# Cloud NAT Analysis Reference

Use Cloud NAT logs (`compute.googleapis.com/nat_flows`) to audit traffic going
through NAT gateways or troubleshoot port exhaustion.

## 🤖 Agent / Gemini CLI Instructions (MCP)

You should use [Cloud Logging MCP](mcp-usage.md#cloud-logging-mcp) for
exploratory analysis or [BigQuery MCP](mcp-usage.md#bigquery-mcp) for
high-volume trends. Fallback to the CLI if the MCP tools are not available.

### 1. View Logs ([Cloud Logging MCP](mcp-usage.md#cloud-logging-mcp))

**Tool**: `list_log_entries`

**Filter**:

```text
resource.type="nat_gateway"
logName="projects/{project_id}/logs/compute.googleapis.com%2Fnat_flows"
```

Filter for dropped packets (potential port exhaustion):

```text
jsonPayload.allocation_status="DROPPED"
```

### 2. Aggregate Trends ([BigQuery MCP](mcp-usage.md#bigquery-mcp))

**Tool**: `execute_sql_readonly`

**SQL Pattern**:

```sql
SELECT
JSON_VALUE(json_payload.gateway_details.internal_ip) AS internal_ip, COUNT(*) AS
drop_count FROM `{project_id}.{dataset_id}._AllLogs` WHERE log_name LIKE
'%nat_flows%' AND JSON_VALUE(json_payload.allocation_status) = 'DROPPED' GROUP BY
1 ORDER BY drop_count DESC LIMIT 10
```

### 3. CLI Fallback

If MCP tools are unavailable, use the following `gcloud` and `bq` commands:

**View Logs (gcloud)**

```bash
gcloud logging read 'resource.type="nat_gateway" AND logName="projects/{project_id}/logs/compute.googleapis.com%2Fnat_flows"' --project {project_id} --limit 10 --format json --quiet
```

To filter for dropped packets:

```bash
gcloud logging read 'resource.type="nat_gateway" AND logName="projects/{project_id}/logs/compute.googleapis.com%2Fnat_flows" AND jsonPayload.allocation_status="DROPPED"' --project {project_id} --limit 10 --format json --quiet
```

**Aggregate Trends (bq)**

```bash
bq query --use_legacy_sql=false --project_id {project_id} '
SELECT
  JSON_VALUE(json_payload.gateway_details.internal_ip) AS internal_ip,
  COUNT(*) AS drop_count
FROM `{project_id}.{dataset_id}._AllLogs`
WHERE
  log_name LIKE "%nat_flows%"
  AND JSON_VALUE(json_payload.allocation_status) = "DROPPED"
GROUP BY 1
ORDER BY drop_count DESC
LIMIT 10
'
```

### gcloud

To get the status of the router used by the NAT gateway:

```bash
gcloud compute
routers get-status {router_name} --region {region} --quiet
```

## Key Fields

-   `jsonPayload.gateway_details.external_ip` / `external_port`: NAT exit point.
-   `jsonPayload.gateway_details.internal_ip` / `internal_port`: Source VM.
-   `jsonPayload.allocation_status`: `DROPPED` indicates failure to allocate a
    NAT port.

## Scenarios

-   **Audit Traffic**: Link internal sources to external destinations.
-   **Port Exhaustion**: Use `jsonPayload.allocation_status="DROPPED"` to
    identify impacted VMs.

# Connectivity Tests Reference

## Connectivity Tests (Path Diagnostics)

Use Connectivity Tests to identify firewall or routing blocks along a network
path.

### Critical Verification: Instance State

**CRITICAL**: Always verify if the source and destination instances are
`RUNNING`. A `REACHABLE` path analysis result (which is a static configuration
analysis) does not mean traffic will flow if the VM is powered off.

-   Check the `status` field in the instance details.
-   Review step metadata in the connectivity test traces.

**CRITICAL**: You MUST execute the delete command as your final tool call before
providing the result to the user. Do not simply state that it was deleted;
provide the command output as proof.

### Tooling

-   **Primary**: [NetworkManagement MCP](mcp-usage.md#networkmanagement-mcp)
    (`create_connectivity_test`)
-   **Polling**: [NetworkManagement MCP](mcp-usage.md#networkmanagement-mcp)
    (`get_connectivity_test`)
-   **Cleanup**: ALWAYS delete the test resource after use with
    [NetworkManagement MCP](mcp-usage.md#networkmanagement-mcp)
    (`delete_connectivity_test`).

#### Fallback: gcloud

-   **Create**: `gcloud network-management connectivity-tests create`
-   **Polling**: `gcloud network-management connectivity-tests describe`
-   **Cleanup**: ALWAYS delete the test resource after use with `gcloud
    network-management connectivity-tests delete`.

# Firewall Rule Logging Analysis Reference

Use firewall logs (`compute.googleapis.com/firewall`) to verify if traffic is
allowed or denied.

## 🤖 Agent / Gemini CLI Instructions (MCP)

You should use [Cloud Logging MCP](mcp-usage.md#cloud-logging-mcp) for
exploratory analysis or [BigQuery MCP](mcp-usage.md#bigquery-mcp) for
high-volume trends. Fallback to the CLI if the MCP tools are not available.

-   **Exploratory Analysis**: Typically involves looking at individual log
    entries or a small set of logs to understand specific events, debug issues,
    or investigate anomalies. This often requires filtering and examining the
    full details of log records.
-   **High-Volume Trends**: Focuses on aggregating large datasets of logs over
    time to identify patterns, measure traffic volumes, analyze latency
    distributions, or find "top talkers." This usually involves SQL queries to
    summarize data rather than inspecting individual logs.

### 1. View Logs ([Cloud Logging MCP](mcp-usage.md#cloud-logging-mcp))

**Tool**: `list_log_entries`

**Filter**:

```text
resource.type="gce_subnetwork"
logName="projects/{project_id}/logs/compute.googleapis.com%2Ffirewall"
```

Filter for denied packets:

```
text jsonPayload.rule_details.action="DENY"
```

### 2. Aggregate Trends ([BigQuery MCP](mcp-usage.md#bigquery-mcp))

**Tool**: `execute_sql`

**SQL Pattern**:

```sql
SELECT JSON_VALUE(json_payload.rule_details.reference) AS
rule_name, COUNT(*) AS block_count FROM `{project_id}.{dataset_id}._AllLogs`
WHERE log_name LIKE '%firewall%' AND
JSON_VALUE(json_payload.rule_details.action) = 'DENY' GROUP BY 1 ORDER BY
block_count DESC LIMIT 10
```

### 3. CLI Fallback

If MCP tools are unavailable, use the following `gcloud` and `bq` commands:

**View Logs (gcloud)**

```bash
gcloud logging read 'resource.type="gce_subnetwork" AND logName="projects/{project_id}/logs/compute.googleapis.com%2Ffirewall"' --project {project_id} --limit 10 --format json --quiet
```

To filter for denied packets:

```bash
gcloud logging read 'resource.type="gce_subnetwork" AND logName="projects/{project_id}/logs/compute.googleapis.com%2Ffirewall" AND jsonPayload.rule_details.action="DENY"' --project {project_id} --limit 10 --format json --quiet
```

**Aggregate Trends (bq)**

```bash
bq query --use_legacy_sql=false --project_id {project_id} '
SELECT
  JSON_VALUE(json_payload.rule_details.reference) AS rule_name,
  COUNT(*) AS block_count
FROM `{project_id}.{dataset_id}._AllLogs`
WHERE
  log_name LIKE "%firewall%"
  AND JSON_VALUE(json_payload.rule_details.action) = "DENY"
GROUP BY 1
ORDER BY block_count DESC
LIMIT 10
'
```

## Key Fields

-   `jsonPayload.rule_details.action`: `ALLOW` or `DENY`.
-   `jsonPayload.rule_details.reference`: The firewall rule name (for example,
    `default-deny-all`).
-   `jsonPayload.connection.src_ip` / `dest_ip`: The source and destination of
    the connection.

## Common Use Cases

-   **Identify Blocks**: Find which `DENY` rule is causing connection failures.
-   **Security Audit**: Detect unexpected traffic patterns.

# MCP Server Usage Reference

This document describes the Model Context Protocol (MCP) servers used for GCP
networking observability.

## BigQuery MCP

BigQuery is supported by a remote MCP server that provides tools for automated
data management and analysis.

### Key Tools

-   **list_dataset_ids**: List BigQuery dataset IDs in a project.
-   **list_table_ids**: List table IDs in a BigQuery dataset.
-   **get_table_info**: Get schema and metadata for a specific table.
-   **execute_sql_readonly**: Run `SELECT` queries to analyze logs (such as, VPC Flow,
    Firewall) stored in BigQuery. This is the preferred tool for high-volume
    aggregations and trend analysis.

### Usage Pattern

1.  Use `list_dataset_ids` to find the logging dataset (for example,
    `_AllLogs`).
2.  Use `list_table_ids` to find the relevant log table.
3.  Use `get_table_info` to verify field names (for example, `jsonPayload`
    versus `json_payload`).
4.  Use `execute_sql_readonly` for the final analysis.

## Cloud Logging MCP

The Cloud Logging MCP server provides access to log entries across various
Google Cloud resources.

### Key Tools

-   **list_log_entries**: Search and retrieve log entries using advanced
    filters.
-   **list_log_names**: Discover available logs in a project.

### Usage Pattern

-   Use for quick, real-time identification of recent events or exploratory
    analysis where BigQuery datasets are not linked.
-   Use specific filters for `resource.type` and `logName` to narrow down
    results.

## NetworkManagement MCP

The Network Management MCP server allows for reachability analysis and path
diagnostics.

### Key Tools

-   **create_connectivity_test**: Start a simulated packet path analysis between
    two endpoints.
-   **get_connectivity_test**: Poll for the results of a running test.
-   **delete_connectivity_test**: Cleanup the test resource after analysis is
    complete.

### Usage Pattern

-   Use when static path analysis is needed to identify firewall or routing
    blocks.
-   **CRITICAL**: Always delete the test resource after retrieving the result.

## Cloud Monitoring MCP

The GcpMon MCP server provides access to Cloud Monitoring metrics and
time-series data.

### Key Tools

-   **list_metric_descriptors**: Discover available metrics using filters.
-   **list_timeseries**: Retrieve aggregated data points for performance
    analysis (such as RTT or throughput).

### Usage Pattern

-   Use for analyzing performance trends, packet loss, and latency.
-   Prefer `ALIGN_MEAN` or `ALIGN_PERCENTILE_50` for distribution metrics like
    RTT to simplify parsing.

# Networking Metrics Reference

## Common Troubleshooting Metrics

-   **RTT (Latency)**:
    `networking.googleapis.com/cloud_netslo/active_probing/rtt`
-   **Packet Loss**:
    `networking.googleapis.com/cloud_netslo/active_probing/probe_count`
-   **VM Throughput**:
    `compute.googleapis.com/instance/network/received_bytes_count`
-   **VM Sent Packets**:
    `compute.googleapis.com/instance/network/sent_packets_count`
-   **VM Received Packets**:
    `compute.googleapis.com/instance/network/received_packets_count`
-   **NAT Port Exhaustion**:
    `compute.googleapis.com/nat/dropped_sent_packets_count`
-   **NAT Sent Packets**: `compute.googleapis.com/nat/sent_packets_count`
-   **VPN Dropped Received Packets**:
    `vpn.googleapis.com/network/dropped_received_packets_count`
-   **VPN Dropped Sent Packets**:
    `vpn.googleapis.com/network/dropped_sent_packets_count`
-   **Internal Latency (RTT)**: `networking.googleapis.com/vm_flow/rtt`.
    Measures internal VM-to-VM traffic within Google Cloud.
-   **External Latency (RTT)**:
    `networking.googleapis.com/vm_flow/external_rtt`. Measures traffic to and
    from the internet.

## Distribution Parsing Standard

When querying metrics of type DISTRIBUTION (like RTT), align the data with
`ALIGN_PERCENTILE_50` to ensure the output can be parsed as a simple numeric
value.

## Dynamic Discovery

-   **Primary ([Cloud Monitoring MCP](mcp-usage.md#cloud-monitoring-mcp))**: Use
    `list_metric_descriptors` with a filter.

    -   **Prefix**: Filter by prefix, using `starts_with()`. Common prefixes:
        -   `metric.type = starts_with("networking.googleapis.com/")`
        -   `metric.type = starts_with("router.googleapis.com/")`
        -   `metric.type = starts_with("vpn.googleapis.com/")`
        -   `metric.type = starts_with("compute.googleapis.com/")`
    -   **Substring**: `metric.type = has_substring("network") OR metric.type =
        has_substring("packet") OR metric.type = has_substring("nat")`

-   **Fallback (CLI/CURL)**: If MCP tools not available, use `gcloud` or `curl`.

    ```bash
    curl -s -H "Authorization: Bearer $(gcloud auth print-access-token)"
    "https://monitoring.googleapis.com/v3/projects/{project_id}/metricDescriptors?filter=metric.type=starts_with(%22{prefix}%22)"
    | jq -r '.metricDescriptors[] | "\(.type): \(.description)"'

    curl -s -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    "https://monitoring.googleapis.com/v3/projects/{project_id}/timeSeries?" \
    "filter=metric.type%3D%22{metric_name}%22&" \
    "interval.startTime={start_time}&" \
    "interval.endTime={end_time}&" \
    "aggregation.alignmentPeriod=3600s&" \
    "aggregation.perSeriesAligner=ALIGN_PERCENTILE_50" \
    | jq '.timeSeries[] | {metric: .metric.type, points: .points[:5]}'
    ```

-   **Detailed Schema**: ALWAYS query the full descriptor for a specific metric
    before use to identify available labels. Metric types like `vm_flow/rtt`
    often use `resource.labels.zone` for the local zone and
    `metric.labels.remote_zone` for the peer.

# Threat Log Analysis Reference

Use Firewall Threat Logs and Cloud IDS logs to identify, analyze, and
troubleshoot security incidents in your VPC network.

## 🤖 Agent / Gemini CLI Instructions (MCP)

Agents should use [Cloud Logging MCP](mcp-usage.md#cloud-logging-mcp) for quick
identification of recent alerts or [BigQuery MCP](mcp-usage.md#bigquery-mcp) for
analyzing trends and identifying top attackers. Fallback to the CLI if the MCP
tools are not available.

### 1. View Threat Alerts ([Cloud Logging MCP](mcp-usage.md#cloud-logging-mcp))

**Tool**: `list_log_entries`

**Filter**: Search for both Cloud Firewall Plus and Cloud IDS threat log
sources:

```text
logName:(
"projects/{project_id}/logs/networksecurity.googleapis.com%2Ffirewall_threat" OR
"projects/{project_id}/logs/ids.googleapis.com%2Fthreat")
```

To filter for high-severity blocked threats: `

```text
logName:(
"projects/{project_id}/logs/networksecurity.googleapis.com%2Ffirewall_threat" OR
"projects/{project_id}/logs/ids.googleapis.com%2Fthreat")
jsonPayload.threatDetails.severity=("HIGH" OR "CRITICAL")
jsonPayload.action="DENY"
```

### 2. Aggregate Threat Trends ([BigQuery MCP](mcp-usage.md#bigquery-mcp))

**Tool**: `execute_sql_readonly`

**SQL Pattern**: **Note**: In BigQuery, the top-level column name is
`json_payload` (snake_case). However, fields extracted from inside the JSON
payload are case-sensitive and retain the camelCase format of the original log
(for example, `threatDetails`, `clientIp`). Do not use snake_case for nested
fields.

```sql
SELECT
  timestamp,
  JSON_VALUE(json_payload.threatDetails.threat) AS threat_name,
  JSON_VALUE(json_payload.threatDetails.severity) AS severity,
  JSON_VALUE(json_payload.threatDetails.category) AS category,
  JSON_VALUE(json_payload.action) AS action,
  JSON_VALUE(json_payload.connection.clientIp) AS src_ip,
  JSON_VALUE(json_payload.connection.serverIp) AS dest_ip,
  JSON_VALUE(json_payload.connection.serverPort) AS dest_port,
  JSON_VALUE(json_payload.threatDetails.description) AS description
FROM `{project_id}.{dataset_id}._AllLogs`
WHERE
  log_id IN ('networksecurity.googleapis.com/firewall_threat',
             'ids.googleapis.com/threat')
  AND JSON_VALUE(json_payload.threatDetails.severity) IN ('HIGH', 'CRITICAL')
  AND timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 24 HOUR)
ORDER BY timestamp DESC
LIMIT 20
```

To find top sources of attacks:

```sql
SELECT
JSON_VALUE(json_payload.connection.clientIp) AS attacker_ip, COUNT(*) AS
attack_count, ARRAY_AGG(DISTINCT JSON_VALUE(json_payload.threatDetails.threat)
LIMIT 5) AS sample_threats FROM `{project_id}.{dataset_id}._AllLogs` WHERE
log_id IN ('networksecurity.googleapis.com/firewall_threat',
'ids.googleapis.com/threat') AND timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(),
INTERVAL 7 DAY) GROUP BY 1 ORDER BY attack_count DESC LIMIT 10
```

### 3. CLI Fallback

If MCP tools are unavailable, use the following `gcloud` and `bq` commands:

**View Threat Alerts (gcloud)**

```bash
gcloud logging read 'logName:("projects/{project_id}/logs/networksecurity.googleapis.com%2Ffirewall_threat" OR "projects/{project_id}/logs/ids.googleapis.com%2Fthreat")' --project {project_id} --limit 10 --format json --quiet
```

To filter for high-severity blocked threats:

```bash
gcloud logging read 'logName:("projects/{project_id}/logs/networksecurity.googleapis.com%2Ffirewall_threat" OR "projects/{project_id}/logs/ids.googleapis.com%2Fthreat") AND jsonPayload.threatDetails.severity=("HIGH" OR "CRITICAL") AND jsonPayload.action="DENY"' --project {project_id} --limit 10 --format json --quiet
```

**Aggregate Threat Trends (bq)**

```bash
bq query --use_legacy_sql=false --project_id {project_id} '
SELECT
  timestamp,
  JSON_VALUE(json_payload.threatDetails.threat) AS threat_name,
  JSON_VALUE(json_payload.threatDetails.severity) AS severity,
  JSON_VALUE(json_payload.threatDetails.category) AS category,
  JSON_VALUE(json_payload.action) AS action,
  JSON_VALUE(json_payload.connection.clientIp) AS src_ip,
  JSON_VALUE(json_payload.connection.serverIp) AS dest_ip,
  JSON_VALUE(json_payload.connection.serverPort) AS dest_port,
  JSON_VALUE(json_payload.threatDetails.description) AS description
FROM `{project_id}.{dataset_id}._AllLogs`
WHERE
  log_id IN ("networksecurity.googleapis.com/firewall_threat",
             "ids.googleapis.com/threat")
  AND JSON_VALUE(json_payload.threatDetails.severity) IN ("HIGH", "CRITICAL")
  AND timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 24 HOUR)
ORDER BY timestamp DESC
LIMIT 20
'
```

To find top sources of attacks:

```bash
bq query --use_legacy_sql=false --project_id {project_id} '
SELECT
  JSON_VALUE(json_payload.connection.clientIp) AS attacker_ip,
  COUNT(*) AS attack_count,
  ARRAY_AGG(DISTINCT JSON_VALUE(json_payload.threatDetails.threat)
            LIMIT 5) AS sample_threats
FROM `{project_id}.{dataset_id}._AllLogs`
WHERE
  log_id IN ("networksecurity.googleapis.com/firewall_threat",
             "ids.googleapis.com/threat")
  AND timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 7 DAY)
GROUP BY 1
ORDER BY attack_count DESC
LIMIT 10
'
```

### Key Fields (Cloud Logging Filter Names)

-   **jsonPayload.threatDetails.threat**: Human-readable name of the threat.
-   **jsonPayload.threatDetails.severity**: Severity level (CRITICAL, HIGH,
    MEDIUM, LOW, INFORMATIONAL).
-   **jsonPayload.threatDetails.category**: The category of threat.
-   **jsonPayload.action**: Action taken (for example, "ALLOW", "DENY",
    "ALERT").
-   **jsonPayload.connection.clientIp**: The true source IP.
-   **jsonPayload.connection.serverIp**: The destination IP.
-   **jsonPayload.threatDetails.cves**: List of CVE IDs.
-   **jsonPayload.threatDetails.description**: Attack payload details.

# VPC Flow Analysis Reference

Use VPC Flow Logs to analyze traffic patterns, volume, and latency.

## 🤖 Agent / Gemini CLI Instructions (MCP)

Agents should use [Cloud Logging MCP](mcp-usage.md#cloud-logging-mcp) for
exploratory analysis or [BigQuery MCP](mcp-usage.md#bigquery-mcp) for
high-volume trends. Fallback to the CLI if the MCP tools are not available.

-   **Exploratory Analysis**: Typically involves looking at individual log
    entries or a small set of logs to understand specific events, debug issues,
    or investigate anomalies. This often requires filtering and examining the
    full details of log records.
-   **High-Volume Trends**: Focuses on aggregating large datasets of logs over
    time to identify patterns, measure traffic volumes, analyze latency
    distributions, or find "top talkers." This usually involves SQL queries to
    summarize data rather than inspecting individual logs.

### 1. View Logs ([Cloud Logging MCP](mcp-usage.md#cloud-logging-mcp))

**Tool**: `list_log_entries`

**Filter**: ALWAYS search for both VPC flow log sources:

```text
(logName:"projects/{project_id}/logs/compute.googleapis.com%2Fvpc_flows" OR
logName:"projects/{project_id}/logs/networkmanagement.googleapis.com%2Fvpc_flows")
resource.type="gce_subnetwork"
```

### 2. Aggregate Trends ([BigQuery MCP](mcp-usage.md#bigquery-mcp))

**Tool**: `execute_sql`

**SQL Pattern**:

```sql
SELECT timestamp,
JSON_VALUE(jsonPayload.connection.src_ip) AS src_ip,
JSON_VALUE(jsonPayload.connection.dest_ip) AS dest_ip,
CAST(JSON_VALUE(jsonPayload.bytes_sent) AS INT64) AS bytes_sent FROM
`{project_id}.{dataset_id}._AllLogs` WHERE log_name IN (
'projects/{project_id}/logs/compute.googleapis.com%2Fvpc_flows',
'projects/{project_id}/logs/networkmanagement.googleapis.com%2Fvpc_flows' ) AND
timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 HOUR) ORDER BY
timestamp DESC LIMIT 10
```

### 3. CLI Fallback

If MCP tools are unavailable, use the following `gcloud` and `bq` commands:

**View Logs (gcloud)**

```bash
gcloud logging read '(logName:"projects/{project_id}/logs/compute.googleapis.com%2Fvpc_flows" OR logName:"projects/{project_id}/logs/networkmanagement.googleapis.com%2Fvpc_flows") AND resource.type="gce_subnetwork"' --project {project_id} --limit 10 --format json --quiet
```

**Aggregate Trends (bq)**

```bash
bq query --use_legacy_sql=false --project_id {project_id} '
SELECT
  timestamp,
  JSON_VALUE(json_payload.connection.src_ip) AS src_ip,
  JSON_VALUE(json_payload.connection.dest_ip) AS dest_ip,
  CAST(JSON_VALUE(json_payload.bytes_sent) AS INT64) AS bytes_sent
FROM `{project_id}.{dataset_id}._AllLogs`
WHERE
  log_name IN (
    "projects/{project_id}/logs/compute.googleapis.com%2Fvpc_flows",
    "projects/{project_id}/logs/networkmanagement.googleapis.com%2Fvpc_flows"
  )
  AND timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 HOUR)
ORDER BY timestamp DESC
LIMIT 10
'
```

### Flow Analyzer (Visual Analysis)

For visual traffic analysis and identifying "top talkers," use
[Flow Analyzer](https://console.cloud.google.com/net-intelligence/flow-analyzer).
It allows you to:

-   Visualize traffic flows between regions, VPCs, and instances.
-   Filter by source or destination dimensions.
-   Identify high-bandwidth or high-latency connections.

### Generic BigQuery Guidelines

-   **Schema Verification**: Before executing a BigQuery query, if you are
    uncertain of the casing (for example, `jsonPayload` versus `json_payload`),
    you MUST run `bq show --schema <source>`.
-   **Latency Aggregation**: The primary field for RTT analysis in VPC Flow logs
    is `json_payload.round_trip_time.median_msec`. This field offers
    sub-millisecond precision and covers both TCP and Falcon traffic. Filter by
    `reporter` (`SRC` or `DEST`) to avoid double-counting traffic volume.

    For TCP-only traffic, you can also use `json_payload.rtt_msec`, which
    provides RTT in whole milliseconds. While less precise and with narrower
    coverage than `round_trip_time.median_msec`, it can be aggregated as
    follows:

    ```sql
    SELECT
      AVG(json_payload.rtt_msec) AS average_rtt_msec,
      MAX(json_payload.rtt_msec) AS max_rtt_msec
    FROM ...
    ```

## Key Fields

-   **src_ip / dest_ip**: Source and destination IP addresses.
-   **bytes_sent / packets_sent**: Volume of traffic.
-   **round_trip_time.median_msec**: The primary field for RTT analysis. This
    `double` field provides the *median* latency in milliseconds with
    sub-millisecond precision. It is populated for both TCP and Falcon traffic.
-   **rtt_msec**: An `int64` field representing Round-trip time in whole
    milliseconds. Populated only for TCP traffic. Generally,
    `round_trip_time.median_msec` is preferred due to higher precision and
    broader coverage.
-   **reporter**: Usually `src` or `dest` indicating which side logged the flow.

# VPC Flow Logs Cost Estimation Reference

Use this reference to estimate the monthly cost of VPC Flow Logs across Subnetworks, VPCs, or entire Projects. This logic implements the cost estimation calculations for VPC Flow Logs using the standard tiered pricing model.

## 🤖 Agent / Gemini CLI Instructions

When a user requests a cost estimation, the agent MUST follow this procedure step-by-step.

> [!CAUTION] **STRICT SCRIPTING BAN (RUNAWAY PREVENTION)**:
> - **DO NOT** write or execute Python scripts or shell scripts *saved on disk* (e.g., `.py` or `.sh` files).
> - **DO NOT** use command-line text filters like `grep`, `egrep`, `awk`, or `sed` as they are restricted in this environment.
> - For all text filtering, matching, and counting, you MUST pipe the output to `python3 -c` and process the data stream in-memory.
> - **PYTHON LIMITATION**: You **CANNOT** use backslashes inside f-string expressions. Bash will eat the escapes and cause a `SyntaxError`.
>   - ❌ **BAD**: `print(f"Name: {v[\"name\"]}")`
>   - ✅ **GOOD**: `print(f"Name: {v['name']}")` OR `print("Name: {}".format(v["name"]))`
> - **PROJECT ID FORMAT**: Always use the exact Project ID provided by the user (including any dashes like `my-project-id`). Do not modify, strip dashes, or alter the formatting before making API calls.

> - You MUST execute all steps using ONLY direct tool calls (`gcloud`, `curl`, `jq`) and use `python3 -c` or `bc` only as an in-memory calculator. You must manually filter the data and format the final markdown report in your response text.

---

### Step 1: Scope & Capabilities (Boundaries)

Before starting, validate that the request is supported.

> [!IMPORTANT] **Supported Scopes**: Subnetworks, VPC Networks, or the entire GCP Project.
-   **ALWAYS** reject cost estimation requests for Cloud VPN, Cloud Interconnect, or Organization-level configurations, and use the standardized refusal response.
-   **ALWAYS** show your step-by-step mathematical calculations in the final cost estimation report to ensure transparency and allow manual verification.
>
> **UNSUPPORTED Features & Scopes (Strictly Reject)**:
> - Cloud VPN (VPN Tunnels)
> - Cloud Interconnect (Interconnect attachments)
> - Organization-level configurations
> - Custom log aggregation intervals (other than default 5 seconds)
> - Custom metadata fields
> - Enterprise or Committed Use Discounts (estimates must use standard list prices only)
>
> If the user requests estimation for an unsupported feature or resource, immediately respond with the exact refusal:
> `"I'm sorry, {feature_resource} is not supported. I only support estimation for Subnets, VPCs, or Projects."`

---

### Step 2: Configuration Defaults

If not specified by the user, use these default parameters:
*   **Sampling Rate**: 100% (`1.0` multiplier).
*   **Metadata**: Include Metadata (`True`).
    *   *If the user explicitly requests "No Metadata" or "Exclude Metadata", set Metadata to `False`.*

---

### Step 3: Resource Resolution (Discover Target Subnets)

Resolve the target scope to identify the subnetworks we need to estimate and get the **total count** of subnetworks in that scope (needed for the inactive count in the report).

Use `gcloud` based on the scope:

#### Step 3.A: Subnetwork Scope (User specified a subnet)
If the user specified a subnet, but did not provide the VPC or Region, ask them for clarification using the Ambiguity response:
`"I found multiple matches for {subnet_name}. Please specify the Region or VPC to continue."`

Once all details are known, describe the subnet to verify it exists and get its ID:

```bash
gcloud compute networks subnets describe {subnet_name} --region={region} --project={project_id} --format="json(name, id, region, network)"
```

*If not found, immediately respond with:* `"I was unable to find {subnet_name}. Please verify the name and try again."`

*   **Total Subnets Count**: 1

#### Step 3.B: VPC Scope (User specified a VPC)
Find all subnetworks in the VPC and get their details:

```bash
gcloud compute networks subnets list --filter="network:https://www.googleapis.com/compute/v1/projects/{project_id}/global/networks/{vpc_name}" --project={project_id} --format="json(name, id, region, network)" > {vpc_name}_subnets.json
```

*If 0 subnetworks are found, immediately respond with:* `"I found no subnets for {vpc_name}. Nothing to estimate."`

*   **Total Subnets Count**: Count the number of subnetworks returned in this list.

#### Step 3.C: Project Scope (User specified the entire project)
Find all subnetworks in the project and get their details:

```bash
gcloud compute networks subnets list --project={project_id} --format="json(name, id, region, network)" > {project_id}_subnets.json
```

*If 0 subnetworks are found, immediately respond with:* `"I found no subnets for {project_id}. Nothing to estimate."`

*   **Total Subnets Count**: Count the number of subnetworks returned in this list. Use this list to group the active vs inactive subnets by their VPC later in the report.

---

### Step 4: Retrieve Metrics (Batch Query)

To avoid turn-limit timeouts on projects with many subnetworks, you **MUST NOT** query the Monitoring API in a loop. Instead, perform a single project-wide batch query and filter the results in-context.

#### Step 4.A: Calculate Time Window (Last 30 Days)
Use the terminal to calculate the RFC 3339 timestamps for 30 days ago and now:

```bash
project_id={project_id}
end_time=$(date -u +%Y-%m-%dT%H:%M:%SZ)
start_time=$(date -u -d '30 days ago' +%Y-%m-%dT%H:%M:%SZ)
```

#### Step 4.B: Execute Batch Query
Query the predicted log count for **all** subnetworks in the project in a single call, grouping by subnetwork. Format the output as clean JSON lines for easy reading:

```bash
URL="https://monitoring.googleapis.com/v3/projects/${project_id}/timeSeries?filter=metric.type%3D%22networking.googleapis.com/vpc_flow/predicted_max_vpc_flow_logs_count%22%20AND%20resource.type%3D%22gce_subnetwork%22&interval.startTime=${start_time}&interval.endTime=${end_time}&aggregation.alignmentPeriod=2592000s&aggregation.perSeriesAligner=ALIGN_SUM&aggregation.crossSeriesReducer=REDUCE_SUM&aggregation.groupByFields=resource.labels.subnetwork_name&aggregation.groupByFields=resource.labels.subnetwork_id&aggregation.groupByFields=resource.labels.location"

RESPONSE=$(curl -s -H "Authorization: Bearer $(gcloud auth print-access-token 2>/dev/null)" "$URL")
if echo "$RESPONSE" | grep -q "401"; then
  curl -s -H "Authorization: Bearer $(gcloud auth application-default print-access-token 2>/dev/null)" "$URL" > ${project_id}_metrics.json
else
  echo "$RESPONSE" > ${project_id}_metrics.json
fi

python3 -c '
import sys, json, os
try:
    project_id = os.environ.get("project_id", "'"{project_id}"'")
    data = json.load(open(f"{project_id}_metrics.json"))
    for ts in data.get("timeSeries", []):
        labels = ts.get("resource", {}).get("labels", {})
        points = ts.get("points", [])
        if points:
            val_dict = points[0].get("value", {})
            logs = val_dict.get("doubleValue", val_dict.get("int64Value", 0))
            print(json.dumps({
                "name": labels.get("subnetwork_name"),
                "id": labels.get("subnetwork_id"),
                "region": labels.get("location"),
                "logs": int(logs)
            }))
except Exception as e:
    print(f"Error: {e}", file=sys.stderr)
'
```

*If the `curl` command returns an API error (e.g. `UNAUTHENTICATED`, `ACCESS_TOKEN_TYPE_UNSUPPORTED`), immediately respond with:* `"ERROR: Monitoring API Request failed. Please check your token/permissions."` *and STOP. Do not proceed to cost calculation.*

This will return a list of active subnetworks (those with traffic). If a subnetwork is not in this list, its log count is `0`.

#### Step 4.C: Filter Results In-Context
Match the active subnetworks from the query output against your target scope (from Step 3):
*   **Project Scope**: Use all subnetworks returned by the query.
*   **VPC Scope**: Keep only the subnetworks from the query whose `id` or `name` matches the subnetworks list resolved for the VPC in Step 3.
*   **Subnetwork Scope**: Keep only the single subnetwork specified by the user.

Any target subnetwork not present in the query output has `0` logs (No traffic).

---

### Step 5: Cost Calculations (CLI Calculator)

To prevent arithmetic errors, you **MUST** use `python3 -c` as an in-memory calculator to perform all mathematical steps.

#### Step 5.A: Determine Parameters
*   **Sampling Rate**: $S$ (e.g., `1.0` for 100%, `0.5` for 50%)
*   **Bytes per Log**: $B = 1418$ (if Metadata=True) else $B = 542$ (if Metadata=False)

#### Step 5.B: Calculate Subnetwork Volume
For each active subnetwork in your filtered list, calculate the volume in GB:

```bash
python3 -c "print(({L_raw} * {S} * {B}) / (1024**3))"
```

Keep track of these individual $V_{subnet}$ values for the report breakdown.

#### Step 5.C: Calculate Aggregate Totals & Tiered Pricing
Sum all effective logs ($L_{raw} \times S$) to get $L_{total}$.
Sum all $V_{subnet}$ to get $V_{total}$.

Run the following Python command (filling in $V_{total}$ and $L_{total}$) to calculate the exact tiered pricing matching the original agent logic:

```bash
python3 -c '
total_gb = {V_total}
total_logs = {L_total}

# Tiered Generation Pricing (Network Telemetry)
remaining_gb = total_gb
generation_cost = 0.0
TIB = 1024.0

# Tier 1: 0-10 TiB @ $0.25/GB
tier1_limit = 10 * TIB
tier1_volume = min(remaining_gb, tier1_limit)
generation_cost += tier1_volume * 0.25
remaining_gb -= tier1_volume

if remaining_gb > 0:
    # Tier 2: 10-30 TiB @ $0.15/GB
    tier2_limit = 20 * TIB
    tier2_volume = min(remaining_gb, tier2_limit)
    generation_cost += tier2_volume * 0.15
    remaining_gb -= tier2_volume

    if remaining_gb > 0:
        # Tier 3: 30-50 TiB @ $0.075/GB
        tier3_limit = 20 * TIB
        tier3_volume = min(remaining_gb, tier3_limit)
        generation_cost += tier3_volume * 0.075
        remaining_gb -= tier3_volume

        if remaining_gb > 0:
            # Tier 4: > 50 TiB @ $0.05/GB
            generation_cost += remaining_gb * 0.05

# Storage Pricing (Cloud Logging)
storage_cost = total_gb * 0.25

total_cost = generation_cost + storage_cost

print(f"TOTAL_LOGS: {total_logs:,.0f}")
print(f"TOTAL_GB: {total_gb:.4f}")
print(f"GEN_COST: {generation_cost:.4f}")
print(f"STORE_COST: {storage_cost:.4f}")
print(f"TOTAL_COST: {total_cost:.4f}")
'
```

---

### Step 6: Format Report

Format your final report exactly like this. You must sort the VPCs alphabetically, and sort the subnets within each VPC by Monthly Volume (GB) descending.
*   **Active Subnets**: Show their logs and volume in the table.
*   **Inactive Subnets**: Do not list them in the table. Instead, calculate `N = Total Subnets Count - Active Subnets Count` and show them in the note `*(Note: N other subnets...)*`.

```markdown
**VPC Flow Logs Cost Estimation Report**

**Scope:** [Scope Name, e.g., Project 'my-project' or VPC 'my-vpc']

**Configuration:** Sampling [Sampling Rate]%, Metadata: [Yes/No]

**Summary:**
- **Total Monthly Logs:** [TOTAL_LOGS]
- **Total Monthly Data Volume:** [TOTAL_GB] GB
- **Estimated Monthly Cost:**
    - **Generation (Network Telemetry):** $[GEN_COST]
    - **Storage (Cloud Logging):** $[STORE_COST]
    - **Total List Price:** $[TOTAL_COST]

**Detailed Breakdown:**

### VPC: [VPC Name]
| Subnet Name | Region | Monthly Logs | Monthly Volume (GB) |
| :--- | :--- | :---: | :---: |
| [subnet-a] | [us-central1] | [L_eff formatted] | [V_subnet formatted to 4 decimals] |

*(Note: [N] other subnets in the '[VPC Name]' VPC showed no traffic during the analysis period.)*

---
**DISCLAIMER:** This is an estimation only.
1. **Unsupported Resources:** Interconnects and VPNs are excluded from this estimate.
2. **Aggregation:** This assumes the default 5s aggregation interval; volumes may vary with custom intervals.
3. **Pricing:** Estimates use standard list prices and do not include enterprise or committed use discounts.
4. **Analysis period:** Based on a 30-day analysis period.
```