Skip to content

Bug Bash 2026-08: Street View Imagery Insights MCP Server #82

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
akshitmeghawat wants to merge 1 commit into
base: feat/street-view-samples
Choose a base branch
from
Open

Bug Bash 2026-08: Street View Imagery Insights MCP Server #82

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion street_view_insights/mcp_server/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM python:3.11-slim
FROM python:3.12-slim

# Prevent python from buffering stdout/stderr
ENV PYTHONUNBUFFERED=1
Expand Down
62 changes: 48 additions & 14 deletions street_view_insights/mcp_server/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,22 +8,34 @@ It implements a **strict Data Loss Prevention (DLP) boundary**: no raw images or

## 1. Using the Hosted MCP Server

To use the already-deployed instance in your MCP-compatible IDE or client (e.g. Cursor, Claude Desktop, Windsurf, or custom python agents), add the following configuration to your `mcp_config.json` file:
To use the already-deployed instance in your MCP-compatible IDE or [Antigravity CLI](https://antigravity.google/docs/cli/mcp#antigravity-cli), add the following configuration to your `mcp_config.json` file:

```json
{
"mcpServers": {
"streetview-imagery-insights": {
"url": "https://streetview-imagery-insights-mcp-769602582640.us-central1.run.app/sse"
"url": "<use the service url generated from your deployment>"
}
}
}
```

Your service url (after deployment) can be retrieved using:

```
gcloud run services describe streetview-imagery-insights-mcp \
--project "${PROJECT_ID}" \
--region "${REGION}"
```

### Supported Tools

Once connected, your agent will have access to the following 5 tools:
1. **`list_assets`**: Find assets by type or within a geographic radius (uses BigQuery).
* **Example:**
```bash
streetview-imagery-insights list_assets <gcp-project>.imagery_insights___us
```
2. **`get_asset_observations`**: Retrieve all historical image captures/observations for a specific asset.
3. **`analyze_cropped_asset`**: Downloads full-frame image, crops to asset bounding box, runs Gemini analysis in GCP, and returns text results.
4. **`analyze_full_frame_context`**: Submits full-frame image (optionally with bounding box drawn) to Gemini for contextual scene understanding.
Expand All @@ -49,7 +61,39 @@ If you wish to host your own instance of the MCP server, follow these steps:
gcloud config set project <YOUR_PROJECT_ID>
```

### Deployment
### Local Development / Running Locally
To test the server locally:
1. Configure the harness by updating the JSON as follows:
```json
{
"mcpServers": {
"streetview-imagery-insights": {
"url": "http://localhost:8080/mcp"
}
}
}
```
2. Create a virtual environment and activate it:
```bash
python -m venv .venv
source .venv/bin/activate
```
3. Install dependencies:
```bash
pip install -r requirements.txt
```
4. Run the server to override the SSE transport with plain unauthenticated HTTP:
```bash
MCP_TRANSPORT=http python main.py
```
*Note: Ensure your terminal has access to GCP credentials (`GOOGLE_APPLICATION_CREDENTIALS` or configured via `gcloud`).*
5. To deactivate and clean up the virtual environment:
```bash
deactivate
rm -rf .venv
```

### Cloud Run Deployment
1. Navigate to the `mcp_server` directory:

@bijanvakili bijanvakili Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Remove this step cd ... as the local development did not include it.

```bash
cd street_view_insights/mcp_server
Expand All @@ -65,14 +109,4 @@ The script will automatically:
3. Push the container image to GCP Artifact Registry.
4. Deploy the container to a new Google Cloud Run service named `streetview-imagery-insights-mcp` with unauthenticated access allowed.

### Local Development / Running Locally
To test the server locally:
1. Install dependencies:
```bash
pip install -r requirements.txt
```
2. Run the server using `stdio` transport:
```bash
python main.py
```
*Note: Ensure your terminal has access to GCP credentials (`GOOGLE_APPLICATION_CREDENTIALS` or configured via `gcloud`).*
> If your GCP project or organization does not allow unauthenticated applications, modify the `--allow-unauthenticated` flag (in `deploy.sh`) with `--no-allow-unauthenticated` when you deploy. You can then use [gcloud run services proxy](https://docs.cloud.google.com/sdk/gcloud/reference/run/services/proxy) to connect from your local machine to the Cloud Run application.
11 changes: 4 additions & 7 deletions street_view_insights/mcp_server/deploy.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,17 +1,14 @@
#!/bin/bash
set -e

PROJECT_ID="${GOOGLE_CLOUD_PROJECT:-YOUR_PROJECT_ID}"
PROJECT_ID="${GOOGLE_CLOUD_PROJECT:-$(gcloud config get-value project 2>/dev/null)}"
PROJECT_ID="${PROJECT_ID:-YOUR_PROJECT_ID}"
REGION="us-central1"
SERVICE_NAME="streetview-imagery-insights-mcp"
IMAGE_TAG="us-central1-docker.pkg.dev/${PROJECT_ID}/cloud-run-source-deploy/${SERVICE_NAME}:latest"

echo "Building container image using Cloud Build..."
gcloud builds submit --tag "${IMAGE_TAG}" --project "${PROJECT_ID}" .

echo "Deploying to Cloud Run service: ${SERVICE_NAME}..."
echo "Building container and deploying to Cloud Run service: ${SERVICE_NAME} (project: ${PROJECT_ID})..."
gcloud run deploy "${SERVICE_NAME}" \
--image "${IMAGE_TAG}" \
--source . \
--project "${PROJECT_ID}" \
--region "${REGION}" \
--allow-unauthenticated \

@bijanvakili bijanvakili Aug 19, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please verify that this deployment to Cloud Run will work?

(ie. deploy it and connect with Jetski CLI to successfully run a specific skill)

  1. The service needs to run as a service account with necessary roles to query BigQuery. Take a look at the deployment script for street-view-samples/street_view_insights/cropped/samples/imagery-insights-batch-processor here which ensures the appropriate service account is set up along with forcing the service deployment to use it.

In particular, the service account will require the bigquery.jobs.create permission which is typically granted through one of the roles/bigquery.jobUser or roles/bigquery.user roles.

  1. You'll need to determine if --allow-unauthenticated or --no-allow-unauthenticated should be used. Based on the comment in the README.md, I was only able to get Jetski CLI to connect successfully if the FastMCP transport was http and service was deployed with --no-allow-unauthenticated along with running the Cloud Run proxy in a separate window.

Expand Down
45 changes: 29 additions & 16 deletions street_view_insights/mcp_server/main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,10 @@
from google.cloud import storage
from google import genai
from google.genai import types
from mcp.server.fastmcp import FastMCP
import uvicorn
from fastmcp import FastMCP

# Initialize FastMCP Server
mcp = FastMCP("Street View Imagery Insights", host="0.0.0.0")
mcp = FastMCP("Street View Imagery Insights")

CACHE_DIR = "/tmp/mcp_image_cache"
os.makedirs(CACHE_DIR, exist_ok=True)
Expand Down Expand Up @@ -109,6 +108,22 @@ def equirectangular_to_perspective(src_image_path, heading_deg, pitch_deg, fov_d
out_data = src_data[px_y, px_x]
return Image.fromarray(out_data)

def resolve_dataset(dataset_ref: str, default_project: str) -> tuple[str, str]:
"""Resolves dataset reference to project and dataset IDs.

Handles:
- "dataset_id" -> (default_project, "dataset_id")
- "project_id.dataset_id" -> ("project_id", "dataset_id")
- "project_id:dataset_id" -> ("project_id", "dataset_id")
"""
if "." in dataset_ref:
project_id, dataset_id = dataset_ref.split(".", 1)
return project_id, dataset_id
elif ":" in dataset_ref:
project_id, dataset_id = dataset_ref.split(":", 1)
return project_id, dataset_id
return default_project, dataset_ref

# --- MCP Tools ---

@mcp.tool()
Expand All @@ -124,7 +139,7 @@ def list_assets(
List assets in the dataset, optionally filtered by asset class or geographic proximity.
"""
client_bq = bigquery.Client()
project_id = client_bq.project
project_id, dataset_id = resolve_dataset(dataset_id, client_bq.project)

where_clauses = []
query_params = []
Expand All @@ -145,7 +160,7 @@ def list_assets(

query = f"""
SELECT asset_id, asset_type, location, detection_time
FROM `{project_id}.{dataset_id}.all_assets`
FROM `{project_id}.{dataset_id}.latest_assets`
{where_str}
LIMIT @limit
"""
Expand All @@ -171,11 +186,11 @@ def get_asset_observations(dataset_id: str, asset_id: str) -> str:
Retrieve all historical observations for a specific asset ID.
"""
client_bq = bigquery.Client()
project_id = client_bq.project
project_id, dataset_id = resolve_dataset(dataset_id, client_bq.project)

query = f"""
SELECT observation_id, gcs_uri, bbox, pano_id, capture_time, camera_pose, asset_type
FROM `{project_id}.{dataset_id}.all_observations`
FROM `{project_id}.{dataset_id}.latest_observations`

@bijanvakili bijanvakili Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be cropped_assets_all

Reasons:

  • latest_observations will be deprecated
  • The function description indicates that it will retrieve all historical observations

WHERE asset_id = @asset_id
ORDER BY capture_time DESC
"""
Expand Down Expand Up @@ -212,11 +227,11 @@ def analyze_cropped_asset(
The raw image never leaves GCP.
"""
client_bq = bigquery.Client()
project_id = client_bq.project
project_id, dataset_id = resolve_dataset(dataset_id, client_bq.project)

query = f"""
SELECT gcs_uri, bbox, asset_type
FROM `{project_id}.{dataset_id}.all_observations`
FROM `{project_id}.{dataset_id}.cropped_observations_all`
WHERE observation_id = @observation_id
LIMIT 1
"""
Expand Down Expand Up @@ -301,11 +316,11 @@ def analyze_full_frame_context(
The raw image never leaves GCP.
"""
client_bq = bigquery.Client()
project_id = client_bq.project
project_id, dataset_id = resolve_dataset(dataset_id, client_bq.project)

query = f"""
SELECT gcs_uri, bbox, asset_type
FROM `{project_id}.{dataset_id}.all_observations`
FROM `{project_id}.{dataset_id}.full_frame_observations_all`
WHERE observation_id = @observation_id
LIMIT 1
"""
Expand Down Expand Up @@ -404,7 +419,7 @@ def analyze_panorama_perspective(
The raw image never leaves GCP.
"""
client_bq = bigquery.Client()
project_id = client_bq.project
project_id, dataset_id = resolve_dataset(dataset_id, client_bq.project)

query = f"""
SELECT gcs_uri
Expand Down Expand Up @@ -455,9 +470,7 @@ def analyze_panorama_perspective(
except Exception as e:
return json.dumps({"error": f"Failed to run model analysis: {str(e)}"})

# Mount SSE transport Starlette app
app = mcp.sse_app()

if __name__ == "__main__":
port = int(os.getenv("PORT", 8080))
uvicorn.run(app, host="0.0.0.0", port=port)
transport = os.getenv("MCP_TRANSPORT", "sse")

@bijanvakili bijanvakili Aug 19, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Based on the MCP specification here, the sse transport is now deprecated and we should be using streamable HTTP instead:
Streamable HTTP was introduced in protocol version 2025-03-26 
as a replacement for the [HTTP+SSE transport](https://modelcontextprotocol.io/specification/2024-11-05/basic/transports#http-with-sse) 
from protocol version 2024-11-05.
  1. Additionally, there is an outstanding bug with the Antigravity CLI which shows that their SSE client implementation is broken.

See other comment. I recommend switching this to the http transport.

mcp.run(transport=transport, host="0.0.0.0", port=port)
16 changes: 8 additions & 8 deletions street_view_insights/mcp_server/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
mcp
google-genai
google-cloud-bigquery
google-cloud-storage
pillow
numpy
uvicorn
fastapi
mcp ~= 1.29

@bijanvakili bijanvakili Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can remove mcp from this list.

fastmcp already includes it here (ie. it's a transitive dependency).

fastmcp ~= 3.4.7
google-genai ~= 2.17
google-cloud-bigquery ~= 3.43
google-cloud-storage ~= 3.13.1
pillow ~= 12.3
numpy ~= 2.5.2
fastapi ~= 0.141.1