Query Petabytes Without

Downloading a Byte

GeoQ uses DuckDB's httpfs extension to stream only the data you need — directly from Amazon S3, Google Cloud Storage, or any HTTPS endpoint — using efficient HTTP range requests.

Get the CLI → ← Back to Home

Storage Providers

Connect Any Cloud, Any Endpoint

All three major cloud object stores plus open HTTP — configured with a single URI prefix.

Amazon S3

Use s3://bucket/path/data.copc.laz directly. Credentials are resolved through the full AWS credential chain: environment variables, ~/.aws/credentials, IAM instance roles, or explicit secrets.

s3://IAM RolesAWS SDK Chain

Google Cloud Storage

Use gs://bucket/path/data.parquet with GCS service account credentials injected via the DuckDB aws extension secret management.

gs://Service AccountGCS

HTTPS / Public Endpoints

Any HTTPS URL pointing to a supported format works out of the box — USGS national datasets, OpenTopography COPCs, or your own CDN. No credentials needed for public data.

https://Public DataCDN

How It Works

HTTP Range Requests

Cloud-native formats like COPC, COG (GeoTIFF), and GeoParquet embed spatial indexes and metadata at known byte offsets. DuckDB reads only those offsets — fetching a 10 MB spatial tile from a 100 GB dataset uses only those 10 MB.

This makes GeoQ fast even over high-latency internet connections, because the query engine pushes spatial predicates into the file reader before any network I/O happens.

STEP 1

Read spatial index / header

(~4 KB range request)

STEP 2

Resolve matching tiles/chunks from bbox

(spatial index lookup)

STEP 3

Fetch only matched byte ranges

(e.g. 3 × 2 MB = 6 MB of 100 GB)

RESULT

SQL query executed in-memory → output

Credential Management

Zero-Config Credentials

AWS Credential Chain

GeoQ resolves credentials in order: explicit --aws-key flags → environment variables (AWS_ACCESS_KEY_ID) → ~/.aws/credentials profile → EC2/ECS instance metadata IAM role.

DuckDB Secrets API

Store credentials once with CREATE PERSISTENT SECRET in the DuckDB secrets manager. They are encrypted at rest and reused across sessions — no env-var juggling.

Public Data — No Config

For anonymous-access S3 buckets or HTTPS endpoints, no credentials are needed. Just pass the URI and GeoQ detects and configures anonymous access automatically.

CLI Examples

Cloud Source Commands

# Query a COPC point cloud directly from S3 (uses IAM role automatically)
geoq cloud point-cloud \
  --source s3://my-lidar-bucket/surveys/city.copc.laz \
  --bbox "-122.5,37.7,-122.3,37.9" \
  --output csv

# Query a Cloud-Optimized GeoTIFF from GCS
geoq cloud raster \
  --source gs://public-dem/ot_be_Brus_2020_LAZ_2021_cloud.tif \
  --band 1 \
  --output csv

# Query a public GeoParquet file over HTTPS
geoq cloud vector \
  --source https://data.source.coop/example/parcels.parquet \
  --sql "SELECT apn, area_sqft, geom FROM features WHERE area_sqft > 5000 LIMIT 100" \
  --output geojson

# Query a multi-file glob from S3 (st_read_multi extension)
geoq cloud vector \
  --source "s3://my-bucket/parcels/*.fgb" \
  --output geojson
    

Your Data, Still in the Cloud

No staging. No egress surprises. GeoQ reads only what the query needs.

Get the CLI → ← Back to Home