Geodallion

Multi-criteria spatial site-scoring pipeline on open geospatial data (OpenStreetMap, SRTM).

Status: work in progress. Scaffold and decision records first; runnable end-to-end pipeline lands incrementally.

What this is

A Medallion-pattern data pipeline that scores candidate locations on open geospatial data (OpenStreetMap power infrastructure, SRTM terrain). Scoring is multi-factor (grid proximity, terrain, residential distance) and multi-preset, rather than single-criterion optimization. The engine generalises to any siting problem combining these criteria; the current showcase application is datacenter siting, with five presets — Balanced, Hyperscaler, Edge, Green, Regulatory Compliant — rendered as a map UI for exploring candidate sites.

Initial geographic scope is Regierungsbezirk Köln (Germany, ~7,400 km²), with NRW as the next-step extension via the Geofabrik PBF.

Stack

Layer	Technology
Object storage	AWS S3 (Raw, Bronze)
Bronze transform	AWS Glue + Apache Sedona (Spark) → Parquet
Warehouse	Snowflake (RAW / BRONZE / SILVER / GOLD)
Transforms	dbt-snowflake (Silver / Gold)
Orchestration	AWS Step Functions
API	FastAPI on Lambda + API Gateway
Frontend	TypeScript + React + Leaflet on S3 + CloudFront
IaC	Terraform (AWS + Snowflake)

Architecture

Two input streams land in S3 raw: OSM (Geofabrik PBF for NRW-scale runs; Overpass JSON for the initial Regierungsbezirk-Köln one-shot ingest) and SRTM (1-arc-second elevation tiles). The Bronze transform (Glue + Sedona) projects both into typed Parquet in S3 bronze/. Snowflake reads via External Stage + COPY INTO into Bronze tables → dbt Silver (geometry assembly, tag normalisation) → dbt Gold (scoring) → API → Map UI.

Initial scope is Regierungsbezirk Köln; the pipeline shape is national — NRW Geofabrik PBF and further Bundesländer slot in via the same Bronze path.

S3 stays the source-of-truth tier for raw and bronze (cheap, audit-friendly); Snowflake holds the queried subset (Silver + Gold).

Architecture decisions

Architectural decisions are captured as Michael-Nygard-style ADRs in docs/adr/. See the index for the current list.

Development workflow

Built with Claude Code (https://claude.com/claude-code) as an AI pair programmer, under a deliberately constrained process:

• Architecture is human-owned. Structural choices are decided and recorded as ADRs (docs/adr/) before implementation — the AI accelerates the build, it does not choose the architecture.
• Every change is reviewed before it lands, and committed by hand.
• The result stands on its own. Pinned dependencies, a committed lockfile, terraform validate in the saved-plan flow, and the ADR trail keep the output reproducible and auditable regardless of how it was produced.

Roadmap

• Bronze layer: hardened S3 raw bucket + Glue/Sedona PBF → typed Parquet (Terraform-managed)
• Warehouse: Snowflake RAW / BRONZE / SILVER / GOLD schemas, External Stage, COPY INTO
• Modeling: dbt Silver (geometry assembly, tag normalisation) + Gold (multi-preset scoring)
• Serving: FastAPI on Lambda + React / Leaflet SPA against generated OpenAPI
• Orchestration: Step Functions stitching fetch (Glue Python Shell) → Bronze transform (Glue + Sedona) → Snowflake COPY INTO → dbt
• H3 hexagonal heatmap aggregation (stretch)

Setup

Python 3.13 required. Reproducible install uses uv (https://docs.astral.sh/uv/):

uv sync

This resolves pyproject.toml + uv.lock into a local .venv/.

Running the infrastructure

Current scope spans the S3 raw tier (the hardened bucket described in ADR-0005), an in-AWS fetch path (Glue Python Shell, ADR-0009), and the Snowflake side of the External-Stage handshake (Storage Integration + warehouse / database / schemas, ADR-0007). Sources are readable without execution; the live deployment requires:

• AWS credentials (AWS_PROFILE or AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY in the environment)
• Terraform >= 1.6 (install via tfenv, Homebrew, or https://developer.hashicorp.com/terraform/install)
• terraform.tfvars with expected_aws_account_id (target AWS account ID; the bucket's precondition fails fast if the active profile resolves to a different account) plus snowflake_organization_name, snowflake_account_name, snowflake_user, and snowflake_token (60-day Programmatic Access Token; PAT-generation and Network-Policy guidance in examples/terraform.tfvars.example).

terraform init
terraform fmt -check
terraform validate
terraform plan -out=tfplan
terraform apply tfplan

(terraform fmt -check is read-only; if it reports formatting drift, run terraform fmt locally before the plan / apply sequence.)

Conventions

• Python 3.13, pinned via requires-python = "==3.13.*" and uv.lock
• ruff + pyright (standard mode); Pylance reads the same [tool.pyright] config in VS Code, with minor default divergences

License

Code is licensed under the MIT License.

Data sources retain their respective upstream licenses:

• OpenStreetMap data — Open Database License (ODbL) — https://www.openstreetmap.org/copyright
• SRTM elevation — public domain (NASA) — https://www2.jpl.nasa.gov/srtm/