Skip to content

Commit 82605fd

Browse files
rustyconoverclaude
rustyconover
and
claude
committed
Add Docker image + release workflow; refactor worker into wheel-importable module
Ship packaging for vgi-news so it can be released and containerized: - Refactor: move the `news` catalog + `NewsWorker` + `main()` from the repo-root PEP 723 script into `vgi_news/worker.py` (wheel-importable). `news_worker.py` becomes a thin PEP 723 shim re-exporting them, so `uv run news_worker.py` (Makefile / ci/run-integration.sh / tests / vgi-lint location) is unchanged. - pyproject: add a `serve` extra (= vgi-python[http]), a `vgi-news-worker` console script (-> vgi_news.worker:main), and project URLs. - bin/vgi-news-worker: `uv run` launch wrapper for a filesystem ATTACH LOCATION. - Docker: Dockerfile (python:3.13-slim, `pip install '.[serve]'`), docker-entrypoint.sh (http default via `vgi-serve --http`; stdio), .dockerignore, and docker-publish.yml (gated on ci.yml -> shared vgi-actions reusable workflow; main -> :edge, vX.Y.Z -> :X.Y.Z/:X.Y/:latest). - Release: release.yml (on Release published -> ci gate -> uv build sdist+wheel -> twine check -> attach; PyPI job gated behind vars.PYPI_PUBLISH, left off) plus ci/check-version.sh (tag == pyproject version). - ci.yml: add `workflow_call:` so docker-publish/release can gate on it. Local gates: ruff format+check clean, mypy --strict clean, pydoclint clean, pytest 28 passed. Docker image builds; HTTP /health -> 200; stdio speaks Arrow IPC. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011aU684xHBa4PXH5sqzLT5z
1 parent f613ca6 commit 82605fd

12 files changed

Lines changed: 575 additions & 303 deletions

File tree

‎.dockerignore‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
.venv
2+
__pycache__
3+
*.pyc
4+
.git
5+
dist
6+
build
7+
*.egg-info
8+
.mypy_cache
9+
.ruff_cache
10+
.pytest_cache
11+
duckdb_unittest_tempdir
12+
.vgi-review-cache.json
13+
.vgi-sim-cache.json
14+
.DS_Store
15+
tests
16+
test
17+
docs

‎.github/workflows/ci.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ on:
1515
branches: [main]
1616
paths-ignore: ['README.md']
1717
workflow_dispatch:
18+
# Allow docker-publish.yml / release.yml to gate on this CI as a reusable workflow.
19+
workflow_call:
1820

1921
permissions:
2022
contents: read

‎.github/workflows/docker-publish.yml‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# Copyright 2026 Query Farm LLC - https://query.farm
2+
#
3+
# Build + publish the multi-arch vgi-news image to ghcr.io. The heavy lifting
4+
# (multi-arch build -> /health smoke -> push-by-digest -> manifest -> cosign-sign) lives
5+
# in the shared reusable workflow Query-farm/vgi-actions/docker-publish.yml; this file
6+
# gates on the repo's own CI and supplies the worker-specific inputs.
7+
#
8+
# Tag-driven: vX.Y.Z -> :X.Y.Z / :X.Y / :latest; push to main -> :edge.
9+
name: Publish image to ghcr.io
10+
11+
on:
12+
push:
13+
branches: [main]
14+
tags: ['v*.*.*']
15+
workflow_dispatch:
16+
17+
permissions:
18+
contents: read
19+
20+
concurrency:
21+
group: docker-${{ github.workflow }}-${{ github.ref }}
22+
cancel-in-progress: true
23+
24+
jobs:
25+
ci:
26+
uses: ./.github/workflows/ci.yml
27+
28+
publish:
29+
needs: [ci]
30+
permissions:
31+
contents: read
32+
packages: write
33+
id-token: write
34+
attestations: write
35+
uses: Query-farm/vgi-actions/.github/workflows/docker-publish.yml@v1
36+
secrets: inherit
37+
with:
38+
image_name: vgi-news
39+
smoke_import: "vgi_news, httpx, pyarrow"
40+
health_port: "8000"
41+
version_check_cmd: "ci/check-version.sh"

‎.github/workflows/release.yml‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
name: Release
2+
3+
# On a published GitHub Release: gate on the full test suite (ci.yml), build the sdist +
4+
# wheel, attach them to the Release, and (opt-in) publish to PyPI.
5+
#
6+
# vgi-python workers ship to PyPI, NOT as a compiled binary. PyPI upload uses Trusted
7+
# Publishing (OIDC — no API tokens) and is GATED behind the repo variable PYPI_PUBLISH:
8+
# it stays skipped until a maintainer sets `PYPI_PUBLISH=true` AND configures a Trusted
9+
# Publisher on PyPI for the `vgi-news` project pointing at:
10+
# owner: Query-farm · repo: vgi-news · workflow: release.yml · environment: pypi
11+
# Until then the Release still ships installable sdist+wheel assets, CI-gated and green.
12+
13+
on:
14+
release:
15+
types: [published]
16+
workflow_dispatch:
17+
18+
permissions:
19+
contents: read
20+
21+
jobs:
22+
ci:
23+
uses: ./.github/workflows/ci.yml
24+
25+
build:
26+
name: Build + attach distributions
27+
needs: [ci]
28+
runs-on: ubuntu-latest
29+
permissions:
30+
contents: write # attach assets to the Release
31+
steps:
32+
- uses: actions/checkout@v4
33+
- name: Install uv
34+
uses: astral-sh/setup-uv@v5
35+
- name: Check version matches tag
36+
if: github.event_name == 'release'
37+
run: ci/check-version.sh "${{ github.event.release.tag_name }}"
38+
- name: Build sdist + wheel
39+
run: uv build --sdist --wheel --out-dir dist .
40+
- name: Validate metadata
41+
run: uvx twine check dist/*
42+
- name: Upload build artifacts
43+
uses: actions/upload-artifact@v4
44+
with:
45+
name: dist
46+
path: dist/
47+
- name: Attach distributions to the Release
48+
if: github.event_name == 'release'
49+
env:
50+
GH_TOKEN: ${{ github.token }}
51+
run: gh release upload "${{ github.event.release.tag_name }}" dist/* --clobber
52+
53+
publish:
54+
name: Publish to PyPI (opt-in)
55+
needs: [build]
56+
# Skipped until the maintainer enables PyPI Trusted Publishing (see header). A skipped
57+
# job does not fail the workflow, so releases stay green before PyPI is configured.
58+
if: ${{ vars.PYPI_PUBLISH == 'true' }}
59+
runs-on: ubuntu-latest
60+
environment: pypi
61+
permissions:
62+
id-token: write # required for Trusted Publishing
63+
steps:
64+
- name: Download artifacts
65+
uses: actions/download-artifact@v4
66+
with:
67+
name: dist
68+
path: dist/
69+
- name: Publish
70+
uses: pypa/gh-action-pypi-publish@release/v1

‎Dockerfile‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Copyright 2026 Query Farm LLC - https://query.farm
2+
#
3+
# Single image serving BOTH transports of the vgi-news worker:
4+
# docker run ... IMG -> HTTP server on $PORT (default 8000; /health, VGI RPC)
5+
# docker run -i ... IMG stdio -> stdio worker DuckDB spawns on-host
6+
# See docker-entrypoint.sh. GDELT (the default provider) is keyless and free; the
7+
# optional NewsAPI provider's key is supplied at query time via a DuckDB `newsapi`
8+
# secret — never baked into the image.
9+
# syntax=docker/dockerfile:1
10+
FROM python:3.13-slim
11+
12+
ARG VERSION=0.0.0
13+
ARG GIT_COMMIT=unknown
14+
ARG SOURCE_URL=https://github.com/Query-farm/vgi-news
15+
16+
LABEL org.opencontainers.image.title="vgi-news" \
17+
org.opencontainers.image.description="Global news search (GDELT, NewsAPI) for DuckDB via VGI (stdio + HTTP)" \
18+
org.opencontainers.image.source="${SOURCE_URL}" \
19+
org.opencontainers.image.version="${VERSION}" \
20+
org.opencontainers.image.revision="${GIT_COMMIT}" \
21+
org.opencontainers.image.licenses="MIT" \
22+
farm.query.vgi.transports='["http","stdio"]'
23+
24+
ENV PYTHONUNBUFFERED=1 \
25+
PIP_NO_CACHE_DIR=1 \
26+
PIP_DISABLE_PIP_VERSION_CHECK=1 \
27+
PORT=8000
28+
29+
WORKDIR /app
30+
31+
# curl backs the HEALTHCHECK and the CI /health smoke.
32+
RUN apt-get update \
33+
&& apt-get install -y --no-install-recommends curl \
34+
&& rm -rf /var/lib/apt/lists/*
35+
36+
# Install the worker + HTTP-serving extra from the source tree.
37+
COPY pyproject.toml README.md LICENSE ./
38+
COPY vgi_news ./vgi_news
39+
RUN pip install '.[serve]'
40+
41+
COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
42+
RUN chmod +x /usr/local/bin/docker-entrypoint.sh
43+
44+
EXPOSE 8000
45+
HEALTHCHECK --interval=30s --timeout=3s --start-period=8s \
46+
CMD curl -fsS "http://localhost:${PORT}/health" || exit 1
47+
48+
ENTRYPOINT ["docker-entrypoint.sh"]

‎bin/vgi-news-worker‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
#!/usr/bin/env bash
2+
# Launch wrapper so DuckDB's vgi extension can spawn this Python worker via a plain
3+
# filesystem LOCATION:
4+
# ATTACH 'news' (TYPE vgi, LOCATION '/path/to/vgi-news/bin/vgi-news-worker');
5+
#
6+
# The worker speaks Arrow IPC over stdin/stdout (Worker.run()). `uv run` resolves the
7+
# project's environment (vgi-python + httpx + pyarrow) and execs the console script;
8+
# uv's own diagnostics go to stderr, so stdout stays a clean Arrow IPC stream.
9+
set -euo pipefail
10+
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
11+
exec uv run --project "$REPO_ROOT" --quiet vgi-news-worker "$@"

‎ci/check-version.sh‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
#!/usr/bin/env bash
2+
# Assert the release tag matches the version in pyproject.toml, so a `vX.Y.Z` release can
3+
# never publish a wheel built from a mismatched version. Called with the tag as $1
4+
# (e.g. "v0.1.0"). Dependency-free (no jq/uv required).
5+
set -euo pipefail
6+
7+
tag="${1:?usage: check-version.sh <tag>}"
8+
want="${tag#v}" # strip a leading 'v'
9+
have="$(grep -m1 '^version' pyproject.toml | sed -E 's/.*"([^"]+)".*/\1/')"
10+
11+
if [ "$want" != "$have" ]; then
12+
echo "Version mismatch: tag ${tag} (-> ${want}) != pyproject.toml version ${have}" >&2
13+
exit 1
14+
fi
15+
echo "Version OK: pyproject.toml ${have} matches tag ${tag}"

‎docker-entrypoint.sh‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
#!/bin/sh
2+
# Dispatch the vgi-news image into one of its transports:
3+
# http (default) HTTP server on $PORT (vgi-serve --http: /health + VGI RPC)
4+
# stdio a worker DuckDB spawns over stdio (on-host execution)
5+
# * exec'd verbatim (debug escape hatch)
6+
set -e
7+
case "${1:-http}" in
8+
http) exec vgi-serve vgi_news.worker:NewsWorker --http --host 0.0.0.0 --port "${PORT:-8000}" ;;
9+
stdio) shift 2>/dev/null || true; exec vgi-news-worker "$@" ;;
10+
*) exec "$@" ;;
11+
esac

0 commit comments

Comments
 (0)