Skip to content

Releases: ttezer/gtfs-analyzer

v0.6.0

Choose a tag to compare

@ttezer ttezer released this 17 Jul 11:56

Scores are not comparable to 0.5.0. False positives were removed from five rules and two rules were regrouped, so a feed's numbers move even though the feed is unchanged. Re-baseline any Golden snapshots after upgrading.

This release comes out of running 250 feeds sampled across 71 countries from the MobilityData catalogue and comparing every finding against MobilityData's own report for the same snapshot.

Our test corpus had been three feeds — BART, Tokyo Toei, TriMet. Every bug below is one that corpus could not surface: they need non-ASCII text, minute-rounded times, a repeated stop name, a ", " separator, or a multi-agency feed.

Fixed

Valid feeds were rejected outright when a header read split a UTF-8 character. K1 reads only the first 8 KB of trips.txt, stop_times.txt and calendar_dates.txt; when that cut landed inside a multi-byte character the truncated tail was read as invalid encoding, and on a required file that became a fatal error — the feed produced no analysis at all. Japanese, Lithuanian and Thai feeds were rejected while MobilityData reported two of them with zero errors.

FLG_002 reported false criticals for valid Fares v2 networks. A network_id may be declared in networks.txt, routes.txt, or route_networks.txt; only the first was recognised. On TriMet this removed all 7 criticals and raised the Publication Score from 89.3 to 100.

STM_012 called a stop pair impossible on a fixed 1 km distance. When two stops share a whole-minute timestamp the real travel time is unknown — rounding hides 0–59 seconds — yet a bus covering 1.1 km within that minute only needs 66 km/h. The distance must now be unreachable even if a full minute had elapsed (max_speed_kmh(route_type) / 60). One feed: 42 findings → the single 29.4 km jump MobilityData also reports.

TRP_020 flagged headsigns that correctly name the last stop. The rule excluded the terminal by stop_id but matched by name, so a trip was flagged whenever an earlier stop shared the terminal's name. One feed: 9,162 → 171. On Tokyo Toei all 156 findings were this false positive.

RTS_019 treated different operators' route numbers as duplicates. Two agencies each running a route "10" is ordinary; GTFS scopes name uniqueness to one route_type under one agency_id. One feed: 32 → 3, matching MobilityData exactly.

STP_030 counted entrances as a station's children. A vehicle only stops at a platform, so a station holding six entrances and no platform was silently accepted.

RTS_023 ignored route_desc that repeats the short name. It compared only against route_long_name; MobilityData checks both. On one feed all 436 of its findings were short-name matches and we reported none.

Changed

STM_014 findings are grouped per route, direction and segment. A single bad segment produced one finding per trip crossing it — on TriMet, 497 findings pointed at the same two segments.

DQ_016 reports one summary per file instead of one finding per row. Trailing whitespace is one producer habit — usually ", " as the separator — so it appears on every row of every file. One feed produced 2,361,873 findings (98.6% of its total, against MobilityData's 12) — the volume that exhausts browser memory. Detection is unchanged and still correct: RFC 4180 makes the space part of the field. That feed drops to 7 findings, its total from 2.4M to 43k.

Added

validate_with_today (WASM). The browser evaluated calendar rules against the machine's clock, so a run could not be reproduced and a diff mixed code changes with the date shift. The date can now be supplied explicitly, mirroring the CLI's --today.

Internal

  • RULES.md / .en / .ja are generated from the rule registry but nothing enforced regeneration and they had drifted; a test now ties them together.
  • ARC_029 (decompression guard) is proven end-to-end: a zip bomb returns the expected fatal, a legitimate small high-ratio file does not trip the guard.

Full changelog: https://github.com/ttezer/gtfs-analyzer/blob/v0.6.0/CHANGELOG.md

v0.5.0 — Authority-based classification + CLI

Choose a tag to compare

@ttezer ttezer released this 11 Jul 11:22

Scores are not comparable to 0.4.0. This release continues the authority-based reclassification started in 0.4.0: more rules moved between classes, so a feed's Overall and Publication scores can shift even though the feed is unchanged and detection is identical. Re-baseline any Golden snapshots after upgrading.

Added

  • Command-line interface. A new gtfs-cli crate ships the gtfs-analyzer binary: gtfs-analyzer validate feed.zip with --json, --summary, --rule <ID>, --severity <level>, --config <file>, --today <YYYYMMDD>. Exit codes: 0 no notices, 1 notices present, 2 fatal / config error. It runs the same validation core as the web app — no separate logic.

Changed

  • Authority-based classification completed. Every Spec rule card must now cite an explicit gtfs.org field anchor, enforced by a test gate. Rules lacking an explicit GTFS Schedule normative basis were reclassified off Spec/Interop:
    • XFL_020 / XFL_021 (transfers cross-file consistency): SpecQuality.
    • TRP_017 (frequencies trip missing stop_times): InteropQuality (MobilityData's unused_trip is a different condition — no exact-notice parity).
    • Google Transit checks grouped under a dedicated GoogleTransitInterop authority.
    • Detection is unchanged — the same issues are still reported; only class labels and scoring weights moved.
  • ATR_009 was mislabelled "attribution_phone invalid"; it actually flags rows where more than one of agency_id / route_id / trip_id is set. Title, message, and en/tr/ja locales corrected (no behaviour change).
  • Interop rules no longer appear in the R1 publishability report.

Fixed

  • On the file map, shape geometry for trip-context rules (e.g. STM_014, OPR_007, STM_017) is now fetched on demand in deferred (large-shape) mode, so the route line renders alongside the stop pins.

Full changelog: https://github.com/ttezer/gtfs-analyzer/blob/main/CHANGELOG.md

v0.3.1 — SHP_005 false-positive fix

Choose a tag to compare

@ttezer ttezer released this 02 Jul 10:45

Fixed

  • SHP_005 (shape_dist_traveled decreasing) now evaluates values in shape_pt_sequence order instead of file-row order. GTFS does not require shapes.txt rows to be pre-sorted by sequence, so feeds that list shape points out of sequence order (valid) produced spurious CRITICAL errors — e.g. all 5,774 on the Athens feed (mdb-3220) were false positives (file order shows 5,774 decreases; sequence order shows 0, matching MobilityData's decreasing_shape_distance). The check moved to the stage that already sorts shape points by sequence; no other rule was affected. On Athens: overall 79.2 → 84.8, publish/spec 86.2 → 100.

v0.3.0 — URL feed loading + calendar-analytics fixes

Choose a tag to compare

@ttezer ttezer released this 02 Jul 08:49

Added

  • Load a GTFS feed directly from a URL (#45): the browser fetches the feed client-side — no backend, and the feed data stays on the device. Works for CORS-enabled hosts such as the Mobility Database catalog; other hosts fall back cleanly to download-and-drop.
  • Run-to-run comparison page (#21): compare the current feed against an older Golden JSON — fixed/new/increased/decreased rules, score, severity, class, feed-size, date-range and normalized notice-density deltas.
  • Golden JSON v4 with a UTC generated_at, file row/byte counts, feed metrics, analysis settings, rule metadata and severity/class totals. Golden v1–v3 remain importable with marked limitations.

Changed

  • The downloaded Golden snapshot is byte-deterministic again for git-tracked regression baselines (#42): it omits the per-second generated_at while keeping the day-granular validate_date; the in-app comparison keeps its live timestamp.
  • CAL_006 (all weekday columns are 0) and RTS_020 (route_url equals agency_url) are now Info, matching MobilityData; CAL_006 is reframed as a dates-only hint rather than "service never runs".
  • The Memory64 package (pkg64) is now optimized with wasm-opt (~2.37 MB → ~2.02 MB).

Fixed

  • STP_021 no longer flags valid Entrance/Exit stops. It now targets Boarding Areas (location_type=4) that lack a platform parent, removing high-severity false positives (e.g. 132 on the BART feed) that skewed the quality score.
  • Calendar-analytics over-fire on feeds that model one operational calendar as many service-id variants (#30): CAL_007/CAL_012 gaps and CAL_013 expiries are aggregated by signature, so each is reported once with the affected services listed.

v0.2.0 — Automatic WASM64 runtime for large feeds

Choose a tag to compare

@ttezer ttezer released this 28 Jun 18:22

GTFS Analyzer v0.2.0 introduces automatic dual-runtime selection. Browsers with WebAssembly Memory64 support use the WASM64 serial engine, allowing feeds that exceed the wasm32 4 GB linear-memory ceiling to be analyzed directly in the browser. Other browsers automatically fall back to WASM32 threaded or serial mode.\n\nHighlights:\n- Automatic WASM64-first runtime selection with safe WASM32 fallback\n- Active engine indicator in the upload interface and diagnostic exports\n- URL overrides for WASM32, WASM64, and serial diagnostics\n- CI builds and validates all three runtime packages\n- Deterministic wasm32/wasm64 Golden output parity check\n- Verified 250 MB feed with 34.5 million stop_times at approximately 5.3 GB memory\n\nAll Rust tests, TypeScript tests, security checks, production builds, output-parity checks, and Playwright E2E tests passed.

v0.1.4 — Interactive GTFS File Map + large-feed memory optimization

Choose a tag to compare

@ttezer ttezer released this 24 Jun 19:42

This release adds the Interactive GTFS File Map and substantially improves the reliability of large feeds under WebAssembly. Before 0.1.4, VBB-class feeds (e.g. VBB Berlin, ~282k trips / 6.3M stop_times) either ran close to the 4 GB wasm32 memory ceiling or failed during result serialization; 0.1.4 brings peak memory down to roughly 2.74 GB, an improvement of about 1.3 GB of headroom.

Added

  • Interactive GTFS File Map that combines GTFS file relationships with the real analyzer findings for the loaded feed.
  • Per-file finding count, severity, row count, and missing/clean status.
  • Direct navigation from a file into the filtered Detail and Fix views.
  • File-type icons, dark theme, and a mobile layout; non-spec files are shown separately.
  • Turkish, English, and Japanese UI strings for the File Map.
  • Diagnostic information showing the last completed pipeline stage on validation errors and timeouts.

Performance

  • Reduced WASM peak memory on the VBB Berlin test feed from roughly 4.0 GB (or a serialization failure) to about 2.74 GB.
  • CompactStopTime row size reduced from 168 to 96 bytes: dead flag fields removed and stop_headsign boxed.
  • Removed about 454 MB of unused capacity in the stop_times row buffer.
  • Removed the second, borrowed copy of trip_stop_set built in the K4 cross-reference stage, and freed the set after K4 so K6 allocations can reuse that space.
  • Added a large-feed mode to the name index to avoid serialization OOM (notices fall back to raw IDs above a deterministic threshold); small and normal feeds are unchanged.
  • Reduced several high-volume intermediate notice emitters (TRP_020, SHP_022, STP_022, PTH_008).

Fixed

  • Large GTFS feeds that previously approached the wasm32 4 GB limit or ran out of memory can now be analyzed more safely.
  • STM_014 (fast travel) false positives on feeds using extended European route_type codes (S-Bahn, U-Bahn, tram, regional rail) — speed thresholds are now mapped to the correct vehicle category.
  • The File Map no longer disappears entirely when a severity filter is applied; dark-theme file/group colors and file-relationship label readability were corrected.

Known caveat: an intermittent rayon worker crash (memory access out of bounds) was observed in one earlier run; high-water was below 4 GB, so it is not proven to be OOM. The error card now reports the last completed pipeline stage; it will be filed separately if it recurs.

Remaining large-feed runtime optimization is tracked in #37.

v0.1.3

Choose a tag to compare

@ttezer ttezer released this 22 Jun 12:47

Real-feed accuracy release.

Highlights

  • Large false-positive fixes on production feeds: TRP_022 (cross-calendar block overlaps), TRP_011 (named routes), VAT_001 (temporal route splits), TRP_020 (consecutive duplicate stops), CAL_007/CAL_012 (duplicate gap reports).
  • New Golden JSON snapshot export — deterministic per-rule aggregate plus feed metrics, for version-to-version comparison.
  • Rule set grew 518 → 526: CAL_024, GEO_022, AGN_017, JPN_011, STP_037/038, FTR_009/010/011.

Full changelog: https://github.com/ttezer/gtfs-analyzer/blob/v0.1.3/CHANGELOG.md

v0.1.2 — GTFS-JP support

Choose a tag to compare

@ttezer ttezer released this 16 Jun 07:24

⭐ GTFS-JP (Japan profile) support

Profile detection with a GTFS-JP badge, plus 10 GTFS-JP rules (JPN_001–010):

  • ja-Hrkt (kana) readings for stop / route / trip / agency names
  • office_jp / agency_jp / routes_jp foreign-key checks
  • the profile's mandatory translations.txt, fare files and feed_info.txt

Coverage-gap rules (MobilityData v8 comparison)

  • STM_050 — missing timepoint value
  • STM_051 / STM_052 — forbidden Flex pickup / drop-off types
  • FRQ_011 — overlapping frequency
  • SHP_028 / SHP_029 — equal shape distance with differing coordinates

Other improvements

  • RULES.md / RULES.en.md / RULES.ja.md are now generated from the rule registry, so the documented rule list can no longer drift.
  • Fix page: collapsible rule-score summary (R9) and multi-select chip filters (R2).

Rule count is now 518 across 37 groups.

Full changelog: https://github.com/ttezer/gtfs-analyzer/blob/v0.1.2/CHANGELOG.md