Skip to content

Commit 4541105

Browse files
feat(bench): simplify suites and enrich comparison metadata (#6)
* feat(bench): simplify suites and enrich comparison metadata - introduce shared task runner (`bench/lib/task-runner.js`) and suite hub (`bench/suites.js`), then refactor `run-full suite`, `run-quick-suite`, and the CLI to use the unified pipeline instead of bespoke scripts - extract fixture metadata (title, domain, link/script/table counts, word count) for every benchmarked file and persist it in `bench/.results/comparison-latest.json`; console summaries now show domain/context and absolute deltas alongside ratios - treat benchmark outputs/fixtures as first-class tracked artifacts by adding `.results` and root `fixtures` to Git LFS; update `.gitignore` so those directories remain visible while developers can opt out via LFS - update README and benchmark artifacts to reflect the new metadata ordering and fresh results * fix(h2m): types * refactor(bench): unify benchmark tooling and refresh baseline - standardise `runBenchmark` defaults (100 iterations, warmups, 5MB cap, full fixtures) - wire the shared config through baseline, regression, and README refresh flows - drop tracked benchmark artifacts now covered by .gitignore - regenerate baseline metrics and comparison outputs with the new configuration - document the workflow updates across README, docs, and bench guides Ensures regression gates, published metrics, and local benches stay aligned while keeping the repo free of generated artifacts. * chore: add changesets * test(fixtures): increase timeout for HTML fixture processing - Added a timeout of 60 seconds to the HTML fixture processing test to prevent potential timeouts during execution. - Refactored the test structure for improved readability. * fix(actions): optimize PR runs, conditions and permissions
1 parent a104ace commit 4541105

146 files changed

Lines changed: 181476 additions & 4240 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
"h2m-parser": minor
3+
---
4+
5+
Refined the htmlparser2 renderer to stream nodes directly, which removes the domhandler dependency and unlocks faster parse times on large fixtures.
6+
Expose granular `convert_parse`, `convert_render`, and `convert_postprocess` telemetry events (propagated through `H2MParser` and `htmlToMarkdown`) so downstream tooling can pinpoint hotspots while staying backwards compatible.

.gitattributes

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
tests/fixtures/** filter=lfs diff=lfs merge=lfs -text
2+
bench/datasets/** filter=lfs diff=lfs merge=lfs -text
3+
bench/.results/** filter=lfs diff=lfs merge=lfs -text
4+
fixtures/** filter=lfs diff=lfs merge=lfs -text

.github/workflows/performance-regression.yml

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,9 @@
11
name: Performance Regression Check
22

3+
concurrency:
4+
group: performance-regression-${{ github.event.pull_request.number || github.ref }}
5+
cancel-in-progress: true
6+
37
on:
48
pull_request:
59
paths:
@@ -15,7 +19,7 @@ permissions:
1519

1620
jobs:
1721
regression:
18-
if: github.event_name == 'pull_request'
22+
if: github.event_name == 'pull_request' && contains(join(github.event.pull_request.labels.*.name, ','), 'test-regression')
1923
runs-on: ubuntu-latest
2024
permissions:
2125
pull-requests: write
@@ -121,13 +125,14 @@ jobs:
121125
update-baseline:
122126
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
123127
runs-on: ubuntu-latest
128+
permissions:
129+
contents: write
124130

125131
steps:
126132
- name: Checkout repository
127133
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
128134
with:
129135
fetch-depth: 0
130-
persist-credentials: false
131136

132137
- name: Setup pnpm
133138
uses: pnpm/action-setup@a7487c7e89a18df4991f7f222e4898a00d66ddda # v4.1.0

.github/workflows/test.yml

Lines changed: 4 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,9 @@
11
name: Test
22

3+
concurrency:
4+
group: test-${{ github.event.pull_request.number || github.sha }}
5+
cancel-in-progress: true
6+
37
on:
48
push:
59
branches: [main]
@@ -47,55 +51,3 @@ jobs:
4751

4852
- name: Build
4953
run: pnpm build
50-
51-
benchmark:
52-
name: Performance Check
53-
runs-on: ubuntu-latest
54-
if: github.event_name == 'pull_request'
55-
permissions:
56-
contents: read
57-
pull-requests: write # Needed to comment on PR
58-
steps:
59-
- name: Checkout PR
60-
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 #v5.0.0
61-
with:
62-
persist-credentials: false
63-
64-
- name: Setup pnpm
65-
uses: pnpm/action-setup@a7487c7e89a18df4991f7f222e4898a00d66ddda #v4.1.0
66-
67-
- name: Setup Node.js
68-
uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 #v5.0.0
69-
with:
70-
node-version: 24
71-
cache: 'pnpm'
72-
73-
- name: Install dependencies
74-
run: pnpm install --frozen-lockfile
75-
76-
- name: Build
77-
run: pnpm build
78-
79-
- name: Run benchmark
80-
run: node bench/check-regression.js
81-
82-
- name: Comment PR
83-
if: always()
84-
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd #v8.0.0
85-
with:
86-
script: |
87-
const fs = require('fs');
88-
const path = require('path');
89-
90-
// Read regression report if it exists
91-
const reportPath = path.join(process.cwd(), 'bench', '.results', 'regression-report.md');
92-
if (fs.existsSync(reportPath)) {
93-
const report = fs.readFileSync(reportPath, 'utf8');
94-
95-
await github.rest.issues.createComment({
96-
issue_number: context.issue.number,
97-
owner: context.repo.owner,
98-
repo: context.repo.repo,
99-
body: report
100-
});
101-
}

.gitignore

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,24 @@ node_modules
22
.DS_Store
33
dist
44
coverage
5-
.bench
65
bench/output
7-
fixtures/
6+
bench/artifacts/
7+
bench/datasets/
8+
# Local benchmark scratch space
9+
.bench
10+
# Ignore volatile benchmark outputs (canonical artifacts are tracked via LFS)
11+
bench/.results/latest.json
12+
bench/.results/regression-report.md
13+
bench/.results/fetch-e2e-latest.json
14+
bench/.results/workflows-latest.json
15+
bench/.results/memory-latest.json
16+
bench/.results/token-usage-latest.json
17+
bench/.results/bundle-size-*.json
18+
bench/output/
19+
bench/Users/
20+
bench/README-benchmarks.md
21+
# Allow tracked benchmark results and fixtures (managed via Git LFS)
22+
# (keep optional local fixture packs elsewhere)
823
.vscode
924
.idea
1025
*.log

CONTRIBUTING.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ Thanks for your interest in improving h2m-parser! Keeping cognitive load low is
1111
## Coding standards
1212

1313
- Write TypeScript in `src/` and keep exports explicit in `src/index.ts`.
14-
- Favor early returns and descriptive helper functions to keep cognitive load low—see `/Users/gustavovalverde/dev/personal/micro-play/cognitive-load.md` for the philosophy behind our style.
14+
- Favor early returns and descriptive helper functions to keep cognitive load low—see [`cognitive-load.md`](cognitive-load.md) for the philosophy behind our style.
1515
- Prefer pure functions; when mutating shared state, encapsulate the mutation and document it.
1616
- Keep custom translators compact—wrap individual tags with clear helpers instead of adding deep inheritance or cross-cutting state.
1717
- Use Biome for formatting and linting (`pnpm lint:fix`, `pnpm format:fix`).

0 commit comments

Comments
 (0)