Skip to content

Commit ef6499c

Browse files
DaemonF0rgeclaude
andcommitted
Harden and speed up the LSP; add workspace-wide rename
Reliability - Isolate diagnostics: a checker exception clears that file's diagnostics and self-heals instead of downing the connection. - Startup indexing: per-file guard + finally so one unreadable file (or an early throw) cannot hang the client forever. - Fix false "multi-line statement" errors from parens inside string literals. - Fill findReferences generic-arg gaps for global functions/vars. Performance (all version-gated, invalidated on index change) - Memoize getClassHierarchyOrdered (the hottest shared primitive). - O(1) findFunctionOverloads via a lazy method index. - Access-modifier checks skip per-token resolution unless the member is actually private/protected. - Precompute a block-comment mask, removing the O(n^2) backward scan. - Reverse index (symbolsByUri) makes per-keystroke index removal O(edited file) instead of O(whole workspace). - Drop redundant docCache scans covered by globalSymbolIndex; memoize the comment-stripped text. Feature: workspace-wide, identity-checked rename - renameSymbol lexes each workspace file and resolves every occurrence, rewriting call sites / member accesses / type refs (not just decls) while never touching a same-named symbol on an unrelated class. - Guards: skip files whose cached text is out of sync with their AST, skip occurrences inside comments/strings, and only rename symbols defined in workspace (not read-only include-path) files. Simplification - Remove dead modules/handlers (printer, scopes, types stubs, documents, includePaths), unused RuleContext plumbing, and duplicated helpers; extract shared diagnostics dispatch into revalidate.ts. Tests: 306 passing (jest); typecheck clean. handoffs/ documents the deferred larger refactors. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 9a2adb1 commit ef6499c

21 files changed

Lines changed: 989 additions & 558 deletions

handoffs/01-callsite-parser-ast.md

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# Hand-off 1 — Emit `CallSite` nodes from the parser; retire the regex re-tokenizers
2+
3+
**Effort:** L **Risk:** medium **Prerequisite for:** hand-off 2, phase B
4+
5+
## Problem
6+
7+
The parser (`server/src/analysis/ast/parser.ts`) stops at declaration level. Per
8+
function body it emits only:
9+
- `bodyTypeRefs: TypeNode[]` — static-call targets (`ClassName.` where ClassName
10+
is PascalCase), **deduped per body** (first occurrence only).
11+
- `bodyIdentifierRefs: BodyIdentifierRef[]` — standalone identifiers **not**
12+
preceded by `.` and **not** followed by `(`, also **deduped per body**.
13+
14+
Neither captures call sites, member accesses, or every occurrence. So the
15+
diagnostics checkers re-derive call structure from **raw text with regex** on the
16+
per-keystroke path:
17+
18+
- `checkFunctionCallArgs` (`graph.ts:6365`, ~350 lines) — regex-matches calls,
19+
then uses brittle heuristics to decide decl-vs-call (`/(?:void|int|float…)\s+$/`
20+
look-behind, backward `[` walks, `endsWith('new')`). These guards are the top
21+
false-positive source.
22+
- `checkTypeMismatches` (`graph.ts:5378`, ~200 lines) — same shape.
23+
- `parseCallArguments` (`graph.ts:5954`) — manually splits argument text on
24+
top-level commas (handles nested parens/brackets/strings/templates by hand).
25+
26+
Cost: O(fileLength) regex passes on every keystroke, plus a maintenance maze.
27+
28+
## Goal
29+
30+
Have the parser emit a minimal, **complete** (non-deduped) `CallSite[]` for each
31+
function body, and migrate the two checkers to consume it — deleting the
32+
decl-vs-call heuristics and `parseCallArguments`.
33+
34+
```ts
35+
interface CallSite {
36+
calleeName: string;
37+
calleeStart: Position; // start of the callee identifier
38+
calleeEnd: Position;
39+
argRanges: { start: number; end: number }[]; // char offsets of each top-level arg
40+
isDeclaration: boolean; // true for `Type name(...)`-style decls the parser already knows aren't calls
41+
}
42+
```
43+
44+
Add `callSites: CallSite[]` to `FunctionDeclNode` (parser.ts interface ~173).
45+
46+
## Where to emit it
47+
48+
The body loop already tracks `parenDepth` (parser.ts:702) and walks tokens with
49+
`prevPrev`/`prev`/`t`. When `prev` is an identifier and `t.value === '('` at the
50+
statement level, record a `CallSite`: `calleeName = prev.value`, positions from
51+
`prev.start/end`, and `argRanges` by scanning to the matching `)` and splitting on
52+
commas at the call's paren depth. `isDeclaration` is derivable from the same
53+
context the parser already has (a preceding type token / modifier run).
54+
55+
Keep the emission O(n) — it must not regress the hot parse path.
56+
57+
## Plan (incremental — one checker at a time)
58+
59+
1. Add the `CallSite` interface + `callSites` field + emission in the body loop.
60+
Add **parser unit tests** for tricky inputs: nested calls `f(g(x), y)`, string
61+
args with commas `f("a,b", c)`, `new X()`, array args `f({1,2})`, chained
62+
`a.b().c()`. Assert `calleeName`, `argRanges`, `isDeclaration`.
63+
2. Migrate `checkFunctionCallArgs` to iterate `func.callSites` instead of its
64+
regex call-detection. **Diff diagnostics output on a corpus** — ranges must be
65+
byte-for-byte identical (the 303 tests assert exact ranges). Commit only when
66+
the diff is empty (or the deltas are verified improvements).
67+
3. Migrate `checkTypeMismatches` the same way.
68+
4. Delete `parseCallArguments` once no caller remains.
69+
70+
## Do NOT touch
71+
72+
`parseExpressionChainBackward`, `parseChainMembers`, `resolveChainReturnType`
73+
(the completion/hover path). They run on **incomplete cursor text** where the
74+
parser has no `CallSite`. Retiring those is hand-off 2, phase B.
75+
76+
## Risks
77+
78+
- **Exact ranges.** The parser-derived offsets must reproduce current highlight
79+
ranges exactly. Golden-diff before every commit.
80+
- **Hot path.** Emission runs on every keystroke parse — keep it O(n), no
81+
backtracking.
82+
- **`isDeclaration` correctness.** Getting this wrong flips real diagnostics;
83+
cover it heavily in the parser unit tests.
84+
85+
## Done when
86+
87+
- `checkFunctionCallArgs` and `checkTypeMismatches` no longer regex-scan for calls.
88+
- `parseCallArguments` is deleted.
89+
- Parser emits `callSites`; unit tests cover the tricky cases.
90+
- Diagnostics are byte-identical on the corpus; `npm test` green.
Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
# Hand-off 2 — Unify the two chain resolvers
2+
3+
**Effort:** M (phase A) / L (phase B) **Risk:** medium **Phase B depends on:** hand-off 1
4+
5+
## Problem
6+
7+
Expression-chain resolution is implemented twice:
8+
9+
- **Completion/hover path** (works on incomplete cursor text): two steppers,
10+
`resolveChainSteps` (`graph.ts:2172`) and `resolveChainStepsWithIndexing`
11+
(`graph.ts:2259`). The former is a **strict subset** of the latter — same walk,
12+
minus array-indexing support.
13+
- **Diagnostics path** (works on full expression text): `resolveChainReturnType`
14+
(`graph.ts:2320`), `resolveVariableChainType` (`graph.ts:2425`), plus the text
15+
helpers `parseChainMembers` (`graph.ts:2093`) and `countIndexingLevels`
16+
(`graph.ts:1581`).
17+
18+
## Phase A — collapse the two steppers (contained, do this first)
19+
20+
`resolveChainSteps``resolveChainStepsWithIndexing`. Delete the former; route
21+
its callers through the latter by mapping their string arrays to segments:
22+
`calls.map(name => ({ name, isIndexed: false }))`.
23+
24+
**The one behavioral divergence to reconcile:** on `Cast``Class`,
25+
`resolveChainSteps` preserves `templateMap` (via a `continue`) while
26+
`resolveChainStepsWithIndexing` resets it. Pick the **preserve** behavior as
27+
canonical, but implement it **without a literal `continue`** — a bare `continue`
28+
in the indexing stepper would skip the `isIndexed` dereference and regress
29+
`.Cast()[0]`. Guard the typedef/genericArgs rebuild block so `Cast` skips it while
30+
still falling through to the `isIndexed` deref.
31+
32+
- Callers of `resolveChainSteps` to update: `graph.ts:2221`, `2264`, `2278`,
33+
`5969` (verify with a fresh grep — line numbers drift).
34+
- Do **not** fold `resolveMethodCallWithTemplates` in blindly: its extends-clause
35+
generic-arg handling (its `~1250-1257` block) is not present in the stepper and
36+
would need explicit preservation + a generic-inheritance test.
37+
38+
**Test before touching:** add completion + type-mismatch tests exercising a
39+
`Cast()` chain and an indexed chain (`x.Cast()[0]`, `arr.Get(i).Field`). Then run
40+
all 303. Net ≈ 45 lines deleted, one drift point removed.
41+
42+
## Phase B — retire the diagnostics-side text resolvers (gated on hand-off 1)
43+
44+
Once the parser emits expression structure (`CallSite`, and ideally chain
45+
segments) from hand-off 1, the diagnostics callers can stop text-parsing.
46+
47+
Why it's gated: those callers resolve the chain **root** via `getVarTypeAtLine`
48+
(line-scoped locals) rather than `resolveVariableType(doc, pos)`, and they consume
49+
full, non-dot-terminated expression text. Unifying requires a forward segment
50+
parser that reproduces the `isCall` / `isIndexed` / indexing-level semantics that
51+
user-visible type-mismatch / return-type / cast diagnostics depend on **exactly**.
52+
53+
Do it **one caller at a time**, each guarded by golden-diffing diagnostics, only
54+
after hand-off 1 lands. Retire `parseChainMembers` and `countIndexingLevels` when
55+
their last caller is migrated.
56+
57+
## Risks
58+
59+
- Phase A: the `Cast` reconciliation is subtle — wrong and you regress
60+
`.Cast()[0]`. The added completion/type-mismatch tests are the guard.
61+
- Phase B: the diagnostics resolvers back user-visible diagnostics with exact
62+
ranges; reproduce their semantics precisely, incrementally, with diffing.
63+
64+
## Done when
65+
66+
- Phase A: `resolveChainSteps` deleted, one canonical stepper, Cast + indexing
67+
chains verified, 303 green.
68+
- Phase B: `resolveChainReturnType` / `resolveVariableChainType` /
69+
`parseChainMembers` / `countIndexingLevels` gone, diagnostics byte-identical.
Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# Hand-off 3 — Targeted removal for `globalSymbolIndex` (the second per-keystroke full scan)
2+
3+
**Effort:** S–M **Risk:** low–medium **Start here** — cheapest, reuses a landed pattern.
4+
5+
## Problem
6+
7+
The review's reverse-index work made `removeIndexEntriesForUri` (`graph.ts`)
8+
touch only the edited file's symbols via a per-URI record, `symbolsByUri`
9+
O(edited file) instead of O(all workspace symbols) per keystroke.
10+
11+
But its sibling, `updateGlobalSymbolIndex` (`graph.ts:626`), still does the old
12+
**full scan** of `globalSymbolIndex` on every re-parse to drop the URI's old
13+
entries:
14+
15+
```ts
16+
for (const [name, entry] of this.globalSymbolIndex) {
17+
if (entry.uri === uri) this.globalSymbolIndex.delete(name);
18+
}
19+
```
20+
21+
That's the same O(all-symbols)-per-keystroke cost, left unoptimized only because
22+
`globalSymbolIndex` also holds `VarDecl` (global variable) names, which
23+
`symbolsByUri` does not currently record (`updateAllIndexes` handles ClassDecl /
24+
EnumDecl / FunctionDecl / Typedef, not VarDecl — vars live only in
25+
`globalSymbolIndex`).
26+
27+
> Note: this is the genuinely-valuable half of the deferred "merge
28+
> `updateGlobalSymbolIndex` + `updateAllIndexes`" item. A full method merge is
29+
> **not** needed and adds little; the win is the targeted removal below.
30+
31+
## Goal
32+
33+
Make `updateGlobalSymbolIndex`'s removal targeted via the reverse index,
34+
eliminating the second per-keystroke full scan.
35+
36+
## Plan
37+
38+
1. **Record vars in the reverse index.** Extend the `symbolsByUri` value type with
39+
`vars: string[]`. Add a `VarDecl` branch to `updateAllIndexes` that pushes the
40+
var name into `contributed.vars` (vars don't go into the five name-keyed maps —
41+
this record is purely so `globalSymbolIndex` removal can be targeted).
42+
2. **Targeted removal in `updateGlobalSymbolIndex`.** Replace the full-scan loop
43+
with: read `symbolsByUri.get(uri)` and, for each recorded name across
44+
`classes ∪ enums ∪ functions ∪ typedefs ∪ vars`, delete it from
45+
`globalSymbolIndex` **only if** `globalSymbolIndex.get(name)?.uri === uri`
46+
(single-entry, last-writer-wins semantics — same as the enum/typedef handling
47+
in the landed `removeIndexEntriesForUri`). Fall back to the existing full scan
48+
when no record exists (first index of the URI / defensive).
49+
50+
### Ordering (verify, it's the crux)
51+
52+
`ensure()` calls `updateGlobalSymbolIndex(uri, ast)` **then**
53+
`updateAllIndexes(uri, ast)`. Inside `updateAllIndexes`, `removeIndexEntriesForUri`
54+
deletes `symbolsByUri[uri]` and then `updateAllIndexes` re-sets it with the new
55+
record. So when `updateGlobalSymbolIndex` runs (first), `symbolsByUri[uri]` still
56+
holds the **previous** run's complete record (including vars) — exactly what the
57+
removal needs. Confirm this ordering hasn't changed before relying on it.
58+
59+
## Risks
60+
61+
`globalSymbolIndex` is load-bearing: completions, `getWorkspaceSymbols`, and —
62+
since the review — `checkUnknownSymbols` / `typeExists` all rely on it. A stale
63+
entry surfaces as a **phantom symbol**. The landed reverse-index test only checks
64+
`classIndex`, so it would not catch a `globalSymbolIndex` leak.
65+
66+
## Test strategy (required)
67+
68+
Extend the `index maintenance (reverse-index cleanup)` describe in
69+
`test/features.test.ts`: after re-indexing a file that **drops a global function
70+
and a global var**, assert they no longer appear via a public path —
71+
`analyzer.getWorkspaceSymbols('<name>')` returns empty (and/or a completion query
72+
no longer offers them). Keep the existing class assertions.
73+
74+
## Done when
75+
76+
- `updateGlobalSymbolIndex` no longer full-scans on the per-keystroke path.
77+
- `symbolsByUri` records vars; ordering invariant confirmed.
78+
- New staleness test (global function + var removal) passes; all tests green.

handoffs/README.md

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# LSP refactor hand-offs
2+
3+
Deferred follow-ups from the LSP review. Each file is a self-contained spec:
4+
context, goal, exact locations, an incremental plan, risks, a test strategy, and
5+
"done" criteria. They are independent — pick up any one on its own.
6+
7+
Baseline when these were written: `server/` typechecks clean, `npm test` = 303
8+
tests passing. The diagnostics checkers assert **exact** diagnostic ranges, so
9+
any change touching offsets must be validated by diffing diagnostics output, not
10+
just by the suite passing.
11+
12+
| # | Hand-off | Effort | Risk | Why it's worth doing |
13+
|---|----------|--------|------|----------------------|
14+
| 1 | [CallSite parser AST](01-callsite-parser-ast.md) | L | med | Deletes ~550 lines of per-keystroke regex re-tokenizing and its false positives |
15+
| 2 | [Chain-resolver unification](02-chain-resolver-unification.md) | M (phase A) / L (phase B) | med | Removes two parallel chain resolvers; phase B gated on #1 |
16+
| 3 | [globalSymbolIndex targeted removal](03-globalsymbolindex-targeted-removal.md) | S–M | low–med | Kills the *second* per-keystroke O(all-symbols) index scan (sibling of one already fixed) |
17+
18+
Start with **#3** — it's the cheapest, lowest-risk, and reuses a pattern already
19+
landed in the codebase. **#1** unlocks the second phase of **#2**.
20+
21+
## Explicitly NOT recommended
22+
23+
- **Extracting the ~3,900-line diagnostics cluster into a separate "provider"
24+
class.** Pure reorganization: it nets *more* code (a ~25-member facade to hand
25+
the provider the analyzer's internals), cuts zero CPU, and gains no real
26+
testability (the provider still needs a fully-populated analyzer). If the only
27+
goal is navigability, split the diagnostics methods into a `graph.diagnostics.ts`
28+
partial via TypeScript declaration merging / a mixin — same file-size relief,
29+
near-zero risk, no new interface. Don't build the provider.
30+
31+
## Intentionally skipped micro-items (not worth a hand-off)
32+
33+
- **`checkTypeCompatibility` pair-memo** — dominated by the `getClassHierarchyOrdered`
34+
memo that already landed; its only unique gain is skipping a free early-exit.
35+
- **`getHover` double chain-resolve** — hover fires on mouse dwell, not per
36+
keystroke; the second resolve is negligible.
37+
- **`_sourceUri` / `_containerClassName` / `_containerIsModded` typing** — moving
38+
these synthetic fields onto `SymbolNodeBase` to drop the `as any` casts is pure
39+
churn across ~24 sites with no runtime change. Only worth doing if all three are
40+
done together in one pass while touching that code for another reason.

server/src/analysis/ast/parser.ts

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1020,17 +1020,18 @@ export function parse(
10201020
const vars: VarDeclNode[] = [];
10211021
let sawDefault = false;
10221022
while (!eof()) {
1023-
const typeNode = structuredClone(baseTypeNode);
1024-
10251023
// Support trailing `T name[]`
10261024
if (peek().value === '[') {
10271025

10281026
// Prevent additional [] after identifier if already declared in type
1029-
if (typeNode.arrayDims.length !== 0) {
1027+
if (baseTypeNode.arrayDims.length !== 0) {
10301028
throwErr(peek(), "not another [");
10311029
}
10321030

1033-
parseArrayDims(doc, typeNode);
1031+
// Consume the trailing '[..]' tokens into a throwaway scratch node.
1032+
// The parsed dims are unused (each var's type is baseTypeNode below),
1033+
// so a shallow copy avoids a per-declaration structuredClone.
1034+
parseArrayDims(doc, { ...baseTypeNode, arrayDims: [] });
10341035
}
10351036

10361037
// value initialization (skip for now)

server/src/analysis/ast/printer.ts

Lines changed: 0 additions & 5 deletions
This file was deleted.

server/src/analysis/diagnostics/engine.ts

Lines changed: 13 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
import { Diagnostic, DiagnosticSeverity } from 'vscode-languageserver';
2-
import { DiagnosticRule, RuleContext } from './rules';
3-
import { File, ClassDeclNode, FunctionDeclNode, VarDeclNode } from '../ast/parser';
2+
import { DiagnosticRule } from './rules';
3+
import { File, ClassDeclNode, FunctionDeclNode } from '../ast/parser';
44

55
/**
66
* Pluggable Diagnostic Rule Engine
@@ -28,11 +28,11 @@ export class DiagnosticEngine {
2828
}
2929

3030
/** Run all registered rules against an AST file */
31-
run(ast: File, context: RuleContext): Diagnostic[] {
31+
run(ast: File): Diagnostic[] {
3232
const diagnostics: Diagnostic[] = [];
3333
for (const rule of this.rules) {
3434
try {
35-
const ruleDiags = rule.check(ast, context);
35+
const ruleDiags = rule.check(ast);
3636
diagnostics.push(...ruleDiags);
3737
} catch (err) {
3838
// Don't let a single rule failure break all diagnostics
@@ -55,7 +55,7 @@ class ConflictingModifiersRule implements DiagnosticRule {
5555
name = 'Conflicting Modifiers';
5656
severity = DiagnosticSeverity.Error;
5757

58-
check(ast: File, _context: RuleContext): Diagnostic[] {
58+
check(ast: File): Diagnostic[] {
5959
const diags: Diagnostic[] = [];
6060

6161
const conflicts: [string, string, string][] = [
@@ -114,19 +114,18 @@ class StrongRefParameterRule implements DiagnosticRule {
114114

115115
private static readonly STRONG_REF_MODIFIERS = ['autoptr', 'ref'];
116116

117-
check(ast: File, _context: RuleContext): Diagnostic[] {
117+
check(ast: File): Diagnostic[] {
118118
const diags: Diagnostic[] = [];
119119

120120
const checkParams = (func: FunctionDeclNode) => {
121121
for (const param of func.parameters) {
122-
for (const mod of StrongRefParameterRule.STRONG_REF_MODIFIERS) {
123-
if (param.modifiers?.includes(mod)) {
124-
diags.push({
125-
message: `Method argument '${param.name}' can't be a strong reference. Remove '${mod}' from the parameter.`,
126-
range: { start: param.nameStart, end: param.nameEnd },
127-
severity: this.severity
128-
});
129-
}
122+
const mod = param.modifiers?.find(m => StrongRefParameterRule.STRONG_REF_MODIFIERS.includes(m));
123+
if (mod) {
124+
diags.push({
125+
message: `Method argument '${param.name}' can't be a strong reference. Remove '${mod}' from the parameter.`,
126+
range: { start: param.nameStart, end: param.nameEnd },
127+
severity: this.severity
128+
});
130129
}
131130
}
132131
};
Lines changed: 2 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
import { Diagnostic, DiagnosticSeverity } from 'vscode-languageserver';
2-
import { ClassDeclNode, File } from '../ast/parser';
2+
import { File } from '../ast/parser';
33

44
/**
55
* Base interface for all diagnostic rules.
@@ -13,18 +13,5 @@ export interface DiagnosticRule {
1313
/** Severity of diagnostics produced by this rule */
1414
severity: DiagnosticSeverity;
1515
/** Run the rule against an AST and return diagnostics */
16-
check(ast: File, context: RuleContext): Diagnostic[];
17-
}
18-
19-
/**
20-
* Context object passed to diagnostic rules, providing
21-
* access to the analyzer's indexes and resolution methods.
22-
*/
23-
export interface RuleContext {
24-
/** Look up a class by name */
25-
findClassByName(name: string): ClassDeclNode | null;
26-
/** Get all classes in inheritance hierarchy */
27-
getClassHierarchy(className: string): ClassDeclNode[];
28-
/** Get the number of indexed files (for threshold checks) */
29-
indexedFileCount: number;
16+
check(ast: File): Diagnostic[];
3017
}

0 commit comments

Comments
 (0)