note · evergreen
Language coverage is a rule, not a table
371 grammars, an extension-name rule, no role taxonomy, no per-framework code — what production systems do.
The tempting design was a table: extension → language, folder → role
(controllers/ = controller), a regex per language for symbols. We surveyed
what production systems do instead, and measured what a table would buy.
What everyone else does
GitHub code search, Sourcegraph, Cursor, aider, Continue, Sweep, Qodo,
Greptile — all of them outsource language coverage to a parser
ecosystem (tree-sitter grammars with upstream tags.scm, or
universal-ctags), chunk with a grammar-generic size cap, keep a universal
line-window fallback, and carry path + symbol as metadata. None of them has
framework-specific logic. None of them keeps a folder→role taxonomy.
| Proposed | Verdict | Why |
|---|---|---|
| extension → language table | replaced by a rule | 371 grammars ship in the pack; if the extension name is a grammar name (.lua .vue .razor .zig) it just works; a small alias table (.ts .cs .kt) is validated against the pack at import |
| regex symbol extractors | no | tree-sitter already yields symbol, kind, parent for 20+ languages |
| folder → role tags | no | a role is a deterministic function of the path, and the path is already in the embedded text; it fixed none of the 11 misses in the golden set |
| vendor-name skip lists (jquery, bootstrap…) | no | a per-repo ignore file generalises; a name list encodes one company |
| per-framework rules (Next.js, Vue, Razor) | no | route files already carry the route in their path; Razor blocks fall out of the grammar |
What we kept hand-written, and why
- Deliberately plain: json, yaml, css, html — grammars exist, but windows measured equal and a parser is only crash surface.
- Never indexed: csv, tsv, diff, po, pem — grammars exist for those too.
- Grammar exists, parser refused:
sql(the generated grammar segfaults on migration files) andcobol(hangs past 300 s on unbalanced brackets). Both were found by a smoke test that runs every grammar in its own subprocess on degenerate input — a crash kills the child, not the indexer.
Adding a language now costs zero lines. The ceiling of that whole axis, per the ablation, is about one question of recall — so zero is the right price.