Overview
microsoft/tgrep treats search as a chain of conditional evidence: repository rules define the corpus, a persistent trigram index narrows candidate files, and the requested matcher verifies the result. Its planner uses intersections, unions, or MatchAll depending on what a pattern safely requires; bounded external construction, memory-mapped postings, live overlays, coverage metadata, and freshness checks keep reuse practical after edits. The series shows why this can outperform repeated scans on large, selective workloads, while one-shot, warm, broad-output, incompatible, or not-yet-ready requests may need a direct scan. The index is an optimization, not a substitute for current corpus membership or exact regex semantics.
Learning outcomes
- A persistent trigram index lets tgrep reuse repository evidence across repeated searches and avoid many file reads.
- Posting intersections, unions, and compact masks narrow candidates conservatively; the full matcher remains authoritative.
- External sorting, mapped postings, live overlays, and coverage metadata make reuse practical without treating partial or stale state as complete.
- The strongest gains are conditional: large, repeated, selective workloads benefit, while warm, broad, one-shot, or incompatible requests may favor direct scans.
Keywords
Preview every chapter · Unlock full text, audio, and video from chapter 4
PlansFAQ
What does microsoft/tgrep index?
microsoft/tgrep indexes eligible text-file content as overlapping three-byte fragments and maps them to file IDs; filename-only paths and coverage metadata are tracked separately.
Why does microsoft/tgrep verify candidates with the full matcher?
Trigrams and compact masks provide necessary evidence, not proof of order or a complete regex match; the full matcher preserves the requested semantics.
When is microsoft/tgrep faster than a direct scan?
microsoft/tgrep is most likely faster for repeated, selective searches over a large repository after indexing is paid for. One-shot, warm, broad-output, or incompatible requests can favor scanning.
How does microsoft/tgrep handle changes and incomplete coverage?
It overlays edits and deletions on the published reader, publishes validated generations, and falls back to broader scanning when freshness or coverage is not proven.