DocNeedle

Current state. Public README and v0.2.0 tag document a focused four-command CLI (index, search, read, check) while keeping benchmark harnesses explicitly separate from normal user workflows; "v2" refers to the retrieval-generalization baseline, not product version 2.0.

Context

DocNeedle is a local-first, deterministic tool for finding — and eventually patching — small facts across large markdown documentation corpora, without loading the haystack into an LLM. It is the most portfolio-ready repo in the evidence set because the README is explicit about what works today and what is still author-environment infrastructure.

Problem

Two problems, one tool:

1. Working with fast-changing docs is expensive. Docs live across PRDs, DRDs, TRDs, implementation plans, and decision logs. Finding one small fact usually means whole-file reads (token-heavy) or rg + manual follow-up (structure-less noise). The thesis: deterministic indexes for the haystack, LLM tokens only for the final candidate sections.
2. Tool stories blur product and machinery. Stable commands, internal experiments, and benchmark harnesses get folded into one narrative, forcing readers to guess what is actually safe to use.

DocNeedle is deliberately not “RAG for docs” — no cloud vector DB, no LLM memory system, no full-doc loading by default.

Approach

A deliberately small, documented CLI surface (python -m docneedle.cli ...):

How it is implemented:

• Deterministic chunking — markdown is split into chunks that carry hashes and line ranges, so re-indexing and patch targeting stay stable across runs; parser-version invalidation drops stale chunks when the parser changes.
• SQLite + FTS5 storage — files and chunks tables plus an FTS5 virtual table (porter unicode61 tokenizer) with per-column BM25 weights.
• Explainable ranking — FTS5 BM25 candidates are re-ranked with explicit score components (lexical score, query-intent classification, metadata hints, project filter), then diversified before returning compact payloads (rank, path, line range, source hash).
• Metadata + hierarchy indexing — document role/status/canonicality is indexed and used during retrieval; a large part of the retrieval gain came from making structure legible, not just from ranking tweaks.
• Zero LLM/API in the retrieval path — indexing and search are fully local and deterministic (Python ≥3.11, no runtime dependencies).
• Tested — 43 tests across indexer, parser, search/ranking, fixture eval, and benchmark verdicting.

Evidence

Public evidence is the README, the repository tree, and the v0.2.0 tag on the GitHub repo. The eval design separates three families — a broad dev set (tuning across failure families), an untouched holdout (overfitting detection), and a real-doc benchmark (live corpus) — but numbers stay qualitative until linked public reports exist. Note: v2 here names the retrieval-generalization phase/baseline, not product version 2.0.

Decisions

• Product surface and benchmark harness stay separate. Benchmark scripts are repo-specific evaluation tools (hardcoded paths, rg baseline) and are explicitly not the normal way to use the tool.
• No invented ADR record; the separation is presented directly.

Outcome / current state

Working and iterating. The four-command CLI is documented and tagged (v0.2.0); deterministic indexing, changed-only refresh, explainable re-ranking, and cross-project stale-term checks are implemented and tested. Known limits are stated openly:

• Patch mechanics are scaffolded but not exposed as a CLI command in this MVP slice.
• Benchmark scripts are still author-environment-oriented and need path cleanup before they become portable.
• Contrastive-negation queries (“do not use X, prefer Y”) are an open current-corpus case.

Next

When public benchmark reports exist, the case study can add attributable results without changing the core story. On the product side: expose patch mechanics behind a documented command, and close the contrastive-negation gap.