dsh-kb-rag

dsh plugin --profile web add dsh-kb-rag

dsh-kb-rag







Static DSH plugin for a local literature knowledge base. It indexes PDFs, folders or a Zotero library into one SQLite file and gives the model ten tools for ingest, hybrid search, cited answers and HTML reports. Embedding and reranking run locally.

Latest version 1.6.8. Install with dsh plugin --profile web add dsh-kb-rag@1.6.8. A DSH profile is a pnpm workspace, so do not run npm install inside it; for a hand-managed deployment see [Option 3](#option-3--manual-npm-install).

This release fixes the DSH 0.2 install path (peerDependencies and the client injection list), the tool cards, and adds engine-rendered HTML answers with retrieval/render time in the footer.

Tools (10)

Tool · Purpose

kb_ingest · Ingest files or folders (PDF/TXT/MD/DOCX, recursive). Incremental skip, dedup, section-aware chunking, vectorisation; metadata_only=true refreshes title/authors/year/journal/DOI in place, rebuild=true re-parses every indexed document, and a large batch moves to a background job

kb_status · Poll a background job by job_id: progress while running, totals and recent files when done

kb_zotero · Migrate a local Zotero library, including PDF attachments

kb_search · Hybrid search returning passages with title, authors, year, journal, DOI, section and PDF page

kb_rag · Retrieve evidence for the model to answer from, with numbered citations; optional self-contained HTML answer

kb_scope · Query scope (kb / both / web), strict mode and retrieval depth

kb_stats · Document, chunk and vector counts, recent ingests, model cache state, stale documents

kb_dedup · Remove duplicate documents, keeping the earliest

kb_clear · Wipe all documents and indexes (requires confirm: true)

kb_fetch · Download a PDF by DOI or arXiv ID (publisher version first, open-access fallback)

Citations render as authors, year, journal when a DOI exists, and as [authors, year, filename] when it does not. Search and RAG results also carry a related-literature list used by the answer's "suggested additions" note.

HTML answers

Pass an answer object to kb_rag and the engine writes a self-contained HTML page: ten templates (instant, review, method, compare, numeric, timeline, concept, dispute, derivation, qa), comparison tables with heat or bars, trend charts, KPI cards and process chains. The footer reports retrieval and render time. Files go to ~/.kb-rag/answers/ next to a gallery; answer_dir overrides the location.

Engine capabilities

• Section-aware chunking: abstract ×1.5, methods ×1.2, inline heading detection, abstract promotion, caption blocks, paragraph merging for non-article documents.
• Hybrid retrieval: BM25 with CJK bigrams, bge-small vector cosine, RRF fusion, section weights.
• Reranking: bge-reranker-base cross-encoder (top 20 into top 3), with a bge-large-en fallback.
• Provenance: section, paragraph range, physical PDF page (schema v3), authors, year, journal, DOI.
• Citation linking: [n] markers resolve to references; Nature-style superscripts are detected from font metrics; references are stored at weight 0 and excluded from retrieval.
• Fast and deep modes: quick returns hybrid hits directly; deep adds reranking, citation linking and related work.
• Incremental indexing and dedup by SHA-256 content hash.
• Metadata refresh and stale detection (schema v4): docs.indexed_with records the parser revision; metadata_only re-reads page 1 in about 90 ms per document, rebuild re-parses in place.
• Query cache: identical query and filters are not recomputed; ingest or an engine rebuild invalidates it.
• Resident daemon: models load once, the daemon recovers from crashes, and it is reclaimed when the plugin stops.
• Background jobs: above KB_ASYNC_THRESHOLD (default 25) pending files the batch is forked under .kb-jobs/, and job_id is polled with kb_status.

Query guidance

Queries go to the engine verbatim. It never translates or rewrites them, so retrieval depends on the query matching the indexed text. A typical library is about 98% English body text:

• Write queries as English term strings. BM25 matches tokens; a CJK query leaves the keyword half idle and the hit rests on cross-language vector similarity alone.
• Use 3–12 words in the pattern material/system + method/process + property/characterisation, e.g. graphene CVD copper single crystal nucleation suppression.
• Put limits in filters (year, journal, author, section, kind), not in the query. journal is filled only by the Zotero path, so it stays NULL for kb_ingest documents; use author, year, title or section instead.
• Send a second query in the original language only when you want documents in that language.

A CJK query against an English library gets a lang_note in the response, and the plugin renders it with the results.

Install

Pick one path. After installing, restart DSH and open a new session; tools are injected when a session is created.

Option 0 — DSH desktop app (recommended, no terminal)

Open Settings → Plugins (设置 → 插件), search kb-rag, and install 1.6.8. Restart DSH and start a new conversation. The desktop app must be 0.2 or newer; 1.6.7 and earlier are rejected by 0.2's compatibility check, which is wh

…(truncated)