@aaronandwork/dsh-evidence-gate

dsh plugin --profile web add @aaronandwork/dsh-evidence-gate

dsh-evidence-gate

English | [中文](#中文)

Make agents in DeepSeek Harness (DSH) stop drawing conclusions from unverified guesses — and make every conclusion auditable.

An agent that says "I read this file" is cross-checked against the session's actual tool activity. A claim with no matching tool log entry is downgraded from VERIFIED to UNVERIFIED on the spot.

Demo / 演示

<p align="center">

<img src="assets/evidence-gate-demo.gif" alt="Evidence gate demo: the shield chip is ON, the agent's load-bearing claim (windchill full-suite load test passed, basis: tested) goes through the evidence_gate tool before it may be stated / 实证门演示：盾形开关开启，agent 的关键结论先过 evidence_gate 再落笔" width="640"><br>

_{Chip ON → the claim goes through <code>evidence_gate</code> before it may be stated / 开关开启 → 结论先过门再落笔}

</p>

<p align="center">

<img src="assets/evidence-gate-toggle.gif" alt="The per-session shield toggle in the composer showing OFF with the tooltip explaining the gate does not apply to this conversation / 输入框旁的会话级开关，关闭态悬停提示本会话不生效" width="640"><br>

_{Per-session toggle: OFF means the gate does not apply to this conversation / 会话级开关：关闭仅影响本会话}

</p>

<p align="center">

<img src="assets/evidence-gate-bilingual.gif" alt="The expanded verdict card following the GUI language, shown in Chinese with basis, tier, evidence and cross-check rows / 展开的判定卡片跟随 GUI 语言，展示依据、证据、核验各行" width="640"><br>

_{The verdict card follows the GUI language and switches live / 判定卡片跟随 GUI 语言实时切换}

</p>

<p align="center">

<img src="assets/evidence-gate-report.gif" alt="The evidence-gate report output: per-session ledger with verified, cited, blocked, converted, cross-checked and floor-passed counts / 实证门报告：本会话台账统计" width="640"><br>

_{<code>/evidence-gate report</code>: the per-session ledger, preserved across restarts / 本会话台账，跨重启保留}

</p>

---

Why

Prompt-only "don't guess" instructions are unenforceable and unauditable. This plugin makes conclusion discipline structural:

1. Policy — a system-prompt section establishing the evidence hierarchy injected before every model step.

2. Gate — a model tool the agent calls before stating load-bearing conclusions; each claim gets a verdict.

3. Cross-check — claimed direct observations are verified against the session's recorded tool activity (tools/result stream).

4. Ledger — per-session counts: how many guesses were blocked, how many were converted into real verification.

How it works

Component · Mechanism

Policy section · systemPrompt.section (order 550), text provider

evidence_gate tool · registered via the tools registry; verdicts VERIFIED / REFERENCED / UNVERIFIED / INVALID / OFF

Evidence hierarchy · tested / measured / read_directly / prior_verified (direct) · documented / community_practice (must cite source) · assumption (only as a labeled hypothesis + next verification step)

Cross-check · tools/result listener records a durable per-session evidence index (400 entries) — tool arguments AND output anchors (paths / URLs / distinctive output words); gate claims of direct observation must match it

Anti-false-kill ladder · own-session hit → recorded-output keyword hit → other-session hit (subagent) → existence floor (cited path really exists → pass with ⚠ weak note) → block

Metrics · blocked (guesses downgraded) vs converted (blocked claims that later came back VERIFIED) vs charity (existence-floor passes, visible as 采信)

Card badges · tool card titles: 🟢 VERIFIED · 🔵 REFERENCED · 🟠 UNVERIFIED · ⚪ OFF

Switch · /evidence-gate on\ · off\ · status\ · report + evidence_gate_switch model tool — per-session, persisted

v0.2 semantics

• Per-session everything: the switch, the ledger, and the prompt section are scoped to sessionId. Turning the gate on in one conversation never leaks into another; the chip tooltip shows 本会话 stats only. The policy section's text provider resolves per assembly (context.agent.sessionId).
• Persistence: ledger + switch + evidence index survive deployment restarts and context compaction (JSON store under DSH_HOME/evidence-gate/store.json, debounced atomic writes; override the path with EVIDENCE_GATE_STORE). Top 50 sessions by recency are kept.
• Quiet startup: mounting the bundle prints nothing to the host console — set EVIDENCE_GATE_DEBUG=1 to print the load breadcrumbs (store restore + each registered component) when diagnosing a mount problem. Error/warning diagnostics are never gated.
• Output-aware cross-check: evidence like "pnpm test → SMOKE PASSED" now correlates against recorded tool OUTPUT (keywords, paths, URLs extracted from the tools/result payload), not just command strings.
• Honest-claim protection: misses fall through to other sessions (subagent work), then to an existence floor (a cited path that really exists on disk passes with a weak/采信 note instead of blocking). Fabricated paths, empty evidence, and assumption still block.

Verified demo (from actual development)

agent claims: "I read C:\Users\...\fake\never-read-this-module.txt"
→ 🟠 UNVERIFIED — cross-check failed (no tool activity matches c:\users\...\fake\never-read-this-module.txt)

agent claims: "I read C:\...\3d12c8d4e809-cordis_inspect_query.txt"
→ 🟢 VERIFIED (read_directly, cross-checked ✓ matched tool activity)

Install

The package is a dsh bundle: it ships its own cordis.patch.yml (which inserts the plugin row) and a web client half — no manual composition editing needed.

Into a dsh profile — four ways, ①–③ need no npm account (all verified):

cd "$DSH_HOME/profiles/<name>"

# ① straight from GitHub (recommended)
pnpm add github:AaronandWork/dsh-evidence-gate

# ② prebuilt tarball from the GitHub Release (no git needed)
pnpm add https://github.com/AaronandWork/dsh-evidence-gate/releases/latest/download/dsh-evidence-gate.tgz

# ③ from a local clone / checkout
pnpm add

…(truncated)