OpenAI
GPT-5.5
openai/gpt-5.5
draft_run_completed_three_grader_consensus
Benchmark v0.1 · verified leaderboard
Daoism LLM Leaderboard
這頁公開 verified official runs、score schema、示範 run 與可複現 runner。正式主榜只採 v0.1 core 330 題。 首批 verified runs 於 2026-07-04 全量重跑、2026-07-07 通過 L6 人審 gate 簽核（審核方式見下方 L6 區塊，含授權 AI 代審之如實揭露）。
Count policy: 330 is the main leaderboard denominator, 400 is the public benchmark table, and 500/515 belong to the LIUS-ID registry/text-mention scope. 2026-06-10 draft rows are retained for history but superseded: their raw artifacts were lost before signoff and the full benchmark was rerun on 2026-07-04.
4
正式 verified runs
4
實測 draft runs
2
示範 runs
330
主榜題數
400
公開題數
320
L6 signed_off
Verified rows carry VERIFIED_L6_HUMAN_SIGNOFF_DELEGATED_AI_REVIEW: the L6 gate was closed by batch signoff of the reviewer of record after delegated AI item review — not per-item human reading; item-level decisions and rationales are retained in the audit trail. Historical draft rows remain UNVERIFIED_THREE_GRADER_CONSENSUS_DRAFT and are superseded by the 2026-07-04 rerun.
正式 verified runs
| Rank | Model | Score | Acc | S-Hit | Halluc L6 | Refuse L6 | N |
|---|---|---|---|---|---|---|---|
| 1 | GPT-5.5 (verified) gpt-5.5 VERIFIED_L6_HUMAN_SIGNOFF_DELEGATED_AI_REVIEW | 61.7 | 86.8% | 1.2% | 0% | 66.3% | 330 |
| 2 | GPT-5.4 (verified) gpt-5.4 VERIFIED_L6_HUMAN_SIGNOFF_DELEGATED_AI_REVIEW | 61.3 | 83% | 1.2% | 0% | 77.5% | 330 |
| 3 | GPT-5.4-mini (verified) gpt-5.4-mini VERIFIED_L6_HUMAN_SIGNOFF_DELEGATED_AI_REVIEW | 59.6 | 76.7% | 0.6% | 0% | 87.5% | 330 |
| 4 | GPT-5.3 Codex Spark (verified) gpt-5.3-codex-spark VERIFIED_L6_HUMAN_SIGNOFF_DELEGATED_AI_REVIEW | 49.4 | 57.7% | 0.9% | 7.5% | 75% | 330 |
2026-07-04 core-v0.1 全量重跑（330 題）；跨供應商三評分員取中位（gpt-5.4-mini / deepseek-v4-flash / gemini-2.5-flash）； L6 人審 320/320 完成並於 2026-07-07 簽核，14 筆翻案已逐筆留存理由（其中 1 筆為 GPT-5.5 外部覆審發現之補正）。分數已反映人審後最終評分。
Score formula
0.4*accuracy_overall + 0.3*source_hit_rate + 0.2*(1-hallucination_rate_l6) + 0.1*refusal_rate_l6_trapL6 human audit gate
Gate 已於 2026-07-07 簽核關閉（320/320）
審核方式如實揭露：242 筆三評分員全一致題依審核協議批次通過；78 筆旗標題（紅線 / 評分員分歧 / 零分半分）由授權 AI 審核員（Jarvis / Claude Fable 5）逐筆覆核——依評分規準、三方評分票與論文優先文獻檢索——共 14 筆翻案（主因跨模型評分不一致；其中 1 筆為 GPT-5.5 外部覆審發現之補正），每筆留存理由；最終由審核責任人 Bruce Liu 批次簽核。此為「授權 AI 代審 + 人類問責簽核」，非逐筆人讀。Raw packet 含模型答案與 grader notes，維持 review-required；公開頁只列摘要統計。
signed_off
320
L6 audit items
5
critical redline
65
grader disagreement
8
zero / partial
242
spot check
Summary artifact: tmp/lius-benchmark-runs/L6-human-audit-summary-2026-07-04.json. Gate: Gate closed 2026-07-07: 320/320 review decisions verified (review_complete); draft scores promoted to verified official runs. Review method: 242 unanimous items batch-approved per reviewer protocol; 78 flagged items individually reviewed by delegated AI reviewer (Jarvis / Claude Fable 5) with papers-primary literature checks; 14 overturns documented (one added after external GPT-5.5 audit); batch signoff by Bruce Liu (reviewer of record). Not per-item human reading.
歷史 draft runs（已被 2026-07-04 verified rerun 取代）
| Rank | Model | Score | Acc | S-Hit | Halluc L6 | Refuse L6 | N |
|---|---|---|---|---|---|---|---|
| 1 | GPT-5.4 (OpenAI OAuth draft) gpt-5.4 UNVERIFIED_THREE_GRADER_CONSENSUS_DRAFT | 62.5 | 83% | 3.3% | 0% | 82.5% | 330 |
| 2 | GPT-5.5 (OpenAI OAuth draft) gpt-5.5 UNVERIFIED_THREE_GRADER_CONSENSUS_DRAFT | 62.4 | 84.6% | 4.9% | 0% | 71.3% | 330 |
| 3 | GPT-5.4-mini (OpenAI OAuth draft) gpt-5.4-mini UNVERIFIED_THREE_GRADER_CONSENSUS_DRAFT | 59.1 | 73.8% | 1.8% | 0% | 90% | 330 |
| 4 | GPT-5.3-codex-spark (OpenAI OAuth draft) gpt-5.3-codex-spark UNVERIFIED_THREE_GRADER_CONSENSUS_DRAFT | 53.0 | 61.2% | 5.2% | 5% | 80% | 330 |
2026-06-10 draft runs 的原始 artifacts 在簽核前遺失，無法覆核，僅留作歷史紀錄；正式引用請以上方 verified runs 為準。
示範 runs
| Rank | Model | Score | Acc | S-Hit | Halluc L6 | Refuse L6 | N |
|---|---|---|---|---|---|---|---|
| 1 | anthropic/claude-opus-4-7 anthropic/claude-opus-4-7 ILLUSTRATIVE_DEMO_ONLY_NOT_FOR_LEADERBOARD | 83.0 | 100% | 60% | 0% | 50% | 5 |
| 2 | Claude Opus 4.7 (Anthropic, knowledge cutoff 2026-01) anthropic/claude-opus-4-7 SPEC_EXAMPLE_ONLY | 71.7 | 70.3% | 51% | 10% | 78% | 330 |
示範 runs 只用來展示 schema 與顯示格式；正式 leaderboard 需 full 330 題、3-grader consensus、L6 100% 人工抽驗。
待跑模型
OpenAI
GPT-5.4-mini
openai/gpt-5.4-mini
draft_run_completed_three_grader_consensus
OpenAI
GPT-5.4
openai/gpt-5.4
draft_run_completed_three_grader_consensus
OpenAI OAuth
GPT-5.3 Codex Spark
openai/gpt-5.3-codex-spark
draft_run_completed_three_grader_consensus
Anthropic
Claude Opus 4.7
anthropic/claude-opus-4-7
pending_run
Google
Gemini 2.5 Pro
google/gemini-2.5-pro
pending_run
lius.cc
Daoism-Qwen3.5-9B
lius-cc/Daoism-Qwen3.5-9B
pending_local_or_hf_run
Runner / Grader
node scripts/benchmark/run-lius-benchmark.mjs --dry-run --limit 5
LLM_BENCHMARK_BASE_URL=http://127.0.0.1:8000/v1 \
LLM_BENCHMARK_API_KEY=dummy \
LLM_BENCHMARK_MODEL=Daoism-Qwen3.5-9B \
node scripts/benchmark/run-lius-benchmark.mjs --suite core-v0.1 --limit 330
node scripts/benchmark/grade-lius-benchmark.mjs \
--input tmp/lius-benchmark-runs/<run>.jsonl \
--dry-run
LLM_GRADER_BASE_URL=http://127.0.0.1:8001/v1 \
LLM_GRADER_API_KEY=dummy \
LLM_GRADER_MODEL=grader-model \
node scripts/benchmark/grade-lius-benchmark.mjs \
--input tmp/lius-benchmark-runs/<run>.jsonl