1. Test groups and scope
October 18, 2026: Five-case diagnostic with five recovered original CiteMax reports (cases 1–5); case 5 lacks a recorded numeric grade. Shorter specialist prompts and responses are preserved by reference in the original project collection, but not available here as an authenticated public verbatim distribution. Specialist names are anonymized as A and B.
October 20, 2026: A contemporaneous summary reports 20 general-purpose responses across Grok, Copilot, Gemini and Perplexity. Original verbatim responses and independent individual grades have not been recovered for publication.
October 31, 2026: Two wash-sale prompts (simplified and comprehensive), with recorded grades for CiteMax, Grok, Copilot, Perplexity, Claude and Gemini. The public paper reproduces the recorded paired scores and times, but these scores have not been independently rescored from original complete transcripts.
Original-source correction log
The first five CiteMax original reports are recovered: 8, 8, 4, 6 and 6 pages. The original Case 5 report distinguishes October 20, 2026 electronic retransmission from a separate paper-filing fallback. The inherited-IRA report lacks facts present in the later expanded protocol; prompt identity is not established. October 20/31 ratings remain historical reviewer observations, not independently regraded response-level evidence.
2. Five-case scoring ledger
Historical blended scores, 0–100: conclusion / exceptions 30%; primary authority 25%; uncertainty 15%; date and jurisdiction 10%; implementation 10%; practitioner usability 10%. Each row is a first-pass editorial grade, not a blinded or independently audited accuracy score.
| Case | CiteMax full-case report | Short Service A | Short Service B |
|---|---|---|---|
| 1 · S-corp insurance | 85 provisional | 58 provisional | 52 provisional |
| 2 · LLC classification | 89 provisional | 55 provisional | 28 · critical legal error |
| 3 · Hawaii depreciation | 69 provisional | 0 · no answer | 57 provisional |
| 4 · Inherited IRA | 79 provisional | 64 provisional | 68 provisional |
| 5 · Rejected e-file | Not retained; not scored | 0 · no answer | 40 · critical deadline error |
Not comparable on equal footing: CiteMax cases were fuller assignments; specialist responses followed shorter prompts. A no-answer score is a result for a particular run, not a categorical claim that the vendor lacks the capability. An internally created research configuration and its scores are excluded because this public study evaluates independently developed comparator systems.
3. Original source file inventory
- October 18 diagnostic protocol and original five research questions — referenced in source project, not embedded in this HTML edition.
- Four CiteMax full research workpapers, cases 1–4 — described in reconciled evidence summary; individual report files are not included in this public ledger.
- “17 question test.pdf” — source of shorter exploratory specialist responses; reproduction permissions and a complete authenticated public extract are not established.
- Original October 20 general-purpose outputs — complete verbatim transcripts not recovered in this publication.
- Original October 31 wash-sale outputs — historical ratings and timings recorded, complete original model transcripts not re-extracted for this version.
- Reconciled comparative evidence memorandum and web publication source — retained by the publisher.
4. Independent primary-source verification
Selected primary sources were checked on November 5, 2026. They substantiate broad regulatory and procedural benchmarks, not all claims or every model score. Particularly: IRS OPR Issue 2026-19 applies Circular 230 duties to AI work; the IRS MeF schedule and IRM establish October 20, 2026 as the relevant retransmission limit; and IRS Publication 1345 separately addresses a paper filing fallback. Hawaii's Form N-11 instructions establish federal bonus-depreciation decoupling, without by itself resolving all passive-loss consequences for a specific taxpayer.
- IRS OPR, AI in Federal Tax Practice, June 24 2026
- 31 CFR §10.22 — due diligence
- 31 CFR §10.35 — competence
- 31 CFR §10.36 — procedures
- 31 CFR §10.37 — written advice
- IRC §301.7701-3 classification regulations
- IRS S corporation medical insurance guidance
- Hawaii 2025 N-11 instructions
- IRS Notice 2024-35
- IRS 2025 Publication 590-B
- IRS 2026 Form 1040 MeF schedule
- IRS IRM 3.42.5.14.6 perfection periods
- IRS Publication 1345, paper fallback
5. Reproducibility and remaining limits
A fully reproducible benchmark would additionally require: each complete dated prompt; unedited model response; model and product version; elapsed time measurement method; source-access environment; redaction and publication permissions; independent legal reviewer decisions; and case-by-case grading on two separate 100-point dimensions (substantive accuracy and professional workpaper quality). These elements cannot be reliably reconstructed where the original evidence is missing. No combined “best AI overall” ranking is presented.
The current publication is a sponsored, observational case study with clearly flagged historical assessments. The omission of records or lack of a verified original is not evidence that any system performed poorly.