Dictate Transcription Plan

Date: 2026-07-05

Status: Active scoped authority for local transcription and release evidence.

This is the authoritative planning document only for Dictate local transcription, recordings, model lanes, timestamping, and their release evidence. Meetings and speaker labelling are shelved (P3, #140) and not part of this plan. Dictate runs on the CPU only; GPU material below is history unless it says otherwise. VISION.md is the current cross-system authority; this plan is subordinate for account, cloud, commerce, and Deck decisions. Older research/cost notes are archived under docs/archive/.

Goal #

Ship a clean local-first transcription stack where:

1. Push-to-talk dictation and plain recordings use the same verbatim ASR path.
2. Meeting capture is not in the app. It was removed on 2026-10-03 (#140) and is kept on the archive/meeting-2026-10-03 branch and tag.
3. Parakeet on the CPU is the local ASR foundation. Dictate does not support GPUs: NVIDIA CUDA and AMD lanes were dropped on 2026-10-01 to focus on CPU latency on the machines people already have. Removing the remaining GPU code, the engine's device selection, is tracked in #111; GPU may return later as a separate decision.
4. Whisper is gone: faster-whisper, WhisperX and whisper.cpp were removed on 2026-09-30. Parakeet v2 (English) and v3 (25 European languages) cover dictation.
5. Latency from releasing the shortcut to text appearing, measured on everyday CPU machines, is the performance target.

Deployable Goal #

Set this document as the active scoped deployment goal for local transcription and release evidence:

Implement docs/TRANSCRIPTION_PLAN.md end to end so Dictate is ready for human testing on everyday CPU machines, with a Windows build verified in the local VM lab.

This document is the source of truth only for the scoped transcription/model deployment. Older goal, model-research, and cost-planning notes are archived or reduced to pointers. If another document conflicts with this plan on local transcription or release evidence, update that document to point here. For product direction, follow VISION.md.

Planning Map #

Use this file as the only active plan for transcription deployment. Other docs have narrower roles:

Do not create a second planning landing page for this work. If implementation details outgrow this document, keep this plan as the goal and link to focused runbooks or specs from the relevant acceptance gate.

Deployment Snapshot #

Current verified state:

1. Linux source/local install is usable for human testing on this workstation.
2. Fresh CPU English installs default to Parakeet v2 where the runtime is available.
3. GPU lanes are no longer supported (2026-10-01). Last evidence, for the record: Parakeet v2 int8 loaded on CUDA 13 (onnxruntime-gpu 1.30, RTX 4090 in win11-gpu WSL2) on 2026-10-01, but only after working around a broken GPU install path (#109, closed), and with 742 CPU/GPU memcpy nodes, so int8 on CUDA was not a clear win.
4. The Windows win11-dev VM passes source install, update, uninstall, and desktop build smokes, including Store MSIX packaging.
5. The Windows human-test artifacts currently proven in the local VM are the no-bundle executable plus dictate-engine.exe sidecar and a Store-style .msix package. MSI/NSIS direct-download installers are still not promoted.
6. Meeting capture, its speaker-attribution backends (pyannote, DiariZen, Sortformer), and the torch/pyannote runtime and model in the builds were removed on 2026-10-03 (#140). Meeting transcripts saved before then stay in history as notes.

Deployment blockers before calling this plan complete:

1. Measure end-to-end dictation latency on CPU on the two daily machines (X Forge and the Framework laptop) and tune where the numbers point.
2. Validate note timestamp quality. The engine, daemon, note store, expanded-note UI, search and Markdown export carry timestamped segments; their accuracy on real recordings is not yet measured.
3. Decide the public Windows promotion path: current VM evidence proves the no-bundle executable and MSIX packaging paths; MSI/NSIS direct-download bundling is still not promoted.
4. Staged update preparation is deferred out of the human-test release. The human-test release uses the tested immediate source update path plus the documented MSIX/manual package path.

Product Rules #

The UI should expose product language such as Record. It should not expose primary controls named diarization, ASR backend, CUDA, MIGraphX, or similar engine terms.

Local ASR Lanes #

NVIDIA CUDA and AMD (ROCm/MIGraphX, DirectML) lanes were dropped on 2026-10-01. The engine's device selection is still present until #111 lands; treat it as unsupported, and do not add GPU evidence gates.

The faster-whisper/large-v3 bridge for this workstation was removed on 2026-09-30.

Implementation Status #

GPU history: the CUDA and AMD lanes were dropped on 2026-10-01. Their benchmark, evidence and audit tooling was removed with #112 and the GPU install paths with #110; the engine's device selection goes with #111. Their last evidence is in this file's git history before 2026-10-01.

Parakeet v2 and v3 are wired through the ONNX/onnx-asr loader on the CPU.

1. The Parakeet backend exposes a plain ASR transcribe_segments(...) path via onnx-asr timestamp adapters when available. The engine and benchmark CLI consume this path for recordings so Parakeet dictation can carry TranscriptSegment timing metadata, not just plain text.
2. A repeatable local flite-smoke benchmark fixture generator exists at scripts/generate-benchmark-fixtures.sh. These fixtures are for speed, JSON-shape, timestamp, and gate smoke evidence only; curated human speech recordings are still required for final WER promotion.
3. A repeatable longer flite benchmark fixture generator exists at scripts/generate-long-benchmark-fixtures.sh. It writes a longer WAV plus manifest.csv and manifest-3x.csv so CPU speed runs can repeat the same synthetic audio with less startup-overhead distortion than the short smoke fixture.

Windows VM evidence on win11-dev:

1. scripts/windows-vm-smoke.sh --vm win11-dev --mode syntax passes.
2. scripts/windows-vm-smoke.sh --vm win11-dev --mode install passes, including clean guest app-data setup, install, Python compile, 314 focused Windows tests, dictate --version, default Parakeet doctor, and cleanup.
3. scripts/windows-vm-smoke.sh --vm win11-dev --mode lifecycle passes, including clean install, update, default Parakeet doctor, and uninstall.
4. scripts/windows-vm-smoke.sh --vm win11-dev --mode build --keep-guest-workdir passes for the no-bundle Windows desktop build. Verified guest artifacts: C:\Users\Public\dictate-vm-smoke\source\ui-shell\src-tauri\target\release\dictate-ui-shell.exe and C:\Users\Public\dictate-vm-smoke\source\ui-shell\src-tauri\target\release\engine\dictate-engine.exe.
5. Current dirty-tree no-bundle build verification after the preflighted lane runner, stricter plan audit, refreshed install smoke, validated evidence import, and segment-aware UI refresh: scripts/windows-vm-smoke.sh --vm win11-dev --mode build --timeout 1800 --keep-guest-workdir passed on 2026-07-05. The guest rebuilt the UI with Vite, froze and smoke-tested the Python engine sidecar, built the Tauri shell with DICTATE_BUNDLES=no-bundle, and verified the same no-bundle shell executable and engine sidecar artifact paths listed above.
6. Current dirty-tree MSIX verification after the preflighted lane runner, stricter plan audit, refreshed install smoke, validated evidence import, segment-aware UI refresh, and refreshed no-bundle build: scripts/windows-vm-smoke.sh --vm win11-dev --mode msix --timeout 2400 --keep-guest-workdir passed on 2026-07-05. The builder regenerated the UI with Vite, froze and smoke-tested the Python engine sidecar, built the Tauri shell, packed the MSIX with Windows SDK makeappx.exe, unpacked it, and validated manifest identity plus dictate-ui-shell.exe, logo assets, and engine\dictate-engine.exe. Verified guest artifact: C:\Users\Public\dictate-vm-smoke\source\packaging\msix\out\ArcForgeDictate_2026.7.4.0_x64.msix.
7. Current dirty-tree Windows install verification after the evidence collector, preflighted lane runner, stricter plan audit, Parakeet-first default, Meeting readiness, and model-preparation changes: scripts/windows-vm-smoke.sh --vm win11-dev --mode install --timeout 1800 --keep-guest-workdir passed on 2026-07-05. The smoke resets the guest Dictate app-data directory, seeds a fresh config, compiles Python sources, runs 314 focused tests with 16 skips in the guest, verifies dictate 2026.7.4, runs dictate doctor --quick --type-backend pynput, and reports STT backend: parakeet with STT model: parakeet-tdt-0.6b-v2 before uninstalling cleanly.
8. Current dirty-tree Windows lifecycle verification after the preflighted lane runner, stricter plan audit, refreshed install smoke, and current package checks: scripts/windows-vm-smoke.sh --vm win11-dev --mode lifecycle --timeout 2400 --keep-guest-workdir passed on 2026-07-05. It performed the clean install path with 196 focused tests and 15 skips, ran update, then ran default dictate doctor --quick --type-backend pynput again and reported the same Parakeet v2 default before uninstalling.

Windows MSI/NSIS installer bundling is not yet promoted. In the current win11-dev VM, Tauri/WiX MSI bundling fails because candle.exe exits with -2146232576 (CLR/.NET runtime load failure), and NSIS bundling also fails inside Tauri. The verified Windows human-test artifacts are currently the no-bundle executable plus engine sidecar and the Store-style MSIX package, not MSI/NSIS installers.

Update scope decision: Staged update preparation is deferred out of the human-test release. The human-test release uses the tested immediate source update path (update-windows.ps1 through scripts/windows-vm-smoke.sh --mode lifecycle) plus the documented MSIX/manual package path. The source lifecycle smoke verifies install, update, post-update doctor, and uninstall on win11-dev; the staged available -> preparing -> ready UX and persistent skipped-version state are deferred until after human testing.

Benchmark evidence foundation:

1. dictate benchmark runs plain ASR. Its speaker options (--diarize, DER and speaker-confusion gates) went with Meeting on 2026-10-03 (#140).
2. JSON output records backend/model/device, WER, RTF, RTFx, peak RSS when the platform exposes it, per-sample hypotheses, optional timed segments, segment-boundary mean absolute error, boundary-pair counts, and promotion gate results.
3. scripts/run-transcription-lane-benchmarks.sh is the canonical local lane runner for writing the CPU, CPU multilingual, and curated-human CPU JSON artifacts consumed by this plan and the plan audit. Every lane runs on the CPU; its CUDA and AMD lanes were removed with #112 and its Meeting lanes with #140. Use --dry-run first on target machines to verify the exact commands without loading models.
4. The canonical lane runner preflights each selected lane with dictate doctor --quick for the exact backend/model before benchmark work starts. This fails fast when the Parakeet runtime is unavailable. --skip-preflight is reserved for intentionally collecting failed benchmark JSON artifacts.
5. scripts/collect-transcription-evidence.sh and scripts/collect-transcription-evidence.ps1 package the plan, benchmark JSON artifacts, audit output, lane-runner dry-run output, per-lane doctor readiness output, and basic non-secret machine/CPU context into a timestamped archive under evidence-bundles/ for CPU and Windows test-machine handoff. The plan audit requires that output directory to be ignored by Git.

AMD GPU Path (dropped) #

AMD GPU lanes were dropped on 2026-10-01 with the rest of GPU support. There is no AMD acceptance gate, and the readiness audit no longer requires the AMD promotion artifacts (benchmark-results/parakeet-v2-amd-human-gated.json and benchmark-results/parakeet-v3-amd-human-gated.json). The AMD benchmark runners and the AMD audit gate were removed with #112, and the amd extra with #110. The DirectML and ROCm/MIGraphX provider checks are still in the tree, unsupported, until #111 removes them. Earlier revisions of this file hold the old AMD runtime plan and gates.

Meetings (shelved) #

Meeting capture, local speaker labelling (pyannote Community-1, DiariZen and NVIDIA Sortformer lanes), the meeting and sortformer extras, the bundled pyannote model and its Hugging Face token were removed from the app and every build on 2026-10-03. The work is P3 and kept on the archive/meeting-2026-10-03 branch and tag; #140 records its state then (two speakers labelled in the frozen Linux engine, never run on Windows, about 20 s added to the first engine state request) and the bar for bringing it back. A saved meeting_stt_backend or meeting_stt_model is removed from config.yaml on upgrade with one notice. Meeting transcripts saved before then stay in history, readable, searchable and exportable with their speaker labels.

UI And App Wiring #

The desktop UI must stay product-level while the engine lanes mature. The app surface should continue to expose Record, push-to-talk dictation and notes; model/provider names remain advanced configuration and diagnostics.

Current UI/backend wiring that belongs to this deployment:

1. Capture home, push-to-talk, and plain recording use the same Parakeet ASR path.
2. Durable notes can carry structured transcript segments with optional timestamps. Speaker labels appear only on meeting transcripts saved before Meeting was removed.
3. Expanded notes, list search, copy, and Markdown export understand structured segments while preserving the plain text path.
4. The UI server exposes live state, history/notes, config, doctor, and update routes over the loopback API used by the Tauri shell.
5. Mock/demo dictations are restricted to dev/test/browser preview mode or an explicit VITE_DICTATE_ENABLE_MOCK=1 opt-in. A packaged shell with no live engine bridge must fail visibly instead of returning canned fixture text.

Remaining app-level gates:

1. Validate segment rendering against real recordings, not only mock/dev output.
2. A .deb update downloads and verifies while the app stays open. Install and restart are a second click. The Windows lifecycle smoke still covers its own installer path.
3. Persistent skipped update versions are deferred with the staged update UX. The current UI-only localStorage skip remains a non-authoritative preview behavior until staged updates are promoted.
4. Dictate is local-only (VISION.md); there is no hosted provider lane to promote and no provider-key or switchable-backend UI.

Timestamping #

All local systems should produce integrated timestamps. Parakeet v3 documents word and segment timestamp support. The implementation should also study WhisperX-style alignment/reconciliation patterns, but timestamping must be owned by Dictate rather than depending on WhisperX as the product stack.

Benchmark timestamp quality separately from WER:

1. Segment start/end error.
2. Word-level timing error where available.
3. Stability on long recordings.

Benchmark Requirements #

Before promoting a lane to default, benchmark:

1. WER for transcript quality.
2. RTFx and live latency.
3. RAM use.
4. First-run download size and install/package weight.
5. Runtime failure rate and fallback behavior.
6. Windows and Linux coverage where that lane is intended to ship.

The required artifact format is the dictate benchmark --json-output ... report documented in benchmarks/README.md. Human-readable console output is not enough for promotion because it cannot be compared reliably across hardware lanes.

There is no GPU tier: GPU lanes were dropped on 2026-10-01. Lanes are benchmarked and promoted on the CPU of everyday machines.

Human-Test Acceptance Checklist #

The deployment is ready for human testing only when these checks are true:

1. CPU English: fresh install selects Parakeet v2, records real microphone speech, stores the note transcript, and does not repeat stale fixture text.
2. Plain recording: pressing Record uses the same ASR behavior as push-to-talk and does not expose speaker-attribution engine terms.
3. Windows VM: install, lifecycle, no-bundle build, and MSIX package smokes pass on win11-dev; the tested artifact paths are documented for human testers.
4. Packaging: the public Windows path is either a tested Store/MSIX route or an explicitly approved temporary artifact; no stale MSI/NSIS assumption is presented as ready.
5. Regression: focused Python tests for STT selection, model preparation, UI server state, tray helpers, Windows platform behavior, and settings migration pass locally.

These five items are exactly the items --readiness reports. The NVIDIA English, NVIDIA multilingual, AMD English and AMD multilingual items were removed on 2026-10-01 when GPU lanes were dropped, and the Meeting item on 2026-10-03 when Meeting was removed (#140).

Run python3 scripts/transcription_plan_audit.py before handing a build to a tester. It summarizes which plan gates have local evidence and which are still blocked. Use --strict in automation; it exits 2 while required promotion evidence is missing. Run python3 scripts/transcription_plan_audit.py --readiness for the tester-facing checklist view. It maps the human-test acceptance items above to the audit gates and reports five items: CPU English, Plain recording, Windows VM, Packaging and Regression. All five are decided by files in the repository. No GPU or Meeting item is reported. Use --readiness --json for a machine-readable handoff report. Use scripts/run-human-test-readiness.sh on Linux source installs, or powershell.exe -NoProfile -ExecutionPolicy Bypass -File scripts\run-human-test-readiness.ps1 on Windows source installs, to print the readiness report, run a quick local Parakeet doctor, and show the manual real-microphone checks that must be completed on each human-test machine. These scripts intentionally print, rather than auto-pass, the microphone checks because the acceptance condition is spoken human audio.

Latest full Python regression evidence: uv run pytest -q passed with 687 passed, 5 subtests passed on 2026-07-05 after the UI mock-mode guard, benchmark fixture tooling, preflighted lane runner, evidence collector, validated evidence importer, human-test readiness report and wrappers, Windows MSIX, and segment-aware UI refresh (and the Meeting lanes, since removed by #140). uv run python -m py_compile src/dictate/__main__.py src/dictate/benchmark.py src/dictate/doctor.py src/dictate/stt/factory.py src/dictate/ui_server.py also succeeded on the same worktree. Latest readiness and plan-audit evidence: tests/test_transcription_plan_audit.py runs python3 scripts/transcription_plan_audit.py --readiness on the checkout in CI and requires all five items READY; the audit has a pass gate for the canonical plan, benchmark fixture tooling, Windows VM package evidence, and the staged-update deferral decision. The Meeting gate was removed with Meeting on 2026-10-03 (#140); before that, on 2026-10-01 (#112), it passed on the operator workstation from local CPU curated-human artifacts. Benchmark JSON reports record Python, platform, machine, and processor context only. scripts/run-human-test-readiness.sh and scripts/run-human-test-readiness.ps1 provide the source-install handoff wrapper for that report plus the manual microphone checks; both run the CPU doctor. Readiness and audit evidence from 2026-07-05, including the CUDA, AMD and Meeting gates, is in earlier revisions of this file.

Latest Parakeet-first local default evidence: uv run pytest tests/test_default_local_model.py tests/test_main_stt_selection.py tests/test_stt_registry.py tests/test_parakeet_backend.py tests/test_ui_server.py tests/test_main_benchmark_dispatch.py tests/test_model_prepare.py -q passed with 142 passed; uv run pytest tests/test_windows_platform.py tests/test_main_benchmark_dispatch.py tests/test_default_local_model.py tests/test_main_stt_selection.py tests/test_ui_server.py tests/test_stt_registry.py -q passed with 156 passed; uv run python -m py_compile src/dictate/stt/factory.py src/dictate/__main__.py src/dictate/doctor.py src/dictate/ui_server.py succeeded; and uv run dictate doctor --quick --type-backend pynput exited 0 with STT backend: parakeet and STT model: parakeet-tdt-0.6b-v2 on 2026-07-05. Fresh local startup defaults now choose Parakeet v2 when the Parakeet runtime is importable; faster-whisper was the fallback when Parakeet was unavailable until it was removed on 2026-09-30; Parakeet is now a core dependency.

Latest Windows fresh-install Parakeet default evidence: scripts/windows-vm-smoke.sh --vm win11-dev --mode install --timeout 1800 --keep-guest-workdir passed again on 2026-07-05 after the smoke expanded to cover benchmark parser, benchmark dispatch, benchmark fixture-generator, preflighted canonical lane-runner dry-run, evidence collector dry-run, and stricter transcription-plan audit tests. The guest seeded %APPDATA%\dictate\config.yaml, ran 314 focused Windows tests with 16 skipped, verified dictate 2026.7.4, and default dictate doctor --quick --type-backend pynput reported STT backend: parakeet with STT model: parakeet-tdt-0.6b-v2. scripts/windows-vm-smoke.sh --vm win11-dev --mode lifecycle --timeout 2400 --keep-guest-workdir also passed again on 2026-07-05 after the preflighted lane runner, stricter plan audit, refreshed install smoke, and current package checks: the install phase ran 196 focused Windows tests with 15 skipped, default doctor reported Parakeet v2, update completed, and post-update doctor again reported Parakeet v2 before uninstall. Staged update preparation is deferred out of the human-test release. The human-test release uses the tested immediate source update path verified by the same lifecycle smoke; staged update preparation and persistent skipped-version state are post-human-test work. After adding tests.test_transcription_plan_audit to the Windows guest selection, the local matching command uv run python -m unittest tests.test_update_status tests.test_windows_platform tests.test_ui_server tests.test_model_prepare tests.test_default_local_model tests.test_main_stt_selection tests.test_stt_registry tests.test_benchmark tests.test_main_benchmark_dispatch tests.test_benchmark_fixtures tests.test_transcription_plan_audit tests.test_config_commands tests.test_config_selection tests.test_daemon_history also passed with 314 tests.

Latest Windows package evidence: scripts/windows-vm-smoke.sh --vm win11-dev --mode build --timeout 1800 --keep-guest-workdir passed on 2026-07-05 after the validated evidence import and segment-aware UI refresh. It rebuilt the UI with Vite, froze and smoke-tested dictate-engine.exe, built dictate-ui-shell.exe with DICTATE_BUNDLES=no-bundle, and verified both the shell executable and engine\dictate-engine.exe sidecar. scripts/windows-vm-smoke.sh --vm win11-dev --mode msix --timeout 2400 --keep-guest-workdir then passed on the same current worktree after the same UI refresh: it rebuilt the UI and engine sidecar, built the Tauri shell, packed the MSIX with Windows SDK makeappx.exe, unpacked it, and validated the manifest, shell executable, logo assets, and engine sidecar payload. Verified MSIX guest artifact: C:\Users\Public\dictate-vm-smoke\source\packaging\msix\out\ArcForgeDictate_2026.7.4.0_x64.msix.

Latest structured segment evidence: uv run pytest tests/test_engine_capabilities.py tests/test_note_store.py tests/test_daemon_history.py tests/test_parakeet_pyannote_backend.py -q passed with 101 passed on 2026-07-05.

Latest benchmark evidence-format regression: uv run pytest tests/test_benchmark.py tests/test_main_benchmark_dispatch.py tests/test_windows_platform.py tests/test_stt_registry.py -q passed with 59 passed on 2026-07-05.

Latest UI evidence: npm test -- --run passed with 62 passed, and npm run build completed successfully from ui/ on 2026-07-05.

Latest stale-fixture UI guard evidence: npm test -- --run src/test/ipc.test.js src/test/app.test.jsx passed with 48 passed from ui/, and npm run build completed successfully on 2026-07-05. Coverage verifies that a Tauri shell without an injected engine bridge is not mock mode, does not seed demo history, and does not return the canned project-note transcript.

Latest segment UI/export evidence: uv run pytest tests/test_ui_server.py tests/test_daemon_history.py tests/test_note_store.py -q passed with 123 passed, and npm test -- --run src/test/app.test.jsx src/test/ipc.test.js passed with 44 passed from ui/ on 2026-07-05.

Latest persisted segment rehydration evidence: uv run pytest tests/test_note_store.py tests/test_ui_server.py tests/test_daemon_history.py -q passed with 125 passed, and npm test -- --run src/test/app.test.jsx src/test/ipc.test.js && npm run build passed with 45 UI tests plus a successful production build from ui/ on 2026-07-05.

Latest segment-aware notes search/export evidence: npm test -- --run passed with 62 UI tests from ui/ on 2026-07-05. The suite now includes a persisted Meeting note shaped like the curated Open Speech Repository fixture: snake_case segment timestamps, speaker ids, speaker labels, and fractional turn boundaries. Notes search can match segment-only text, the list preview renders normalized speaker segment content instead of only a generic note title, Markdown export preserves speaker labels and formatted timestamps, and the expanded note renders the same real-fixture-style segment rows.

Latest non-meeting recording rule evidence: uv run pytest tests/test_daemon_history.py tests/test_provider_supervisor.py -q passed with 129 passed, and uv run python -m py_compile src/dictate/daemon.py succeeded on 2026-07-05. Note/plain recordings now use the same non-speaker ASR path as push-to-talk, including the long-recording remote retry path. Meeting, the one mode that asked for speaker attribution, was removed with #140.

Latest timestamp/DER benchmark-gate evidence: uv run pytest tests/test_benchmark.py tests/test_main_benchmark_dispatch.py -q passed with 12 passed, and uv run python -m py_compile src/dictate/benchmark.py succeeded on 2026-07-05. The benchmark CLI now supports failing promotion gates for WER, RTF, DER, speaker confusion, segment-boundary MAE, required timestamp metrics, and required DER metrics. Runtime failures during benchmark transcription now write the requested JSON artifact with a failed benchmark_runtime gate instead of producing only a traceback. The DER and speaker-confusion gates were removed with Meeting (#140).

Latest Parakeet timestamp-segment evidence: uv run pytest tests/test_parakeet_backend.py tests/test_engine_capabilities.py tests/test_benchmark.py tests/test_main_benchmark_dispatch.py -q passed with 40 passed; uv run pytest tests/test_parakeet_backend.py tests/test_parakeet_pyannote_backend.py tests/test_engine_capabilities.py tests/test_benchmark.py tests/test_stt_registry.py -q passed with 66 passed; and uv run python -m py_compile src/dictate/stt/parakeet_backend.py src/dictate/engine.py src/dictate/benchmark.py succeeded on 2026-07-05.

Latest evidence-import evidence: scripts/import-transcription-evidence.py imports returned Linux .tar.gz or Windows .zip evidence bundles by copying benchmark-results/*.json into the local repo and can rerun the audit with --audit, so returned promotion artifacts can be validated without manual file copying. It now validates each candidate artifact as benchmark JSON with config and summary objects before copying, supports evidence directories plus .zip, .tar.gz, and .tgz bundles, and ignores unsafe archive member paths. uv run pytest tests/test_benchmark_fixtures.py tests/test_transcription_plan_audit.py -q passed with 27 passed on 2026-07-05 after adding zip dry-run, tar import, no-overwrite, malformed-JSON rejection, and audit coverage.

Latest CPU Parakeet v2 long-fixture evidence: scripts/generate-long-benchmark-fixtures.sh generated a longer synthetic flite fixture under benchmark-fixtures/flite-long/ with a 47.81s WAV and a three-row repeat manifest. uv run dictate benchmark --manifest benchmark-fixtures/flite-long/manifest-3x.csv --audio-root benchmark-fixtures/flite-long --stt-backend parakeet --model parakeet-tdt-0.6b-v2 --device cpu --language en --require-timestamp-metrics --max-mean-rtf 1.00 --max-mean-segment-boundary-mae-s 1.00 --json-output benchmark-results/parakeet-v2-cpu-flite-long-3x-gated.json --run-label local-cpu-flite-long-3x-gated passed through the preflighted canonical lane runner on 2026-07-05 with mean_rtf=0.0345, mean_rtfx=28.98, mean_wer=0.0976, and mean_segment_boundary_mae_s=0.3830. It remains synthetic; curated human speech benchmarks are still required before final product promotion.

Latest long fixture and lane-runner reproducibility evidence: bash -n scripts/generate-long-benchmark-fixtures.sh scripts/generate-benchmark-fixtures.sh scripts/generate-meeting-benchmark-fixtures.sh scripts/run-transcription-lane-benchmarks.sh scripts/collect-transcription-evidence.sh passed, and uv run pytest tests/test_benchmark_fixtures.py -q passed with 8 passed on 2026-07-05. The lane runner dry-run lists the canonical CPU, multilingual CPU, and Meeting smoke artifact names before loading models, and shows the preflight doctor commands that will fail fast for a missing runtime or gated Meeting model access (lane set reduced to CPU on 2026-10-01, #112). It also exposes explicit cpu-human, cpu-human-v3, meeting-human, meeting-diarizen-human, and meeting-sortformer-human lanes that require curated manifests through DICTATE_HUMAN_MANIFEST or DICTATE_MEETING_HUMAN_MANIFEST and write artifacts marked fixture_class=curated-human. dictate benchmark --validate-manifest-only now validates curated-human manifests before model loading, rejecting generated benchmark-fixtures/ paths, flite sample IDs or filenames, missing audio files, missing timestamp references, and missing timestamped speaker references for Meeting/DER lanes. The Bash and PowerShell evidence collectors now include human-lane-dry-run.txt with the exact curated-human CPU and Meeting promotion commands plus promotion-status.txt showing whether each required promotion artifact is present or missing. The collector dry-runs list the handoff bundle contents without exposing token or environment-variable values and include lane-readiness.txt with quick doctor output for CPU, pyannote, DiariZen, and Sortformer lanes. The default evidence-bundles/ output path is ignored by Git. scripts/windows-vm-smoke.sh --vm win11-dev --mode syntax --timeout 900 --keep-guest-workdir also passed again on 2026-07-05 after the collector promotion-status changes and parsed scripts/collect-transcription-evidence.ps1 in the Windows guest. The Meeting lanes, the meeting fixture generators and their collector entries were removed with #140.

Implementation Phases #

1. Consolidate defaults

   • Keep fresh CPU English installs on Parakeet v2.
   • Keep recordings and push-to-talk on the same ASR path.

2. Parakeet on the CPU

   • Benchmark Parakeet v3 CPU feasibility.
   • Measure and tune release-to-text latency on everyday CPU machines.
   • The CUDA and AMD runtime tasks that were here were dropped on 2026-10-01; their tooling and install paths went with #112 and #110, and the engine's device selection goes with #111.

3. Meeting speaker attribution — removed from the app on 2026-10-03 and shelved as P3 (#140); see "Meetings (shelved)" above.

4. Remove Whisper product dependency — done 2026-09-30, ahead of the Parakeet multilingual benchmark gate, by owner decision. The desktop UI already forced Parakeet, and faster-whisper pulled a GPL FFmpeg build (through PyAV) into the bundle. Benchmark comparison rows stay as historical evidence only.

Model Landscape Review (2026-09-20) #

Periodic check of the open STT landscape against the lanes above. Numbers are from the Open ASR Leaderboard English short-form track (cleaned datasets, so they are not comparable to pre-2026 snapshots). RTFx is the leaderboard's GPU figure and only useful as a relative ordering.

Parakeet Unified 0.6b evaluation #

nvidia/parakeet-unified-en-0.6b (2026-04-07) is a FastConformer-RNNT that runs offline and streaming (down to 160 ms) from one checkpoint. NVIDIA claims it beats parakeet-tdt-0.6b-v2 offline. It was evaluated on 2026-09-20 on a 16-core CPU workstation, int8, as a possible v2 successor, using two community ONNX exports (an onnx-asr layout and the k2/sherpa-onnx maintainer's export) so the export was not the variable. Fixtures: 8 LibriSpeech test-clean clips (216 reference words, 79 s) and the repo's open-speech-harvard fixture (scripts/generate-curated-human-asr-fixture.sh, 80 words, peak level 0.34).

Findings:

1. On clean, well-levelled audio unified matches v2 exactly.
2. Unified is strongly input-level sensitive; v2 is not. Peak-normalising the Harvard fixture takes unified from 23.75% to 8.75%, still more than twice v2. Dictate's always-on AGC (audio_preprocess.py, target -18 dBFS RMS) would narrow but not close that gap.
3. Unified degrades under additive noise where v2 does not.
4. Unified is roughly 30% slower than v2 on CPU int8 with no accuracy gain.
5. Streaming needs the sherpa-onnx runtime; onnx-asr has no streaming path.

Decision: parakeet-tdt-0.6b-v2 stays the English default. Unified is a streaming-lane candidate only, to be revisited if live-text-while-holding-key becomes a product goal, and then together with nemotron-speech-streaming and a sherpa-onnx backend decision. Any such evaluation must include quiet-input and noisy-input fixtures, not only clean read speech.

Runtime notes #

1. onnx-asr 0.12.0 (2026-07-15) adds convolution-based ONNX preprocessors for the GPU path.
2. ONNX Runtime is now pinned to 1.30.x for both the CPU and CUDA lanes. onnxruntime-gpu 1.27+ ships CUDA 13 runtime wheels (nvidia-cuda-runtime, nvidia-cudnn-cu13), which need NVIDIA driver 580 or newer; 1.26 was the last CUDA 12 build. The old <1.24 pin existed for the external-data path check; Dictate stages Parakeet as flat real files, and an external-data model loads on 1.30 (verified 2026-09-20 on CPU). The CUDA lane was dropped on 2026-10-01 and is not re-validated.
3. onnxruntime-directml stopped at 1.24.4; the amd extra was capped at <1.25 until #110 removed it. Microsoft has retired that wheel. The Windows AMD lane would have needed a replacement runtime; that question closed when GPU lanes were dropped on 2026-10-01.
4. sherpa-onnx 1.13.8 supports Parakeet Unified (offline and streaming), Nemotron streaming, Moonshine, Qwen3-ASR, and Cohere Transcribe. It is the runtime to evaluate if a streaming lane is opened.

Archived Notes #

Archived notes are reference material only. They do not override this plan.

Sources #

• Open ASR Leaderboard paper/table: https://arxiv.org/html/2510.06961v4
• Parakeet v2 model card: https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2
• Parakeet v3 model card: https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3
• Canary/Parakeet v3 technical report: https://arxiv.org/abs/2509.14128