Kev 1.0

Kev 1.0 is the first versioned release of the whole Kev family: four decision models that read a document and a set of typed questions and return calibrated probabilities over the options, in one forward pass, behind TypeSafe’s System One API. Nothing in it is newly trained. It pins the checkpoints, model cards, evaluation suites and serving code that the next generation of Kev is measured against, with the same tag (v1.0) on every Hub repo.

What’s in 1.0

Headline numbers (fp32 evaluation path, each model at its shipped temperature; the transfer-v4 test is locked and was read once per model):

hard-v1, devtools-v1 and documents-v1 have training splits that every Kev trained on: those rows measure held-out items of trained families, not transfer. The breadth-v1 and transfer-v4 rows are datasets no Kev trained on. Jev was read on the development partitions and on breadth-v1 test only. Every number traces to a committed report through docs/claims.json; the model cards (docs/model-cards/) have the rest, with intervals.

What changed since the last family release

Measured from the GitHub release kev-family as first assembled for the current family on 2026-09-24 (Kev-27B v1, Kev-9B v1, and the same Kev-4B and Kev-0.8B as here). Its 2026-09-30 updates (Kev-27B v2, Kev-9B v2) are listed here too, since 1.0 is where they become part of a versioned release.

• Kev-27B v2: full weights. Every weight of Qwen3.8-27B fine-tuned for one epoch on a 145,840-record corpus, then averaged 0.85 / 0.15 with v1. Against v1 on test: held-out datasets +1.2 pp [+0.3, +2.2], held-out task families +5.3 [+3.7, +6.8], skills, tooling and documents +8.9 [+7.5, +10.3]; locked out-of-domain test 0.889 against 0.896, Brier 0.154 against 0.160. Worse and overconfident on long contracts (CUAD ECE 0.053 against 0.007). v1 is at jaredpalmer/kev-27b@v1-lora.
• Kev-9B v2. v1 plus one epoch on the documents and skills data Kev-4B and Kev-0.8B already had. Against v1 on test: hard-v1 + devtools-v1 +18.7 pp [+16.7, +20.8], documents-v1 +7.1 [+4.7, +9.2]; level on the locked out-of-domain test (0.852 both) with Brier 0.199 against 0.224. v1 is at jaredpalmer/kev-9b@v1.
• No silent truncation. The server used to cut a state longer than its limit without saying so. It now refuses a state over 65,536 tokens with a 422 that names the token count and the limit; KEV_TRUNCATE_STATES=1 opts back into truncation, and every response from such a server then says truncated. The deploy and fine-tune skills pin KEV_REF to a commit that has this fix and the long-document and MLX changes below (71d4829), and the Space was republished after the fix.
• Long documents on every size. The evaluation path kept fp32 attention for long rows on the math kernel, so Kev-0.8B, 4B and 9B ran out of GPU memory on 32k–64k-token states. Long rows now run the memory-efficient kernel in fp32: Kev-4B reads a 61k-token state in 17.2 s with 17.2 GiB above the weights on an H100, and shorter rows keep their logits bit for bit. This is what makes the validated context lengths above measurable.
• Apple Silicon. The MLX backend loads full-weight checkpoints as saved, with no merge, which gives Kev-27B a Mac path (expected to need about 51 GB plus working memory; not yet run at that size). Long states are prefilled 1,024 tokens at a time and the cache evicts before a pass, so Kev-4B serves a 65,000-token state on a 32 GB M5 at a 13.0 GB peak (84.5 s new, 716 ms cached).
• Kernel provenance. Every evaluation report and trial now records the kernel set its logits depend on (package versions, GPU, dtype, attention and DeltaNet implementations), after a kernel change in the evaluation image was found to move Kev-27B v1’s reads by 0.03–0.06 in probability with no change to Kev’s own code.
• Evaluation audit. Three suites were removed as unsound for selecting models: scienthoon (templated tickets, one question the text cannot answer), WANLI-v2 / WANLI-v1 (a quarter of gold labels are one of two disagreeing annotators) and TypeSafe’s public evals (gold from two closed models, too few questions). Headline panels exclude items the audit found unanswerable or unlabelled. Past releases’ figures on those suites are kept in their records, not on the 1.0 cards.
• Calibration. Kev-4B and Kev-0.8B ship temperatures fitted on held-out items of their training data. A registered refit on held-out datasets was evaluated for both and adopted for neither: it did not improve Kev-4B (Brier difference −0.0001 [−0.0005, +0.0003]), and it made Kev-0.8B worse calibrated on its document and skill families by more than the registered tolerance. Kev-9B and Kev-27B already ship held-out-dataset temperatures.
• Training data published. The documents-v1 and hard-v1 training partitions are in the jaredpalmer/kev-suites dataset, so the small models’ training data can be fetched and hash-checked.
• Validated context length. Each card now states the longest state at which accuracy on CUAD contracts stays within 3 pp (95 % lower bound) of the same model at 8k tokens. Kev-27B holds to 65,536 tokens, the serving limit (its 64k lower bound is −2.4 pp). Kev-0.8B, 4B and 9B validate only their trained 8,192: each already misses the tolerance at 16k (lower bounds −8.5, −3.4 and −3.7 pp), so past 8k tokens their answers on long documents are not covered by the measurement.
• Formal model cards. All four cards follow one structure: summary, details, intended and out-of-scope uses, how to use, training data and procedure, evaluation, limitations, risks, compute, provenance.

Known limitations

• In-distribution gains. The large gains of the last year are on suites whose training splits are in the training data. On datasets no Kev trained on, Kev-27B is 1.7 index points below Jev on breadth-v1 test, and the smaller sizes are 13–31 points below.
• Untrained lengths. Kev-0.8B, 4B and 9B trained on states of at most 7,552 tokens and Kev-27B on at most 32,768; the server accepts 65,536. Use the validated context length, not the serving limit.
• Kev-27B on long contracts is less accurate than v1 and overconfident (CUAD test ECE 0.053 against 0.007); refit the temperature on your own documents or use @v1-lora for contract review.
• Kev-0.8B and tool routing. Its When2Call accuracy fell below chance (0.133 on test) after its documents-and-skills stage; do not use it for tool-call routing.
• Date arithmetic is the weakest family at every size below 27B (deadline policy accuracy 0.35 / 0.65 / 0.725 against Jev’s 0.95); KEV_DATE_FACTS=1 helps.
• Knowledge is set by the base (MMLU-Pro 0.230–0.675 against Jev’s 0.840).
• Kev-9B on a Mac has not been measured, and Kev-27B on a Mac is expected to fit 96–128 GB but has not been run.
• Selection. Kev-27B v2 and Kev-9B v2 were re-selected under the audited rule with earlier development reads known; their test margins are optimistic.
• One temperature per model cannot reorder confidences, so at a 5 % error budget the models automate fewer decisions than Jev out of domain.

How to run

git clone https://github.com/jaredpalmer/kev.git && cd kev && uv sync --extra serve
uv run --extra serve python -m kev.serve --run jaredpalmer/kev-4b@v1.0 --port 8009    # CUDA, or MLX on Apple Silicon

From a release tarball:

shasum -a 256 -c SHA256SUMS.txt
tar -xzf kev-4b.tar.gz
uv run --extra serve python -m kev.serve --run kev-4b --port 8009

The TypeSafe SDK works unchanged: TypeSafeClient(api_key="local", base_url="http://127.0.0.1:8009", model="kev-latest"). Kev-27B needs one B200, H200 or H100 80 GB: --run jaredpalmer/kev-27b@v1.0. To deploy an HTTPS endpoint on Modal, see skills/kev-deploy.

Assets

Each tarball holds one checkpoint as it is on the Hub at its weights revision (LoRA adapter, head.pt with the temperature, tokenizer files, the training trial’s result.json, provenance.json, training_config.json, training_metrics.json and log), the Kev 1.0 model card as README.md, and the locked transfer-v4 read as locked_test.json. They are built by scripts/build_release_assets.py from docs/releases/kev-1.0-assets.json, and rebuilding gives the same bytes. Every file in them is dated 2026-10-01 00:00 UTC, which kev.serve reports as the release date of an unpacked checkpoint. The tarballs first attached on 2026-10-01 dated their files 1970-01-01, so kev.serve reported 1969-12-31; they were replaced the same day by these. The weights and every other file are byte-identical; only the tarball hashes changed.

Kev-27B is not attached, because its 51 GB of weights are over GitHub’s 2 GB limit per asset. Download it from the Hub: jaredpalmer/kev-27b@v1.0 (weights commit 28be62e9, head.pt 7968f17b…).

On every Hub repo, the v1.0 tag points to the commit that uploaded the Kev 1.0 card. That commit changed only README.md, so its weights are the same bytes as the weights revision in the first table: kev-0.8b bf75a6a8, kev-4b 6cfce5c2, kev-9b db029f08, kev-27b af0e6d55.

Release plan (for the maintainer; not part of the published notes)

Done 2026-10-01 (record runs/release/kev-1.0.json; PLAN.md “Released: Kev 1.0”). Steps 3 and 4: card-only commits, with v1.0 on each card commit (0.8B bf75a6a8, 4B 6cfce5c2, 9B db029f08, 27B af0e6d55; every other file unchanged). Steps 5 and 6: assets built twice with identical hashes, the release published and marked Latest. Step 7: kev-family kept, its assets removed, its body a pointer to kev-1.0, its old notes in runs/release/kev-family-notes-retired.md. Step 8: pins unchanged. Step 9: collection and Space checked; the Space was not republished. The plan as written before the release follows. Order:

1. Placeholders: filled (2026-10-01) from round 28’s registered context read-out, runs/r28-readout/context.json (scripts/longdoc_report.py --context-margin -0.03 over runs/r28-{4b-r10,08b-r15}-longdoc, runs/r29-9b-r18a-longdoc and runs/r23-27b-k-w85-longdoc; raw runs/r28-context, ECE at the shipped T runs/r28-context-served), with the numbers in docs/claims.json.

2. Merge this PR.

3. Hub cards. Upload each 1.0 card as README.md only (no weights): kev.publish is not needed for a card-only commit; hf upload jaredpalmer/kev-<size> docs/model-cards/kev-<size>.md README.md --commit-message "Kev 1.0 model card (weights unchanged)". Check with HfApi().model_info(..., files_metadata=True) that adapter_model.safetensors / head.pt (27B: model.safetensors.index.json and every shard) hash as below.

4. Hub tags. v1.0 on all four repos. Default (as specified): the exact weight revisions; if step 3 ran first, tag the card commit instead so that @v1.0 shows the 1.0 card (the weights are byte-identical; record both commits in PLAN.md).

   Repo	v1.0 target (weights)	adapter / head sha256
   jaredpalmer/kev-0.8b	9a45d25eb2ab761841196625383fa1dff0e56c1e	9b908623… / f400bd12…
   jaredpalmer/kev-4b	139fdd94f1b6a6ad80cc15e08fcb99cac885a101	90e81735… / dd633435…
   jaredpalmer/kev-9b	b5d8c18e44c60888d138b65cb6507ff0a5a448a0	2b2a70cf… / 8e1dab2c…
   jaredpalmer/kev-27b	main (today ef78cc8a34d5f426fb229c52089db189218cfe5c: weights 28be62e9, then three card-only commits)	weights d27af6ab… / head 7968f17b…

   hf repos tag create jaredpalmer/kev-0.8b v1.0 --revision 9a45d25eb2ab761841196625383fa1dff0e56c1e -m "Kev 1.0"
   hf repos tag create jaredpalmer/kev-4b   v1.0 --revision 139fdd94f1b6a6ad80cc15e08fcb99cac885a101 -m "Kev 1.0"
   hf repos tag create jaredpalmer/kev-9b   v1.0 --revision b5d8c18e44c60888d138b65cb6507ff0a5a448a0 -m "Kev 1.0"
   hf repos tag create jaredpalmer/kev-27b  v1.0 --revision <main at release> -m "Kev 1.0"

5. Assets. uv run python scripts/build_release_assets.py --release docs/releases/kev-1.0-assets.json --out /tmp/kev-1.0-assets builds kev-0.8b.tar.gz, kev-4b.tar.gz, kev-9b.tar.gz (each: the Hub snapshot at the revision above, i.e. adapter, head.pt with the temperature, tokenizer files, the trial’s result.json, provenance.json, training_config.json and training_metrics.json; the 1.0 card as README.md; the locked read as locked_test.json), SHA256SUMS.txt and manifest.json (every member’s sha256). It refuses a download whose adapter or head hash differs from the spec. Kev-27B is not an asset (51 GB; GitHub caps an asset at 2 GB): the notes point to the Hub.

6. GitHub release. Tag kev-1.0 on the merge commit; create the release as a draft with the published part of these notes as its body (everything above this section), attach the three tarballs and SHA256SUMS.txt, download them back, shasum -a 256 -c SHA256SUMS.txt, extract one and serve it, then publish and mark it Latest.

7. One release per size. The release policy keeps only the best version of each size in a GitHub release. Once kev-1.0 is published, kev-family duplicates it: delete its three tarballs and SHA256SUMS.txt and replace its body with a pointer to kev-1.0 (or delete the release; Jared’s call). Earlier versions stay on the Hub tags listed in each card.

8. Deploy pins. skills/kev-deploy and skills/kev-finetune pin KEV_REF 71d4829; the 1.0 checkpoints need no newer code. Move the pin only if a later serving fix should ship with 1.0.

9. Collection and Space. The Kev collection already lists the four repos. The Space serves Kev-4B and Kev-0.8B from main, which is the 1.0 weights; nothing to republish unless kev/model.py, kev/api.py or kev/checkpoint.py changed after its last publish.