The measured track — v2 (2026-10-28)

Supersedes: 2026-10-28-measured-track-agentic-ai.md (v1, filed this morning — kept, not deleted,

because a roadmap that changed should show what it changed from).

Why v2: three measured facts landed after v1 was written, and each one moves a milestone.

AWS certs are dropped from the track entirely. They were the v1 lane on the strength of a plan from

2026-10-26. Nothing in the 98-posting sample asks for one, and the biggest gap (GPU serving) is not an AWS

Educate topic. A purchase before the artifact is a substitute for the work.

What an orchestrator actually needs when agents write the code

Four things survive automation of the writing. Each is a ticket that already exists.

1. Judging output, not producing it

Skill: build the golden set, the deterministic grader, the pre-registered decision rule, and read the

number honestly — including when it says keep the old thing.

Evidence it is purchasable: LLM evaluation, 56 of 98 postings (57%) — the single most-named

requirement in the sample.

Already built: EVAL-7. Golden set of 127 hand-reviewed pairs, grader with a 5/5 selftest, before/after

across 20 model calls, and a pre-registered rule the data contradicted — which I reported rather than

re-graded.

2. Knowing when NOT to build

Skill: the gap between "can't" and "should". Today: RAG cannot be built honestly on this box, and

saying so is correct — while the 204K-token corpus still means retrieval is the right technique. *"Can't

today" and "shouldn't ever"* are different sentences.

3. Systems that fail closed

Skill: deciding that a wrong number is worse than no number, then building the boundary that enforces

it. Guarantee over capability.

Evidence: guardrails named in 25 of 98 postings; tool calling in 32.

Already built: TOOL-42. Six bad-input classes rejected, rejection text proven actionable on a live

model, fail-closed proven, and the natural error rate reported as UNMEASURED at n=3, not 0%.

4. Tasting the difference between what is true and what is purchasable

The agent economy is the genuine thesis and it is early. Applied agent engineering and evaluation are

hiring now. Both are true. Choosing what to lead with is the orchestrator's call — that is the

judgment work, and it is why the org's front page carries both stories.

The plan, in order

1. RAG-9 — local embeddings on the VPS · 1 day, starts now

bge-small is ~160MB and runs on CPU. No endpoint needed, no GPU, no Forge.

• chunk the 204K-token textbook two ways (fixed-size vs structure-aware)
• embed locally, store a vector index
• measure hit@3 / hit@5 / MRR against the 127-pair golden set
• write one paragraph defending the chunk size from the numbers, not from a blog post

Closes: RAG (34 of 98 postings), and the "defend your chunking" interview question.

Honest scope: 160MB model on CPU — real retrieval, small model. Say both.

2. Forge — the same index at full scale · weekend

Dump the whole textbook on Forge, embed there, re-measure hit@k. **This is where the 8GB card earns its

place**: not as a server, as a workstation that can chew the full corpus without touching the VPS.

Closes: the gap between "I tried retrieval on a sample" and "I built an index over a real book."

3. Forge — local inference with measured numbers · weekend

vLLM or llama.cpp serving Qwen 2.5 7B Q4 (~5.0GB — fits in 8GB; 12B and Q8 do not).

Measure TTFT and tokens/sec before and after ONE optimisation (quantisation level, batch size, or

KV-cache settings). Report the 8GB constraint in the artifact title.

Closes: inference (Tier B), and it is the artifact that makes "I served a model" a claim with a

number behind it.

Honest limit: 8GB / 7B is a learning rig, not credential-grade for a role running multi-GPU serving.

It is the free rehearsal for…

4. Rented GPU — only when a posting requires it · 1–2 days, ~$10–30

Same benchmark, bigger model, real GPU. Do not do this before step 3 — pay for hardware after the skill

is already demonstrated, not instead of it.

5. MCP server — the orchestrator's product · weekend

Your org README already advertises an MCP server. Ship one that exposes the EDU tools (extract, evaluate,

record) so another agent can use them. That is what an orchestrator sells.

Closes: agent/tool-integration requirements, and it is the only artifact on this list that is a

product rather than a demonstration.

What each milestone is worth, ranked by (posting frequency × distance)

*Note what is not on this list: a certificate, a course, or a bootcamp.* Nothing in the sample asked

for any of them, and the two things that would move the needle — evaluation and inference — are artifacts,

not credentials.

What would change this track

• A live Q4 pull showing evaluation below 40% → milestone 1 drops in priority (but it is already done).
• A posting that requires a certificate → the "don't buy" answer changes.
• Forge turning out to have more than 8GB → step 3 gets bigger and step 4 may disappear.
• The agent economy maturing into posted roles → the org's current ten-repo story becomes the door rather

than the ceiling.

Not measured, and named as such

Interview performance, comp floor, relocation, and whether a human reading the resume agrees. **Posting

coverage is not eligibility, and it is not proficiency.** An 8GB 3070 estimate from quantisation bytes is

not a benchmark — it becomes one in step 3.