How to make deterministic skills in Claude Code

Most people treat Claude Code skills like well-written markdown files, a prompt with good instructions. But a skill is a folder, and that folder gives you control over five distinct parts of the execution. Use them and the result stops depending on which day you happened to run it.

We’ll walk through a concrete case: a skill called ai-news built to find and summarize AI news. We’ll follow it from its first version, a single file that “decides” everything again on every run, through to turning it into a nearly deterministic system.

There’s no trick involved. Just five small decisions about where each type of work lives.

A markdown file is not a system

This is how a skill usually gets written the first time. One folder, one file. It tells Claude what we want, and nothing about how.

1
.claude/skills/
2
└── ai-news/
3
    └── SKILL.md   ← a single file, instructions only

1
---
2
name: ai-news
3
description: Find the 50 most relevant AI news
4
  stories from the last 7 days.
5
---
6

7
# AI News
8

9
Find the 50 most relevant AI news stories from
10
the last 7 days. Summarize each one and present
11
the result as a clean report.

Every gap in the “how” is a decision Claude makes again, differently, on every run. The same skill launched Monday, Friday, or by someone else on the team, produces three different reports. Claude isn’t making a mistake, the execution was simply never fixed in place.

It’s worth putting the two versions of the same skill side by side, the one that changes on every run and the one that already has the five controls in place.

The mess:

• Sources change on every run: sometimes a solid outlet, sometimes whatever SEO blog ranked well that day.
• The method changes: a web search this time, a hand-rolled scraper the next, an MCP after that.
• The shape of the result changes: a table, then prose, then a bullet list.
• It’s slow: sources get visited one after another.
• There’s no way to know if a story is real, recent, or made up.

The system:

• A fixed, verified set of sources, decided once.
• A single process, captured in scripts, repeated on every run.
• One output template: the report looks the same week after week.
• Sources get visited in parallel: the run takes as long as the slowest one.
• Every story gets checked for accuracy, relevance, and recency before publishing.

The jump from the first list to the second is what this whole piece is about. Each of the five controls removes one source of variation, decided once and applied on every subsequent run. A longer prompt doesn’t fix anything on its own. What fixes the execution is giving Claude an artifact to follow instead of a decision to make.

The five controls

A skill folder lets you control five distinct parts of a run. Four of them increase determinism, meaning how consistently the result repeats. One, speed, increases efficiency.

Each control is an artifact inside a subfolder. None of those folder names is special to Claude Code, the only file the skill loader reads automatically is SKILL.md. Every other folder gets read because SKILL.md tells Claude to read it.

Here’s the usual order, from cheapest to most work:

1. Input · references/: decide where to read from, once. A closed world, no surprise sources.
2. Process · scripts/: capture the method as code, so it doesn’t get reinvented on every run.
3. Output · assets/: give Claude a literal template to fill in, not a design to invent.
4. Speed · agents/: split independent work across parallel subagents and keep the main context clean.
5. Tests · tests/: evaluate the substance with a fresh judge (LLM as judge) against a written rubric.

You almost never need all five at once. The rule is to add a control to fix a failure you’re actually seeing, not preventively. The ai-news skill only earned all five because each one closed an observed gap.

The names references/, scripts/, assets/ and agents/ are conventions, the same ones Anthropic’s official skill-creator uses for building Claude skills, so it’s worth sticking to them; tests/ is our own convention, the skill-creator keeps its cases in evals/. But you could rename any of them, as long as you wire it up from SKILL.md.

Control 1: input, with references/

The first source of variation is also the cheapest to fix. The skill never says where to look, so Claude picks the sources on its own, and picks differently every time.

One week it’s quality outlets, the next it’s a marketing blog that happened to rank well that day. Coverage can’t be repeated, and an unverified source is exactly where a made-up story slips in.

The fix is deciding the sources once. You put one markdown file per source in references/sources/, and have SKILL.md say: read every file in that folder, and use only those sources. The run now operates in a closed world that you defined.

references/ is the usual place for material Claude should read before or during a run: frameworks, schemas, whitelists. Claude doesn’t scan the folder by magic, the control happens in SKILL.md, which points to the folder. The reference files just hold the detail the instruction leaves out.

1
.claude/skills/ai-news/
2
├── SKILL.md
3
└── references/
4
    └── sources/
5
        ├── techcrunch-ai.md
6
        ├── the-verge-ai.md
7
        ├── arxiv-cs-ai.md
8
        ├── hacker-news.md
9
        └── anthropic-blog.md

Each source has a markdown file describing how to read it. Treat it as a small spec, detailed enough that someone new (or Claude) can pick it up cold:

1
# Source: TechCrunch AI
2

3
url: techcrunch.com/category/artificial-intelligence
4
language: en
5
pages_to_scrape: 3
6
list_selector: article.post-block
7
follow_to_article: yes
8
ignore: sponsored posts, event promos,
9
  newsletter subscription blocks
10
recency: each card shows a relative date,
11
  keep only items <= 7 days old
12
navigation: paginate with ?page=N
13
trust: mid tier-1, primary reporting

And here’s how it connects in SKILL.md:

1
## Step 1: Load the sources
2

3
Read every file in references/sources/.
4
Each file defines an approved source and how
5
to read it.
6

7
Use ONLY these sources. Don't fall back to open
8
web search. Don't add a source that isn't in this
9
folder. If a source is unavailable, mark it as
10
skipped, don't replace it with another.

The “don’t replace it” line does important work. Without it, Claude would fix a downed source on its own by finding something similar, and your whitelist would quietly stop being one.

The closed world gains repeatability, and it gains trust too. When the source list is fixed, Claude can’t end up on a low-quality or spoofed page. Adding a source becomes a deliberate edit you review, not an assumption made mid-run.

Control 2: process, with scripts/

The sources are fixed now, but the method isn’t. Claude reinvents how to fetch them on every run, which costs tokens and shifts coverage.

Without scripts, the same task gets solved differently each time: pure web search one run, a hand-written scraper the next, an MCP after that. A different method means different errors, different coverage, and plenty of tokens spent reasoning through the same “how” all over again.

The fix here is putting the scripts in scripts/ and having SKILL.md tell Claude to run them, instead of improvising. You generate the scripts once, with Claude’s help, review them, and from then on the skill just runs them.

Not everything can be a script. Some steps are, by nature, calls to a tool or an MCP, there’s no code to write, and it’s common for a Claude skill to combine both kinds of step in the same SKILL.md. The pattern is using two tools in parallel:

• Scripts, for deterministic work. Fetching data, deduplicating, ranking, validating: anything with a correct answer. A script runs the same way every time and barely costs any tokens to “think through.”
• Plain-text steps, for tool or MCP work. When the step is “call this MCP” or “use this tool,” it can’t be scripted. So you name the exact tool and its arguments in SKILL.md so Claude doesn’t pick a different one.

1
.claude/skills/ai-news/
2
├── SKILL.md
3
├── references/sources/      ← control 1
4
└── scripts/
5
    ├── fetch_source.py     ← takes a source file, scrapes it, emits normalized JSON
6
    ├── dedupe.py           ← groups the same story across sources
7
    ├── rank.py             ← scores and keeps the top 50
8
    └── validate.py         ← checks the shape: 50 items, dates in range, URLs present

Before, SKILL.md leaves the method open. Claude has to decide how to do everything:

1
# vague: decided again every time
2
Go through the sources and gather the news.
3
Remove duplicates and keep the 50 most
4
relevant.

After, the method is named explicitly:

1
# fixed: same path every time
2
For each source file, run:
3
  python scripts/fetch_source.py <file>
4

5
Then, on the combined results:
6
  python scripts/dedupe.py
7
  python scripts/rank.py --top 50
8
  python scripts/validate.py

A curious detail, the second version is also shorter. Being specific ends up shrinking the prompt, not inflating it.

When Claude improvises a process, it reasons through every step inside the context window, tokens spent thinking and more tokens spent doing. A script cuts that down to one line, run it and read the result. The deterministic part costs nothing to “think about” because it was already thought through once, when the skill was built.

It’s worth leaving an escape hatch. Determinism shouldn’t mean fragility. Tell the skill that if a script fails or a site has changed, Claude can improvise to recover, and must report it. Then you update the script. The script is the default path, not the only one.

Control 3: output, with assets/

An HTML file with {{SLOT}} markers, saved as assets/output-template.html. That’s the entire control 3, and it fixes something very specific: with sources and process already fixed, the report still came out different every week, an HTML table, then prose, then a bullet list. The data was stable, the shape wasn’t.

assets/ is the usual spot for files a skill includes but doesn’t reason about as prose: templates, logos, fixed images, fonts (the CSV version and the JSON schema for the deliverable live here too). What a template changes is the difference between “describe the report you want” and “fill in this exact report.”

1
.claude/skills/ai-news/
2
├── SKILL.md
3
├── references/sources/      ← control 1
4
├── scripts/                ← control 2
5
└── assets/
6
    ├── output-template.html  ← the report's skeleton, the deliverable
7
    ├── output-template.csv   ← flat export, one row per story
8
    ├── story.json             ← the schema every story must match
9
    └── brand-logo.svg        ← fixed brand asset, inserted unchanged

The template is just HTML with {{SLOT}} markers, nothing elaborate. What matters is that the structure is already decided:

1
<!-- header: do not edit -->
2
<h1>AI News: week of {{WEEK}}</h1>
3
<p class="meta">{{COUNT}} stories ·
4
  {{SOURCE_COUNT}} sources</p>
5

6
<!-- repeat this block per story -->
7
<article>
8
  <span class="tag">{{CATEGORY}}</span>
9
  <h3>{{HEADLINE}}</h3>
10
  <p>{{SUMMARY}}</p>
11
  <a href="{{URL}}">{{SOURCE}}</a>
12
</article>

In SKILL.md, step 4 reads like this:

1
## Step 4: Render the output
2

3
Load assets/output-template.html.
4

5
Fill each {{SLOT}} with the ranked stories.
6
Repeat the <article> block once per story,
7
in ranking order.
8

9
Don't change the structure, headings, classes,
10
or styling. The template IS the design, you
11
only supply the content.

The slots still contain generated text, the wording of a summary varies on every run, and that’s normal and expected. What the template fixes is everything around the text: structure, order, sections, style. The container stays fixed, and only the content inside it varies.

Control 4: speed, with agents/

The skill already does a lot, and it does it in sequence. Going through five sources one after another is five times slower than it needs to be. And every raw page piles up in the main context, which gets more expensive and less precise as it fills.

The fix is splitting independent work across subagents and running them in parallel. The main thread becomes an orchestrator, it dispatches one subagent per source, all run at once, and only their clean results come back.

A subagent runs in its own, separate context window. It does its job (scraping one source, dozens of tool calls) and returns only the result to the main thread, not the full transcript of how it got there. The total time for the step drops to the duration of the slowest source, and the main thread stays clean because the raw scraping never touches it.

1
             main thread (orchestrator)
2
                       │
3
              dispatches in parallel
4
                       │
5
   ┌─────────┬─────────┼─────────┬─────────┐
6
   ▼         ▼         ▼         ▼         ▼
7
 scraper   scraper   scraper   scraper   scraper
8
TechCrunch  Verge    arXiv      HN     Anthropic
9
   │         │         │         │         │
10
   └─────────┴─────────┴─────────┴─────────┘
11
                       │
12
                 5 JSON arrays
13
                       ▼
14
             dedupe → rank → render

1
.claude/skills/ai-news/
2
├── SKILL.md                ← the orchestrator
3
├── references/sources/      ← control 1
4
├── scripts/                ← control 2
5
├── assets/                 ← control 3
6
└── agents/
7
    └── source-scraper.md   ← worker definition, reused per source

One file defines the scraper subagent. The frontmatter fixes the model (Haiku is plenty for scraping) and the tools it can use. The body is short on purpose: the worker does one thing and returns only the result, no extra commentary:

1
---
2
name: source-scraper
3
description: Scrapes ONE ai-news source and
4
  returns normalized JSON.
5
model: haiku
6
tools: Bash, Read, WebFetch
7
---
8

9
You receive ONE source file from
10
references/sources/.
11

12
Run scripts/fetch_source.py against it.
13
Return ONLY the normalized JSON array.
14
Don't summarize, don't rank, don't comment.

The parallel dispatch, in SKILL.md:

1
## Step 2: Scrape, in parallel
2

3
Dispatch one source-scraper subagent per file
4
in references/sources/, all at once, not in
5
sequence.
6

7
Wait for all of them to finish. Each returns
8
a JSON array. Concatenate the arrays and pass
9
them to step 3 (dedupe + rank).

The two words doing the heavy lifting are “all at once.” Without them, Claude happily dispatches the workers in series, which preserves determinism but throws speed out the window.

One nuance about the agents/ folder. Subagents are a first-class concept in Claude Code, but Claude Code only registers them automatically from .claude/agents/ (or ~/.claude/agents/, or a plugin’s agents/), not from a folder nested inside a skill. So there are two valid setups. Either you put the actual subagent definition in .claude/agents/ so you can dispatch it by name, or you keep an agents/ folder inside the skill with plain instruction files, and have SKILL.md tell the orchestrator to launch a general subagent that reads that file.

The second pattern is what Anthropic’s official skill-creator uses, and it keeps the skill portable in a single directory. Just be clear that the nested file’s model or tools frontmatter doesn’t apply on its own, Claude Code doesn’t read it. When you’re sharing workers across several skills, promote them to .claude/agents/.

Control 5: quality, with tests/

A deterministic process can still publish a wrong report. A summary can assert something the source never said, a “last 7 days” story can turn out to be three weeks old. Determinism isn’t the same as correctness.

You need a verification layer that runs before the report gets published, an LLM as judge. A fresh subagent reads the output and a rubric you wrote, and decides whether each item passes. Independence is what makes the check valid, the model judging the work isn’t the one that produced it.

Some kinds of errors can only be caught by reading and reasoning. Is the claim actually backed by the source? Is the tone right? Is the date plausibly recent? A script can’t answer that, but a model with a clear rubric can. Writing the rubric down turns a subjective judgment into a repeatable check.

The pattern is one markdown file per criterion inside tests/. Each file is a rubric with pass criteria that the judge applies to every item in the output.

1
.claude/skills/ai-news/
2
├── SKILL.md
3
├── references/ scripts/ assets/ agents/   ← controls 1-4
4
└── tests/
5
    ├── accuracy-test.md    ← rubric: does every claim trace back to its source?
6
    ├── relevance-test.md   ← rubric: is the story actually about AI?
7
    ├── recency-test.md     ← rubric: is the story within the 7-day window?
8
    └── safety-test.md      ← rubric: no personal data, no unsafe or defamatory content

A content rubric:

1
# Rubric: Accuracy
2

3
Evaluate every story in the final report.
4

5
Approve it only if:
6
- every claim in the summary is backed by the
7
  linked source, open it and verify
8
- no number, name, or date appears that isn't
9
  in the source article
10
- the headline isn't editorialized beyond what
11
  the source says
12

13
For each story, return:
14
  { id, passed, evidence }
15

16
Flag failures, don't silently fix them.

The quality gate, in SKILL.md:

1
## Step 5: Quality gate (before rendering)
2

3
For EACH rubric file in tests/, dispatch a
4
separate judge subagent (fresh context, can't be
5
the agent that wrote the report). Each judge reads
6
its rubric, evaluates the output, and returns a
7
result per story: { id, passed, evidence }.
8

9
Collect all evaluations. If a story fails any
10
rubric, flag it in the report with the judge's
11
evidence, or remove it and note the gap. Never
12
silently publish a story that fails.

A model reviewing its own work in the same context is a weak check, it’s biased toward agreeing with what it just wrote. A fresh subagent, with only the rubric and the output, no memory of having produced it, gives an independent read. The subagent infrastructure from control 4 is what makes this cheap.

An LLM as judge against a rubric is the qualitative check that runs every time the skill executes. If you also want quantitative evaluations (assertion-level scoring, variance benchmarks, with-and-without-skill comparisons), Anthropic’s official skill-creator skill is built for exactly that. They’re complementary, not substitutes, the rubrics stop a wrong report from getting published today, and the benchmarks tell you whether yesterday’s change quietly lowered the hit rate.

The finished skill

Five controls, five folders, one orchestrator. The same skill that started as a single markdown file that changed its result on every run now runs fast, verified, with sources and process already decided in advance.

1
.claude/skills/ai-news/
2
├── SKILL.md                    ← the orchestrator, wires the five controls
3
├── references/                 ← CONTROL 1: input
4
│   └── sources/                ← one file per approved source
5
│       ├── techcrunch-ai.md
6
│       ├── the-verge-ai.md
7
│       ├── arxiv-cs-ai.md
8
│       ├── hacker-news.md
9
│       └── anthropic-blog.md
10
├── scripts/                    ← CONTROL 2: process
11
│   ├── fetch_source.py
12
│   ├── dedupe.py
13
│   ├── rank.py
14
│   └── validate.py
15
├── assets/                     ← CONTROL 3: output
16
│   ├── output-template.html
17
│   ├── output-template.csv
18
│   ├── story.json
19
│   └── brand-logo.svg
20
├── agents/                     ← CONTROL 4: speed
21
│   └── source-scraper.md
22
└── tests/                      ← CONTROL 5: tests (LLM as judge)
23
    ├── accuracy-test.md
24
    ├── relevance-test.md
25
    ├── recency-test.md
26
    └── safety-test.md

And the orchestrator, six steps that read like a recipe:

1
# SKILL.md: the orchestrator
2

3
Step 1  Read every file in references/sources/. Use ONLY these sources.
4
Step 2  Dispatch one source-scraper subagent per source, all in parallel.
5
        Each runs scripts/fetch_source.py and returns normalized JSON.
6
Step 3  Combine the results. Run scripts/dedupe.py, then scripts/rank.py --top 50.
7
Step 4  Run scripts/validate.py to confirm the output's shape. Stop if it fails.
8
Step 5  Dispatch one judge subagent per rubric in tests/. Flag any story that fails.
9
Step 6  Fill assets/output-template.html with the ranked, verified stories.

It’s those five decisions, applied where they were needed, nothing else added.

Once a skill has agents and reaches this point, the next step, if you have several related skills, is packaging commands, skills, and agents into a single installable, versioned plugin. That’s a packaging decision, not another control. For a standalone skill, the five controls above are the whole kit.

Which controls you actually need

Determinism is a dial, not a default. You almost never need all five controls, and over-controlling a skill that should stay flexible only makes it fragile. Add a control to fix a failure you’re seeing, not preventively.

Write the plain skill first and run it several times. What you observe changing tells you exactly which control to add next. A skill that works fine as a single markdown file doesn’t need a folder full of scaffolding.

Each control you add closes one concrete source of variation, and it stays closed on every run after that.