More Than Prompting

Originally published on jieyideng.substack.com.

What is SKILL.md?

In late 2025, Anthropic introduced the concept of SKILL.md: a standardized set of instructions that teaches an Agent how to handle specific tasks or workflows.

Think of SKILL.md not just as a prompt, but as a tool to preserve context for repeatable workflows. Whether you are generating frontend designs from specs, conducting research with a consistent methodology, or orchestrating multi-step processes, SKILL.md acts as the bridge between raw intelligence and reliable execution.

(Note: While you can apply any LLM to your SKILL.md, choosing models that have specific tuning or optimization towards SKILL use is recommended.)

How to Define Your SKILL.md

SKILL.md is fundamentally different from README.md. While a README explains software to humans, a SKILL file explains a workflow to an AI. Before structuring your file, you need to clarify four pillars:

  • User Case: What is the exact goal the user wants to accomplish?
  • Workflow Steps: What is the critical path? What multi-step sequence is required to finish the job?
  • Tools & Data: What specific datasets or external tools are necessary context?
  • Domain Knowledge: What industry best practices or “tribal knowledge” must be embedded to ensure quality?

Before writing SKILL.md, the most important step is not formatting but problem framing. You need to clarify what users are trying to accomplish, what sequence of steps is required, what tools and data are needed, and what domain knowledge should be embedded as non-negotiable guidance. This is where many teams fail. A weak SKILL.md behaves like static documentation. It sounds comprehensive, but it does not reliably improve decision quality. A strong SKILL.md behaves like an operational protocol. It defines when to activate, what to verify first, what is not allowed, and how conclusions must be structured before they are considered usable.

Once you grasp these basics, you can move from writing prompts to designing skills. Let’s look at a concrete example in Ad Analytics.

Example: The Ads Analytics SKILL.md

In Ads Analytics, teams are constantly under pressure to explain performance shifts, allocate budgets, and avoid expensive errors. The challenge isn’t whether a model can generate fluent text; the challenge is whether that text is grounded in the right objective, correct metric logic, and a valid sequence of reasoning.

In practical scenarios, logic beats fluency. A common mistake in performance analysis is metric misuse across incompatible objectives. For example, comparing an “Install-focused” campaign directly with a “Purchase-focused” campaign is a fundamental error. If objective alignment isn’t enforced early, the downstream analysis might look elegant, but it will be wrong.

Process Control over Prompt Engineering

A strong skill design prevents this by making “Objective Classification” a prerequisite, not an afterthought. It enforces a discipline most teams underestimate: Data Quality Checks before interpretation.

Many bad business decisions stem from small technical mismatches, for instance, missing fields, formatting inconsistencies, or data type errors. The fix isn’t a better prompt; the fix is process control. A good skill architecture inserts validation gates before the model attempts to generate insights.

Only after passing these gates does the model reach the interpretation phase. Here, structure beats improvisation. Instead of generic advice like optimize creatives, a skill-governed output follows a diagnostic chain:

  • Cost efficiency worsened? Isolate the cause: Is it media cost? Engagement quality? Or post-click conversion dynamics?

This chain turns the output from a narrative artifact into an execution artifact.

Laid out end to end, the division of labour becomes the whole design. The runtime does the deterministic work — validating the query before a single provider call, compiling a typed execution plan, reconciling totals, emitting diagnostic signals — and only then hands a validated evidence package to the agent for synthesis.

Sequence diagram across four participants: User, Agent, Insights Skill Runtime, and Ads Data Backend. Phase 1, Understand Intent and Form Query: the agent interprets scope, objective, date range and KPI intent, optionally clarifies missing information with the user, then submits a canonical semantic query carrying metrics, dimensions, filters, entity scope and date range. Phase 2, Validate, Plan and Execute: the runtime validates the query before any provider call, compiles a typed SemanticExecutionPlan, executes it against the selected backend, receives the provider response and source metadata, normalizes data and calculates metrics, validates completeness, coverage and totals, generates diagnostic signals and action eligibility, and returns a validated evidence package; on validation, no-data or integrity errors it returns a hint and next_action for correction and retry. Phase 3, Interpret and Recommend: the agent interprets the evidence using goal-, mode- and grain-specific guardrails, identifies performance signals, likely drivers and limitations, prioritizes eligible actions based on confidence, and returns an executive summary with findings, limitations and prioritized actions to the user. A closing note states that the skill owns deterministic calculation, validation and diagnostic signals, while the agent owns synthesis, prioritization and final communication.
The three phases, and the line between what the skill computes and what the agent is allowed to conclude.

That boundary is what the folder structure has to encode. Here is the shape I use for analytics-heavy skills:

ads-insights-skill/

├── SKILL.md                        # Required — when to activate, the phase contract, hard limits

├── workflow/                       # One file per phase above
│   ├── phase1-form-query.md        # Interpret scope, objective, date range and KPI intent;
│   │                               #   clarify with the user when anything is missing
│   ├── phase2-validate-execute.md  # Validate → plan → execute → normalise → verify
│   └── phase3-interpret.md         # Guardrailed interpretation and action prioritisation

├── contracts/                      # The typed handoffs between agent and skill
│   ├── semantic-query.md           # Metrics, dimensions, filters, entity scope, date range
│   ├── execution-plan.md           # The SemanticExecutionPlan the runtime compiles
│   └── evidence-package.md         # What comes back, including hint + next_action on failure

├── rules/                          # Hard boundaries and error prevention
│   ├── query-validation.md         # Reject invalid queries before any provider call
│   ├── integrity-checks.md         # Completeness, coverage, totals reconciliation
│   └── guardrails.md               # Goal-, mode- and grain-specific interpretation limits

├── references/                     # Domain documentation
│   ├── metrics-definition.md       # KPI formulas and interpretation standards
│   └── schema-guide.md             # Data fields and structure reference

├── templates/                      # Standardized reporting outputs
│   └── report-template.md          # Executive summary → findings → limitations → actions

└── scripts/                        # Deterministic helpers — the skill calculates, it does not narrate
    ├── validate_query.py           # Type checks and missing-field validation
    ├── execute_plan.py             # Run the compiled plan against the selected backend
    ├── normalize_metrics.py        # Normalise provider responses into comparable metrics
    └── diagnostic_signals.py       # Emit signals and action eligibility for the agent

Pitfalls: Soft vs. Hard Constraints

While the initial version of SKILL.md performed well, extensive multi-round testing has revealed some limitations. Specifically, in complex cross-campaign comparisons, the model still occasionally commits the specific mistakes outlined in our Error Prevention guidelines.

This occurs because SKILL.md functions essentially as a soft constraint for the Agent. Instructions describe what should happen; they cannot guarantee it. The issue is particularly pronounced when the underlying foundation model is not Claude.

That is why the validator in the diagram above sits outside the model rather than inside the prompt. The runtime rejects an invalid query before any provider call, reconciles completeness, coverage and totals once the data comes back, and on a validation, no-data or integrity error returns a hint and a next_action so the agent corrects and retries instead of narrating around the gap. A soft constraint asks the model to be careful. A hard constraint makes the careless path unavailable. This level of rigor is critical for commercialization; when dealing with financial data, we cannot afford any margin for error.

Summary

Advertising analytics is the example, but the principle is broader: reliable agentic analytics requires deterministic calculation and validation before model interpretation. The more capable the model becomes, the more important it is to define what the system must verify before it is allowed to conclude.

That shift moves AI from response generation to workflow execution. Done well, it shortens time to insight, reduces analytical rework and makes decision quality more consistent without giving up the problem framing and judgment that data science still requires.