Instruction following · v1
Answer in prose when the content wants to be a list
A question whose natural answer is a bulleted list, asked with lists explicitly forbidden. Tests a negative constraint against the model's strongest formatting habit — and whether it holds for the whole response rather than the first paragraph.
Results
Not yet run
No model has been run against this probe yet, so there are no results to show. The definition, rubric and scoring below are complete and final for v1 — this page is published now so that the test is on the record before any score exists, rather than appearing alongside one.
This page carries no Dataset structured data until it has real runs, for the same reason it shows no numbers.
Prompt
Exactly what every model receives. Nothing else is sent.
system
You write in continuous prose. You never use bullet points, numbered lists, headings, or markdown formatting of any kind, regardless of how naturally the content would suit them.
user
Explain what a database index is, why it makes reads faster, and what it costs. Write in continuous prose only. Do not use bullet points, numbered lists, headings, bold, italics, or any markdown formatting. Two paragraphs maximum.
Rubric
Published so you can disagree with it. A score you cannot argue with is a rumour.
Four prohibited constructs, each checked separately, plus one positive check that the question was actually answered. Line-anchored patterns are used for the list markers so an ordinary hyphen mid-sentence is not mistaken for a bullet — a check that punishes normal punctuation would be worse than no check.
- No bullet markers at the start of any line
- No numbered list items
- No markdown headings
- No bold or italic markup
- The question is genuinely answered, not evaded to satisfy the constraint
Scoring
Scored by regex rules. Deterministic — the same output always produces the same score.
Rules, in order
no_bullet_markers
must NOT match /^\s*[-*•]\s+\S/m
Anchored to the start of a line, so a hyphen inside a sentence does not count. Only an actual list item fails this.
no_numbered_list
must NOT match /^\s*\d+[.)]\s+\S/m
Switching from hyphens to numbers is the most common way this constraint gets half-followed.
no_headings
must NOT match /^\s*#{1,6}\s+\S/m
Headings are markdown formatting, which the instruction excludes explicitly.
no_bold_or_italic_markup
must NOT match /\*\*[^*\n]+\*\*|__[^_\n]+__/
Emphasis tends to survive when list formatting is dropped, because it does not look like a list.
actually_answered_the_question
must match /(index|indexes|indices)[\s\S]{0,400}(write|insert|update|storage|space|cost)/i
Both halves of the question must be present. An answer that satisfies every prohibition by saying almost nothing has not complied, it has evaded.Worked examples
Hand-written outputs the rubric is tested against on every build.
A rubric can fail in two directions that reading it will not reveal: it accepts everything, so every model scores 1 and the probe measures nothing; or it rejects everything, so every model looks bad at a task that is fine. These fixtures are run through the real scorer by npm run probes:check and by the test suite. The correct answer must score 1, and every wrong answer must not.
Correct — must score 1.00
A database index is a separate data structure, usually a B-tree, that stores the values of one or more columns in sorted order alongside pointers back to the rows they came from. When a query filters or sorts on those columns, the engine can walk the index instead of reading every row in the table, which turns a scan proportional to the size of the table into a lookup proportional to the depth of the tree. That speed is bought with space and with write cost. The index is a real structure on disk, so it consumes storage, and every insert, update or delete that touches an indexed column has to modify the index as well as the table. On a write-heavy table with several indexes the accumulated cost can outweigh the read benefit, which is why indexes are added in response to measured query patterns rather than added everywhere by default.
Rejected — reverted to bullets for the second half — the common partial failure
A database index is a sorted structure that lets the engine find rows without scanning the whole table. The costs are: - Extra disk space for the index itself - Slower inserts and updates - Maintenance overhead during bulk loads
Rejected — swapped hyphens for numbers
An index speeds up reads by keeping column values in sorted order. 1. It consumes additional storage. 2. It slows down writes. 3. It must be maintained as data changes.
Rejected — prose, but with headings
## What an index is A sorted structure that lets the engine find rows without scanning the table. ## What it costs Storage, and slower writes on every indexed column.
Rejected — prose with bold emphasis
A database index is a **sorted data structure** that makes lookups fast by avoiding a full table scan. The cost is **storage** and **slower writes**, since every insert has to update the index as well as the table.
Rejected — obeyed the format by saying nothing — the degenerate win
A database index is a feature of relational databases that helps with performance. It is generally a good idea to use them where appropriate, depending on your workload.
Version history
A probe version is immutable. Changing a prompt or a rubric creates the next version; existing runs stay attached to the one that produced them.
- v1 · current
- Initial version.
Parameters
- maxTokens
- 1600
- temperature
- 0
Run it yourself
The probe exactly as it stands at v1. Swap the model slug for any model you want to compare — the API key is a shell variable, never a value.
curl https://openrouter.ai/api/v1/chat/completions \
-H "Authorization: Bearer $OPENROUTER_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "anthropic/claude-opus-5",
"messages": [
{
"role": "system",
"content": "You write in continuous prose. You never use bullet points, numbered lists, headings, or markdown formatting of any kind, regardless of how naturally the content would suit them."
},
{
"role": "user",
"content": "Explain what a database index is, why it makes reads faster, and what it costs.\n\nWrite in continuous prose only. Do not use bullet points, numbered lists, headings,\nbold, italics, or any markdown formatting. Two paragraphs maximum."
}
],
"max_tokens": 1600,
"temperature": 0
}'