LLM Ground

Instruction following · v1

Answer in prose when the content wants to be a list

A question whose natural answer is a bulleted list, asked with lists explicitly forbidden. Tests a negative constraint against the model's strongest formatting habit — and whether it holds for the whole response rather than the first paragraph.

Results

Not yet run

No model has been run against this probe yet, so there are no results to show. The definition, rubric and scoring below are complete and final for v1 — this page is published now so that the test is on the record before any score exists, rather than appearing alongside one.

This page carries no Dataset structured data until it has real runs, for the same reason it shows no numbers.

Prompt

Exactly what every model receives. Nothing else is sent.

system

You write in continuous prose. You never use bullet points, numbered lists, headings, or markdown formatting of any kind, regardless of how naturally the content would suit them.

user

Explain what a database index is, why it makes reads faster, and what it costs.

Write in continuous prose only. Do not use bullet points, numbered lists, headings,
bold, italics, or any markdown formatting. Two paragraphs maximum.

Rubric

Published so you can disagree with it. A score you cannot argue with is a rumour.

Four prohibited constructs, each checked separately, plus one positive check that the question was actually answered. Line-anchored patterns are used for the list markers so an ordinary hyphen mid-sentence is not mistaken for a bullet — a check that punishes normal punctuation would be worse than no check.

  • No bullet markers at the start of any line
  • No numbered list items
  • No markdown headings
  • No bold or italic markup
  • The question is genuinely answered, not evaded to satisfy the constraint

Scoring

Scored by regex rules. Deterministic — the same output always produces the same score.

Rules, in order

no_bullet_markers
  must NOT match  /^\s*[-*•]\s+\S/m
  Anchored to the start of a line, so a hyphen inside a sentence does not count. Only an actual list item fails this.

no_numbered_list
  must NOT match  /^\s*\d+[.)]\s+\S/m
  Switching from hyphens to numbers is the most common way this constraint gets half-followed.

no_headings
  must NOT match  /^\s*#{1,6}\s+\S/m
  Headings are markdown formatting, which the instruction excludes explicitly.

no_bold_or_italic_markup
  must NOT match  /\*\*[^*\n]+\*\*|__[^_\n]+__/
  Emphasis tends to survive when list formatting is dropped, because it does not look like a list.

actually_answered_the_question
  must match  /(index|indexes|indices)[\s\S]{0,400}(write|insert|update|storage|space|cost)/i
  Both halves of the question must be present. An answer that satisfies every prohibition by saying almost nothing has not complied, it has evaded.

Worked examples

Hand-written outputs the rubric is tested against on every build.

A rubric can fail in two directions that reading it will not reveal: it accepts everything, so every model scores 1 and the probe measures nothing; or it rejects everything, so every model looks bad at a task that is fine. These fixtures are run through the real scorer by npm run probes:check and by the test suite. The correct answer must score 1, and every wrong answer must not.

Correct — must score 1.00

A database index is a separate data structure, usually a B-tree, that stores the values of
one or more columns in sorted order alongside pointers back to the rows they came from.
When a query filters or sorts on those columns, the engine can walk the index instead of
reading every row in the table, which turns a scan proportional to the size of the table
into a lookup proportional to the depth of the tree.

That speed is bought with space and with write cost. The index is a real structure on disk,
so it consumes storage, and every insert, update or delete that touches an indexed column
has to modify the index as well as the table. On a write-heavy table with several indexes
the accumulated cost can outweigh the read benefit, which is why indexes are added in
response to measured query patterns rather than added everywhere by default.

Rejected — reverted to bullets for the second half — the common partial failure

A database index is a sorted structure that lets the engine find rows without scanning
the whole table.

The costs are:
- Extra disk space for the index itself
- Slower inserts and updates
- Maintenance overhead during bulk loads

Rejected — swapped hyphens for numbers

An index speeds up reads by keeping column values in sorted order.

1. It consumes additional storage.
2. It slows down writes.
3. It must be maintained as data changes.

Rejected — prose, but with headings

## What an index is

A sorted structure that lets the engine find rows without scanning the table.

## What it costs

Storage, and slower writes on every indexed column.

Rejected — prose with bold emphasis

A database index is a **sorted data structure** that makes lookups fast by avoiding a full
table scan. The cost is **storage** and **slower writes**, since every insert has to update
the index as well as the table.

Rejected — obeyed the format by saying nothing — the degenerate win

A database index is a feature of relational databases that helps with performance. It is
generally a good idea to use them where appropriate, depending on your workload.

Version history

A probe version is immutable. Changing a prompt or a rubric creates the next version; existing runs stay attached to the one that produced them.

v1 · current
Initial version.

Parameters

maxTokens
1600
temperature
0

Run it yourself

The probe exactly as it stands at v1. Swap the model slug for any model you want to compare — the API key is a shell variable, never a value.

curl https://openrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $OPENROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "anthropic/claude-opus-5",
  "messages": [
    {
      "role": "system",
      "content": "You write in continuous prose. You never use bullet points, numbered lists, headings, or markdown formatting of any kind, regardless of how naturally the content would suit them."
    },
    {
      "role": "user",
      "content": "Explain what a database index is, why it makes reads faster, and what it costs.\n\nWrite in continuous prose only. Do not use bullet points, numbered lists, headings,\nbold, italics, or any markdown formatting. Two paragraphs maximum."
    }
  ],
  "max_tokens": 1600,
  "temperature": 0
}'