Code · v1

Produce one shell command that meets four constraints

Delete regular files older than 30 days under a given path, leaving directories intact, in a single command. Each constraint is scored separately, because a command that satisfies three of four is not partially right — it is a command that deletes the wrong things.

Results

Not yet run

No model has been run against this probe yet, so there are no results to show. The definition, rubric and scoring below are complete and final for v1 — this page is published now so that the test is on the record before any score exists, rather than appearing alongside one.

This page carries no Dataset structured data until it has real runs, for the same reason it shows no numbers.

Prompt

Exactly what every model receives. Nothing else is sent.

system

You are a shell expert. Reply with a single command and nothing else. No explanation, no code fence, no preamble.

user

Write one shell command that deletes every regular file under /var/log which was last
modified more than 30 days ago. Directories must not be deleted. Do not use a loop or
multiple commands joined by ; or &&.

Reply with the command only.

Rubric

Published so you can disagree with it. A score you cannot argue with is a rumour.

Five properties, scored independently: uses find, targets /var/log, restricts to regular files, filters on a modification age greater than 30 days, and does not fall back to a recursive rm. The negative rule is weighted the same as the others but matters most — it is the answer that removes the directories the task said to keep.

Uses find rather than a loop or a bare rm
Targets /var/log
Restricts to regular files with -type f
Filters on -mtime +30
Does not use a recursive rm anywhere in the command

Scoring

Scored by regex rules. Deterministic — the same output always produces the same score.

Rules, in order

uses_find
  must match  /\bfind\b/
  The task asks for one command; find is the tool that expresses all four constraints.

targets_the_path
  must match  //var/log(?![A-Za-z0-9_])/
  A command that operates on the wrong path is not partially correct.

regular_files_only
  must match  /-type\s+f\b/
  Without -type f the command also matches directories, which the task forbids.

filters_by_age
  must match  /-mtime\s*\+\s*30\b/
  -mtime 30 means exactly 30 days, not more than 30. The plus sign is the whole predicate.

no_recursive_rm
  must NOT match  /rm\s+(-[a-zA-Z]*[rR][a-zA-Z]*\s|--recursive)/
  A recursive rm removes the directory tree the task explicitly said to preserve, and it is the fallback a model reaches for when it does not know find's -delete.

Worked examples

Hand-written outputs the rubric is tested against on every build.

A rubric can fail in two directions that reading it will not reveal: it accepts everything, so every model scores 1 and the probe measures nothing; or it rejects everything, so every model looks bad at a task that is fine. These fixtures are run through the real scorer by npm run probes:check and by the test suite. The correct answer must score 1, and every wrong answer must not.

Correct — must score 1.00

find /var/log -type f -mtime +30 -delete

Rejected — exactly 30 days rather than more than 30 — the subtle one

find /var/log -type f -mtime 30 -delete

Rejected — omitted -type f, so directories match too

find /var/log -mtime +30 -delete

Rejected — fell back to a recursive rm

find /var/log -mtime +30 -exec rm -rf {} +

Rejected — right idea, wrong path

find /var/logs -type f -mtime +30 -delete

Rejected — answered with prose instead of a command

You can use the find command with -mtime to locate files older than 30 days and delete them.

Version history

A probe version is immutable. Changing a prompt or a rubric creates the next version; existing runs stay attached to the one that produced them.

v1 · current: Initial version.

Parameters

maxTokens: 1200
temperature: 0

Run it yourself

The probe exactly as it stands at v1. Swap the model slug for any model you want to compare — the API key is a shell variable, never a value.

curl https://openrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $OPENROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "anthropic/claude-opus-5",
  "messages": [
    {
      "role": "system",
      "content": "You are a shell expert. Reply with a single command and nothing else. No explanation, no code fence, no preamble."
    },
    {
      "role": "user",
      "content": "Write one shell command that deletes every regular file under /var/log which was last\nmodified more than 30 days ago. Directories must not be deleted. Do not use a loop or\nmultiple commands joined by ; or &&.\n\nReply with the command only."
    }
  ],
  "max_tokens": 1200,
  "temperature": 0
}'