Prepare Tasks

Tracer supports a local curator source and a compatible Hugging Face source. Task preparation decides what is staged; the processed-task ledger and manual exclusion list decide what Harbor skips.

Manifest-filtered copy

scripts/prepare_tasks.sh copies task directories from the configured source into artifacts/tasks/<dataset>/:

bash scripts/prepare_tasks.sh

For a local source with verifiable_tasks.txt, only listed task IDs are copied.
For a local source without the manifest, the current compatibility behavior copies every structurally valid task directory and reports the fallback.
For provider: huggingface, the dataset snapshot is downloaded under artifacts/tasks/.cache/; archives are extracted if needed and a compatible Harbor task root is staged.
The copy is idempotent: it skips when the target already holds valid Harbor task dirs. Pass --overwrite to rebuild.

The task source is configured in config.yaml under runtime_info.input.task_source and is normally wired from curator's output via meta_info.dependencies.

The processed-tasks ledger

artifacts/processed_tasks.yaml is the source of truth for which task IDs have already been processed. Each entry has a status:

Status	Meaning
`pending`	Eligible, not yet run
`running`	Currently being rolled out
`done`	Completed; trajectory captured
`failed`	Errored out (timeout/OOM); excluded from future runs
`skipped`	Intentionally not run; excluded

After each job, the /tracer:run procedure inspects results and appends or updates one entry per task, including timestamps, trajectory path, reward, and an operator note.

The exclude list

runtime_info.input.env_extra.HARBOR_EXCLUDE_TASKS names exclusion sources, not task ids. In production it is one line naming two files:

env_extra:
  HARBOR_EXCLUDE_TASKS: excluded_tasks.txt artifacts/processed_tasks.yaml

Source	Holds	Tracked in git
`excluded_tasks.txt`	tasks a human retired for good (chronic timeout, OOM)	yes — the decision must survive a fresh clone
`artifacts/processed_tasks.yaml`	run history: what this block already consumed	no — `artifacts/` is gitignored runtime state

At launch start.sh hands the field to scripts/resolve_exclude_tasks.py, which expands each whitespace-separated token:

Token	Expands to
a YAML ledger path	every terminal entry (`done \| failed \| skipped`)
a plain-text file path	one task id per line (`#` comments ignored)
anything else	a literal task id

so an operator can still pin a one-off id inline without touching the ledger. Smoke runs opt out of the ledger-derived half with HARBOR_LEDGER_EXCLUDE=0; literal ids and text lists survive that, because those are decisions rather than run history.

To retire a task permanently, append its id to excluded_tasks.txt instead of growing config.yaml.

local/HF source ─▶ prepare_tasks.sh ─▶ artifacts/tasks/<dataset>
                                          │
                                          ▼
                                Harbor job selection
                                   ▲             ▲
                                   │             │
                     processed_tasks.yaml   excluded_tasks.txt
                        (run history)        (human decision)

The manifest gates verified local inputs when present. Exactly-once protection depends on completing the post-run ledger bookkeeping; start.sh then enforces the recorded terminal state automatically on the next launch.

Prepare Tasks

Manifest-filtered copy

The processed-tasks ledger

The exclude list

On this page