Test Cases
Curator tests are split into a cheap drift-detection suite and an optional
end-to-end smoke run. Use them when you are changing curator code, plugin
skills, docs, or CI wiring. For normal environment diagnosis, start with
/curator:check.
Run the Suite
# Cheap checks only. Safe for local development.
bash blocks/curator/tests/run.sh
# Full path with the real-LLM, real-Docker smoke run.
bash blocks/curator/tests/run.sh --with-smokeEach test returns 0 for pass, 77 for skip, and any other code for failure.
The aggregate run.sh fails only when at least one test fails. Skipped tests
are reported but do not fail the suite.
Check Cases
| Case | What it checks | Pass condition |
|---|---|---|
01_config_schema.sh | Curator config.yaml shape and required runtime keys. | Required fields are present; meta_info.name is curator; tracked config does not contain a real GitHub token. |
02_repo_pin.sh | repos/swegen checkout and Python environment. | The repo exists, matches its configured pin when a pin is set, and swegen imports inside the configured venv. |
03_github_tokens.sh | GitHub token hydration. | Every token in GITHUB_TOKENS authenticates against GitHub /rate_limit. |
04_llm_endpoint.sh | Cross-provider LLM configuration and chat completion. | The hydrated endpoint can run a small chat completion and return content. |
05_docker_daemon.sh | Docker availability. | docker is available and docker info succeeds. |
06_harbor_smoke.sh | Known verified task fixture. | If the fixture exists, tox-dev__tox-3813 still yields NOP reward 0 and oracle reward 1. Missing fixture is skipped. |
07_create_tasks_skill.sh | Curator task-generation command surface. | The canonical /curator:create-tasks skill, compatibility adapter, and plugin metadata stay wired correctly. |
08_create_tasks_docs.sh | Public curator documentation contract. | Docs recommend the canonical command and keep the expected Pages target. |
Smoke Run
The smoke test is smoke/10_pr_demo.sh. It runs swegen create over a small
10-PR fixture and is meant for self-hosted or manually triggered validation,
not routine PR checks.
It passes when at least one verified task is produced under the smoke output
directory within the time budget. If it times out or produces no verified task,
inspect artifacts/swe_tasks/.swegen-smoke-py.log first; the tail usually
shows whether the slow step was GitHub access, Docker, validation, or the LLM
call.
Files
blocks/curator/tests/
|-- cases/
|-- smoke/
`-- run.shSee blocks/curator/tests/README.md for detailed failure recipes.