03 / AOSNot every record is a lesson.
Not every record is a lesson.
Evidence comes first.
The execution loop and model development pipeline are separate processes. Moving between them requires quality checks, evaluation and an explicit decision.
Make the outcome visible.
- Task and success criterion
- Allowed action
- Fresh observation
- Outcome verification
- Correction and fresh evidence if needed
A correction does not erase the original failure. Actions, observations, verification and human intervention are preserved as distinct events.
Evaluate the candidate with evidence.
- Data preparation and redaction
- Quality review and versioned dataset
- Offline training → candidate
- Held-out evaluation
- Explicit promotion decision
- Controlled activation and rollback
No silent training during tasks or automatic activation of new models. LoRA and adapter switching are targets awaiting verification.
Four questions. Four kinds of evidence.
01
Did the tool call succeed?
02
Did the intended outcome occur?
03
Was the outcome independently verified?
04
Was the record found suitable for training?
Raw records are not automatically good training data. Usage rights, privacy, label quality, duplicates and data leakage require separate checks.
Source: docs/TRAJECTORY_DATA.md · docs/DATASET_PIPELINE.md · docs/EVALUATION_AND_PROMOTION.md 2026-09-19
Explore the development diagram ↗