# Iteration Run Folder Architecture This doc describes how `runs/iteration-x` works now. The important mental model is: - `iteration-0` is the active mutable execution sandbox - `iteration-1`, `iteration-2`, and higher are archived older runs - when groups are enabled, the real run unit is usually `iteration-x/` Older docs and comments may still sound like every execution directly targets a new `iteration-N`. That is no longer the main runtime model. ## Current Model For normal workflow execution, the controller always resolves execution into `iteration-0`. If `runs/iteration-0` already exists: - it is moved to the next available numbered archive such as `iteration-7` - a fresh `runs/iteration-0` is created - the new execution runs in that fresh `iteration-0` So in practice: - `iteration-0` = latest working run - `iteration-N` = preserved history This is implemented in [controller_run_manager.go](../../agent_go/pkg/orchestrator/agents/workflow/step_based_workflow/controller_run_manager.go). ## Why It Works This Way The architecture is optimizing for a stable active workspace: - builder mode can always point at `iteration-0` - schedulers can always launch against `iteration-0` - shell working directories and bridge paths stay predictable - the most recent run is easy to inspect without guessing the latest number - historical runs are still preserved by moving older `iteration-0` to `iteration-N` ## Standard Execution Normal full workflow execution uses this flow: 1. Resolve run folder. 2. If `iteration-0` exists, move it to the next available archive. 3. Create a fresh `runs/iteration-0`. 4. Execute the workflow there. The main orchestration path does this in [controller.go](../../agent_go/pkg/orchestrator/agents/workflow/step_based_workflow/controller.go) and [controller_run_manager.go](../../agent_go/pkg/orchestrator/agents/workflow/step_based_workflow/controller_run_manager.go). The base run folder structure currently creates: - `runs/iteration-0/` - `runs/iteration-0/execution/` - `runs/iteration-0/execution/Downloads/` Logs and step outputs are then written under that run during execution. ## Group-Scoped Runs When variables/groups are enabled, the runtime uses nested folders under an iteration: - `runs/iteration-0//` - `runs/iteration-3//` Current behavior: - group folders are always nested under an iteration - the folder name uses sanitized `display_name` when available - otherwise it falls back to `group_id` This logic lives in [controller_batch_execution.go](../../agent_go/pkg/orchestrator/agents/workflow/step_based_workflow/controller_batch_execution.go). Examples: - `iteration-0/production` - `iteration-0/staging` - `iteration-4/manish` ## Partial Group Runs Partial group runs are a special case. If the user runs only a subset of enabled groups: - the controller reuses `iteration-0` - it does **not** back up `iteration-0` first - this preserves outputs for the other groups already present in the current latest run - cleanup happens per-group instead of rotating the whole iteration This behavior is handled in [controller.go](../../agent_go/pkg/orchestrator/agents/workflow/step_based_workflow/controller.go). This is the main exception to the simple “old `iteration-0` becomes `iteration-N`” rule. ## What The Frontend Sees The backend run-folder listing does not always expose bare iteration folders. Current listing behavior: - if an iteration has group subfolders, the API returns the group paths - if an iteration has no group subfolders, the API can return the bare iteration folder So the UI often works with: - `iteration-8/production` - `iteration-8/staging` instead of just: - `iteration-8` This behavior comes from [workflow.go](../../agent_go/cmd/server/workflow.go). ## Builder Mode Workflow builder mode is pinned to `iteration-0`. That means: - any incoming builder selection is normalized to `iteration-0` or `iteration-0/` - the builder should be thought of as operating on the latest mutable run, not on archived iterations This behavior lives in [interactive_workshop_manager.go](../../agent_go/pkg/orchestrator/agents/workflow/step_based_workflow/interactive_workshop_manager.go) and is reflected in the canvas logic in [WorkflowCanvas.tsx](../../frontend/src/components/workflow/canvas/WorkflowCanvas.tsx). ## Scheduler Behavior Schedulers also target `iteration-0`. The scheduler request path sets: - `run_mode = use_same_run` - `selected_run_folder = iteration-0` Then the controller applies the same backup-and-refresh logic for full runs. This behavior lives in [scheduler.go](../../agent_go/cmd/server/scheduler.go). ## Evaluation And Report Generation Evaluation and final report generation have their own internal sandbox behavior. They do **not** mean “evaluate directly inside the archived target iteration.” Instead: - evaluation executes in `evaluation/runs/iteration-0[/group]` - final report generation uses an internal `iteration-0`-based report-generation area and then publishes output back to the requested target run So `iteration-0` is also the internal scratch space for non-primary execution modes. See: - [evaluation_execution.go](../../agent_go/pkg/orchestrator/agents/workflow/step_based_workflow/evaluation_execution.go) ## Selected Run Folder Semantics One source of confusion is `selected_run_folder`. Current reality: - for observability and UI context, it is a real run-folder selector - for historical inspection, it can point at archived runs like `iteration-9/production` - for standard execution, the controller still resolves execution into `iteration-0` So `selected_run_folder` is not “execute exactly in any arbitrary archived iteration” in the normal workflow path. ## Practical Rule Of Thumb Use this mental model: - inspect the latest run in `iteration-0` - treat `iteration-N` as archived history - expect grouped workflows to use `iteration-x/` - expect builder, scheduler, evaluation sandboxes, and report generation to revolve around `iteration-0` ## Related Docs - [workflow_manifest_architecture.md](./workflow_manifest_architecture.md) - [workflow_monitoring.md](./workflow_monitoring.md) - [workflow_shell_working_directory.md](./workflow_shell_working_directory.md)