feat: propagate experiment assignments in agent context to workflow_call and dispatch#29762
Closed
feat: propagate experiment assignments in agent context to workflow_call and dispatch#29762
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
…all and dispatch
- Add `experiments` field to `buildAwContext()` in aw_context.cjs, sourced from
`GH_AW_EXPERIMENTS_JSON` env var (JSON string of name→variant assignments)
- Inject `aw_context` as a workflow_call trigger input (mirroring the existing
workflow_dispatch injection) via new `injectAwContextIntoWorkflowCallYAML`
- Pass `GH_AW_EXPERIMENTS_JSON` env var to the safe_outputs handler step so
buildAwContext has access to current experiment assignments
- Expose `call_workflow_aw_context` as a safe_outputs job output
- Forward `aw_context` in the `with:` block of compiler-generated call-* fan-out jobs
- Pass `GH_AW_EXPERIMENT_CONTEXT` env var to the pick-experiment step so it can
inherit parent experiment assignments via `${{ inputs.aw_context || '' }}`
- Update `pick_experiment.cjs` to read parent context from `GH_AW_EXPERIMENT_CONTEXT`:
inherited variants reuse the parent's selection without updating state counters;
local-only experiments are picked normally; invalid inherited variants fall through
- Update `generate_aw_info.cjs` to also check `GH_AW_INPUT_AW_CONTEXT` env var for
workflow_call workflows (workflow_call inputs are not in ctx.payload.inputs)
- Pass `GH_AW_INPUT_AW_CONTEXT` env var in the generate_aw_info step for workflow_call
- Add `parseParentExperimentAssignments` exported for testing
- Add tests: compiler_aw_context_test.go, updated dispatch/call/pick_experiment tests
- Recompile all lock files to include new changes
Agent-Logs-Url: https://github.com/github/gh-aw/sessions/80e7104a-6d75-4f0a-b864-98dd1bb21dc0
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
- Use Object.hasOwn() instead of Object.prototype.hasOwnProperty.call() - Log warnings for non-string experiment values in parseParentExperimentAssignments - Extract reservedCallWorkflowInputs map for centralized reserved input tracking - Update test to verify warning emission for non-string values Agent-Logs-Url: https://github.com/github/gh-aw/sessions/80e7104a-6d75-4f0a-b864-98dd1bb21dc0 Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot created this pull request from a session on behalf of
pelikhan
May 2, 2026 11:44
View session
Collaborator
|
@copilot merge main and recompile |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR ensures A/B experiment variant assignments are propagated through chained Agentic Workflows by embedding experiment assignments into aw_context and teaching called/dispatched workflows to inherit compatible parent assignments rather than re-picking.
Changes:
- Add
experiments(JSON string) toaw_contextand propagate it throughdispatch_workflowandcall_workflowflows. - Inject
aw_contextas aworkflow_callinput and forward it through compiler-generated call-* fan-out jobs. - Update experiment picking to inherit parent assignments when valid, plus add tests for inheritance + propagation.
Show a summary per file
| File | Description |
|---|---|
| pkg/workflow/compiler_yaml.go | Injects aw_context into workflow_call triggers and passes GH_AW_INPUT_AW_CONTEXT to generate_aw_info. |
| pkg/workflow/compiler_safe_outputs_steps.go | Exposes GH_AW_EXPERIMENTS_JSON to handlers so aw_context can include experiment assignments. |
| pkg/workflow/compiler_safe_outputs_job.go | Exposes call_workflow_aw_context as a safe_outputs job output. |
| pkg/workflow/compiler_safe_output_jobs.go | Forwards aw_context into uses: call-* jobs and centralizes reserved workflow_call inputs. |
| pkg/workflow/compiler_experiments.go | Passes parent aw_context into pick-experiment for inheritance via GH_AW_EXPERIMENT_CONTEXT. |
| pkg/workflow/compiler_aw_context.go | Adds workflow_call injection helper to declare aw_context as an input. |
| pkg/workflow/compiler_aw_context_test.go | Adds unit tests for workflow_call aw_context injection behavior. |
| actions/setup/js/aw_context.cjs | Adds experiments field (string) to the generated aw_context. |
| actions/setup/js/call_workflow.cjs | Adds call_workflow_aw_context output so fan-out jobs can forward context. |
| actions/setup/js/generate_aw_info.cjs | Adds workflow_call fallback env var for aw_context (GH_AW_INPUT_AW_CONTEXT). |
| actions/setup/js/pick_experiment.cjs | Adds parsing of parent experiment assignments and inheritance logic. |
| actions/setup/js/pick_experiment.test.cjs | Adds tests for parent assignment parsing and inheritance behavior. |
| actions/setup/js/dispatch_workflow.test.cjs | Verifies experiments is present in dispatched aw_context. |
| actions/setup/js/call_workflow.test.cjs | Verifies call_workflow_aw_context output and experiment propagation. |
| .github/workflows/*.lock.yml | Updates compiled workflows to include the new injected input/env/output wiring. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 19/19 changed files
- Comments generated: 2
Comment on lines
+178
to
+194
| // Look for an "inputs:" key directly inside workflow_call (at wcIndent+2 depth). | ||
| // Only the first non-empty, non-comment line after wcLineIdx matters. | ||
| inputsLineIdx := -1 | ||
| for i := wcLineIdx + 1; i < len(lines); i++ { | ||
| stripped := strings.TrimLeft(lines[i], " \t") | ||
| if stripped == "" || strings.HasPrefix(stripped, "#") { | ||
| continue | ||
| } | ||
| lineIndent := len(lines[i]) - len(stripped) | ||
| if lineIndent <= wcIndent { | ||
| break // left workflow_call block entirely | ||
| } | ||
| if strings.HasPrefix(stripped, "inputs:") { | ||
| inputsLineIdx = i | ||
| } | ||
| break // only inspect the first substantive child key | ||
| } |
Comment on lines
+51
to
+86
| { | ||
| name: "workflow_call with existing inputs – aw_context added first", | ||
| input: `on: | ||
| workflow_call: | ||
| inputs: | ||
| payload: | ||
| type: string | ||
| required: false`, | ||
| wantSubs: []string{ | ||
| AwContextInputName + ":", | ||
| "payload:", | ||
| "type: string", | ||
| }, | ||
| }, | ||
| { | ||
| name: "idempotent – aw_context not duplicated on second call", | ||
| input: `on: | ||
| workflow_call: | ||
| inputs: | ||
| aw_context: | ||
| default: "" | ||
| type: string`, | ||
| // No duplicate should be added | ||
| wantSubs: []string{AwContextInputName + ":"}, | ||
| }, | ||
| { | ||
| name: "both workflow_dispatch and workflow_call – both injected", | ||
| input: `on: | ||
| workflow_dispatch: | ||
| workflow_call:`, | ||
| wantSubs: []string{ | ||
| "workflow_dispatch:", | ||
| "workflow_call:", | ||
| }, | ||
| }, | ||
| } |
Contributor
🧪 Test Quality Sentinel ReportTest Quality Score: 86/100✅ Excellent test quality
Test Classification DetailsView all 17 test classifications
NotesTest inflation flag (informational):
Go tests — assertion messages: All Go tests — build tags: Language SupportTests analyzed:
Verdict
📖 Understanding Test ClassificationsDesign Tests (High Value) verify what the system does:
Implementation Tests (Low Value) verify how the system does it:
Goal: Shift toward tests that describe the system's behavioral contract — the promises it makes to its users and collaborators.
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
…flow chains Auto-generated by Design Decision Gate (run 25251362027). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
|
Commit pushed:
|
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Agent-Logs-Url: https://github.com/github/gh-aw/sessions/7094efcd-a959-40fd-b0fb-a0a77e6f49f1 Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Contributor
Author
Done — merged main and recompiled in e38e7ba. Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
Agent-Logs-Url: REDACTED ache/uv/0.11.8/x86_64/bash -f owner=github -f infocmp` (http block)
|
This was referenced May 2, 2026
Collaborator
|
@copilot review changes, keep things minimal. The context should be integrated in the payload, not additional args |
…te workflow_call input Agent-Logs-Url: https://github.com/github/gh-aw/sessions/8d4ba1dd-fe13-4f99-b570-8a66ed4809bd Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
…ession Agent-Logs-Url: https://github.com/github/gh-aw/sessions/8d4ba1dd-fe13-4f99-b570-8a66ed4809bd Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Contributor
Author
Done in 1b8486e. Removed Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ensures that experiment variant assignments are included in the
aw_contextwhen a workflow dispatches or calls another workflow, and that the called workflow resolves its experiments from the combination of the parent's context and its own local experiment configuration.Problem
When a workflow with experiments dispatches or calls another workflow, each workflow independently picks its own experiment variants. This means the same user interaction could experience different variants at different stages of the workflow chain, breaking A/B experiment consistency.
Changes
JavaScript (
actions/setup/js/)aw_context.cjs: Addedexperimentsfield tobuildAwContext()- a JSON string of current experiment assignments fromGH_AW_EXPERIMENTS_JSONenv var. Stored as primitive string to preserve the flat-object constraint onaw_context.call_workflow.cjs: Embedsaw_context(serialized as a JSON string) directly inside thecall_workflow_payloadJSON alongside the agent's message inputs. This propagates caller metadata and experiment assignments to called workflows without requiring a newworkflow_callinput declaration.pick_experiment.cjs: AddedparseParentExperimentAssignments()that readsGH_AW_EXPERIMENT_CONTEXTenv var to extract parent workflow's experiment assignments. For each local experiment whose name matches a parent assignment (and the variant is valid), the parent's variant is reused without updating state counters. Local-only experiments are picked normally.generate_aw_info.cjs: Added fallback toGH_AW_INPUT_AW_CONTEXTenv var forworkflow_callworkflows, sincectx.payload.inputsis not populated for workflow_call triggers.Go (
pkg/workflow/)compiler_yaml.go: PassesGH_AW_INPUT_AW_CONTEXT: ${{ fromJSON(inputs.payload || '{}').aw_context || '' }}in thegenerate_aw_infostep for workflow_call workflows, extracting the embedded aw_context from the payload.compiler_experiments.go: SetsGH_AW_EXPERIMENT_CONTEXTusing a combined expression that handles both triggers transparently:${{ inputs.aw_context || fromJSON(inputs.payload || '{}').aw_context || '' }}. Forworkflow_dispatch,inputs.aw_contextis used directly; forworkflow_call, the aw_context is extracted from the payload.compiler_safe_outputs_steps.go: AddsGH_AW_EXPERIMENTS_JSON: ${{ needs.activation.outputs.experiments || '' }}env var to the safe_outputs handler step (when experiments are configured) sobuildAwContext()can include them when building aw_context for dispatch/call.Tests
pkg/workflow/compiler_aw_context_test.go- Tests forinjectAwContextIntoOnYAML(dispatch injection only)actions/setup/js/pick_experiment.test.cjs- Tests forparseParentExperimentAssignmentsand context inheritance behavioractions/setup/js/dispatch_workflow.test.cjs- Tests forexperimentsfield in aw_contextactions/setup/js/call_workflow.test.cjs- Tests verifyingaw_contextis embedded in the call payload and that experiment assignments are propagatedBehavior
Dispatch flow: Parent picks experiments →
experimentsincluded inaw_contextsent to dispatched workflow → dispatched workflow'spick_experimentinherits parent variants for matching experiment names.Call flow: Parent picks experiments →
aw_context(with experiments) embedded insidecall_workflow_payloadJSON → compiler-generatedcall-*job passes the payload to the called workflow → called workflow'spick_experimentextracts aw_context fromfromJSON(inputs.payload).aw_contextand inherits parent variants.Combination logic: Experiments in both parent context and local spec → use parent's variant (no state update). Experiments only in local spec → pick normally (state updated). Parent experiments not in local spec → ignored.