feat(openai_agents): expose real usage, response_id, plumb previous_response_id, opt-in prompt_cache_key for stateful responses and prompt caching

[x]

Summary

Four changes to the streaming Responses API path in temporal_streaming_model.py.

My goal is to:

• Make it easier to track prompt cache hits (usage)
• Make it easier to get cache hits (prompt_cache_key)
• Make it easier to use the responses API (response_id, previous_response_id)

1. Capture real usage from ResponseCompletedEvent.response.usage and surface in the span's output dict.

   TemporalStreamingModel.get_response was constructing a zero-filled Usage and discarding event.response.usage. It now reads usage off the streaming protocol and falls back to zeros only on the error path (stream ends without a completed event).

   The len(''.join(reasoning_contents)) // 4 reasoning-tokens estimator is dropped — the real value arrives in the API response.

   The streaming_model_get_response span now carries {input_tokens, output_tokens, total_tokens, cached_input_tokens, reasoning_tokens} so traces show cache-hit rate.

2. Return real response_id captured from ResponseCompletedEvent.response.id instead of the previously fabricated f"resp_{uuid.uuid4().hex[:8]}".

   OpenAI Agents SDK reads this field for previous_response_id chaining (run.py:145), exposes it as RunResult.last_response_id (result.py:108), and writes it into traces (tracing/span_data.py:164). A client-side UUID has never been issued by any server, so any caller that picks it up and tries to chain (the documented use case) gets a 400 "response not found".

   Latent since this file was added (commit 2f2a6ed7) because nothing in the codebase chains previous_response_id yet.

3. Forward previous_response_id, conversation_id, and prompt from the SDK kwarg.

   The SDK's abstract Model.get_response has previous_response_id, conversation_id, prompt as required keyword-only params; the SDK threads previous_response_id down through _ServerConversationTracker when callers set it on Runner.run / RunConfig.

   Our implementation declared **kwargs # noqa: ARG002 and silently swallowed all three. Callers who used the official chaining API got no stateful behavior and no error. Replaced with explicit named params.

   This change is not necessarily required we need to pass previous_response_id but we could be doing that via extra_args. That said, the silent behavior is sketch. I could be convinced that the right thing to here is to raise an exception when we see them in kwargs and expect them in extra_args but this felt less dangerous and more intuitive to the caller.

4. Plumb prompt_cache_key to responses.create as an opt-in parameter.

   Callers set it via model_settings.extra_args["prompt_cache_key"]. We do not auto-inject a default. I don't trust prompt_cache_key to be standard across all OpenAI-compatible endpoints.

   When unset, the parameter resolves to NOT_GIVEN and is omitted from the request body entirely.

Test plan

All 38 tests in test_streaming_model.py pass locally:

• test_usage_captured_from_completed_event
• test_usage_falls_back_when_no_completed_event
• test_usage_emitted_in_span_output
• test_response_id_captured_from_completed_event
• test_response_id_is_none_when_no_completed_event — guards against the fake-UUID footgun
• test_prompt_cache_key_not_sent_by_default — verifies NOT_GIVEN fallback for non-OpenAI compat
• test_prompt_cache_key_forwarded_when_opted_in
• test_previous_response_id_not_sent_by_default — verifies NOT_GIVEN fallback
• test_previous_response_id_forwarded_via_sdk_kwarg — verifies the SDK's official chaining API now works
• test_conversation_id_and_prompt_accepted_but_not_forwarded — verifies SDK contract compliance without surface-area expansion
• All 28 pre-existing tests still pass (after stripping vestigial task_id kwargs)

Why these changes together

The motivating workstream (downstream of this PR) is a stateful-Responses-API migration for ST&S agents to chain via previous_response_id for 40–80% better cache utilization on reasoning models. That migration needs:

• Real Usage to measure cache hit rate before/after.
• Real response_id to actually pass back as previous_response_id.
• previous_response_id actually reaching responses.create instead of being silently swallowed.
• Optional prompt_cache_key for callers who want it without forcing it on everyone.

Once a release containing these changes lands and aimi-scale bumps its agentex-sdk pin, the runtime monkey-patch in st_s/*/agentex_usage_patch.py and the corresponding _apply_agentex_usage_patch() calls in each run_worker.py can be deleted.

Greptile Summary

This PR fixes four latent issues in TemporalStreamingModel.get_response: (1) real Usage is now read from ResponseCompletedEvent.response.usage instead of zeros, (2) the real server-issued response_id replaces a fabricated UUID, (3) previous_response_id/conversation_id/prompt are surfaced as explicit kwargs and forwarded to responses.create, and (4) prompt_cache_key is added as an opt-in via model_settings.extra_args. The changes are well-tested and the core logic is correct, though the existing review comments on prompt_cache_key SDK compatibility warrant attention before merge.

Confidence Score: 4/5

Safe to merge once the open prompt_cache_key SDK-parameter concern from the previous review thread is resolved.

The four core fixes (real usage, real response_id, forwarding SDK kwargs, opt-in cache key) are logically correct and well-tested. Score is capped at 4 because the previously raised concern about prompt_cache_key — always passing it as an explicit named kwarg when the OpenAI Python SDK may not declare it — appears unresolved and could raise a TypeError for every caller.

temporal_streaming_model.py line 625: prompt_cache_key is passed as a named kwarg regardless of whether the SDK accepts it as a declared parameter

Important Files Changed

Filename	Overview
src/agentex/lib/core/temporal/plugins/openai_agents/models/temporal_streaming_model.py	Captures real usage and response_id from ResponseCompletedEvent, plumbs previous_response_id/conversation_id/prompt as explicit kwargs, and adds opt-in prompt_cache_key. Core logic is correct; previous review comments flagged the prompt_cache_key SDK naming concern.
src/agentex/lib/core/temporal/plugins/openai_agents/tests/test_streaming_model.py	Adds 10 new targeted tests for usage capture, response_id, prompt_cache_key opt-in, and SDK kwarg forwarding. Removes vestigial task_id kwargs from existing tests. Previous review flagged a vacuously-true assertion in test_prompt_cache_key_forwarded_when_opted_in.

Sequence Diagram

sequenceDiagram
    participant SDK as OpenAI Agents SDK
    participant TSM as TemporalStreamingModel
    participant OAI as responses.create

    SDK->>TSM: get_response(...,previous_response_id,conversation_id,prompt)
    TSM->>TSM: pop prompt_cache_key from extra_args (opt-in)
    TSM->>OAI: responses.create(previous_response_id,conversation,prompt,prompt_cache_key,...)
    OAI-->>TSM: stream of events
    loop stream events
        TSM->>TSM: process delta events
        TSM->>TSM: on ResponseCompletedEvent capture usage + response.id
    end
    TSM->>TSM: Build Usage from captured_usage (fallback to zeros if None)
    TSM-->>SDK: ModelResponse(output, usage, response_id=captured_response_id)

Loading

_{Reviews (6): Last reviewed commit: "Merge branch 'main' into dpeticolas/prom..." | Re-trigger Greptile}