feat: add StreamReservation context manager for streaming DX#39
Merged
feat: add StreamReservation context manager for streaming DX#39
Conversation
Add StreamReservation and AsyncStreamReservation context managers that automate the reserve → commit/release lifecycle for streaming use cases, reducing boilerplate from ~50 lines to ~15. - StreamUsage dataclass for accumulating tokens/cost during streaming - Auto-commit on successful exit, auto-release on exception - Heartbeat-based TTL extension for long-running streams - Commit retry via existing CommitRetryEngine - Cost resolution: explicit actual_cost > cost_fn > estimate fallback - Respects user-set ctx.metrics during streaming - Full spec validation (TTL, grace_period, subject constraints) - Handles IDEMPOTENCY_MISMATCH correctly (no release) - Client convenience: CyclesClient.stream_reservation() - 64 tests, 97% module coverage, 99.38% total coverage - Version bump: 0.2.0 → 0.3.0
- Fix weak async cost_fn test: use distinct value (1500) vs estimate (1000) so test fails if cost_fn is ignored - Fix README streaming example: add missing variable definitions (max_tokens, openai_client, stream creation) so example is runnable - Update AUDIT.md: correct test count (64), add validation and IDEMPOTENCY_MISMATCH details - Update examples/README.md: streaming_usage.py description
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
StreamReservationandAsyncStreamReservationcontext managers that automate the reserve → commit/release lifecycle for streaming use cases, reducing boilerplate from ~50 lines to ~15StreamUsagedataclass for accumulating tokens/cost during streamingCyclesClient.stream_reservation()/AsyncCyclesClient.stream_reservation()convenience factory methodsBefore (~50 lines manual boilerplate)
After (~15 lines)
Key details
usage.actual_cost>cost_fn(usage)> estimate fallbackCommitRetryEngine/AsyncCommitRetryEngineCyclesContextviaContextVar; respects user-setctx.metricsvalidate_ttl_ms,validate_grace_period_ms,validate_subject— matches lifecycle.pyRESERVATION_FINALIZED,RESERVATION_EXPIRED,IDEMPOTENCY_MISMATCHdo not trigger release — matches lifecycle.py exactlyFiles changed
runcycles/streaming.pyruncycles/client.pystream_reservation()to both clientsruncycles/__init__.pytests/test_streaming.pyexamples/streaming_usage.pyREADME.mdAUDIT.mdexamples/README.mdpyproject.tomlTest plan
pytest --cov=runcycles— 364 passed, 99.38% coverageruff check— all cleanruff format --check— all cleanmypy— no errors