Skip to content

chore: sync OpenAPI contract#2

Closed
JasonLovesDoggo wants to merge 1 commit intomainfrom
contract-sync/25274156
Closed

chore: sync OpenAPI contract#2
JasonLovesDoggo wants to merge 1 commit intomainfrom
contract-sync/25274156

Conversation

@JasonLovesDoggo
Copy link
Copy Markdown
Member

Automated sync of `contract/openapi.yaml` from the monorepo.

Source commit: trylitmus/litmus@25274156c014dca5bee2a4dd236a2b947dbe6fa2

Review the diff, then merge. If the SDK has codegen (e.g. `pnpm generate`),
run it after merging to update generated types.

Source commit: trylitmus/litmus@25274156c014dca5bee2a4dd236a2b947dbe6fa2
@greptile-apps
Copy link
Copy Markdown

greptile-apps bot commented Apr 17, 2026

Greptile Summary

This PR syncs the OpenAPI contract to add the $sessionend event type (with a clear $abandon vs $sessionend disambiguation note) and introduces 8 new top-level Event fields: model, provider, input_tokens, output_tokens, total_tokens, duration_ms, ttft_ms, and cost.

  • The Python SDK currently stores model inside metadata (see Feature.generation() and tests/test_client.py:251), not as a top-level field. Once the server reads these new fields at the top level of the event object, model and all other new generation-metadata fields will be silently dropped by the current SDK. The SDK's track() method needs to be updated before (or immediately after) the server-side change goes live.

Confidence Score: 4/5

Safe to merge as a contract-only change, but the SDK must be updated before the server-side change is deployed or model/token/cost data will silently drop.

One P1 SDK-contract drift issue: model (and all other new fields) are sent via metadata by the Python SDK but are now top-level Event fields in the contract. Merging is safe as a contract-only change, but the SDK needs a follow-up update to avoid silent data loss in production.

contract/openapi.yaml is fine on its own; src/litmus/client.py needs updating to expose the new top-level fields as first-class track() parameters.

Important Files Changed

Filename Overview
contract/openapi.yaml Adds $sessionend event type and 8 new top-level Event fields (model, provider, token counts, latency, cost); the Python SDK currently sends model via metadata rather than as a top-level field, creating a contract-SDK drift.

Sequence Diagram

sequenceDiagram
    participant App
    participant LitmusClient
    participant Consumer
    participant IngestAPI

    App->>LitmusClient: generation(session_id, model_name)
    LitmusClient->>LitmusClient: build msg, model stored in metadata
    LitmusClient->>Consumer: enqueue $generation event

    App->>LitmusClient: track(event_type, metadata with token counts)
    Note over LitmusClient: New fields like input_tokens<br/>not yet first-class params
    LitmusClient->>Consumer: enqueue event with token data in metadata

    Consumer->>IngestAPI: POST /v1/events
    Note over IngestAPI: Contract expects top-level<br/>model, input_tokens, cost etc.
    IngestAPI-->>Consumer: 202 Accepted
Loading
Prompt To Fix All With AI
This is a comment left during a code review.
Path: contract/openapi.yaml
Line: 138-162

Comment:
**SDK sends `model` (and new fields) via `metadata`, not as top-level fields**

The contract now defines `model`, `provider`, `input_tokens`, `output_tokens`, `total_tokens`, `duration_ms`, `ttft_ms`, and `cost` as top-level `Event` properties, but the Python SDK never serializes them at the top level. `client.py`'s `track()` signature has no parameters for any of these fields, and `Feature.generation()` explicitly puts `model` into `metadata` — confirmed by the test at `tests/test_client.py:251` which asserts `gen_event["metadata"]["model"] == "gpt-4o"`. If the server-side is updated to read `model` (and the other new fields) from the event's top-level object, the SDK will silently fail to deliver this data. The SDK's `track()` method, `Generation.event()`, and the event-building block in the constructor all need to be updated to forward these fields at the wire level.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: contract/openapi.yaml
Line: 144-161

Comment:
**Missing `minimum: 0` constraints on numeric fields**

`input_tokens`, `output_tokens`, `total_tokens`, `duration_ms`, `ttft_ms`, and `cost` are all non-negative by definition, but the schema allows any integer/number value. Adding `minimum: 0` closes the door on accidental negative submissions that would corrupt reporting.

```suggestion
        input_tokens:
          type: integer
          minimum: 0
          description: Number of input tokens consumed.
        output_tokens:
          type: integer
          minimum: 0
          description: Number of output tokens produced.
        total_tokens:
          type: integer
          minimum: 0
          description: Total tokens (input + output).
        duration_ms:
          type: integer
          minimum: 0
          description: End-to-end generation latency in milliseconds.
        ttft_ms:
          type: integer
          minimum: 0
          description: Time to first token in milliseconds.
        cost:
          type: number
          format: double
          minimum: 0
          description: Generation cost in USD.
```

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "chore: sync OpenAPI contract from monore..." | Re-trigger Greptile

Comment thread contract/openapi.yaml
Comment on lines +138 to +162
model:
type: string
description: Model identifier (e.g. "claude-sonnet-4-20250514", "gpt-4o"). Typically set on $generation events.
provider:
type: string
description: LLM provider (e.g. "anthropic", "openai", "openrouter").
input_tokens:
type: integer
description: Number of input tokens consumed.
output_tokens:
type: integer
description: Number of output tokens produced.
total_tokens:
type: integer
description: Total tokens (input + output).
duration_ms:
type: integer
description: End-to-end generation latency in milliseconds.
ttft_ms:
type: integer
description: Time to first token in milliseconds.
cost:
type: number
format: double
description: Generation cost in USD.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 SDK sends model (and new fields) via metadata, not as top-level fields

The contract now defines model, provider, input_tokens, output_tokens, total_tokens, duration_ms, ttft_ms, and cost as top-level Event properties, but the Python SDK never serializes them at the top level. client.py's track() signature has no parameters for any of these fields, and Feature.generation() explicitly puts model into metadata — confirmed by the test at tests/test_client.py:251 which asserts gen_event["metadata"]["model"] == "gpt-4o". If the server-side is updated to read model (and the other new fields) from the event's top-level object, the SDK will silently fail to deliver this data. The SDK's track() method, Generation.event(), and the event-building block in the constructor all need to be updated to forward these fields at the wire level.

Prompt To Fix With AI
This is a comment left during a code review.
Path: contract/openapi.yaml
Line: 138-162

Comment:
**SDK sends `model` (and new fields) via `metadata`, not as top-level fields**

The contract now defines `model`, `provider`, `input_tokens`, `output_tokens`, `total_tokens`, `duration_ms`, `ttft_ms`, and `cost` as top-level `Event` properties, but the Python SDK never serializes them at the top level. `client.py`'s `track()` signature has no parameters for any of these fields, and `Feature.generation()` explicitly puts `model` into `metadata` — confirmed by the test at `tests/test_client.py:251` which asserts `gen_event["metadata"]["model"] == "gpt-4o"`. If the server-side is updated to read `model` (and the other new fields) from the event's top-level object, the SDK will silently fail to deliver this data. The SDK's `track()` method, `Generation.event()`, and the event-building block in the constructor all need to be updated to forward these fields at the wire level.

How can I resolve this? If you propose a fix, please make it concise.

Comment thread contract/openapi.yaml
Comment on lines +144 to +161
input_tokens:
type: integer
description: Number of input tokens consumed.
output_tokens:
type: integer
description: Number of output tokens produced.
total_tokens:
type: integer
description: Total tokens (input + output).
duration_ms:
type: integer
description: End-to-end generation latency in milliseconds.
ttft_ms:
type: integer
description: Time to first token in milliseconds.
cost:
type: number
format: double
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Missing minimum: 0 constraints on numeric fields

input_tokens, output_tokens, total_tokens, duration_ms, ttft_ms, and cost are all non-negative by definition, but the schema allows any integer/number value. Adding minimum: 0 closes the door on accidental negative submissions that would corrupt reporting.

Suggested change
input_tokens:
type: integer
description: Number of input tokens consumed.
output_tokens:
type: integer
description: Number of output tokens produced.
total_tokens:
type: integer
description: Total tokens (input + output).
duration_ms:
type: integer
description: End-to-end generation latency in milliseconds.
ttft_ms:
type: integer
description: Time to first token in milliseconds.
cost:
type: number
format: double
input_tokens:
type: integer
minimum: 0
description: Number of input tokens consumed.
output_tokens:
type: integer
minimum: 0
description: Number of output tokens produced.
total_tokens:
type: integer
minimum: 0
description: Total tokens (input + output).
duration_ms:
type: integer
minimum: 0
description: End-to-end generation latency in milliseconds.
ttft_ms:
type: integer
minimum: 0
description: Time to first token in milliseconds.
cost:
type: number
format: double
minimum: 0
description: Generation cost in USD.
Prompt To Fix With AI
This is a comment left during a code review.
Path: contract/openapi.yaml
Line: 144-161

Comment:
**Missing `minimum: 0` constraints on numeric fields**

`input_tokens`, `output_tokens`, `total_tokens`, `duration_ms`, `ttft_ms`, and `cost` are all non-negative by definition, but the schema allows any integer/number value. Adding `minimum: 0` closes the door on accidental negative submissions that would corrupt reporting.

```suggestion
        input_tokens:
          type: integer
          minimum: 0
          description: Number of input tokens consumed.
        output_tokens:
          type: integer
          minimum: 0
          description: Number of output tokens produced.
        total_tokens:
          type: integer
          minimum: 0
          description: Total tokens (input + output).
        duration_ms:
          type: integer
          minimum: 0
          description: End-to-end generation latency in milliseconds.
        ttft_ms:
          type: integer
          minimum: 0
          description: Time to first token in milliseconds.
        cost:
          type: number
          format: double
          minimum: 0
          description: Generation cost in USD.
```

How can I resolve this? If you propose a fix, please make it concise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant