You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[BUG] Translated OpenAI streams keep the first nonzero usage counters #4263
The translated OpenAI streaming path keeps the first nonzero usage count, even when the provider sends a later cumulative snapshot.
I reproduced this through the actual proxy and installed AnyLLM/OpenAI SDK, using a loopback OpenAI-compatible streaming server. It sends these usage snapshots on successive content chunks:
On main fbf7bc3278c11975299b9bd0a2ecf200dfd25a05, /stats records 10 input / 1 output token, and /metrics records 2 cache-read tokens. The client receives both frames intact, so this is accounting after delivery, not a missing upstream event.
The fix records the final 13 / 5 / 9 counts. Usage frames update only fields they actually contain; an absent field leaves the last known value alone, while an explicit zero replaces it. The stateful SSE parser already handles completion-only frames, so the redundant single-chunk parser and its implementation-only tests are removed.
The eight replacement regressions exercise progressive, partial, zero, and absent usage through stream finalization, in Chat and Responses shapes. Six fail before the fix; all eight pass after it. The related streaming/outcome suites pass 71 tests; the shared parser's malformed-event, billed-input, cache-TTL and handler-helper suites pass another 469. The same real HTTP replay reports the corrected counts after the patch.
I'll keep the branch on the fork until I'm below the ten-open-PR limit.
Activity