Skip to content

SemanticCache.put spins forever when max_entries is zero or negative #4282

Open

Description

@oxura

A nonpositive SemanticCacheConfig.max_entries traps put() in its eviction loop. Once the map is empty, _evict_oldest() does nothing and len(cache) >= max_entries stays true. This can pin a request worker indefinitely. The same problem occurs when the public config object is changed after entries have been stored.

I reproduced both 0 and -1 in bounded Python subprocesses; neither store returned within two seconds. The permanent regression for changing the capacity to 0 timed out after 30 seconds before the fix.

from headroom.cache import SemanticCache
from headroom.cache.semantic import SemanticCacheConfig

cache = SemanticCache(SemanticCacheConfig(max_entries=0))
cache.put("query", "answer", messages_hash="request")  # does not return on main

The fix rejects a nonpositive capacity with ValueError before mutating the cache. It does not introduce a new meaning for zero or a silent no-cache mode. A public SemanticCacheLayer smoke check now rejects both values, preserves the previous response, and resumes normal stores after a positive capacity is restored. The semantic/client integration profile passes 40 tests.

Fix and regression coverage: oxura@5a28bb8

The normal commit/pre-push hooks passed, including a native rebuild, 1658 Rust tests and 177 critical Python tests (four skips). The 10-PR author cap is still full, so the fix is published as this exact fork commit rather than an eleventh PR.

Activity

  1. rominparekh commented on Oct 19, 2026

    @rominparekh
    Contributor

    I am preparing a focused upstream PR for this: validate the capacity at the start of put(), before cleanup or eviction, with regressions for initial invalid values and runtime configuration changes (including existing-key updates). I will independently reproduce the bounded hang and exercise the public SemanticCacheLayer API, and credit the diagnosis and published fork fix linked above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions