Skip to content

fix(files): reset cache eviction order on clear - #9518

Merged
Vidit Ostwal (Vidit-Ostwal) merged 1 commit into
crewAIInc:mainfrom
Kiko0301:fix/upload-cache-clear-order
Nov 19, 2026
Merged

Vidit Ostwal (Vidit-Ostwal) merged 1 commit into
crewAIInc:mainfrom
Kiko0301:fix/upload-cache-clear-order

Conversation

@Kiko0301

Copy link
Copy Markdown
Contributor

Fixes #9513

Summary

UploadCache.clear() cleared the backing cache and the per-provider key sets, but left _key_access_order holding the keys that had just been removed. Because the cache's size is derived from _provider_keys, a refilled cache could reach max_entries and then have its eviction step delete stale keys that no longer existed, freeing no room and letting the new entry push the cache past the limit.

This change clears the access-order list together with the other bookkeeping, so the eviction list is as empty after clear() as it is in a freshly constructed cache.

Verification

  • The new regression test fails on main (assert 3 <= 2 after clearing a two-entry cache and inserting three entries) and passes with this change.
  • The rest of lib/crewai-files/tests/test_upload_cache.py could not be run to completion on this Windows machine: an existing test constructs an ImageFile, and libmagic crashes the interpreter here before that test finishes. The failure is unrelated to this change and occurs with the same module on main.

The test uses set_by_hash() and the in-memory backend only; no provider or network access is involved.

AI assistance

I used an AI coding agent to investigate, implement, and test this change, and reviewed its results. Per CONTRIBUTING.md, please apply the llm-generated label; external contributors cannot add it with my current permissions.

@strix-security

strix-security Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Strix Security Review

Warning

This pull request has 23 commits after the last Strix review (01f7d39). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

✅ No open security findings.

PR summary: Fixes `UploadCache.clear()` so it also resets the cache's access-order list, keeping eviction bookkeeping consistent with a freshly built cache so a refilled cache cannot grow past `max_entries`; adds a regression test for that scenario.

Review summary

This PR is a one-line bookkeeping fix in UploadCache.clear(): it now empties _key_access_order together with the backing cache and _provider_keys, plus a regression test asserting max_entries still holds after a clear. The review traced that line through the whole cache class (eviction ordering, _provider_keys maintenance, __len__, the get/set paths), its callers (cleanup_uploaded_files, FileResolver.clear_cache, reset_upload_cache, the atexit hook) and both backing-store backends, and considered whether the change could weaken eviction enforcement, cross up records between providers or callers, expose a deserialization path, or make any file-reading path reachable. No security vulnerability was confirmed: the added line only restores consistency between two tracking structures that were already keyed to the backing store, and the order list it clears feeds no authorization or isolation decision.

Ruled out (3)
  • Can a concurrent clear() and upload leave an entry uncounted and grow the cache past max_entries?
    The default in-memory backend's clear never suspends, and clear() has no route, tool or user-supplied trigger, so the only party that could interleave is the application's own code acting on its own cache.
  • Can one caller's clear() or cleanup delete other users' or tenants' cached uploads and provider-side files?
    Those behaviors predate the PR and are unchanged by it, and no route, tool or CLI path exposes clear() or cleanup_uploaded_files to untrusted input.
  • Can the added line make eviction raise an exception and drop a user's file?
    The unguarded list removal predates the PR and already races between concurrent evictions, and the failure aborts only the calling request's own cache write without touching another party's data.

Review 1 on this PR · 01f7d39 · requested Nov 17 12:42 UTC when the PR was opened


Reviewed by Strix
Re-run review · Configure security review settings · Try a deeper review with @strix-security /deep

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct. Before this, aclear() left the old keys in _key_access_order, so the next _evict_oldest picked those stale keys first. Deleting them was a no-op and len(cache) didn't go down, so the cache grew past max_entries. The new test walks through exactly that.

I checked the other removal paths: aremove, aremove_by_file_id and aclear_expired all go through _untrack_key, and _evict_oldest removes from the list itself, so aclear was the only one that skipped the list.

Small test nit: assert len(cache) <= 2 would also pass if eviction removed too much. Checking that new-0 is gone and new-1/new-2 are still there would also pin down which entry gets evicted.

Reviewed with help from Claude Code.

UploadCache.clear() cleared the backing cache and the per-provider key
sets but left the shared access-order list populated. A later set then
spent its eviction step deleting stale keys that were already gone, so
the cache could grow past max_entries after a clear.

Clear the access-order list with the other bookkeeping and cover reuse
after clear with a regression test.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this is a clean fix. aclear() was the only removal path that skipped _key_access_order, and upload_cache.py:378 now resets it, so a cleared cache matches a fresh one.

Verified: the issue's repro fails on base (len 3 with max_entries=2) and passes on this branch, along with your new test and the existing clear/eviction tests.

Optional nit: asserting that new-0 is evicted and new-1/new-2 remain would make the test stricter than len(cache) <= 2.

@Vidit-Ostwal
Vidit Ostwal (Vidit-Ostwal) merged commit 7af028b into crewAIInc:main Nov 19, 2026
54 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] UploadCache.clear() leaves eviction bookkeeping behind, so max_entries stops being enforced

3 participants