Repository navigation
feat(flow): add paginated read-only sqlite state history - #8253
shenjm (Jiaming07) wants to merge 12 commits into
Conversation
Expose bounded history metadata (row id, method name, timestamp) for a flow UUID from SQLiteFlowPersistence without loading state payloads. Adds a module-level read-only reader for inspection tools that avoids schema and journal initialization side effects. Closes crewAIInc#8138 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Strix Security ReviewWarning This pull request has 267 commits after the last Strix review ( No security issues found. Review summaryReviewed all 8 changed files (4 docs, 3 source, 1 test). The PR introduces a bounded, paginated, read-only state-history listing for SQLiteFlowPersistence. The new SQL query uses bound parameters throughout, pagination inputs are strictly validated, and the read-only inspection path carefully avoids creating files/tables or silently truncating history. No injection, path-traversal, secret, cryptographic, or authorization issues were identified in the changed code. Updated for Reviewed by Strix |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
📝 Walkthrough
Priority: ⬇️ Low Merge Risk: 🔵 Low · up to
|
|
Lorenze Jay (@lorenzejay) Per CONTRIBUTING.md this PR should carry the llm-generated label, but I don't have permission to add it. Could you apply it? Thanks! |
Do not fall back to immutable=1 when a hot rollback journal exists, since reading the main file would surface uncommitted rows. Bound offset to the SQLite integer range so an out-of-range value raises ValueError instead of OverflowError. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ory reader Note in the docs that reading a WAL-mode database may create or leave behind -wal/-shm sidecar files, and state in the reader docstring that an immutable read requires the database to stay unchanged for the whole read with no protection against a concurrent writer. Also scope the frozen-datetime monkeypatch in the equal-timestamps test with monkeypatch.context() instead of undo(). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Resolve conflict in flow/persistence/sqlite.py: keep both list_state_history (this branch) and load_state_from_history (crewAIInc#8272). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Preview approval requiredThis pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit |
|
Review: simplify and keep the history API on the persistence class Nice work on the feature. I'd like to cut it back, and put the behavior in the class so another backend can reuse it. 1. Drop the 2. Move the read-only path into the class.
The history API then lives in one place per backend, with the same signature. 3. Remove extra helpers. 4. Scope. Keep: |
Address review feedback on crewAIInc#8253: - Drop the immutable=1 fallback; mode=ro errors now propagate. - Replace module-level read_flow_state_history with the SQLiteFlowPersistence.read_history staticmethod, which opens the database read-only without calling __init__. - Inline _HISTORY_QUERY and remove _fetch_history_uri. - Add a test that rows still in an un-checkpointed WAL are returned. - Update the flow docs in en, ar, ko and pt-BR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks for the review! Addressed all points in 02fc530:
Kept |
|
Thanks for the updates. The 1. Make def list_state_history(
self,
flow_uuid: str,
*,
limit: int = DEFAULT_HISTORY_LIMIT,
offset: int = 0,
) -> list[FlowStateHistoryRecord]:
return self.read_history(self.db_path, flow_uuid, limit=limit, offset=offset)This keeps the signature that matches the base-class 2. Remove the module-level helpers.
Net result: 3. Keep as-is: bound parameters, 4. Tests: the existing tests should pass unchanged. Please add one asserting that 5. Housekeeping (not code):
|
…ation Address review feedback on crewAIInc#8253: - list_state_history now delegates to read_history with self.db_path. - Fold _fetch_history and _validate_page into read_history so sqlite.py adds no module-level functions. - Add a test that list_state_history and read_history return identical records, and docstrings for the test helpers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks! Addressed in a5b1d4b:
|
Closes #8138
Summary
SQLiteFlowPersistenceretains every saved snapshot inflow_states, but the public API only exposed the latest one viaload_state. This PR adds a bounded, paginated history listing of snapshot metadata (row id, method name, timestamp) that never loads state payloads.FlowStateHistoryRecord(inbase.py): frozen typed record withid,method_name,timestamp.idis the storage row id, notstate.id.FlowPersistence.list_state_history(): new non-abstract base method that raisesNotImplementedErrorby default, so existing custom backends are unaffected.SQLiteFlowPersistence.list_state_history(): thin wrapper that delegates toread_history(self.db_path, ...), so there is a single implementation.SQLiteFlowPersistence.read_history(db_path, flow_uuid, *, limit, offset): the single implementation: bound parameters,ORDER BY id ASC, metadata-only SELECT, validatedlimit(1–MAX_HISTORY_LIMIT=1000) andoffset(≥0). Usable as a staticmethod by inspection tools such as the CLI. It opens the database withmode=rowithout calling__init__, so it never creates the file or tables and never changes the journal mode. Returns[]when the file, table, or flow does not exist; any other SQLite error propagates.FlowStateHistoryRecord,DEFAULT_HISTORY_LIMIT,MAX_HISTORY_LIMITfromcrewai.flow.persistence.docs/edge/{en,ar,ko,pt-BR}/concepts/flows.mdx.Writes,
load_state, pending feedback, and resume code paths are untouched (the only changed lines in existing methods are import statements).Design notes
-wal/-shmsidecars. If they cannot be created (e.g. the directory is not writable),sqlite3.OperationalErroris raised rather than falling back to animmutableread that could miss rows still in the WAL.Tests
lib/crewai/tests/test_flow_state_history.py(28 tests, no API calls): multiple flow UUIDs,list_state_historyandread_historyreturn identical records, empty histories (unknown uuid / missing table / missing file), pagination, equal timestamps ordered by id, nostate_jsonread, limit/offset validation, rows still in an un-checkpointed WAL are returned, read-only file, unwritable directory raises, naive-timestamp normalisation, base class default, and unchangedload_state/pending-feedback behaviour.Quality checks run locally:
ruff check,ruff format --check,mypy lib/crewai/src/crewai/flow/persistence/(no errors inflow/persistence), and the flow persistence test suites.🤖 Generated with Claude Code