Repository navigation
fix(chat): serialize zstd decompression to prevent concurrent segfault - #44
Merged
Merged
Conversation
The module-level ZstdDecompressor is shared across Flask request threads, but python-zstandard decompressors are not thread-safe. Two concurrent /api/chat/messages requests decoding an uncached shard (e.g. scrolling up in a large group chat) race on the same decompression context, producing wrong content or crashing the serve process with SIGSEGV. Guard the decompress call with a lock. Verified: - 8-thread repro calling _decode_content: exit 139 before, 3/3 clean runs after - python -m unittest discover -s tests: Ran 863 tests, OK (skipped=20) Fixes ImUpXuu#43
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
这个 PR 做了什么
给
siwx/api_chat.py里全局共享的ZstdDecompressor加锁,修复聊天翻页时两个并发请求同时解码分片导致serve进程段错误崩溃的问题。python-zstandard 的
ZstdDecompressor不是线程安全的,而 Flask 开发服务器用多线程处理请求。两个/api/chat/messages请求同时对未缓存的分片做全量解码时,会并发操作同一个解压上下文:轻则解出错误内容,重则 SIGSEGV 直接退出。详细分析、日志和复现脚本见 issue。关联 issue:Fixes #43
改动类型
改动清单
siwx/api_chat.py:新增模块级_ZCTX_LOCK = threading.Lock(),_decode_content中的_ZCTX.decompressobj().decompress(raw)改为在锁内执行。全仓只有这一处在用共享的ZstdDecompressor。验证方式
并发复现脚本(#43 中的
zstd_race.py,8 个线程并发调用真实的_decode_content,不依赖真实数据):node --check <改动的 js 文件>(不涉及前端)没有把并发压测加进单测:修复前一旦复现就是段错误,会直接带崩整个测试进程,而且压测本身也偏慢,不符合「轻量、自包含」的要求。如果需要,我可以补一个轻量版本。
兼容性检查
SIWX_NO_PLUGINS=1零插件模式下,行为与改动前一致(不涉及插件)paths.data_dir(),未依赖当前工作目录(不涉及)截图 / 日志
崩溃时
logs/crash.log的 faulthandler 栈(已打码):