Skip to content

C unpack: optional compile-time array/map element limits - #1220

Open
aravindor wants to merge 1 commit into
msgpack:c_masterfrom
aravindor:harden/unpack-array-map-limit
Open

aravindor wants to merge 1 commit into
msgpack:c_masterfrom
aravindor:harden/unpack-array-map-limit

Conversation

@aravindor

Copy link
Copy Markdown

msgpack-c PR draft (array/map unpack element limits)

Target: msgpack/msgpack-c, branch off c_master. Addresses issue #1013.
One commit, one file (src/unpack.c), +32 / -0.

Commit message

C unpack: optional compile-time array/map element limits

A map32/array32 header declares a 32-bit element count that is passed
straight to the backing-store allocation (n * sizeof(msgpack_object[_kv]))
before any element byte is read. A 5-byte input such as 0xdf 0xff 0x00
0x00 0x00 (map32, count 0xff000000) therefore requests ~205 GB up front,
making a resource-exhaustion attack very cheap (issue #1013). The two
sites are the ones already marked /* FIXME security guard */.

Add two compile-time limits, MSGPACK_UNPACK_ARRAY_LIMIT and
MSGPACK_UNPACK_MAP_LIMIT, checked against the declared count before the
allocation. Both default to 0 (no limit): with the default the checks
compile out entirely, so existing behavior is preserved byte for byte.
Defining either to a non-zero value rejects an over-limit array/map with
MSGPACK_UNPACK_PARSE_ERROR.

Only arrays and maps are guarded: str/bin/ext point into the parse buffer
(zero-copy), so their length is already bounded by the bytes present and
is not an up-front amplification.

PR description

Follow-up to #1013 for the C library.

Problem. In template_callback_array / template_callback_map
(src/unpack.c), the element count n comes straight from the wire
(array32/map32 carry a full 32-bit count) and is used to size the
zone allocation n * sizeof(...) before a single element is parsed. A
5-byte message can thus force a ~100 GB (array) / ~205 GB (map)
allocation. These are the two lines already flagged /* FIXME security guard */.

Approach. Per the guidance in #1013 (preserve current behavior,
selectable by a default-disabled preprocessor macro), this adds:

#ifndef MSGPACK_UNPACK_ARRAY_LIMIT
#define MSGPACK_UNPACK_ARRAY_LIMIT 0
#endif
#ifndef MSGPACK_UNPACK_MAP_LIMIT
#define MSGPACK_UNPACK_MAP_LIMIT 0
#endif

and, right after the existing 32-bit overflow guard in each callback:

#if MSGPACK_UNPACK_ARRAY_LIMIT
    if (n > MSGPACK_UNPACK_ARRAY_LIMIT)
        return MSGPACK_UNPACK_PARSE_ERROR;
#endif

With the default 0 the #if is false and the check is not compiled in,
so behavior is unchanged. A build that opts in, e.g.
-DMSGPACK_UNPACK_MAP_LIMIT=1000000, rejects any larger array/map with
MSGPACK_UNPACK_PARSE_ERROR. This mirrors the intent of the C++
binding's element-count limit, which the C library currently lacks.

Only arrays and maps are limited; str/bin/ext are zero-copy
references into the parse buffer, so their size is already bounded by the
input and does not allocate up front.

Testing. Built the library both ways (Debug). With limits unset, a
benign map unpacks to SUCCESS and a map32 count=2000 header returns
CONTINUE as before. With -DMSGPACK_UNPACK_ARRAY_LIMIT=1000 -DMSGPACK_UNPACK_MAP_LIMIT=1000, the benign map still unpacks to
SUCCESS, map32 count=2000 returns PARSE_ERROR, and the #1013 input
df ff 00 00 00 returns PARSE_ERROR with no large allocation.

Happy to add a CI job building the test suite with the limits set, or to
switch the rejection code to MSGPACK_UNPACK_NOMEM_ERROR if that reads
better alongside the existing overflow guard.

Before you post

  • Branch from c_master (not cpp_master).
  • Check whether the project wants a DCO Signed-off-by: line (add -s on
    commit if so). msgpack-c has no CLA bot.
  • One open PR per project (CLAUDE.md). This is your first on msgpack-c.

A map32/array32 header declares a 32-bit element count that is passed
straight to the backing-store allocation (n * sizeof(msgpack_object[_kv]))
before any element byte is read. A 5-byte input such as 0xdf 0xff 0x00
0x00 0x00 (map32, count 0xff000000) therefore requests ~205 GB up front,
making a resource-exhaustion attack very cheap (issue msgpack#1013). The two
sites are the ones already marked /* FIXME security guard */.

Add two compile-time limits, MSGPACK_UNPACK_ARRAY_LIMIT and
MSGPACK_UNPACK_MAP_LIMIT, checked against the declared count before the
allocation. Both default to 0 (no limit): with the default the checks
compile out entirely, so existing behavior is preserved byte for byte.
Defining either to a non-zero value rejects an over-limit array/map with
MSGPACK_UNPACK_PARSE_ERROR.

Only arrays and maps are guarded: str/bin/ext point into the parse buffer
(zero-copy), so their length is already bounded by the bytes present and
is not an up-front amplification.
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.66%. Comparing base (eabfe29) to head (695a371).
⚠️ Report is 21 commits behind head on c_master.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@             Coverage Diff              @@
##           c_master    #1220      +/-   ##
============================================
+ Coverage     57.47%   62.66%   +5.18%     
============================================
  Files            10       10              
  Lines          1124     1149      +25     
  Branches        129      141      +12     
============================================
+ Hits            646      720      +74     
+ Misses          478      429      -49     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants