Skip to content

Harden the decoder against malformed EXI input - #103

Open
prandogabriel wants to merge 1 commit into
ekrich:mainfrom
prandogabriel:fix/untrusted-input
Open

prandogabriel wants to merge 1 commit into
ekrich:mainfrom
prandogabriel:fix/untrusted-input

Conversation

@prandogabriel

Copy link
Copy Markdown

Problem

Malformed EXI input can crash the process (segfault or abort) instead of making the parser return an error. A service that decodes EXI from the network can be taken down with a single request. I found this while using EXIP to decode IEEE 2030.5 (SEP2) messages, and reproduced every case with the fixtures already in tests/test-set.

There are five root causes:

  1. Out-of-bounds reads in the value tables (decodeStringValue): the local (vxTable) and global value partition indexes read from the stream were not checked against the table sizes.
  2. Asserts reachable from input: content after xsi:nil="true" (SE(*), SE(qname) and CH) and a QName value outside AT(xsi:type) hit assert() and abort.
  3. realloc() of static memory: when a stream adds string table entries beyond the capacity of a static grammar (for example the EXI options grammar ops_schema while it decodes header options), addEmptyDynEntry calls EXIP_REALLOC on a static array.
  4. Use-after-free across streams: SE(*) creates a built-in element grammar and stores its index in LnEntry.elemGrammar of a schema local name. freeAllMem frees the built-in grammars at the end of the stream, but the index stays in the schema. The next stream that uses the same schema follows the freed grammar.
  5. Value outside any element: a CH event with no current element decodes a value with an empty QName (INDEX_MAX) and indexes the string table with it.

Changes

  • decodeStringValue: reject a QName that is not in the string table, empty value tables and out-of-range local and global indexes with EXIP_INVALID_EXI_INPUT.
  • decodeEventContent / decodeValueItem: the input-reachable asserts now return EXIP_INVALID_EXI_INPUT.
  • DynArray: new heapAllocated member. createDynArray and the first allocation in addEmptyDynEntry set it. Static grammars (exipg output, staticEXIOptions.c, staticXmlSchema.c) leave it zero because they use positional initializers with inner braces. When a static array has to grow, it is copied to the heap instead of passed to realloc. destroyDynArray only frees heap arrays. A failed realloc no longer overwrites the base pointer.
  • freeAllMem: before freeing the built-in grammars, reset to INDEX_MAX every elemGrammar that points to one of them.

ABI note

Adding a member to DynArray (include/procTypes.h) changes the size of the public structs that embed it. Sources that are compiled together, including generated static grammars, are not affected. Prebuilt objects compiled against the old header would need a rebuild. If you prefer to avoid that, I can move the flag elsewhere. Let me know which you'd rather have.

Tests

New "Malformed input" test case in tests/check_exip.c:

  • test_malformed_schemaless: xsWhitespace/whitespace-instance.xml.exi with every single-bit flip, every truncation and 50,000 seeded random corruptions.
  • test_malformed_schema_informed: the same for testStates/acceptance_a_01.exi and simpleContentExt/test.xml.exi with their schemas. All streams reuse one schema object, and the valid file must still decode afterwards.
  • test_malformed_header_options: an EXI options header that grows the string table of the static options grammar.

On main the test binary aborts (Abort trap: 6). With this change make TARGET=pc clean all check examples utils passes on macOS (clang), with the exip suite going from 10 to 13 checks, and the patch adds no new compiler warnings. I also checked that each test catches its fix: without the freeAllMem change the schema-informed test segfaults, and without the DynArray change the header options test aborts.

I did not test the Windows (MSVC) build. No new files were added, so the Visual Studio projects should not need changes.

Malformed EXI streams could crash the process instead of returning an
error: out-of-bounds reads in the value tables, asserts reachable from
the input, realloc() of static string tables, a use-after-free across
streams that share a schema, and a value decoded outside any element.

Add malformed input tests (bit flips, truncations and random corruption
of existing test-set files, in schema-informed and schemaless modes,
plus an EXI options header that grows the static options grammar).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant