Optimize endpoint resolution by mtdowling · Pull Request #1417 · smithy-lang/smithy-java

mtdowling · 2026-10-15T06:27:01Z

This branch turns endpoint BDD bytecode into a load-time format rather than interpreting it on every resolution, then optimizes the remaining URI construction, validation, evaluator setup, and endpoint-property allocation costs.

Overall, the finished branch is 50–56% faster than main across the representative S3 cases rerun after the final register-filling work. Projected endpoint allocation is down to 80 B/op for common cases and 136 B/op when dynamic auth properties are required.

Main changes

Lazily compiles reached bytecode conditions and results into sealed CompiledNode trees.
- Removes the runtime operand stack, program counter, and large opcode dispatch loop.
- Retains bytecode as the serialization format.
- Includes fused nodes for templates, substring equality, host validation, and common collection shapes.
- Preserves substring null, range, reverse, and non-ASCII semantics.
Adds SmithyUriTemplate, a specialized endpoint URI template—not an RFC URI Template.
- Validates static literals once.
- Concatenates host placeholders directly.
- Allows path placeholders to occupy partial or multiple segments.
- Uses fixed-arity expansion methods without staging arrays.
- Reuses the evaluator’s character buffer instead of allocating component buffers.
- Treats values as already encoded URI text; it validates but does not percent-encode them.
Speeds up validation.
- Host labels and non-subdomain S3 bucket names are scanned eight Latin-1 bytes at a time using SWAR.
- Non-Latin-1 values use the equivalent scalar path.
- Shared ASCII support lives in codec-commons/rulesengine utilities.
Reduces evaluator overhead.
- Parameters write directly into evaluator registers instead of going through a staging array.
- Register filling and extension classification are prepared at resolver construction.
- Built-ins already satisfied by register defaults are omitted from the runtime filling loops.
- Evaluators are reused through a resolver-local striped pool.
- aws.partition reuses its hot-region result.
Projects endpoint properties instead of rebuilding them.
- PropertyPlan compiles and interns property shapes and fully static subtrees at load time.
- Dynamic leaves receive stable slots.
- ProjectedEndpoint0 through ProjectedEndpoint3 store small dynamic results without an Object[]; larger shapes use the general representation.
- Auth schemes and ordinary endpoint properties are read directly from the projected endpoint.
- Auth schemes are constructed lazily and constant schemes are shared.
- PropertyViews materializes the compatibility representation only when an extension declines direct binding.
- The auth-specific binder SPI was removed. The generalized unstable RulesExtension.bindEndpointProperties hook returns null to request the existing materialized path.
Improves benchmark and correctness coverage.
- Adds generated-client-style bound parameter benchmarks.
- Adds fixed-cost rulesets for isolating individual operations.
- Adds conformance coverage for every AWS ruleset.
- Adds exhaustive and randomized equivalence tests for the SWAR validators.

Benchmark results

The comparisons below use generated-client-style bound parameters.

Steady-state endpoint resolution

The main baseline is the bytecode interpreter. The branch-tip results include compiled nodes, URI templates, SWAR validation, evaluator reuse, projected endpoint properties, and defaulted-built-in elimination.

Case	`main`	Branch tip	Change
Virtual addressing	487.0 ns	212.5 ns	−56.4%
Express data plane	547.7 ns	265.1 ns	−51.6%
Access-point ARN	811.7 ns	401.8 ns	−50.5%
S3 Outposts	905.2 ns	423.9 ns	−53.2%

The additional path-style branch-tip case measured 242.3 ns.

First-use overhead

The compiled representation does not require eagerly compiling the whole ruleset. Conditions, results, and property plans are compiled only when reached.

The following controlled comparison uses a warmed JVM and a fresh resolver for every sample, and times the complete first resolveEndpoint invocation. Resolver construction and bytecode loading occur before the timed call on both sides.

Case	`main`	Branch tip	Absolute cost
Virtual addressing	1.33 µs	2.08 µs	+0.75 µs
Express data plane	1.48 µs	2.48 µs	+1.00 µs
Access-point ARN	1.79 µs	2.95 µs	+1.17 µs

Although compilation makes the first call 56–67% slower in relative terms, the absolute penalty is under 1.2 µs in these cases. There is no whole-ruleset compilation pause, and subsequent calls receive the 50–56% steady-state reduction above.

Endpoint resolution spent most of its time in the bytecode interpreter rather than in the BDD. Profiling S3 showed the BDD walk at 6-7% while BytecodeEvaluator.runLoop took 65-75%: a 3535-bytecode method where all ~55 opcodes share one tableswitch, so every instruction paid a mispredicted indirect branch (~6.5ns measured). A vanilla resolve walks 16 BDD nodes but interprets ~50 instructions. NodeCompiler now decodes each condition and result from the existing bytecode into a CompiledNode tree by simulating the operand stack symbolically, recovering the structured coalesce and ite idioms. Operands land in final fields and the operand stack is gone except where a node needs scratch space. Peepholes fold the shapes that dominate, notably coalesce(substring(reg, ...), "") == "literal" into one bounded regionMatches, and merge constant template parts. The bytecode is now purely the serialization format. The interpreter is deleted: run, runLoop, the temp-array machinery and the pc field, 415 lines of BytecodeEvaluator. Opcodes, the reader, writer, compiler, disassembler and walker are untouched, since they are the .bdd format. An instruction NodeCompiler cannot decode raises a RulesEvaluationError instead of degrading silently; because compilation is lazy that surfaces on the first resolution reaching it, which is when the interpreter would have tripped over it too. Compilation is lazy per condition and result. A resolve walks one BDD path and returns one result -- for S3, 15 to 30 of 76 conditions and 1 of 116 results -- so compiling eagerly threw away ~90% of the work, and Bytecode is built per client builder rather than once per JVM. Eager compilation cost +113us per load; lazy costs nothing measurable. It also costs nothing in steady state because the hot path already had the array load and null check it reuses. Trees are published by a plain array write and may be computed twice under a race, which is safe because every field of every node is final; the hierarchy is sealed so that invariant can be checked by reading one file. Measured on S3 (az+express, 6x4s), interpreter vs compiled: vanillaVirtualAddressing 477 -> 364 ns vanillaPathStyle 527 -> 409 ns dataPlaneShortZone 585 -> 437 ns vanillaAccessPointArn 778 -> 535 ns s3OutpostsVanilla 896 -> 645 ns Confirmed independently in the aws-sdk-java-v2 bdd-endpoints-benchmarks harness built against this build: 1.24x to 1.59x across its five S3 cases, and 1.12x to 1.32x on Lambda. Also tried and rejected: a 4-wide BDD node table fusing the condition kind and operand into each node to drop two scattered dependent loads per node. No consistent gain and one case 7% worse -- these BDDs are ~550 nodes, so the loads it removed were already L1 hits and the larger table cost as much as it saved. The walk is not the bottleneck, so memory-layout tricks there do not pay. MAP3 is decoded with the same value-then-key layout as every other arity. The retired interpreter read its slots in a different order, building a map of values keyed by values and throwing ClassCastException unless they happened to be strings, so no working program can have depended on it. Correctness is gated by EndpointRulesetConformanceTest, which resolves every endpoint test case of every AWS ruleset on the classpath and compares against the URL the model itself declares: 868 tests. It replaced a test that compared against the interpreter, which could not outlive it and could not catch a case where both paths agreed and both were wrong. Switching oracles immediately surfaced two such pre-existing defects, which reproduce identically on the interpreter and are characterized by shape so a new divergence cannot hide behind them: SmithyUri lowercases the host of a custom endpoint, and an endpoint template carrying an explicit port resolves to "scheme://[host:port]" because BUILD_URI always passes port -1 and the colon makes SmithyUri bracket it as an IPv6 literal. This adds no public API.

Endpoint.Builder.addAuthScheme went straight to new ArrayList<>(), which cost three objects to hold one element: the list, the capacity-10 Object[] its first add grows into, and the unmodifiable wrapper built in the EndpointImpl constructor. An allocation profile of S3 endpoint resolution put that at ~31% of the bytes allocated per resolve, for a list that essentially always has exactly one entry. Hold the first scheme in a field and only allocate a list once a second one arrives, then expose the single case as List.of(scheme). Measured on S3 (az+express, isolated JVM per benchmark, 3 forks x 8 iterations): vanillaVirtualAddressing 186.7 -> 168.0 B/op, 315.3 -> 310.3 ns/op s3OutpostsVanilla 336.0 -> 298.7 B/op, 581.7 -> 577.4 ns/op The time difference is ~1% and at the edge of significance, as expected when the allocation is TLAB-cheap and the benchmark does not stress GC. It also made vanilla's allocation deterministic: it was bimodal across forks before (+/-20.7 B/op) and is now exact, because the builder no longer sits on an escape-analysis cliff.

The S3 endpoint benchmark passed every parameter by name through ADDITIONAL_ENDPOINT_PARAMS, which is the escape hatch rather than how a generated client works. That made the resolver re-resolve 6-7 names to register indices on every call, and a CPU profile attributed ~18% of a resolve to it. There was a "binding" param mode for this, but it was commented out and would not have measured anything: the old buildParams set ADDITIONAL_ENDPOINT_PARAMS unconditionally. Add a live "bound" mode that supplies parameters the way a generated client does: the bucket through its smithy.rules#contextParam input member, and the region and feature flags through the typed context keys the builtins read, whose register indices are resolved once at load. Nothing is passed by name. All five cases were checked to resolve to the same URL in both modes before comparing. The by-name path turns out to cost far less than the profile claimed (isolated JVM per benchmark, 3 forks x 8 iterations, az+express): vanillaVirtualAddressing 309.7 -> 307.3 ns (not significant) dataPlaneShortZone 401.8 -> 370.9 ns (-7.7%) vanillaAccessPointArn 534.0 -> 511.9 ns (-4.1%) s3OutpostsVanilla 624.9 -> 622.1 ns (not significant) So removing by-name parameter passing entirely buys 0-8%, not 18%. That is consistent with three separate attempts to optimize that path -- a per-map index cache, an open-addressed String-to-int table replacing the HashMap<String,Integer>, and arity-specialized struct nodes -- each of which measured as noise and was not kept. The sampling profiler over-attributes to this path, and the honest reading is that no single item dominates what remains after the interpreter was removed. Note the bound mode is, if anything, pessimistic about the win: the dynamic client's input is Document-backed, so reading the contextParam member is a map lookup where a generated client would read a field. Cycle-accurate profilers would settle this directly but none are available on macOS: perf, perfasm, perfnorm and perfc2c all shell out to Linux perf, and dtraceasm needs sudo. Running these on the EC2 metal instance that scripts/run-metal-benchmarks.py provisions would give real cycle attribution.

…wrapper Several attempts to speed up endpoint resolution were guided by a sampling CPU profile and all measured as noise, so this adds a benchmark that decomposes a resolve by construction instead of by sampling. EndpointFixedCostBenchmark resolves minimal rulesets that differ by one rules-language operation, so each delta is that operation's own cost. Parameters always arrive through the region builtin, so parameter passing is identical across rulesets. What it shows (isolated JVM per ruleset, 3 forks x 8 iterations): constant 32.7 ns 24 B/op no conditions, hardcoded URL template 56.3 ns 72 B/op URL interpolated from the region oneCondition 58.2 ns 72 B/op plus one isSet substring 77.4 ns 88 B/op partition 88.0 ns 88 B/op virtualHostable 89.9 ns 88 B/op hostLabel 92.6 ns 88 B/op So the engine's fixed per-call overhead -- building the register array, filling builtins from the context, acquiring and releasing a pooled evaluator, walking to a terminal and constructing the Endpoint -- is 32.7 ns and one 24-byte allocation. A BDD node plus a trivial condition is under 2 ns. Neither register setup nor the BDD walk is where the time goes; the rules-language operations are, at 12-27 ns each. That is why optimizing the register path, the property lookups, the node arities and the BDD node layout all measured as nothing. Acting on that: aws.partition allocated a fresh PartitionMap on every resolution, even though a client almost always resolves for one region and the function appears in nearly every AWS ruleset. Cache it on the region using the same single-volatile hot-key pattern AWS_PARSE_ARN in the same file already uses: partition 88.0 +/- 1.5 ns, 88 B/op -> 72.3 +/- 3.1 ns, 72 B/op -17.8% and exactly the 16 bytes of the record. Full S3 does not show the 16 ns, but S3's run-to-run drift is +/-15-40 ns, so an effect this size is below that benchmark's resolution -- which is the reason the decomposition benchmark exists. Also fixes a bug in the new benchmark worth noting, because it is easy to repeat: the stub operation originally built its Schema inside schema(), and ContextProvider reads operation.schema().id() on every resolve, so 93% of the measured allocation was the benchmark's own. The first numbers were 57/139/140 ns; after holding the schema in a field they are 33/56/58 ns.

The single-operation rulesets build a three-part URL while the TEMPLATE baseline builds a two-part one, so their deltas included the extra URL part and overstated what each operation costs. templateLong is the same three-part URL with no condition. With it, the per-operation costs are (3 forks x 8 iterations): template 58.4 ns two-part URL templateLong 64.9 ns three-part URL, no condition partition 72.9 ns -> aws.partition ~ +8 ns (vs template) substring 81.3 ns -> fused substring-eq ~ +16 ns (vs templateLong) hostLabel 90.8 ns -> isValidHostLabel ~ +26 ns virtualHostable 91.4 ns -> isVirtualHostable ~ +26 ns aws.partition is now the cheapest of the four, having been the most expensive before its wrapper was cached. The two host-validation predicates are the expensive ones at ~26 ns each for a 14-character label, and both are char-at scans in smithy library code.

isValidHostLabel was the most expensive single operation in an endpoint resolve. The decomposition benchmark put it at 32.6ns for a 14-character label, which is far more than the comparison work implies: the library version routes every character through String.charAt into a separate isAlphanumeric helper. Read the string's Latin1 bytes through CompactStringAccess and test eight at a time with SWAR range checks, falling back to a character scan when the bytes are not directly reachable. smithy IsValidHostLabel 32.6 ns flat scalar loop 27.4 ns SWAR + Latin1 13.4 ns So the SWAR is doing the work, not merely removing the call layering: flattening the loop alone recovers 5 of the 19 ns. End to end on S3 (5 forks x 8 iterations, paramMode=bound): vanillaAccessPointArn 528.3 -> 476.5 ns (-9.8%) vanillaVirtualAddressing no measurable change accessPointArn validates about six labels so the saving compounds; vanilla validates one, on a nine-character region, which is inside the benchmark's noise. aws.isVirtualHostableS3Bucket is the matching ~26.5ns predicate and is still unconverted. The first version of this was wrong: subtracting straight from the word let a lane that underflows borrow from the lane above and corrupt its answer, so isValidHostLabel("op-01234567890123456", false) returned false. Testing '-' (0x2D) against '0' (0x30) underflows. Setting a guard bit per lane first keeps the subtraction inside the lane. HostLabelEquivalenceTest compares against the library implementation rather than asserting hand-picked expectations, since being faster is the only reason this exists. The original version of that test passed despite the bug, because the word loop only engages at nine characters and it only covered one- to three-character strings; it now sweeps every length to 26 with each awkward character at every offset, plus 200k seeded random labels. The endpoint conformance suite caught the bug. rulesengine now depends on codec-commons through its shadow configuration, matching how aws-sigv4 and http-api already consume it.

Companion to the host-label scan. aws.isVirtualHostableS3Bucket was the other ~25ns predicate in the decomposition benchmark, and S3 evaluates it on the common virtual-addressing path. Only the no-subdomains form is rewritten, which is the form that path uses: it reduces to "first and last characters are lowercase alphanumeric, everything between is that or a hyphen". The subdomain form compares each character against its predecessor and then tests the whole name for an IP address, which has no word-at-a-time equivalent, so it still calls the library. Note the bucket character class is lowercase-only, unlike the host-label one. The SWAR primitives moved out of EndpointUtils into AsciiBytes so both scans can share them, since the AWS functions live in another package. That is new @SmithyInternalApi surface in rulesengine, which seemed better than duplicating the word tricks in two modules. Isolated (5 forks x 8 iterations, against a same-URL control): virtualHostable predicate 24.8 -> 11.6 ns (-53%) End to end on S3 the change is not resolvable: vanillaVirtualAddressing 310.7 -> 303.9 and vanillaAccessPointArn 492.6 -> 483.7, both with overlapping intervals. Identical code measured 476.5 and 492.6 for accessPointArn across two runs, so that benchmark has ~16ns of run-to-run drift and a single short-string predicate call sits inside it. The direction is consistent across every run, and the host-label sibling did clear the bars at -9.8% on accessPointArn, which evaluates about six labels. BucketNameEquivalenceTest compares against the library rather than asserting expectations, and its coverage is shaped by the bug in the host-label scan: the word loop only engages at nine characters, so it sweeps every length to 26 with each awkward character at every offset plus 200k seeded random names, rather than relying on short strings.

Bytecode result nodes were lazy, but EndpointFactory defeated that by walking and compiling every result during resolver construction. The S3 program currently contains 203 results. Build and cache only the selected ResultPlan on first use while preserving property-plan interning under the cold-path lock. Steady-state endpoint creation remains a direct array lookup plus a null check. Resolver-cold medians on JDK 25 (2,000 fresh samples): eager lazy resolver construction 205-209 us 0.5-0.9 us virtual total 225.2 us 25.9 us express total 215.5 us 24.5 us access-point total 218.6 us 18.1 us Warm S3 resolution (3 forks x 8 iterations): virtual addressing 219.044 +/- 29.078 -> 209.244 +/- 9.942 ns/op express data plane 230.936 +/- 6.365 -> 237.634 +/- 5.108 ns/op access-point ARN 412.111 +/- 47.190 -> 418.007 +/- 13.231 ns/op The warmed differences are within the run confidence intervals.

mtdowling force-pushed the optimize-bdd-more branch from b5b73a1 to dc57e30 Compare October 15, 2026 15:01

mtdowling added 12 commits October 15, 2026 10:32

Optimize endpoint URI templates

ff2c461

Optimize compiled endpoint rule evaluation

af0accf

Preserve substring equality semantics

94b3add

Project endpoint properties without materialization

9e4d372

mtdowling force-pushed the optimize-bdd-more branch from dc57e30 to 16cc972 Compare October 15, 2026 15:32

mtdowling added 2 commits October 15, 2026 10:56

Reduce rules engine node compilation overhead

5d30306

Skip defaulted builtins during register filling

f43a162

mtdowling requested a review from adwsingh October 16, 2026 14:23

mtdowling enabled auto-merge (rebase) October 16, 2026 14:23

adwsingh approved these changes Oct 16, 2026

View reviewed changes

mtdowling merged commit 4bb3eaa into main Oct 16, 2026
6 checks passed

mtdowling deleted the optimize-bdd-more branch October 16, 2026 18:48

Navigation Menu

Repository navigation

Optimize endpoint resolution - #1417

Optimize endpoint resolution#1417
mtdowling merged 14 commits into
mainfrom
optimize-bdd-more

mtdowling commented Oct 15, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

mtdowling commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Main changes

Benchmark results

Steady-state endpoint resolution

First-use overhead

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mtdowling commented Oct 15, 2026 •

edited

Loading