Repository navigation
Conversation
Signed-off-by: importcpp <57519936+importcpp@users.noreply.github.com>
| if (!host_ordered_segments.empty()) { | ||
| // Keep the vector's preference order while avoiding a | ||
| // growing linear search for every host-ordered name. | ||
| std::unordered_set<std::string> seen; |
There was a problem hiding this comment.
Could we use a request-local std::unordered_set<std::string_view> here, while keeping preferred_segments as the owning, insertion-ordered vector?
The current implementation is safe, but the temporary set owns another copy of every unique name. For names beyond SSO, this adds character-buffer allocations and frees on top of the hash-node allocations, all inside the segment shared-lock scope.
This should not be a mechanical type replacement: views seeded from preferred_segments can dangle when subsequent push_back calls reallocate that vector, especially for SSO strings. One bounded-scope alternative is:
const size_t max_size =
preferred_segments.size() + host_ordered_segments.size();
preferred_segments.reserve(max_size); // Before creating any views.
std::unordered_set<std::string_view> seen;
seen.reserve(max_size);
for (const auto& name : preferred_segments) {
seen.insert(std::string_view{name});
}
for (const auto& name : host_ordered_segments) {
if (!name.empty() && seen.insert(std::string_view{name}).second) {
preferred_segments.push_back(name);
}
}With this scope, the existing preference strings cannot move during insertion, host_ordered_segments remains unmodified, and seen is destroyed before either backing vector. Preference order and name-based deduplication stay unchanged.
This could reduce temporary bytes/copies and long-name allocations without changing placement policy. It still allocates hash nodes and hashes all names, so it should not be assumed to eliminate the small-pool regression. Could the existing A/B harness and allocation census compare this variant for both small pools and large short-/long-name pools?
This is an optimization suggestion, not a correctness finding against the current owning-string implementation.
Description
MasterService::AllocateReplicasprepares a host-ordered preference list for single-replicalocal_firstplacement andprefer_alloc_in_same_node. It generates the complete host-order list before allocation, then appends each name through a growingstd::findsearch of the output vector. With no explicit preference and N distinct names, the merge performs N(N-1)/2 string comparisons: 499,500 at N=1,000. The preparation cost is paid even when the first local candidate can satisfy the write.Use the existing vector for preference order and a request-local
unordered_set<string>for membership, only while merging a nonempty host-order list. Seed the set with the already resolved explicit preferences and append a nonempty name only on its first insertion. This makes the automatic merge expected O(P+N), with P resolved explicit preferences; the explicit-preference parsing/deduplication itself is unchanged, and worst-case hash collisions can still degrade complexity.Minimal scope and preserved behavior
Related work: preserves the local-first behavior introduced in #2638. This patch is based directly on main, independently of the owner-snapshot optimization #5524 and bounded fallback coverage #5527. Shared directory snapshots in #5106 and serving-count/client-index work in #4429 address separate costs and are not included here. Searches for existing host-order/quadratic-dedup work found no direct duplicate.
Real Master measurements and tradeoffs
Base:
504f2ae4a307abe095d3baa75d40ed2661b6481b.CPU-only Linux aarch64/GCC 13.3,
-O3, local shared Colima VM (container: 3 CPU quota / 6 GiB). Direct, single-writerMasterService::PutStartwith real Offset allocators: one 64 KiB segment per client/host, 4 KiB incoming objects, fixed seed and occupancy. Mount/fill, PutRevoke, verification and output are outside the timer. No RPC, network/payload transfer, NoF or device I/O is included. These are local diagnostics, not a concurrent-throughput or production-topology acceptance test.The broad run uses six alternating AB/BA pairs, 20 scenarios, 300 warmups and 5,000 timed calls per scenario/process: 1,200,000 timed calls. Each pair matches success counts and endpoint-sequence fingerprints; full-pool failures are expected and remain failures in both versions. Table values are medians of the individual run quantiles, in microseconds:
Small-pool p99 regression: an explicit cost
The hash table adds hashing, temporary allocations and lock-held construction/destruction. A separate small-pool follow-up uses 20,000 warmups / 100,000 timed calls, four AB/BA pairs over 12 scenarios, plus same-binary A/A comparisons: 9,600,000 timed calls. It does not replace the original six-pair results.
All four N=16 fit pairs worsen in both wall and CPU quantiles: CPU p99 paired medians increase by 0.417 us for short names and 0.771 us for long names. This is reported as a real cost, not dismissed as noise. N=16 short-name same-base A/A p99 changes are +0.167 / -0.041 us; fixed A/A changes are 0 / -0.125 us. They provide drift context and are not subtracted from A/B. The all-full +3.292 us pair also has CPU p99 +3.5 us. N=64 results have both positive and negative pairs.
An independent ordinary-C++-operator-new census, excluded from latency analysis, finds per-request increments of 2 calls / 64 requested bytes at N=1; 1,001 / 56,248 bytes at N=1,000 short names; 2,001 / 163,248 bytes for long names. Ordinary placement adds zero counted allocations. These are main-thread allocation-request counts and summed bytes, not all malloc calls or peak retained memory.
The submitter explicitly accepts the disclosed microsecond-scale small-pool cost. This patch does not claim p99 non-regression for every pool size or group. It keeps the minimal implementation without an arbitrary small-pool threshold or allocator tuning.
Separately retained: unresolved ordinary-path p99 bad pair
The original multi-scenario run contains an ordinary-placement N=1,000 pair with p99 +35.712 us, even though this branch constructs no new set. Other retained adverse points include N=16 all-full +3.917 us and N=64 local-fit +4.239 us. None is replaced by follow-up medians.
A fresh-process control running only ordinary N=1,000 placement uses 5,000 warmups / 20,000 timed calls, four pairs and same-binary A/A: 160,000 timed calls. Run-quantile p99 medians are 89.0965 us main / 84.5535 us patch; paired changes have median -7.2925 us and range -11.461 to +2.375 us, with one adverse pair. Same-base A/A changes are -14.836 / +15.044 us; fixed A/A changes are -7.167 / +1.876 us.
The original +35.712 us point was not reproduced by that control. Its cause is not fully explained, and A/A variation does not prove it was entirely noise. Runtime-state carryover between scenarios remains an unproven explanation. The original measurements are retained; no snapshot, allocator or other framework change is folded into this patch to make them disappear. Local observations are not tail-latency bounds.
Suggested Merge Priority
Module
mooncake-store)Type of Change
How Has This Been Tested?
415 passed, 4 strategy-inapplicable skips, 0 failed, deduplicated across six suites and the HCA-enabled rerun (419 unique cases). The four skips are FreeRatioFirst-specific parameterized cases under Random. The three added tests also pass against the original base library, asserting preserved behavior rather than timing thresholds.
Regression coverage includes the full preference sequence for both local-first entry points; explicit single/array precedence, duplicates and empty names; cross-host same-name registrations; unknown-host lower-bound/wrap; draining/unmount; absent automatic candidates and ordinary placement. Existing local-full remote fallback, snapshot invalidation and HA barrier tests pass.
Test commands (binary paths in the CPU-only build):
Build provenance: reused the separately preserved main store archive and unchanged dependencies, then recompiled the changed Master source and test sources with the existing CMake flags. All 172 archive members were compared byte-for-byte; only
master_service.cpp.odiffers. This is an incremental source build, not a clean rebuild of every dependency. USE_NOF is off; no device I/O validation is claimed.Raw CSV/XML/logs and exact build/diagnostic commands are retained locally, outside the two-file patch. Codex self-review and independent review in a separate chat checked the diff and validation evidence, including recomputing all 384 CSVs / 39 XMLs, 10,960,000 latency samples and 4,000 separate allocation-diagnostic samples. This is AI-assisted review, not an upstream human-review claim.
Test results:
Checklist
Signed-off-bytrailer./scripts/code_format.sh(clang-format 20.1.8)git diff --checkpassAI Assistance Disclosure