clibmdbx is an independent, high-performance CPython binding for
libmdbx. Both the distribution and
import name are clibmdbx. It is a community project and is not an official
Mithril-mine/libmdbx package.
The hot path is hand-written CPython C API code calling the libmdbx C API
directly. The extension embeds the official libmdbx 0.14.3 amalgamation, so a
wheel contains libmdbx itself. Python typing compatibility uses typing_extensions.
It does not use ctypes, CFFI, Cython, Rust, or a helper service.
Stable release: 1.0.5 deliberately uses a CPython-version-specific ABI for maximum hot-path performance. Public API compatibility follows Semantic Versioning within the 1.x line.
import clibmdbx
with clibmdbx.Environment("cache.mdbx", max_dbs=8) as env:
with env.write() as txn:
txn.put(b"answer", b"42")
with env.read() as txn:
assert txn.get(b"answer") == b"42"
assert txn[b"answer"] == b"42"Named databases, DUPSORT, cursors and batches use the same transaction model:
with env.write() as txn:
events = txn.open_db(b"events", flags=clibmdbx.MDBX_DUPSORT, create=True)
txn.put_many([(b"job", b"started"), (b"job", b"finished")], db=events)
with env.read() as txn, txn.cursor(events) as cursor:
print(cursor.set(b"job"))
print(cursor.count())All keys and values accept bytes or a contiguous bytes-like object. Returned
data is copied to safe bytes. MDBX_RESERVE and borrowed zero-copy views are
not exposed because their native pointers can outlive neither the write call nor
the mapped transaction safely.
For a single warm lookup, env.get(key, db=None, default=None) performs the
complete short read transaction in one C call and returns a safe copy. It does
not retain an active snapshot between calls. A bounded private reset handle
avoids repeated allocation; max_spare_txns=0 disables it. With the default
max_spare_txns=1, budget one idle reader slot per environment/process; explicit
transactions, reader_check(), management operations and close release that
spare. Use an explicit read transaction and
get_many() when several values must share one snapshot.
The reset-handle optimization and max_spare_txns option were added in 1.0.5.
Version 1.0.4 allocates/aborts each one-shot read and retains no idle slot.
- An
Environmentmay be shared by threads. Open a given database path only once per process and share that object; upstream libmdbx rejects a second open of the same environment in one process. libmdbx permits only one write transaction at a time per environment. Environment.get()keeps the GIL on the uncontended warm-read path. While a writer is pending or active it yields during transaction begin, preventing tight reader loops from starving write progress.- A
Transactionand every cursor belonging to it are bound to the thread that created the transaction. Cross-thread use raisesThreadErrorbefore calling libmdbx. - Transactions keep their environment alive; cursors and DB handles keep their
owners alive. Closing an environment with active transactions is rejected.
A
Database.close()closes only that Python view because repeated opens may share one native DBI; ordinary native DBIs remain environment-owned, avoiding upstream's double-close/concurrent-close corruption hazard. Repeatedclose()/abort()is safe. - A process created with
fork()must open a fresh environment. Inherited objects raiseForkError; their destructors intentionally do not call into the parent's native handles. - The current single-phase high-performance extension supports CPython's main
interpreter only. Import in a subinterpreter raises
ImportErrorexplicitly rather than sharing interpreter-owned exception/type objects unsafely. Use a separate process when interpreter isolation is required. commit,sync,copy, defragmentation and warm-up release the GIL once around the native operation when no current-thread writer is involved. A commit with an open cursor retains the GIL because libmdbx updates the transaction's cursor list while committing; close cursors first when concurrent Python progress during commit matters. Warm point reads keep the GIL to avoid a costly release/reacquire on every key. Batch methods cross Python/C once and loop in C. Their default fast path retains the GIL;detached=Truefirst copies every input, then releases the GIL exactly once around the native loop. Use detached mode for cold pages, large batches or latency-sensitive RPC thread pools. Detached mode requires all cursors in the transaction and parent chain to be closed, preventing cross-thread cursor finalization from racing native work.- The wrapper always enables native
MDBX_NOSTICKYTHREADS, then applies the stricter Python owner-thread checks itself. Read transactions can therefore be finalized safely by a different Python worker. libmdbx still requires a write transaction to finish on its creator OS thread: an accidental cross-thread final reference is marked broken and queued, never unlocked on the wrong thread. Owner identity uses CPython's unique thread-state ID, so a newly created thread cannot impersonate an exited owner when the OS reuses a numeric thread ID. The real owner reaps the transaction automatically on its next environment operation or explicitly withenv.reap_orphaned_transactions(). If that owner has exited, the environment is faulted: new transactions and native environment operations raiseBusyErrorimmediately. Monitorenv.orphaned_write_transactionsand restart the process after draining service traffic. Root writers and every environment operation that can acquire MDBX's write lock (sync, online geometry/flag/option changes anddefrag) are queued at a binding-level gate before entering libmdbx; an off-owner finalizer wakes them withBusyErroron every supported platform. Never try to abort the native writer from another thread. drop(delete=True)is rejected while an unrelated transaction may still reference the shared DBI. It invalidates all duplicate Python aliases after the native delete-and-close succeeds. Creation rolled back at any nesting level likewise invalidates its provisional aliases.
Durability defaults are upstream libmdbx defaults. Unsafe modes such as
SAFE_NOSYNC, UTTERLY_NOSYNC, WRITEMAP, and NOMEMINIT are opt-in constants;
the binding never enables them silently.
For capacity thresholds, long-reader detection, disk-full recovery, bounded
writer waits and shutdown ordering, follow the
production operations runbook. In particular,
plan the geometry upper bound before opening every process: online map growth
can legitimately return UnableExtendMapError, including when address-space
layout prevents remapping, and the supported recovery is a coordinated close
and reopen with the larger upper bound.
Environment: one-shot point get, geometry, options, flags, sync, consistent copy, online defragmentation, warm-up, reader cleanup, statistics and information.Transaction: read/write/nested transactions, commit timing, abort, reset/renew, refresh, park/unpark, canary, GC information, DB enumeration, CRUD, replace and batch operations.Database: flags/statistics, sequence, clear/drop, close and rename.Cursor: exact/range positioning, forward/reverse and DUPSORT traversal, count, write/delete/renew, iterator protocol and bounded range collection.- Stable exception classes for common libmdbx failures, plus
ThreadError,ForkError, andClosedErrorfor wrapper safety checks. Native exceptions includecode,what, andreason, matching the diagnostic ergonomics expected from mature LMDB bindings.
See the complete C API coverage matrix, the API/lifecycle guide, the wtdcode/mdbx-py ctypes compatibility audit, the python-lmdb comparison, and the benchmark method.
A C11 compiler and CPython development headers are required. Builds use the vendored source and perform no network access.
python -m venv .venv
. .venv/bin/activate
python -m pip install -U pip build pytest twine
python -m pip install -e .
python -m pytest -q
python -m build
python -m twine check dist/*Strict warnings are enabled by default. Sanitizer builds are supported on Clang/GCC:
CLIBMDBX_SANITIZE=address,undefined python -m pip install -e . --no-build-isolation
ASAN_OPTIONS=detect_leaks=0:abort_on_error=1 \
UBSAN_OPTIONS=halt_on_error=1 PYTHONMALLOC=malloc python -m pytest -qFor source provenance, wheel policy, release commands, and supported-platform evidence, see VENDORING.md, docs/BUILDING.md, and RELEASING.md.
This source tree pins official libmdbx 0.14.3, upstream release tag
v0.14.3. The release tag resolves to commit
f7a3a9323cacacfa9dc6137ae7a7252a67744ff0; the official amalgamation records
source commit 251562b2dc55266d8e6d0e6627ec88ecb410702f.
The official amalgamation archive SHA-256 is
dbc4a791c44d3e51a8159eedfee0dedada7b21d46c22588f0fa99294983f33cd.
The wrapper and libmdbx are Apache-2.0 licensed. The sdist and wheels retain the
wrapper and upstream LICENSE, NOTICE, and COPYRIGHT; see
THIRD_PARTY_NOTICES.md and SBOM.cdx.json.