Skip to content

mod_mam_mnesia: reduce paged query memory usage - #4636

Merged
badlop merged 2 commits into
processone:masterfrom
catap:mnesia-paginate
Oct 28, 2026
Merged

badlop merged 2 commits into
processone:masterfrom
catap:mnesia-paginate

Conversation

@catap

@catap catap commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

Reduce retained memory for paginated archive queries; preserve query filters, ordering, page limits, counts and completeness, including records with equal archive IDs.

Support count only queries without message retrieval or conversion; preserve existing behavior for unlimited queries. Concurrent archive deletions can shorten a returned page.

Add regression coverage for pagination, filters, equal archive IDs, count only requests and XML-heavy queries under a heap limit.

OTP 28.5.0.8 references:
[1] https://github.com/erlang/otp/blob/OTP-28.5.0.8/lib/stdlib/src/dets.erl#L1755-L1764
[2] https://github.com/erlang/otp/blob/OTP-28.5.0.8/lib/stdlib/src/dets.erl#L2958-L2987

@badlop

badlop commented Oct 28, 2026

Copy link
Copy Markdown
Member

Hi, nice improvement!

There's a recent merged commit that also involves mod_mam_mnesia.erl, and there's a source code conflict.

Do you have spare time to resolve the conflict? If not, I can take a look.

@catap

catap commented Oct 28, 2026

Copy link
Copy Markdown
Contributor Author

Sure, will do it in an hour.

catap added 2 commits October 28, 2026 22:49
Reduce retained memory for paginated archive queries;
preserve query filters, ordering, page limits, counts and completeness,
including records with equal archive IDs.

Support count only queries without message retrieval or conversion;
preserve existing behavior for unlimited queries. Concurrent archive
deletions can shorten a returned page.

Add regression coverage for pagination, filters, equal archive IDs,
count only requests and XML-heavy queries under a heap limit.

OTP 28.5.0.8 references:
[1] https://github.com/erlang/otp/blob/OTP-28.5.0.8/lib/stdlib/src/dets.erl#L1755-L1764
[2] https://github.com/erlang/otp/blob/OTP-28.5.0.8/lib/stdlib/src/dets.erl#L2958-L2987
@catap

catap commented Oct 28, 2026

Copy link
Copy Markdown
Contributor Author

@badlop here it is. I had rebased to the last master and included one more commit which extracts only timestamp, without payload. I can open it as separated PR, but I think this is related to current one.

@badlop badlop added this to the ejabberd 26.xx milestone Oct 28, 2026
@badlop
badlop merged commit a4a59d7 into processone:master Oct 28, 2026
19 checks passed
@catap
catap deleted the mnesia-paginate branch October 28, 2026 23:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants