Feature : Hibernation Layer and plugins. by forensicxlab · Pull Request #1036 · volatilityfoundation/volatility3

forensicxlab · 2023-11-11T17:44:10Z

Hello,

Adding a new translation layer and plugins to allow modern Windows (Win8-Win11) hibernation file conversion to a raw memory image
Adding new codecs : LZ77, LZ77Huffman decompression algorithms.

Details about the feature : https://www.forensicxlab.com/posts/hibernation/

Best regards.

Hibernation layerwriter is now working from Win1809 to Win22h2 hibernation.

…ugin

…ded from Win8 to Win11. Testing in progress on hibernation files from the different OS versions.

…nation.Dump' plugin (provide the version of Windows)

ikelos · 2023-11-15T00:45:18Z

Thanks very much for your contribution! This is quite a big change, so may take us some time to review (I also need to pull in some of the other guys who wrote our previous hibernation code to check it). So please don't despair, but this may take us some time...

…gins (Info + Dump)

forensicxlab · 2024-02-04T17:12:04Z

Hello @ikelos,

Finally found some time to make adjustments to the code.
Added a bunch of comments for better readability for the reviewer(s).

I still have doubts about some of my choices for the implementation 🧐
Any news on your side about this PR review ?

Kind regards,
Félix.

ikelos · 2024-02-08T12:31:04Z

Thanks Félix, unfortunately the chap that knows the hibernation stuff best ( @awalters ) is a little busy at the moment, so we haven't made much progress with it, but we haven't forgotten about it either. Could you describe your worries about the implementations choices you're not sure about please and hopefully one of us will be able to check out your concerns?

forensicxlab · 2024-02-08T17:43:39Z


-"""Codecs used for encoding or decoding data should live here

+# Temporary comment:


I wonder what's the best way to integrate algorithms into the "codecs" from the framework standpoint.
What are your recommandations about that ? Should I implement algorithms In seperate files ? I feel like I should but I have put both algorithms here for now :)

Yeah, I'd probably put them both in different files, but there needs to be a way to register them (either the main code seeks them out, or they add themselves to a shared singleton object when they're imported). I don't know if it'll be overkill, but it might be pretty cool to build them into the existing python codec framework?

https://docs.python.org/3/library/codecs.html

Thanks, I'll get right on that.

forensicxlab · 2024-02-08T17:50:12Z

+            {}
+        )  # This will hold the mapping between the PageNumber in the decompressed data vs the physical page number.
+
+        if "plugins.Dump.version" in context.config:


The fact that this translation layer is meant to be used with the hibernation.Dump plugin makes me feel bad about this part.
Indeed the FirstKernelRestorePage and KernelPagesProcessed offsets can only be determine via the user input for now.

Is making a translation layer partially dependant to a plugin is an acceptable concept in the volatility3 framework ?

This concept is working well with this implementation and seems to have no side effects from all the tests I've made but an external eye from your side will be delightful :D I propably missed something.

If this is the only requirement for the layer, then make it a requirement (like the kernel_virtual_offset for Intel layers), then anything can instantiate it as long as they provide the right version number for the layer (it might be better if the layer could figure it out for itself, but if not then ask the instantiator to provide it).

Did not find a way to figure the version out by itself in my research at time of writing, I'll look at what’s done in the Intel layer thanks for the tip!

Probably add a PluginRequirement and that should do you. 5:)

forensicxlab · 2024-02-08T17:57:39Z

+            ),
+        ]
+
+    def _generator(self):


The Idea here is to check if the HibernationLayer is indeed stacked before trying to make the conversion to the raw file using the layerwriter plugin. Feel like it's ugly though, did not find another cleaner way right now. I'll be happy to have your insights :D

I mean, that's ok and it does reuse the LayerWriter class rather than implementing itself. If this whole plugin doesn't do anything other than call layerwriter, why don't we just document that to write out a hibernation layer, you should use layerwriter? A specific plugin for a specific layer type doesn't make a lot of sense? We might be able to update layerwriter so you can easily ask it to write all layers of a particular type (so you could say write out all HibernationLayer layers or something)? Essentially, improve the UI of the existing plugin to avoid the need for the new plugin.

That's a very interesting idea.

I’ll leave both plugins in this PR so that the tests by the reviewer(s) can all be done on this branch. If the conclusion goes indeed in this way, I will implement your idea as soon as I can in a dedicated PR (if not proposed by somebody else in time :-) )

forensicxlab · 2024-02-08T18:00:40Z

Thanks Félix, unfortunately the chap that knows the hibernation stuff best ( @awalters ) is a little busy at the moment, so we haven't made much progress with it, but we haven't forgotten about it either. Could you describe your worries about the implementations choices you're not sure about please and hopefully one of us will be able to check out your concerns?

Hi @ikelos,
I've added some comments about some of my worries and questions :D

Thank you for your quick responses every time.

Kind Regards.
Félix.

KamilPacanek · 2024-03-11T10:23:09Z

FYI, just used that one on my local vol3 installation during the weekend and must say worked flawlessly. I could convert with no issues the hiberfil.sys to the raw image file that can be analyzed later with vol3 plugins.
Thank you @forensicxlab, you're my savior 💚

EDIT:
As a bonus, it helped me tremendously at first stage of solving HTB CTF challenge: https://blog.cyberethical.me/htb-cyber-apocalypse-forensics-oblique-final, I really wish the PR got completed soon :) !

ikelos · 2024-03-12T00:33:18Z

Thanks for the feedback, much appreciated! This is still waiting on @awalters for review at the moment, but we'll get to it once things have calmed down for everyone... 5:)

k1nd0ne · 2024-07-15T20:51:38Z

Hello,
Implemented a version using python3 codecs. Still not happy with the performances though.

Since I have a bit of time, I am working on a native python extension module using pyo3 for the decompression algorithms.

The decompression speed is already a lot better with a pyo3 version of the xpress_lz77_plain_decompression algorithm tested.

Currently working on the xpress_lz77_huffman rust binding before updating this PR.

To be continued...

k1nd0ne · 2024-07-23T12:47:21Z

Hello @ikelos,

Just finished the implementation of the algorithms + packaging:

Performances are now a lot better for the conversion of the hibernation file.

It will also be useful for the prefetch plugin. I will post a PR for it if you are happy with adding xpress_lz77 in the requirements.

Python 3.8 is required though, making some checks failed. Updated the pipeline, but I can still go back if we choose to check if the requirement is present for this translation layer (like in the cloudstorage/pcileech layers for example).

Freezing to this point for now :)

FYI: Rust will not be needed as a requirement to run volatility3

Thanks you!

…layer

ikelos

Some errors that ruff picked up could do with some fixing, and the last merge seems to have made a mess of a few of the files. I'll give it a more thorough review once all that's cleaned up.

ikelos · 2024-07-24T21:51:57Z

-
-
-"""
+# Register codecs here.


So, had you been going to go the codec route?

ikelos · 2025-02-08T23:54:02Z

      fail-fast: false
      matrix:
+<<<<<<< HEAD
+        host: [ubuntu-latest, windows-latest]


This seems like a failed merge, could you clear it up please?

ikelos · 2025-02-08T23:54:58Z

-      uses: actions/setup-python@v5
-      with:
-        python-version: ${{ matrix.python-version }}
+      - uses: actions/checkout@v4


Changes just for formatting should be avoided if possible (some people will format it one way, and other a different way, and we'll end up playing pingpong...). I don't think any of the changes in this file are significant, so leave it out of the PR please...

ikelos · 2025-02-08T23:55:18Z

+>>>>>>> develop
    steps:
-    - uses: actions/checkout@v4
+      - uses: actions/checkout@v4


Another mis-merge

ikelos · 2025-02-08T23:55:58Z

-        curl -sLO https://downloads.volatilityfoundation.org/volatility3/symbols/linux.zip
-        unzip linux.zip
-        cd -
+


More format only changes, look like this file can also be left out of the PR.

k1nd0ne · 2025-02-09T10:27:48Z

Hi there. Thanks for the review.

I am still working on some enhancements to guess the Windows version without any arguments.
Also had a bug reported from a fellow that I need to address on Win10 23H
I just need to find the time... Will keep you posted!

Kind regards

ikelos · 2025-02-09T10:29:38Z

No problem, just wanted to make sure it wasn't waiting on me for some reason, take as long as you need! 5:)

Ozer0x777 · 2026-09-26T16:46:56Z

Hi @k1nd0ne, thanks for the layer. I was trying it on a Windows 11 24H2 Fast Startup hiberfil.sys and found two things that may be useful for your work on version detection. I have not published anything else, this is just a comment.

1. On 24H2 the kernel section is skipped with --version 0. KernelPagesProcessed is read at 0x230 and is 0 in that file, so only the boot section is mapped, and the resulting image looks valid but is almost empty.

2. The offset of that counter depends on the build, not only on the release. For each file below I walked the kernel chain of compression sets until it ends by itself, and looked for the header qwords that hold its page count and its byte length. Those relations hold exactly in every file, only the position of the block changes:

file	kernel pages counter	kernel section length (bytes)
Windows 10 22H2, Fast Startup and full hibernation (lab VM)	`0x230`	`0x240`
Windows 11 24H2 26100.1742, Fast Startup (lab VM, original ISO)	`0x238`	`0x248`
a real Windows 11 24H2 file from a more recent update	`0x250`	`0x260`

So a fixed offset per version would work for one 24H2 build and silently read the wrong counter on the other (the image comes out almost empty, with no error). Checking the counter against the chain (follow the sets, the page count must match exactly) picks the right one without a build number.

The compression-set header is the same on both 24H2 files: the size is in bits 8-28 (bit 29 is not part of it, it is set on most sets), and bit 30 selects LZ77+Huffman. Every set of the 26100.1742 kernel section decompresses with those rules.

I have a small patch for --version 4 that does this, tested on the two 24H2 files (on the real one the output is byte-identical to a fixed-offset version), and a script that prints the counters without dumping anything. Happy to send them if that is useful, in whatever form you prefer. Not tested: 25H2, full hibernation on Windows 11, other 26100 builds.

…uffman set size Two independent problems of the hibernation layer on Windows 11 Fast Startup files, validated on 21H2, 22H2, 23H2, 24H2, 25H2 and 26H1 lab VMs (see contrib/hibernation_checks/campaign_2026-10). 1. KernelPagesProcessed is not at 0x230 on build 26100 and later, and its offset depends on the build: 0x238 in 26100.1742, 26100.6584 and 28000.1340, 0x250 in a 26100.9445 file. With --version 0 the kernel section is skipped or mis-walked. New --version 4 (Windows 11 24H2 and later): the counter is read at the known offsets and accepted only if the chain of compression sets of the kernel section adds up to exactly that number of pages; a file where no candidate matches is refused with an explicit message. On those builds the set header carries the size in bits 8-28 (bit 29 is not part of it) and bit 30 selects LZ77+Huffman. 2. _uncompress asked the LZ77+Huffman decoder for 65536 bytes for every set. A set of fewer than 16 pages (present at the end of the kernel section of every Windows 11 21H2-24H2 file of the campaign) fails with "EOF Error" and aborts the whole conversion, including with the existing --version 0. The layer now passes the size of the set (pages * 4096). Versions 0 to 3 are otherwise unchanged.

…campaign logs) - hib_check.py: walks the chain of compression sets of a hiberfil.sys with the 23h2, 24h2-early (0x238) and 24h2 (0x250) layouts, finds the header qwords that bound the kernel section, reports set kinds and header bits, and with --decompress decodes every compressed set with the output size checked. Counters only, nothing identifying by default. - compare_outputs.py, count_nonzero_pages.py: page-by-page comparison of two Dump outputs, non-zero page count of a raw image. - test_hib_layout.py (16 tests of the tools) and test_hib_layer.py (3 tests of the real layer) on synthetic files generated in memory, no sample data. - campaign_2026-10/: logs of the campaign on Windows 11 21H2/22H2/23H2/24H2/ 25H2/26H1 lab VMs: per file hib_check output, Dump with this branch (twice), with the original code and with the wrong --version; unit tests x5, with the decoder fix reverted, and ruff.

hibernation: Windows 11 24H2+ support (build-dependent counter offset) and a decoder size fix.

volatility3/framework/layers/codecs/__init__.py imported the standard `codecs` module without using it (F401), which makes the lint-and-format workflow of the upstream PR volatilityfoundation#1036 fail with ruff 0.17.1. The file is now identical to the one on volatilityfoundation/volatility3 develop.

Ozer0x777 · 2026-10-16T18:34:24Z

Hi @k1nd0ne, thanks for merging forensicxlab#1.

The lint-and-format job of this PR now fails on a line that predates the merge: volatility3/framework/layers/codecs/__init__.py:3:8: F401 'codecs' imported but unused (ruff 0.17.1). The import codecs was added to that __init__.py in 2024 and nothing uses it; develop does not have it.

forensicxlab#2 removes it (the file becomes identical to the one on develop). Checked with ruff 0.17.1 locally and the workflow on your repository is green on that branch; the other jobs here (build, tests, CodeQL) already pass.

codecs: drop the unused import that fails the ruff job of volatilityfoundation#1036

k1nd0ne and others added 8 commits October 26, 2023 02:49

Adding hibernation support

b0cbd3c

Fixing Huffman

c91e417

Hibernation layerwriter is now working from Win1809 to Win22h2 hibernation.

Adapting comments to the codec instead of the volatility3 prefetch pl…

5c44467

…ugin

Update the Xpress LZ77+Huffman decoder

a13e105

Code comments and cleaning.

3568d3f

Plugins added : hibernation.Info and hibernation.Dump. Support for ad…

7e87e2d

…ded from Win8 to Win11. Testing in progress on hibernation files from the different OS versions.

Adding support for Windows 10 2016 1607

142baa6

Only parse the kernel section if the user is using the 'windows.hiber…

cfab14f

…nation.Dump' plugin (provide the version of Windows)

forensicxlab changed the title ~~Feature/hibernation~~ Feature : Hibernation Layer and plugins. Nov 12, 2023

ikelos requested review from awalters and ikelos November 15, 2023 00:45

ikelos assigned awalters Nov 29, 2023

k1nd0ne added 2 commits February 4, 2024 17:52

Quick code review and comments to make it more readable. Enhanced plu…

f6b960a

…gins (Info + Dump)

using black on the codecs

8db03c3

github-advanced-security AI found potential problems Feb 4, 2024

View reviewed changes

Comment thread volatility3/framework/layers/codecs/__init__.py Fixed

Comment thread volatility3/framework/layers/codecs/__init__.py Fixed

Comment thread volatility3/framework/layers/codecs/__init__.py Fixed

fixing mistakes in the lz77+huffman decompression algorithm

d152b48

forensicxlab commented Feb 8, 2024

View reviewed changes

k1nd0ne mentioned this pull request Jun 20, 2024

[2024] AArch64 support #1088

Draft

Adding codecs + black

6213d45

github-advanced-security AI found potential problems Jul 15, 2024

View reviewed changes

Comment thread volatility3/framework/layers/codecs/lz77plain.py Fixed

Formatting using good black version

2bebe91

conflict

2ffa3a9

k1nd0ne added 2 commits July 23, 2024 16:44

Merging to resolve conflict

9379e16

requirements

b17eab6

github-advanced-security AI found potential problems Jul 23, 2024

View reviewed changes

Comment thread volatility3/framework/layers/hib.py Fixed

k1nd0ne and others added 6 commits July 23, 2024 17:05

Fixing requirements

3f428f7

Fixing unused import

ad31052

Merge branch 'volatilityfoundation:develop' into feature/hibernation-…

9d478f3

…layer

Upgrading pipeline and python3 version minimum requirement

d779c0e

Upgrading pipeline and python3 version minimum requirement

c947999

Using symbol Tables (1/2)

0d6c1ea

github-advanced-security AI found potential problems Aug 5, 2024

View reviewed changes

Comment thread volatility3/framework/plugins/windows/hibernation.py Fixed

forensicxlab and others added 4 commits August 5, 2024 22:24

Merge branch 'volatilityfoundation:develop' into feature/hibernation-…

f378a1a

…layer

Sync

3b1f2ae

Merge branch 'develop' into feature/hibernation-layer

3778c97

Merge branch 'volatilityfoundation:develop' into feature/hibernation-…

8d9c917

…layer

ikelos reviewed Feb 8, 2025

View reviewed changes

eve-mem mentioned this pull request Oct 7, 2025

Image copy #1887

Closed

Merge branch 'develop' into feature/hibernation-layer

603b53f

Ozer0x777 added 2 commits October 14, 2026 12:59

Ozer0x777 mentioned this pull request Oct 14, 2026

hibernation: Windows 11 24H2+ support (build-dependent counter offset) and a decoder size fix forensicxlab/volatility3#1

Merged

forensicxlab and others added 2 commits October 16, 2026 09:47

Merge pull request #1 from Ozer0x777/hibernation-24h2-autodetect

365e13a

hibernation: Windows 11 24H2+ support (build-dependent counter offset) and a decoder size fix.

Ozer0x777 mentioned this pull request Oct 16, 2026

codecs: drop the unused import that fails the ruff job of #1036 forensicxlab/volatility3#2

Merged

Merge pull request #2 from Ozer0x777/ci-lint-fix

2e12140

codecs: drop the unused import that fails the ruff job of volatilityfoundation#1036


		"""Codecs used for encoding or decoding data should live here

		# Temporary comment:

Conversation

forensicxlab commented Nov 11, 2023

Uh oh!

ikelos commented Nov 15, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

forensicxlab commented Feb 4, 2024

Uh oh!

ikelos commented Feb 8, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ikelos Feb 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

forensicxlab Feb 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

forensicxlab commented Feb 8, 2024

Uh oh!

KamilPacanek commented Mar 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ikelos commented Mar 12, 2024

Uh oh!

Uh oh!

k1nd0ne commented Jul 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

k1nd0ne commented Jul 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ikelos left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

k1nd0ne commented Feb 9, 2025

Uh oh!

ikelos commented Feb 9, 2025

Uh oh!

Ozer0x777 commented Sep 26, 2026

Uh oh!

Ozer0x777 commented Oct 16, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

ikelos Feb 18, 2024 •

edited

Loading

forensicxlab Feb 8, 2024 •

edited

Loading

KamilPacanek commented Mar 11, 2024 •

edited

Loading

k1nd0ne commented Jul 15, 2024 •

edited

Loading

k1nd0ne commented Jul 23, 2024 •

edited

Loading