We now use mapping to skip unmapped pages when scanning. This
should be slightly quicker and likely more accurate. We also now
don't chunk over the size requested by the scanner. We don't quite
handle overlaps correctly yet (we overlap, but don't prevent
duplicates if the pattern is entirely within the overlap segment).
This uses the exceptions to indicate how much can be skipped in the
virtual intel translation layer. This means large holes in the memory
can be skipped more quickly. It also eliminates the is_valid/mapping
loop.
Layers now accept metadata dictionaries (and chain/stack them on top of
those from lower layers). Metadata can only be set at construction
time, and the metadata dictionary is readonly. The hope is this will
make enumerating metadata keys across the codebase simpler.
The current metadata items that layers hold is:
architecture (Unknown | Intel32 | Intel64)
os (Unknown | Windows | Linux)
pae (bool)
page_map_offset (int)
This patchset may develop further to help enumerate all of these
(through a registration/reporting system).
Unfortunately in tests it turned out re was faster than a python
implementation of Wu-Manber. The Wu-Manber code has been left (because
why not) but it's not really advantageous over the re builtin since
it's compiled in C.
This effectively means that the FileTranslationLayer will reuse the
cached file even with multiple opens, but rerunning python, or starting
a new context (with a new FileTranslationLayer but on the same URL) will
cause a redownload whether necessary or not. This ensures that running
volatility as an engine (inside a long lived python session) will not
prevent a file being checked again later.
Other caching mechanisms (such as last-modified) should be used to
determine if the cached file is still valid.
Note this may cause issues if plugins run concurrently.
Mike Auty