This check was both causing an `InvalidAddressException` due to the
member access, while at the same time not being a useful check, since it
prevents VADs from being mapped in children of the `System` process.
This was preventing enumeration of valid processes (confirmed by
disassembly of the start address/Win32 start address). Heuristic-based
filtering should probably be left to consumers of the APIs.
A `SymbolError` can occur when a type contains a pointer to an opaque
type. For example, `_EPROCESS` can have a member that points to an
`_EPROCESS_QUOTA_BLOCK`, but there is no definition for that type, so
its size and readability can't be determined.
This wraps the block in a try/except, and reports that the type has an
unknown size in the suffix if a `SymbolError` occurs.
Improves the naming of a couple of the new extension class methods to
more accurately reflect the return type, and adds docstrings to
extensions class methods.
Updates type hints on some fields of the result namedtuples to be their
`objects.Primitive` types instead of Python primitives, and does any
conversion to Python primitives in the generator methods.
This simplifies the design of these plugins by moving as much MFTEntry
specific data into the extension class (caching attributes, since
they'll need to be accessed repeatedly) and moving away from the
callback-based implementation to one where classmethods consume
`mft.MFTEntry` objects in order to produce their values.
These changes do two important things:
- They allow us to preserve `object.String` objects until the generator
function, which makes the public interface much better since people
can navigate back the the source of the data within their context
- Completely eliminates the `record_map` that was causing so much memory
consumption.
There was a subtle issue that was causing substantial performance issues
in the MFTScan plugins. The `record_map` was purportedly of type
`Dict[str, Tuple[int, str, int]]`, but in reality, the second member was
a list, and its `str` item was actually being populated with unprocessed
values from method calls on the MFT extension classes, which actually
return `object.String`. These objects are substantially larger than
basic `str` types:
```
[ins] In [5]: pympler.asizeof.asizeof(rec_name)
Out[5]: 312648
[ins] In [6]: pympler.asizeof.asizeof(str(rec_name))
Out[6]: 64
```
This caused this dictionary to grow in size to several gigabytes on
larger samples, resulting in thrashing and OOM errors.
The new layer data type renders the output a little differently, and the
plugin also seems to render 'N/A' for a missing value where previously
it was an empty string.
Mike Auty