mirror of
https://github.com/volatilityfoundation/volatility3.git
synced 2026-08-17 20:35:40 +02:00
177 lines
13 KiB
ReStructuredText
177 lines
13 KiB
ReStructuredText
Volatility 3 Basics
|
|
===================
|
|
|
|
Volatility splits memory analysis down to several components. The main ones are:
|
|
|
|
* Memory layers
|
|
* Templates and Objects
|
|
* Symbol Tables
|
|
|
|
Volatility 3 stores all of these within a :py:class:`Context <volatility3.framework.interfaces.context.ContextInterface>`,
|
|
which acts as a container for all the various layers and tables necessary to conduct memory analysis.
|
|
|
|
Memory layers
|
|
-------------
|
|
|
|
A memory layer is a body of data that can be accessed by requesting data at a specific address. At its lowest level
|
|
this data is stored on a phyiscal medium (RAM) and very early computers addressed locations in memory directly. However,
|
|
as the size of memory increased and it became more difficult to manage memory most architectures moved to a "paged" model
|
|
of memory, where the available memory is cut into specific fixed-sized pages. To help further, programs can ask for any address
|
|
and the processor will look up their (virtual) address in a map, to find out where the (physical) address that it lives at is,
|
|
in the actual memory of the system.
|
|
|
|
Volatility can work with these layers as long as it knows the map (so, for example that virtual address `1` looks up at physical
|
|
address `9`). The automagic that runs at the start of every volatility session often locates the kernel's memory map, and creates
|
|
a kernel virtual layer, which allows for kernel addresses to be looked up and the correct data returned. There can, however, be
|
|
several maps, and in general there is a different map for each process (although a portion of the operating system's memory is
|
|
usually mapped to the same location across all processes). The maps may take the same address but point to a different part of
|
|
physical memory. It also means that two processes could theoretically share memory, both having a virtual address mapped to the
|
|
same physical address. See the worked example below for more information.
|
|
|
|
To translate an address on a layer, call :py:meth:`layer.mapping(offset, length, ignore_errors) <volatility3.framework.interfaces.layers.TranslationLayerInterface.mapping>` and it will return a list of chunks without overlap, in order,
|
|
for the requested range. If a portion cannot be mapped, an exception will be thrown unless `ignore_errors` is true. Each
|
|
chunk will contain the original offset of the chunk, the translated offset, the original size and the translated size of
|
|
the chunk, as well as the lower layer the chunk lives within.
|
|
|
|
Worked example
|
|
^^^^^^^^^^^^^^
|
|
|
|
The operating system and two programs may all appear to have access to all of physical memory, but actually the maps they each have
|
|
mean they each see something different:
|
|
|
|
.. code-block::
|
|
:caption: Memory mapping example
|
|
|
|
Operating system map Physical Memory
|
|
1 -> 9 1 - Free
|
|
2 -> 3 2 - OS.4, Process 1.4, Process 2.4
|
|
3 -> 7 3 - OS.2
|
|
4 -> 2 4 - Free
|
|
5 - Free
|
|
Process 1 map 6 - Process 1.2, Process 2.3
|
|
1 -> 12 7 - OS.3
|
|
2 -> 6 8 - Process1.3
|
|
3 -> 8 9 - OS.1
|
|
4 -> 2 10 - Process2.1
|
|
11 - Free
|
|
Process 2 map 12 - Process1.1
|
|
1 -> 10 13 - Free
|
|
2 -> 15 14 - Free
|
|
3 -> 6 15 - Process2.2
|
|
4 -> 2 16 - Free
|
|
|
|
In this example, part of the operating system is visible across all processes (although not all processes can write to the memory, there
|
|
is a permissions model for Intel addressing which is not discussed further here).
|
|
|
|
In Volatility 3 mappings are represented by a directed graph of layers, whose end nodes are
|
|
:py:class:`DataLayers <volatility3.framework.interfaces.layers.DataLayerInterface>` and whose internal nodes are :py:class:`TranslationLayers <volatility3.framework.interfaces.layers.TranslationLayerInterface>`.
|
|
In this way, a raw memory image in the LiME file format and a page file can be combined to form a single Intel virtual
|
|
memory layer. When requesting addresses from the Intel layer, it will use the Intel memory mapping algorithm, along
|
|
with the address of the directory table base or page table map, to translate that
|
|
address into a physical address, which will then either be directed towards the swap layer or the LiME layer. Should it
|
|
be directed towards the LiME layer, the LiME file format algorithm will translate the new address to determine where
|
|
within the file the data is stored. When the :py:meth:`layer.read() <volatility3.framework.interfaces.layers.TranslationLayerInterface.read>`
|
|
method is called, the translation is done automatically and the correct data gathered and combined.
|
|
|
|
.. note:: Volatility 2 had a similar concept, called address spaces, but these could only stack linearly one on top of another.
|
|
|
|
The list of layers supported by Volatility can be determined by running the `frameworkinfo` plugin.
|
|
|
|
Templates and Objects
|
|
---------------------
|
|
|
|
Once we can address contiguous chunks of memory with a means to translate a virtual address (as seen by the programs)
|
|
into the actual data used by the processor, we can start pulling out
|
|
:py:class:`Objects <volatility3.framework.interfaces.objects.ObjectInterface>` by taking a
|
|
:py:class:`~volatility3.framework.interfaces.objects.Template` and constructing
|
|
it on the memory layer at a specific offset. A :py:class:`~volatility3.framework.interfaces.objects.Template` contains
|
|
all the information you can know about the structure of the object without actually being populated by any data.
|
|
As such a :py:class:`~volatility3.framework.interfaces.objects.Template` can tell you the size of a structure and its
|
|
members, how far into the structure a particular member lives and potentially what various values in that field would
|
|
mean, but not what resides in a particular member.
|
|
|
|
Using a :py:class:`~volatility3.framework.interfaces.objects.Template` on a memory layer at a particular offset, an
|
|
:py:class:`Object <volatility3.framework.interfaces.objects.ObjectInterface>` can be constructed. In Volatility 3, once an
|
|
:py:class:`Object <volatility3.framework.interfaces.objects.ObjectInterface>` has been created, the data has been read from the
|
|
layer and is not read again. An object allows its members to be interrogated and in particular allows pointers to be
|
|
followed, providing easy access to the data contained in the object.
|
|
|
|
.. note:: Volatility 2 would re-read the data which was useful for live memory forensics but quite inefficient for the
|
|
more common static memory analysis typically conducted. Volatility 3 requires that objects be manually reconstructed
|
|
if the data may have changed. Volatility 3 also constructs actual Python integers and floats whereas Volatility 2
|
|
created proxy objects which would sometimes cause problems with type checking.
|
|
|
|
Symbol Tables
|
|
-------------
|
|
|
|
Most compiled programs know of their own templates, and define the structure (and location within the program) of these
|
|
templates as a :py:class:`Symbol <volatility3.framework.interfaces.symbols.SymbolInterface>`. A
|
|
:py:class:`Symbol <volatility3.framework.interfaces.symbols.SymbolInterface>` is often an address and a template and can
|
|
be used to refer to either independently. Lookup tables of these symbols are often produced as debugging information
|
|
alongside the compilation of the program. Volatility 3 provides access to these through a
|
|
:py:class:`SymbolTable <volatility3.framework.interfaces.symbols.SymbolTableInterface>`, many of which can be collected
|
|
within a :py:class:`~volatility3.framework.contexts.Context` as a :py:class:`SymbolSpace <volatility.framework.interfaces.symbols.SymbolSpaceInterface>`.
|
|
A :py:class:`~volatility3.framework.contexts.Context` can store only one :py:class:`~volatility.framework.symbols.SymbolSpace`
|
|
at a time, although a :py:class:`~volatility3.framework.symbols.SymbolSpace` can store as
|
|
many :py:class:`~volatility3.framework.symbols.SymbolTable` items as necessary.
|
|
|
|
Volatility 3 uses the de facto naming convention for symbols of `module!symbol` to refer to them. It reads them from its
|
|
own JSON formatted file, which acts as a common intermediary between Windows PDB files, Linux DWARF files, other symbol
|
|
formats and the internal Python format that Volatility 3 uses to represent
|
|
a :py:class:`~volatility3.framework.interfaces.objects.Template` or
|
|
a :py:class:`Symbol <volatility3.framework.interfaces.symbols.SymbolInterface>`.
|
|
|
|
.. note:: Volatility 2's name for a :py:class:`~volatility3.framework.symbols.SymbolSpace` was a profile, but it could
|
|
not differentiate between symbols from different modules and required special handling for 32-bit programs that
|
|
used Wow64 on Windows. This meant that all symbols lived in a single namespace with the possibility of symbol name
|
|
collisions. It read the symbols using a format called *vtypes*, written in Python code directly.
|
|
This made it less transferable or able to be used by other software.
|
|
|
|
Plugins
|
|
-------
|
|
|
|
A plugin acts as a means of requesting data from the user interface (and so the user) and then using it to carry out a
|
|
specific form of analysis on the :py:class:`Context <volatility3.framework.interfaces.context.ContextInterface>`
|
|
(containing whatever symbol tables and memory layers it may). The means of communication between the user interface and
|
|
the library is the configuration tree, which is used by components within the :py:class:`~volatility3.framework.contexts.Context`
|
|
to store configurable data. After the plugin has been run, it then returns the results in a specific format known as a
|
|
:py:class:`~volatility3.framework.interfaces.renderers.TreeGrid`. This ensures that the data can be handled by consumers of
|
|
the library, without knowing exactly what the data is or how it's formatted.
|
|
|
|
Output Renderers
|
|
----------------
|
|
|
|
User interfaces can choose how best to present the output of the results to their users. The library always responds from
|
|
every plugin with a :py:class:`~volatility3.framework.renderers.TreeGrid`, and the user interface can then determine how
|
|
best to display it. For the Command Line Interface, that might be via text output as a table, or it might output to an
|
|
SQLite database or a CSV file. For a web interface, the best output is probably as JSON where it could be displayed as
|
|
a table, or inserted into a database like Elastic Search and trawled using an existing frontend such as Kibana.
|
|
|
|
The renderers only need to know how to process very basic types (booleans, strings, integers, bytes) and a few additional specific
|
|
ones (disassembly and various absent values).
|
|
|
|
Renderers can also be added to volatility automatically. There is an additional arrow/parquet format renderer available (but requires
|
|
the pyarrow dependency to be installed), but this is not shipped with the EXE version because it doubles the size of the executable.
|
|
|
|
Configuration Tree
|
|
------------------
|
|
|
|
The configuration tree acts as the interface between the calling program and Volatility 3 library. Elements of the
|
|
library (such as a :py:class:`Plugin <volatility3.framework.interfaces.plugins.PluginInterface>`,
|
|
a :py:class:`TranslationLayer <volatility3.framework.interfaces.layers.TranslationLayerInterface>`,
|
|
an :py:class:`Automagic <volatility3.framework.interfaces.automagic.AutomagicInterface>`, etc.) can use the configuration
|
|
tree to inform the calling program of the options they require and/or optionally support, and allows the calling program
|
|
to provide that information when the library is then called.
|
|
|
|
Automagic
|
|
---------
|
|
|
|
There are certain setup tasks that establish the context in a way favorable to a plugin before it runs, removing
|
|
several tasks that are repetitive and also easy to get wrong. These are called
|
|
:py:class:`Automagic <volatility3.framework.interfaces.automagic.AutomagicInterface>`, since they do things like magically
|
|
taking a raw memory image and automatically providing the plugin with an appropriate Intel translation layer and an
|
|
accurate symbol table without either the plugin or the calling program having to specify all the necessary details. Automagics are a core component which consumers of the library can call or not at their discretion.
|
|
|
|
.. note:: Volatility 2 used to do this as well, but it wasn't a particularly modular mechanism, and was used only for
|
|
stacking address spaces (rather than identifying profiles), and it couldn't really be disabled/configured easily.
|