WEBDIENSTE/volatility3
1
0
Fork 0
mirror of https://github.com/volatilityfoundation/volatility3.git synced 2026-09-06 09:47:38 +02:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'upstream/develop' into linux_sockstats_plugin

Browse Source
This commit is contained in:
Gustavo Moreira
2022-10-13 16:08:37 +11:00
parent 6a0b6eb20f 9333dab52a
commit f3ef9a6ded
95 changed files with 2538 additions and 669 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/test.yaml
+53
View File
@@ -0,0 +1,53 @@
name: Test Volatility3
on: [push, pull_request]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.6
uses: actions/setup-python@v2
with:
python-version: '3.6'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install Cmake
pip install setuptools wheel
pip install -r ./test/requirements-testing.txt
- name: Build PyPi packages
run: |
python setup.py sdist --formats=gztar,zip
python setup.py bdist_wheel
- name: Download images
run: |
curl -sLO "https://downloads.volatilityfoundation.org/volatility3/images/linux-sample-1.bin.gz"
gunzip linux-sample-1.bin.gz
curl -sLO "https://downloads.volatilityfoundation.org/volatility3/images/win-xp-laptop-2005-06-25.img.gz"
gunzip win-xp-laptop-2005-06-25.img.gz
- name: Download and Extract symbols
run: |
cd ./volatility3/symbols
curl -sLO https://downloads.volatilityfoundation.org/volatility3/symbols/linux.zip
unzip linux.zip
cd -
- name: Testing...
run: |
py.test ./test/test_volatility.py --volatility=vol.py --image win-xp-laptop-2005-06-25.img -k test_windows -v
py.test ./test/test_volatility.py --volatility=vol.py --image linux-sample-1.bin -k test_linux -v
- name: Clean up post-test
run: |
rm -rf *.lime
rm -rf *.img
cd volatility3/symbols
rm -rf linux
rm -rf linux.zip
cd -
.gitignore
+15
View File
@@ -27,3 +27,18 @@ config*.json
# Pyinstaller files
build
dist
# Environments
.env
.venv
env/
venv/
ENV/
# Memory dump files
*.dmp
*.vmem
*.img
# PyTest cache files
.pytest_cache/
.style.yapf
+1 -1
View File
@@ -107,7 +107,7 @@ each_dict_entry_on_separate_line=True
i18n_comment=
# The i18n function call names. The presence of this function stops
# reformattting on that line, because the string it has cannot be moved
# reformatting on that line, because the string it has cannot be moved
# away from the i18n comment.
i18n_function_call=
API_CHANGES.md
+18 -2
View File
@@ -4,13 +4,29 @@ API Changes
When an addition to the existing API is made, the minor version is bumped.
When an API feature or function is removed or changed, the major version is bumped.
2.4.0
=====
Add a `get_size()` method to Windows VAD structures and fix several off-by-one issues when calculating VAD sizes.
2.3.1
=====
Update in the windows `_EPROCESS.owning_process` method to support Windows Vista and later versions.
2.3.0
=====
Add in `child_template` to template class
2.2.0
=====
Changes to linux core calls
2.1.0
=====
Add in the linux `task.get_threads` method added to the API.
Add in the linux `task.get_threads` method to the API.
2.0.3
=====
`DEVICE_OBJECT.get_attached_devices` and `DRIVER_OBJECT.get_devices` added to the API.
Add in the windows `DEVICE_OBJECT.get_attached_devices` and `DRIVER_OBJECT.get_devices` methods to the API.
2.0.2
=====
LICENSE.txt
+1 -1
View File
@@ -31,7 +31,7 @@ If you make any Additions available to others, such as by providing copies of th
- You are responsible to ensure you have rights in Additions necessary to comply with this section.
Contributing
If you contribute (or offer to contribute) any materials to Volatility Foundation for the software, such as by submitting a pull request to the repository for the software or related content run by Volatility Foundation, you agree to contribute them under the under the BSD 2-Clause Plus Patent License (in the case of software) or the Creative Commons Zero Public Domain Dedication (in the case of content), unless you clearly mark them "Not a Contribution."
If you contribute (or offer to contribute) any materials to Volatility Foundation for the software, such as by submitting a pull request to the repository for the software or related content run by Volatility Foundation, you agree to contribute them under the BSD 2-Clause Plus Patent License (in the case of software) or the Creative Commons Zero Public Domain Dedication (in the case of content), unless you clearly mark them "Not a Contribution."
Trademarks
This license grants you no rights to any trademarks or service marks.
README.md
+3
View File
@@ -94,6 +94,9 @@ Symbol tables zip files must be placed, as named, into the `volatility3/symbols`
Windows symbols that cannot be found will be queried, downloaded, generated and cached. Mac and Linux symbol tables must be manually produced by a tool such as [dwarf2json](https://github.com/volatilityfoundation/dwarf2json).
Important: The first run of volatility with new symbol files will require the cache to be updated. The symbol packs contain a large number of symbol files and so may take some time to update!
However, this process only needs to be run once on each new symbol file, so assuming the pack stays in the same location will not need to be done again. Please also note it can be interrupted and next run will restart itself.
Please note: These are representative and are complete up to the point of creation for Windows and Mac. Due to the ease of compiling Linux kernels and the inability to uniquely distinguish them, an exhaustive set of Linux symbol tables cannot easily be supplied.
## Documentation
doc/source/simple-plugin.rst
+104 -53
View File
@@ -6,6 +6,12 @@ This guide will step through how to construct a simple plugin using Volatility 3
The example plugin we'll use is :py:class:`~volatility3.plugins.windows.dlllist.DllList`, which features the main traits
of a normal plugin, and reuses other plugins appropriately.
.. note::
This document will not include the complete code necessary for a
working plugin (such as imports, etc) since it's designed to focus on the necessary components for writing a plugin.
For complete and functioning plugins, the ``framework/plugins`` directory should be consulted.
Inherit from PluginInterface
----------------------------
@@ -30,20 +36,20 @@ to be able to run properly. Any that are defined as optional need not necessari
::
_version = (1, 0, 0)
_required_framework_version = (2, 0, 0)
@classmethod
def get_requirements(cls):
return [requirements.TranslationLayerRequirement(name = 'primary',
description = 'Memory layer for the kernel',
architectures = ["Intel32", "Intel64"]),
requirements.SymbolTableRequirement(name = "nt_symbols",
description = "Windows kernel symbols"),
requirements.PluginRequirement(name = 'pslist',
plugin = pslist.PsList,
version = (1, 0, 0)),
return [requirements.ModuleRequirement(name = 'kernel', description = 'Windows kernel',
architectures = ["Intel32", "Intel64"]),
requirements.ListRequirement(name = 'pid',
element_type = int,
description = "Process IDs to include (all other processes are excluded)",
optional = True)]
optional = True),
requirements.PluginRequirement(name = 'pslist',
plugin = pslist.PsList,
version = (2, 0, 0))]
This is a classmethod, because it is called before the specific plugin object has been instantiated (in order to know how
@@ -51,69 +57,112 @@ to instantiate the plugin). At the moment these requirements are fairly straigh
::
requirements.TranslationLayerRequirement(name = 'primary',
description = 'Memory layer for the kernel',
architectures = ["Intel32", "Intel64"]),
requirements.ModuleRequirement(name = 'kernel', description = 'Windows kernel',
architectures = ["Intel32", "Intel64"]),
This requirement indicates that the plugin will operate on a single
:py:class:`TranslationLayer <volatility3.framework.interfaces.layers.TranslationLayerInterface>`. The name of the
loaded layer will appear in the plugin's configuration under the name ``primary``. Requirement values can be
accessed within the plugin through the plugin's `config` attribute (for example ``self.config['pid']``).
This requirement specifies the need for a particular submodule. Each module requires a
:py:class:`TranslationLayer <volatility3.framework.interfaces.layers.TranslationLayerInterface>` and a
:py:class:`SymbolTable <volatility3.framework.interfaces.symbols.SymbolTableInterface>`, which are fulfilled by two
subrequirements: a
:py:class:`~volatility3.framework.configuration.requirements.TranslationLayerRequirement` and a
:py:class:`~volatility3.framework.configuration.requirements.SymbolTableRequirement`. At the moment, the automagic
only fills `ModuleRequirements` with kernels, and so has relatively few parameters. It requires the architecture for
the underlying TranslationLayer, and the offset of the module within that layer.
.. note:: The name itself is dynamic depending on the other layers already present in the Context. Always use the value
from the configuration rather than attempting to guess what the layer will be called.
The name of the module will be stored in the ``kernel`` configuration option, and the module object itself
can be accessed from the ``context.modules`` collection. This requirement is a Complex Requirement and therefore will
not be requested directly from the user.
Finally, this defines that the translation layer must be on the Intel Architecture. At the moment, this acts as a filter,
failing to be satisfied by memory images that do not match the architecture required.
Most plugins will only operate on a single layer, but it is entirely possible for a plugin to request two different
layers, for example a plugin that carries out some form of difference or statistics against multiple memory images.
.. note::
This requirement (and the next two) are known as Complex Requirements, and user interfaces will likely not directly
request a value for this from a user. The value stored in the configuration tree for a
:py:class:`~volatility3.framework.configuration.requirements.TranslationLayerRequirement` is
the string name of a layer present in the context's memory that satisfies the requirement.
In previous versions of volatility 3, there was no `ModuleRequirement`, and instead two requirements were defined
a :py:class:`TranslationLayer <volatility3.framework.interfaces.layers.TranslationLayerInterface>` and a `SymbolTableRequirement`. These still exist, and can be used, most plugins just
define a single `ModuleRequirement` for the kernel, which the automagic will populate. The `ModuleRequirement` has
two automatic sub-requirements, a `TranslationLayerRequirement` and a `SymbolTableRequirement`, but the module also
includes the offset of the module, and will allow future expansion to specify specific modules when application
level plugins become more common. Below are how the requirements would be specified:
::
::
requirements.SymbolTableRequirement(name = "nt_symbols",
description = "Windows kernel symbols"),
requirements.TranslationLayerRequirement(name = 'primary',
description = 'Memory layer for the kernel',
architectures = ["Intel32", "Intel64"]),
This requirement specifies the need for a particular
:py:class:`SymbolTable <volatility3.framework.interfaces.symbols.SymbolTableInterface>`
to be loaded. This gets populated by various
:py:class:`Automagic <volatility3.framework.interfaces.automagic.AutoMagicInterface>` as the nearest sibling to a particular
:py:class:`~volatility3.framework.configuration.requirements.TranslationLayerRequirement`.
This means that if the :py:class:`~volatility3.framework.configuration.requirements.TranslationLayerRequirement`
is satisfied and the :py:class:`Automagic <volatility3.framework.interfaces.automagic.AutoMagicInterface>` can determine
the appropriate :py:class:`SymbolTable <volatility3.framework.interfaces.symbols.SymbolTableInterface>`, the
name of the :py:class:`SymbolTable <volatility3.framework.interfaces.symbols.SymbolTableInterface>` will be stored in the configuration.
This requirement indicates that the plugin will operate on a single
:py:class:`TranslationLayer <volatility3.framework.interfaces.layers.TranslationLayerInterface>`. The name of the
loaded layer will appear in the plugin's configuration under the name ``primary``. Requirement values can be
accessed within the plugin through the plugin's `config` attribute (for example ``self.config['pid']``).
This requirement is also a Complex Requirement and therefore will not be requested directly from the user.
.. note:: The name itself is dynamic depending on the other layers already present in the Context. Always use the value
from the configuration rather than attempting to guess what the layer will be called.
::
Finally, this defines that the translation layer must be on the Intel Architecture. At the moment, this acts as a filter,
failing to be satisfied by memory images that do not match the architecture required.
requirements.PluginRequirement(name = 'pslist',
plugin = pslist.PsList,
version = (1, 0, 0)),
Most plugins will only operate on a single layer, but it is entirely possible for a plugin to request two different
layers, for example a plugin that carries out some form of difference or statistics against multiple memory images.
This requirement indicates that the plugin will make use of another plugin's code, and specifies the version requirements
on that plugin. The version is specified in terms of Semantic Versioning, meaning that to be compatible, the major
versions must be identical and the minor version must be equal to or higher than the one provided. This requirement
does not make use of any data from the configuration, even if it were provided, it is merely a functional check before
running the plugin.
This requirement (and the next two) are known as Complex Requirements, and user interfaces will likely not directly
request a value for this from a user. The value stored in the configuration tree for a
:py:class:`~volatility3.framework.configuration.requirements.TranslationLayerRequirement` is
the string name of a layer present in the context's memory that satisfies the requirement.
::
requirements.SymbolTableRequirement(name = "nt_symbols",
description = "Windows kernel symbols"),
This requirement specifies the need for a particular
:py:class:`SymbolTable <volatility3.framework.interfaces.symbols.SymbolTableInterface>`
to be loaded. This gets populated by various
:py:class:`Automagic <volatility3.framework.interfaces.automagic.AutoMagicInterface>` as the nearest sibling to a particular
:py:class:`~volatility3.framework.configuration.requirements.TranslationLayerRequirement`.
This means that if the :py:class:`~volatility3.framework.configuration.requirements.TranslationLayerRequirement`
is satisfied and the :py:class:`Automagic <volatility3.framework.interfaces.automagic.AutoMagicInterface>` can determine
the appropriate :py:class:`SymbolTable <volatility3.framework.interfaces.symbols.SymbolTableInterface>`, the
name of the :py:class:`SymbolTable <volatility3.framework.interfaces.symbols.SymbolTableInterface>` will be stored in the configuration.
This requirement is also a Complex Requirement and therefore will not be requested directly from the user.
::
requirements.ListRequirement(name = 'pid',
description = 'Filter on specific process IDs',
element_type = int,
optional = True)
optional = True),
The final requirement is a List Requirement, populated by integers. The description will be presented to the user to
The next requirement is a List Requirement, populated by integers. The description will be presented to the user to
describe what the value represents. The optional flag indicates that the plugin can function without the ``pid`` value
being defined within the configuration tree at all.
::
requirements.PluginRequirement(name = 'pslist',
plugin = pslist.PsList,
version = (2, 0, 0))]
This requirement indicates that the plugin will make use of another plugin's code, and specifies the version requirements
on that plugin. The version is specified in terms of Semantic Versioning meaning that, to be compatible, the major
versions must be identical and the minor version must be equal to or higher than the one provided. This requirement
does not make use of any data from the configuration, even if it were provided, it is merely a functional check before
running the plugin. To define the version of a plugin, populate the `_version` class variable as a tuple of version
numbers `(major, minor, patch)`. So for example:
::
_version = (1, 0, 0)
The plugin may also require a specific version of the framework, and this also uses Semantic Versioning, and can be
set by defining the `_required_framework_version`. The major version should match the version of volatility the plugin
is to be used with, which at the time of writing would be 2.2.0, and so would be specified as below. If only features, for example,
from 2.0.0 are used, then the lowest applicable version number should be used to support the greatest number of
installations:
::
_required_framework_version = (2, 0, 0)
Define the `run` method
-----------------------
@@ -129,6 +178,7 @@ that will be output as part of the :py:class:`~volatility3.framework.interfaces.
def run(self):
filter_func = pslist.PsList.create_pid_filter(self.config.get('pid', None))
kernel = self.context.modules[self.config['kernel']]
return renderers.TreeGrid([("PID", int),
("Process", str),
@@ -137,8 +187,8 @@ that will be output as part of the :py:class:`~volatility3.framework.interfaces.
("Name", str),
("Path", str)],
self._generator(pslist.PsList.list_processes(self.context,
self.config['primary'],
self.config['nt_symbols'],
kernel.layer_name,
kernel.symbol_table_name,
filter_func = filter_func)))
In this instance, the plugin constructs a filter (using the PsList plugin's *classmethod* for creating filters).
@@ -157,7 +207,8 @@ the :py:class:`~volatility3.plugins.windows.pslist.PsList` plugin. That plugin
so that other plugins can call it. As such, it takes all the necessary parameters rather than accessing them
from a configuration. Since it must be portable code, it takes a context, as well as the layer name,
symbol table and optionally a filter. In this instance we unconditionally
pass it the values from the configuration for the ``primary`` and ``nt_symbols`` requirements. This will generate a list
pass it the values from the configuration for the layer and symbol table from the kernel module object, constructed from
the ``kernel`` configuration requirement. This will generate a list
of :py:class:`~volatility3.framework.symbols.windows.extensions.EPROCESS` objects, as provided by the :py:class:`~volatility.plugins.windows.pslist.PsList` plugin,
and is not covered here but is used as an example for how to share code across plugins
(both as the provider and the consumer of the shared code).
doc/source/symbol-tables.rst
+13 -12
View File
@@ -12,20 +12,22 @@ Volatility will automatically decompress them on use. It will also cache their
under the user's home directory, in :file:`.cache/volatility3`, along with other useful data. The cache directory currently
cannot be altered.
Symbol table JSON files live, by default, under the :file:`volatility3/symbols`, underneath an operating system directory
(currently one of :file:`windows`, :file:`mac` or :file:`linux`). The symbols directory is configurable within the framework and can
usually be set within the user interface.
Symbol table JSON files live, by default, under the :file:`volatility3/symbols` directory. The symbols directory is
configurable within the framework and can usually be set within the user interface.
These files can also be compressed into ZIP files, which Volatility will process in order to locate symbol files.
The ZIP file must be named after the appropriate operating system (such as `linux.zip`, `mac.zip` or `windows.zip`).
Inside the ZIP file, the directory structure should match the uncompressed operating system directory.
Volatility maintains a cache mapping the appropriate identifier for each symbol file against its filename. This cache
is updated by automagic called as part of the standard automagic that's run each time a plugin is run. If a large number of new
symbols file are detected, this may take some time, but can be safely interrupted and restarted and will not need to run again
as long as the symbol files stay in the same location.
Windows symbol tables
---------------------
For Windows systems, Volatility accepts a string made up of the GUID and Age of the required PDB file. It then
searches all files under the configured symbol directories under the windows subdirectory. Any that match the filename
pattern of :file:`<pdb-name>/<GUID>-<AGE>.json` (or any compressed variant) will be used. If such a symbol table cannot be found, then
searches all files under the configured symbol directories under the windows subdirectory. Any that contain metadata
which matches the pdb name and GUID/age (or any compressed variant) will be used. If such a symbol table cannot be found, then
the associated PDB file will be downloaded from Microsoft's Symbol Server and converted into the appropriate JSON
format, and will be saved in the correct location.
@@ -41,11 +43,10 @@ or a virtual environment.
Mac/Linux symbol tables
-----------------------
For Mac/Linux systems, both use the same mechanism for identification. JSON files live under the symbol directories,
under either the :file:`linux` or :file:`mac` directories. The generated files contain an identifying string (the operating system
For Mac/Linux systems, both use the same mechanism for identification. The generated files contain an identifying string (the operating system
banner), which Volatility's automagic can detect. Volatility caches the mapping between the strings and the symbol
tables they come from, meaning the precise file names don't matter and can be organized under any necessary hierarchy
under the operating system directory.
under the symbols directory.
Linux and Mac symbol tables can be generated from a DWARF file using a tool called `dwarf2json <https://github.com/volatilityfoundation/dwarf2json>`_.
Currently a kernel with debugging symbols is the only suitable means for recovering all the information required by
@@ -63,7 +64,7 @@ To determine the string for a particular memory image, use the `banners` plugin.
try to locate that exact kernel debugging package for the operating system. Unfortunately each distribution provides
its debugging packages under different package names and there are so many that the distribution may not keep all old
versions of the debugging symbols, and therefore **it may not be possible to find the right symbols to analyze a linux
memory image with volatlity**. With Macs there are far fewer kernels and only one distribution, making it easier to
memory image with volatility**. With Macs there are far fewer kernels and only one distribution, making it easier to
ensure that the right symbols can be found.
Once a kernel with debugging symbols/appropriate DWARF file has been located, `dwarf2json <https://github.com/volatilityfoundation/dwarf2json>`_ will convert it into an
@@ -93,4 +94,4 @@ file, the banners must match exactly (down to the compilation date).
* Copy the `.json` file to the symbols directory into `[symbols directory]/linux`
* For Mac change `linux` to `mac`
* For Mac change `linux` to `mac`
doc/source/vol-cli.rst
+23 -1
View File
@@ -9,7 +9,11 @@ Synopsis
**volatility** [-h] [-c CONFIG] [--parallelism [{processes,threads,off}]]
[-e EXTEND] [-p PLUGIN_DIRS] [-s SYMBOL_DIRS] [-v] [-l LOG]
[-o OUTPUT_DIR] [-q] [-r RENDERER] [-f FILE]
[--write-config] [--single-location SINGLE_LOCATION]
[--write-config] [--save-config SAVE_CONFIG]
[--clear-cache] [--cache-path CACHE_PATH]
[--offline]
[--single-location SINGLE_LOCATION]
[--stackers [STACKERS ...]]
[--single-swap-locations SINGLE_SWAP_LOCATIONS]
<plugin> ...
@@ -98,6 +102,10 @@ Options
attempt to build upon, and can be considered the input for the program.
--write-config
*Deprecated*
Use of `--write-config` has been deprecated, replaced by `--save-config`
--save-config
This flag specifies that volatility should write or overwrite a file
called config.json in the current directory. The file will contain
the necessary JSON configuration to recreate the environment that the
@@ -105,11 +113,25 @@ Options
other plugins, but there's no guarantee that plugins use the same
configuration options.
--clear-cache
Clears out all short-term cached items.
--cache-path
Change the default path used to store the cache.
--offline
Do not search online for additional JSON files.
Run offline mode (defaults to false) and for
remote windows symbol tables, linux/mac banner repositories.
--single-location SINGLE_LOCATION
This specifies a URL which will be downloaded if necessary, and built
upon by the automagic and, since most plugins require a single memory
image, can be considered the input for the program.
--stackers STACKERS
Creates the list of stackers to use based on the config option.
--single-swap-locations SINGLE_SWAP_LOCATIONS
A comma-separated list of swap files to be considered as part of the
memory image specified by the single-location or file parameters.
doc/source/volshell.rst
+1 -1
View File
@@ -110,7 +110,7 @@ This means that pointers do not need to be explicitly dereferenced to access und
Running plugins
---------------
It's possible to run any plugin by importing it appropriately and passing it to the `display_plugin_ouptut` or `dpo`
It's possible to run any plugin by importing it appropriately and passing it to the `display_plugin_output` or `dpo`
method. In the following example we'll provide no additional parameters. Volatility will show us which parameters
were required:
requirements-dev.txt
+26
View File
@@ -0,0 +1,26 @@
# The following packages are required for core functionality.
pefile>=2017.8.1
# The following packages are optional.
# If certain packages are not necessary, place a comment (#) at the start of the line.
# This is required for the yara plugins
yara-python>=3.8.0
# This is required for several plugins that perform malware analysis and disassemble code.
# It can also improve accuracy of Windows 8 and later memory samples.
capstone>=3.0.5
# This is required by plugins that decrypt passwords, password hashes, etc.
pycryptodome
# This can improve error messages regarding improperly configured ISF files,
# but is only recommended for development
# jsonschema>=2.3.0
# This is required for memory acquisition via leechcore/pcileech.
leechcorepyc>=2.4.0
# This is required for analyzing Linux samples compressed using AVMLs native
# compression format. It is not required for AVML's standard LiME compression.
python-snappy==0.6.0
requirements.txt
-3
View File
@@ -14,9 +14,6 @@ capstone>=3.0.5
# This is required by plugins that decrypt passwords, password hashes, etc.
pycryptodome
# This can improve error messages regarding improperly configured ISF files.
jsonschema>=2.3.0
# This is required for memory acquisition via leechcore/pcileech.
leechcorepyc>=2.4.0
test/README.md
+34
View File
@@ -0,0 +1,34 @@
# Volatility 3 Testing Framework
## Requirements
The Volatility 3 Testing Framework requires the same version of Python as Volatility3 itself. To install the current set of dependencies that the framework requires, use a command like this:
```shell
pip3 install -r requirements-testing.txt
```
NOTE: `requirements-testing.txt` can be found in this current `test/` directory.
## Quick Start: Manual Testing
1. To test Volatility 3 on an image, first download one with a command such as:
```shell
curl -sLO "https://downloads.volatilityfoundation.org/volatility3/images/win-xp-laptop-2005-06-25.img.gz"
gunzip win-xp-laptop-2005-06-25.img.gz
```
2. In many cases, more symbols are required to be downloaded to the `./volatility3/symbols` directory.
3. To manually run the tests, run a command, such as:
```shell
py.test ./test/test_volatility.py --volatility=vol.py --image win-xp-laptop-2005-06-25.img -k test_windows
```
The above command runs all available tests for windows on the `win-xp-laptop-2005-06-25.img` image. To choose a more specific set of tests, change the phrase after `-k` in this command.
## Github Actions
This framework currently tests two images (one linux image and one windows image) after every push on any branch. For more information/context, find the actions setup in `./github/workflows/test.yaml`
test/conftest.py
+40
View File
@@ -0,0 +1,40 @@
# This file is used to augment the test configuration
import os
import pytest
def pytest_addoption(parser):
parser.addoption("--volatility", action="store", default=None,
required=True,
help="path to the volatility script")
parser.addoption("--python", action="store", default="python3",
help="The name of the interpreter to use when running the volatility script")
parser.addoption("--image", action="append", default=[],
help="path to an image to test")
parser.addoption("--image-dir", action="append", default=[],
help="path to a directory containing images to test")
def pytest_generate_tests(metafunc):
"""Parameterize tests based on image names"""
images = metafunc.config.getoption('image')
for image_dir in metafunc.config.getoption('image_dir'):
images = images + [os.path.join(image_dir, dir) for dir in os.listdir(image_dir)]
# tests with "image" parameter are run against images
if 'image' in metafunc.fixturenames:
metafunc.parametrize("image",
images,
ids=[os.path.basename(image) for image in images])
# Fixtures
@pytest.fixture
def volatility(request):
return request.config.getoption("--volatility")
@pytest.fixture
def python(request):
return request.config.getoption("--python")
test/known_files.json
+19
View File
@@ -0,0 +1,19 @@
{
"windows_dumpfiles": {
"win-xp-laptop-2005-06-25.img": {
"0x82220e78": [
"9bdd5532286f1660f3778e68bc36efe6",
"e3bc1e9e7370e3b5a661ebe591ecf4ec"
],
"0x82350bf8": [
"e5c5e8d97b6280745b41f6572c85d1f0",
"8589f1463422884dbf1411aaad278465"
],
"0x81eaf418": [
"f7a1ae2060a58f8470b97affdb46dccf",
"54fd611021fa784912530b8007545986"
],
"0x820588e8": "458efbc8fdb859488a6ab2b200cce809"
}
}
}
test/requirements-testing.txt
+10
View File
@@ -0,0 +1,10 @@
# These packages are required for core functionality.
pefile>=2017.8.1 #foo
# The following packages are optional.
# If certain packages are not necessary, place a comment (#) at the start of the line.
# This is required for the yara plugins
yara-python>=3.8.0
pytest>=7.0.0
test/test_volatility.py
+384
View File
@@ -0,0 +1,384 @@
# volatility3 tests
#
#
# IMPORTS
#
import os
import subprocess
import sys
import shutil
import tempfile
import hashlib
import ntpath
import json
#
# HELPER FUNCTIONS
#
def runvol(args, volatility, python):
volpy = volatility
python_cmd = python
cmd = [python_cmd, volpy] + args
print(" ".join(cmd))
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = p.communicate()
print("stdout:")
sys.stdout.write(str(stdout))
print("")
print("stderr:")
sys.stdout.write(str(stderr))
print("")
return p.returncode, stdout, stderr
def runvol_plugin(plugin, img, volatility, python, pluginargs=[], globalargs=[]):
args = globalargs + [
"--single-location",
img,
"-q",
plugin,
] + pluginargs
return runvol(args, volatility, python)
#
# TESTS
#
# WINDOWS
def test_windows_pslist(image, volatility, python):
rc, out, err = runvol_plugin("windows.pslist.PsList", image, volatility, python)
out = out.lower()
assert out.find(b"system") != -1
assert out.find(b"csrss.exe") != -1
assert out.find(b"svchost.exe") != -1
assert out.count(b"\n") > 10
assert rc == 0
rc, out, err = runvol_plugin(
"windows.pslist.PsList", image, volatility, python, pluginargs=["--pid", "4"])
out = out.lower()
assert out.find(b"system") != -1
assert out.count(b"\n") < 10
assert rc == 0
def test_windows_psscan(image, volatility, python):
rc, out, err = runvol_plugin("windows.psscan.PsScan", image, volatility, python)
out = out.lower()
assert out.find(b"system") != -1
assert out.find(b"csrss.exe") != -1
assert out.find(b"svchost.exe") != -1
assert out.count(b"\n") > 10
assert rc == 0
def test_windows_dlllist(image, volatility, python):
rc, out, err = runvol_plugin("windows.dlllist.DllList", image, volatility, python)
out = out.lower()
assert out.count(b"\n") > 10
assert rc == 0
def test_windows_modules(image, volatility, python):
rc, out, err = runvol_plugin("windows.modules.Modules", image, volatility, python)
out = out.lower()
assert out.count(b"\n") > 10
assert rc == 0
def test_windows_hivelist(image, volatility, python):
rc, out, err = runvol_plugin("windows.registry.hivelist.HiveList", image, volatility, python)
out = out.lower()
not_xp = out.find(b"\\systemroot\\system32\\config\\software")
if not_xp == -1:
assert out.find(b"\\device\\harddiskvolume1\\windows\\system32\\config\\software") != -1
assert out.count(b"\n") > 10
assert rc == 0
def test_windows_dumpfiles(image, volatility, python):
json_file = open('./test/known_files.json')
known_files = json.load(json_file)
failed_chksms = 0
if sys.platform == 'win32':
file_name = ntpath.basename(image)
else:
file_name = os.path.basename(image)
try:
for addr in known_files["windows_dumpfiles"][file_name]:
path = tempfile.mkdtemp()
rc, out, err = runvol_plugin("windows.dumpfiles.DumpFiles", image, volatility, python, globalargs=["-o", path], pluginargs=["--virtaddr", addr])
for file in os.listdir(path):
with open(os.path.join(path, file), "rb") as fp:
if hashlib.md5(fp.read()).hexdigest() not in known_files["windows_dumpfiles"][file_name][addr]:
failed_chksms += 1
shutil.rmtree(path)
json_file.close()
assert failed_chksms == 0
assert rc == 0
except Exception as e:
json_file.close()
print("Key Error raised on " + str(e))
assert False
def test_windows_handles(image, volatility, python):
rc, out, err = runvol_plugin(
"windows.handles.Handles", image, volatility, python, pluginargs=["--pid", "4"])
assert out.find(b"System Pid 4") != -1
assert out.find(b"MACHINE\\SYSTEM\\CONTROLSET001\\CONTROL\\SESSION MANAGER\\MEMORY MANAGEMENT\\PREFETCHPARAMETERS") != -1
assert out.find(b"MACHINE\\SYSTEM\\SETUP") != -1
assert out.count(b"\n") > 500
assert rc == 0
def test_windows_svcscan(image, volatility, python):
rc, out, err = runvol_plugin("windows.svcscan.SvcScan", image, volatility, python)
assert out.find(b"Microsoft ACPI Driver") != -1
assert out.count(b"\n") > 250
assert rc == 0
def test_windows_privileges(image, volatility, python):
rc, out, err = runvol_plugin(
"windows.privileges.Privs", image, volatility, python, pluginargs=["--pid", "4"])
assert out.find(b"SeCreateTokenPrivilege") != -1
assert out.find(b"SeCreateGlobalPrivilege") != -1
assert out.find(b"SeAssignPrimaryTokenPrivilege") != -1
assert out.count(b"\n") > 20
assert rc == 0
def test_windows_getsids(image, volatility, python):
rc, out, err = runvol_plugin(
"windows.getsids.GetSIDs", image, volatility, python, pluginargs=["--pid", "4"])
assert out.find(b"Local System") != -1
assert out.find(b"Administrators") != -1
assert out.find(b"Everyone") != -1
assert out.find(b"Authenticated Users") != -1
assert rc == 0
def test_windows_envars(image, volatility, python):
rc, out, err = runvol_plugin("windows.envars.Envars", image, volatility, python)
assert out.find(b"PATH") != -1
assert out.find(b"PROCESSOR_ARCHITECTURE") != -1
assert out.find(b"USERNAME") != -1
assert out.find(b"SystemRoot") != -1
assert out.find(b"CommonProgramFiles") != -1
assert out.count(b"\n") > 500
assert rc == 0
def test_windows_callbacks(image, volatility, python):
rc, out, err = runvol_plugin("windows.callbacks.Callbacks", image, volatility, python)
assert out.find(b"PspCreateProcessNotifyRoutine") != -1
assert out.find(b"KeBugCheckCallbackListHead") != -1
assert out.find(b"KeBugCheckReasonCallbackListHead") != -1
assert out.count(b"KeBugCheckReasonCallbackListHead ") > 5
assert rc == 0
def test_windows_devicetree(image, volatility, python):
rc, out, err = runvol_plugin("windows.devicetree.DeviceTree", image, volatility, python)
assert out.find(b"DEV") != -1
assert out.find(b"DRV") != -1
assert out.find(b"ATT") != -1
assert out.find(b"FILE_DEVICE_CONTROLLER") != -1
assert out.find(b"FILE_DEVICE_DISK") != -1
assert out.find(b"FILE_DEVICE_DISK_FILE_SYSTEM") != -1
assert rc == 0
# LINUX
def test_linux_pslist(image, volatility, python):
rc, out, err = runvol_plugin("linux.pslist.PsList", image, volatility, python)
out = out.lower()
assert ((out.find(b"init") != -1) or (out.find(b"systemd") != -1))
assert out.find(b"watchdog") != -1
assert out.count(b"\n") > 10
assert rc == 0
def test_linux_check_idt(image, volatility, python):
rc, out, err = runvol_plugin("linux.check_idt.Check_idt", image, volatility, python)
out = out.lower()
assert out.count(b"__kernel__") >= 10
assert out.count(b"\n") > 10
assert rc == 0
def test_linux_check_syscall(image, volatility, python):
rc, out, err = runvol_plugin("linux.check_syscall.Check_syscall", image, volatility, python)
out = out.lower()
assert out.find(b"sys_close") != -1
assert out.find(b"sys_open") != -1
assert out.count(b"\n") > 100
assert rc == 0
def test_linux_lsmod(image, volatility, python):
rc, out, err = runvol_plugin("linux.lsmod.Lsmod", image, volatility, python)
out = out.lower()
assert out.count(b"\n") > 10
assert rc == 0
def test_linux_lsof(image, volatility, python):
rc, out, err = runvol_plugin("linux.lsof.Lsof", image, volatility, python)
out = out.lower()
assert out.count(b"socket:") >= 10
assert out.count(b"\n") > 35
assert rc == 0
def test_linux_proc_maps(image, volatility, python):
rc, out, err = runvol_plugin("linux.proc.Maps", image, volatility, python)
out = out.lower()
assert out.count(b"anonymous mapping") >= 10
assert out.count(b"\n") > 100
assert rc == 0
def test_linux_tty_check(image, volatility, python):
rc, out, err = runvol_plugin("linux.tty_check.tty_check", image, volatility, python)
out = out.lower()
assert out.find(b"__kernel__") != -1
assert out.count(b"\n") >= 5
assert rc == 0
# MAC
def test_mac_pslist(image, volatility, python):
rc, out, err = runvol_plugin("mac.pslist.PsList", image, volatility, python)
out = out.lower()
assert ((out.find(b"kernel_task") != -1) or (out.find(b"launchd") != -1))
assert out.count(b"\n") > 10
assert rc == 0
def test_mac_check_syscall(image, volatility, python):
rc, out, err = runvol_plugin("mac.check_syscall.Check_syscall", image, volatility, python)
out = out.lower()
assert out.find(b"chmod") != -1
assert out.find(b"chown") != -1
assert out.find(b"nosys") != -1
assert out.count(b"\n") > 100
assert rc == 0
def test_mac_check_sysctl(image, volatility, python):
rc, out, err = runvol_plugin("mac.check_sysctl.Check_sysctl", image, volatility, python)
out = out.lower()
assert out.find(b"__kernel__") != -1
assert out.count(b"\n") > 250
assert rc == 0
def test_mac_check_trap_table(image, volatility, python):
rc, out, err = runvol_plugin("mac.check_trap_table.Check_trap_table", image, volatility, python)
out = out.lower()
assert out.count(b"kern_invalid") >= 10
assert out.count(b"\n") > 50
assert rc == 0
def test_mac_ifconfig(image, volatility, python):
rc, out, err = runvol_plugin("mac.ifconfig.Ifconfig", image, volatility, python)
out = out.lower()
assert out.find(b"127.0.0.1") != -1
assert out.find(b"false") != -1
assert out.count(b"\n") > 9
assert rc == 0
def test_mac_lsmod(image, volatility, python):
rc, out, err = runvol_plugin("mac.lsmod.Lsmod", image, volatility, python)
out = out.lower()
assert out.find(b"com.apple") != -1
assert out.count(b"\n") > 10
assert rc == 0
def test_mac_lsof(image, volatility, python):
rc, out, err = runvol_plugin("mac.lsof.Lsof", image, volatility, python)
out = out.lower()
assert out.count(b"\n") > 50
assert rc == 0
def test_mac_malfind(image, volatility, python):
rc, out, err = runvol_plugin("mac.malfind.Malfind", image, volatility, python)
out = out.lower()
assert out.count(b"\n") > 20
assert rc == 0
def test_mac_mount(image, volatility, python):
rc, out, err = runvol_plugin("mac.mount.Mount", image, volatility, python)
out = out.lower()
assert out.find(b"/dev") != -1
assert out.count(b"\n") > 7
assert rc == 0
def test_mac_netstat(image, volatility, python):
rc, out, err = runvol_plugin("mac.netstat.Netstat", image, volatility, python)
assert out.find(b"TCP") != -1
assert out.find(b"UDP") != -1
assert out.find(b"UNIX") != -1
assert out.count(b"\n") > 10
assert rc == 0
def test_mac_proc_maps(image, volatility, python):
rc, out, err = runvol_plugin("mac.proc_maps.Maps", image, volatility, python)
out = out.lower()
assert out.find(b"[heap]") != -1
assert out.count(b"\n") > 100
assert rc == 0
def test_mac_psaux(image, volatility, python):
rc, out, err = runvol_plugin("mac.psaux.Psaux", image, volatility, python)
out = out.lower()
assert out.find(b"executable_path") != -1
assert out.count(b"\n") > 50
assert rc == 0
def test_mac_socket_filters(image, volatility, python):
rc, out, err = runvol_plugin("mac.socket_filters.Socket_filters", image, volatility, python)
out = out.lower()
assert out.count(b"\n") > 9
assert rc == 0
def test_mac_timers(image, volatility, python):
rc, out, err = runvol_plugin("mac.timers.Timers", image, volatility, python)
out = out.lower()
assert out.count(b"\n") > 6
assert rc == 0
def test_mac_trustedbsd(image, volatility, python):
rc, out, err = runvol_plugin("mac.trustedbsd.Trustedbsd", image, volatility, python)
out = out.lower()
assert out.count(b"\n") > 10
assert rc == 0
vol.spec
+1 -1
View File
@@ -26,7 +26,7 @@ except ImportError:
# Volatility must be findable in sys.path in order for collect_submodules to work
# This adds the current working directory, which should usually do the trick
sys.path.append(os.getcwd())
sys.path.append(os.path.dirname(os.path.abspath(SPEC)))
vol_analysis = Analysis(['vol.py'],
pathex = [],
volatility3/__init__.py
+3 -3
View File
@@ -37,9 +37,9 @@ class WarningFindSpec(abc.MetaPathFinder):
first."""
if fullname.startswith("volatility3.framework.plugins."):
warning = "Please do not use the volatility3.framework.plugins namespace directly, only use volatility3.plugins"
# Pyinstaller uses walk_packages to import, but needs to read the modules to figure out dependencies
# As such, we only print the warning when directly imported rather than from within walk_packages
if inspect.stack()[-2].function != 'walk_packages':
# Pyinstaller uses walk_packages/_collect_submodules to import, but needs to read the modules to figure out dependencies
# As such, we only print the warning when directly imported rather than from within walk_packages/_collect_submodules
if inspect.stack()[-2].function in ['walk_packages', '_collect_submodules']:
raise Warning(warning)
volatility3/cli/__init__.py
+2 -1
View File
@@ -332,6 +332,7 @@ class CommandLine:
parser.error(f"Cannot write configuration: file {args.save_config} already exists")
with open(args.save_config, "w") as f:
json.dump(dict(constructed.build_configuration()), f, sort_keys = True, indent = 2)
f.write("\n")
except exceptions.UnsatisfiedException as excp:
self.process_unsatisfied_exceptions(excp)
parser.exit(1, f"Unable to validate the plugin requirements: {[x for x in excp.unsatisfied]}\n")
@@ -423,7 +424,7 @@ class CommandLine:
detail = f"{excp}"
caused_by = ["A required python module is not installed (install the module and re-run)"]
else:
general = "Volatilty encountered an unexpected situation."
general = "Volatility encountered an unexpected situation."
detail = ""
caused_by = [
"Please re-run using with -vvv and file a bug with the output", f"at {constants.BUG_URL}"
volatility3/cli/text_renderer.py
+2 -2
View File
@@ -224,7 +224,7 @@ class CSVRenderer(CLIRenderer):
# Ignore the type because namedtuples don't realize they have accessible attributes
header_list.append(f"{column.name}")
writer = csv.DictWriter(outfd, header_list)
writer = csv.DictWriter(outfd, header_list, lineterminator='\n')
writer.writeheader()
def visitor(node: interfaces.renderers.TreeNode, accumulator):
@@ -345,7 +345,7 @@ class JsonRenderer(CLIRenderer):
def output_result(self, outfd, result):
"""Outputs the JSON data to a file in a particular format"""
outfd.write(json.dumps(result, indent = 2, sort_keys = True))
outfd.write("{}\n".format(json.dumps(result, indent = 2, sort_keys = True)))
def render(self, grid: interfaces.renderers.TreeGrid):
outfd = sys.stdout
volatility3/cli/volshell/__init__.py
+1 -1
View File
@@ -246,6 +246,7 @@ class VolShell(cli.CommandLine):
parser.error(f"Cannot write configuration: file {args.save_config} already exists")
with open(args.save_config, "w") as f:
json.dump(dict(constructed.build_configuration()), f, sort_keys = True, indent = 2)
f.write("\n")
except exceptions.UnsatisfiedException as excp:
self.process_unsatisfied_exceptions(excp)
parser.exit(1, f"Unable to validate the plugin requirements: {[x for x in excp.unsatisfied]}\n")
@@ -256,7 +257,6 @@ class VolShell(cli.CommandLine):
constructed.run()
except exceptions.VolatilityException as excp:
self.process_exceptions(excp)
parser.exit(1, f"Unable to validate the plugin requirements: {[x for x in excp.unsatisfied]}\n")
def main():
volatility3/cli/volshell/linux.py
+2 -2
View File
@@ -56,13 +56,13 @@ class Volshell(generic.Volshell):
"""Display Type describes the members of a particular object in alphabetical order"""
if isinstance(object, str):
if constants.BANG not in object:
object = self.config['vmlinux'] + constants.BANG + object
object = self.current_symbol_table + constants.BANG + object
return super().display_type(object, offset)
def display_symbols(self, symbol_table: str = None):
"""Prints an alphabetical list of symbols for a symbol table"""
if symbol_table is None:
symbol_table = self.config['vmlinux']
symbol_table = self.current_symbol_table
return super().display_symbols(symbol_table)
@property
volatility3/cli/volshell/mac.py
+1 -1
View File
@@ -56,7 +56,7 @@ class Volshell(generic.Volshell):
"""Display Type describes the members of a particular object in alphabetical order"""
if isinstance(object, str):
if constants.BANG not in object:
object = self.config['darwin'] + constants.BANG + object
object = self.current_symbol_table + constants.BANG + object
return super().display_type(object, offset)
def display_symbols(self, symbol_table: str = None):
volatility3/framework/__init__.py
+1 -1
View File
@@ -51,7 +51,7 @@ def require_interface_version(*args) -> None:
if args[1] > interface_version()[1]:
raise RuntimeError(
"Framework interface version {} is an older revision than the required version {}".format(
".".join([str(x) for x in interface_version()[0:1]]), ".".join([str(x) for x in args[0:2]])))
".".join([str(x) for x in interface_version()[0:2]]), ".".join([str(x) for x in args[0:2]])))
class NonInheritable(object):
volatility3/framework/automagic/linux.py
+16 -21
View File
@@ -3,10 +3,12 @@
#
import logging
import os
from typing import Optional, Tuple, Type
from volatility3.framework import interfaces, constants
from volatility3.framework import constants, interfaces
from volatility3.framework.automagic import symbol_cache, symbol_finder
from volatility3.framework.configuration import requirements
from volatility3.framework.layers import intel, scanners
from volatility3.framework.symbols import linux
@@ -23,6 +25,13 @@ class LinuxIntelStacker(interfaces.automagic.StackerLayerInterface):
layer_name: str,
progress_callback: constants.ProgressCallback = None) -> Optional[interfaces.layers.DataLayerInterface]:
"""Attempts to identify linux within this layer."""
# Version check the SQlite cache
required = (1, 0, 0)
if not requirements.VersionRequirement.matches_required(required, symbol_cache.SqliteCache.version):
vollog.info(
f"SQLiteCache version not suitable: required {required} found {symbol_cache.SqliteCache.version}")
return None
# Bail out by default unless we can stack properly
layer = context.layers[layer_name]
join = interfaces.configuration.path_join
@@ -32,7 +41,9 @@ class LinuxIntelStacker(interfaces.automagic.StackerLayerInterface):
if isinstance(layer, intel.Intel):
return None
linux_banners = LinuxBannerCache.load_banners()
identifiers_path = os.path.join(constants.CACHE_PATH, constants.IDENTIFIERS_FILENAME)
linux_banners = symbol_cache.SqliteCache(identifiers_path).get_identifier_dictionary(
operating_system = 'linux')
# If we have no banners, don't bother scanning
if not linux_banners:
vollog.info("No Linux banners found - if this is a linux plugin, please check your symbol files location")
@@ -43,15 +54,8 @@ class LinuxIntelStacker(interfaces.automagic.StackerLayerInterface):
dtb = None
vollog.debug(f"Identified banner: {repr(banner)}")
symbol_files = linux_banners.get(banner, None)
if symbol_files:
if len(symbol_files) > 1:
using = "*"
vollog.warning(f"Multiple symbol files identified (using {using}):")
for symbol_file in symbol_files:
vollog.warning(f" {using} {symbol_file}")
using = " "
isf_path = symbol_files[0]
isf_path = linux_banners.get(banner, None)
if isf_path:
table_name = context.symbol_space.free_table_name('LintelStacker')
table = linux.LinuxKernelIntermedSymbols(context,
'temporary.' + table_name,
@@ -147,20 +151,11 @@ class LinuxIntelStacker(interfaces.automagic.StackerLayerInterface):
return addr - 0xc0000000
class LinuxBannerCache(symbol_cache.SymbolBannerCache):
"""Caches the banners found in the Linux symbol files."""
os = "linux"
symbol_name = "linux_banner"
banner_path = constants.LINUX_BANNERS_PATH
exclusion_list = ['mac', 'windows']
class LinuxSymbolFinder(symbol_finder.SymbolFinder):
"""Linux symbol loader based on uname signature strings."""
banner_config_key = "kernel_banner"
banner_cache = LinuxBannerCache
operating_system = 'linux'
symbol_class = "volatility3.framework.symbols.linux.LinuxKernelIntermedSymbols"
find_aslr = lambda cls, *args: LinuxIntelStacker.find_aslr(*args)[1]
exclusion_list = ['mac', 'windows']
volatility3/framework/automagic/mac.py
+16 -14
View File
@@ -3,11 +3,13 @@
#
import logging
import os
import struct
from typing import Optional
from volatility3.framework import interfaces, constants, layers, exceptions
from volatility3.framework import constants, exceptions, interfaces, layers
from volatility3.framework.automagic import symbol_cache, symbol_finder
from volatility3.framework.configuration import requirements
from volatility3.framework.layers import intel, scanners
from volatility3.framework.symbols import mac
@@ -24,6 +26,13 @@ class MacIntelStacker(interfaces.automagic.StackerLayerInterface):
layer_name: str,
progress_callback: constants.ProgressCallback = None) -> Optional[interfaces.layers.DataLayerInterface]:
"""Attempts to identify mac within this layer."""
# Version check the SQlite cache
required = (1, 0, 0)
if not requirements.VersionRequirement.matches_required(required, symbol_cache.SqliteCache.version):
vollog.info(
f"SQLiteCache version not suitable: required {required} found {symbol_cache.SqliteCache.version}")
return None
# Bail out by default unless we can stack properly
layer = context.layers[layer_name]
new_layer = None
@@ -34,7 +43,9 @@ class MacIntelStacker(interfaces.automagic.StackerLayerInterface):
if isinstance(layer, intel.Intel):
return None
mac_banners = MacBannerCache.load_banners()
identifiers_path = os.path.join(constants.CACHE_PATH, constants.IDENTIFIERS_FILENAME)
mac_banners = symbol_cache.SqliteCache(identifiers_path).get_identifier_dictionary(
operating_system = 'mac')
# If we have no banners, don't bother scanning
if not mac_banners:
vollog.info("No Mac banners found - if this is a mac plugin, please check your symbol files location")
@@ -46,9 +57,8 @@ class MacIntelStacker(interfaces.automagic.StackerLayerInterface):
dtb = None
vollog.debug(f"Identified banner: {repr(banner)}")
symbol_files = mac_banners.get(banner, None)
if symbol_files:
isf_path = symbol_files[0]
isf_path = mac_banners.get(banner, None)
if isf_path:
table_name = context.symbol_space.free_table_name('MacintelStacker')
table = mac.MacKernelIntermedSymbols(context = context,
config_path = join('temporary', table_name),
@@ -197,19 +207,11 @@ class MacIntelStacker(interfaces.automagic.StackerLayerInterface):
yield offset, banner
class MacBannerCache(symbol_cache.SymbolBannerCache):
"""Caches the banners found in the Mac symbol files."""
os = "mac"
symbol_name = "version"
banner_path = constants.MAC_BANNERS_PATH
exclusion_list = ['windows', 'linux']
class MacSymbolFinder(symbol_finder.SymbolFinder):
"""Mac symbol loader based on uname signature strings."""
banner_config_key = 'kernel_banner'
banner_cache = MacBannerCache
operating_system = 'mac'
find_aslr = MacIntelStacker.find_aslr
symbol_class = "volatility3.framework.symbols.mac.MacKernelIntermedSymbols"
exclusion_list = ['windows', 'linux']
volatility3/framework/automagic/module.py
+4
View File
@@ -1,3 +1,7 @@
# This file is Copyright 2022 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
from volatility3.framework import interfaces, constants, configuration
volatility3/framework/automagic/pdbscan.py
+8 -7
View File
@@ -7,10 +7,11 @@ from loaded PE files.
This module contains a standalone scanner, and also a :class:`~volatility3.framework.interfaces.layers.ScannerInterface`
based scanner for use within the framework by calling :func:`~volatility3.framework.interfaces.layers.DataLayerInterface.scan`.
"""
import contextlib
import logging
import math
import os
from typing import Any, Dict, Iterable, List, Optional, Set, Tuple, Union, Callable
from typing import Any, Callable, Dict, Iterable, List, Optional, Set, Tuple, Union
from volatility3.framework import constants, exceptions, interfaces, layers
from volatility3.framework.configuration import requirements
@@ -139,7 +140,8 @@ class KernelPDBScanner(interfaces.automagic.AutomagicInterface):
vlayer: layers.intel.Intel,
progress_callback: constants.ProgressCallback = None) -> Optional[ValidKernelType]:
def test_virtual_kernel(physical_layer_name, virtual_layer_name: str, kernel: Dict[str, Any]) -> Optional[ValidKernelType]:
def test_virtual_kernel(physical_layer_name, virtual_layer_name: str, kernel: Dict[str, Any]) -> Optional[
ValidKernelType]:
# It seems the kernel is loaded at a fixed mapping (presumably because the memory manager hasn't started yet)
if kernel['mz_offset'] is None or not isinstance(kernel['mz_offset'], int):
# Rule out kernels that couldn't find a suitable MZ header
@@ -148,7 +150,7 @@ class KernelPDBScanner(interfaces.automagic.AutomagicInterface):
vollog.debug("Kernel base determination - optimized scan virtual layer")
valid_kernel = self._method_layer_pdb_scan(context, vlayer, test_virtual_kernel, True, False, progress_callback)
if valid_kernel != None:
if valid_kernel is not None:
return valid_kernel
vollog.debug("Kernel base determination - slow scan virtual layer")
@@ -159,7 +161,8 @@ class KernelPDBScanner(interfaces.automagic.AutomagicInterface):
vlayer: layers.intel.Intel,
progress_callback: constants.ProgressCallback = None) -> Optional[ValidKernelType]:
def test_physical_kernel(physical_layer_name:str , virtual_layer_name: str, kernel: Dict[str, Any]) -> Optional[ValidKernelType]:
def test_physical_kernel(physical_layer_name: str, virtual_layer_name: str, kernel: Dict[str, Any]) -> Optional[
ValidKernelType]:
# It seems the kernel is loaded at a fixed mapping (presumably because the memory manager hasn't started yet)
if kernel['mz_offset'] is None or not isinstance(kernel['mz_offset'], int):
# Rule out kernels that couldn't find a suitable MZ header
@@ -274,7 +277,7 @@ class KernelPDBScanner(interfaces.automagic.AutomagicInterface):
kernel_pdb_names = [bytes(name + ".pdb", "utf-8") for name in constants.windows.KERNEL_MODULE_NAMES]
virtual_layer_name = vlayer.name
try:
with contextlib.suppress(exceptions.InvalidAddressException):
if vlayer.read(address, 0x2) == b'MZ':
res = list(
PDBUtility.pdbname_scan(ctx = context,
@@ -286,8 +289,6 @@ class KernelPDBScanner(interfaces.automagic.AutomagicInterface):
end = address + self.max_pdb_size))
if res:
valid_kernel = (virtual_layer_name, address, res[0])
except exceptions.InvalidAddressException:
pass
return valid_kernel
# List of methods to be run, in order, to determine the valid kernels
volatility3/framework/automagic/symbol_cache.py
+389 -161
View File
@@ -2,18 +2,19 @@
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
import base64
import gc
import json
import logging
import os
import pickle
import sqlite3
import urllib
import urllib.parse
import urllib.request
import zipfile
from typing import Dict, List, Optional
from abc import abstractmethod
from typing import Dict, Generator, Iterable, List, Optional, Tuple
from volatility3.framework import constants, exceptions, interfaces
from volatility3 import framework, schemas
from volatility3.framework import constants, interfaces
from volatility3.framework.configuration import requirements
from volatility3.framework.layers import resources
from volatility3.framework.symbols import intermed
@@ -22,164 +23,392 @@ vollog = logging.getLogger(__name__)
BannersType = Dict[bytes, List[str]]
class SymbolBannerCache(interfaces.automagic.AutomagicInterface):
"""Runs through all symbols tables and caches their banners."""
### Identifiers
# Since this is necessary for ConstructionMagic, we set a lower priority
# The user would run it eventually either way, but running it first means it can be used that run
class IdentifierProcessor:
operating_system = None
def __init__(self):
pass
@classmethod
@abstractmethod
def get_identifier(cls, json) -> Optional[bytes]:
"""Method to extract the identifier from a particular operating system's JSON
Returns:
identifier is valid or None if not found
"""
raise NotImplementedError("This base class has no get_identifier method defined")
class WindowsIdentifier(IdentifierProcessor):
operating_system = 'windows'
separator = '|'
@classmethod
def get_identifier(cls, json) -> Optional[bytes]:
"""Returns the identifier for the file if one can be found"""
windows_metadata = json.get('metadata', {}).get('windows', {}).get('pdb', {})
if windows_metadata:
guid = windows_metadata.get('GUID', None)
age = windows_metadata.get('age', None)
database = windows_metadata.get('database', None)
if guid and age and database:
return cls.generate(database, guid, age)
return None
@classmethod
def generate(cls, pdb_name: str, guid: str, age: int) -> bytes:
return bytes(cls.separator.join([pdb_name, guid.upper(), str(age)]), 'latin-1')
class MacIdentifier(IdentifierProcessor):
operating_system = 'mac'
@classmethod
def get_identifier(cls, json) -> Optional[bytes]:
mac_banner = json.get('symbols', {}).get('version', {}).get('constant_data', None)
if mac_banner:
return base64.b64decode(mac_banner)
return None
class LinuxIdentifier(IdentifierProcessor):
operating_system = 'linux'
@classmethod
def get_identifier(cls, json) -> Optional[bytes]:
linux_banner = json.get('symbols', {}).get('linux_banner', {}).get('constant_data', None)
if linux_banner:
return base64.b64decode(linux_banner)
return None
### CacheManagers
class CacheManagerInterface(interfaces.configuration.VersionableInterface):
def __init__(self, filename: str):
super().__init__()
self._filename = filename
self._classifiers = {}
for subclazz in framework.class_subclasses(IdentifierProcessor):
self._classifiers[subclazz.operating_system] = subclazz
def add_identifier(self, location: str, operating_system: str, identifier: str):
"""Adds an identifier to the store"""
pass
def find_location(self, identifier: bytes, operating_system: Optional[str]) -> Optional[str]:
"""Returns the location of the symbol file given the identifier
Args:
identifier: string that uniquely identifies a particular symbol table
operating_system: optional string to restrict identifiers to just those for a particular operating system
Returns:
The location of the symbols file that matches the identifier
"""
pass
def get_local_locations(self) -> Iterable[str]:
"""Returns a list of all the local locations"""
pass
def update(self):
"""Locates all files under the symbol directories. Updates the cache with additions, modifications and removals.
This also updates remote locations based on a cache timeout.
"""
pass
def get_identifier_dictionary(self, operating_system: Optional[str] = None, local_only: bool = False) -> \
Dict[bytes, str]:
"""Returns a dictionary of identifiers and locations
Args:
operating_system: If set, limits responses to a specific operating system
local_only: Returns only local locations
Returns:
A dictionary of identifiers mapped to a location
"""
pass
def get_identifier(self, location: str) -> Optional[bytes]:
"""Returns an identifier based on a specific location or None"""
pass
def get_identifiers(self, operating_system: Optional[str]) -> List[bytes]:
"""Returns all identifiers for a particular operating system"""
pass
def get_location_statistics(self, location: str) -> Optional[Tuple[int, int, int, int]]:
"""Returns ISF statistics based on the location
Returns:
A tuple of base_types, types, enums, symbols, or None is location not found"""
def get_hash(self, location: str) -> Optional[str]:
"""Returns the hash of the JSON from within a location ISF"""
class SqliteCache(CacheManagerInterface):
_required_framework_version = (2, 0, 0)
_version = (1, 0, 0)
cache_period = '-3 days'
def __init__(self, filename: str):
super().__init__(filename)
try:
self._database = self._connect_storage(filename)
except sqlite3.DatabaseError:
os.unlink(filename)
self._database = self._connect_storage(filename)
def _connect_storage(self, path: str) -> sqlite3.Connection:
database = sqlite3.connect(path)
database.row_factory = sqlite3.Row
database.cursor().execute(
f'CREATE TABLE IF NOT EXISTS database_info (schema_version INT DEFAULT {constants.CACHE_SQLITE_SCHEMA_VERSION})')
schema_version = database.cursor().execute('SELECT schema_version FROM database_info').fetchone()
if not schema_version:
database.cursor().execute(f'INSERT INTO database_info VALUES ({constants.CACHE_SQLITE_SCHEMA_VERSION})')
elif schema_version['schema_version'] == constants.CACHE_SQLITE_SCHEMA_VERSION:
# All good, so pass and move on
pass
else:
vollog.info(f"Previous cache schema version found: {schema_version['schema_version']}")
# TODO: Implement code if the schema changes
# Current this should never happen so we start over again
database.close()
os.unlink(path)
return self._connect_storage(path)
database.cursor().execute(
'CREATE TABLE IF NOT EXISTS cache (location TEXT UNIQUE NOT NULL, identifier TEXT, operating_system TEXT, hash TEXT,'
'stats_base_types INT DEFAULT 0, stats_types INT DEFAULT 0, stats_enums INT DEFAULT 0, stats_symbols INT DEFAULT 0, local BOOL, cached DATETIME)')
database.commit()
return database
def find_location(self, identifier: bytes, operating_system: Optional[str]) -> Optional[str]:
"""Returns the location of the symbol file given the identifier.
If multiple locations exist for an identifier, the last found is returned
Args:
identifier: string that uniquely identifies a particular symbol table
operating_system: optional string to restrict identifiers to just those for a particular operating system
Returns:
The location of the symbols file that matches the identifier or None
"""
statement = 'SELECT location FROM cache WHERE identifier = ?'
parameters = (identifier,)
if operating_system is not None:
statement = 'SELECT location FROM cache WHERE identifier = ? AND operating_system = ?'
parameters = (identifier, operating_system)
results = self._database.cursor().execute(statement, parameters).fetchall()
result = None
for row in results:
result = row['location']
return result
def get_local_locations(self) -> Generator[str, None, None]:
result = self._database.cursor().execute('SELECT DISTINCT location FROM cache WHERE local = 1').fetchall()
for row in result:
yield row['location']
def is_url_local(self, url: str) -> bool:
"""Determines whether an url is local or not"""
parsed = urllib.parse.urlparse(url)
if parsed.scheme in ['file', 'jar']:
return True
def get_identifier(self, location: str) -> Optional[bytes]:
results = self._database.cursor().execute('SELECT identifier FROM cache WHERE location = ?',
(location,)).fetchall()
for row in results:
return row['identifier']
return None
def get_location_statistics(self, location: str) -> Optional[Tuple[int, int, int, int]]:
results = self._database.cursor().execute(
'SELECT stats_base_types, stats_types, stats_enums, stats_symbols FROM cache WHERE location = ?',
(location,)).fetchall()
for row in results:
return row['stats_base_types'], row['stats_types'], row['stats_enums'], row['stats_symbols']
return None
def get_hash(self, location: str) -> Optional[str]:
results = self._database.cursor().execute('SELECT hash FROM cache WHERE location = ?',
(location,)).fetchall()
for row in results:
return row['hash']
def update(self, progress_callback = None):
"""Locates all files under the symbol directories. Updates the cache with additions, modifications and removals.
This also updates remote locations based on a cache timeout.
"""
on_disk_locations = set([filename for filename in intermed.IntermediateSymbolTable.file_symbol_url('')])
cached_locations = set(self.get_local_locations())
new_locations = on_disk_locations.difference(cached_locations)
missing_locations = cached_locations.difference(on_disk_locations)
cache_update = set()
files_to_timestamp = on_disk_locations.intersection(cached_locations)
if files_to_timestamp:
result = self._database.cursor().execute("SELECT location FROM cache WHERE local = 1 "
f"AND cached < date('now', '{self.cache_period}');")
for row in result:
if row['location'] in files_to_timestamp:
cache_update.add(row['location'])
idextractors = list(framework.class_subclasses(IdentifierProcessor))
# New or not recently updated
files_to_process = new_locations.union(cache_update)
number_files_to_process = len(files_to_process)
cursor = self._database.cursor()
try:
for counter, location in enumerate(files_to_process):
# Open location
progress_callback(counter * 100 / number_files_to_process,
f"Updating caches for {number_files_to_process} files...")
try:
with resources.ResourceAccessor().open(location) as fp:
json_obj = json.load(fp)
hash = schemas.create_json_hash(json_obj)
identifier = None
# Get stats
stats_base_types = len(json_obj.get('base_types', {}))
stats_types = len(json_obj.get('types', {}))
stats_enums = len(json_obj.get('enums', {}))
stats_symbols = len(json_obj.get('symbols', {}))
operating_system = None
for idextractor in idextractors:
identifier = idextractor.get_identifier(json_obj)
if identifier is not None:
operating_system = idextractor.operating_system
break
# We don't try to validate schemas here, we do that on first use
# Store in database
cursor.execute(
"INSERT OR REPLACE INTO cache (location, identifier, operating_system, hash,"
"stats_base_types, stats_types, stats_enums, stats_symbols, "
"local, cached) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, datetime('now'))",
(
location,
identifier,
operating_system,
hash,
stats_base_types,
stats_types,
stats_enums,
stats_symbols,
self.is_url_local(location)
))
if identifier is not None:
vollog.log(constants.LOGLEVEL_VV, f"Identified {location} as {identifier}")
else:
vollog.log(constants.LOGLEVEL_VVVV, f"No identifier found for {location}")
except Exception as excp:
vollog.log(constants.LOGLEVEL_VVVV, excp)
finally:
self._database.commit()
# Remote Entries
if not constants.OFFLINE and constants.REMOTE_ISF_URL:
progress_callback(0, 'Reading remote ISF list')
cursor = self._database.cursor()
cursor.execute(
f"SELECT cached FROM cache WHERE local = 0 and cached < datetime('now', {self.cache_period})")
remote_identifiers = RemoteIdentifierFormat(constants.REMOTE_ISF_URL)
progress_callback(50, 'Reading remote ISF list')
for operating_system in constants.OS_CATEGORIES:
identifiers = remote_identifiers.process({}, operating_system = operating_system)
for identifier, location in identifiers:
cursor.execute(
"INSERT OR REPLACE INTO cache(identifier, location, operating_system, local, cached) VALUES (?, ?, ?, ?, datetime('now'))",
(location, identifier, operating_system, False)
)
progress_callback(100, 'Reading remote ISF list')
self._database.commit()
# Missing entries
if missing_locations:
self._database.cursor().execute(
f"DELETE FROM cache WHERE location IN ({','.join(['?'] * len(missing_locations))})",
[x for x in missing_locations])
self._database.commit()
def get_identifier_dictionary(self, operating_system: Optional[str] = None, local_only: bool = False) -> \
Dict[bytes, str]:
output = {}
additions = []
statement = 'SELECT location, identifier FROM cache'
if local_only:
additions.append('local = 1')
if operating_system:
additions.append(f"operating_system = '{operating_system}'")
if additions:
statement += f" WHERE {' AND '.join(additions)}"
results = self._database.cursor().execute(statement)
for row in results:
if row['identifier'] in output and row['identifier'] and row['location']:
vollog.debug(
f"Duplicate entry for identifier {row['identifier']}: {row['location']} and {output[row['identifier']]}")
output[row['identifier']] = row['location']
return output
def get_identifiers(self, operating_system: Optional[str]) -> List[bytes]:
if operating_system:
results = self._database.cursor().execute('SELECT identifier FROM cache WHERE operating_system = ?',
(operating_system,)).fetchall()
else:
results = self._database.cursor().execute('SELECT identifier FROM cache').fetchall()
output = []
for row in results:
output.append(row['identifier'])
return output
### Automagic
class SymbolCacheMagic(interfaces.automagic.AutomagicInterface):
"""Runs through all symbol tables and caches their identifiers"""
priority = 0
os: Optional[str] = None
symbol_name: str = "banner_name"
banner_path: Optional[str] = None
@classmethod
def load_banners(cls) -> BannersType:
if not cls.banner_path:
raise ValueError("Banner_path not appropriately set")
banners: BannersType = {}
if os.path.exists(cls.banner_path):
with open(cls.banner_path, "rb") as f:
# We use pickle over JSON because we're dealing with bytes objects
banners.update(pickle.load(f))
# Remove possibilities that can't exist locally.
remove_banners = []
for banner in banners:
for path in banners[banner]:
url = urllib.parse.urlparse(path)
if url.scheme == 'file' and not os.path.exists(urllib.request.url2pathname(url.path)):
vollog.log(
constants.LOGLEVEL_VV, "Removing cached path {} for banner {}: file does not exist".format(
path, str(banner or b'', 'latin-1')))
banners[banner].remove(path)
# This is probably excessive, but it's here if we need it
if url.scheme == 'jar':
zip_file, zip_path = url.path.split("!")
zip_file = urllib.parse.urlparse(zip_file).path
if ((not os.path.exists(zip_file)) or (zip_path not in zipfile.ZipFile(zip_file).namelist())):
vollog.log(constants.LOGLEVEL_VV,
"Removing cached path {} for banner {}: file does not exist".format(path, banner))
banners[banner].remove(path)
if not banners[banner]:
remove_banners.append(banner)
for remove_banner in remove_banners:
del banners[remove_banner]
return banners
@classmethod
def save_banners(cls, banners):
with open(cls.banner_path, "wb") as f:
pickle.dump(banners, f)
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
identifiers_path = os.path.join(constants.CACHE_PATH, constants.IDENTIFIERS_FILENAME)
self._cache = SqliteCache(identifiers_path)
def __call__(self, context, config_path, configurable, progress_callback = None):
"""Runs the automagic over the configurable."""
# Bomb out if we're just the generic interface
if self.os is None:
return
# We only need to be called once, so no recursion necessary
banners = self.load_banners()
cacheables = self.find_new_banner_files(banners, self.os)
new_banners = self.read_new_banners(context, config_path, cacheables, self.symbol_name, self.os,
progress_callback)
# Add in any new banners to the existing list
for new_banner in new_banners:
banner_list = banners.get(new_banner, [])
banners[new_banner] = list(set(banner_list + new_banners[new_banner]))
# Do remote banners *after* the JSON loading, so that it doesn't pull down all the remote JSON
self.remote_banners(banners, self.os)
# Rewrite the cached banners each run, since writing is faster than the banner_cache validation portion
self.save_banners(banners)
if progress_callback is not None:
progress_callback(100, f"Built {self.os} caches")
self._cache.update(progress_callback)
@classmethod
def read_new_banners(cls, context: interfaces.context.ContextInterface, config_path: str, new_urls: List[str],
symbol_name: str, operating_system: str = None,
progress_callback = None) -> Optional[Dict[bytes, List[str]]]:
"""Reads the any new banners for the OS in question"""
if operating_system is None:
return None
banners = {}
total = len(new_urls)
if total > 0:
vollog.info(f"Building {operating_system} caches...")
for current in range(total):
if progress_callback is not None:
progress_callback(current * 100 / total, f"Building {operating_system} caches")
isf_url = new_urls[current]
isf = None
try:
# Loading the symbol table will be very slow until it's been validated
isf = intermed.IntermediateSymbolTable(context, config_path, "temp", isf_url, validate = False)
# We should store the banner against the filename
# We don't bother with the hash (it'll likely take too long to validate)
# but we should check at least that the banner matches on load.
banner = isf.get_symbol(symbol_name).constant_data
vollog.log(constants.LOGLEVEL_VV, f"Caching banner {banner} for file {isf_url}")
bannerlist = banners.get(banner, [])
bannerlist.append(isf_url)
banners[banner] = bannerlist
except exceptions.SymbolError:
pass
except json.JSONDecodeError:
vollog.log(constants.LOGLEVEL_VV, f"Caching file {isf_url} failed due to JSON error")
finally:
# Get rid of the loaded file, in case it sits in memory
if isf:
del isf
gc.collect()
return banners
@classmethod
def find_new_banner_files(cls, banners: Dict[bytes, List[str]], operating_system: str) -> List[str]:
"""Gathers all files and remove existing banners"""
cacheables = list(intermed.IntermediateSymbolTable.file_symbol_url(operating_system))
for banner in banners:
for json_file in banners[banner]:
if json_file in cacheables:
cacheables.remove(json_file)
return cacheables
@classmethod
def remote_banners(cls, banners: Dict[bytes, List[str]], operating_system = None, banner_location = None):
"""Adds remote URLs to the banner list"""
if operating_system is None:
return None
if banner_location is None:
banner_location = constants.REMOTE_ISF_URL
if not constants.OFFLINE and banner_location is not None:
try:
rbf = RemoteBannerFormat(banner_location)
rbf.process(banners, operating_system)
except urllib.error.URLError:
vollog.debug(f"Unable to download remote banner list from {banner_location}")
def get_requirements(cls) -> List[interfaces.configuration.RequirementInterface]:
"""Returns a list of RequirementInterface objects required by this
object."""
return [requirements.VersionRequirement(name = 'SQLiteCache', component = SqliteCache, version = (1, 0, 0))]
class RemoteBannerFormat:
class RemoteIdentifierFormat:
def __init__(self, location: str):
self._location = location
with resources.ResourceAccessor().open(url = location) as fp:
self._data = json.load(fp)
if not self._verify():
raise ValueError("Unsupported version for remote banner list format")
raise ValueError("Unsupported version for remote identifier list format")
def _verify(self) -> bool:
version = self._data.get('version', 0)
@@ -188,23 +417,22 @@ class RemoteBannerFormat:
return True
return False
def process(self, banners: Dict[bytes, List[str]], operating_system: Optional[str]):
raise ValueError("Banner List version not verified")
def process(self, identifiers: Dict[bytes, List[str]], operating_system: Optional[str]) -> Generator[
Tuple[bytes, str], None, None]:
raise ValueError("Identifier List version not verified")
def process_v1(self, banners: Dict[bytes, List[str]], operating_system: Optional[str]):
def process_v1(self, identifiers: Optional[Dict[bytes, List[str]]], operating_system: Optional[str]) -> Generator[
Tuple[bytes, str], None, None]:
if operating_system in self._data:
for banner in self._data[operating_system]:
binary_banner = base64.b64decode(banner)
file_list = banners.get(binary_banner, [])
for value in self._data[operating_system][banner]:
if value not in file_list:
file_list = file_list + [value]
banners[binary_banner] = file_list
for identifier in self._data[operating_system]:
binary_identifier = base64.b64decode(identifier)
for value in self._data[operating_system][identifier]:
yield binary_identifier, value
if 'additional' in self._data:
for location in self._data['additional']:
try:
subrbf = RemoteBannerFormat(location)
subrbf.process(banners, operating_system)
subrbf = RemoteIdentifierFormat(location)
yield from subrbf.process(identifiers, operating_system)
except IOError:
vollog.debug(f"Remote file not found: {location}")
return banners
return identifiers
volatility3/framework/automagic/symbol_finder.py
+19 -11
View File
@@ -3,9 +3,10 @@
#
import logging
from typing import Any, Iterable, List, Tuple, Type, Optional, Callable
import os
from typing import Any, Callable, Iterable, List, Optional, Tuple
from volatility3.framework import interfaces, constants, layers
from volatility3.framework import constants, interfaces, layers
from volatility3.framework.automagic import symbol_cache
from volatility3.framework.configuration import requirements
from volatility3.framework.layers import scanners
@@ -18,7 +19,7 @@ class SymbolFinder(interfaces.automagic.AutomagicInterface):
priority = 40
banner_config_key: str = "banner"
banner_cache: Optional[Type[symbol_cache.SymbolBannerCache]] = None
operating_system: Optional[str] = None
symbol_class: Optional[str] = None
find_aslr: Optional[Callable] = None
@@ -27,14 +28,22 @@ class SymbolFinder(interfaces.automagic.AutomagicInterface):
self._requirements: List[Tuple[str, interfaces.configuration.RequirementInterface]] = []
self._banners: symbol_cache.BannersType = {}
@classmethod
def get_requirements(cls) -> List[interfaces.configuration.RequirementInterface]:
return [
requirements.VersionRequirement(name = 'SQLiteCache',
component = symbol_cache.SqliteCache,
version = (1, 0, 0))
]
@property
def banners(self) -> symbol_cache.BannersType:
"""Creates a cached copy of the results, but only it's been
requested."""
if not self._banners:
if not self.banner_cache:
raise RuntimeError(f"Cache has not been properly defined for {self.__class__.__name__}")
self._banners = self.banner_cache.load_banners()
identifiers_path = os.path.join(constants.CACHE_PATH, constants.IDENTIFIERS_FILENAME)
cache = symbol_cache.SqliteCache(identifiers_path)
self._banners = cache.get_identifier_dictionary(operating_system = self.operating_system)
return self._banners
def __call__(self,
@@ -103,8 +112,8 @@ class SymbolFinder(interfaces.automagic.AutomagicInterface):
vollog.debug(f"Identified banner: {repr(banner)}")
symbol_files = self.banners.get(banner, None)
if symbol_files:
isf_path = symbol_files[0]
vollog.debug(f"Using symbol library: {symbol_files[0]}")
isf_path = symbol_files
vollog.debug(f"Using symbol library: {symbol_files}")
clazz = self.symbol_class
# Set the discovered options
path_join = interfaces.configuration.path_join
@@ -116,9 +125,8 @@ class SymbolFinder(interfaces.automagic.AutomagicInterface):
requirement.construct(context, config_path)
break
else:
if symbol_files:
vollog.debug(f"Symbol library path not found: {symbol_files[0]}")
# print("Kernel", banner, hex(banner_offset))
vollog.debug(f"Symbol library path not found for: {banner}")
# print("Kernel", banner, hex(banner_offset))
else:
vollog.debug("No existing banners found")
# TODO: Fallback to generic regex search?
volatility3/framework/automagic/windows.py
+3
View File
@@ -214,6 +214,9 @@ class WindowsIntelStacker(interfaces.automagic.StackerLayerInterface):
context.config[interfaces.configuration.path_join(
config_path, "page_map_offset")] = base_layer.metadata['page_map_offset']
layer = layer_type(context, config_path = config_path, name = new_layer_name, metadata = {'os': 'Windows'})
page_map_offset = context.config[interfaces.configuration.path_join(config_path, "page_map_offset")]
vollog.debug(f"DTB was given to us by base layer: {hex(page_map_offset)}")
return layer
# Self Referential finder
for description, tests, sections in cls.test_sets:
volatility3/framework/configuration/requirements.py
+9 -3
View File
@@ -408,13 +408,19 @@ class VersionRequirement(interfaces.configuration.RequirementInterface):
config_path: str) -> Dict[str, interfaces.configuration.RequirementInterface]:
# Mypy doesn't appreciate our classproperty implementation, self._plugin.version has no type
config_path = interfaces.configuration.path_join(config_path, self.name)
if len(self._version) > 0 and self._component.version[0] != self._version[0]:
return {config_path: self}
if len(self._version) > 1 and self._component.version[1] < self._version[1]:
if not self.matches_required(self._version, self._component.version):
return {config_path: self}
context.config[interfaces.configuration.path_join(config_path, self.name)] = True
return {}
@classmethod
def matches_required(cls, required: Tuple[int, ...], version: Tuple[int, int, int]) -> bool:
if len(required) > 0 and version[0] != required[0]:
return False
if len(required) > 1 and version[1] < required[1]:
return False
return True
class PluginRequirement(VersionRequirement):
volatility3/framework/constants/__init__.py
+6 -6
View File
@@ -39,7 +39,7 @@ BANG = "!"
# We use the SemVer 2.0.0 versioning scheme
VERSION_MAJOR = 2 # Number of releases of the library with a breaking change
VERSION_MINOR = 1 # Number of changes that only add to the interface
VERSION_MINOR = 4 # Number of changes that only add to the interface
VERSION_PATCH = 0 # Number of changes that do not change the interface
VERSION_SUFFIX = ""
@@ -64,14 +64,14 @@ CACHE_PATH = os.path.join(os.path.expanduser("~"), ".cache", "volatility3")
"""Default path to store cached data"""
if sys.platform == 'win32':
CACHE_PATH = os.path.join(os.environ.get("APPDATA", os.path.expanduser("~")), "volatility3")
CACHE_PATH = os.path.realpath(os.path.join(os.environ.get("APPDATA", os.path.expanduser("~")), "volatility3"))
os.makedirs(CACHE_PATH, exist_ok = True)
LINUX_BANNERS_PATH = os.path.join(CACHE_PATH, "linux_banners.cache")
""""Default location to record information about available linux banners"""
IDENTIFIERS_FILENAME = "identifier.cache"
"""Default location to record information about available identifiers"""
MAC_BANNERS_PATH = os.path.join(CACHE_PATH, "mac_banners.cache")
""""Default location to record information about available mac banners"""
CACHE_SQLITE_SCHEMA_VERSION = 1
"""Version for the sqlite3 cache schema"""
BUG_URL = "https://github.com/volatilityfoundation/volatility3/issues"
volatility3/framework/contexts/__init__.py
+1 -1
View File
@@ -321,7 +321,7 @@ class SizedModule(Module):
The mapping should be sorted and should be quicker than reading
the data We turn it into JSON to make a common string and use a
quick hash, because collissions are unlikely
quick hash, because collisions are unlikely
"""
layer = self._context.layers[self.layer_name]
if not isinstance(layer, interfaces.layers.TranslationLayerInterface):
volatility3/framework/interfaces/automagic.py
+5 -4
View File
@@ -9,9 +9,9 @@ that a user has not filled.
"""
import logging
from abc import ABCMeta
from typing import Any, List, Optional, Tuple, Union, Type
from typing import Any, List, Optional, Tuple, Type, Union
from volatility3.framework import interfaces, constants
from volatility3.framework import constants, interfaces
from volatility3.framework.configuration import requirements
vollog = logging.getLogger(__name__)
@@ -47,9 +47,10 @@ class AutomagicInterface(interfaces.configuration.ConfigurableInterface, metacla
super().__init__(context, config_path)
for requirement in self.get_requirements():
if not isinstance(requirement, (interfaces.configuration.SimpleTypeRequirement,
requirements.ChoiceRequirement, requirements.ListRequirement)):
requirements.ChoiceRequirement, requirements.ListRequirement,
requirements.VersionRequirement)):
raise TypeError(
"Automagic requirements must be a SimpleTypeRequirement, ChoiceRequirement or ListRequirement")
"Automagic requirements must be a SimpleTypeRequirement, ChoiceRequirement, ListRequirement or VersionRequirement")
def __call__(self,
context: interfaces.context.ContextInterface,
volatility3/framework/interfaces/configuration.py
+1 -1
View File
@@ -523,7 +523,7 @@ class ConstructableRequirementInterface(RequirementInterface):
must happen after the class configuration value has been provided).
These values are then provided to the object's constructor by name
as arguments (as well as the standard `context` and `config_path`
arguments.
arguments).
"""
def __init__(self, *args, **kwargs) -> None:
volatility3/framework/interfaces/layers.py
+2 -2
View File
@@ -307,7 +307,7 @@ class DataLayerInterface(interfaces.configuration.ConfigurableInterface, metacla
while length > 0:
chunk_size = min(length, scanner.chunk_size + scanner.overlap)
yield [(layer_name, mapped_offset, chunk_size)], offset + chunk_size
# It we've got more than the scanner's chunk_size, only move up by the chunk_size
# If we've got more than the scanner's chunk_size, only move up by the chunk_size
if chunk_size > scanner.chunk_size:
chunk_size -= scanner.overlap
length -= chunk_size
@@ -517,7 +517,7 @@ class TranslationLayerInterface(DataLayerInterface, metaclass = ABCMeta):
yield output, chunk_position
output = []
chunk_position = chunk_start
# Take from chunk_position as far as far as the block can go,
# Take from chunk_position as far as the block can go,
# or as much left of a scanner chunk as we can
chunk_size = min(block_end - chunk_position,
scanner.chunk_size + scanner.overlap - (chunk_position - chunk_start))
volatility3/framework/interfaces/objects.py
+12 -3
View File
@@ -6,6 +6,7 @@ interpreted values of data from a layer."""
import abc
import collections
import collections.abc
import contextlib
import logging
from typing import Any, Dict, List, Mapping, Optional
@@ -187,11 +188,9 @@ class ObjectInterface(metaclass = abc.ABCMeta):
"""
if self.has_member(member_name):
# noinspection PyBroadException
try:
with contextlib.suppress(Exception):
_ = getattr(self, member_name)
return True
except Exception:
pass
return False
def has_valid_members(self, member_names: List[str]) -> bool:
@@ -241,6 +240,12 @@ class ObjectInterface(metaclass = abc.ABCMeta):
the child member."""
raise KeyError(f"Template does not contain any children: {template.vol.type_name}")
@classmethod
@abc.abstractmethod
def child_template(cls, template: 'Template', child: str) -> 'interfaces.objects.Template':
"""Returns the template of the child member from the parent."""
raise KeyError(f"Template does not contain any children: {template.vol.type_name}")
@classmethod
@abc.abstractmethod
def has_member(cls, template: 'Template', member_name: str) -> bool:
@@ -305,6 +310,10 @@ class Template:
"""Returns the relative offset of the `child` member from its parent
offset."""
@abc.abstractmethod
def child_template(self, child: str) -> 'interfaces.objects.Template':
"""Returns the `child` member template from its parent."""
@abc.abstractmethod
def replace_child(self, old_child: 'Template', new_child: 'Template') -> None:
"""Replaces `old_child` with `new_child` in the list of children."""
volatility3/framework/interfaces/symbols.py
+1 -1
View File
@@ -169,7 +169,7 @@ class BaseSymbolTableInterface:
def optional_set_type_class(self, name: str, clazz: Type[objects.ObjectInterface]) -> bool:
"""Calls the set_type_class function but does not throw an exception.
Returns whether setting the type class was successfull.
Returns whether setting the type class was successful.
Args:
name: The name of the type to override the class for
clazz: The actual class to override for the provided type name
volatility3/framework/layers/avml.py
+4
View File
@@ -1,3 +1,7 @@
# This file is Copyright 2022 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
"""Functions that read AVML files.
The user of the file doesn't have to worry about the compression,
volatility3/framework/layers/codecs/__init__.py
+4
View File
@@ -1,3 +1,7 @@
# This file is Copyright 2022 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
"""Codecs used for encoding or decoding data should live here
volatility3/framework/layers/crash.py
+2 -3
View File
@@ -1,6 +1,7 @@
# This file is Copyright 2021 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
import contextlib
import logging
import struct
from typing import Tuple, Optional
@@ -202,11 +203,9 @@ class WindowsCrashDumpStacker(interfaces.automagic.StackerLayerInterface):
layer_name: str,
progress_callback: constants.ProgressCallback = None) -> Optional[interfaces.layers.DataLayerInterface]:
for layer in [WindowsCrashDump32Layer, WindowsCrashDump64Layer]:
try:
with contextlib.suppress(WindowsCrashDumpFormatException):
layer.check_header(context.layers[layer_name])
new_name = context.layers.free_layer_name(layer.__name__)
context.config[interfaces.configuration.path_join(new_name, "base_layer")] = layer_name
return layer(context, new_name, new_name)
except WindowsCrashDumpFormatException:
pass
return None
volatility3/framework/layers/leechcore.py
+4
View File
@@ -1,3 +1,7 @@
# This file is Copyright 2022 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
import io
import logging
import urllib.parse
volatility3/framework/layers/linear.py
+4
View File
@@ -1,3 +1,7 @@
# This file is Copyright 2022 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
import functools
from typing import List, Optional, Tuple, Iterable
volatility3/framework/layers/physical.py
+7 -5
View File
@@ -5,7 +5,7 @@ import logging
import threading
from typing import Any, Dict, IO, List, Optional, Union
from volatility3.framework import exceptions, interfaces, constants
from volatility3.framework import constants, exceptions, interfaces
from volatility3.framework.configuration import requirements
from volatility3.framework.layers import resources
@@ -88,6 +88,7 @@ class FileLayer(interfaces.layers.DataLayerInterface):
self._accessor = resources.ResourceAccessor()
self._file_: Optional[IO[Any]] = None
self._size: Optional[int] = None
self._maximum_address: Optional[int] = None
# Construct the lock now (shared if made before threading) in case we ever need it
self._lock: Union[DummyLock, threading.Lock] = DummyLock()
if constants.PARALLELISM == constants.Parallelism.Threading:
@@ -113,14 +114,15 @@ class FileLayer(interfaces.layers.DataLayerInterface):
def maximum_address(self) -> int:
"""Returns the largest available address in the space."""
# Zero based, so we return the size of the file minus 1
if self._size:
return self._size
if self._maximum_address:
return self._maximum_address
with self._lock:
orig = self._file.tell()
self._file.seek(0, 2)
self._size = self._file.tell()
self._file.seek(orig)
return self._size
self._maximum_address = self._size - 1
return self._maximum_address
@property
def minimum_address(self) -> int:
@@ -189,7 +191,7 @@ class FileLayer(interfaces.layers.DataLayerInterface):
"""Closes the file handle."""
self._file.close()
def __del__(self) -> None:
def __exit__(self, type, value, traceback) -> None:
self.destroy()
@classmethod
volatility3/framework/layers/qemu.py
+122 -10
View File
@@ -3,12 +3,17 @@
#
import functools
import json
from typing import Optional, Dict, Any, Tuple, List, Set
import logging
import re
import struct
from typing import Any, Dict, List, Optional, Set, Tuple
from volatility3.framework import interfaces, exceptions, constants
from volatility3.framework.layers import segmented
from volatility3.framework import constants, exceptions, interfaces
from volatility3.framework.layers import scanners, segmented
from volatility3.framework.symbols import intermed
vollog = logging.getLogger(__name__)
class QemuSuspendLayer(segmented.NonLinearlySegmentedLayer):
"""A Qemu suspend-to-disk translation layer."""
@@ -32,6 +37,34 @@ class QemuSuspendLayer(segmented.NonLinearlySegmentedLayer):
SEGMENT_FLAG_XBZRLE = 0x40
SEGMENT_FLAG_HOOK = 0x80
# See https://qemu.readthedocs.io/en/latest/devel/memory.html for more info
#
# At least the following values could occur for devices using > 3-4 GB RAM:
# +--------------------------------+--------------------------------+------------+-------------+
# | Architecture | Reference Code | Hole Start | Hole End |
# +--------------------------------+--------------------------------+------------+-------------+
# | PC i440FX + PIIX "New Default" | qemu/hw/i386/pc_piix.c:98 | 0xc0000000 | 0x100000000 |
# | PC i440FX + PIIX "Old Default" | qemu/hw/i386/pc_piix.c:98 | 0xe0000000 | 0x100000000 |
# | PC Q35 + ICH9 | qemu/hw/i386/pc_q35.c:141 | 0x80000000 | 0x100000000 |
# | MicroVM | qemu/hw/i386/microvm.c:291 | 0xc0000000 | 0x100000000 |
# | Xen | qemu/hw/i386/xen/xen-hvm.c:248 | 0xf0000000 | 0x100000000 |
# +--------------------------------+--------------------------------+------------+-------------+
#
# For now, we assume that the parameter max-ram-below-4g is not set, since this parameter influences the size
# and location of the memory gap. Deviating hole sizes could eventually be detected for Linux by e.g. scanning
# for dmesg entries with a regex like rb'\[mem (0x[0-9a-f]{4,10})-0x[0-9a-f]{4,10}\] available for PCI devices'
distro_re = r"(\w+[\d{1,2}\.]*)"
pci_hole_table = {re.compile(r"^pc-i440fx-([23456789]|\d\d+)\.\d$"): (0xe0000000, 0xc0000000, 0x100000000),
re.compile(r"^pc-i440fx-[01]\.\d$"): (0xe0000000, 0xe0000000, 0x100000000),
re.compile(r"^pc-q35-\d\.\d$"): (0xb0000000, 0x80000000, 0x100000000),
re.compile(r"^microvm$"): (0xc0000000, 0xc0000000, 0x100000000),
re.compile(r"^xen$"): (0xf0000000, 0xf0000000, 0x100000000),
re.compile(r"^pc-i440fx-" + distro_re + r"$"): (0xe0000000, 0xc0000000, 0x100000000),
re.compile(r"^pc-q35-" + distro_re + r"$"): (0xb0000000, 0x80000000, 0x100000000),
}
def __init__(self,
context: interfaces.context.ContextInterface,
config_path: str,
@@ -39,8 +72,12 @@ class QemuSuspendLayer(segmented.NonLinearlySegmentedLayer):
metadata: Optional[Dict[str, Any]] = None) -> None:
self._qemu_table_name = intermed.IntermediateSymbolTable.create(context, config_path, 'generic', 'qemu')
self._configuration = None
self._architecture = None
self._compressed: Set[int] = set()
self._current_segment_name = b''
self._pci_hole_start = 0
self._pci_hole_end = 0
self._pci_hole_minimum = 0
super().__init__(context = context, config_path = config_path, name = name, metadata = metadata)
@classmethod
@@ -50,6 +87,7 @@ class QemuSuspendLayer(segmented.NonLinearlySegmentedLayer):
raise exceptions.LayerException(name, 'No QEMU magic bytes')
if header[4:] != b'\x00\x00\x00\x03':
raise exceptions.LayerException(name, 'Unsupported QEMU version found')
vollog.debug("QEVM header found")
def _read_configuration(self, base_layer: interfaces.layers.DataLayerInterface, name: str) -> Any:
"""Reads the JSON configuration from the end of the file"""
@@ -73,12 +111,13 @@ class QemuSuspendLayer(segmented.NonLinearlySegmentedLayer):
done = None
segments = []
size_array = {}
base_layer = self.context.layers[self._base_layer]
while not done:
addr = self.context.object(self._qemu_table_name + constants.BANG + 'unsigned long long',
offset = index,
layer_name = self._base_layer)
# Use struct.unpack here for performance improvements
addr = struct.unpack('>Q', base_layer.read(index, 8))[0]
# Flags are stored in the n least significant bits, where n equals the bit-length of pagesize
flags = addr & (page_size - 1)
# addr equals the highest multiple of pagesize <= offset
@@ -86,19 +125,29 @@ class QemuSuspendLayer(segmented.NonLinearlySegmentedLayer):
addr = addr ^ (addr & (page_size - 1))
index += 8
if addr >= self._pci_hole_start:
addr += self._pci_hole_end - self._pci_hole_start
if flags & self.SEGMENT_FLAG_MEM_SIZE:
namelen = self._context.object(self._qemu_table_name + constants.BANG + 'unsigned char',
offset = index,
layer_name = self._base_layer)
while namelen != 0:
# if base_layer.read(index + 1, namelen) == b'pc.ram':
# total_size = self._context.object(self._qemu_table_name + constants.BANG + 'unsigned long long',
# offset = index + 1 + namelen,
# layer_name = self._base_layer)
total_size = self._context.object(self._qemu_table_name + constants.BANG + 'unsigned long long',
offset = index + 1 + namelen,
layer_name = self._base_layer)
size_array[base_layer.read(index + 1, namelen)] = total_size
index += 1 + namelen + 8
namelen = self._context.object(self._qemu_table_name + constants.BANG + 'unsigned char',
offset = index,
layer_name = self._base_layer)
highest_possible_maximum = max([x[0] for x in self.pci_hole_table.values()]) + 1
if size_array.get(b'pc.ram', highest_possible_maximum) < self._pci_hole_minimum:
# Turns off the pci_hole if it's not supposed to be there
vollog.debug(
f"QEVM turning off PCI hole due to small image size: 0x{size_array.get(b'pc.ram'):x} < 0x{self._pci_hole_minimum:x}")
self._pci_hole_start, self._pci_hole_end = 0, 0
if flags & (self.SEGMENT_FLAG_COMPRESS | self.SEGMENT_FLAG_PAGE):
if not (flags & self.SEGMENT_FLAG_CONTINUE):
namelen = self._context.object(self._qemu_table_name + constants.BANG + 'unsigned char',
@@ -130,7 +179,26 @@ class QemuSuspendLayer(segmented.NonLinearlySegmentedLayer):
index = 8
section_info = dict()
current_section_id = -1
arch_detected = False
while section_byte != self.QEVM_EOF and index <= base_layer.maximum_address:
if index > 20 and not arch_detected:
# We're past where the QEVM_CONFIGURATION might be, so set the values
# If no architecture has been set, try to determine it using fallback mechanisms
if not self._architecture:
self._architecture = self._fallback_determine_architecture()
if self._architecture is None:
vollog.log(constants.LOGLEVEL_VV, f"QEVM architecture could not be determined")
# Once all segments have been read, determine the PCI hole if any
for regex in self.pci_hole_table:
if regex.match(self._architecture):
self._pci_hole_minimum, self._pci_hole_start, self._pci_hole_end = self.pci_hole_table[regex]
vollog.log(constants.LOGLEVEL_VVVV, f"QEVM architecture detected as: {self._architecture}")
break
else:
vollog.log(constants.LOGLEVEL_VVVV, f"QEVM unknown architecture found: {self._architecture}")
arch_detected = True
section_byte = self.context.object(self._qemu_table_name + constants.BANG + 'unsigned char',
offset = index,
layer_name = self._base_layer)
@@ -139,6 +207,9 @@ class QemuSuspendLayer(segmented.NonLinearlySegmentedLayer):
section_len = self.context.object(self._qemu_table_name + constants.BANG + 'unsigned long',
offset = index,
layer_name = self._base_layer)
self._architecture = self.context.object(self._qemu_table_name + constants.BANG + 'string',
offset = index + 4, layer_name = self._base_layer,
max_length = section_len)
index += 4 + section_len
elif section_byte == self.QEVM_SECTION_START or section_byte == self.QEVM_SECTION_FULL:
section_id = self.context.object(self._qemu_table_name + constants.BANG + 'unsigned long',
@@ -189,6 +260,47 @@ class QemuSuspendLayer(segmented.NonLinearlySegmentedLayer):
else:
raise exceptions.LayerException(self._name, f'QEMU unknown section encountered: {section_byte}')
def _fallback_determine_architecture(self) -> str:
architecture_pattern = rb'pc-(i440fx|q35)-(\d{1,2}\.\d{1,2}|\w+[\d{1,2}\.]*)'
default_suffix = "-2.0"
base_layer = self.context.layers[self._base_layer]
vollog.log(constants.LOGLEVEL_VVVV, "QEVM fallback architecture detection used")
res = scanners.RegExScanner(architecture_pattern)
for offset in base_layer.scan(context = self.context, scanner = res):
line = base_layer.read(offset, 64)
regex_results = re.search(architecture_pattern, line)
architecture = regex_results.group().decode()
return architecture
# If that does not work, look in configuration JSON for devices specific to a certain architecture
architecture = None
for device in self._configuration.get('devices', []):
device_name = device.get('vmsd_name', '').lower()
if 'i440fx' in device_name or 'piix' in device_name:
architecture = 'pc-i440fx' + default_suffix
break
elif 'ich9' in device_name:
architecture = 'pc-q35' + default_suffix
break
if architecture:
vollog.log(constants.LOGLEVEL_VVV, f'Architecture version unknown, default used: {default_suffix}')
return architecture
# Still haven't found architecture, switch to fallback-method
architecture_pattern = rb'Standard PC \((i440FX|Q35)'
res = scanners.RegExScanner(architecture_pattern)
for offset in base_layer.scan(context = self.context, scanner = res):
line = base_layer.read(offset, 64)
regex_results = re.search(architecture_pattern, line)
architecture = "pc-" + regex_results.groups()[0].decode().lower() + default_suffix
vollog.log(constants.LOGLEVEL_VVV, f'Architecture version unknown, default used: {default_suffix}')
return architecture
vollog.warning("Could not determine QEMU target architecture!")
return None
def extract_data(self, index, name, version_id):
if name == 'ram':
if version_id != 4:
volatility3/framework/layers/registry.py
+9 -12
View File
@@ -1,7 +1,7 @@
# This file is Copyright 2019 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
import contextlib
import logging
from typing import Any, Callable, Dict, Iterable, List, Optional, Tuple, Union
@@ -92,11 +92,9 @@ class RegistryHive(linear.LinearlyMappedLayer):
@property
def root_cell_offset(self) -> int:
"""Returns the offset for the root cell in this hive."""
try:
with contextlib.suppress(InvalidAddressException):
if self._base_block.Signature.cast("string", max_length = 4, encoding = "latin-1") == 'regf':
return self._base_block.RootCell
except InvalidAddressException:
pass
return 0x20
def get_cell(self, cell_offset: int) -> 'objects.StructType':
@@ -201,11 +199,11 @@ class RegistryHive(linear.LinearlyMappedLayer):
if offset & 0x7fffffff > self._get_hive_maxaddr(volatile):
vollog.log(constants.LOGLEVEL_VVV,
"Layer {} couldn't translate offset {}, greater than {} in {} store of {}".format(
self.name,
hex(offset & 0x7fffffff),
hex(self._get_hive_maxaddr(volatile)),
"volative" if volatile else "non-volatile",
self.get_name()))
self.name,
hex(offset & 0x7fffffff),
hex(self._get_hive_maxaddr(volatile)),
"volative" if volatile else "non-volatile",
self.get_name()))
raise RegistryInvalidIndex(self.name, "Mapping request for value greater than maxaddr")
storage = self.hive.Storage[volatile]
@@ -252,14 +250,13 @@ class RegistryHive(linear.LinearlyMappedLayer):
def is_valid(self, offset: int, length: int = 1) -> bool:
"""Returns a boolean based on whether the offset is valid or not."""
try:
with contextlib.suppress(exceptions.InvalidAddressException):
# Pass this to the lower layers for now
return all([
self.context.layers[layer].is_valid(offset, length)
for (_, _, offset, length, layer) in self.mapping(offset, length)
])
except exceptions.InvalidAddressException:
return False
return False
@property
def minimum_address(self) -> int:
volatility3/framework/layers/resources.py
+3 -3
View File
@@ -171,6 +171,8 @@ class ResourceAccessor(object):
cache_file.write(block)
block = fp.read(block_size)
cache_file.close()
else:
vollog.debug(f"Using already cached file at: {temp_filename}")
# Re-open the cache with a different mode
# Since we don't want people thinking they're able to save to the cache file,
# open it in read mode only and allow breakages to happen if they wanted to write
@@ -182,14 +184,12 @@ class ResourceAccessor(object):
stop = False
while not stop:
detected = None
try:
with contextlib.suppress(AttributeError, IOError):
# Detect the content
detected = magic.detect_from_fobj(curfile)
IMPORTED_MAGIC = True
# This is because python-magic and file provide a magic module
# Only file's python has magic.detect_from_fobj
except (AttributeError, IOError):
pass
if detected:
if detected.mime_type == 'application/x-xz':
volatility3/framework/layers/vmware.py
+11 -15
View File
@@ -1,14 +1,14 @@
# This file is Copyright 2019 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
import contextlib
import logging
import struct
from typing import Any, Dict, List, Optional
from volatility3.framework import interfaces, constants, exceptions
from volatility3.framework import constants, exceptions, interfaces
from volatility3.framework.configuration import requirements
from volatility3.framework.layers import physical, segmented, resources
from volatility3.framework.layers import physical, resources, segmented
from volatility3.framework.symbols import native
vollog = logging.getLogger(__name__)
@@ -87,13 +87,13 @@ class VmwareLayer(segmented.SegmentedLayer):
offset = offset + name_len + 2 + (index * index_len),
layer_name = self._meta_layer))
data_len = flags & 0x3f
if data_len in [62, 63]: # Handle special data sizes that indicate a longer data stream
data_len = 4 if version == 0 else 8
# Read the size of the data
data_size = self._context.object(self._choose_type(data_len),
layer_name = self._meta_layer,
offset = offset + 2 + name_len + (indices_len * index_len))
layer_name = self._meta_layer,
offset = offset + 2 + name_len + (indices_len * index_len))
# Skip two bytes of padding (as it seems?)
# Read the actual data
data = self._context.object("vmware!bytes",
@@ -113,9 +113,9 @@ class VmwareLayer(segmented.SegmentedLayer):
if tags[("regionsCount", ())][1] == 0:
raise VmwareFormatException(self.name, "VMware VMEM is not split into regions")
for region in range(tags[("regionsCount", ())][1]):
offset = tags[("regionPPN", (region, ))][1] * self._page_size
mapped_offset = tags[("regionPageNum", (region, ))][1] * self._page_size
length = tags[("regionSize", (region, ))][1] * self._page_size
offset = tags[("regionPPN", (region,))][1] * self._page_size
mapped_offset = tags[("regionPageNum", (region,))][1] * self._page_size
length = tags[("regionSize", (region,))][1] * self._page_size
self._segments.append((offset, mapped_offset, length, length))
@property
@@ -153,23 +153,19 @@ class VmwareStacker(interfaces.automagic.StackerLayerInterface):
current_layer_name)
vmss_success = False
try:
with contextlib.suppress(IOError):
_ = resources.ResourceAccessor().open(vmss).read(10)
context.config[interfaces.configuration.path_join(current_config_path, "location")] = vmss
context.layers.add_layer(physical.FileLayer(context, current_config_path, current_layer_name))
vmss_success = True
except IOError:
pass
vmsn_success = False
if not vmss_success:
try:
with contextlib.suppress(IOError):
_ = resources.ResourceAccessor().open(vmsn).read(10)
context.config[interfaces.configuration.path_join(current_config_path, "location")] = vmsn
context.layers.add_layer(physical.FileLayer(context, current_config_path, current_layer_name))
vmsn_success = True
except IOError:
pass
vollog.log(constants.LOGLEVEL_VVVV, f"Metadata found: VMSS ({vmss_success}) or VMSN ({vmsn_success})")
volatility3/framework/objects/__init__.py
+17
View File
@@ -602,6 +602,14 @@ class Array(interfaces.objects.ObjectInterface, collections.abc.Sequence):
return 0
raise IndexError(f"Member not present in array template: {child}")
@classmethod
def child_template(cls, template: interfaces.objects.Template, child: str) -> interfaces.objects.Template:
"""Returns the template of the child member."""
if 'subtype' in template.vol and child == 'subtype':
return template.vol.subtype
raise IndexError(f"Member not present in array template: {child}")
@overload
def __getitem__(self, i: int) -> interfaces.objects.Template:
...
@@ -715,6 +723,15 @@ class AggregateType(interfaces.objects.ObjectInterface):
raise IndexError(f"Member not present in template: {child}")
return retlist[0]
@classmethod
def child_template(cls, template: interfaces.objects.Template, child: str) -> interfaces.objects.Template:
"""Returns the template of a child to its parent."""
retlist = template.vol.members.get(child, None)
if retlist is None:
raise IndexError(f"Member not present in template: {child}")
return retlist[1]
@classmethod
def has_member(cls, template: interfaces.objects.Template, member_name: str) -> bool:
"""Returns whether the object would contain a member called
volatility3/framework/objects/templates.py
+8 -1
View File
@@ -48,6 +48,12 @@ class ObjectTemplate(interfaces.objects.Template):
plateProxy`)"""
return self.vol.object_class.VolTemplateProxy.relative_child_offset(self, child)
def child_template(self, child: str) -> interfaces.objects.Template:
"""Returns the template of a child of the templated object (see
:class:`~volatility3.framework.interfaces.objects.ObjectInterface.VolTem
plateProxy`)"""
return self.vol.object_class.VolTemplateProxy.child_template(self, child)
def replace_child(self, old_child: interfaces.objects.Template, new_child: interfaces.objects.Template) -> None:
"""Replaces `old_child` for `new_child` in the templated object's child
list (see :class:`~volatility3.framework.interfaces.objects.ObjectInterf
@@ -63,7 +69,7 @@ class ObjectTemplate(interfaces.objects.Template):
object_info: interfaces.objects.ObjectInformation) -> interfaces.objects.ObjectInterface:
"""Constructs the object.
Returns: an object adhereing to the :class:`~volatility3.framework.interfaces.objects.ObjectInterface`
Returns: an object adhering to the :class:`~volatility3.framework.interfaces.objects.ObjectInterface`
"""
arguments: Dict[str, Any] = {}
for arg in self.vol:
@@ -99,6 +105,7 @@ class ReferenceTemplate(interfaces.objects.Template):
size: ClassVar[Any] = property(_unresolved)
replace_child: ClassVar[Any] = _unresolved
relative_child_offset: ClassVar[Any] = _unresolved
child_template: ClassVar[Any] = _unresolved
has_member: ClassVar[Any] = _unresolved
def __call__(self, context: interfaces.context.ContextInterface, object_info: interfaces.objects.ObjectInformation):
volatility3/framework/plugins/frameworkinfo.py
+4
View File
@@ -1,3 +1,7 @@
# This file is Copyright 2022 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
from typing import List
from volatility3 import framework
volatility3/framework/plugins/isfinfo.py
+55 -36
View File
@@ -1,17 +1,16 @@
# This file is Copyright 2020 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
import base64
import json
import logging
import os
import pathlib
import zipfile
from typing import List, Type, Any, Generator
from typing import Generator, List
from volatility3 import schemas, symbols
from volatility3.framework import interfaces, renderers, constants
from volatility3.framework.automagic import mac, linux, symbol_cache
from volatility3.framework import constants, interfaces, renderers
from volatility3.framework.automagic import symbol_cache
from volatility3.framework.configuration import requirements
from volatility3.framework.interfaces import plugins
from volatility3.framework.layers import resources
@@ -23,7 +22,7 @@ class IsfInfo(plugins.PluginInterface):
"""Determines information about the currently available ISF files, or a specific one"""
_required_framework_version = (2, 0, 0)
_version = (1, 0, 0)
_version = (2, 0, 0)
@classmethod
def get_requirements(cls) -> List[interfaces.configuration.RequirementInterface]:
@@ -39,6 +38,13 @@ class IsfInfo(plugins.PluginInterface):
requirements.BooleanRequirement(name = 'validate',
description = 'Validate against schema if possible',
default = False,
optional = True),
requirements.VersionRequirement(name = 'SQLiteCache',
component = symbol_cache.SqliteCache,
version = (1, 0, 0)),
requirements.BooleanRequirement(name = 'live',
description = 'Traverse all files, rather than use the cache',
default = False,
optional = True)
]
@@ -62,14 +68,6 @@ class IsfInfo(plugins.PluginInterface):
if filename.endswith(extension):
yield pathlib.Path(base_name).as_uri()
def _get_banner(self, clazz: Type[symbol_cache.SymbolBannerCache], data: Any) -> str:
"""Gets a banner from an ISF file"""
banner_symbol = data.get('symbols', {}).get(clazz.symbol_name, {}).get('constant_data',
renderers.NotAvailableValue())
if not isinstance(banner_symbol, interfaces.renderers.BaseAbsentValue):
banner_symbol = str(base64.b64decode(banner_symbol), encoding = 'latin-1')
return banner_symbol
def _generator(self):
if self.config.get('isf', None) is not None:
file_list = [self.config['isf']]
@@ -98,33 +96,54 @@ class IsfInfo(plugins.PluginInterface):
def check_valid(data):
return "Unknown"
# Process the filtered list
for entry in filtered_list:
num_types = num_enums = num_bases = num_symbols = 0
windows_info = linux_banner = mac_banner = renderers.NotAvailableValue()
valid = "Unknown"
with resources.ResourceAccessor().open(url = entry) as fp:
try:
data = json.load(fp)
num_symbols = len(data.get('symbols', []))
num_types = len(data.get('user_types', []))
num_enums = len(data.get('enums', []))
num_bases = len(data.get('base_types', []))
if self.config['live']:
# Process the filtered list
for entry in filtered_list:
num_types = num_enums = num_bases = num_symbols = 0
valid = "Unknown"
with resources.ResourceAccessor().open(url = entry) as fp:
try:
data = json.load(fp)
num_symbols = len(data.get('symbols', []))
num_types = len(data.get('user_types', []))
num_enums = len(data.get('enums', []))
num_bases = len(data.get('base_types', []))
linux_banner = self._get_banner(linux.LinuxBannerCache, data)
mac_banner = self._get_banner(mac.MacBannerCache, data)
if not linux_banner and not mac_banner:
windows_info = os.path.splitext(os.path.basename(entry))[0]
valid = check_valid(data)
except (UnicodeDecodeError, json.decoder.JSONDecodeError):
vollog.warning(f"Invalid ISF: {entry}")
yield (0, (entry, valid, num_bases, num_types, num_symbols, num_enums, windows_info, linux_banner,
mac_banner))
identifiers_path = os.path.join(constants.CACHE_PATH, constants.IDENTIFIERS_FILENAME)
identifier_cache = symbol_cache.SqliteCache(identifiers_path)
identifier = identifier_cache.get_identifier(location = entry)
if identifier:
identifier = identifier.decode('utf-8', errors = 'replace')
else:
identifier = renderers.NotAvailableValue()
valid = check_valid(data)
except (UnicodeDecodeError, json.decoder.JSONDecodeError):
vollog.warning(f"Invalid ISF: {entry}")
yield (0, (entry, valid, num_bases, num_types, num_symbols, num_enums, identifier))
else:
identifiers_path = os.path.join(constants.CACHE_PATH, constants.IDENTIFIERS_FILENAME)
cache = symbol_cache.SqliteCache(identifiers_path)
valid = 'Unknown'
for identifier, location in cache.get_identifier_dictionary().items():
num_bases, num_types, num_enums, num_symbols = cache.get_location_statistics(location)
if identifier:
json_hash = cache.get_hash(location)
if json_hash and json_hash in schemas.cached_validations:
valid = 'True (cached)'
if self.config['validate']:
# Even if we're not live, if we've been explicitly asked to validate, then do-so
with resources.ResourceAccessor().open(url = location) as fp:
try:
data = json.load(fp)
valid = check_valid(data)
except (UnicodeDecodeError, json.decoder.JSONDecodeError):
vollog.warning(f"Invalid ISF: {location}")
yield (0, (location, valid, num_bases, num_types, num_symbols, num_enums, str(identifier)))
# Try to open the file, load it as JSON, read the data from it
def run(self):
return renderers.TreeGrid([("URI", str), ("Valid", str),
("Number of base_types", int), ("Number of types", int), ("Number of symbols", int),
("Number of enums", int), ("Windows info", str), ("Linux banner", str),
("Mac banner", str)], self._generator())
("Number of enums", int), ("Identifying information", str)], self._generator())
volatility3/framework/plugins/linux/check_syscall.py
+4 -6
View File
@@ -3,11 +3,11 @@
#
"""A module containing a collection of plugins that produce data typically
found in Linux's /proc file system."""
import contextlib
import logging
from typing import List
from volatility3.framework import exceptions, interfaces
from volatility3.framework import renderers, constants
from volatility3.framework import constants, exceptions, interfaces, renderers
from volatility3.framework.configuration import requirements
from volatility3.framework.interfaces import plugins
from volatility3.framework.renderers import format_hints
@@ -40,11 +40,9 @@ class Check_syscall(plugins.PluginInterface):
symbol_list = []
for sn in vmlinux.symbols:
try:
with contextlib.suppress(exceptions.SymbolError):
# When requesting the symbol from the module, a full resolve is performed
symbol_list.append((vmlinux.get_symbol(sn).address, sn))
except exceptions.SymbolError:
pass
sorted_symbols = sorted(symbol_list)
sym_address = 0
@@ -80,7 +78,7 @@ class Check_syscall(plugins.PluginInterface):
def _get_table_info_disassembly(self, ptr_sz, vmlinux):
"""Find the size of the system call table by disassembling functions
that immediately reference it in their first isntruction This is in the
that immediately reference it in their first instruction This is in the
form 'cmp reg,NR_syscalls'."""
table_size = 0
volatility3/framework/plugins/linux/mountinfo.py
+219
View File
@@ -0,0 +1,219 @@
# This file is Copyright 2021 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
import logging
from collections import namedtuple
from typing import Tuple, List, Iterable, Union
from volatility3.framework import renderers, interfaces
from volatility3.framework.configuration import requirements
from volatility3.framework.interfaces import plugins
from volatility3.plugins.linux import pslist
vollog = logging.getLogger(__name__)
MountInfoData = namedtuple("MountInfoData", ("mnt_id", "parent_id", "st_dev", "mnt_root_path", "path_root",
"mnt_opts", "fields", "mnt_type", "devname", "sb_opts"))
class MountInfo(plugins.PluginInterface):
"""Lists mount points on processes mount namespaces"""
_required_framework_version = (2, 2, 0)
_version = (1, 0, 0)
@classmethod
def get_requirements(cls) -> List[interfaces.configuration.RequirementInterface]:
return [
requirements.ModuleRequirement(name="kernel", description="Linux kernel",
architectures=["Intel32", "Intel64"]),
requirements.PluginRequirement(name="pslist",
plugin=pslist.PsList, version=(2, 0, 0)),
requirements.ListRequirement(name="pids",
description="Filter on specific process IDs.",
element_type=int,
optional=True),
requirements.ListRequirement(name="mntns",
description="Filter results by mount namespace. "
"Otherwise, all of them are shown.",
element_type=int,
optional=True),
requirements.BooleanRequirement(name="mount-format",
description="Shows a brief summary of the mount points information "
"with similar output format to the older /proc/[pid]/mounts or the "
"user-land command 'mount -l'.",
optional=True,
default=False),
]
@classmethod
def _do_get_path(cls, mnt, fs_root) -> Union[None, str]:
"""It mimics the Linux kernel prepend_path function."""
vfsmnt = mnt.mnt
dentry = vfsmnt.get_mnt_root()
path_reversed = []
while dentry != fs_root.dentry or vfsmnt.vol.offset != fs_root.mnt:
if dentry == vfsmnt.get_mnt_root() or dentry.is_root():
parent = mnt.get_mnt_parent().dereference()
# Escaped?
if dentry != vfsmnt.get_mnt_root():
return None
# Global root?
if mnt.vol.offset != parent.vol.offset:
dentry = mnt.get_mnt_mountpoint()
mnt = parent
vfsmnt = mnt.mnt
continue
return None
parent = dentry.d_parent
dname = dentry.d_name.name_as_str()
path_reversed.append(dname.strip("/"))
dentry = parent
path = "/" + "/".join(reversed(path_reversed))
return path
@classmethod
def get_mountinfo(cls, mnt, task) -> Union[None, Tuple[int, int, str, str, str, List[str],
List[str], str, str, List[str]]]:
"""Extract various information about a mount point.
It mimics the Linux kernel show_mountinfo function.
"""
mnt_root = mnt.get_mnt_root()
if not mnt_root:
return None
path_root = cls._do_get_path(mnt, task.fs.root)
if path_root is None:
return None
mnt_root_path = mnt_root.path()
superblock = mnt.get_mnt_sb()
mnt_id: int = mnt.mnt_id
parent_id: int = mnt.mnt_parent.mnt_id
st_dev = f"{superblock.major}:{superblock.minor}"
mnt_opts: List[str] = []
mnt_opts.append(mnt.get_flags_access())
mnt_opts.extend(mnt.get_flags_opts())
# Tagged fields
fields: List[str] = []
if mnt.is_shared():
fields.append(f"shared:{mnt.mnt_group_id}")
if mnt.is_slave():
master = mnt.mnt_master.mnt_group_id
fields.append(f"master:{master}")
dominating_id = mnt.get_dominating_id(task.fs.root)
if dominating_id and dominating_id != master:
fields.append(f"propagate_from:{dominating_id}")
if mnt.is_unbindable():
fields.append("unbindable")
mnt_type = superblock.get_type()
devname = mnt.get_devname()
if not devname:
devname = "none"
sb_opts: List[str] = []
sb_opts.append(superblock.get_flags_access())
sb_opts.extend(superblock.get_flags_opts())
return MountInfoData(mnt_id, parent_id, st_dev, mnt_root_path, path_root, mnt_opts, fields,
mnt_type, devname, sb_opts)
def _get_tasks_mountpoints(self, tasks: Iterable[interfaces.objects.ObjectInterface], per_namespace: bool):
seen_namespaces = set()
for task in tasks:
if not (task and task.fs and task.fs.root and task.nsproxy and task.nsproxy.mnt_ns):
# This task doesn't have all the information required
continue
mnt_namespace = task.nsproxy.mnt_ns
mnt_ns_id = mnt_namespace.get_inode()
if per_namespace:
if mnt_ns_id in seen_namespaces:
continue
else:
seen_namespaces.add(mnt_ns_id)
for mount in mnt_namespace.get_mount_points():
yield task, mount, mnt_ns_id
def _generator(
self,
tasks: Iterable[interfaces.objects.ObjectInterface],
mnt_ns_ids: List[int],
mount_format: bool,
per_namespace: bool) -> Iterable[Tuple[int, Tuple]]:
for task, mnt, mnt_ns_id in self._get_tasks_mountpoints(tasks, per_namespace):
if mnt_ns_ids and mnt_ns_id not in mnt_ns_ids:
continue
mnt_info = self.get_mountinfo(mnt, task)
if mnt_info is None:
continue
if mount_format:
all_opts = set()
all_opts.update(mnt_info.mnt_opts)
all_opts.update(mnt_info.sb_opts)
all_opts_str = ",".join(all_opts)
extra_fields_values = [mnt_info.devname, mnt_info.path_root, mnt_info.mnt_type, all_opts_str]
else:
mnt_opts_str = ",".join(mnt_info.mnt_opts)
fields_str = " ".join(mnt_info.fields)
sb_opts_str = ",".join(mnt_info.sb_opts)
extra_fields_values = [mnt_info.mnt_id, mnt_info.parent_id, mnt_info.st_dev, mnt_info.mnt_root_path,
mnt_info.path_root, mnt_opts_str, fields_str, mnt_info.mnt_type,
mnt_info.devname, sb_opts_str]
fields_values = [mnt_ns_id]
if not per_namespace:
fields_values.append(task.pid)
fields_values.extend(extra_fields_values)
yield (0, fields_values)
def run(self):
pids = self.config.get('pids')
mount_ns_ids = self.config.get('mntns')
mount_format = self.config.get('mount-format')
pid_filter = pslist.PsList.create_pid_filter(pids)
tasks = pslist.PsList.list_tasks(self.context, self.config['kernel'], filter_func=pid_filter)
columns = [("MNT_NS_ID", int)]
# The PID column does not make sense when a PID filter is not specified. In that case, the default behavior is
# to displays the mountpoints per namespace.
if pids:
columns.append(("PID", int))
per_namespace = False
else:
per_namespace = True
if self.config.get('mount-format'):
extra_columns = [("DEVNAME", str), ("PATH", str), ("FSTYPE", str), ("MNT_OPTS", str)]
else:
# /proc/[pid]/mountinfo output format
extra_columns = [("MOUNT ID", int), ("PARENT_ID", int), ("MAJOR:MINOR", str), ("ROOT", str),
("MOUNT_POINT", str), ("MOUNT_OPTIONS", str), ("FIELDS", str), ("FSTYPE", str),
("MOUNT_SRC", str), ("SB_OPTIONS", str)]
columns.extend(extra_columns)
return renderers.TreeGrid(columns, self._generator(tasks, mount_ns_ids, mount_format, per_namespace))
volatility3/framework/plugins/linux/psaux.py
+111
View File
@@ -0,0 +1,111 @@
# This file is Copyright 2022 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
from typing import Optional
from volatility3.framework import exceptions, interfaces, renderers
from volatility3.framework.configuration import requirements
from volatility3.framework.interfaces import plugins
from volatility3.framework.objects import utility
from volatility3.plugins.linux import pslist
class PsAux(plugins.PluginInterface):
""" Lists processes with their command line arguments """
_required_framework_version = (2, 0, 0)
@classmethod
def get_requirements(cls):
# Since we're calling the plugin, make sure we have the plugin's requirements
return [
requirements.ModuleRequirement(name = 'kernel', description = 'Linux kernel',
architectures = ["Intel32", "Intel64"]),
requirements.PluginRequirement(name = 'pslist', plugin = pslist.PsList, version = (2, 0, 0)),
requirements.ListRequirement(name = 'pid',
description = 'Filter on specific process IDs',
element_type = int,
optional = True)
]
def _get_command_line_args(self, task: interfaces.objects.ObjectInterface,
name: str) -> Optional[str]:
"""
Reads the command line arguments of a process
These are stored on the userland stack
Kernel threads re-use the process data structure, but do not have a valid 'mm' pointer
Parameters:
task: task_struct object of the process
name: string name of the process (from task.comm)
"""
# kernel threads never have an mm as they do not have userland mappings
try:
mm = task.mm
except exceptions.InvalidAddressException:
mm = None
if mm:
proc_layer_name = task.add_process_layer()
if proc_layer_name is None:
return renderers.UnreadableValue()
proc_layer = self.context.layers[proc_layer_name]
# read argv from userland
start = task.mm.arg_start
# get the size of the arguments with sanity checking
size_to_read = task.mm.arg_end - task.mm.arg_start
if not (0 < size_to_read <= 4096):
return renderers.UnreadableValue()
# attempt to read it all as partial values are invalid and misleading
try:
argv = proc_layer.read(start, size_to_read)
except exceptions.InvalidAddressException:
return renderers.UnreadableValue()
# the arguments are null byte terminated, replace the nulls with spaces
s = argv.decode().split('\x00')
args = " ".join(s)
else:
# kernel thread
# [ ] mimics ps on a live system
# also helps identify malware masquerading as a kernel thread, which is fairly common
args = "[" + name + "]"
# remove trailing space, if present
if len(args) > 1 and args[-1] == " ":
args = args[:-1]
return args
def _generator(self, tasks):
""" Generates a listing of processes along with command line arguments """
# walk the process list and report the arguments
for task in tasks:
pid = task.pid
try:
ppid = task.parent.pid
except exceptions.InvalidAddressException:
ppid = 0
name = utility.array_to_string(task.comm)
args = self._get_command_line_args(task, name)
yield (0, (pid, ppid, name, args))
def run(self):
filter_func = pslist.PsList.create_pid_filter(self.config.get('pid', None))
return renderers.TreeGrid([("PID", int), ("PPID", int), ("COMM", str), ("ARGS", str)],
self._generator(
pslist.PsList.list_tasks(self.context,
self.config['kernel'],
filter_func = filter_func)))
volatility3/framework/plugins/linux/pstree.py
+1 -1
View File
@@ -19,7 +19,7 @@ class PsTree(pslist.PsList):
"""Finds how deep the PID is in the tasks hierarchy.
Args:
pid: PID to find the level in the hierachy
pid: PID to find the level in the hierarchy
"""
seen = set([pid])
level = 0
volatility3/framework/plugins/mac/kauth_listeners.py
+1 -1
View File
@@ -1,4 +1,4 @@
# This file is opyright 2020 Volatility Foundation and licensed under the Volatility Software License 1.0
# This file is Copyright 2020 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
volatility3/framework/plugins/mac/kevents.py
+1 -1
View File
@@ -74,7 +74,7 @@ class Kevents(interfaces.plugins.PluginInterface):
@classmethod
def _walk_klist_array(cls, kernel, fdp, array_pointer_member, array_size_member):
"""
Convience wrapper for walking an array of lists of kernel events
Convenience wrapper for walking an array of lists of kernel events
Handles invalid address references
"""
try:
volatility3/framework/plugins/timeliner.py
+3 -3
View File
@@ -101,7 +101,7 @@ class Timeliner(interfaces.plugins.PluginInterface):
return [sortable(timestamp) for timestamp in data[2:]]
def _generator(self, runable_plugins: List[TimeLinerInterface]) -> Optional[Iterable[Tuple[int, Tuple]]]:
def _generator(self, runnable_plugins: List[TimeLinerInterface]) -> Optional[Iterable[Tuple[int, Tuple]]]:
"""Takes a timeline, sorts it and output the data from each relevant
row from each plugin."""
# Generate the results for each plugin
@@ -115,9 +115,9 @@ class Timeliner(interfaces.plugins.PluginInterface):
file_data = None
fp = None
for plugin in runable_plugins:
for plugin in runnable_plugins:
plugin_name = plugin.__class__.__name__
self._progress_callback((runable_plugins.index(plugin) * 100) // len(runable_plugins),
self._progress_callback((runnable_plugins.index(plugin) * 100) // len(runnable_plugins),
f"Running plugin {plugin_name}...")
try:
vollog.log(logging.INFO, f"Running {plugin_name}")
volatility3/framework/plugins/windows/cachedump.py
+1 -1
View File
@@ -46,7 +46,7 @@ class Cachedump(interfaces.plugins.PluginInterface):
rc4 = ARC4.new(rc4key)
data = rc4.encrypt(edata) # lgtm [py/weak-cryptographic-algorithm]
else:
# based on Based on code from http://lab.mediaservice.net/code/cachedump.rb
# Based on code from http://lab.mediaservice.net/code/cachedump.rb
aes = AES.new(nlkm[16:32], AES.MODE_CBC, ch)
data = b""
for i in range(0, len(edata), 16):
volatility3/framework/plugins/windows/callbacks.py
+63 -20
View File
@@ -11,7 +11,6 @@ from volatility3.framework.renderers import format_hints
from volatility3.framework.symbols import intermed
from volatility3.framework.symbols.windows import versions
from volatility3.plugins.windows import ssdt
from volatility3.plugins.windows import svcscan
vollog = logging.getLogger(__name__)
@@ -28,7 +27,6 @@ class Callbacks(interfaces.plugins.PluginInterface):
requirements.ModuleRequirement(name = 'kernel', description = 'Windows kernel',
architectures = ["Intel32", "Intel64"]),
requirements.PluginRequirement(name = 'ssdt', plugin = ssdt.SSDT, version = (1, 0, 0)),
requirements.PluginRequirement(name = 'svcscan', plugin = svcscan.SvcScan, version = (1, 0, 0))
]
@staticmethod
@@ -111,30 +109,19 @@ class Callbacks(interfaces.plugins.PluginInterface):
yield symbol_name, callback.Callback, None
@classmethod
def list_registry_callbacks(cls, context: interfaces.context.ContextInterface, layer_name: str, symbol_table: str,
callback_table_name: str) -> Iterable[Tuple[str, int, None]]:
"""Lists all registry callbacks.
Args:
context: The context to retrieve required elements (layers, symbol tables) from
layer_name: The name of the layer on which to operate
symbol_table: The name of the table containing the kernel symbols
callback_table_name: The nae of the table containing the callback symbols
Yields:
A name, location and optional detail string
def _list_registry_callbacks_legacy(cls, context: interfaces.context.ContextInterface, layer_name: str, symbol_table: str,
callback_table_name: str) -> Iterable[Tuple[str, int, None]]:
"""
Lists all registry callbacks from the old format via the CmpCallBackVector.
"""
kvo = context.layers[layer_name].config['kernel_virtual_offset']
ntkrnlmp = context.module(symbol_table, layer_name = layer_name, offset = kvo)
full_type_name = callback_table_name + constants.BANG + "_EX_CALLBACK_ROUTINE_BLOCK"
try:
symbol_offset = ntkrnlmp.get_symbol("CmpCallBackVector").address
symbol_count_offset = ntkrnlmp.get_symbol("CmpCallBackCount").address
except exceptions.SymbolError:
vollog.debug("Cannot find CmpCallBackVector or CmpCallBackCount")
return
symbol_offset = ntkrnlmp.get_symbol("CmpCallBackVector").address
symbol_count_offset = ntkrnlmp.get_symbol("CmpCallBackCount").address
callback_count = ntkrnlmp.object(object_type = "unsigned int", offset = symbol_count_offset)
@@ -155,6 +142,62 @@ class Callbacks(interfaces.plugins.PluginInterface):
if callback.Function != 0:
yield "CmRegisterCallback", callback.Function, None
@classmethod
def _list_registry_callbacks_new(cls, context: interfaces.context.ContextInterface, layer_name: str, symbol_table: str,
callback_table_name: str) -> Iterable[Tuple[str, int, None]]:
"""
Lists all registry callbacks via the CallbackListHead.
"""
kvo = context.layers[layer_name].config['kernel_virtual_offset']
ntkrnlmp = context.module(symbol_table, layer_name = layer_name, offset = kvo)
full_type_name = callback_table_name + constants.BANG + "_CM_CALLBACK_ENTRY"
symbol_offset = ntkrnlmp.get_symbol("CallbackListHead").address
symbol_count_offset = ntkrnlmp.get_symbol("CmpCallBackCount").address
callback_count = ntkrnlmp.object(object_type = "unsigned int", offset = symbol_count_offset)
if callback_count == 0:
return
callback_list = ntkrnlmp.object(object_type = "_LIST_ENTRY", offset = symbol_offset)
for callback in callback_list.to_list(full_type_name, "Link"):
yield "CmRegisterCallbackEx", callback.Function, f"Altitude: {callback.Altitude.String}"
@classmethod
def list_registry_callbacks(cls, context: interfaces.context.ContextInterface, layer_name: str, symbol_table: str,
callback_table_name: str) -> Iterable[Tuple[str, int, None]]:
"""Lists all registry callbacks.
Args:
context: The context to retrieve required elements (layers, symbol tables) from
layer_name: The name of the layer on which to operate
symbol_table: The name of the table containing the kernel symbols
callback_table_name: The nae of the table containing the callback symbols
Yields:
A name, location and optional detail string
"""
kvo = context.layers[layer_name].config['kernel_virtual_offset']
ntkrnlmp = context.module(symbol_table, layer_name = layer_name, offset = kvo)
if ntkrnlmp.has_symbol("CmpCallBackVector") and ntkrnlmp.has_symbol("CmpCallBackCount"):
yield from cls._list_registry_callbacks_legacy(context, layer_name, symbol_table, callback_table_name)
elif ntkrnlmp.has_symbol("CallbackListHead") and ntkrnlmp.has_symbol("CmpCallBackCount"):
yield from cls._list_registry_callbacks_new(context, layer_name, symbol_table, callback_table_name)
else:
symbols_to_check = ["CmpCallBackVector", "CmpCallBackCount", "CallbackListHead"]
vollog.debug("Failed to get registry callbacks!")
for symbol_name in symbols_to_check:
symbol_status = "does not exist"
if ntkrnlmp.has_symbol(symbol_name):
symbol_status = "exists"
vollog.debug(f"symbol {symbol_name} {symbol_status}.")
return
@classmethod
def list_bugcheck_reason_callbacks(cls, context: interfaces.context.ContextInterface, layer_name: str,
symbol_table: str, callback_table_name: str) -> Iterable[Tuple[str, int, str]]:
volatility3/framework/plugins/windows/devicetree.py
+5 -5
View File
@@ -78,7 +78,7 @@ class DeviceTree(interfaces.plugins.PluginInterface):
"""Listing tree based on drivers and attached devices in a particular windows memory image."""
_required_framework_version = (2, 0, 3)
_version = (1, 0, 0)
_version = (1, 0, 1)
@classmethod
def get_requirements(cls) -> List[interfaces.configuration.RequirementInterface]:
@@ -96,7 +96,7 @@ class DeviceTree(interfaces.plugins.PluginInterface):
try:
try:
driver_name = driver.get_driver_name()
except (ValueError, exceptions.PagedInvalidAddressException):
except (ValueError, exceptions.InvalidAddressException):
vollog.log(constants.LOGLEVEL_VVVV,
f"Failed to get Driver name : {driver.vol.offset:x}")
driver_name = renderers.UnparsableValue()
@@ -114,7 +114,7 @@ class DeviceTree(interfaces.plugins.PluginInterface):
for device in driver.get_devices():
try:
device_name = device.get_device_name()
except (ValueError, exceptions.PagedInvalidAddressException):
except (ValueError, exceptions.InvalidAddressException):
vollog.log(constants.LOGLEVEL_VVVV,
f"Failed to get Device name : {device.vol.offset:x}")
device_name = renderers.UnparsableValue()
@@ -134,7 +134,7 @@ class DeviceTree(interfaces.plugins.PluginInterface):
for level, attached_device in enumerate(device.get_attached_devices(), start=2):
try:
device_name = attached_device.get_device_name()
except (ValueError, exceptions.PagedInvalidAddressException):
except (ValueError, exceptions.InvalidAddressException):
vollog.log(constants.LOGLEVEL_VVVV,
f"Failed to get Attached Device Name: {attached_device.vol.offset:x}")
device_name = renderers.UnparsableValue()
@@ -151,7 +151,7 @@ class DeviceTree(interfaces.plugins.PluginInterface):
attached_device_type
))
except(exceptions.PagedInvalidAddressException):
except(exceptions.InvalidAddressException):
vollog.log(constants.LOGLEVEL_VVVV,
f"Invalid address identified in drivers and devices: {driver.vol.offset:x}")
continue
volatility3/framework/plugins/windows/dlllist.py
+7 -8
View File
@@ -1,18 +1,19 @@
# This file is Copyright 2019 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
import contextlib
import datetime
import logging
import ntpath
from typing import List, Optional, Type
from volatility3.framework import exceptions, renderers, interfaces, constants
from volatility3.framework import constants, exceptions, interfaces, renderers
from volatility3.framework.configuration import requirements
from volatility3.framework.renderers import format_hints, conversion
from volatility3.framework.renderers import conversion, format_hints
from volatility3.framework.symbols import intermed
from volatility3.framework.symbols.windows.extensions import pe
from volatility3.plugins import timeliner
from volatility3.plugins.windows import pslist, info
from volatility3.plugins.windows import info, pslist
vollog = logging.getLogger(__name__)
@@ -28,7 +29,7 @@ class DllList(interfaces.plugins.PluginInterface, timeliner.TimeLinerInterface):
# Since we're calling the plugin, make sure we have the plugin's requirements
return [
requirements.ModuleRequirement(name = 'kernel', description = 'Windows kernel',
architectures = ["Intel32", "Intel64"]),
architectures = ["Intel32", "Intel64"]),
requirements.VersionRequirement(name = 'pslist', component = pslist.PsList, version = (2, 0, 0)),
requirements.VersionRequirement(name = 'info', component = info.Info, version = (1, 0, 0)),
requirements.ListRequirement(name = 'pid',
@@ -65,7 +66,7 @@ class DllList(interfaces.plugins.PluginInterface, timeliner.TimeLinerInterface):
try:
name = dll_entry.FullDllName.get_string()
except exceptions.InvalidAddressException:
name = 'UnreadbleDLLName'
name = 'UnreadableDLLName'
if layer_name is None:
layer_name = dll_entry.vol.layer_name
@@ -107,12 +108,10 @@ class DllList(interfaces.plugins.PluginInterface, timeliner.TimeLinerInterface):
for entry in proc.load_order_modules():
BaseDllName = FullDllName = renderers.UnreadableValue()
try:
with contextlib.suppress(exceptions.InvalidAddressException):
BaseDllName = entry.BaseDllName.get_string()
# We assume that if the BaseDllName points to an invalid buffer, so will FullDllName
FullDllName = entry.FullDllName.get_string()
except exceptions.InvalidAddressException:
pass
if dll_load_time_field:
# Versions prior to 6.1 won't have the LoadTime attribute
volatility3/framework/plugins/windows/dumpfiles.py
+29 -36
View File
@@ -5,6 +5,7 @@
import logging
import ntpath
from typing import List, Tuple, Type, Optional, Generator
from volatility3.framework import interfaces, renderers, exceptions, constants
from volatility3.framework.configuration import requirements
from volatility3.framework.renderers import format_hints
@@ -32,8 +33,9 @@ class DumpFiles(interfaces.plugins.PluginInterface):
def get_requirements(cls) -> List[interfaces.configuration.RequirementInterface]:
# Since we're calling the plugin, make sure we have the plugin's requirements
return [
requirements.ModuleRequirement(name = 'kernel', description = 'Windows kernel',
architectures = ["Intel32", "Intel64"]),
requirements.ModuleRequirement(name = 'kernel',
description = 'Windows kernel',
architectures = ["Intel32", "Intel64"]),
requirements.IntRequirement(name = 'pid',
description = "Process ID to include (all other processes are excluded)",
optional = True),
@@ -63,29 +65,28 @@ class DumpFiles(interfaces.plugins.PluginInterface):
:return: result status
"""
filedata = open_method(desired_file_name)
try:
# Description of these variables:
# memoffset: offset in the specified layer where the page begins
# fileoffset: write to this offset in the destination file
# datasize: size of the page
# Description of these variables:
# memoffset: offset in the specified layer where the page begins
# fileoffset: write to this offset in the destination file
# datasize: size of the page
# track number of bytes written so we don't write empty files to disk
bytes_written = 0
# track number of bytes written so we don't write empty files to disk
bytes_written = 0
try:
for memoffset, fileoffset, datasize in memory_object.get_available_pages():
data = layer.read(memoffset, datasize, pad = True)
bytes_written += len(data)
filedata.seek(fileoffset)
filedata.write(data)
if not bytes_written:
vollog.debug(f"No data is cached for the file at {file_object.vol.offset:#x}")
return None
else:
vollog.debug(f"Stored {filedata.preferred_filename}")
return filedata
except exceptions.InvalidAddressException:
vollog.debug(f"Unable to dump file at {file_object.vol.offset:#x}")
return None
if not bytes_written:
vollog.debug(f"No data is cached for the file at {file_object.vol.offset:#x}")
return None
vollog.debug(f"Stored {filedata.preferred_filename}")
return filedata
@classmethod
def process_file_object(cls, context: interfaces.context.ContextInterface, primary_layer_name: str,
@@ -98,12 +99,10 @@ class DumpFiles(interfaces.plugins.PluginInterface):
:param open_method: class for constructing output files
:param file_obj: the FILE_OBJECT
"""
# Filtering by these types of devices prevents us from processing other types of devices that
# use the "File" object type, such as \Device\Tcp and \Device\NamedPipe.
if file_obj.DeviceObject.DeviceType not in [FILE_DEVICE_DISK, FILE_DEVICE_NETWORK_FILE_SYSTEM]:
vollog.log(constants.LOGLEVEL_VVV,
f"The file object at {file_obj.vol.offset:#x} is not a file on disk")
vollog.log(constants.LOGLEVEL_VVV, f"The file object at {file_obj.vol.offset:#x} is not a file on disk")
return
# Depending on the type of object (DataSection, ImageSection, SharedCacheMap) we may need to
@@ -120,7 +119,7 @@ class DumpFiles(interfaces.plugins.PluginInterface):
# layer to read from,
# file extension to apply,
# )
dump_parameters = []
dump_parameters = list()
# The DataSectionObject and ImageSectionObject caches are handled in basically the same way.
# We carve these "pages" from the memory_layer.
@@ -131,8 +130,7 @@ class DumpFiles(interfaces.plugins.PluginInterface):
if control_area.is_valid():
dump_parameters.append((control_area, memory_layer, extension))
except exceptions.InvalidAddressException:
vollog.log(constants.LOGLEVEL_VVV,
f"{member_name} is unavailable for file {file_obj.vol.offset:#x}")
vollog.log(constants.LOGLEVEL_VVV, f"{member_name} is unavailable for file {file_obj.vol.offset:#x}")
# The SharedCacheMap is handled differently than the caches above.
# We carve these "pages" from the primary_layer.
@@ -142,8 +140,7 @@ class DumpFiles(interfaces.plugins.PluginInterface):
if shared_cache_map.is_valid():
dump_parameters.append((shared_cache_map, primary_layer, "vacb"))
except exceptions.InvalidAddressException:
vollog.log(constants.LOGLEVEL_VVV,
f"SharedCacheMap is unavailable for file {file_obj.vol.offset:#x}")
vollog.log(constants.LOGLEVEL_VVV, f"SharedCacheMap is unavailable for file {file_obj.vol.offset:#x}")
for memory_object, layer, extension in dump_parameters:
cache_name = EXTENSION_CACHE_MAP[extension]
@@ -151,7 +148,7 @@ class DumpFiles(interfaces.plugins.PluginInterface):
memory_object.vol.offset, cache_name,
ntpath.basename(obj_name), extension)
file_handle = DumpFiles.dump_file_producer(file_obj, memory_object, open_method, layer, desired_file_name)
file_handle = cls.dump_file_producer(file_obj, memory_object, open_method, layer, desired_file_name)
file_output = "Error dumping file"
if file_handle:
@@ -185,8 +182,7 @@ class DumpFiles(interfaces.plugins.PluginInterface):
try:
object_table = proc.ObjectTable
except exceptions.InvalidAddressException:
vollog.log(constants.LOGLEVEL_VVV,
f"Cannot access _EPROCESS.ObjectTable at {proc.vol.offset:#x}")
vollog.log(constants.LOGLEVEL_VVV, f"Cannot access _EPROCESS.ObjectTable at {proc.vol.offset:#x}")
continue
for entry in handles_plugin.handles(object_table):
@@ -218,12 +214,10 @@ class DumpFiles(interfaces.plugins.PluginInterface):
if not file_obj.is_valid():
continue
for result in self.process_file_object(self.context, kernel.layer_name, self.open,
file_obj):
for result in self.process_file_object(self.context, kernel.layer_name, self.open, file_obj):
yield (0, result)
except exceptions.InvalidAddressException:
vollog.log(constants.LOGLEVEL_VVV,
f"Cannot extract file from VAD at {vad.vol.offset:#x}")
vollog.log(constants.LOGLEVEL_VVV, f"Cannot extract file from VAD at {vad.vol.offset:#x}")
elif offsets:
# Now process any offsets explicitly requested by the user.
@@ -234,10 +228,9 @@ class DumpFiles(interfaces.plugins.PluginInterface):
if not is_virtual:
layer_name = self.context.layers[layer_name].config["memory_layer"]
file_obj = self.context.object(
kernel.symbol_table_name + constants.BANG + "_FILE_OBJECT",
file_obj = self.context.object(kernel.symbol_table_name + constants.BANG + "_FILE_OBJECT",
layer_name = layer_name,
native_layer_name = kernel.layer_name,
native_layer_name = kernel.layer_name,
offset = offset)
for result in self.process_file_object(self.context, kernel.layer_name, self.open, file_obj):
yield (0, result)
@@ -246,9 +239,9 @@ class DumpFiles(interfaces.plugins.PluginInterface):
def run(self):
# a list of tuples (<int>, <bool>) where <int> is the address and <bool> is True for virtual.
offsets = []
offsets = list()
# a list of processes matching the pid filter. all files for these process(es) will be dumped.
procs = []
procs = list()
kernel = self.context.modules[self.config['kernel']]
if self.config.get("virtaddr", None) is not None:
volatility3/framework/plugins/windows/envars.py
+7 -14
View File
@@ -1,9 +1,10 @@
# This file is Copyright 2020 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
import contextlib
import logging
from typing import List
from volatility3.framework import renderers, interfaces, objects, exceptions, constants
from volatility3.framework import constants, exceptions, interfaces, objects, renderers
from volatility3.framework.configuration import requirements
from volatility3.framework.layers import registry
from volatility3.plugins.windows import pslist
@@ -23,7 +24,7 @@ class Envars(interfaces.plugins.PluginInterface):
# Since we're calling the plugin, make sure we have the plugin's requirements
return [
requirements.ModuleRequirement(name = 'kernel', description = 'Windows kernel',
architectures = ["Intel32", "Intel64"]),
architectures = ["Intel32", "Intel64"]),
requirements.ListRequirement(name = 'pid',
description = 'Filter on specific process IDs',
element_type = int,
@@ -61,13 +62,11 @@ class Envars(interfaces.plugins.PluginInterface):
key = hive.get_key('CurrentControlSet\\Control\\Session Manager\\Environment')
sys = True
except KeyError:
try:
with contextlib.suppress(KeyError):
key = hive.get_key('ControlSet001\\Control\\Session Manager\\Environment')
sys = True
except KeyError:
pass
if sys:
try:
with contextlib.suppress(KeyError):
for node in key.get_values():
try:
value_node_name = node.get_name()
@@ -78,17 +77,13 @@ class Envars(interfaces.plugins.PluginInterface):
constants.LOGLEVEL_VVV,
"Error while parsing global environment variables keys (some keys might be excluded)")
continue
except KeyError:
pass
## The user-specific variables
try:
with contextlib.suppress(KeyError):
key = hive.get_key('Environment')
ntuser = True
except KeyError:
pass
if ntuser:
try:
with contextlib.suppress(KeyError):
for node in key.get_values():
try:
value_node_name = node.get_name()
@@ -99,8 +94,6 @@ class Envars(interfaces.plugins.PluginInterface):
constants.LOGLEVEL_VVV,
"Error while parsing user environment variables keys (some keys might be excluded)")
continue
except KeyError:
pass
## The volatile user variables
try:
volatility3/framework/plugins/windows/joblinks.py
+75
View File
@@ -0,0 +1,75 @@
# This file is Copyright 2022 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
import logging
from typing import Iterator, List, Tuple
from volatility3.framework import exceptions, interfaces, renderers
from volatility3.framework.configuration import requirements
from volatility3.framework.objects import utility
from volatility3.framework.renderers import format_hints
from volatility3.plugins.windows import pslist
vollog = logging.getLogger(__name__)
class JobLinks(interfaces.plugins.PluginInterface):
"""Print process job link information"""
_required_framework_version = (2, 0, 0)
_version = (1, 0, 0)
@classmethod
def get_requirements(cls) -> List[interfaces.configuration.RequirementInterface]:
return [
requirements.ModuleRequirement(name = 'kernel',
description = 'Windows kernel',
architectures = ["Intel32", "Intel64"]),
requirements.BooleanRequirement(name = 'physical',
description = "Display physical offset instead of virtual",
default = False,
optional = True),
requirements.VersionRequirement(name = 'pslist', component = pslist.PsList, version = (2, 0, 0))
]
def _generator(self) -> Iterator[Tuple]:
kernel = self.context.modules[self.config['kernel']]
memory = self.context.layers[kernel.layer_name]
for proc in pslist.PsList.list_processes(self.context, kernel.layer_name, kernel.symbol_table_name):
try:
if not self.config['physical']:
offset = proc.vol.offset
else:
(_, _, offset, _, _) = list(memory.mapping(offset = proc.vol.offset, length = 0))[0]
job = proc.Job.dereference()
yield (0, (format_hints.Hex(offset), utility.array_to_string(proc.ImageFileName), proc.UniqueProcessId,
proc.InheritedFromUniqueProcessId, proc.get_session_id(), job.SessionId, proc.get_is_wow64(),
job.TotalProcesses, job.ActiveProcesses, job.TotalTerminatedProcesses,
renderers.NotApplicableValue(), "(Original Process)"))
for entry in job.ProcessListHead.to_list(proc.vol.type_name, "JobLinks"):
if not self.config['physical']:
offset = entry.vol.offset
else:
(_, _, offset, _, _) = list(memory.mapping(offset = entry.vol.offset, length = 0))[0]
yield (1, (format_hints.Hex(offset), utility.array_to_string(entry.ImageFileName),
entry.UniqueProcessId, entry.InheritedFromUniqueProcessId, entry.get_session_id(), 0,
entry.get_is_wow64(), 0, 0, 0, "Yes",
entry.get_peb().ProcessParameters.ImagePathName.get_string()))
except (exceptions.InvalidAddressException):
continue
def run(self) -> renderers.TreeGrid:
offsettype = "(V)" if not self.config.get('physical', pslist.PsList.PHYSICAL_DEFAULT) else "(P)"
return renderers.TreeGrid([(f"Offset{offsettype}", format_hints.Hex), ("Name", str),
("PID", int), ("PPID", int), ("Sess", int), ("JobSess", int), ("Wow64", bool),
("Total", int), ("Active", int), ("Term", int), ("JobLink", str), ("Process", str)],
self._generator())
volatility3/framework/plugins/windows/ldrmodules.py
+7 -8
View File
@@ -1,5 +1,4 @@
from volatility3.framework import interfaces, constants
from volatility3.framework import renderers, interfaces, exceptions
from volatility3.framework import constants, exceptions, interfaces, renderers
from volatility3.framework.configuration import requirements
from volatility3.framework.renderers import format_hints
from volatility3.framework.symbols import intermed
@@ -71,14 +70,14 @@ class LdrModules(interfaces.plugins.PluginInterface):
mem_mod = mem_order_mod.get(base, None)
yield (0, [int(proc.UniqueProcessId),
str(proc.ImageFileName.cast("string",
str(proc.ImageFileName.cast("string",
max_length = proc.ImageFileName.vol.count,
errors = 'replace')),
format_hints.Hex(base),
load_mod != None,
init_mod != None,
mem_mod != None,
mapped_files[base]])
format_hints.Hex(base),
load_mod is not None,
init_mod is not None,
mem_mod is not None,
mapped_files[base]])
def run(self):
filter_func = pslist.PsList.create_pid_filter(self.config.get('pid', None))
volatility3/framework/plugins/windows/malfind.py
+2 -2
View File
@@ -17,7 +17,7 @@ vollog = logging.getLogger(__name__)
class Malfind(interfaces.plugins.PluginInterface):
"""Lists process memory ranges that potentially contain injected code."""
_required_framework_version = (2, 0, 0)
_required_framework_version = (2, 4, 0)
@classmethod
def get_requirements(cls):
@@ -56,7 +56,7 @@ class Malfind(interfaces.plugins.PluginInterface):
all_zero_page = b"\x00" * CHUNK_SIZE
offset = 0
vad_length = vad.get_end() - vad.get_start()
vad_length = vad.get_size()
while offset < vad_length:
next_addr = vad.get_start() + offset
volatility3/framework/plugins/windows/mftscan.py
+2 -5
View File
@@ -1,7 +1,7 @@
# This file is Copyright 2022 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
import contextlib
import datetime
import logging
@@ -56,7 +56,7 @@ class MFTScan(interfaces.plugins.PluginInterface, timeliner.TimeLinerInterface):
# Scan the layer for Raw MFT records and parse the fields
for offset, _rule_name, _name, _value in layer.scan(context = self.context,
scanner = yarascan.YaraScanner(rules = rules)):
try:
with contextlib.suppress(exceptions.PagedInvalidAddressException):
mft_record = self.context.object(mft_object, offset = offset, layer_name = layer.name)
# We will update this on each pass in the next loop and use it as the new offset.
attr_base_offset = mft_record.FirstAttrOffset
@@ -131,9 +131,6 @@ class MFTScan(interfaces.plugins.PluginInterface, timeliner.TimeLinerInterface):
offset = offset + attr_base_offset,
layer_name = layer.name)
except exceptions.PagedInvalidAddressException:
pass
def generate_timeline(self):
for row in self._generator():
_depth, row_data = row
volatility3/framework/plugins/windows/modscan.py
+1 -1
View File
@@ -25,7 +25,7 @@ class ModScan(interfaces.plugins.PluginInterface):
return [
requirements.ModuleRequirement(name = 'kernel', description = 'Windows kernel',
architectures = ["Intel32", "Intel64"]),
requirements.VersionRequirement(name = 'poolerscanner',
requirements.VersionRequirement(name = 'poolscanner',
component = poolscanner.PoolScanner,
version = (1, 0, 0)),
requirements.VersionRequirement(name = 'pslist', component = pslist.PsList, version = (2, 0, 0)),
volatility3/framework/plugins/windows/netstat.py
+1 -1
View File
@@ -433,7 +433,7 @@ class NetStat(interfaces.plugins.PluginInterface, timeliner.TimeLinerInterface):
self.context, interfaces.configuration.path_join(self.config_path, 'tcpip'),
kernel.layer_name, "tcpip.pdb", tcpip_module.DllBase, tcpip_module.SizeOfImage)
except exceptions.VolatilityException:
vollog.warning("Unable to locate symbols for the memory image's tcpip module")
vollog.error("Unable to locate symbols for the memory image's tcpip module")
for netw_obj in self.list_sockets(self.context, kernel.layer_name, kernel.symbol_table_name,
netscan_symbol_table, tcpip_module.DllBase, tcpip_symbol_table):
volatility3/framework/plugins/windows/psscan.py
+1 -1
View File
@@ -22,7 +22,7 @@ vollog = logging.getLogger(__name__)
class PsScan(interfaces.plugins.PluginInterface, timeliner.TimeLinerInterface):
"""Scans for processes present in a particular windows memory image."""
_required_framework_version = (2, 0, 0)
_required_framework_version = (2, 3, 1)
_version = (1, 1, 0)
@classmethod
volatility3/framework/plugins/windows/registry/userassist.py
+7 -11
View File
@@ -3,17 +3,18 @@
#
import codecs
import contextlib
import datetime
import json
import logging
import os
from typing import Any, List, Tuple, Generator
from typing import Any, Generator, List, Tuple
from volatility3.framework import exceptions, renderers, constants, interfaces
from volatility3.framework import constants, exceptions, interfaces, renderers
from volatility3.framework.configuration import requirements
from volatility3.framework.layers.physical import BufferDataLayer
from volatility3.framework.layers.registry import RegistryHive
from volatility3.framework.renderers import format_hints, conversion
from volatility3.framework.renderers import conversion, format_hints
from volatility3.framework.symbols import intermed
from volatility3.plugins.windows.registry import hivelist
@@ -38,7 +39,7 @@ class UserAssist(interfaces.plugins.PluginInterface):
def get_requirements(cls) -> List[interfaces.configuration.RequirementInterface]:
return [
requirements.ModuleRequirement(name = 'kernel', description = 'Windows kernel',
architectures = ["Intel32", "Intel64"]),
architectures = ["Intel32", "Intel64"]),
requirements.IntRequirement(name = 'offset', description = "Hive Offset", default = None, optional = True),
requirements.PluginRequirement(name = 'hivelist', plugin = hivelist.HiveList, version = (1, 0, 0))
]
@@ -126,11 +127,9 @@ class UserAssist(interfaces.plugins.PluginInterface):
hive_name = hive.hive.cast(kernel.symbol_table_name + constants.BANG + "_CMHIVE").get_name()
if self._win7 is None:
try:
with contextlib.suppress(exceptions.SymbolError):
self._win7 = self._win7_or_later()
except exceptions.SymbolError:
# self._win7 will be None and only registry value rawdata will be output
pass
self._determine_userassist_type()
@@ -163,7 +162,6 @@ class UserAssist(interfaces.plugins.PluginInterface):
# output any subkeys under Count
for subkey in countkey.get_subkeys():
subkey_name = subkey.get_name()
result = (1, (
renderers.format_hints.Hex(hive.hive_offset),
@@ -185,10 +183,8 @@ class UserAssist(interfaces.plugins.PluginInterface):
for value in countkey.get_values():
value_name = value.get_name()
try:
with contextlib.suppress(UnicodeDecodeError):
value_name = codecs.encode(value_name, "rot_13")
except UnicodeDecodeError:
pass
if self._win7:
guid = value_name.split("\\")[0]
volatility3/framework/plugins/windows/skeleton_key_check.py
+3 -3
View File
@@ -9,7 +9,7 @@
# For a thorough walkthrough on how the R&D was performed to develop this plugin,
# please see our blogpost here:
#
# <insert blog URL once published>
# https://volatility-labs.blogspot.com/2021/10/memory-forensics-r-illustrated.html
import io
import logging
@@ -41,7 +41,7 @@ vollog = logging.getLogger(__name__)
class Skeleton_Key_Check(interfaces.plugins.PluginInterface):
""" Looks for signs of Skeleton Key malware """
_required_framework_version = (2, 0, 0)
_required_framework_version = (2, 4, 0)
@classmethod
def get_requirements(cls):
@@ -262,7 +262,7 @@ class Skeleton_Key_Check(interfaces.plugins.PluginInterface):
if isinstance(filename, str) and filename.lower().endswith("cryptdll.dll"):
base = vad.get_start()
return base, vad.get_end() - base
return base, vad.get_size()
return None, None
volatility3/framework/plugins/windows/ssdt.py
+2 -2
View File
@@ -95,10 +95,10 @@ class SSDT(plugins.PluginInterface):
if is_kernel_64:
array_subtype = "long"
def kvo_calulator(func: int) -> int:
def kvo_calculator(func: int) -> int:
return kvo + service_table_address + (func >> 4)
find_address = kvo_calulator
find_address = kvo_calculator
else:
array_subtype = "unsigned long"
volatility3/framework/plugins/windows/vadinfo.py
+5 -4
View File
@@ -33,7 +33,7 @@ winnt_protections = {
class VadInfo(interfaces.plugins.PluginInterface):
"""Lists process memory ranges."""
_required_framework_version = (2, 0, 0)
_required_framework_version = (2, 4, 0)
_version = (2, 0, 0)
MAXSIZE_DEFAULT = 1024 * 1024 * 1024 # 1 Gb
@@ -132,7 +132,7 @@ class VadInfo(interfaces.plugins.PluginInterface):
vollog.debug("Unable to find the starting/ending VPN member")
return None
if maxsize > 0 and (vad_end - vad_start) > maxsize:
if 0 < maxsize < vad.get_size():
vollog.debug(f"Skip VAD dump {vad_start:#x}-{vad_end:#x} due to maxsize limit")
return None
@@ -151,8 +151,9 @@ class VadInfo(interfaces.plugins.PluginInterface):
file_handle = open_method(file_name)
chunk_size = 1024 * 1024 * 10
offset = vad_start
while offset < vad_end:
to_read = min(chunk_size, vad_end - offset)
vad_size = vad.get_size()
while offset < vad_start + vad_size:
to_read = min(chunk_size, vad_start + vad_size - offset)
data = proc_layer.read(offset, to_read, pad = True)
if not data:
break
volatility3/framework/plugins/windows/vadyarascan.py
+2 -4
View File
@@ -17,7 +17,7 @@ vollog = logging.getLogger(__name__)
class VadYaraScan(interfaces.plugins.PluginInterface):
"""Scans all the Virtual Address Descriptor memory maps using yara."""
_required_framework_version = (2, 0, 0)
_required_framework_version = (2, 4, 0)
_version = (1, 0, 0)
@classmethod
@@ -82,9 +82,7 @@ class VadYaraScan(interfaces.plugins.PluginInterface):
"""
vad_root = task.get_vad_root()
for vad in vad_root.traverse():
end = vad.get_end()
start = vad.get_start()
yield (start, end - start)
yield (vad.get_start(), vad.get_size())
def run(self):
return renderers.TreeGrid([('Offset', format_hints.Hex), ('PID', int), ('Rule', str), ('Component', str),
volatility3/framework/renderers/conversion.py
+2 -4
View File
@@ -1,7 +1,7 @@
# This file is Copyright 2019 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
import contextlib
import datetime
import ipaddress
import socket
@@ -27,10 +27,8 @@ def unixtime_to_datetime(unixtime: int) -> Union[interfaces.renderers.BaseAbsent
ret: Union[interfaces.renderers.BaseAbsentValue, datetime.datetime] = renderers.UnparsableValue()
if unixtime > 0:
try:
with contextlib.suppress(ValueError):
ret = datetime.datetime.utcfromtimestamp(unixtime)
except ValueError:
pass
return ret
volatility3/framework/symbols/intermed.py
+1 -2
View File
@@ -202,8 +202,7 @@ class IntermediateSymbolTable(interfaces.symbols.SymbolTableInterface):
pass
# Finally try looking in zip files
zip_path = os.path.join(path, sub_path + ".zip")
if os.path.exists(zip_path):
for zip_path in pathlib.Path(path).joinpath(sub_path).resolve().rglob(filename + '.zip'):
# We have a zipfile, so run through it and look for sub files that match the filename
with zipfile.ZipFile(zip_path) as zfile:
for name in zfile.namelist():
volatility3/framework/symbols/linux/__init__.py
+6 -2
View File
@@ -1,10 +1,10 @@
# This file is Copyright 2019 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
from typing import List, Tuple, Iterator, Optional
from typing import Iterator, List, Tuple, Optional
from volatility3 import framework
from volatility3.framework import exceptions, constants, interfaces, objects
from volatility3.framework import constants, exceptions, interfaces, objects
from volatility3.framework.objects import utility
from volatility3.framework.symbols import intermed
from volatility3.framework.symbols.linux import extensions
@@ -39,9 +39,13 @@ class LinuxKernelIntermedSymbols(intermed.IntermediateSymbolTable):
self.set_type_class('netlink_sock', extensions.netlink_sock)
self.set_type_class('vsock_sock', extensions.vsock_sock)
self.set_type_class('packet_sock', extensions.packet_sock)
if 'bt_sock' in self.types:
self.set_type_class('bt_sock', extensions.bt_sock)
if 'mnt_namespace' in self.types:
self.set_type_class('mnt_namespace', extensions.mnt_namespace)
if 'module' in self.types:
self.set_type_class('module', extensions.module)
volatility3/framework/symbols/linux/extensions/__init__.py
+189 -3
View File
@@ -294,6 +294,29 @@ class super_block(objects.StructType):
# include/linux/kdev_t.h
MINORBITS = 20
# Superblock flags
SB_RDONLY = 1 # Mount read-only
SB_NOSUID = 2 # Ignore suid and sgid bits
SB_NODEV = 4 # Disallow access to device special files
SB_NOEXEC = 8 # Disallow program execution
SB_SYNCHRONOUS = 16 # Writes are synced at once
SB_MANDLOCK = 64 # Allow mandatory locks on an FS
SB_DIRSYNC = 128 # Directory modifications are synchronous
SB_NOATIME = 1024 # Do not update access times
SB_NODIRATIME = 2048 # Do not update directory access times
SB_SILENT = 32768
SB_POSIXACL = (1 << 16) # VFS does not apply the umask
SB_KERNMOUNT = (1 << 22) # this is a kern_mount call
SB_I_VERSION = (1 << 23) # Update inode I_version field
SB_LAZYTIME = (1 << 25) # Update the on-disk [acm]times lazily
SB_OPTS = {
SB_SYNCHRONOUS: "sync",
SB_DIRSYNC: "dirsync",
SB_MANDLOCK: "mand",
SB_LAZYTIME: "lazytime"
}
@property
def major(self) -> int:
return self.s_dev >> self.MINORBITS
@@ -302,6 +325,20 @@ class super_block(objects.StructType):
def minor(self) -> int:
return self.s_dev & ((1 << self.MINORBITS) - 1)
def get_flags_access(self) -> str:
return 'ro' if self.s_flags & self.SB_RDONLY else 'rw'
def get_flags_opts(self) -> Iterable[str]:
sb_opts = [self.SB_OPTS[sb_opt] for sb_opt in self.SB_OPTS if sb_opt & self.s_flags]
return sb_opts
def get_type(self):
mnt_sb_type = utility.pointer_to_string(self.s_type.name, count=255)
if self.s_subtype:
mnt_sb_subtype = utility.pointer_to_string(self.s_subtype, count=255)
mnt_sb_type += "." + mnt_sb_subtype
return mnt_sb_type
class vm_area_struct(objects.StructType):
perm_flags = {
@@ -378,7 +415,7 @@ class vm_area_struct(objects.StructType):
fname = linux.LinuxUtilities.path_for_file(context, task, self.vm_file)
elif self.vm_start <= task.mm.start_brk and self.vm_end >= task.mm.brk:
fname = "[heap]"
elif self.vm_start <= task.mm.start_stack and self.vm_end >= task.mm.start_stack:
elif self.vm_start <= task.mm.start_stack <= self.vm_end:
fname = "[stack]"
elif self.vm_mm.context.has_member("vdso") and self.vm_start == self.vm_mm.context.vdso:
fname = "[vdso]"
@@ -421,7 +458,50 @@ class qstr(objects.StructType):
class dentry(objects.StructType):
def path(self) -> str:
return self.d_name.name_as_str()
"""Based on __dentry_path Linux kernel function"""
reversed_path = []
dentry_seen = set()
current_dentry = self
while (not current_dentry.is_root() and
current_dentry.vol.offset not in dentry_seen):
parent = current_dentry.d_parent
reversed_path.append(current_dentry.d_name.name_as_str())
dentry_seen.add(current_dentry.vol.offset)
current_dentry = parent
return "/" + "/".join(reversed(reversed_path))
def is_root(self) -> bool:
return self.vol.offset == self.d_parent
def is_subdir(self, old_dentry):
"""Is this dentry a subdirectory of old_dentry?
Returns true if this dentry is a subdirectory of the parent (at any depth).
Otherwise, it returns false.
"""
if self.vol.offset == old_dentry:
return True
return self.d_ancestor(old_dentry)
def d_ancestor(self, ancestor_dentry):
"""Search for an ancestor
Returns the ancestor dentry which is a child of "ancestor_dentry",
if "ancestor_dentry" is an ancestor of "child_dentry", else None.
"""
dentry_seen = set()
current_dentry = self
while (not current_dentry.is_root() and
current_dentry.vol.offset not in dentry_seen):
if current_dentry.d_parent == ancestor_dentry.vol.offset:
return current_dentry
dentry_seen.add(current_dentry.vol.offset)
current_dentry = current_dentry.d_parent
return None
class struct_file(objects.StructType):
@@ -516,6 +596,27 @@ class files_struct(objects.StructType):
class mount(objects.StructType):
MNT_NOSUID = 0x01
MNT_NODEV = 0x02
MNT_NOEXEC = 0x04
MNT_NOATIME = 0x08
MNT_NODIRATIME = 0x10
MNT_RELATIME = 0x20
MNT_READONLY = 0x40
MNT_SHRINKABLE = 0x100
MNT_WRITE_HOLD = 0x200
MNT_SHARED = 0x1000
MNT_UNBINDABLE = 0x2000
MNT_FLAGS = {
MNT_NOSUID: "nosuid",
MNT_NODEV: "nodev",
MNT_NOEXEC: "noexec",
MNT_NOATIME: "noatime",
MNT_NODIRATIME: "nodiratime",
MNT_RELATIME: "relatime",
}
def get_mnt_sb(self):
if self.has_member("mnt"):
return self.mnt.mnt_sb
@@ -546,6 +647,82 @@ class mount(objects.StructType):
def get_mnt_mountpoint(self):
return self.mnt_mountpoint
def get_flags_access(self) -> str:
return "ro" if self.get_mnt_flags() & self.MNT_READONLY else "rw"
def get_flags_opts(self) -> Iterable[str]:
flags = [self.MNT_FLAGS[mntflag] for mntflag in self.MNT_FLAGS if mntflag & self.get_mnt_flags()]
return flags
def is_shared(self) -> bool:
return self.get_mnt_flags() & self.MNT_SHARED
def is_unbindable(self) -> bool:
return self.get_mnt_flags() & self.MNT_UNBINDABLE
def is_slave(self) -> bool:
return self.mnt_master and self.mnt_master.vol.offset != 0
def get_devname(self) -> str:
return utility.pointer_to_string(self.mnt_devname, count=255)
def has_parent(self) -> bool:
return self.vol.offset != self.mnt_parent
def get_dominating_id(self, root) -> int:
"""Get ID of closest dominating peer group having a representative under the given root."""
mnt_seen = set()
current_mnt = self.mnt_master
while (current_mnt and
current_mnt.vol.offset != 0 and
current_mnt.vol.offset not in mnt_seen):
peer = current_mnt.get_peer_under_root(self.mnt_ns, root)
if peer and peer.vol.offset != 0:
return peer.mnt_group_id
mnt_seen.add(current_mnt.vol.offset)
current_mnt = current_mnt.mnt_master
return 0
def get_peer_under_root(self, ns, root):
"""Return true if path is reachable from root.
It mimics the kernel function is_path_reachable(), ref: fs/namespace.c
"""
mnt_seen = set()
current_mnt = self
while current_mnt.vol.offset not in mnt_seen:
if current_mnt.mnt_ns == ns and current_mnt.is_path_reachable(current_mnt.mnt.mnt_root, root):
return current_mnt
mnt_seen.add(current_mnt.vol.offset)
current_mnt = current_mnt.next_peer()
if current_mnt.vol.offset == self.vol.offset:
break
return None
def is_path_reachable(self, current_dentry, root):
"""Return true if path is reachable.
It mimics the kernel function with same name, ref fs/namespace.c:
"""
mnt_seen = set()
current_mnt = self
while (current_mnt.mnt.vol.offset != root.mnt and
current_mnt.has_parent() and
current_mnt.vol.offset not in mnt_seen):
current_dentry = current_mnt.mnt_mountpoint
mnt_seen.add(current_mnt.vol.offset)
current_mnt = current_mnt.mnt_parent
return current_mnt.mnt.vol.offset == root.mnt and current_dentry.is_subdir(root.dentry)
def next_peer(self):
table_name = self.vol.type_name.split(constants.BANG)[0]
mount_struct = "{0}{1}mount".format(table_name, constants.BANG)
offset = self._context.symbol_space.get_type(mount_struct).relative_child_offset("mnt_share")
return self._context.object(mount_struct, self.vol.layer_name, offset=self.mnt_share.next.vol.offset - offset)
class vfsmount(objects.StructType):
@@ -576,7 +753,6 @@ class vfsmount(objects.StructType):
def get_mnt_root(self):
return self.mnt_root
class kobject(objects.StructType):
def reference_count(self):
@@ -597,6 +773,16 @@ class mnt_namespace(objects.StructType):
else:
raise AttributeError("Unable to find mnt_namespace inode")
def get_mount_points(self):
table_name = self.vol.type_name.split(constants.BANG)[0]
mnt_type = table_name + constants.BANG + "mount"
if not self._context.symbol_space.has_type(mnt_type):
# Old kernels ~ 2.6
mnt_type = table_name + constants.BANG + "vfsmount"
for mount in self.list.to_list(mnt_type, "mnt_list"):
yield mount
class net(objects.StructType):
def get_inode(self):
if self.has_member("proc_inum"):
volatility3/framework/symbols/mac/extensions/__init__.py
+11 -21
View File
@@ -1,19 +1,18 @@
# This file is Copyright 2019 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
import contextlib
import logging
from typing import Generator, Iterable, Optional, Set, Tuple
import logging
from volatility3.framework import constants, objects, renderers
from volatility3.framework import exceptions, interfaces
from volatility3.framework import constants, exceptions, interfaces, objects
from volatility3.framework.objects import utility
from volatility3.framework.renderers import conversion
from volatility3.framework.symbols import generic
vollog = logging.getLogger(__name__)
class proc(generic.GenericIntelProcess):
def get_task(self):
@@ -29,10 +28,8 @@ class proc(generic.GenericIntelProcess):
if not isinstance(parent_layer, interfaces.layers.TranslationLayerInterface):
raise TypeError("Parent layer is not a translation layer, unable to construct process layer")
try:
with contextlib.suppress(exceptions.InvalidAddressException):
dtb = self.get_task().map.pmap.pm_cr3
except exceptions.InvalidAddressException:
return None
if preferred_name is None:
preferred_name = self.vol.layer_name + f"_Process{self.p_pid}"
@@ -41,10 +38,8 @@ class proc(generic.GenericIntelProcess):
return self._add_process_layer(self._context, dtb, config_prefix, preferred_name)
def get_map_iter(self) -> Iterable[interfaces.objects.ObjectInterface]:
try:
with contextlib.suppress(exceptions.InvalidAddressException):
task = self.get_task()
except exceptions.InvalidAddressException:
return
try:
current_map = task.map.hdr.links.next
@@ -55,9 +50,9 @@ class proc(generic.GenericIntelProcess):
for i in range(task.map.hdr.nentries):
if (not current_map or
current_map.vol.offset in seen or
not self._context.layers[task.vol.native_layer_name].is_valid(current_map.dereference().vol.offset, current_map.dereference().vol.size)):
current_map.vol.offset in seen or
not self._context.layers[task.vol.native_layer_name].is_valid(current_map.dereference().vol.offset,
current_map.dereference().vol.size)):
vollog.log(constants.LOGLEVEL_VVV, "Breaking process maps iteration due to invalid state.")
break
@@ -102,10 +97,8 @@ class fileglob(objects.StructType):
if self.has_member("fg_type"):
ret = self.fg_type
elif self.fg_ops != 0:
try:
with contextlib.suppress(exceptions.InvalidAddressException):
ret = self.fg_ops.fo_type
except exceptions.InvalidAddressException:
pass
if ret:
ret = str(ret.description).replace("DTYPE_", "")
@@ -456,7 +449,7 @@ class queue_entry(objects.StructType):
seen = set()
for attr in ['next', 'prev']:
try:
with contextlib.suppress(exceptions.InvalidAddressException):
n = getattr(self, attr).dereference().cast(type_name)
while n is not None and n.vol.offset != list_head:
@@ -473,9 +466,6 @@ class queue_entry(objects.StructType):
n = getattr(n.member(attr = member_name), attr).dereference().cast(type_name)
except exceptions.InvalidAddressException:
pass
class ifnet(objects.StructType):
volatility3/framework/symbols/metadata.py
+1 -1
View File
@@ -38,4 +38,4 @@ class WindowsMetadata(interfaces.symbols.MetadataInterface):
class LinuxMetadata(interfaces.symbols.MetadataInterface):
"""Class to handle the etadata from a Linux symbol table."""
"""Class to handle the metadata from a Linux symbol table."""
volatility3/framework/symbols/windows/__init__.py
+6 -8
View File
@@ -1,10 +1,11 @@
# This file is Copyright 2020 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
import contextlib
from volatility3.framework.symbols import intermed
from volatility3.framework.symbols.windows import extensions
from volatility3.framework.symbols.windows.extensions import registry, pool, pe
from volatility3.framework.symbols.windows.extensions import pe, pool, registry
class WindowsKernelIntermedSymbols(intermed.IntermediateSymbolTable):
@@ -39,26 +40,23 @@ class WindowsKernelIntermedSymbols(intermed.IntermediateSymbolTable):
self.set_type_class('_VACB', extensions.VACB)
self.set_type_class('_POOL_TRACKER_BIG_PAGES', pool.POOL_TRACKER_BIG_PAGES)
self.set_type_class('_IMAGE_DOS_HEADER', pe.IMAGE_DOS_HEADER)
# Might not necessarily defined in every version of windows
self.optional_set_type_class('_IMAGE_NT_HEADERS', pe.IMAGE_NT_HEADERS)
self.optional_set_type_class('_IMAGE_NT_HEADERS64', pe.IMAGE_NT_HEADERS)
# This doesn't exist in very specific versions of windows
try:
with contextlib.suppress(ValueError):
if self.get_type("_POOL_TRACKER_BIG_PAGES").has_member("PoolType"):
self.set_type_class('_POOL_HEADER', pool.POOL_HEADER_VISTA)
else:
self.set_type_class('_POOL_HEADER', pool.POOL_HEADER)
except ValueError:
pass
# these don't exist in windows XP
self.optional_set_type_class('_MMADDRESS_NODE', extensions.MMVAD_SHORT)
# these were introduced starting in windows 8
self.optional_set_type_class('_MM_AVL_NODE', extensions.MMVAD_SHORT)
# these were introduced starting in windows 7
self.optional_set_type_class('_RTL_BALANCED_NODE', extensions.MMVAD_SHORT)
volatility3/framework/symbols/windows/callbacks-x64.json
+43
View File
@@ -8,6 +8,12 @@
"signed": false,
"endian": "little"
},
"unsigned long long": {
"kind": "int",
"size": 8,
"signed": false,
"endian": "little"
},
"unsigned char": {
"kind": "char",
"size": 1,
@@ -137,6 +143,43 @@
},
"kind": "struct",
"size": 64
},
"_CM_CALLBACK_ENTRY": {
"fields": {
"Link": {
"type": {
"kind": "struct",
"name": "nt_symbols!_LIST_ENTRY"
},
"offset": 0
},
"Cookie": {
"type": {
"kind": "base",
"name": "unsigned long long"
},
"offset": 24
},
"Function": {
"type": {
"kind": "pointer",
"subtype": {
"kind": "base",
"name": "void"
}
},
"offset": 40
},
"Altitude": {
"type": {
"kind": "struct",
"name": "nt_symbols!_UNICODE_STRING"
},
"offset": 48
}
},
"kind": "struct",
"size": 64
}
},
"metadata": {
volatility3/framework/symbols/windows/callbacks-x86.json
+43
View File
@@ -8,6 +8,12 @@
"signed": false,
"endian": "little"
},
"unsigned long long": {
"kind": "int",
"size": 8,
"signed": false,
"endian": "little"
},
"unsigned char": {
"kind": "char",
"size": 1,
@@ -137,6 +143,43 @@
},
"kind": "struct",
"size": 28
},
"_CM_CALLBACK_ENTRY": {
"fields": {
"Link": {
"type": {
"kind": "struct",
"name": "nt_symbols!_LIST_ENTRY"
},
"offset": 0
},
"Cookie": {
"type": {
"kind": "base",
"name": "unsigned long long"
},
"offset": 16
},
"Function": {
"type": {
"kind": "pointer",
"subtype": {
"kind": "base",
"name": "void"
}
},
"offset": 28
},
"Altitude": {
"type": {
"kind": "struct",
"name": "nt_symbols!_UNICODE_STRING"
},
"offset": 32
}
},
"kind": "struct",
"size": 40
}
},
"metadata": {
volatility3/framework/symbols/windows/extensions/__init__.py
+32 -26
View File
@@ -3,6 +3,7 @@
#
import collections.abc
import contextlib
import datetime
import functools
import logging
@@ -196,8 +197,8 @@ class MMVAD_SHORT(objects.StructType):
raise AttributeError("Unable to find the parent member")
def get_start(self):
"""Get the VAD's starting virtual address."""
def get_start(self) -> int:
"""Get the VAD's starting virtual address. This is the first accessible byte in the range."""
if self.has_member("StartingVpn"):
@@ -215,8 +216,8 @@ class MMVAD_SHORT(objects.StructType):
raise AttributeError("Unable to find the starting VPN member")
def get_end(self):
"""Get the VAD's ending virtual address."""
def get_end(self) -> int:
"""Get the VAD's ending virtual address. This is the last accessible byte in the range."""
if self.has_member("EndingVpn"):
@@ -233,6 +234,10 @@ class MMVAD_SHORT(objects.StructType):
raise AttributeError("Unable to find the ending VPN member")
def get_size(self) -> int:
"""Get the size of the VAD region. The OS ensures page granularity."""
return (self.get_end() - self.get_start()) + 1
def get_commit_charge(self):
"""Get the VAD's commit charge (number of committed pages)"""
@@ -305,7 +310,7 @@ class MMVAD(MMVAD_SHORT):
file_name = renderers.NotApplicableValue()
try:
with contextlib.suppress(exceptions.InvalidAddressException):
# this is for xp and 2003
if self.has_member("ControlArea"):
filename_obj = self.ControlArea.FilePointer.FileName
@@ -318,9 +323,6 @@ class MMVAD(MMVAD_SHORT):
if filename_obj.Length > 0:
file_name = filename_obj.get_string()
except exceptions.InvalidAddressException:
pass
return file_name
@@ -364,6 +366,7 @@ class DEVICE_OBJECT(objects.StructType, pool.ExecutiveObject):
yield device
device = device.AttachedDevice.dereference()
class DRIVER_OBJECT(objects.StructType, pool.ExecutiveObject):
"""A class for kernel driver objects."""
@@ -374,7 +377,7 @@ class DRIVER_OBJECT(objects.StructType, pool.ExecutiveObject):
def get_devices(self) -> Generator[ObjectInterface, None, None]:
"""Enumerate the driver's device objects"""
device = self.DeviceObject.dereference()
device = self.DeviceObject.dereference()
while device:
yield device
device = device.NextDevice.dereference()
@@ -413,15 +416,11 @@ class FILE_OBJECT(objects.StructType, pool.ExecutiveObject):
# this pointer needs to be checked against native_layer_name because the object may
# be instantiated from a primary (virtual) layer or a memory (physical) layer.
if self._context.layers[self.vol.native_layer_name].is_valid(self.DeviceObject):
try:
with contextlib.suppress(ValueError):
name = f"\\Device\\{self.DeviceObject.get_device_name()}"
except ValueError:
pass
try:
with contextlib.suppress(TypeError, exceptions.InvalidAddressException):
name += self.FileName.String
except (TypeError, exceptions.InvalidAddressException):
pass
return name
@@ -448,9 +447,17 @@ class KMUTANT(objects.StructType, pool.ExecutiveObject):
class ETHREAD(objects.StructType):
"""A class for executive thread objects."""
def owning_process(self, kernel_layer: str = None) -> interfaces.objects.ObjectInterface:
def owning_process(self) -> interfaces.objects.ObjectInterface:
"""Return the EPROCESS that owns this thread."""
return self.ThreadsProcess.dereference(kernel_layer)
# For Windows XPs
if(self.has_member("ThreadsProcess")):
return self.ThreadsProcess.dereference().cast("_EPROCESS")
# For Windows Vista and later versions
elif(self.has_member("Tcb") and self.Tcb.has_member("Process")):
return self.Tcb.Process.dereference().cast("_EPROCESS")
else:
raise AttributeError("Unable to find the owning process of ethread")
def get_cross_thread_flags(self) -> str:
dictCrossThreadFlags = {
@@ -485,7 +492,7 @@ class UNICODE_STRING(objects.StructType):
# We manually construct an object rather than casting a dereferenced pointer in case
# the buffer length is 0 and the pointer is a NULL pointer
return self._context.object(self.vol.type_name.split(constants.BANG)[0] + constants.BANG + 'string',
layer_name = self.Buffer.vol.layer_name,
layer_name = self.Buffer.vol.native_layer_name,
offset = self.Buffer,
max_length = self.Length, errors = 'replace', encoding = 'utf16')
@@ -719,7 +726,7 @@ class EPROCESS(generic.GenericIntelProcess, pool.ExecutiveObject):
env = envar[:split_index]
var = envar[split_index + 1:]
# Exlude parse problem with some types of env
# Exclude parse problem with some types of env
if env and var:
yield env, var
@@ -746,7 +753,10 @@ class LIST_ENTRY(objects.StructType, collections.abc.Iterable):
trans_layer = self._context.layers[layer]
try:
trans_layer.is_valid(self.vol.offset)
is_valid = trans_layer.is_valid(self.vol.offset)
if not is_valid:
return
link = getattr(self, direction).dereference()
except exceptions.InvalidAddressException:
return
@@ -761,9 +771,7 @@ class LIST_ENTRY(objects.StructType, collections.abc.Iterable):
while link.vol.offset not in seen:
obj_offset = link.vol.offset - relative_offset
try:
trans_layer.is_valid(obj_offset)
except exceptions.InvalidAddressException:
if not trans_layer.is_valid(obj_offset):
return
obj = self._context.object(symbol_type,
@@ -1113,12 +1121,10 @@ class SHARED_CACHE_MAP(objects.StructType):
iterval = 0
while (iterval < full_blocks) and (full_blocks <= 4):
vacb_obj = self.InitialVacbs[iterval]
try:
with contextlib.suppress(exceptions.InvalidAddressException):
# Make sure that the SharedCacheMap member of the VACB points back to the parent object.
if vacb_obj.SharedCacheMap == self.vol.offset:
self.save_vacb(vacb_obj, vacb_list)
except exceptions.InvalidAddressException:
pass
iterval += 1
# We also have to account for the spill over data that is not found in the full blocks.
volatility3/framework/symbols/windows/extensions/pool.py
+12 -16
View File
@@ -1,12 +1,14 @@
import contextlib
import functools
import logging
import struct
from typing import Optional, Tuple, List, Dict, Union
from typing import Dict, List, Optional, Tuple, Union
from volatility3.framework import objects, interfaces, constants, symbols, exceptions, renderers
from volatility3.framework.renderers import conversion
from volatility3.plugins.windows.poolscanner import PoolConstraint
from volatility3.framework import constants, exceptions, interfaces, objects, renderers, symbols
from volatility3.framework.renderers import conversion
vollog = logging.getLogger(__name__)
@@ -138,7 +140,7 @@ class POOL_HEADER(objects.StructType):
if addr - optional_headers_length >= padding_length > addr:
continue
try:
with contextlib.suppress(TypeError, exceptions.InvalidAddressException):
mem_object = self._context.object(symbol_table_name + constants.BANG + type_name,
layer_name = self.vol.layer_name,
offset = addr + body_offset + start_offset,
@@ -147,15 +149,13 @@ class POOL_HEADER(objects.StructType):
if mem_object.is_valid():
yield mem_object
except (TypeError, exceptions.InvalidAddressException):
pass
# use the bottom up approach for windows 7 and earlier
else:
type_size = self._context.symbol_space.get_type(symbol_table_name + constants.BANG + type_name).size
if constraint.additional_structures:
for additional_structure in constraint.additional_structures:
type_size += self._context.symbol_space.get_type(symbol_table_name + constants.BANG + additional_structure).size
type_size += self._context.symbol_space.get_type(
symbol_table_name + constants.BANG + additional_structure).size
rounded_size = conversion.round(type_size, alignment, up = True)
@@ -164,11 +164,9 @@ class POOL_HEADER(objects.StructType):
offset = self.vol.offset + self.BlockSize * alignment - rounded_size,
native_layer_name = native_layer_name)
try:
with contextlib.suppress(TypeError, exceptions.InvalidAddressException):
if mem_object.is_valid():
yield mem_object
except (TypeError, exceptions.InvalidAddressException):
pass
@classmethod
@functools.lru_cache()
@@ -177,20 +175,18 @@ class POOL_HEADER(objects.StructType):
headers = []
sizes = []
for header in [
'CREATOR_INFO', 'NAME_INFO', 'HANDLE_INFO', 'QUOTA_INFO', 'PROCESS_INFO', 'AUDIT_INFO', 'EXTENDED_INFO',
'HANDLE_REVOCATION_INFO', 'PADDING_INFO'
'CREATOR_INFO', 'NAME_INFO', 'HANDLE_INFO', 'QUOTA_INFO', 'PROCESS_INFO', 'AUDIT_INFO', 'EXTENDED_INFO',
'HANDLE_REVOCATION_INFO', 'PADDING_INFO'
]:
try:
with contextlib.suppress(AttributeError, exceptions.SymbolError):
type_name = f"{symbol_table_name}{constants.BANG}_OBJECT_HEADER_{header}"
header_type = context.symbol_space.get_type(type_name)
headers.append(header)
sizes.append(header_type.size)
except (AttributeError, exceptions.SymbolError):
# Some of these may not exist, for example:
# if build < 9200: PADDING_INFO else: AUDIT_INFO
# if build == 10586: HANDLE_REVOCATION_INFO else EXTENDED_INFO
# based on what's present and what's not, this list should be the right order and the right length
pass
return headers, sizes
def is_free_pool(self):
volatility3/framework/symbols/windows/extensions/registry.py
+2 -4
View File
@@ -1,7 +1,7 @@
# This file is Copyright 2019 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
import contextlib
import enum
import logging
import struct
@@ -75,12 +75,10 @@ class CMHIVE(objects.StructType):
"""
for attr in ["FileFullPath", "FileUserName", "HiveRootPath"]:
try:
with contextlib.suppress(AttributeError, exceptions.InvalidAddressException):
name = getattr(self, attr)
if name.Length > 0:
return name.get_string()
except (AttributeError, exceptions.InvalidAddressException):
pass
return None
volatility3/framework/symbols/windows/pdbconv.py
+1 -1
View File
@@ -521,7 +521,7 @@ class PdbReader:
self.metadata['windows']['pdb'] = {
"GUID": self.convert_bytes_to_guid(pdb_info.GUID),
"age": pdb_info.age,
"age": self._dbiheader.age,
"database": self._database_name or 'unknown.pdb',
"machine_type": self._dbiheader.machine
}
volatility3/framework/symbols/windows/pdbutil.py
+72 -16
View File
@@ -10,10 +10,12 @@ import os
import re
import struct
from typing import Any, Dict, Generator, List, Optional, Tuple, Union
from urllib import request, parse
from urllib import parse, request
from volatility3 import symbols
from volatility3.framework import constants, interfaces, exceptions
from volatility3.framework import constants, contexts, exceptions, interfaces
from volatility3.framework.automagic import symbol_cache
from volatility3.framework.configuration import requirements
from volatility3.framework.configuration.requirements import SymbolTableRequirement
from volatility3.framework.symbols import intermed
from volatility3.framework.symbols.windows import pdbconv
@@ -24,7 +26,7 @@ vollog = logging.getLogger(__name__)
class PDBUtility(interfaces.configuration.VersionableInterface):
"""Class to handle and manage all getting symbols based on MZ header"""
_version = (1, 0, 0)
_version = (1, 0, 1)
_required_framework_version = (2, 0, 0)
@classmethod
@@ -74,9 +76,16 @@ class PDBUtility(interfaces.configuration.VersionableInterface):
isf_path = None
# Take the first result of search for the intermediate file
for value in intermed.IntermediateSymbolTable.file_symbol_url("windows", filter_string):
if not requirements.VersionRequirement.matches_required((1, 0, 0), symbol_cache.SqliteCache.version):
vollog.debug(f"Required version of SQLiteCache not found")
return None
identifiers_path = os.path.join(constants.CACHE_PATH, constants.IDENTIFIERS_FILENAME)
value = symbol_cache.SqliteCache(identifiers_path).find_location(
symbol_cache.WindowsIdentifier.generate(pdb_name.strip('\x00'), guid.upper(), age), 'windows')
if value:
isf_path = value
break
else:
# If none are found, attempt to download the pdb, convert it and try again
cls.download_pdb_isf(context, guid.upper(), age, pdb_name, progress_callback)
@@ -131,14 +140,14 @@ class PDBUtility(interfaces.configuration.VersionableInterface):
# Check it is actually the MZ header
if mz_sig != b"MZ":
return None
nt_header_start, = struct.unpack("<I", layer.read(offset + 0x3C, 4))
pe_sig = layer.read(offset + nt_header_start, 2)
# Check it is actually the Nt Headers
if pe_sig != b"PE":
return None
optional_header_size, = struct.unpack('<H', layer.read(offset + nt_header_start + 0x14, 2))
# Just enough to tell us the max size
pe_header = layer.read(offset, nt_header_start + 0x16 + optional_header_size)
@@ -146,7 +155,7 @@ class PDBUtility(interfaces.configuration.VersionableInterface):
max_size = pe_data.OPTIONAL_HEADER.SizeOfImage
# Proper data
virtual_data = layer.read(offset, max_size, pad=True)
virtual_data = layer.read(offset, max_size, pad = True)
pe_data = pefile.PE(data = virtual_data)
# De-virtualize the memory
@@ -291,7 +300,7 @@ class PDBUtility(interfaces.configuration.VersionableInterface):
@classmethod
def symbol_table_from_pdb(cls, context: interfaces.context.ContextInterface, config_path: str, layer_name: str,
pdb_name: str, module_offset: int, module_size: int) -> str:
pdb_name: str, module_offset: int = None, module_size: int = None) -> str:
"""Creates symbol table for a module in the specified layer_name.
Searches the memory section of the loaded module for its PDB GUID
@@ -307,6 +316,19 @@ class PDBUtility(interfaces.configuration.VersionableInterface):
Returns:
The name of the constructed and loaded symbol table
"""
_, symbol_table_name = cls._modtable_from_pdb(context, config_path, layer_name, pdb_name, module_offset,
module_size)
return symbol_table_name
@classmethod
def _modtable_from_pdb(cls, context: interfaces.context.ContextInterface, config_path: str, layer_name: str,
pdb_name: str, module_offset: int = None, module_size: int = None,
create_module: bool = False) -> Tuple[Optional[str], Optional[str]]:
if module_offset is None:
module_offset = context.layers[layer_name].minimum_address
if module_size is None:
module_size = context.layers[layer_name].maximum_address - module_offset
guids = list(
cls.pdbname_scan(context,
@@ -323,12 +345,46 @@ class PDBUtility(interfaces.configuration.VersionableInterface):
vollog.debug(f"Found {guid['pdb_name']}: {guid['GUID']}-{guid['age']}")
return cls.load_windows_symbol_table(context,
guid["GUID"],
guid["age"],
guid["pdb_name"],
"volatility3.framework.symbols.intermed.IntermediateSymbolTable",
config_path = config_path)
module_name = guid["pdb_name"].strip('.pdb')
symbol_table_name = cls.load_windows_symbol_table(context,
guid["GUID"],
guid["age"],
guid["pdb_name"],
"volatility3.framework.symbols.intermed.IntermediateSymbolTable",
config_path = config_path)
new_module_name = None
if create_module:
new_module = contexts.Module.create(context, module_name, layer_name, offset = guid['mz_offset'],
symbol_table_name = symbol_table_name)
new_module_name = new_module.name
return new_module_name, symbol_table_name
@classmethod
def module_from_pdb(cls, context: interfaces.context.ContextInterface, config_path: str, layer_name: str,
pdb_name: str, module_offset: int = None, module_size: int = None) -> str:
"""Creates a module in the specified layer_name based on a pdb name.
Searches the memory section of the loaded module for its PDB GUID
and loads the associated symbol table into the symbol space.
Args:
context: The context to retrieve required elements (layers, symbol tables) from
config_path: The config path where to find symbol files
layer_name: The name of the layer on which to operate
module_offset: This memory dump's module image offset
module_size: The size of the module for this dump
Returns:
The name of the constructed and loaded symbol table
"""
module_name, _ = cls._modtable_from_pdb(context, config_path, layer_name, pdb_name, module_offset,
module_size, create_module = True)
return module_name
class PdbSignatureScanner(interfaces.layers.ScannerInterface):
volatility3/plugins/windows/registry/certificates.py
+39 -19
View File
@@ -1,11 +1,14 @@
import contextlib
import logging
import struct
from typing import List, Iterator, Tuple
from typing import List, Iterator, Optional, Tuple, Type
from volatility3.framework import interfaces, renderers
from volatility3.framework import constants, exceptions, interfaces, renderers
from volatility3.framework.configuration import requirements
from volatility3.framework.symbols.windows.extensions.registry import RegValueTypes
from volatility3.plugins.windows.registry import hivelist, printkey
vollog = logging.getLogger(__name__)
class Certificates(interfaces.plugins.PluginInterface):
"""Lists the certificates in the registry's Certificate Store."""
@@ -15,12 +18,14 @@ class Certificates(interfaces.plugins.PluginInterface):
@classmethod
def get_requirements(cls) -> List[interfaces.configuration.RequirementInterface]:
return [
requirements.TranslationLayerRequirement(name = 'primary',
description = 'Memory layer for the kernel',
requirements.ModuleRequirement(name = 'kernel', description = 'Windows kernel',
architectures = ["Intel32", "Intel64"]),
requirements.SymbolTableRequirement(name = "nt_symbols", description = "Windows kernel symbols"),
requirements.PluginRequirement(name = 'hivelist', plugin = hivelist.HiveList, version = (1, 0, 0)),
requirements.PluginRequirement(name = 'printkey', plugin = printkey.PrintKey, version = (1, 0, 0))
requirements.PluginRequirement(name = 'printkey', plugin = printkey.PrintKey, version = (1, 0, 0)),
requirements.BooleanRequirement(name = 'dump',
description = "Extract listed certificates",
default = False,
optional = True)
]
def parse_data(self, data: bytes) -> Tuple[str, bytes]:
@@ -34,36 +39,51 @@ class Certificates(interfaces.plugins.PluginInterface):
elif ctype == 0x100000020:
certificate_data = cvalue
return (name, certificate_data)
@classmethod
def dump_certificate(cls, certificate_data: bytes, hive_offset: int,
reg_section: str, key_hash: str,
open_method: Type[interfaces.plugins.FileHandlerInterface]) -> \
Optional[interfaces.plugins.FileHandlerInterface]:
try:
if not isinstance(certificate_data, interfaces.renderers.BaseAbsentValue):
dump_name = "{}-{}-{}.crt".format(hive_offset, reg_section, key_hash)
file_handle = open_method(dump_name)
file_handle.write(certificate_data)
return file_handle
except exceptions.InvalidAddressException:
vollog.debug(f"Unable to certificate file at {hive_offset:#x}")
return None
def _generator(self) -> Iterator[Tuple[int, Tuple[str, str, str, str]]]:
kernel = self.context.modules[self.config['kernel']]
for hive in hivelist.HiveList.list_hives(self.context,
base_config_path = self.config_path,
layer_name = self.config['primary'],
symbol_table = self.config['nt_symbols']):
layer_name = kernel.layer_name,
symbol_table = kernel.symbol_table_name):
for top_key in [
"Microsoft\\SystemCertificates",
"Software\\Microsoft\\SystemCertificates",
]:
try:
with contextlib.suppress(KeyError, exceptions.InvalidAddressException):
# Walk it
node_path = hive.get_key(top_key, return_list = True)
for (depth, is_key, last_write_time, key_path, volatility,
node) in printkey.PrintKey.key_iterator(hive, node_path, recurse = True):
for (_depth, is_key, _last_write_time, key_path, _volatility, node) in printkey.PrintKey.key_iterator(hive, node_path, recurse = True):
if not is_key and RegValueTypes(node.Type).name == "REG_BINARY":
name, certificate_data = self.parse_data(node.decode_data())
unique_key_offset = key_path.index(top_key) + len(top_key) + 1
unique_key_offset = key_path.casefold().index(top_key.casefold()) + len(top_key) + 1
reg_section = key_path[unique_key_offset:key_path.index("\\", unique_key_offset)]
key_hash = key_path[key_path.rindex("\\") + 1:]
if not isinstance(certificate_data, interfaces.renderers.BaseAbsentValue):
with self.open("{} - {} - {}.crt".format(hex(hive.hive_offset), reg_section,
key_hash)) as file_data:
file_data.write(certificate_data)
if self.config['dump']:
file_handle = self.dump_certificate(certificate_data, hive.hive_offset, reg_section, key_hash, self.open)
if file_handle:
file_handle.close()
yield (0, (top_key, reg_section, key_hash, name))
except KeyError:
# Key wasn't found in this hive, carry on
pass
def run(self) -> renderers.TreeGrid:
return renderers.TreeGrid([("Certificate path", str), ("Certificate section", str), ("Certificate ID", str),
volatility3/schemas/__init__.py
+14 -2
View File
@@ -6,7 +6,7 @@ import hashlib
import json
import logging
import os
from typing import Set, Any, Dict
from typing import Any, Dict, Optional, Set
from volatility3.framework import constants
@@ -51,9 +51,21 @@ def validate(input: Dict[str, Any], use_cache: bool = True) -> bool:
return valid(input, schema, use_cache)
def create_json_hash(input: Dict[str, Any], schema: Dict[str, Any]) -> str:
def create_json_hash(input: Dict[str, Any], schema: Optional[Dict[str, Any]] = None) -> Optional[str]:
"""Constructs the hash of the input and schema to create a unique
identifier for a particular JSON file."""
if schema is None:
format = input.get('metadata', {}).get('format', None)
if not format:
vollog.debug("No schema format defined")
return None
basepath = os.path.abspath(os.path.dirname(__file__))
schema_path = os.path.join(basepath, 'schema-' + format + '.json')
if not os.path.exists(schema_path):
vollog.debug(f"Schema for format not found: {schema_path}")
return None
with open(schema_path, 'r') as s:
schema = json.load(s)
return hashlib.sha1(bytes(json.dumps((input, schema), sort_keys = True), 'utf-8')).hexdigest()