Merge remote-tracking branch 'upstream/develop' into linux_sockstats_plugin

API_CHANGES.md

@@ -4,6 +4,25 @@ API Changes
 When an addition to the existing API is made, the minor version is bumped.
 When an API feature or function is removed or changed, the major version is bumped.
 
+2.1.0
+=====
+Add in the linux `task.get_threads` method added to the API.
+
+2.0.3
+=====
+`DEVICE_OBJECT.get_attached_devices` and `DRIVER_OBJECT.get_devices` added to the API.
+
+2.0.2
+=====
+Fix the behaviour of the offsets returned by the PDB scanner.
+
+2.0.0
+=====
+Remove the `symbol_shift` mechanism, where symbol tables could alter their own symbols.
+Symbols from a symbol table are now always the offset values.  They can be added to a Module
+and when symbols are requested from a Module they are shifted by the module's offset to get
+an absolute offset.  This can be done with `Module.get_absolute_symbol_address` or as part of
+`Module.object_from_symbol(absolute = False, ...)`.
 
 1.2.0
 =====

doc/source/complex-plugin.rst

@@ -300,7 +300,7 @@ This will mean that when a specific structure is loaded from the symbol_space, i
 `StructType`, but instead is instantiated using the NewStructureClass, meaning new methods can be called directly on it.
 
 If the situation really calls for an entirely new object, that isn't covered by one of the existing
-:py:class:`~volatility3.framework.objects.PrimativeObject` objects (such as
+:py:class:`~volatility3.framework.objects.PrimitiveObject` objects (such as
 :py:class:`~volatility3.framework.objects.Integer`,
 :py:class:`~volatility3.framework.objects.Boolean`,
 :py:class:`~volatility3.framework.objects.Float`,

doc/source/glossary.rst

@@ -145,7 +145,7 @@ Struct, Structure
 
 Symbol
     This is used in many different contexts, as a short term for many things.  Within Volatility, a symbol is a
-    construct that usually encompasses a specific type :ref:`type<Type>` at a specfific :ref:`offset<Offset>`,
+    construct that usually encompasses a specific type :ref:`type<Type>` at a specific :ref:`offset<Offset>`,
     representing a particular instance of that type within the memory of a compiled and running program.  An example
     would be the location in memory of a list of active tcp endpoints maintained by the networking stack
     within an operating system.

doc/source/simple-plugin.rst

@@ -196,7 +196,7 @@ The plugin then defaults the ``BaseDllName`` and ``FullDllName`` variables to an
 which is a way of indicating to the user interface that the value couldn't be read for some reason (but that it isn't fatal).
 There are currently four different reasons a value may be unreadable:
 
-* **Unreadble**: values which are empty because the data cannot be read
+* **Unreadable**: values which are empty because the data cannot be read
 * **Unparsable**: values which are empty because the data cannot be interpreted correctly
 * **NotApplicable**: values which are empty because they don't make sense for this particular entry
 * **NotAvailable**: values which cannot be provided now (but might in a future run, via new symbols or an updated plugin)
@@ -206,9 +206,9 @@ information may not be provided.
 
 The plugin then takes the process's ``BaseDllName`` value, and calls :py:meth:`~volatility3.framework.symbols.windows.extensions.UNICODE_STRING.get_string` on it.  All structure attributes,
 as defined by the symbols, are directly accessible and use the case-style of the symbol library it came from (in Windows,
-attributes are CamelCase), such as ``entry.BaseDllName`` in this instance.  Any attribtues not defined by the symbol but added
+attributes are CamelCase), such as ``entry.BaseDllName`` in this instance.  Any attributes not defined by the symbol but added
 by Volatility extensions cannot be properties (in case they overlap with the attributes defined in the symbol libraries)
-and are therefore always methods and prepended with ``get_``, in this example ``BaseDllName.get_string()``.
+and are therefore always methods and pretended with ``get_``, in this example ``BaseDllName.get_string()``.
 
 Finally, ``FullDllName`` is populated.  These operations read from memory, and as such, the memory image may be unable to
 read the data at a particular offset.  This will cause an exception to be thrown.  In Volatility 3, exceptions are thrown

doc/source/volshell.rst

@@ -24,13 +24,15 @@ operating system mode for volshell, and the current layer available for use.
 
 ::
 
-    Volshell (Volatility 3 Framework) 1.0.1
+    Volshell (Volatility 3 Framework) 2.0.2
     Readline imported successfully	PDB scanning finished
 
         Call help() to see available functions
 
-        Volshell mode: Generic
-        Current Layer: primary
+        Volshell mode        : Generic
+        Current Layer        : primary
+        Current Symbol Table : None
+        Current Kernel Name  : None
 
     (primary) >>>
 
@@ -55,9 +57,9 @@ python environment, we can do the following:
 
 ::
 
-    (primary) >>> proc = ps()[0]
-    (primary) >>> proc
-    <EPROCESS nt_symbols1!_EPROCESS: primary @ 0x8c0bcac87040 #2624>
+    (layer_name) >>> proc = ps()[0]
+    (layer_name) >>> proc
+    <EPROCESS symbol_table_name1!_EPROCESS: layer_name @ 0xe08ff2459040 #1968>
 
 When printing a volatility structure, various information is output, in this case the `type_name`, the `layer` and
 `offset` that it's been constructed on, and the size of the structure.
@@ -70,31 +72,31 @@ automagic).
 
 ::
 
-    (primary) >>> dt('_EPROCESS')
-    nt_symbols1!_EPROCESS (2624 bytes)
-       0x0 :   Pcb                                    nt_symbols1!_KPROCESS
-     0x438 :   ProcessLock                            nt_symbols1!_EX_PUSH_LOCK
-     0x440 :   UniqueProcessId                        nt_symbols1!pointer
-     0x448 :   ActiveProcessLinks                     nt_symbols1!_LIST_ENTRY
+    (layer_name) >>> dt('_EPROCESS')
+    symbol_table_name1!_EPROCESS (1968 bytes)
+       0x0 :   Pcb                                         symbol_table_name1!_KPROCESS
+     0x2d8 :   ProcessLock                                 symbol_table_name1!_EX_PUSH_LOCK
+     0x2e0 :   RundownProtect                              symbol_table_name1!_EX_RUNDOWN_REF
+     0x2e8 :   UniqueProcessId                             symbol_table_name1!pointer
     ...
 
 It can also be provided with an object and will interpret the data for each in the process:
 
 ::
 
-    (primary) >>> dt(proc)
-    nt_symbols1!_EPROCESS (2624 bytes)
-       0x0 :   Pcb                                    nt_symbols1!_KPROCESS                               0x8c0bccf8d040
-     0x438 :   ProcessLock                            nt_symbols1!_EX_PUSH_LOCK                           0x8c0bccf8d478
-     0x440 :   UniqueProcessId                        nt_symbols1!pointer                                 356
-     0x448 :   ActiveProcessLinks                     nt_symbols1!_LIST_ENTRY                             0x8c0bccf8d488
+    (layer_name) >>> dt(proc)
+    symbol_table_name1!_EPROCESS (1968 bytes)
+       0x0 :   Pcb                                         symbol_table_name1!_KPROCESS                           0xe08ff2459040
+     0x2d8 :   ProcessLock                                 symbol_table_name1!_EX_PUSH_LOCK                       0xe08ff2459318
+     0x2e0 :   RundownProtect                              symbol_table_name1!_EX_RUNDOWN_REF                     0xe08ff2459320
+     0x2e8 :   UniqueProcessId                             symbol_table_name1!pointer                             4
     ...
 
 These values can be accessed directory as attributes
 
 ::
 
-    (primary) >>> proc.UniqueProcessId
+    (layer_name) >>> proc.UniqueProcessId
     356
 
 Pointer structures contain the value they point to, but attributes accessed are forwarded to the object they point to.
@@ -102,7 +104,7 @@ This means that pointers do not need to be explicitly dereferenced to access und
 
 ::
 
-    (primary) >>> proc.Pcb.DirectoryTableBase
+    (layer_name) >>> proc.Pcb.DirectoryTableBase
     4355817472
 
 Running plugins
@@ -114,26 +116,26 @@ were required:
 
 ::
 
-    (primary) >>> from volatility3.plugins.windows import pslist
-    (primary) >>> display_plugin_output(pslist.PsList)
-    Unable to validate the plugin requirements: ['plugins.Volshell.9QZLXJKFWESI0BAP3M1U7Y5VCT468GRN.PsList.primary', 'plugins.Volshell.9QZLXJKFWESI0BAP3M1U7Y5VCT468GRN.PsList.nt_symbols']
+    (layer_name) >>> from volatility3.plugins.windows import pslist
+    (layer_name) >>> display_plugin_output(pslist.PsList)
+    Unable to validate the plugin requirements: ['plugins.Volshell.VH3FSA1JBG0QP9E62Z8OT5UCIMLNYKW4.PsList.kernel']
 
-We can see that it's made a temporary configuration path for the plugin, and that neither `primary` nor `nt_symbols`
-was fulfilled.
+We can see that it's made a temporary configuration path for the plugin, and that the `kernel` requirement
+was not fulfilled.
 
 We can see all the options that the plugin can accept by access the `get_requirements()` method of the plugin.
 This is a classmethod, so can be called on an uninstantiated copy of the plugin.
 
 ::
 
-    (primary) >>> pslist.PsList.get_requirements()
-    [<TranslationLayerRequirement: primary>, <SymbolTableRequirement: nt_symbols>, <BooleanRequirement: physical>, <ListRequirement: pid>, <BooleanRequirement: dump>]
+    (layer_name) >>> pslist.PsList.get_requirements()
+    [<ModuleRequirement: kernel>, <BooleanRequirement: physical>, <ListRequirement: pid>, <BooleanRequirement: dump>]
 
 We can provide arguments via the `dpo` method call:
 
 ::
 
-    (primary) >>> display_plugin_output(pslist.PsList, primary = self.current_layer, nt_symbols = self.config['nt_symbols'])
+    (layer_name) >>> display_plugin_output(pslist.PsList, kernel = self.config['kernel'])
 
     PID	PPID	ImageFileName	Offset(V)	Threads	Handles	SessionId	Wow64	CreateTime	ExitTime	File output
 
@@ -142,8 +144,9 @@ We can provide arguments via the `dpo` method call:
     356	4	smss.exe	0x8c0bccf8d040	3	-	N/A	False	2021-03-13 17:25:33.000000 	N/A	Disabled
     ...
 
-Here's we've provided the current layer as the TranslationLayerRequirement, and used the symbol tables requirement
-requested by the volshell plugin itself.  A different table could be loaded and provided instead.  The context used
+Here's we've provided the kernel name that was requested by the volshell plugin itself (the generic volshell does not
+load a kernel module, and instead only has a TranslationLayerRequirement).
+A different module could be created and provided instead.  The context used
 by the `dpo` method is always `context`.
 
 Instead of print the results directly to screen, they can be gathered into a TreeGrid objects for direct access by
@@ -151,8 +154,8 @@ using the `generate_treegrid` or `gt` command.
 
 ::
 
-    (primary) >>> treegrid = gt(pslist.PsList, primary = self.current_layer, nt_symbols = self.config['nt_symbols'])
-    (primary) >>> treegrid.populate()
+    (layer_name) >>> treegrid = gt(pslist.PsList, kernel = self.config['kernel'])
+    (layer_name) >>> treegrid.populate()
 
 Treegrids must be populated before the data in them can be accessed.  This is where the plugin actually runs and
 produces data.

volatility3/cli/__init__.py

@@ -19,14 +19,14 @@ import os
 import sys
 import tempfile
 import traceback
-from typing import Dict, Type, Union, Any
+from typing import Any, Dict, Type, Union
 from urllib import parse, request
 
 import volatility3.plugins
 import volatility3.symbols
 from volatility3 import framework
 from volatility3.cli import text_renderer, volargparse
-from volatility3.framework import automagic, constants, contexts, exceptions, interfaces, plugins, configuration
+from volatility3.framework import automagic, configuration, constants, contexts, exceptions, interfaces, plugins
 from volatility3.framework.automagic import stacker
 from volatility3.framework.configuration import requirements
 
@@ -157,6 +157,10 @@ class CommandLine:
                             help = "Write configuration JSON file out to config.json",
                             default = False,
                             action = 'store_true')
+        parser.add_argument("--save-config",
+                            help = "Save configuration JSON file to a file",
+                            default = None,
+                            type = str)
         parser.add_argument("--clear-cache",
                             help = "Clears out all short-term cached items",
                             default = False,
@@ -320,8 +324,13 @@ class CommandLine:
                                                    self.file_handler_class_factory())
 
             if args.write_config:
-                vollog.debug("Writing out configuration data to config.json")
-                with open("config.json", "w") as f:
+                vollog.warning('Use of --write-config has been deprecated, replaced by --save-config <filename>')
+                args.save_config = 'config.json'
+            if args.save_config:
+                vollog.debug("Writing out configuration data to {args.save_config}")
+                if os.path.exists(os.path.abspath(args.save_config)):
+                    parser.error(f"Cannot write configuration: file {args.save_config} already exists")
+                with open(args.save_config, "w") as f:
                     json.dump(dict(constructed.build_configuration()), f, sort_keys = True, indent = 2)
         except exceptions.UnsatisfiedException as excp:
             self.process_unsatisfied_exceptions(excp)
@@ -443,16 +452,17 @@ class CommandLine:
 
             print(f"Unsatisfied requirement {config_path}: {excp.unsatisfied[config_path].description}")
 
-        if symbols_failed:
-            print("\nA symbol table requirement was not fulfilled.  Please verify that:\n"
-                  "\tYou have the correct symbol file for the requirement\n"
-                  "\tThe symbol file is under the correct directory or zip file\n"
-                  "\tThe symbol file is named appropriately or contains the correct banner\n")
         if translation_failed:
             print("\nA translation layer requirement was not fulfilled.  Please verify that:\n"
                   "\tA file was provided to create this layer (by -f, --single-location or by config)\n"
                   "\tThe file exists and is readable\n"
-                  "\tThe necessary symbols are present and identified by volatility3")
+                  "\tThe file is a valid memory image and was acquired cleanly")
+        if symbols_failed:
+            print("\nA symbol table requirement was not fulfilled.  Please verify that:\n"
+                  "\tThe associated translation layer requirement was fulfilled\n"
+                  "\tYou have the correct symbol file for the requirement\n"
+                  "\tThe symbol file is under the correct directory or zip file\n"
+                  "\tThe symbol file is named appropriately or contains the correct banner\n")
 
     def populate_config(self, context: interfaces.context.ContextInterface,
                         configurables_list: Dict[str, Type[interfaces.configuration.ConfigurableInterface]],

volatility3/cli/text_renderer.py

@@ -1,6 +1,7 @@
 # This file is Copyright 2019 Volatility Foundation and licensed under the Volatility Software License 1.0
 # which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
 #
+import csv
 import datetime
 import json
 import logging
@@ -8,7 +9,7 @@ import random
 import string
 import sys
 from functools import wraps
-from typing import Callable, Any, List, Tuple, Dict
+from typing import Any, Callable, Dict, List, Tuple
 
 from volatility3.framework import interfaces, renderers
 from volatility3.framework.renderers import format_hints
@@ -66,7 +67,6 @@ def multitypedata_as_text(value: format_hints.MultiTypeData) -> str:
 
 
 def optional(func: Callable) -> Callable:
-
     @wraps(func)
     def wrapped(x: Any) -> str:
         if isinstance(x, interfaces.renderers.BaseAbsentValue):
@@ -80,7 +80,6 @@ def optional(func: Callable) -> Callable:
 
 
 def quoted_optional(func: Callable) -> Callable:
-
     @wraps(func)
     def wrapped(x: Any) -> str:
         result = optional(func)(x)
@@ -102,7 +101,7 @@ def display_disassembly(disasm: interfaces.renderers.Disassembly) -> str:
         disasm: Input disassembly objects
 
     Returns:
-        A string as rendererd by capstone where available, otherwise output as if it were just bytes
+        A string as rendered by capstone where available, otherwise output as if it were just bytes
     """
 
     if CAPSTONE_PRESENT:
@@ -193,16 +192,17 @@ class NoneRenderer(CLIRenderer):
         if not grid.populated:
             grid.populate(lambda x, y: True, True)
 
+
 class CSVRenderer(CLIRenderer):
     _type_renderers = {
-        format_hints.Bin: quoted_optional(lambda x: f"0b{x:b}"),
-        format_hints.Hex: quoted_optional(lambda x: f"0x{x:x}"),
-        format_hints.HexBytes: quoted_optional(hex_bytes_as_text),
-        format_hints.MultiTypeData: quoted_optional(multitypedata_as_text),
-        interfaces.renderers.Disassembly: quoted_optional(display_disassembly),
-        bytes: quoted_optional(lambda x: " ".join([f"{b:02x}" for b in x])),
-        datetime.datetime: quoted_optional(lambda x: x.strftime("%Y-%m-%d %H:%M:%S.%f %Z")),
-        'default': quoted_optional(lambda x: f"{x}")
+        format_hints.Bin: optional(lambda x: f"0b{x:b}"),
+        format_hints.Hex: optional(lambda x: f"0x{x:x}"),
+        format_hints.HexBytes: optional(hex_bytes_as_text),
+        format_hints.MultiTypeData: optional(multitypedata_as_text),
+        interfaces.renderers.Disassembly: optional(display_disassembly),
+        bytes: optional(lambda x: " ".join([f"{b:02x}" for b in x])),
+        datetime.datetime: optional(lambda x: x.strftime("%Y-%m-%d %H:%M:%S.%f %Z")),
+        'default': optional(lambda x: f"{x}")
     }
 
     name = "csv"
@@ -219,28 +219,28 @@ class CSVRenderer(CLIRenderer):
         """
         outfd = sys.stdout
 
-        line = ['"TreeDepth"']
+        header_list = ['TreeDepth']
         for column in grid.columns:
             # Ignore the type because namedtuples don't realize they have accessible attributes
-            line.append("{}".format('"' + column.name + '"'))
-        outfd.write(f"{','.join(line)}")
+            header_list.append(f"{column.name}")
+
+        writer = csv.DictWriter(outfd, header_list)
+        writer.writeheader()
 
         def visitor(node: interfaces.renderers.TreeNode, accumulator):
-            accumulator.write("\n")
             # Nodes always have a path value, giving them a path_depth of at least 1, we use max just in case
-            accumulator.write(str(max(0, node.path_depth - 1)) + ",")
-            line = []
+            row = {'TreeDepth': str(max(0, node.path_depth - 1))}
             for column_index in range(len(grid.columns)):
                 column = grid.columns[column_index]
                 renderer = self._type_renderers.get(column.type, self._type_renderers['default'])
-                line.append(renderer(node.values[column_index]))
-            accumulator.write(f"{','.join(line)}")
+                row[f'{column.name}'] = renderer(node.values[column_index])
+            accumulator.writerow(row)
             return accumulator
 
         if not grid.populated:
-            grid.populate(visitor, outfd)
+            grid.populate(visitor, writer)
         else:
-            grid.visit(node = None, function = visitor, initial_accumulator = outfd)
+            grid.visit(node = None, function = visitor, initial_accumulator = writer)
 
         outfd.write("\n")
 
@@ -274,7 +274,8 @@ class PrettyTextRenderer(CLIRenderer):
         max_column_widths = dict([(column.name, len(column.name)) for column in grid.columns])
 
         def visitor(
-            node: interfaces.renderers.TreeNode, accumulator: List[Tuple[int, Dict[interfaces.renderers.Column, bytes]]]
+                node: interfaces.renderers.TreeNode,
+                accumulator: List[Tuple[int, Dict[interfaces.renderers.Column, bytes]]]
         ) -> List[Tuple[int, Dict[interfaces.renderers.Column, bytes]]]:
             # Nodes always have a path value, giving them a path_depth of at least 1, we use max just in case
             max_column_widths[tree_indent_column] = max(max_column_widths.get(tree_indent_column, 0), node.path_depth)

volatility3/cli/volshell/__init__.py

@@ -7,12 +7,11 @@ import json
 import logging
 import os
 import sys
-import glob
 
 import volatility3.plugins
 import volatility3.symbols
 from volatility3 import cli, framework
-from volatility3.cli.volshell import generic, windows, linux, mac
+from volatility3.cli.volshell import generic, linux, mac, windows
 from volatility3.framework import automagic, constants, contexts, exceptions, interfaces, plugins
 
 # Make sure we log everything
@@ -86,6 +85,10 @@ class VolShell(cli.CommandLine):
                             help = "Write configuration JSON file out to config.json",
                             default = False,
                             action = 'store_true')
+        parser.add_argument("--save-config",
+                            help = "Save configuration JSON file to a file",
+                            default = None,
+                            type = str)
         parser.add_argument("--clear-cache",
                             help = "Clears out all short-term cached items",
                             default = False,
@@ -235,8 +238,13 @@ class VolShell(cli.CommandLine):
                                                    self.file_handler_class_factory())
 
             if args.write_config:
-                vollog.debug("Writing out configuration data to config.json")
-                with open("config.json", "w") as f:
+                vollog.warning('Use of --write-config has been deprecated, replaced by --save-config <filename>')
+                args.save_config = 'config.json'
+            if args.save_config:
+                vollog.debug("Writing out configuration data to {args.save_config}")
+                if os.path.exists(os.path.abspath(args.save_config)):
+                    parser.error(f"Cannot write configuration: file {args.save_config} already exists")
+                with open(args.save_config, "w") as f:
                     json.dump(dict(constructed.build_configuration()), f, sort_keys = True, indent = 2)
         except exceptions.UnsatisfiedException as excp:
             self.process_unsatisfied_exceptions(excp)

volatility3/cli/volshell/generic.py

@@ -8,11 +8,11 @@ import random
 import string
 import struct
 import sys
-from typing import Any, Dict, List, Optional, Tuple, Union, Type, Iterable
-from urllib import request, parse
+from typing import Any, Dict, Iterable, List, Optional, Tuple, Type, Union
+from urllib import parse, request
 
 from volatility3.cli import text_renderer, volshell
-from volatility3.framework import renderers, interfaces, objects, plugins, exceptions
+from volatility3.framework import exceptions, interfaces, objects, plugins, renderers
 from volatility3.framework.configuration import requirements
 from volatility3.framework.layers import intel, physical, resources
 
@@ -31,6 +31,8 @@ class Volshell(interfaces.plugins.PluginInterface):
     def __init__(self, *args, **kwargs):
         super().__init__(*args, **kwargs)
         self.__current_layer: Optional[str] = None
+        self.__current_symbol_table: Optional[str] = None
+        self.__current_kernel_name: Optional[str] = None
         self.__console = None
 
     def random_string(self, length: int = 32) -> str:
@@ -57,8 +59,6 @@ class Volshell(interfaces.plugins.PluginInterface):
             Return a TreeGrid but this is always empty since the point of this plugin is to run interactively
         """
 
-        self.__current_layer = self.config['primary']
-
         # Try to enable tab completion
         try:
             import readline
@@ -79,9 +79,11 @@ class Volshell(interfaces.plugins.PluginInterface):
         banner = f"""
     Call help() to see available functions
 
-    Volshell mode: {mode}
-    Current Layer: {self.current_layer}
-        """
+    Volshell mode        : {mode}
+    Current Layer        : {self.current_layer}
+    Current Symbol Table : {self.current_symbol_table}
+    Current Kernel Name  : {self.current_kernel_name}
+"""
 
         sys.ps1 = f"({self.current_layer}) >>> "
         self.__console = code.InteractiveConsole(locals = self._construct_locals_dict())
@@ -121,7 +123,10 @@ class Volshell(interfaces.plugins.PluginInterface):
                 (['dw', 'display_words'], self.display_words), (['dd',
                                                                  'display_doublewords'], self.display_doublewords),
                 (['dq', 'display_quadwords'], self.display_quadwords), (['dis', 'disassemble'], self.disassemble),
-                (['cl', 'change_layer'], self.change_layer), (['context'], self.context), (['self'], self),
+                (['cl', 'change_layer'], self.change_layer),
+                (['cs', 'change_symboltable'], self.change_symbol_table),
+                (['ck', 'change_kernel'], self.change_kernel),
+                (['context'], self.context), (['self'], self),
                 (['dpo', 'display_plugin_output'], self.display_plugin_output),
                 (['gt', 'generate_treegrid'], self.generate_treegrid), (['rt',
                                                                          'render_treegrid'], self.render_treegrid),
@@ -174,15 +179,58 @@ class Volshell(interfaces.plugins.PluginInterface):
 
     @property
     def current_layer(self):
+        if self.__current_layer is None:
+            self.__current_layer = self.config['primary']
         return self.__current_layer
 
-    def change_layer(self, layer_name = None):
+    @property
+    def current_symbol_table(self):
+        if self.__current_symbol_table is None and self.kernel:
+            self.__current_symbol_table = self.kernel.symbol_table_name
+        return self.__current_symbol_table
+
+    @property
+    def current_kernel_name(self):
+        if self.__current_kernel_name is None:
+            self.__current_kernel_name = self.config.get('kernel', None)
+        return self.__current_kernel_name
+
+    @property
+    def kernel(self):
+        """Returns the current kernel object"""
+        if self.current_kernel_name not in self.context.modules:
+            return None
+        return self.context.modules[self.current_kernel_name]
+
+    def change_layer(self, layer_name: str = None):
         """Changes the current default layer"""
         if not layer_name:
-            layer_name = self.config['primary']
-        self.__current_layer = layer_name
+            layer_name = self.current_layer
+        if layer_name not in self.context.layers:
+            print(f"Layer {layer_name} not present in context")
+        else:
+            self.__current_layer = layer_name
         sys.ps1 = f"({self.current_layer}) >>> "
 
+    def change_symbol_table(self, symbol_table_name: str = None):
+        """Changes the current_symbol_table"""
+        if not symbol_table_name:
+            print("No symbol table provided, not changing current symbol table")
+        if symbol_table_name not in self.context.symbol_space:
+            print(f"Symbol table {symbol_table_name} not present in context symbol_space")
+        else:
+            self.__current_symbol_table = symbol_table_name
+        print(f"Current Symbol Table: {self.current_symbol_table}")
+
+    def change_kernel(self, kernel_name: str = None):
+        if not kernel_name:
+            print("No kernel module name provided, not changing current kernel")
+        if kernel_name not in self.context.modules:
+            print(f"Kernel module {kernel_name} not found in the context module list")
+        else:
+            self.__current_kernel_name = kernel_name
+        print(f"Current kernel : {self.current_kernel_name}")
+
     def display_bytes(self, offset, count = 128, layer_name = None):
         """Displays byte values and ASCII characters"""
         remaining_data = self._read_data(offset, count = count, layer_name = layer_name)

volatility3/cli/volshell/linux.py

@@ -5,7 +5,7 @@
 from typing import Any, List, Tuple, Union
 
 from volatility3.cli.volshell import generic
-from volatility3.framework import interfaces, constants
+from volatility3.framework import constants, interfaces
 from volatility3.framework.configuration import requirements
 from volatility3.plugins.linux import pslist
 
@@ -15,8 +15,8 @@ class Volshell(generic.Volshell):
 
     @classmethod
     def get_requirements(cls):
-        return (super().get_requirements() + [
-            requirements.SymbolTableRequirement(name = "vmlinux", description = "Linux kernel symbols"),
+        return ([
+            requirements.ModuleRequirement(name = "kernel", description = "Linux kernel module"),
             requirements.PluginRequirement(name = 'pslist', plugin = pslist.PsList, version = (2, 0, 0)),
             requirements.IntRequirement(name = 'pid', description = "Process ID", optional = True)
         ])
@@ -37,14 +37,14 @@ class Volshell(generic.Volshell):
     def list_tasks(self):
         """Returns a list of task objects from the primary layer"""
         # We always use the main kernel memory and associated symbols
-        return list(pslist.PsList.list_tasks(self.context, self.config['primary'], self.config['vmlinux']))
+        return list(pslist.PsList.list_tasks(self.context, self.current_kernel_name))
 
     def construct_locals(self) -> List[Tuple[List[str], Any]]:
         result = super().construct_locals()
         result += [
             (['ct', 'change_task', 'cp'], self.change_task),
             (['lt', 'list_tasks', 'ps'], self.list_tasks),
-            (['symbols'], self.context.symbol_space[self.config['vmlinux']]),
+            (['symbols'], self.context.symbol_space[self.current_symbol_table]),
         ]
         if self.config.get('pid', None) is not None:
             self.change_task(self.config['pid'])
@@ -64,3 +64,9 @@ class Volshell(generic.Volshell):
         if symbol_table is None:
             symbol_table = self.config['vmlinux']
         return super().display_symbols(symbol_table)
+
+    @property
+    def current_layer(self):
+        if self.__current_layer is None:
+            self.__current_layer = self.kernel.layer_name
+        return self.__current_layer

volatility3/cli/volshell/mac.py

@@ -15,9 +15,9 @@ class Volshell(generic.Volshell):
 
     @classmethod
     def get_requirements(cls):
-        return (super().get_requirements() + [
-            requirements.SymbolTableRequirement(name = "darwin", description = "Darwin kernel symbols"),
-            requirements.PluginRequirement(name = 'pslist', plugin = pslist.PsList, version = (1, 0, 0)),
+        return ([
+            requirements.ModuleRequirement(name = "kernel", description = "Darwin kernel module"),
+            requirements.PluginRequirement(name = 'pslist', plugin = pslist.PsList, version = (3, 0, 0)),
             requirements.IntRequirement(name = 'pid', description = "Process ID", optional = True)
         ])
 
@@ -34,17 +34,17 @@ class Volshell(generic.Volshell):
                 return
         print(f"No task with task ID {pid} found")
 
-    def list_tasks(self):
+    def list_tasks(self, method = None):
         """Returns a list of task objects from the primary layer"""
         # We always use the main kernel memory and associated symbols
-        return list(pslist.PsList.list_tasks(self.context, self.config['primary'], self.config['darwin']))
+        return list(pslist.PsList.get_list_tasks(method)(self.context, self.current_kernel_name))
 
     def construct_locals(self) -> List[Tuple[List[str], Any]]:
         result = super().construct_locals()
         result += [
             (['ct', 'change_task', 'cp'], self.change_task),
             (['lt', 'list_tasks', 'ps'], self.list_tasks),
-            (['symbols'], self.context.symbol_space[self.config['darwin']]),
+            (['symbols'], self.context.symbol_space[self.current_symbol_table]),
         ]
         if self.config.get('pid', None) is not None:
             self.change_task(self.config['pid'])
@@ -62,5 +62,11 @@ class Volshell(generic.Volshell):
     def display_symbols(self, symbol_table: str = None):
         """Prints an alphabetical list of symbols for a symbol table"""
         if symbol_table is None:
-            symbol_table = self.config['darwin']
+            symbol_table = self.current_symbol_table
         return super().display_symbols(symbol_table)
+
+    @property
+    def current_layer(self):
+        if self.__current_layer is None:
+            self.__current_layer = self.kernel.layer_name
+        return self.__current_layer

volatility3/cli/volshell/windows.py

@@ -5,7 +5,7 @@
 from typing import Any, List, Tuple, Union
 
 from volatility3.cli.volshell import generic
-from volatility3.framework import interfaces, constants
+from volatility3.framework import constants, interfaces
 from volatility3.framework.configuration import requirements
 from volatility3.plugins.windows import pslist
 
@@ -15,8 +15,8 @@ class Volshell(generic.Volshell):
 
     @classmethod
     def get_requirements(cls):
-        return (super().get_requirements() + [
-            requirements.SymbolTableRequirement(name = "nt_symbols", description = "Windows kernel symbols"),
+        return ([
+            requirements.ModuleRequirement(name = 'kernel', description = 'Windows kernel'),
             requirements.PluginRequirement(name = 'pslist', plugin = pslist.PsList, version = (2, 0, 0)),
             requirements.IntRequirement(name = 'pid', description = "Process ID", optional = True)
         ])
@@ -34,14 +34,14 @@ class Volshell(generic.Volshell):
     def list_processes(self):
         """Returns a list of EPROCESS objects from the primary layer"""
         # We always use the main kernel memory and associated symbols
-        return list(pslist.PsList.list_processes(self.context, self.config['primary'], self.config['nt_symbols']))
+        return list(pslist.PsList.list_processes(self.context, self.current_layer, self.current_symbol_table))
 
     def construct_locals(self) -> List[Tuple[List[str], Any]]:
         result = super().construct_locals()
         result += [
             (['cp', 'change_process'], self.change_process),
             (['lp', 'list_processes', 'ps'], self.list_processes),
-            (['symbols'], self.context.symbol_space[self.config['nt_symbols']]),
+            (['symbols'], self.context.symbol_space[self.current_symbol_table]),
         ]
         if self.config.get('pid', None) is not None:
             self.change_process(self.config['pid'])
@@ -53,11 +53,17 @@ class Volshell(generic.Volshell):
         """Display Type describes the members of a particular object in alphabetical order"""
         if isinstance(object, str):
             if constants.BANG not in object:
-                object = self.config['nt_symbols'] + constants.BANG + object
+                object = self.current_symbol_table + constants.BANG + object
         return super().display_type(object, offset)
 
     def display_symbols(self, symbol_table: str = None):
         """Prints an alphabetical list of symbols for a symbol table"""
         if symbol_table is None:
-            symbol_table = self.config['nt_symbols']
+            symbol_table = self.current_symbol_table
         return super().display_symbols(symbol_table)
+
+    @property
+    def current_layer(self):
+        if self.__current_layer is None:
+            self.__current_layer = self.kernel.layer_name
+        return self.__current_layer

volatility3/framework/automagic/linux.py

@@ -45,6 +45,12 @@ class LinuxIntelStacker(interfaces.automagic.StackerLayerInterface):
 
             symbol_files = linux_banners.get(banner, None)
             if symbol_files:
+                if len(symbol_files) > 1:
+                    using = "*"
+                    vollog.warning(f"Multiple symbol files identified (using {using}):")
+                    for symbol_file in symbol_files:
+                        vollog.warning(f" {using} {symbol_file}")
+                        using = " "
                 isf_path = symbol_files[0]
                 table_name = context.symbol_space.free_table_name('LintelStacker')
                 table = linux.LinuxKernelIntermedSymbols(context,

volatility3/framework/automagic/pdbscan.py

@@ -146,8 +146,13 @@ class KernelPDBScanner(interfaces.automagic.AutomagicInterface):
                 return None
             return (virtual_layer_name, kernel['mz_offset'], kernel)
 
+        vollog.debug("Kernel base determination - optimized scan virtual layer")
+        valid_kernel = self._method_layer_pdb_scan(context, vlayer, test_virtual_kernel, True, False, progress_callback)
+        if valid_kernel != None:
+            return valid_kernel
+
         vollog.debug("Kernel base determination - slow scan virtual layer")
-        return self._method_layer_pdb_scan(context, vlayer, test_virtual_kernel, False, progress_callback)
+        return self._method_layer_pdb_scan(context, vlayer, test_virtual_kernel, False, False, progress_callback)
 
     def method_fixed_mapping(self,
                              context: interfaces.context.ContextInterface,
@@ -175,12 +180,13 @@ class KernelPDBScanner(interfaces.automagic.AutomagicInterface):
                 vollog.debug(f"Potential kernel_virtual_offset caused a page fault: {hex(kvo)}")
 
         vollog.debug("Kernel base determination - testing fixed base address")
-        return self._method_layer_pdb_scan(context, vlayer, test_physical_kernel, True, progress_callback)
+        return self._method_layer_pdb_scan(context, vlayer, test_physical_kernel, False, True, progress_callback)
 
     def _method_layer_pdb_scan(self,
                                context: interfaces.context.ContextInterface,
                                vlayer: layers.intel.Intel,
                                test_kernel: Callable,
+                               optimized: bool = False,
                                physical: bool = True,
                                progress_callback: constants.ProgressCallback = None) -> Optional[ValidKernelType]:
         # TODO: Verify this is a windows image
@@ -192,9 +198,15 @@ class KernelPDBScanner(interfaces.automagic.AutomagicInterface):
         if not physical:
             layer_to_scan = virtual_layer_name
 
+        start_scan_address = 0
+        if optimized and not physical and context.layers[layer_to_scan].metadata.architecture in ["Intel64"]:
+            # TODO: change this value accordingly when 5-Level paging is supported.
+            start_scan_address = (0x1f0 << 39)
+
         kernel_pdb_names = [bytes(name + ".pdb", "utf-8") for name in constants.windows.KERNEL_MODULE_NAMES]
         kernels = PDBUtility.pdbname_scan(ctx = context,
                                           layer_name = layer_to_scan,
+                                          start = start_scan_address,
                                           page_size = vlayer.page_size,
                                           pdb_names = kernel_pdb_names,
                                           progress_callback = progress_callback)

volatility3/framework/automagic/windows.py

@@ -28,9 +28,9 @@ The self-referential indices for older versions of windows are listed below:
 """
 import logging
 import struct
-from typing import Generator, List, Optional, Tuple, Type, Iterable
+from typing import Generator, Iterable, List, Optional, Tuple, Type
 
-from volatility3.framework import interfaces, layers, constants
+from volatility3.framework import constants, interfaces, layers
 from volatility3.framework.configuration import requirements
 from volatility3.framework.layers import intel
 
@@ -116,10 +116,27 @@ class DtbSelfRefPae(DtbSelfReferential):
                          mask = 0x3FFFFFFFFFF000,
                          reserved_bits = 0x0)
 
-    def __call__(self, *args, **kwargs):
-        dtb = super().__call__(*args, **kwargs)
+    @staticmethod
+    def _and_bytes(abytes, bbytes):
+        return bytes([a & b for a, b in zip(abytes[::-1], bbytes[::-1])][::-1])
+
+    def __call__(self, data: bytes, data_offset: int, page_offset: int) -> Optional[Tuple[int, int]]:
+        dtb = super().__call__(data, data_offset, page_offset)
         if dtb:
-            return dtb[0] - 0x4000, dtb[1]
+            # Find the top page
+            top_pae_page = dtb[0] - 0x4000
+            # The top page should map to the next four pages after it
+            # Build what we expect the page table to be
+            expected_table = b''.join([struct.pack(self.ptr_struct, top_pae_page + (i * 0x1000)) for i in range(1, 5)])
+            # Mask off the page bits of top level page map
+            page_table_mask = b"\x00\xf0\xff\xff\xff\xff\xff\xff" * 4
+            page_table = data[top_pae_page - data_offset: top_pae_page - data_offset + (4 * self.ptr_size)]
+            # Compare them
+            anded_bytes = self._and_bytes(page_table, page_table_mask)
+            if (anded_bytes == expected_table):
+                return top_pae_page, dtb[1]
+            # Return None since the dtb value *isn't* None
+            return None
         return dtb
 
 
@@ -202,30 +219,50 @@ class WindowsIntelStacker(interfaces.automagic.StackerLayerInterface):
         for description, tests, sections in cls.test_sets:
             vollog.debug(description)
             # There is a very high chance that the DTB will live in these very narrow segments, assuming we couldn't find them previously
-            hits = context.layers[layer_name].scan(context,
-                                                   PageMapScanner(tests = tests),
-                                                   sections = sections,
-                                                   progress_callback = progress_callback)
+            hits = base_layer.scan(context,
+                                   PageMapScanner(tests = tests),
+                                   sections = sections,
+                                   progress_callback = progress_callback)
 
             # Flatten the generator
             def sort_by_tests(x):
+                """Key used to sort by tests"""
                 return tests.index(x[0]), x[1]
 
+            def get_max_pointer(page_table, test, ptr_size: int):
+                """Determines a pointer from a page_table"""
+                max_ptr = 0
+                for index in range(0, len(page_table), ptr_size):
+                    pointer = struct.unpack(test.ptr_struct, page_table[index:index + ptr_size])[0]
+                    # Make sure the pointer is valid, ignore large pages which would require more calculation
+                    if pointer & 0x1 and not pointer & 0x80:
+                        max_ptr = max(max_ptr, (pointer ^ (pointer & 0xfff)) % test.layer_type.maximum_address)
+                return max_ptr
+
             hits = sorted(list(hits), key = sort_by_tests)
 
-            if hits:
-                # TODO: Decide which to use if there are multiple options
-                test, page_map_offset = hits[0]
-                vollog.debug(f"{test.__class__.__name__} test succeeded at {hex(page_map_offset)}")
-                new_layer_name = context.layers.free_layer_name("IntelLayer")
-                config_path = interfaces.configuration.path_join("IntelHelper", new_layer_name)
-                context.config[interfaces.configuration.path_join(config_path, "memory_layer")] = layer_name
-                context.config[interfaces.configuration.path_join(config_path, "page_map_offset")] = page_map_offset
-                # TODO: Need to determine the layer type (chances are high it's x64, hence this default)
-                layer = test.layer_type(context,
-                                        config_path = config_path,
-                                        name = new_layer_name,
-                                        metadata = {'os': 'Windows'})
+            for test, page_map_offset in hits:
+                # Turn the page tables into integers and find the largest one
+                page_table = base_layer.read(page_map_offset, 0x1000)
+                ptr_size = struct.calcsize(test.ptr_struct)
+                max_pointer = get_max_pointer(page_table, test, ptr_size)
+
+                if max_pointer <= base_layer.maximum_address:
+                    vollog.debug(f"{test.__class__.__name__} test succeeded at {hex(page_map_offset)}")
+                    new_layer_name = context.layers.free_layer_name("IntelLayer")
+                    config_path = interfaces.configuration.path_join("IntelHelper", new_layer_name)
+                    context.config[interfaces.configuration.path_join(config_path, "memory_layer")] = layer_name
+                    context.config[
+                        interfaces.configuration.path_join(config_path, "page_map_offset")] = page_map_offset
+                    layer = test.layer_type(context,
+                                            config_path = config_path,
+                                            name = new_layer_name,
+                                            metadata = {'os': 'Windows'})
+                    break
+                else:
+                    vollog.debug(
+                        f"Max pointer for hit with test {test.__class__.__name__} not met: {hex(max_pointer)} > {hex(base_layer.maximum_address)}")
+            if layer is not None and config_path:
                 break
 
         if layer is not None and config_path:

volatility3/framework/configuration/requirements.py

@@ -10,7 +10,7 @@ expect to be in the context (such as particular layers or symboltables).
 """
 import abc
 import logging
-from typing import Any, ClassVar, List, Optional, Type, Dict, Tuple
+from typing import Any, ClassVar, Dict, List, Optional, Tuple, Type
 
 from volatility3.framework import constants, interfaces
 
@@ -303,7 +303,8 @@ class TranslationLayerRequirement(interfaces.configuration.ConstructableRequirem
         args = {"context": context, "config_path": config_path, "name": name}
 
         if any(
-            [subreq.unsatisfied(context, config_path) for subreq in self.requirements.values() if not subreq.optional]):
+                [subreq.unsatisfied(context, config_path) for subreq in self.requirements.values() if
+                 not subreq.optional]):
             return None
 
         obj = self._construct_class(context, config_path, args)
@@ -358,7 +359,8 @@ class SymbolTableRequirement(interfaces.configuration.ConstructableRequirementIn
         args = {"context": context, "config_path": config_path, "name": name}
 
         if any(
-            [subreq.unsatisfied(context, config_path) for subreq in self.requirements.values() if not subreq.optional]):
+                [subreq.unsatisfied(context, config_path) for subreq in self.requirements.values() if
+                 not subreq.optional]):
             return None
 
         # Fill out the parameter for class creation
@@ -462,6 +464,15 @@ class ModuleRequirement(interfaces.configuration.ConstructableRequirementInterfa
                        "TypeError - Module Requirement only accepts string labels: {}".format(repr(value)))
             return {config_path: self}
 
+        result = {}
+        for subreq in self._requirements:
+            req_unsatisfied = self._requirements[subreq].unsatisfied(context, config_path)
+            if req_unsatisfied:
+                result.update(req_unsatisfied)
+        if not result:
+            result = {config_path: self}
+        return result
+
         ### NOTE: This validate method has side effects (the dependencies can change)!!!
 
         self._validate_class(context, interfaces.configuration.parent_path(config_path))
@@ -482,7 +493,8 @@ class ModuleRequirement(interfaces.configuration.ConstructableRequirementInterfa
         args = {"context": context, "config_path": config_path, "name": name}
 
         if any(
-            [subreq.unsatisfied(context, config_path) for subreq in self.requirements.values() if not subreq.optional]):
+                [subreq.unsatisfied(context, config_path) for subreq in self.requirements.values() if
+                 not subreq.optional]):
             return None
 
         obj = self._construct_class(context, config_path, args)

volatility3/framework/constants/__init__.py

@@ -9,7 +9,7 @@ volatility This includes default scanning block sizes, etc.
 import enum
 import os.path
 import sys
-from typing import Optional, Callable
+from typing import Callable, Optional
 
 import volatility3.framework.constants.linux
 import volatility3.framework.constants.windows
@@ -39,8 +39,8 @@ BANG = "!"
 
 # We use the SemVer 2.0.0 versioning scheme
 VERSION_MAJOR = 2  # Number of releases of the library with a breaking change
-VERSION_MINOR = 0  # Number of changes that only add to the interface
-VERSION_PATCH = 2  # Number of changes that do not change the interface
+VERSION_MINOR = 1  # Number of changes that only add to the interface
+VERSION_PATCH = 0  # Number of changes that do not change the interface
 VERSION_SUFFIX = ""
 
 # TODO: At version 2.0.0, remove the symbol_shift feature
@@ -63,7 +63,7 @@ LOGLEVEL_VVVV = 6
 CACHE_PATH = os.path.join(os.path.expanduser("~"), ".cache", "volatility3")
 """Default path to store cached data"""
 
-if sys.platform == 'windows':
+if sys.platform == 'win32':
     CACHE_PATH = os.path.join(os.environ.get("APPDATA", os.path.expanduser("~")), "volatility3")
 os.makedirs(CACHE_PATH, exist_ok = True)
 
@@ -80,6 +80,7 @@ ProgressCallback = Optional[Callable[[float, str], None]]
 
 OS_CATEGORIES = ['windows', 'mac', 'linux']
 
+
 class Parallelism(enum.IntEnum):
     """An enumeration listing the different types of parallelism applied to
     volatility."""

volatility3/framework/constants/linux/__init__.py

@@ -227,3 +227,6 @@ BLUETOOTH_PROTOCOLS = (
     "HIDP",
     "AVDTP",
 )
+
+# include/linux/sched.h
+PF_KTHREAD = 0x00200000	 # I'm a kernel thread

volatility3/framework/constants/windows/__init__.py

@@ -8,3 +8,5 @@ Windows-specific values that aren't found in debug symbols
 
 KERNEL_MODULE_NAMES = ["ntkrnlmp", "ntkrnlpa", "ntkrpamp", "ntoskrnl"]
 """The list of names that kernel modules can have within the windows OS"""
+
+PE_MAX_EXTRACTION_SIZE = 1024 * 1024 * 256

volatility3/framework/contexts/__init__.py

@@ -141,7 +141,7 @@ class Context(interfaces.context.ContextInterface):
             layer_name: The layer within the context in which the module exists
             offset: The offset at which the module exists in the layer
             native_layer_name: The default native layer for objects constructed by the module
-            size: The size, in bytes, that the module occupys from offset location within the layer named layer_name
+            size: The size, in bytes, that the module occupies from offset location within the layer named layer_name
         """
         if size:
             return SizedModule.create(self,

volatility3/framework/interfaces/configuration.py

@@ -73,7 +73,7 @@ class HierarchicalDict(collections.abc.Mapping):
                  separator: str = CONFIG_SEPARATOR) -> None:
         """
         Args:
-            initial_dict: A dictionary to populate the HierachicalDict with initially
+            initial_dict: A dictionary to populate the HierarchicalDict with initially
             separator: A custom hierarchy separator (defaults to CONFIG_SEPARATOR)
         """
         if not (isinstance(separator, str) and len(separator) == 1):

volatility3/framework/interfaces/context.py

@@ -129,7 +129,7 @@ class ContextInterface(metaclass = ABCMeta):
             layer_name: The layer the module is associated with (which layer the module lives within)
             offset: The initial/base offset of the module (used as the offset for relative symbols)
             native_layer_name: The default native_layer_name to use when the module constructs objects
-            size: The size, in bytes, that the module occupys from offset location within the layer named layer_name
+            size: The size, in bytes, that the module occupies from offset location within the layer named layer_name
 
         Returns:
             A module object

volatility3/framework/interfaces/objects.py

@@ -115,7 +115,7 @@ class ObjectInterface(metaclass = abc.ABCMeta):
         mask = context.layers[object_info.layer_name].address_mask
         normalized_offset = object_info.offset & mask
 
-        self._vol = collections.ChainMap({}, object_info, {'type_name': type_name, 'offset': normalized_offset}, kwargs)
+        self._vol = collections.ChainMap({}, {'type_name': type_name, 'offset': normalized_offset}, object_info, kwargs)
         self._context = context
 
     def __getattr__(self, attr: str) -> Any:

volatility3/framework/interfaces/renderers.py

@@ -1,7 +1,7 @@
 # This file is Copyright 2019 Volatility Foundation and licensed under the Volatility Software License 1.0
 # which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
 #
-"""All plugins output a TreeGrid object which must then be rendered (eithe by a
+"""All plugins output a TreeGrid object which must then be rendered (either by a
 GUI, or as text output, html output or in some other form.
 
 This module defines both the output format (:class:`TreeGrid`) and the

volatility3/framework/layers/resources.py

@@ -10,10 +10,11 @@ import logging
 import lzma
 import os
 import ssl
+import sys
 import urllib.parse
 import urllib.request
 import zipfile
-from typing import Optional, Any, IO, List
+from typing import Any, IO, List, Optional
 from urllib import error
 
 from volatility3 import framework
@@ -100,6 +101,21 @@ class ResourceAccessor(object):
         """
         urllib.request.install_opener(urllib.request.build_opener(*self._handlers))
 
+        # Python bug 46654
+        if sys.platform == 'win32':
+            # We only need to worry about UNC paths on windows, on linux they'd be smb:// and need pysmb or similar
+            parsed_url = urllib.parse.urlparse(url, scheme = 'file')
+            # Only worry about file scheme URLs, make sure that there's either a host or
+            # the unparsing left an extra slash at the start (which will get lost with urlunparse)
+            if parsed_url.scheme == 'file' and (parsed_url.netloc or parsed_url.path.startswith('//')):
+                # Change the netloc to '/' and then prepend the netloc to the path
+                # Urlunparse will remove extra initial slashes from path, hence setting netloc
+                new_url = urllib.parse.urlunparse((parsed_url.scheme, '/',
+                                                   '/' + parsed_url.netloc + parsed_url.path, parsed_url.params,
+                                                   parsed_url.query, parsed_url.fragment))
+                vollog.log(constants.LOGLEVEL_VVVV, f'UNC path detected, converted path {url} to {new_url}')
+                url = new_url
+
         try:
             fp = urllib.request.urlopen(url, context = self._context)
         except error.URLError as excp:

volatility3/framework/layers/scanners/__init__.py

@@ -31,7 +31,7 @@ class BytesScanner(layers.ScannerInterface):
 
 class RegExScanner(layers.ScannerInterface):
     """A scanner that can be provided with a bytes-object regular expression pattern
-    The scanner will scqn all blocks for the regular expression and report the absolute offset of any finds
+    The scanner will scan all blocks for the regular expression and report the absolute offset of any finds
 
     The default flags include DOTALL, since the searches are through binary data and the newline character should
     have no specific significance in such searches"""
@@ -95,7 +95,7 @@ class MultiStringScanner(layers.ScannerInterface):
                 else:
                     suffixes.append(re.escape(bytes([entry])))
             else:
-                # If we've fininshed one of the strings at this point, remember it for later
+                # If we've finished one of the strings at this point, remember it for later
                 finished = True
 
         if len(suffixes) == 1:

volatility3/framework/objects/__init__.py

@@ -136,12 +136,15 @@ class PrimitiveObject(interfaces.objects.ObjectInterface):
             if k not in ["context", "data_format", "object_info", "type_name"]:
                 kwargs[k] = v
         kwargs['new_value'] = self.__new_value
-        return (self._context, self._vol.maps[-2]['type_name'], self._vol.maps[-3], self._data_format), kwargs
+        return (self._context, self._vol.maps[-3]['type_name'], self._vol.maps[-2], self._data_format), kwargs
 
     @classmethod
     def _unmarshall(cls, context: interfaces.context.ContextInterface, data_format: DataFormatInfo,
                     object_info: interfaces.objects.ObjectInformation) -> TUnion[int, float, bool, bytes, str]:
-        data = context.layers.read(object_info.layer_name, object_info.offset, data_format.length)
+        # Don't try to lookup a 0 length data format, incase it's at an invalid offset.  Length 0 means b''
+        data = b''
+        if data_format.length > 0:
+            data = context.layers.read(object_info.layer_name, object_info.offset, data_format.length)
         return convert_data_to_value(data, cls._struct_type, data_format)
 
     class VolTemplateProxy(interfaces.objects.ObjectInterface.VolTemplateProxy):
@@ -203,7 +206,7 @@ class Bytes(PrimitiveObject, bytes):
                 length: int = 1,
                 **kwargs) -> 'Bytes':
         """Creates the appropriate class and returns it so that the native type
-        is inherritted.
+        is inherited.
 
         The only reason the kwargs is added, is so that the
         inheriting types can override __init__ without needing to
@@ -701,7 +704,7 @@ class AggregateType(interfaces.objects.ObjectInterface):
                     tmp_list[member] = (relative_offset, new_child)
                     # If there's trouble with mutability, consider making update_vol return a clone with the changes
                     # (there will be a few other places that will be necessary) and/or making these part of the
-                    # permanent dictionaries rather than the non-clonable ones
+                    # permanent dictionaries rather than the non-cloneable ones
                     template.update_vol(members = tmp_list)
 
         @classmethod

volatility3/framework/plugins/linux/pslist.py

@@ -1,11 +1,12 @@
-# This file is Copyright 2019 Volatility Foundation and licensed under the Volatility Software License 1.0
+# This file is Copyright 2021 Volatility Foundation and licensed under the Volatility Software License 1.0
 # which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
 #
-from typing import Callable, Iterable, List, Any
+from typing import Callable, Iterable, List, Any, Tuple
 
 from volatility3.framework import renderers, interfaces
 from volatility3.framework.configuration import requirements
 from volatility3.framework.objects import utility
+from volatility3.framework.renderers import format_hints
 
 
 class PsList(interfaces.plugins.PluginInterface):
@@ -13,7 +14,7 @@ class PsList(interfaces.plugins.PluginInterface):
 
     _required_framework_version = (2, 0, 0)
 
-    _version = (2, 0, 0)
+    _version = (2, 1, 0)
 
     @classmethod
     def get_requirements(cls) -> List[interfaces.configuration.RequirementInterface]:
@@ -23,7 +24,15 @@ class PsList(interfaces.plugins.PluginInterface):
             requirements.ListRequirement(name = 'pid',
                                          description = 'Filter on specific process IDs',
                                          element_type = int,
-                                         optional = True)
+                                         optional = True),
+            requirements.BooleanRequirement(name="threads",
+                                            description="Include user threads",
+                                            optional=True,
+                                            default=False),
+            requirements.BooleanRequirement(name="decorate_comm",
+                                            description="Show `user threads` comm in curly brackets, and `kernel threads` comm in square brackets",
+                                            optional=True,
+                                            default=False),
         ]
 
     @classmethod
@@ -48,31 +57,76 @@ class PsList(interfaces.plugins.PluginInterface):
         else:
             return lambda _: False
 
-    def _generator(self):
+    def _get_task_fields(
+            self,
+            task: interfaces.objects.ObjectInterface,
+            decorate_comm: bool = False) -> Tuple[int, int, int, str]:
+        """Extract the fields needed for the final output
+
+        Args:
+            task: A task object from where to get the fields.
+            decorate_comm: If True, it decorates the comm string of
+                            - User threads: in curly brackets,
+                            - Kernel threads: in square brackets
+                           Defaults to False.
+        Returns:
+            A tuple with the fields to show in the plugin output.
+        """
+        pid = task.tgid
+        tid = task.pid
+        ppid = task.parent.tgid if task.parent else 0
+        name = utility.array_to_string(task.comm)
+        if decorate_comm:
+            if task.is_kernel_thread:
+                name = f"[{name}]"
+            elif task.is_user_thread:
+                name = f"{{{name}}}"
+
+        task_fields = (format_hints.Hex(task.vol.offset), pid, tid, ppid, name)
+        return task_fields
+
+    def _generator(
+            self,
+            pid_filter: Callable[[Any], bool],
+            include_threads: bool = False,
+            decorate_comm: bool = False):
+        """Generates the tasks list.
+
+        Args:
+            pid_filter: A function which takes a process object and returns True if the process should be ignored/filtered
+            include_threads: If True, the output will also show the user threads
+                             If False, only the thread group leaders will be shown
+                             Defaults to False.
+            decorate_comm: If True, it decorates the comm string of
+                            - User threads: in curly brackets,
+                            - Kernel threads: in square brackets
+                           Defaults to False.
+        Yields:
+            Each rows
+        """
         for task in self.list_tasks(self.context,
                                     self.config['kernel'],
-                                    filter_func = self.create_pid_filter(self.config.get('pid', None))):
-            pid = task.pid
-            ppid = 0
-            if task.parent:
-                ppid = task.parent.pid
-            name = utility.array_to_string(task.comm)
-            yield (0, (pid, ppid, name))
+                                    pid_filter,
+                                    include_threads):
+            row = self._get_task_fields(task, decorate_comm)
+            yield (0, row)
 
     @classmethod
     def list_tasks(
             cls,
             context: interfaces.context.ContextInterface,
             vmlinux_module_name: str,
-            filter_func: Callable[[int], bool] = lambda _: False) -> Iterable[interfaces.objects.ObjectInterface]:
+            filter_func: Callable[[int], bool] = lambda _: False,
+            include_threads: bool = False) -> Iterable[interfaces.objects.ObjectInterface]:
         """Lists all the tasks in the primary layer.
 
         Args:
             context: The context to retrieve required elements (layers, symbol tables) from
             vmlinux_module_name: The name of the kernel module on which to operate
-
+            filter_func: A function which takes a process object and returns True if the process should be ignored/filtered
+            include_threads: If True, it will also return user threads.
         Yields:
-            Process objects
+            Task objects
         """
         vmlinux = context.modules[vmlinux_module_name]
 
@@ -80,8 +134,19 @@ class PsList(interfaces.plugins.PluginInterface):
 
         # Note that the init_task itself is not yielded, since "ps" also never shows it.
         for task in init_task.tasks:
-            if not filter_func(task):
-                yield task
+            if filter_func(task):
+                continue
+
+            yield task
+
+            if include_threads:
+                yield from task.get_threads()
 
     def run(self):
-        return renderers.TreeGrid([("PID", int), ("PPID", int), ("COMM", str)], self._generator())
+        pids = self.config.get('pid')
+        include_threads = self.config.get('threads')
+        decorate_comm = self.config.get('decorate_comm')
+        filter_func = self.create_pid_filter(pids)
+
+        columns = [("OFFSET (V)", format_hints.Hex), ("PID", int), ("TID", int), ("PPID", int), ("COMM", str)]
+        return renderers.TreeGrid(columns, self._generator(filter_func, include_threads, decorate_comm))

volatility3/framework/plugins/linux/pstree.py

@@ -1,8 +1,7 @@
-# This file is Copyright 2019 Volatility Foundation and licensed under the Volatility Software License 1.0
+# This file is Copyright 2021 Volatility Foundation and licensed under the Volatility Software License 1.0
 # which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
 #
 
-from volatility3.framework.objects import utility
 from volatility3.plugins.linux import pslist
 
 
@@ -12,44 +11,74 @@ class PsTree(pslist.PsList):
 
     def __init__(self, *args, **kwargs):
         super().__init__(*args, **kwargs)
-        self._processes = {}
+        self._tasks = {}
         self._levels = {}
         self._children = {}
 
-    def find_level(self, pid):
-        """Finds how deep the pid is in the processes list."""
-        seen = set([])
-        seen.add(pid)
-        level = 0
-        proc = self._processes.get(pid, None)
-        while proc is not None and proc.parent != 0 and proc.parent.pid not in seen:
-            ppid = int(proc.parent.pid)
+    def find_level(self, pid: int) -> None:
+        """Finds how deep the PID is in the tasks hierarchy.
 
-            child_list = self._children.get(ppid, set([]))
+        Args:
+            pid: PID to find the level in the hierachy
+        """
+        seen = set([pid])
+        level = 0
+        proc = self._tasks.get(pid)
+        while proc and proc.parent and proc.parent.pid not in seen:
+            if proc.is_thread_group_leader:
+                parent_pid = proc.parent.pid
+            else:
+                parent_pid = proc.tgid
+
+            child_list = self._children.setdefault(parent_pid, set())
             child_list.add(proc.pid)
-            self._children[ppid] = child_list
-            proc = self._processes.get(ppid, None)
+
+            proc = self._tasks.get(parent_pid)
             level += 1
+
         self._levels[pid] = level
 
-    def _generator(self):
-        """Generates the."""
+    def _generator(
+            self,
+            pid_filter,
+            include_threads: bool = False,
+            decorate_com: bool = False):
+        """Generates the tasks hierarchy tree.
+
+        Args:
+            pid_filter: A function which takes a process object and returns True if the process should be ignored/filtered
+            include_threads: If True, the output will also show the user threads
+                             If False, only the thread group leaders will be shown
+                             Defaults to False.
+            decorate_comm: If True, it decorates the comm string of
+                            - User threads: in curly brackets,
+                            - Kernel threads: in square brackets
+                           Defaults to False.
+        Yields:
+            Each rows
+        """
         vmlinux = self.context.modules[self.config['kernel']]
-        for proc in self.list_tasks(self.context, vmlinux.name):
-            self._processes[proc.pid] = proc
+        for proc in self.list_tasks(self.context,
+                                    vmlinux.name,
+                                    filter_func=pid_filter,
+                                    include_threads=include_threads):
+            self._tasks[proc.pid] = proc
 
         # Build the child/level maps
-        for pid in self._processes:
+        for pid in self._tasks:
             self.find_level(pid)
 
         def yield_processes(pid):
-            proc = self._processes[pid]
-            row = (proc.pid, proc.parent.pid, utility.array_to_string(proc.comm))
+            task = self._tasks[pid]
 
-            yield (self._levels[pid] - 1, row)
-            for child_pid in self._children.get(pid, []):
+            row = self._get_task_fields(task, decorate_com)
+
+            tid = task.pid
+            yield (self._levels[tid] - 1, row)
+
+            for child_pid in sorted(self._children.get(tid, [])):
                 yield from yield_processes(child_pid)
 
-        for pid in self._levels:
-            if self._levels[pid] == 1:
+        for pid, level in self._levels.items():
+            if level == 1:
                 yield from yield_processes(pid)

volatility3/framework/plugins/mac/__init__.py

@@ -0,0 +1,8 @@
+# This file is Copyright 2022 Volatility Foundation and licensed under the Volatility Software License 1.0
+# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
+#
+"""All core mac plugins.
+
+These modules should only be imported from volatility3.plugins NOT
+volatility3.framework.plugins
+"""

volatility3/framework/plugins/mac/ifconfig.py

@@ -9,7 +9,7 @@ from volatility3.framework.symbols import mac
 
 
 class Ifconfig(plugins.PluginInterface):
-    """Lists loaded kernel modules"""
+    """Lists network interface information for all devices"""
 
     _required_framework_version = (2, 0, 0)
 

volatility3/framework/plugins/mac/kauth_scopes.py

@@ -1,4 +1,4 @@
-# This file is opyright 2020 Volatility Foundation and licensed under the Volatility Software License 1.0
+# This file is Copyright 2020 Volatility Foundation and licensed under the Volatility Software License 1.0
 # which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
 #
 import logging

volatility3/framework/plugins/mac/kevents.py

@@ -1,4 +1,4 @@
-# This file is opyright 2020 Volatility Foundation and licensed under the Volatility Software License 1.0
+# This file is Copyright 2020 Volatility Foundation and licensed under the Volatility Software License 1.0
 # which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
 #
 

volatility3/framework/plugins/mac/mount.py

@@ -12,7 +12,7 @@ from volatility3.framework.symbols import mac
 
 class Mount(plugins.PluginInterface):
     """A module containing a collection of plugins that produce data typically
-    foundin Mac's mount command"""
+    found in Mac's mount command"""
 
     _required_framework_version = (2, 0, 0)
 

volatility3/framework/plugins/mac/vfsevents.py

@@ -1,4 +1,4 @@
-# This file is opyright 2020 Volatility Foundation and licensed under the Volatility Software License 1.0
+# This file is Copyright 2020 Volatility Foundation and licensed under the Volatility Software License 1.0
 # which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
 #
 

volatility3/framework/plugins/windows/bigpools.py

@@ -21,7 +21,7 @@ class BigPools(interfaces.plugins.PluginInterface):
     """List big page pools."""
 
     _required_framework_version = (2, 0, 0)
-    _version = (1, 0, 0)
+    _version = (1, 1, 0)
 
     @classmethod
     def get_requirements(cls) -> List[interfaces.configuration.RequirementInterface]:
@@ -32,7 +32,11 @@ class BigPools(interfaces.plugins.PluginInterface):
             requirements.StringRequirement(name = 'tags',
                                            description = "Comma separated list of pool tags to filter pools returned",
                                            optional = True,
-                                           default = None)
+                                           default = None),
+            requirements.BooleanRequirement(name = 'show-free',
+                                            description = 'Show freed regions (otherwise only show allocations in use)',
+                                            default = False,
+                                            optional = True)
         ]
 
     @classmethod
@@ -40,7 +44,8 @@ class BigPools(interfaces.plugins.PluginInterface):
                        context: interfaces.context.ContextInterface,
                        layer_name: str,
                        symbol_table: str,
-                       tags: Optional[list] = None):
+                       tags: Optional[list] = None,
+                       show_free: bool = False):
         """Returns the big page pool objects from the kernel PoolBigPageTable array.
 
         Args:
@@ -97,7 +102,7 @@ class BigPools(interfaces.plugins.PluginInterface):
 
         for big_pool in big_pools:
             if big_pool.is_valid():
-                if tags is None or big_pool.get_key() in tags:
+                if (tags is None or big_pool.get_key() in tags) and (show_free or not big_pool.is_free()):
                     yield big_pool
 
     def _generator(self) -> Iterator[Tuple[int, Tuple[int, str]]]:  # , str, int]]]:
@@ -110,13 +115,19 @@ class BigPools(interfaces.plugins.PluginInterface):
         for big_pool in self.list_big_pools(context = self.context,
                                             layer_name = kernel.layer_name,
                                             symbol_table = kernel.symbol_table_name,
-                                            tags = tags):
+                                            tags = tags,
+                                            show_free = self.config.get("show-free")):
 
             num_bytes = big_pool.get_number_of_bytes()
             if not isinstance(num_bytes, interfaces.renderers.BaseAbsentValue):
                 num_bytes = format_hints.Hex(num_bytes)
 
-            yield (0, (format_hints.Hex(big_pool.Va), big_pool.get_key(), big_pool.get_pool_type(), num_bytes))
+            if big_pool.is_free():
+                status = "Free"
+            else:
+                status = "Allocated"
+
+            yield (0, (format_hints.Hex(big_pool.Va), big_pool.get_key(), big_pool.get_pool_type(), num_bytes, status))
 
     def run(self):
         return renderers.TreeGrid([
@@ -124,4 +135,5 @@ class BigPools(interfaces.plugins.PluginInterface):
             ('Tag', str),
             ('PoolType', str),
             ('NumberOfBytes', format_hints.Hex),
+            ('Status', str),
         ], self._generator())

volatility3/framework/plugins/windows/devicetree.py

@@ -0,0 +1,167 @@
+# This file is Copyright 2022 Volatility Foundation and licensed under the Volatility Software License 1.0
+# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
+#
+
+import logging
+
+from typing import Iterator, List, Tuple
+
+from volatility3.framework import constants, renderers, exceptions, interfaces
+from volatility3.framework.configuration import requirements
+from volatility3.framework.renderers import format_hints
+from volatility3.plugins.windows import driverscan
+
+DEVICE_CODES = {
+    0x00000027 : "FILE_DEVICE_8042_PORT",
+    0x00000032 : "FILE_DEVICE_ACPI",
+    0x00000029 : "FILE_DEVICE_BATTERY",
+    0x00000001 : "FILE_DEVICE_BEEP",
+    0x0000002a : "FILE_DEVICE_BUS_EXTENDER",
+    0x00000002 : "FILE_DEVICE_CD_ROM",
+    0x00000003 : "FILE_DEVICE_CD_ROM_FILE_SYSTEM",
+    0x00000030 : "FILE_DEVICE_CHANGER",
+    0x00000004 : "FILE_DEVICE_CONTROLLER",
+    0x00000005 : "FILE_DEVICE_DATALINK",
+    0x00000006 : "FILE_DEVICE_DFS",
+    0x00000035 : "FILE_DEVICE_DFS_FILE_SYSTEM",
+    0x00000036 : "FILE_DEVICE_DFS_VOLUME",
+    0x00000007 : "FILE_DEVICE_DISK",
+    0x00000008 : "FILE_DEVICE_DISK_FILE_SYSTEM",
+    0x00000033 : "FILE_DEVICE_DVD",
+    0x00000009 : "FILE_DEVICE_FILE_SYSTEM",
+    0x0000003a : "FILE_DEVICE_FIPS",
+    0x00000034 : "FILE_DEVICE_FULLSCREEN_VIDEO",
+    0x0000000a : "FILE_DEVICE_INPORT_PORT",
+    0x0000000b : "FILE_DEVICE_KEYBOARD",
+    0x0000002f : "FILE_DEVICE_KS",
+    0x00000039 : "FILE_DEVICE_KSEC",
+    0x0000000c : "FILE_DEVICE_MAILSLOT",
+    0x0000002d : "FILE_DEVICE_MASS_STORAGE",
+    0x0000000d : "FILE_DEVICE_MIDI_IN",
+    0x0000000e : "FILE_DEVICE_MIDI_OUT",
+    0x0000002b : "FILE_DEVICE_MODEM",
+    0x0000000f : "FILE_DEVICE_MOUSE",
+    0x00000010 : "FILE_DEVICE_MULTI_UNC_PROVIDER",
+    0x00000011 : "FILE_DEVICE_NAMED_PIPE",
+    0x00000012 : "FILE_DEVICE_NETWORK",
+    0x00000013 : "FILE_DEVICE_NETWORK_BROWSER",
+    0x00000014 : "FILE_DEVICE_NETWORK_FILE_SYSTEM",
+    0x00000028 : "FILE_DEVICE_NETWORK_REDIRECTOR",
+    0x00000015 : "FILE_DEVICE_NULL",
+    0x00000016 : "FILE_DEVICE_PARALLEL_PORT",
+    0x00000017 : "FILE_DEVICE_PHYSICAL_NETCARD",
+    0x00000018 : "FILE_DEVICE_PRINTER",
+    0x00000019 : "FILE_DEVICE_SCANNER",
+    0x0000001c : "FILE_DEVICE_SCREEN",
+    0x00000037 : "FILE_DEVICE_SERENUM",
+    0x0000001a : "FILE_DEVICE_SERIAL_MOUSE_PORT",
+    0x0000001b : "FILE_DEVICE_SERIAL_PORT",
+    0x00000031 : "FILE_DEVICE_SMARTCARD",
+    0x0000002e : "FILE_DEVICE_SMB",
+    0x0000001d : "FILE_DEVICE_SOUND",
+    0x0000001e : "FILE_DEVICE_STREAMS",
+    0x0000001f : "FILE_DEVICE_TAPE",
+    0x00000020 : "FILE_DEVICE_TAPE_FILE_SYSTEM",
+    0x00000038 : "FILE_DEVICE_TERMSRV",
+    0x00000021 : "FILE_DEVICE_TRANSPORT",
+    0x00000022 : "FILE_DEVICE_UNKNOWN",
+    0x0000002c : "FILE_DEVICE_VDM",
+    0x00000023 : "FILE_DEVICE_VIDEO",
+    0x00000024 : "FILE_DEVICE_VIRTUAL_DISK",
+    0x00000025 : "FILE_DEVICE_WAVE_IN",
+    0x00000026 : "FILE_DEVICE_WAVE_OUT",
+}
+
+vollog = logging.getLogger(__name__)
+
+class DeviceTree(interfaces.plugins.PluginInterface):
+    """Listing tree based on drivers and attached devices in a particular windows memory image."""
+
+    _required_framework_version = (2, 0, 3)
+    _version = (1, 0, 0)
+
+    @classmethod
+    def get_requirements(cls) -> List[interfaces.configuration.RequirementInterface]:
+        return [
+            requirements.ModuleRequirement(name = "kernel", description = "Windows kernel",
+                                           architectures = ["Intel32", "Intel64"]),
+            requirements.PluginRequirement(name = "driverscan", plugin = driverscan.DriverScan, version = (1, 0, 0)),
+        ]
+
+    def _generator(self) -> Iterator[Tuple]:
+        kernel = self.context.modules[self.config["kernel"]]
+
+        # Scan the Layer for drivers
+        for driver in driverscan.DriverScan.scan_drivers(self.context, kernel.layer_name, kernel.symbol_table_name):
+            try:
+                try:
+                    driver_name = driver.get_driver_name()
+                except (ValueError, exceptions.PagedInvalidAddressException):
+                    vollog.log(constants.LOGLEVEL_VVVV,
+                        f"Failed to get Driver name : {driver.vol.offset:x}")
+                    driver_name = renderers.UnparsableValue()
+
+                yield (0, (
+                    format_hints.Hex(driver.vol.offset),
+                    "DRV",
+                    driver_name,
+                    renderers.NotApplicableValue(),
+                    renderers.NotApplicableValue(),
+                    renderers.NotApplicableValue()
+                ))
+
+                # Scan to get the device information of driver.
+                for device in driver.get_devices():
+                    try:
+                        device_name = device.get_device_name()
+                    except (ValueError, exceptions.PagedInvalidAddressException):
+                        vollog.log(constants.LOGLEVEL_VVVV,
+                            f"Failed to get Device name : {device.vol.offset:x}")
+                        device_name = renderers.UnparsableValue()
+                    
+                    device_type = DEVICE_CODES.get(device.DeviceType, "UNKNOWN")
+
+                    yield (1, (
+                        format_hints.Hex(driver.vol.offset),
+                        "DEV",
+                        driver_name,
+                        device_name,
+                        renderers.NotApplicableValue(),
+                        device_type
+                    ))
+                    
+                    # Scan to get the attached devices information of device.
+                    for level, attached_device in enumerate(device.get_attached_devices(), start=2):
+                        try:
+                            device_name = attached_device.get_device_name()
+                        except (ValueError, exceptions.PagedInvalidAddressException):
+                            vollog.log(constants.LOGLEVEL_VVVV,
+                                f"Failed to get Attached Device Name: {attached_device.vol.offset:x}")
+                            device_name = renderers.UnparsableValue()
+                        
+                        attached_device_driver_name = attached_device.DriverObject.DriverName.get_string()
+                        attached_device_type = DEVICE_CODES.get(attached_device.DeviceType, "UNKNOWN")
+
+                        yield (level, (
+                            format_hints.Hex(driver.vol.offset),
+                            "ATT",
+                            driver_name,
+                            device_name,
+                            attached_device_driver_name,
+                            attached_device_type
+                        ))
+                    
+            except(exceptions.PagedInvalidAddressException):
+                vollog.log(constants.LOGLEVEL_VVVV,
+                    f"Invalid address identified in drivers and devices: {driver.vol.offset:x}")
+                continue
+
+    def run(self) -> renderers.TreeGrid:
+        return renderers.TreeGrid([
+            ("Offset", format_hints.Hex),
+            ("Type", str),
+            ("DriverName", str),
+            ("DeviceName", str),
+            ("DriverNameOfAttDevice", str),
+            ("DeviceType", str),
+        ], self._generator())

volatility3/framework/plugins/windows/hashdump.py

@@ -4,10 +4,10 @@
 import binascii
 import hashlib
 import logging
-from struct import unpack, pack
-from typing import List, Tuple, Optional
+from struct import pack, unpack
+from typing import List, Optional, Tuple
 
-from Crypto.Cipher import ARC4, DES, AES
+from Crypto.Cipher import AES, ARC4, DES
 from Crypto.Hash import MD5
 
 from volatility3.framework import interfaces, renderers
@@ -28,7 +28,7 @@ class Hashdump(interfaces.plugins.PluginInterface):
     def get_requirements(cls):
         return [
             requirements.ModuleRequirement(name = 'kernel', description = 'Windows kernel',
-                                                     architectures = ["Intel32", "Intel64"]),
+                                           architectures = ["Intel32", "Intel64"]),
             requirements.PluginRequirement(name = 'hivelist', plugin = hivelist.HiveList, version = (1, 0, 0))
         ]
 
@@ -63,7 +63,8 @@ class Hashdump(interfaces.plugins.PluginInterface):
     def get_hive_key(cls, hive: registry.RegistryHive, key: str):
         result = None
         try:
-            result = hive.get_key(key)
+            if hive:
+                result = hive.get_key(key)
         except KeyError:
             vollog.info(
                 f"Unable to load the required registry key {hive.get_name()}\\{key} from this memory image")
@@ -132,7 +133,7 @@ class Hashdump(interfaces.plugins.PluginInterface):
             rc4_key = md5.digest()
 
             rc4 = ARC4.new(rc4_key)
-            hbootkey = rc4.encrypt(sam_data[0x80:0xA0]) # lgtm [py/weak-cryptographic-algorithm]
+            hbootkey = rc4.encrypt(sam_data[0x80:0xA0])  # lgtm [py/weak-cryptographic-algorithm]
             return hbootkey
         elif revision == 3:
             # AES encrypted
@@ -151,7 +152,7 @@ class Hashdump(interfaces.plugins.PluginInterface):
         des2 = DES.new(des_k2, DES.MODE_ECB)
         cipher = AES.new(hbootkey[:16], AES.MODE_CBC, salt)
         obfkey = cipher.decrypt(enc_hash)
-        return des1.decrypt(obfkey[:8]) + des2.decrypt(obfkey[8:16]) # lgtm [py/weak-cryptographic-algorithm]
+        return des1.decrypt(obfkey[:8]) + des2.decrypt(obfkey[8:16])  # lgtm [py/weak-cryptographic-algorithm]
 
     @classmethod
     def get_user_hashes(cls, user: registry.CM_KEY_NODE, samhive: registry.RegistryHive,
@@ -229,9 +230,9 @@ class Hashdump(interfaces.plugins.PluginInterface):
         md5.update(hbootkey[:0x10] + pack("<L", rid) + lmntstr)
         rc4_key = md5.digest()
         rc4 = ARC4.new(rc4_key)
-        obfkey = rc4.encrypt(enc_hash) # lgtm [py/weak-cryptographic-algorithm]
+        obfkey = rc4.encrypt(enc_hash)  # lgtm [py/weak-cryptographic-algorithm]
 
-        return des1.decrypt(obfkey[:8]) + des2.decrypt(obfkey[8:]) # lgtm [py/weak-cryptographic-algorithm]
+        return des1.decrypt(obfkey[:8]) + des2.decrypt(obfkey[8:])  # lgtm [py/weak-cryptographic-algorithm]
 
     @classmethod
     def get_user_name(cls, user: registry.CM_KEY_NODE, samhive: registry.RegistryHive) -> Optional[bytes]:
@@ -253,13 +254,9 @@ class Hashdump(interfaces.plugins.PluginInterface):
     # replaces the dump_hashes method in vol2
     def _generator(self, syshive: registry.RegistryHive, samhive: registry.RegistryHive):
         if syshive is None:
-            vollog.debug("SYSTEM address is None: Did you use the correct profile?")
-            yield (0, (renderers.NotAvailableValue(), renderers.NotAvailableValue(), renderers.NotAvailableValue(),
-                       renderers.NotAvailableValue()))
+            vollog.debug("SYSTEM address is None: No system hive found")
         if samhive is None:
-            vollog.debug("SAM address is None: Did you use the correct profile?")
-            yield (0, (renderers.NotAvailableValue(), renderers.NotAvailableValue(), renderers.NotAvailableValue(),
-                       renderers.NotAvailableValue()))
+            vollog.debug("SAM address is None: No SAM hive found")
         bootkey = self.get_bootkey(syshive)
         hbootkey = self.get_hbootkey(samhive, bootkey)
         if hbootkey:

volatility3/framework/plugins/windows/malfind.py

@@ -53,7 +53,7 @@ class Malfind(interfaces.plugins.PluginInterface):
         """
 
         CHUNK_SIZE = 0x1000
-        all_zero_page = "\x00" * CHUNK_SIZE
+        all_zero_page = b"\x00" * CHUNK_SIZE
 
         offset = 0
         vad_length = vad.get_end() - vad.get_start()

volatility3/framework/plugins/windows/mbrscan.py

@@ -0,0 +1,200 @@
+# This file is Copyright 2022 Volatility Foundation and licensed under the Volatility Software License 1.0
+# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
+#
+
+import logging
+import hashlib
+
+from typing import Iterator, List, Tuple
+
+from volatility3.framework import constants, exceptions, interfaces, renderers, symbols
+from volatility3.framework.configuration import requirements
+from volatility3.framework.layers import scanners
+from volatility3.framework.renderers import format_hints
+from volatility3.framework.symbols import intermed
+from volatility3.framework.symbols.windows.extensions import mbr
+
+vollog = logging.getLogger(__name__)
+
+class MBRScan(interfaces.plugins.PluginInterface):
+    """Scans for and parses potential Master Boot Records (MBRs)"""
+
+    _required_framework_version = (2, 0, 1)
+    _version = (1, 0, 0)
+
+    @classmethod
+    def get_requirements(cls)-> List[interfaces.configuration.RequirementInterface]:
+        return [
+            requirements.ModuleRequirement(name = 'kernel', description = 'Windows kernel',
+                                           architectures = ["Intel32", "Intel64"]),
+            requirements.BooleanRequirement(name = 'full',
+                                            description ="It analyzes and provides all the information in the partition entry and bootcode hexdump. (It returns a lot of information, so we recommend you render it in CSV.)",
+                                            default = False,
+                                            optional = True)            
+        ]
+
+    @classmethod
+    def get_hash(cls, data:bytes) -> str:
+        return hashlib.md5(data).hexdigest()
+
+    def _generator(self) -> Iterator[Tuple]:
+        kernel = self.context.modules[self.config['kernel']]
+        physical_layer_name = self.context.layers[kernel.layer_name].config.get('memory_layer', None)
+        
+        # Decide of Memory Dump Architecture
+        layer = self.context.layers[physical_layer_name]
+        architecture = "intel" if not symbols.symbol_table_is_64bit(self.context, kernel.symbol_table_name) else "intel64"
+
+        # Read in the Symbol File
+        symbol_table = intermed.IntermediateSymbolTable.create(context = self.context,
+                                                               config_path = self.config_path,
+                                                               sub_path = "windows",
+                                                               filename = "mbr",
+                                                               class_types = {
+                                                                'PARTITION_TABLE': mbr.PARTITION_TABLE,
+                                                                'PARTITION_ENTRY': mbr.PARTITION_ENTRY
+                                                               })
+
+        partition_table_object = symbol_table + constants.BANG + "PARTITION_TABLE"
+        
+        # Define Signature and Data Length
+        mbr_signature = b"\x55\xAA"
+        mbr_length = 0x200
+        bootcode_length = 0x1B8
+
+        # Scan the Layer for Raw Master Boot Record (MBR) and parse the fields
+        for offset, _value in layer.scan(context = self.context, scanner = scanners.MultiStringScanner(patterns = [mbr_signature])):
+            try:
+                mbr_start_offset = offset - (mbr_length - len(mbr_signature))
+                partition_table = self.context.object(partition_table_object, offset = mbr_start_offset, layer_name = layer.name)
+
+                # Extract only BootCode
+                full_mbr = layer.read(mbr_start_offset, mbr_length, pad = True)
+                bootcode = full_mbr[:bootcode_length]
+                
+                all_zeros = None
+
+                if bootcode:
+                    all_zeros = bootcode.count(b"\x00") == len(bootcode)
+
+                if not all_zeros:
+
+                    partition_entries = [
+                        partition_table.FirstEntry, partition_table.SecondEntry,
+                        partition_table.ThirdEntry, partition_table.FourthEntry
+                    ]
+
+                    if not self.config.get("full", True):
+                        yield (0, (
+                            format_hints.Hex(offset),
+                            partition_table.get_disk_signature(),
+                            self.get_hash(bootcode),
+                            self.get_hash(full_mbr),
+                            renderers.NotApplicableValue(),
+                            renderers.NotApplicableValue(),
+                            renderers.NotApplicableValue(),
+                            renderers.NotApplicableValue(),
+                            interfaces.renderers.Disassembly(bootcode, 0, architecture)
+                        ))
+                    else:
+                        yield (0, (
+                            format_hints.Hex(offset),
+                            partition_table.get_disk_signature(),
+                            self.get_hash(bootcode),
+                            self.get_hash(full_mbr),
+                            renderers.NotApplicableValue(),
+                            renderers.NotApplicableValue(),
+                            renderers.NotApplicableValue(),
+                            renderers.NotApplicableValue(),
+                            renderers.NotApplicableValue(),
+                            renderers.NotApplicableValue(),
+                            renderers.NotApplicableValue(),
+                            renderers.NotApplicableValue(),
+                            renderers.NotApplicableValue(),
+                            renderers.NotApplicableValue(),
+                            renderers.NotApplicableValue(),
+                            renderers.NotApplicableValue(),
+                            renderers.NotApplicableValue(),
+                            interfaces.renderers.Disassembly(bootcode, 0, architecture),
+                            format_hints.HexBytes(bootcode)
+                        ))
+
+                    for partition_index, partition_entry_object in enumerate(partition_entries, start=1):
+                        
+                        if not self.config.get("full", True):
+                            yield (1, (
+                                format_hints.Hex(offset),
+                                partition_table.get_disk_signature(),
+                                self.get_hash(bootcode),
+                                self.get_hash(full_mbr),
+                                partition_index,
+                                partition_entry_object.is_bootable(),
+                                partition_entry_object.get_partition_type(),
+                                format_hints.Hex(partition_entry_object.get_size_in_sectors()),
+                                renderers.NotApplicableValue()
+                            ))
+                        else:
+                            yield (1, (
+                                format_hints.Hex(offset),
+                                partition_table.get_disk_signature(),
+                                self.get_hash(bootcode),
+                                self.get_hash(full_mbr),
+                                partition_index,
+                                partition_entry_object.is_bootable(),
+                                format_hints.Hex(partition_entry_object.get_bootable_flag()),
+                                partition_entry_object.get_partition_type(),
+                                format_hints.Hex(partition_entry_object.PartitionType),
+                                format_hints.Hex(partition_entry_object.get_starting_lba()),
+                                partition_entry_object.get_starting_cylinder(),
+                                partition_entry_object.get_starting_chs(),
+                                partition_entry_object.get_starting_sector(),
+                                partition_entry_object.get_ending_cylinder(),
+                                partition_entry_object.get_ending_chs(),
+                                partition_entry_object.get_ending_sector(),
+                                format_hints.Hex(partition_entry_object.get_size_in_sectors()),
+                                renderers.NotApplicableValue(),
+                                renderers.NotApplicableValue()
+                            ))
+                else:
+                    vollog.log(constants.LOGLEVEL_VVVV, f"Not a valid MBR: Data all zeroed out : {format_hints.Hex(offset)}")
+                    continue
+            
+            except exceptions.PagedInvalidAddressException as excp:
+                vollog.log(constants.LOGLEVEL_VVVV, f"Invalid address identified in guessed MBR: {hex(excp.invalid_address)}")
+                continue
+                
+    def run(self)-> renderers.TreeGrid:
+        if not self.config.get("full", True):
+            return renderers.TreeGrid([
+                ("Potential MBR at Physical Offset", format_hints.Hex),
+                ("Disk Signature", str),
+                ("Bootcode MD5", str),
+                ("Full MBR MD5", str),
+                ("PartitionIndex", int),
+                ("Bootable", bool),
+                ("PartitionType", str),
+                ("SectorInSize", format_hints.Hex),
+                ("Disasm", interfaces.renderers.Disassembly)
+            ], self._generator())
+        else:
+            return renderers.TreeGrid([
+                ("Potential MBR at Physical Offset", format_hints.Hex),
+                ("Disk Signature", str),
+                ("Bootcode MD5", str),
+                ("Full MBR MD5", str),
+                ("PartitionIndex", int),
+                ("Bootable", bool),
+                ("BootFlag", format_hints.Hex),
+                ("PartitionType", str),
+                ("PartitionTypeRaw", format_hints.Hex),
+                ("StartingLBA", format_hints.Hex),
+                ("StartingCylinder", int),
+                ("StartingCHS", int),
+                ("StartingSector", int),
+                ("EndingCylinder", int),
+                ("EndingCHS", int),
+                ("EndingSector", int),
+                ("SectorInSize", format_hints.Hex),            
+                ("Disasm", interfaces.renderers.Disassembly),
+                ("Bootcode", format_hints.HexBytes)
+            ], self._generator())

volatility3/framework/plugins/windows/privileges.py

@@ -55,14 +55,14 @@ class Privs(interfaces.plugins.PluginInterface):
             try:
                 process_token = task.Token.dereference().cast("_TOKEN")
             except exceptions.InvalidAddressException:
-                vollog.log(constants.LOGLEVEL_VVV, 'Skeep invalid token.')
+                vollog.log(constants.LOGLEVEL_VVV, 'Skip invalid token.')
                 continue
 
             for value, present, enabled, default in process_token.privileges():
                 # Skip privileges whose bit positions cannot be
                 # translated to a privilege name
                 if not self.privilege_info.get(int(value)):
-                    vollog.log(constants.LOGLEVEL_VVV, f'Skeep invalid privilege ({value}).')
+                    vollog.log(constants.LOGLEVEL_VVV, f'Skip invalid privilege ({value}).')
                     continue
 
                 name, desc = self.privilege_info.get(int(value))

volatility3/framework/plugins/windows/psscan.py

@@ -85,7 +85,7 @@ class PsScan(interfaces.plugins.PluginInterface, timeliner.TimeLinerInterface):
             context: The context to retrieve required elements (layers, symbol tables) from
             layer_name: The name of the layer on which to operate
             symbol_table: The name of the table containing the kernel symbols
-            proc: the process object with phisical address
+            proc: the process object with physical address
 
         Returns:
             A process object on virtual address layer

volatility3/framework/symbols/intermed.py

@@ -11,13 +11,13 @@ import os
 import pathlib
 import zipfile
 from abc import ABCMeta
-from typing import Any, Dict, Generator, Iterable, List, Optional, Type, Tuple, Mapping
+from typing import Any, Dict, Generator, Iterable, List, Mapping, Optional, Tuple, Type
 
 from volatility3 import schemas, symbols
 from volatility3.framework import class_subclasses, constants, exceptions, interfaces, objects
 from volatility3.framework.configuration import requirements
 from volatility3.framework.layers import resources
-from volatility3.framework.symbols import native, metadata
+from volatility3.framework.symbols import metadata, native
 
 vollog = logging.getLogger(__name__)
 
@@ -113,6 +113,9 @@ class IntermediateSymbolTable(interfaces.symbols.SymbolTableInterface):
 
         metadata = json_object.get('metadata', None)
 
+        if not metadata:
+            raise exceptions.SymbolSpaceError(f"Invalid ISF file attempted to be parsed: {isf_url}")
+
         # Determine the delegate or throw an exception
         self._delegate = self._closest_version(metadata.get('format', "0.0.0"),
                                                self._versions)(context, config_path, name, json_object, native_types,
@@ -540,7 +543,8 @@ class Version3Format(Version2Format):
         if 'type' in symbol:
             symbol_type = self._interdict_to_template(symbol['type'])
 
-        self._symbol_cache[name] = interfaces.symbols.SymbolInterface(name = name, address = address, type = symbol_type)
+        self._symbol_cache[name] = interfaces.symbols.SymbolInterface(name = name, address = address,
+                                                                      type = symbol_type)
         return self._symbol_cache[name]
 
 

volatility3/framework/symbols/linux/extensions/__init__.py

@@ -207,6 +207,48 @@ class task_struct(generic.GenericIntelProcess):
 
             yield (start, end - start)
 
+    @property
+    def is_kernel_thread(self) -> bool:
+        """Checks if this task is a kernel thread.
+
+        Returns:
+            bool: True, if this task is a kernel thread. Otherwise, False.
+        """
+        return (self.flags & constants.linux.PF_KTHREAD) != 0
+
+    @property
+    def is_thread_group_leader(self) -> bool:
+        """Checks if this task is a thread group leader.
+
+        Returns:
+            bool: True, if this task is a thread group leader. Otherwise, False.
+        """
+        return self.tgid == self.pid
+
+    @property
+    def is_user_thread(self) -> bool:
+        """Checks if this task is a user thread.
+
+        Returns:
+            bool: True, if this task is a user thread. Otherwise, False.
+        """
+        return not self.is_kernel_thread and self.tgid != self.pid
+
+    def get_threads(self) -> Iterable[interfaces.objects.ObjectInterface]:
+        """Returns a list of the task_struct based on the list_head
+        thread_node structure."""
+
+        task_symbol_table_name = self.get_symbol_table_name()
+
+        # iterating through the thread_list from thread_group
+        # this allows iterating through pointers to grab the
+        # threads and using the thread_group offset to get the
+        # corresponding task_struct
+        for task in self.thread_group.to_list(
+            f"{task_symbol_table_name}{constants.BANG}task_struct",
+            "thread_group"
+        ):
+            yield task
 
 class fs_struct(objects.StructType):
 

volatility3/framework/symbols/windows/__init__.py

@@ -1,7 +1,7 @@
 # This file is Copyright 2020 Volatility Foundation and licensed under the Volatility Software License 1.0
 # which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
 #
-import volatility3.framework.symbols.windows.extensions.pool
+
 from volatility3.framework.symbols import intermed
 from volatility3.framework.symbols.windows import extensions
 from volatility3.framework.symbols.windows.extensions import registry, pool

volatility3/framework/symbols/windows/extensions/__init__.py

@@ -7,16 +7,18 @@ import datetime
 import functools
 import logging
 import math
-from typing import Iterable, Iterator, Optional, Union, Tuple, List
+from typing import Generator, Iterable, Iterator, List, Optional, Tuple, Union
 
 from volatility3.framework import constants, exceptions, interfaces, objects, renderers, symbols
+from volatility3.framework.interfaces.objects import ObjectInterface
 from volatility3.framework.layers import intel
 from volatility3.framework.renderers import conversion
 from volatility3.framework.symbols import generic
-from volatility3.framework.symbols.windows.extensions import pool, pe, kdbg
+from volatility3.framework.symbols.windows.extensions import kdbg, pe, pool
 
 vollog = logging.getLogger(__name__)
 
+
 # Keep these in a basic module, to prevent import cycles when symbol providers require them
 
 
@@ -306,12 +308,15 @@ class MMVAD(MMVAD_SHORT):
         try:
             # this is for xp and 2003
             if self.has_member("ControlArea"):
-                file_name = self.ControlArea.FilePointer.FileName.get_string()
+                filename_obj = self.ControlArea.FilePointer.FileName
 
             # this is for vista through windows 7
             else:
-                file_name = self.Subsection.ControlArea.FilePointer.dereference().cast(
-                    "_FILE_OBJECT").FileName.get_string()
+                filename_obj = self.Subsection.ControlArea.FilePointer.dereference().cast(
+                    "_FILE_OBJECT").FileName
+
+            if filename_obj.Length > 0:
+                file_name = filename_obj.get_string()
 
         except exceptions.InvalidAddressException:
             pass
@@ -348,17 +353,32 @@ class DEVICE_OBJECT(objects.StructType, pool.ExecutiveObject):
     """A class for kernel device objects."""
 
     def get_device_name(self) -> str:
+        """Get device's name from the object header."""
         header = self.get_object_header()
         return header.NameInfo.Name.String  # type: ignore
 
+    def get_attached_devices(self) -> Generator[ObjectInterface, None, None]:
+        """Enumerate the attached device's objects"""
+        device = self.AttachedDevice.dereference()
+        while device:
+            yield device
+            device = device.AttachedDevice.dereference()
 
 class DRIVER_OBJECT(objects.StructType, pool.ExecutiveObject):
     """A class for kernel driver objects."""
 
     def get_driver_name(self) -> str:
+        """Get driver's name from the object header."""
         header = self.get_object_header()
         return header.NameInfo.Name.String  # type: ignore
 
+    def get_devices(self) -> Generator[ObjectInterface, None, None]:
+        """Enumerate the driver's device objects"""
+        device =  self.DeviceObject.dereference()
+        while device:
+            yield device
+            device = device.NextDevice.dereference()
+
     def is_valid(self) -> bool:
         """Determine if the object is valid."""
         return True
@@ -461,10 +481,13 @@ class UNICODE_STRING(objects.StructType):
         # We explicitly do *not* catch errors here, we allow an exception to be thrown
         # (otherwise there's no way to determine anything went wrong)
         # It's up to the user of this method to catch exceptions
-        return self.Buffer.dereference().cast("string",
-                                              max_length = self.Length,
-                                              errors = "replace",
-                                              encoding = "utf16")
+
+        # We manually construct an object rather than casting a dereferenced pointer in case
+        # the buffer length is 0 and the pointer is a NULL pointer
+        return self._context.object(self.vol.type_name.split(constants.BANG)[0] + constants.BANG + 'string',
+                                    layer_name = self.Buffer.vol.layer_name,
+                                    offset = self.Buffer,
+                                    max_length = self.Length, errors = 'replace', encoding = 'utf16')
 
     String = property(get_string)
 
@@ -898,8 +921,8 @@ class CONTROL_AREA(objects.StructType):
                 return False
 
             # The first SubsectionBase should not be page aligned
-            #subsection = self.get_subsection()
-            #if subsection.SubsectionBase & self.PAGE_MASK == 0:
+            # subsection = self.get_subsection()
+            # if subsection.SubsectionBase & self.PAGE_MASK == 0:
             #    return False
         except exceptions.InvalidAddressException:
             return False
@@ -948,7 +971,7 @@ class CONTROL_AREA(objects.StructType):
             subsection_offset = starting_sector * 0x200
 
             # Similar to the check in is_valid(), make sure the SubsectionBase is not page aligned.
-            #if subsection.SubsectionBase & self.PAGE_MASK == 0:
+            # if subsection.SubsectionBase & self.PAGE_MASK == 0:
             #    break
 
             ptecount = 0
@@ -979,8 +1002,8 @@ class CONTROL_AREA(objects.StructType):
                     # Currently just a temporary workaround to deal with custom bit flag
                     # in the PFN field for pages in transition state.
                     # See https://github.com/volatilityfoundation/volatility3/pull/475
-                    physoffset = (mmpte.u.Trans.PageFrameNumber & (( 1 << 33 ) - 1 ) ) << 12
-                    
+                    physoffset = (mmpte.u.Trans.PageFrameNumber & ((1 << 33) - 1)) << 12
+
                     yield physoffset, file_offset, self.PAGE_SIZE
 
                 # Go to the next PTE entry

volatility3/framework/symbols/windows/extensions/mbr.py

@@ -0,0 +1,62 @@
+# This file is Copyright 2022 Volatility Foundation and licensed under the Volatility Software License 1.0
+# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
+#
+
+from volatility3.framework import objects
+
+class PARTITION_TABLE(objects.StructType):
+
+    def get_disk_signature(self) -> str:
+        """Get Disk Signature (GUID)."""
+        return "{0:02x}-{1:02x}-{2:02x}-{3:02x}".format(
+                self.DiskSignature[0], 
+                self.DiskSignature[1],
+                self.DiskSignature[2],
+                self.DiskSignature[3]
+        )
+
+class PARTITION_ENTRY(objects.StructType):
+    
+    def get_bootable_flag(self) -> int:
+        """Get Bootable Flag."""
+        return self.BootableFlag
+    
+    def is_bootable(self) -> bool:
+        """Check Bootable Partition."""
+        return False if not (self.get_bootable_flag() == 0x80) else True
+
+    def get_partition_type(self) -> str:
+        """Get Partition Type."""
+        return self.PartitionType.lookup() if self.PartitionType.is_valid_choice else "Not Defined PartitionType"
+
+    def get_starting_chs(self) -> int:
+        """Get Starting CHS (Cylinder Header Sector) Address."""
+        return self.StartingCHS[0]
+
+    def get_ending_chs(self) -> int:
+        """Get Ending CHS (Cylinder Header Sector) Address."""
+        return self.EndingCHS[0]
+
+    def get_starting_sector(self) -> int:
+        """Get Starting Sector."""
+        return self.StartingCHS[1] % 64
+
+    def get_ending_sector(self) -> int:
+        """Get Ending Sector."""
+        return self.EndingCHS[1] % 64
+
+    def get_starting_cylinder(self) -> int:
+        """Get Starting Cylinder."""
+        return (self.StartingCHS[1] - self.get_starting_sector()) * 4 + self.StartingCHS[2]
+
+    def get_ending_cylinder(self) -> int:
+        """Get Ending Cylinder."""
+        return (self.EndingCHS[1] - self.get_ending_sector()) * 4 + self.EndingCHS[2]
+    
+    def get_starting_lba(self) -> int:
+        """Get Starting LBA (Logical Block Addressing)."""
+        return self.StartingLBA
+    
+    def get_size_in_sectors(self) -> int:
+        """Get Size in Sectors."""
+        return self.SizeInSectors

volatility3/framework/symbols/windows/extensions/pe.py

@@ -2,15 +2,15 @@
 # which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
 #
 
-from typing import Generator, Tuple
 import logging
+from typing import Generator, Tuple
 
-from volatility3.framework import constants
-from volatility3.framework import objects, interfaces
+from volatility3.framework import constants, interfaces, objects
 from volatility3.framework.renderers import conversion
 
 vollog = logging.getLogger(__name__)
 
+
 class IMAGE_DOS_HEADER(objects.StructType):
 
     def get_nt_header(self) -> interfaces.objects.ObjectInterface:
@@ -77,12 +77,13 @@ class IMAGE_DOS_HEADER(objects.StructType):
         image_base_type = nt_header.OptionalHeader.ImageBase.vol.type_name
         member_size = self._context.symbol_space.get_type(image_base_type).size
         try:
-            newval = objects.convert_value_to_data(self.vol.offset, int, nt_header.OptionalHeader.ImageBase.vol.data_format)
+            newval = objects.convert_value_to_data(self.vol.offset, int,
+                                                   nt_header.OptionalHeader.ImageBase.vol.data_format)
             new_pe = raw_data[:image_base_offset] + newval + raw_data[image_base_offset + member_size:]
         except OverflowError:
             vollog.warning("Volatility was unable to fix the image base for the PE file at base address {:#x}. " \
-                        "This will cause issues with many static analysis tools if you do not inform the " \
-                        "tool of the in-memory load address.".format(self.vol.offset))
+                           "This will cause issues with many static analysis tools if you do not inform the " \
+                           "tool of the in-memory load address.".format(self.vol.offset))
             new_pe = raw_data
 
         return new_pe
@@ -109,7 +110,7 @@ class IMAGE_DOS_HEADER(objects.StructType):
         size_of_image = nt_header.OptionalHeader.SizeOfImage
 
         # no legitimate PE is going to be larger than this
-        if size_of_image > (1024 * 1024 * 100):
+        if size_of_image > constants.windows.PE_MAX_EXTRACTION_SIZE:
             raise ValueError(f"The claimed SizeOfImage is too large: {size_of_image}")
 
         read_layer = self._context.layers[layer_name]

volatility3/framework/symbols/windows/extensions/pool.py

@@ -233,7 +233,10 @@ class POOL_TRACKER_BIG_PAGES(objects.StructType):
 
     def is_valid(self) -> bool:
         return self.Key > 0
-        # return self.Va > 0x1
+
+    def is_free(self) -> bool:
+        """Returns if the allocation is freed (True) or in-use (False)"""
+        return self.Va & 1 == 1
 
     def get_key(self) -> str:
         """Returns the Key value as a 4 character string"""

volatility3/framework/symbols/windows/extensions/registry.py

@@ -5,10 +5,10 @@
 import enum
 import logging
 import struct
-from typing import Optional, Iterable, Union
+from typing import Iterable, Optional, Union
 
-from volatility3.framework import constants, exceptions, objects, interfaces
-from volatility3.framework.layers.registry import RegistryHive, RegistryInvalidIndex, RegistryFormatException
+from volatility3.framework import constants, exceptions, interfaces, objects
+from volatility3.framework.layers.registry import RegistryFormatException, RegistryHive, RegistryInvalidIndex
 
 vollog = logging.getLogger(__name__)
 
@@ -76,7 +76,9 @@ class CMHIVE(objects.StructType):
 
         for attr in ["FileFullPath", "FileUserName", "HiveRootPath"]:
             try:
-                return getattr(self, attr).get_string()
+                name = getattr(self, attr)
+                if name.Length > 0:
+                    return name.get_string()
             except (AttributeError, exceptions.InvalidAddressException):
                 pass
 
@@ -269,7 +271,7 @@ class CM_KEY_VALUE(objects.StructType):
         if self_type == RegValueTypes.REG_DWORD_BIG_ENDIAN:
             if len(data) != struct.calcsize(">L"):
                 raise ValueError(f"Size of data does not match the type of registry value {self.get_name()}")
-            res, =  struct.unpack(">L", data)
+            res, = struct.unpack(">L", data)
             return res
         if self_type == RegValueTypes.REG_QWORD:
             if len(data) != struct.calcsize("<Q"):
@@ -277,9 +279,9 @@ class CM_KEY_VALUE(objects.StructType):
             res, = struct.unpack("<Q", data)
             return res
         if self_type in [
-                RegValueTypes.REG_SZ, RegValueTypes.REG_EXPAND_SZ, RegValueTypes.REG_LINK, RegValueTypes.REG_MULTI_SZ,
-                RegValueTypes.REG_BINARY, RegValueTypes.REG_FULL_RESOURCE_DESCRIPTOR, RegValueTypes.REG_RESOURCE_LIST,
-                RegValueTypes.REG_RESOURCE_REQUIREMENTS_LIST
+            RegValueTypes.REG_SZ, RegValueTypes.REG_EXPAND_SZ, RegValueTypes.REG_LINK, RegValueTypes.REG_MULTI_SZ,
+            RegValueTypes.REG_BINARY, RegValueTypes.REG_FULL_RESOURCE_DESCRIPTOR, RegValueTypes.REG_RESOURCE_LIST,
+            RegValueTypes.REG_RESOURCE_REQUIREMENTS_LIST
         ]:
             return data
         if self_type == RegValueTypes.REG_NONE:

volatility3/framework/symbols/windows/mbr.json

@@ -0,0 +1,240 @@
+{
+    "metadata": {
+        "producer": {
+          "version": "0.0.1",
+          "name": "Donghyun Kim (@digitalisx99)",
+          "comment": "Using structures defined in File System Forensic Analysis pg 88+",
+          "datetime": "2022-03-05T10:53:00"
+        },
+        "format": "6.1.0"
+    },
+    "base_types": {
+        "unsigned long": {
+            "kind": "int", 
+            "size": 4, 
+            "signed": false, 
+            "endian": "little"
+        }, 
+        "unsigned long long": {
+            "kind": "int", 
+            "size": 8, 
+            "signed": false, 
+            "endian": "little"
+        }, 
+        "long": {
+            "kind": "int", 
+            "size": 4, 
+            "signed": true, 
+            "endian": "little"
+        },
+        "unsigned int": {
+            "kind": "int",
+            "size": 4,
+            "signed": false,
+            "endian": "little"
+        },
+        "int": {
+            "kind": "int",
+            "size": 4,
+            "signed": true,
+            "endian": "little"
+        }, 
+        "unsigned short": {
+            "kind": "int", 
+            "size": 2, 
+            "signed": false, 
+            "endian": "little"
+        },
+        "unsigned char": {
+            "kind": "int", 
+            "size": 1, 
+            "signed": false, 
+            "endian": "little"
+        },
+        "wchar": {
+            "kind": "int", 
+            "size": 2, 
+            "signed": true, 
+            "endian": "little"
+        }
+    },
+    "symbols": {},
+    "enums": {
+        "PartitionTypes": {
+            "base": "unsigned char",
+            "constants": {
+                "Empty": 0,
+                "FAT12,CHS": 1,
+                "FAT16 16-32MB,CHS": 4,
+                "Microsoft Extended": 5,
+                "FAT16 32MB,CHS": 6,
+                "NTFS": 7,
+                "FAT32,CHS": 11,
+                "FAT32,LBA": 12,
+                "FAT16, 32MB-2GB,LBA": 14,
+                "Microsoft Extended, LBA": 15,
+                "Hidden FAT12,CHS": 17,
+                "Hidden FAT16,16-32MB,CHS": 20,
+                "Hidden FAT16,32MB-2GB,CHS": 22,
+                "AST SmartSleep Partition": 24,
+                "Hidden FAT32,CHS": 27,
+                "Hidden FAT32,LBA": 28,
+                "Hidden FAT16,32MB-2GB,LBA": 30,
+                "PQservice": 39,
+                "Plan 9 partition": 57,
+                "PartitionMagic recovery partition": 60,
+                "Microsoft MBR,Dynamic Disk": 66,
+                "GoBack partition": 68,
+                "Novell": 81,
+                "CP/M": 82,
+                "Unix System V": 99,
+                "PC-ARMOUR protected partition": 100,
+                "Solaris x86 or Linux Swap": 130,
+                "Linux": 131,
+                "Hibernation": 132,
+                "Linux Extended": 133,
+                "NTFS Volume Set": 134,
+                "NTFS Volume Set": 135,
+                "BSD/OS": 159,
+                "Hibernation": 160,
+                "Hibernation": 161,
+                "FreeBSD": 165,
+                "OpenBSD": 166,
+                "Mac OSX": 168,
+                "NetBSD": 169,
+                "Mac OSX Boot": 171,
+                "MacOS X HFS": 175,
+                "BSDI": 183,
+                "BSDI Swap": 184,
+                "Boot Wizard hidden": 187,
+                "Solaris 8 boot partition": 190,
+                "CP/M-86": 216,
+                "Dell PowerEdge Server utilities (FAT fs)": 222,
+                "DG/UX virtual disk manager partition": 223,
+                "BeOS BFS": 235,
+                "EFI GPT Disk": 238,
+                "EFI System Partition": 239,
+                "VMWare File System": 251,
+                "VMWare Swap": 252
+            },
+            "size": 1
+        }
+    },
+    "user_types": {
+        "PARTITION_ENTRY":{
+            "fields": {
+                "BootableFlag": {
+                    "offset": 0,
+                    "type": {
+                        "kind": "base",
+                        "name": "unsigned char"
+                    }
+                },
+                "StartingCHS": {
+                    "offset": 1,
+                    "type": {
+                        "count": 3,
+                        "kind": "array",
+                        "subtype": {
+                            "kind": "base",
+                            "name": "unsigned char"
+                        }
+                    }
+                },
+                "PartitionType": {
+                    "offset": 4,
+                    "type": {
+                        "kind": "enum",
+                        "name": "PartitionTypes"
+                    }
+                },
+                "EndingCHS": {
+                    "offset": 5,
+                    "type": {
+                        "count": 3,
+                        "kind": "array",
+                        "subtype": {
+                            "kind": "base",
+                            "name": "unsigned char"
+                        }
+                    }
+                },
+                "StartingLBA": {
+                    "offset": 8,
+                    "type": {
+                        "kind": "base",
+                        "name": "unsigned int"
+                    }
+                },
+                "SizeInSectors": {
+                    "offset": 12,
+                    "type": {
+                        "kind": "base",
+                        "name": "unsigned int"
+                    }
+                }
+            },
+            "kind": "struct",
+            "size": 16
+        },
+        "PARTITION_TABLE":{
+            "fields":{
+                "DiskSignature": {
+                    "offset": 440,
+                    "type": {
+                        "count": 4,
+                        "kind": "array",
+                        "subtype": {
+                            "kind": "base",
+                            "name": "unsigned char"
+                        }
+                    }
+                },
+                "Unused": {
+                    "offset": 444,
+                    "type": {
+                        "kind": "base",
+                        "name": "unsigned short"
+                    }
+                },
+                "FirstEntry":{
+                    "offset": 446,
+                    "type": {
+                        "kind": "struct",
+                        "name": "PARTITION_ENTRY"
+                    }
+                },
+                "SecondEntry":{
+                    "offset": 462,
+                    "type": {
+                        "kind": "struct",
+                        "name": "PARTITION_ENTRY"
+                    }
+                },
+                "ThirdEntry":{
+                    "offset": 478,
+                    "type": {
+                        "kind": "struct",
+                        "name": "PARTITION_ENTRY"
+                    }
+                },
+                "FourthEntry":{
+                    "offset": 494,
+                    "type": {
+                        "kind": "struct",
+                        "name": "PARTITION_ENTRY"
+                    }
+                },
+                "Signature":{
+                    "offset": 510,
+                    "type": {
+                        "kind": "base",
+                        "name": "unsigned short"
+                    }
+                }
+            },
+            "kind": "struct",
+            "size": 512
+        }
+    }
+}

volatility3/framework/symbols/windows/mft.json

@@ -134,7 +134,7 @@
                           "kind": "base",
                           "name": "unsigned char"
                         }
-                      }
+                    }
                 },
                 "UpdateSequenceOffset": {
                     "offset": 4,
@@ -192,7 +192,7 @@
                         "name": "unsigned int"
                     }
                 },
-                "AlocatedSize": {
+                "AllocatedSize": {
                     "offset": 28,
                     "type":{
                         "kind": "base",
@@ -270,7 +270,8 @@
                     "offset": 8,
                     "type": {
                         "kind": "base",
-                        "name": "unsigned char"                      }
+                        "name": "unsigned char"                      
+                    }
                 },
                 "NameLength": {
                     "offset": 9,
@@ -322,7 +323,8 @@
                     "offset": 8,
                     "type": {
                         "kind": "base",
-                        "name": "unsigned short"                      }
+                        "name": "unsigned short"                      
+                    }
                 }
             },
             "kind": "struct",

volatility3/framework/symbols/windows/pdbutil.py

@@ -146,7 +146,7 @@ class PDBUtility(interfaces.configuration.VersionableInterface):
         max_size = pe_data.OPTIONAL_HEADER.SizeOfImage
 
         # Proper data
-        virtual_data = layer.read(offset, max_size)
+        virtual_data = layer.read(offset, max_size, pad=True)
         pe_data = pefile.PE(data = virtual_data)
 
         # De-virtualize the memory