From 8801a8974b72c19527d85587e312dbf3433d2dc9 Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Sun, 27 Mar 2022 21:19:04 +0530 Subject: [PATCH 01/67] Added Caption To make it look organized in the left side of the readthedocs. --- doc/source/index.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/doc/source/index.rst b/doc/source/index.rst index 3b5a5d2a8..50eaab694 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -10,6 +10,7 @@ Volatility 3 is Open Source. Here are some guidelines for using Volatility 3 effectively: .. toctree:: + :caption: Documentation basics development @@ -18,10 +19,10 @@ Here are some guidelines for using Volatility 3 effectively: volshell glossary -Python Packages -=============== .. toctree:: + :caption: Python Packages + volatility3 Indices and tables From 26251f28c23a7b1ef361ef4d083ce1c2df95e4e5 Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Sun, 27 Mar 2022 21:33:35 +0530 Subject: [PATCH 02/67] Structure for Getting started added --- doc/source/index.rst | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/doc/source/index.rst b/doc/source/index.rst index 50eaab694..0d35b02ba 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -25,6 +25,15 @@ Here are some guidelines for using Volatility 3 effectively: volatility3 + +.. toctree:: + :caption: Getting Started + + FAQ + Installation + Linux + Windows + Indices and tables ================== From 0f6bb99d115fd4b629bbdd5085c354cea98d1d72 Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Tue, 29 Mar 2022 17:41:37 +0530 Subject: [PATCH 03/67] Cross document linked for symbol table Received help from my friend to resolve issues with it Co-authored-by: Abhinandhan S Signed-off-by: Tejas <47889755+tejas15802@users.noreply.github.com> --- doc/source/Linux.rst | 6 ++++++ doc/source/conf.py | 4 +++- doc/source/symbol-tables.rst | 2 +- 3 files changed, 10 insertions(+), 2 deletions(-) create mode 100644 doc/source/Linux.rst diff --git a/doc/source/Linux.rst b/doc/source/Linux.rst new file mode 100644 index 000000000..280126cb0 --- /dev/null +++ b/doc/source/Linux.rst @@ -0,0 +1,6 @@ +Linux +===== + +How to create symbol tables + +- :ref:`symbol-tables:Mac or Linux symbol tables`. diff --git a/doc/source/conf.py b/doc/source/conf.py index 731a73d56..cadf6d3f2 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -111,9 +111,11 @@ needs_sphinx = '2.0' # ones. extensions = [ 'sphinx.ext.autodoc', 'sphinx.ext.doctest', 'sphinx.ext.napoleon', 'sphinx.ext.intersphinx', 'sphinx.ext.todo', - 'sphinx.ext.coverage', 'sphinx.ext.viewcode' + 'sphinx.ext.coverage', 'sphinx.ext.viewcode', 'sphinx.ext.autosectionlabel' ] +autosectionlabel_prefix_document = True + try: import sphinx_autodoc_typehints diff --git a/doc/source/symbol-tables.rst b/doc/source/symbol-tables.rst index 245dd9c67..36b283fff 100644 --- a/doc/source/symbol-tables.rst +++ b/doc/source/symbol-tables.rst @@ -38,7 +38,7 @@ following command: The :envvar:`PYTHONPATH` environment variable is not required if the Volatility library is installed in the system's library path or a virtual environment. -Mac/Linux symbol tables +Mac or Linux symbol tables ----------------------- For Mac/Linux systems, both use the same mechanism for identification. JSON files live under the symbol directories, From 7ac6a60ff9546e89919e33598142d19b30f8082c Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Wed, 30 Mar 2022 00:05:08 +0530 Subject: [PATCH 04/67] Updated linux page similar to vol2 wiki --- doc/source/Linux.rst | 73 +++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 69 insertions(+), 4 deletions(-) diff --git a/doc/source/Linux.rst b/doc/source/Linux.rst index 280126cb0..decc0c246 100644 --- a/doc/source/Linux.rst +++ b/doc/source/Linux.rst @@ -1,6 +1,71 @@ -Linux -===== +Linux Tutorial +============== + +This guide gives you a brief introduction to how volatility3 works and some demonstration on suite of plugins available from + +Procedure to create symbol tables for linux +-------------------------------------------- + +To create symbol table please refer this :ref:`symbol-tables:Mac or Linux symbol tables`. +You can also find some ISF files from this website `Linux ISF Server `_ Which is built and maintained by `kevthehermit `_. + +Using plugins +------------- + +The following is the syntax to run volatility tool. + +.. code-block:: shell-session + + $ python3 vol.py -f plugin_name plugin_option + +List of Plugins +---------------- + +Following are the list of linux plugins available for volatility3. More plugins will be available on future releases. +For plugin requests, Please create an issue with description of the plugin. + +.. code-block:: shell-session + + $ vol3 --help | grep -i linux + + banners.Banners Attempts to identify potential linux banners in an + linux.bash.Bash Recovers bash command history from memory. + linux.check_afinfo.Check_afinfo + linux.check_creds.Check_creds + linux.check_idt.Check_idt + linux.check_modules.Check_modules + linux.check_syscall.Check_syscall + linux.elfs.Elfs Lists all memory mapped ELF files for all processes. + linux.keyboard_notifiers.Keyboard_notifiers + linux.kmsg.Kmsg Kernel log buffer reader + linux.lsmod.Lsmod Lists loaded kernel modules. + linux.lsof.Lsof Lists all memory maps for all processes. + linux.malfind.Malfind + linux.proc.Maps Lists all memory maps for all processes. + linux.pslist.PsList + Lists the processes present in a particular linux + linux.pstree.PsTree + linux.tty_check.tty_check + + +Acquiring memory +---------------- + +Volatility does not provide the ability to acquire memory. We recommend using `Lime `_ for this purpose. +It supports 32 and 64 bit captures from native Intel hardware as well as virtual machine guests. +It also supports capture from Android devices. See below for example commands building and running LiME: + +.. code-block:: shell-session + + $ tar -xvzf lime-forensics-1.1-r14.tar.gz + $ cd lime-forensics-1.1-r14/src + $ make + .... + CC [M] /home/mhl/Downloads/src/tcp.o + CC [M] /home/mhl/Downloads/src/disk.o + .... + $ sudo insmod lime-3.2.0-23-generic.ko "path=/home/mhl/ubuntu.lime format=lime" + $ ls -alh /home/mhl/ubuntu.lime + -r--r--r-- 1 root root 2.0G Aug 17 19:37 /home/mhl/ubuntu.lime -How to create symbol tables -- :ref:`symbol-tables:Mac or Linux symbol tables`. From 0d4c4b58081b808f248f5fe51516a7b5a7081cf2 Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Wed, 30 Mar 2022 08:38:06 +0530 Subject: [PATCH 05/67] Making changes as per review --- doc/source/Linux.rst | 82 +++++++++++++++++++------------------------- 1 file changed, 36 insertions(+), 46 deletions(-) diff --git a/doc/source/Linux.rst b/doc/source/Linux.rst index decc0c246..6ee797c76 100644 --- a/doc/source/Linux.rst +++ b/doc/source/Linux.rst @@ -3,55 +3,10 @@ Linux Tutorial This guide gives you a brief introduction to how volatility3 works and some demonstration on suite of plugins available from -Procedure to create symbol tables for linux --------------------------------------------- - -To create symbol table please refer this :ref:`symbol-tables:Mac or Linux symbol tables`. -You can also find some ISF files from this website `Linux ISF Server `_ Which is built and maintained by `kevthehermit `_. - -Using plugins -------------- - -The following is the syntax to run volatility tool. - -.. code-block:: shell-session - - $ python3 vol.py -f plugin_name plugin_option - -List of Plugins ----------------- - -Following are the list of linux plugins available for volatility3. More plugins will be available on future releases. -For plugin requests, Please create an issue with description of the plugin. - -.. code-block:: shell-session - - $ vol3 --help | grep -i linux - - banners.Banners Attempts to identify potential linux banners in an - linux.bash.Bash Recovers bash command history from memory. - linux.check_afinfo.Check_afinfo - linux.check_creds.Check_creds - linux.check_idt.Check_idt - linux.check_modules.Check_modules - linux.check_syscall.Check_syscall - linux.elfs.Elfs Lists all memory mapped ELF files for all processes. - linux.keyboard_notifiers.Keyboard_notifiers - linux.kmsg.Kmsg Kernel log buffer reader - linux.lsmod.Lsmod Lists loaded kernel modules. - linux.lsof.Lsof Lists all memory maps for all processes. - linux.malfind.Malfind - linux.proc.Maps Lists all memory maps for all processes. - linux.pslist.PsList - Lists the processes present in a particular linux - linux.pstree.PsTree - linux.tty_check.tty_check - - Acquiring memory ---------------- -Volatility does not provide the ability to acquire memory. We recommend using `Lime `_ for this purpose. +Volatility does not provide the ability to acquire memory. In this tutorial we will see how we can use `Lime `_ for this purpose. It supports 32 and 64 bit captures from native Intel hardware as well as virtual machine guests. It also supports capture from Android devices. See below for example commands building and running LiME: @@ -68,4 +23,39 @@ It also supports capture from Android devices. See below for example commands bu $ ls -alh /home/mhl/ubuntu.lime -r--r--r-- 1 root root 2.0G Aug 17 19:37 /home/mhl/ubuntu.lime +Procedure to create symbol tables for linux +-------------------------------------------- + +To create a symbol table please refer this :ref:`symbol-tables:Mac or Linux symbol tables`. +We can also find some ISF files from `Linux ISF Server `_ , which is built and maintained by `kevthehermit `_. + + +Using plugins +------------- + +The following is the syntax to run volatility tool. + +.. code-block:: shell-session + + $ python3 vol.py -f plugin_name plugin_option + +Listing plugins +--------------- + +Following are the list of linux plugins available for volatility3. More plugins will be available on future releases. +For plugin requests, Please create an issue with description of the plugin. + +.. code-block:: shell-session + + $ vol3 --help | grep -i linux. | head -n 5 + banners.Banners Attempts to identify potential linux banners in an + linux.bash.Bash Recovers bash command history from memory. + linux.check_afinfo.Check_afinfo + linux.check_creds.Check_creds + linux.check_idt.Check_idt + + + + + From f4e1533628dcfe5bf45e775259afa68af3e3983e Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Sat, 2 Apr 2022 17:28:36 +0530 Subject: [PATCH 06/67] Order changed --- doc/source/Linux.rst | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/doc/source/Linux.rst b/doc/source/Linux.rst index 6ee797c76..e7f81ff75 100644 --- a/doc/source/Linux.rst +++ b/doc/source/Linux.rst @@ -30,15 +30,6 @@ To create a symbol table please refer this :ref:`symbol-tables:Mac or Linux symb We can also find some ISF files from `Linux ISF Server `_ , which is built and maintained by `kevthehermit `_. -Using plugins -------------- - -The following is the syntax to run volatility tool. - -.. code-block:: shell-session - - $ python3 vol.py -f plugin_name plugin_option - Listing plugins --------------- @@ -55,6 +46,15 @@ For plugin requests, Please create an issue with description of the plugin. linux.check_idt.Check_idt +Using plugins +------------- + +The following is the syntax to run volatility tool. + +.. code-block:: shell-session + + $ python3 vol.py -f plugin_name plugin_option + From b6830a1f3172588e310b149da1bad28377903ad3 Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Sat, 2 Apr 2022 17:38:46 +0530 Subject: [PATCH 07/67] Additional context for proceudre to create symbol --- doc/source/Linux.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/source/Linux.rst b/doc/source/Linux.rst index e7f81ff75..afbccec49 100644 --- a/doc/source/Linux.rst +++ b/doc/source/Linux.rst @@ -28,6 +28,7 @@ Procedure to create symbol tables for linux To create a symbol table please refer this :ref:`symbol-tables:Mac or Linux symbol tables`. We can also find some ISF files from `Linux ISF Server `_ , which is built and maintained by `kevthehermit `_. +After creating the file or downloading the file from the ISF server, please place the file under the directory ``volatility3/symbols/linux``. Make a directory linux under symbols. Listing plugins From 20f1fc39f243e61d9e6fba9c1472a671791462a8 Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Sat, 2 Apr 2022 18:46:16 +0530 Subject: [PATCH 08/67] Example 1 Added --- doc/source/Linux.rst | 128 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 128 insertions(+) diff --git a/doc/source/Linux.rst b/doc/source/Linux.rst index afbccec49..2913a93b2 100644 --- a/doc/source/Linux.rst +++ b/doc/source/Linux.rst @@ -57,6 +57,134 @@ The following is the syntax to run volatility tool. $ python3 vol.py -f plugin_name plugin_option +Example +------- + +Example 1 +~~~~~~~~~ + +In this example we will be using memory dump from Insomni'hack teaser 2020 CTF. Challenge name Getdents, you can find the memory dump +in the link `here `_ . We will limit the discussion to memory forensics with volatility3 and not extend to other parts of the challenges. +I'd like to say thanks to `stuxnet `_ for providing this memory dump and `writeup `_. +.. code-block:: shell-session + $ python3 vol.py -f memory.vmem banners + + Volatility 3 Framework 2.0.3 + + Progress: 100.00 PDB scanning finished + Offset Banner + + 0x141c1390 Linux version 4.15.0-42-generic (buildd@lgw01-amd64-023) (gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #45-Ubuntu SMP Thu Nov 15 19:32:57 UTC 2018 (Ubuntu 4.15.0-42.45-generic 4.15.18) + 0x63a00160 Linux version 4.15.0-72-generic (buildd@lcy01-amd64-026) (gcc version 7.4.0 (Ubuntu 7.4.0-1ubuntu1~18.04.1)) #81-Ubuntu SMP Tue Nov 26 12:20:02 UTC 2019 (Ubuntu 4.15.0-72.81-generic 4.15.18) + 0x6455c4d4 Linux version 4.15.0-72-generic (buildd@lcy01-amd64-026) (gcc version 7.4.0 (Ubuntu 7.4.0-1ubuntu1~18.04.1)) #81-Ubuntu SMP Tue Nov 26 12:20:02 UTC 2019 (Ubuntu 4.15.0-72.81-generic 4.15.18) + 0x6e1e055f Linux version 4.15.0-72-generic (buildd@lcy01-amd64-026) (gcc version 7.4.0 (Ubuntu 7.4.0-1ubuntu1~18.04.1)) #81-Ubuntu SMP Tue Nov 26 12:20:02 UTC 2019 (Ubuntu 4.15.0-72.81-generic 4.15.18) + 0x7fde0010 Linux version 4.15.0-72-generic (buildd@lcy01-amd64-026) (gcc version 7.4.0 (Ubuntu 7.4.0-1ubuntu1~18.04.1)) #81-Ubuntu SMP Tue Nov 26 12:20:02 UTC 2019 (Ubuntu 4.15.0-72.81-generic 4.15.18) + + +This above command helps us to find the memory dump's kernel version and the distribution version. Now using the above banner we can search for ISF file from the ISF server. +If you do not find the ISF file then, please follow the instructions on :ref:`Linux:Procedure to create symbol tables for linux`. After that place the ISF file under ``volatility3/symbols/linux`` directory. + +.. tip:: Use the banner text which is most repeated to search from ISF Server. + + +.. code-block:: shell-session + + $ python3 vol.py -f memory.vmem linux.pslist + + Volatility 3 Framework 2.0.3 Stacking attempts finished + + PID PPID COMM + + 1 0 systemd + 2 0 kthreadd + 3 2 kworker/0:0 + 4 2 kworker/0:0H + 5 2 kworker/u256:0 + 6 2 mm_percpu_wq + 7 2 ksoftirqd/0 + 8 2 rcu_sched + 9 2 rcu_bh + 10 2 migration/0 + 11 2 watchdog/0 + 12 2 cpuhp/0 + 13 2 kdevtmpfs + 14 2 netns + 15 2 rcu_tasks_kthre + 16 2 kauditd + ..... + +``linux.pslist`` helps us to list the processes which are running, their PIDs and PPIDs. + +.. code-block:: shell-session + + $ python3 vol.py -f memory.vmem linux.pstree + Volatility 3 Framework 2.0.3 + Progress: 100.00 Stacking attempts finished + PID PPID COMM + + 1 0 systemd + * 636 1 polkitd + * 514 1 acpid + * 1411 1 pulseaudio + * 517 1 rsyslogd + * 637 1 cups-browsed + * 903 1 whoopsie + * 522 1 ModemManager + * 525 1 cron + * 526 1 avahi-daemon + ** 542 526 avahi-daemon + * 657 1 unattended-upgr + * 914 1 kerneloops + * 532 1 dbus-daemon + * 1429 1 ibus-x11 + * 929 1 kerneloops + * 1572 1 gsd-printer + * 933 1 upowerd + * 1071 1 rtkit-daemon + * 692 1 gdm3 + ** 1234 692 gdm-session-wor + *** 1255 1234 gdm-x-session + **** 1257 1255 Xorg + **** 1266 1255 gnome-session-b + ***** 1537 1266 gsd-clipboard + ***** 1539 1266 gsd-color + ***** 1542 1266 gsd-datetime + ***** 2950 1266 deja-dup-monito + ***** 1546 1266 gsd-housekeepin + ***** 1548 1266 gsd-keyboard + ***** 1550 1266 gsd-media-keys + +``linux.pstree`` helps us to display the parent child relation of processes. + +Now to find the commands ran in bash shell. Lets use ``linux.bash``. + +.. code-block:: shell-session + + $ python3 vol.py -f memory.vmem linux.bash + + Volatility 3 Framework 2.0.3 + Progress: 100.00 Stacking attempts finished + PID Process CommandTime Command + + 1733 bash 2020-01-16 14:00:36.000000 sudo reboot + 1733 bash 2020-01-16 14:00:36.000000 AWAVH�� + 1733 bash 2020-01-16 14:00:36.000000 sudo apt upgrade + 1733 bash 2020-01-16 14:00:36.000000 sudo apt upgrade + 1733 bash 2020-01-16 14:00:36.000000 sudo reboot + 1733 bash 2020-01-16 14:00:36.000000 sudo apt update + 1733 bash 2020-01-16 14:00:36.000000 sudo apt update + 1733 bash 2020-01-16 14:00:36.000000 sudo reboot + 1733 bash 2020-01-16 14:00:36.000000 sudo apt upgrade + 1733 bash 2020-01-16 14:00:36.000000 sudo apt update + 1733 bash 2020-01-16 14:00:36.000000 rub + 1733 bash 2020-01-16 14:00:36.000000 sudo apt upgrade + 1733 bash 2020-01-16 14:00:36.000000 uname -a + 1733 bash 2020-01-16 14:00:36.000000 uname -a + 1733 bash 2020-01-16 14:00:36.000000 sudo apt autoclean + 1733 bash 2020-01-16 14:00:36.000000 sudo reboot + 1733 bash 2020-01-16 14:00:36.000000 sudo apt upgrade + 1733 bash 2020-01-16 14:00:41.000000 chmod +x meterpreter + 1733 bash 2020-01-16 14:00:42.000000 sudo ./meterpreter From ada212da9232a0fff1cdc21bfc2ed7c1793aa20f Mon Sep 17 00:00:00 2001 From: TEJENDRA SARADHI <47889755+tejas15802@users.noreply.github.com> Date: Sun, 3 Apr 2022 06:13:53 +0530 Subject: [PATCH 09/67] lime to LiME Fix inconsistency --- doc/source/Linux.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/source/Linux.rst b/doc/source/Linux.rst index 2913a93b2..294f8ff63 100644 --- a/doc/source/Linux.rst +++ b/doc/source/Linux.rst @@ -6,7 +6,7 @@ This guide gives you a brief introduction to how volatility3 works and some demo Acquiring memory ---------------- -Volatility does not provide the ability to acquire memory. In this tutorial we will see how we can use `Lime `_ for this purpose. +Volatility does not provide the ability to acquire memory. In this tutorial we will see how we can use `LiME `_ for this purpose. It supports 32 and 64 bit captures from native Intel hardware as well as virtual machine guests. It also supports capture from Android devices. See below for example commands building and running LiME: From bbffe9f620924c10de7a05efe1a41692aad8d26d Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Wed, 6 Apr 2022 21:08:07 +0530 Subject: [PATCH 10/67] Windows page added and few commands in example1 --- doc/source/Windows.rst | 88 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) create mode 100644 doc/source/Windows.rst diff --git a/doc/source/Windows.rst b/doc/source/Windows.rst new file mode 100644 index 000000000..0d8c92c41 --- /dev/null +++ b/doc/source/Windows.rst @@ -0,0 +1,88 @@ +Windows Tutorial +================ + +This guide gives you a brief introduction to how volatility3 works and some demonstration on suite of plugins available from + +Acquiring memory +---------------- + +Volatility does not provide the ability to acquire memory. In this tutorial we will see how we can use `WinPmem `_ for this purpose. + +Listing Plugins +--------------- + + $ vol3 --help | grep windows | head -n 5 + windows.bigpools.BigPools + windows.cmdline.CmdLine + windows.crashinfo.Crashinfo + windows.dlllist.DllList + Lists the loaded modules in a particular windows + +Using plugins +------------- + +The following is the syntax to run volatility tool. + +.. code-block:: shell-session + + $ python3 vol.py -f plugin_name plugin_option + + +Example +------- + +Example 1 +~~~~~~~~~ + +In this example we will be using memory dump from PragyanCTF'22. The dump is available `here `_. +We will limit the discussion to memory forensics with volatility3 and not extend to other parts of the challenges. + +In windows memory forensics using volatility3, most of the times we do not require creating a ISF file. + +.. code-block:: shell-session + + $ vol3 -f MemDump.DMP windows.pslist | head -n 10 + + Volatility 3 Framework 2.0.2 PDB scanning finished + + PID PPID ImageFileName Offset(V) Threads Handles SessionId Wow64 CreateTime ExitTime File output + + 4 0 System 0xfa8000cbc040 85 492 N/A False 2022-02-07 16:30:12.000000 N/A Disabled + 276 4 smss.exe 0xfa8001e04040 2 29 N/A False 2022-02-07 16:30:12.000000 N/A Disabled + 352 336 csrss.exe 0xfa8002110b30 9 375 0 False 2022-02-07 16:30:13.000000 N/A Disabled + 404 336 wininit.exe 0xfa800219f060 3 74 0 False 2022-02-07 16:30:13.000000 N/A Disabled + 412 396 csrss.exe 0xfa80021c5b30 9 224 1 False 2022-02-07 16:30:13.000000 N/A Disabled + 468 396 winlogon.exe 0xfa8002284060 5 113 1 False 2022-02-07 16:30:14.000000 N/A Disabled + +``windows.pslist`` helps us list the processes running while the memory dump was taken. + +.. code-block:: shell-session + + $ vol3 -f MemDump.DMP windows.pstree | head -n 20 + Volatility 3 Framework 2.0.2 PDB scanning finished + + PID PPID ImageFileName Offset(V) Threads Handles SessionId Wow64 CreateTime ExitTime + + 4 0 System 0xfa8000cbc040 85 492 N/A False 2022-02-07 16:30:12.000000 N/A + * 276 4 smss.exe 0xfa8001e04040 2 29 N/A False 2022-02-07 16:30:12.000000 N/A + 352 336 csrss.exe 0xfa8002110b30 9 375 0 False 2022-02-07 16:30:13.000000 N/A + 404 336 wininit.exe 0xfa800219f060 3 74 0 False 2022-02-07 16:30:13.000000 N/A + * 504 404 services.exe 0xfa80022ccb30 7 190 0 False 2022-02-07 16:30:14.000000 N/A + ** 960 504 svchost.exe 0xfa8001c17b30 39 1003 0 False 2022-02-07 16:30:14.000000 N/A + ** 1216 504 svchost.exe 0xfa80026e0b30 18 311 0 False 2022-02-07 16:30:15.000000 N/A + ** 1312 504 svchost.exe 0xfa8002740380 19 287 0 False 2022-02-07 16:30:15.000000 N/A + ** 1984 504 taskhost.exe 0xfa8002eb1b30 8 129 1 False 2022-02-07 16:30:27.000000 N/A + ** 804 504 svchost.exe 0xfa80024ca5f0 20 450 0 False 2022-02-07 16:30:14.000000 N/A + *** 100 804 audiodg.exe 0xfa80025b4b30 6 131 0 False 2022-02-07 16:30:14.000000 N/A + ** 1568 504 SearchIndexer. 0xfa800254b480 12 616 0 False 2022-02-07 16:30:32.000000 N/A + ** 744 504 svchost.exe 0xfa8002477b30 8 265 0 False 2022-02-07 16:30:14.000000 N/A + ** 1096 504 svchost.exe 0xfa800260db30 14 357 0 False 2022-02-07 16:30:14.000000 N/A + ** 616 504 svchost.exe 0xfa8002b86ab0 13 314 0 False 2022-02-07 16:32:16.000000 N/A + ** 624 504 svchost.exe 0xfa8002410630 10 350 0 False 2022-02-07 16:30:14.000000 N/A + +``windows.pstree`` helps us to display the parent child relation of processes. + + + + + From 39db890ebd7dd0ef191e96831133e46812ed112b Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Wed, 6 Apr 2022 21:10:41 +0530 Subject: [PATCH 11/67] Fix code block syntax highlight --- doc/source/Windows.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/source/Windows.rst b/doc/source/Windows.rst index 0d8c92c41..5a197a73e 100644 --- a/doc/source/Windows.rst +++ b/doc/source/Windows.rst @@ -11,6 +11,8 @@ Volatility does not provide the ability to acquire memory. In this tutorial we w Listing Plugins --------------- +.. code-block:: shell-session + $ vol3 --help | grep windows | head -n 5 windows.bigpools.BigPools windows.cmdline.CmdLine From 215a43478932c9fa2f6db52487aab4f3fc672576 Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Wed, 6 Apr 2022 21:13:16 +0530 Subject: [PATCH 12/67] Update alias vol3 to python3 vol.py --- doc/source/Windows.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/source/Windows.rst b/doc/source/Windows.rst index 5a197a73e..b086cc57b 100644 --- a/doc/source/Windows.rst +++ b/doc/source/Windows.rst @@ -12,8 +12,8 @@ Listing Plugins --------------- .. code-block:: shell-session - - $ vol3 --help | grep windows | head -n 5 + + $ python3 vol.py --help | grep windows | head -n 5 windows.bigpools.BigPools windows.cmdline.CmdLine windows.crashinfo.Crashinfo @@ -43,7 +43,7 @@ In windows memory forensics using volatility3, most of the times we do not requi .. code-block:: shell-session - $ vol3 -f MemDump.DMP windows.pslist | head -n 10 + $ python3 vol.py -f MemDump.DMP windows.pslist | head -n 10 Volatility 3 Framework 2.0.2 PDB scanning finished @@ -60,7 +60,7 @@ In windows memory forensics using volatility3, most of the times we do not requi .. code-block:: shell-session - $ vol3 -f MemDump.DMP windows.pstree | head -n 20 + $ python3 vol.py -f MemDump.DMP windows.pstree | head -n 20 Volatility 3 Framework 2.0.2 PDB scanning finished PID PPID ImageFileName Offset(V) Threads Handles SessionId Wow64 CreateTime ExitTime From 69c50e3c6511bf8dc5dbbd80d908ab82f1b926ff Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Sun, 3 Jul 2022 19:57:16 +0530 Subject: [PATCH 13/67] last command added to example 1 --- doc/source/Windows.rst | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/doc/source/Windows.rst b/doc/source/Windows.rst index b086cc57b..3e6844f22 100644 --- a/doc/source/Windows.rst +++ b/doc/source/Windows.rst @@ -84,6 +84,22 @@ In windows memory forensics using volatility3, most of the times we do not requi ``windows.pstree`` helps us to display the parent child relation of processes. +.. code-block:: shell-session + + $ python3 vol.py -f MemDump.DMP windows.hashdump + Volatility 3 Framework 2.0.3 + Progress: 100.00 PDB scanning finished + User rid lmhash nthash + + Administrator 500 aad3b435b51404eeaad3b435b51404ee 31d6cfe0d16ae931b73c59d7e0c089c0 + Guest 501 aad3b435b51404eeaad3b435b51404ee 31d6cfe0d16ae931b73c59d7e0c089c0 + Frank Reynolds 1000 aad3b435b51404eeaad3b435b51404ee a88d1e18706d3aa676e01e5943d15911 + HomeGroupUser$ 1002 aad3b435b51404eeaad3b435b51404ee af10ecac6ea817d2bb56e3e5c33ce1cd + Dennis 1003 aad3b435b51404eeaad3b435b51404ee cf96684bbc7877920adaa9663698bf54 + +``windows.hashdump`` helps us to list the hashes of the users in the system. + + From 179d35d03dded64676339ceea37b04ef4a7107c5 Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Tue, 5 Jul 2022 19:15:48 +0530 Subject: [PATCH 14/67] volatility to volatiliy3 --- doc/source/Linux.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/source/Linux.rst b/doc/source/Linux.rst index 294f8ff63..d8b30ff28 100644 --- a/doc/source/Linux.rst +++ b/doc/source/Linux.rst @@ -1,7 +1,7 @@ Linux Tutorial ============== -This guide gives you a brief introduction to how volatility3 works and some demonstration on suite of plugins available from +This guide gives you a brief introduction to how volatility3 works and some demonstration of several of the plugins available from Acquiring memory ---------------- From 92f308b5e7c7b8556072e411d66a5e089828e658 Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Tue, 5 Jul 2022 19:16:07 +0530 Subject: [PATCH 15/67] volatility3 specified --- doc/source/Linux.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/source/Linux.rst b/doc/source/Linux.rst index d8b30ff28..28fa4e0cd 100644 --- a/doc/source/Linux.rst +++ b/doc/source/Linux.rst @@ -6,7 +6,7 @@ This guide gives you a brief introduction to how volatility3 works and some demo Acquiring memory ---------------- -Volatility does not provide the ability to acquire memory. In this tutorial we will see how we can use `LiME `_ for this purpose. +Volatility3 does not provide the ability to acquire memory. In this tutorial we will see how we can use `LiME `_ for this purpose. It supports 32 and 64 bit captures from native Intel hardware as well as virtual machine guests. It also supports capture from Android devices. See below for example commands building and running LiME: From b782e1d751d57fabea3e382ebdee55966eb8eeaf Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Tue, 5 Jul 2022 19:27:36 +0530 Subject: [PATCH 16/67] path adjustments made to have relative and generic --- doc/source/Linux.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/source/Linux.rst b/doc/source/Linux.rst index 28fa4e0cd..3c77a16b6 100644 --- a/doc/source/Linux.rst +++ b/doc/source/Linux.rst @@ -16,12 +16,12 @@ It also supports capture from Android devices. See below for example commands bu $ cd lime-forensics-1.1-r14/src $ make .... - CC [M] /home/mhl/Downloads/src/tcp.o - CC [M] /home/mhl/Downloads/src/disk.o + CC [M] lime-forensics-1.1-r14/src/tcp.o + CC [M] lime-forensics-1.1-r14/src/disk.o .... - $ sudo insmod lime-3.2.0-23-generic.ko "path=/home/mhl/ubuntu.lime format=lime" - $ ls -alh /home/mhl/ubuntu.lime - -r--r--r-- 1 root root 2.0G Aug 17 19:37 /home/mhl/ubuntu.lime + $ sudo insmod lime-3.2.0-23-generic.ko "path=/tmp/ubuntu.lime format=lime" + $ ls -alh /tmp/ubuntu.lime + -r--r--r-- 1 root root 2.0G Aug 17 19:37 /tmp/ubuntu.lime Procedure to create symbol tables for linux -------------------------------------------- From fc48e7b83f59320642952466e40e515b242aa0ec Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Tue, 5 Jul 2022 19:33:33 +0530 Subject: [PATCH 17/67] regarding ISF server its moved to tips section --- doc/source/Linux.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/doc/source/Linux.rst b/doc/source/Linux.rst index 3c77a16b6..21b8659fa 100644 --- a/doc/source/Linux.rst +++ b/doc/source/Linux.rst @@ -27,8 +27,9 @@ Procedure to create symbol tables for linux -------------------------------------------- To create a symbol table please refer this :ref:`symbol-tables:Mac or Linux symbol tables`. -We can also find some ISF files from `Linux ISF Server `_ , which is built and maintained by `kevthehermit `_. -After creating the file or downloading the file from the ISF server, please place the file under the directory ``volatility3/symbols/linux``. Make a directory linux under symbols. + +.. tip:: We can also find some ISF files from `Linux ISF Server `_ , which is built and maintained by `kevthehermit `_. + After creating the file or downloading the file from the ISF server, please place the file under the directory ``volatility3/symbols/linux``. Make a directory linux under symbols. Listing plugins From 89f1116374a1bd2e57696d2109fc7a9475d43c5c Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Tue, 5 Jul 2022 19:36:58 +0530 Subject: [PATCH 18/67] Sentence reframed and clarrified regarding sample plugin list --- doc/source/Linux.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/doc/source/Linux.rst b/doc/source/Linux.rst index 21b8659fa..82b42b251 100644 --- a/doc/source/Linux.rst +++ b/doc/source/Linux.rst @@ -35,7 +35,7 @@ To create a symbol table please refer this :ref:`symbol-tables:Mac or Linux symb Listing plugins --------------- -Following are the list of linux plugins available for volatility3. More plugins will be available on future releases. +Following are the sample of linux plugins available for volatility3. More plugins will be available on future releases. For plugin requests, Please create an issue with description of the plugin. .. code-block:: shell-session @@ -47,6 +47,8 @@ For plugin requests, Please create an issue with description of the plugin. linux.check_creds.Check_creds linux.check_idt.Check_idt +.. note:: Here the the command is piped to grep and head in-order to give you sample list of plugins. + Using plugins ------------- From 3da043c60192a706e82b115952065b84d892de9e Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Tue, 5 Jul 2022 19:39:08 +0530 Subject: [PATCH 19/67] Command syntax angular bracket added --- doc/source/Linux.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/source/Linux.rst b/doc/source/Linux.rst index 82b42b251..bf6b28be9 100644 --- a/doc/source/Linux.rst +++ b/doc/source/Linux.rst @@ -57,7 +57,7 @@ The following is the syntax to run volatility tool. .. code-block:: shell-session - $ python3 vol.py -f plugin_name plugin_option + $ python3 vol.py -f Example From ed361893c7e3b2a055d7d5c5785102da453e3f4c Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Tue, 5 Jul 2022 19:40:43 +0530 Subject: [PATCH 20/67] command fix vol.py to python3 vol.py --- doc/source/Linux.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/source/Linux.rst b/doc/source/Linux.rst index bf6b28be9..8919a83ce 100644 --- a/doc/source/Linux.rst +++ b/doc/source/Linux.rst @@ -40,7 +40,7 @@ For plugin requests, Please create an issue with description of the plugin. .. code-block:: shell-session - $ vol3 --help | grep -i linux. | head -n 5 + $ python3 vol.py --help | grep -i linux. | head -n 5 banners.Banners Attempts to identify potential linux banners in an linux.bash.Bash Recovers bash command history from memory. linux.check_afinfo.Check_afinfo From 20b76830f7ccefcb6b2d01dd6c0423083891a9a1 Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Tue, 5 Jul 2022 19:42:49 +0530 Subject: [PATCH 21/67] Removed external link to memory dump --- doc/source/Linux.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/doc/source/Linux.rst b/doc/source/Linux.rst index 8919a83ce..773aa80e4 100644 --- a/doc/source/Linux.rst +++ b/doc/source/Linux.rst @@ -66,8 +66,7 @@ Example Example 1 ~~~~~~~~~ -In this example we will be using memory dump from Insomni'hack teaser 2020 CTF. Challenge name Getdents, you can find the memory dump -in the link `here `_ . We will limit the discussion to memory forensics with volatility3 and not extend to other parts of the challenges. +In this example we will be using memory dump from Insomni'hack teaser 2020 CTF. Challenge name Getdents. We will limit the discussion to memory forensics with volatility3 and not extend to other parts of the challenges. I'd like to say thanks to `stuxnet `_ for providing this memory dump and `writeup `_. From 0844929610893ca5e239037c1b1f2031316acbb2 Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Tue, 5 Jul 2022 19:45:13 +0530 Subject: [PATCH 22/67] Use same voltility3 version in documentation volatility3 2.0.1 --- doc/source/Linux.rst | 8 ++++---- doc/source/Windows.rst | 4 ++-- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/doc/source/Linux.rst b/doc/source/Linux.rst index 773aa80e4..8132453ce 100644 --- a/doc/source/Linux.rst +++ b/doc/source/Linux.rst @@ -74,7 +74,7 @@ I'd like to say thanks to `stuxnet `_ for provid $ python3 vol.py -f memory.vmem banners - Volatility 3 Framework 2.0.3 + Volatility 3 Framework 2.0.1 Progress: 100.00 PDB scanning finished Offset Banner @@ -96,7 +96,7 @@ If you do not find the ISF file then, please follow the instructions on :ref:`Li $ python3 vol.py -f memory.vmem linux.pslist - Volatility 3 Framework 2.0.3 Stacking attempts finished + Volatility 3 Framework 2.0.1 Stacking attempts finished PID PPID COMM @@ -123,7 +123,7 @@ If you do not find the ISF file then, please follow the instructions on :ref:`Li .. code-block:: shell-session $ python3 vol.py -f memory.vmem linux.pstree - Volatility 3 Framework 2.0.3 + Volatility 3 Framework 2.0.1 Progress: 100.00 Stacking attempts finished PID PPID COMM @@ -167,7 +167,7 @@ Now to find the commands ran in bash shell. Lets use ``linux.bash``. $ python3 vol.py -f memory.vmem linux.bash - Volatility 3 Framework 2.0.3 + Volatility 3 Framework 2.0.1 Progress: 100.00 Stacking attempts finished PID Process CommandTime Command diff --git a/doc/source/Windows.rst b/doc/source/Windows.rst index 3e6844f22..e722be0a3 100644 --- a/doc/source/Windows.rst +++ b/doc/source/Windows.rst @@ -45,7 +45,7 @@ In windows memory forensics using volatility3, most of the times we do not requi $ python3 vol.py -f MemDump.DMP windows.pslist | head -n 10 - Volatility 3 Framework 2.0.2 PDB scanning finished + Volatility 3 Framework 2.0.1 PDB scanning finished PID PPID ImageFileName Offset(V) Threads Handles SessionId Wow64 CreateTime ExitTime File output @@ -61,7 +61,7 @@ In windows memory forensics using volatility3, most of the times we do not requi .. code-block:: shell-session $ python3 vol.py -f MemDump.DMP windows.pstree | head -n 20 - Volatility 3 Framework 2.0.2 PDB scanning finished + Volatility 3 Framework 2.0.1 PDB scanning finished PID PPID ImageFileName Offset(V) Threads Handles SessionId Wow64 CreateTime ExitTime From 1ccd31b506768ef5e9201d5722aeff59be309919 Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Tue, 5 Jul 2022 19:47:47 +0530 Subject: [PATCH 23/67] Added note regarding pipe in windows doc and moved winPEM to tip --- doc/source/Windows.rst | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/doc/source/Windows.rst b/doc/source/Windows.rst index e722be0a3..2f8d58a04 100644 --- a/doc/source/Windows.rst +++ b/doc/source/Windows.rst @@ -6,7 +6,9 @@ This guide gives you a brief introduction to how volatility3 works and some demo Acquiring memory ---------------- -Volatility does not provide the ability to acquire memory. In this tutorial we will see how we can use `WinPmem `_ for this purpose. +Volatility does not provide the ability to acquire memory. + +.. tip:: You could use `WinPmem `_ for collecting windows memory dump. Listing Plugins --------------- @@ -20,6 +22,8 @@ Listing Plugins windows.dlllist.DllList Lists the loaded modules in a particular windows +.. note:: Here the the command is piped to grep and head in-order to give you sample list of plugins. + Using plugins ------------- From 73eec3386dc843761e37bfff6420e173cff891e8 Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Tue, 5 Jul 2022 19:48:49 +0530 Subject: [PATCH 24/67] Reference to memory dump removed --- doc/source/Windows.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/source/Windows.rst b/doc/source/Windows.rst index 2f8d58a04..a9e712fc9 100644 --- a/doc/source/Windows.rst +++ b/doc/source/Windows.rst @@ -40,7 +40,7 @@ Example Example 1 ~~~~~~~~~ -In this example we will be using memory dump from PragyanCTF'22. The dump is available `here `_. +In this example we will be using memory dump from PragyanCTF'22. We will limit the discussion to memory forensics with volatility3 and not extend to other parts of the challenges. In windows memory forensics using volatility3, most of the times we do not require creating a ISF file. From bcc923b1b92567c66f3e96c5996084fdff892895 Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Tue, 5 Jul 2022 19:50:33 +0530 Subject: [PATCH 25/67] Info regarding pipe added --- doc/source/Windows.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/doc/source/Windows.rst b/doc/source/Windows.rst index a9e712fc9..d26f41aa8 100644 --- a/doc/source/Windows.rst +++ b/doc/source/Windows.rst @@ -88,6 +88,9 @@ In windows memory forensics using volatility3, most of the times we do not requi ``windows.pstree`` helps us to display the parent child relation of processes. +.. note:: Here the the command is piped to head in-order to give you smaller output of process here top 20. + + .. code-block:: shell-session $ python3 vol.py -f MemDump.DMP windows.hashdump From 596047c251a6363e49656e79284a56e974b0c8a3 Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Tue, 5 Jul 2022 19:51:32 +0530 Subject: [PATCH 26/67] small adjustment made in note regarding pipe --- doc/source/Linux.rst | 2 +- doc/source/Windows.rst | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/source/Linux.rst b/doc/source/Linux.rst index 8132453ce..b5a3db09c 100644 --- a/doc/source/Linux.rst +++ b/doc/source/Linux.rst @@ -47,7 +47,7 @@ For plugin requests, Please create an issue with description of the plugin. linux.check_creds.Check_creds linux.check_idt.Check_idt -.. note:: Here the the command is piped to grep and head in-order to give you sample list of plugins. +.. note:: Here the the command is piped to grep and head in-order to give you sample list of linux plugins. Using plugins diff --git a/doc/source/Windows.rst b/doc/source/Windows.rst index d26f41aa8..55677a67c 100644 --- a/doc/source/Windows.rst +++ b/doc/source/Windows.rst @@ -22,7 +22,7 @@ Listing Plugins windows.dlllist.DllList Lists the loaded modules in a particular windows -.. note:: Here the the command is piped to grep and head in-order to give you sample list of plugins. +.. note:: Here the the command is piped to grep and head in-order to give you sample list of windows plugins. Using plugins ------------- From e7b33f6c841b250db1a1014d2acd412f456705df Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Tue, 5 Jul 2022 19:54:37 +0530 Subject: [PATCH 27/67] Description on listing plugins in windows added --- doc/source/Windows.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/doc/source/Windows.rst b/doc/source/Windows.rst index 55677a67c..a6c67780e 100644 --- a/doc/source/Windows.rst +++ b/doc/source/Windows.rst @@ -13,6 +13,9 @@ Volatility does not provide the ability to acquire memory. Listing Plugins --------------- +Following are the sample of linux plugins available for volatility3. More plugins will be available on future releases. +For plugin requests, Please create an issue with description of the plugin. + .. code-block:: shell-session $ python3 vol.py --help | grep windows | head -n 5 From 3714fa8c9254c6a29bc2657788521de285baf6f3 Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Tue, 5 Jul 2022 19:56:45 +0530 Subject: [PATCH 28/67] Note regarding using sudo added --- doc/source/Linux.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/source/Linux.rst b/doc/source/Linux.rst index b5a3db09c..180e7c697 100644 --- a/doc/source/Linux.rst +++ b/doc/source/Linux.rst @@ -23,6 +23,8 @@ It also supports capture from Android devices. See below for example commands bu $ ls -alh /tmp/ubuntu.lime -r--r--r-- 1 root root 2.0G Aug 17 19:37 /tmp/ubuntu.lime +.. note:: The above command required sudo inorder to access the files which are root only. + Procedure to create symbol tables for linux -------------------------------------------- From 99672cbe759f22ff0518014a8c93d05bc2d188ca Mon Sep 17 00:00:00 2001 From: Paul Kermann Date: Mon, 15 Aug 2022 15:27:19 +0300 Subject: [PATCH 29/67] improv commit --- volatility3/framework/symbols/windows/pdbutil.py | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/volatility3/framework/symbols/windows/pdbutil.py b/volatility3/framework/symbols/windows/pdbutil.py index 430ad6a30..9f83ad973 100644 --- a/volatility3/framework/symbols/windows/pdbutil.py +++ b/volatility3/framework/symbols/windows/pdbutil.py @@ -253,7 +253,8 @@ class PDBUtility(interfaces.configuration.VersionableInterface): pdb_names: List[bytes], progress_callback: constants.ProgressCallback = None, start: Optional[int] = None, - end: Optional[int] = None) -> Generator[Dict[str, Optional[Union[bytes, str, int]]], None, None]: + end: Optional[int] = None, + maximum_invalid_count: int = 100) -> Generator[Dict[str, Optional[Union[bytes, str, int]]], None, None]: """Scans through `layer_name` at `ctx` looking for RSDS headers that indicate one of four common pdb kernel names (as listed in `self.pdb_names`) and returns the tuple (GUID, age, pdb_name, @@ -278,10 +279,15 @@ class PDBUtility(interfaces.configuration.VersionableInterface): sections = [(start, end - start)]): mz_offset = None sig_pfn = signature_offset // page_size + current_invalid_counter = 0 for i in range(sig_pfn, min_pfn, -1): - if not ctx.layers[layer_name].is_valid(i * page_size, 2): + if current_invalid_counter > maximum_invalid_count: break + + if not ctx.layers[layer_name].is_valid(i * page_size, 2): + current_invalid_counter += 1 + continue data = ctx.layers[layer_name].read(i * page_size, 2) if data == b'MZ': From afb17dfac8ef35950803e549bb127f920ae7eef0 Mon Sep 17 00:00:00 2001 From: Paul Kermann Date: Wed, 24 Aug 2022 14:14:50 +0300 Subject: [PATCH 30/67] use the maximum_invalid_count --- volatility3/framework/automagic/pdbscan.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/volatility3/framework/automagic/pdbscan.py b/volatility3/framework/automagic/pdbscan.py index 5cbdbfe0e..0cb01485e 100644 --- a/volatility3/framework/automagic/pdbscan.py +++ b/volatility3/framework/automagic/pdbscan.py @@ -212,7 +212,8 @@ class KernelPDBScanner(interfaces.automagic.AutomagicInterface): start = start_scan_address, page_size = vlayer.page_size, pdb_names = kernel_pdb_names, - progress_callback = progress_callback) + progress_callback = progress_callback, + maximum_invalid_count = constants.windows.PE_MAX_EXTRACTION_SIZE // 0x1000) for kernel in kernels: valid_kernel = test_kernel(physical_layer_name, virtual_layer_name, kernel) if valid_kernel is not None: From 26d15a3ad5069a99702c2d672d8d10c68f64f0b6 Mon Sep 17 00:00:00 2001 From: Paul Kermann Date: Wed, 31 Aug 2022 09:40:22 +0300 Subject: [PATCH 31/67] code review --- volatility3/framework/automagic/pdbscan.py | 3 +-- volatility3/framework/symbols/windows/pdbutil.py | 8 ++++++++ 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/volatility3/framework/automagic/pdbscan.py b/volatility3/framework/automagic/pdbscan.py index 0cb01485e..5cbdbfe0e 100644 --- a/volatility3/framework/automagic/pdbscan.py +++ b/volatility3/framework/automagic/pdbscan.py @@ -212,8 +212,7 @@ class KernelPDBScanner(interfaces.automagic.AutomagicInterface): start = start_scan_address, page_size = vlayer.page_size, pdb_names = kernel_pdb_names, - progress_callback = progress_callback, - maximum_invalid_count = constants.windows.PE_MAX_EXTRACTION_SIZE // 0x1000) + progress_callback = progress_callback) for kernel in kernels: valid_kernel = test_kernel(physical_layer_name, virtual_layer_name, kernel) if valid_kernel is not None: diff --git a/volatility3/framework/symbols/windows/pdbutil.py b/volatility3/framework/symbols/windows/pdbutil.py index 9f83ad973..311cc6451 100644 --- a/volatility3/framework/symbols/windows/pdbutil.py +++ b/volatility3/framework/symbols/windows/pdbutil.py @@ -264,6 +264,14 @@ class PDBUtility(interfaces.configuration.VersionableInterface): The UI should always provide the user an opportunity to specify the appropriate types and PDB values themselves + Args: + layer_name: The layer name to scan + page_size: Size of page constant + pdb_names: List of pdb names to scan + progress_callback: Means of providing the user with feedback during long processes + start: Start address to start scanning from the pdb_names + end: Minimum address to scan the pdb_names + maximum_invalid_count: Amount of pages that can be invalid during scanning before aborting signature search """ min_pfn = 0 From 2a3212e77b74d0ab0d8c3aeaebea3d0a55866a29 Mon Sep 17 00:00:00 2001 From: Paul Kermann Date: Wed, 31 Aug 2022 10:26:38 +0300 Subject: [PATCH 32/67] remove whitespace --- volatility3/framework/symbols/windows/pdbutil.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/volatility3/framework/symbols/windows/pdbutil.py b/volatility3/framework/symbols/windows/pdbutil.py index 311cc6451..8b58442cc 100644 --- a/volatility3/framework/symbols/windows/pdbutil.py +++ b/volatility3/framework/symbols/windows/pdbutil.py @@ -292,7 +292,7 @@ class PDBUtility(interfaces.configuration.VersionableInterface): for i in range(sig_pfn, min_pfn, -1): if current_invalid_counter > maximum_invalid_count: break - + if not ctx.layers[layer_name].is_valid(i * page_size, 2): current_invalid_counter += 1 continue From 3523985d0a7123f2cf4648568a7e41865c9edd57 Mon Sep 17 00:00:00 2001 From: Donghyun Kim Date: Fri, 23 Sep 2022 07:47:03 +0900 Subject: [PATCH 33/67] Fix: to find_namepsace_packages method --- setup.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/setup.py b/setup.py index f6bb687f2..a4bd3fffe 100644 --- a/setup.py +++ b/setup.py @@ -40,7 +40,7 @@ setuptools.setup(name = "volatility3", '': ['development', 'development.*'], 'development': ['*'] }, - packages = setuptools.find_packages(exclude = ["development", "development.*"]), + packages = setuptools.find_namespace_packages(exclude = ["development", "development.*"]), entry_points = { 'console_scripts': [ 'vol = volatility3.cli:main', From 949b15a36812d3f4ad33cf55e3e8fb387a0d7cee Mon Sep 17 00:00:00 2001 From: Donghyun Kim Date: Fri, 23 Sep 2022 08:01:11 +0900 Subject: [PATCH 34/67] Fix: unsused module for objects initialize code --- volatility3/framework/objects/__init__.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/volatility3/framework/objects/__init__.py b/volatility3/framework/objects/__init__.py index 62e6de553..eedd22bb2 100644 --- a/volatility3/framework/objects/__init__.py +++ b/volatility3/framework/objects/__init__.py @@ -9,7 +9,7 @@ import struct from typing import Any, ClassVar, Dict, Iterable, List, Optional, Tuple, Type, Union as TUnion, overload from volatility3.framework import constants, interfaces -from volatility3.framework.objects import templates, utility +from volatility3.framework.objects import templates vollog = logging.getLogger(__name__) From 1ffe9f222f984512a2441ee65e7317b7b4953531 Mon Sep 17 00:00:00 2001 From: a5hlynx Date: Fri, 7 Oct 2022 00:57:20 +0900 Subject: [PATCH 35/67] correct ImageFileName --- volatility3/framework/plugins/windows/handles.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/volatility3/framework/plugins/windows/handles.py b/volatility3/framework/plugins/windows/handles.py index ab11d30d6..bdff88075 100644 --- a/volatility3/framework/plugins/windows/handles.py +++ b/volatility3/framework/plugins/windows/handles.py @@ -323,7 +323,7 @@ class Handles(interfaces.plugins.PluginInterface): obj_name = item.file_name_with_device() elif obj_type == "Process": item = entry.Body.cast("_EPROCESS") - obj_name = f"{utility.array_to_string(proc.ImageFileName)} Pid {item.UniqueProcessId}" + obj_name = f"{utility.array_to_string(item.ImageFileName)} Pid {item.UniqueProcessId}" elif obj_type == "Thread": item = entry.Body.cast("_ETHREAD") obj_name = f"Tid {item.Cid.UniqueThread} Pid {item.Cid.UniqueProcess}" From 146afc0f0786a9e849e480b54e85562f9a2a19a1 Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Sun, 16 Oct 2022 08:07:43 +0530 Subject: [PATCH 36/67] Incomplete sentence - fixedf --- doc/source/Linux.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/source/Linux.rst b/doc/source/Linux.rst index 180e7c697..223c9fc07 100644 --- a/doc/source/Linux.rst +++ b/doc/source/Linux.rst @@ -1,7 +1,7 @@ Linux Tutorial ============== -This guide gives you a brief introduction to how volatility3 works and some demonstration of several of the plugins available from +This guide will give you a brief overview of how volatility3 works as well as a demonstration of several of the plugins available in the suite of plugins. Acquiring memory ---------------- From 95e4078b77fd802147b5b3d4662ca8734ef2d6df Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Sun, 16 Oct 2022 08:25:35 +0530 Subject: [PATCH 37/67] Added FTK as another example to avoid favouritism --- doc/source/Windows.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/doc/source/Windows.rst b/doc/source/Windows.rst index a6c67780e..d732e0cdc 100644 --- a/doc/source/Windows.rst +++ b/doc/source/Windows.rst @@ -8,7 +8,8 @@ Acquiring memory Volatility does not provide the ability to acquire memory. -.. tip:: You could use `WinPmem `_ for collecting windows memory dump. +.. tip:: - You could use `WinPmem `_ for collecting windows memory dump. + - You could also use `FTK Imager `_ Listing Plugins --------------- From fba734b284e0bed5f5cb7b4ac91c3194ea8bab2b Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Sun, 16 Oct 2022 19:10:55 +0530 Subject: [PATCH 38/67] AVML added. Restructured Acquiring Memory. --- doc/source/Linux.rst | 7 ++++--- doc/source/Windows.rst | 6 +++--- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/doc/source/Linux.rst b/doc/source/Linux.rst index 223c9fc07..9fb5a686e 100644 --- a/doc/source/Linux.rst +++ b/doc/source/Linux.rst @@ -6,9 +6,10 @@ This guide will give you a brief overview of how volatility3 works as well as a Acquiring memory ---------------- -Volatility3 does not provide the ability to acquire memory. In this tutorial we will see how we can use `LiME `_ for this purpose. -It supports 32 and 64 bit captures from native Intel hardware as well as virtual machine guests. -It also supports capture from Android devices. See below for example commands building and running LiME: +Volatility3 does not provide the ability to acquire memory. +You can use any of the following tools to Acquire memory or the ones you are convenient with: + - `AVML - Acquire Volatile Memory for Linux `_ + - `LIME - Linux Memory Extract `_ .. code-block:: shell-session diff --git a/doc/source/Windows.rst b/doc/source/Windows.rst index d732e0cdc..80bc6ddc2 100644 --- a/doc/source/Windows.rst +++ b/doc/source/Windows.rst @@ -7,9 +7,9 @@ Acquiring memory ---------------- Volatility does not provide the ability to acquire memory. - -.. tip:: - You could use `WinPmem `_ for collecting windows memory dump. - - You could also use `FTK Imager `_ +You can use any of the following tools to Acquire memory or the ones you are convenient with: + - `WinPmem `_ + - `FTK Imager `_ Listing Plugins --------------- From 9d9eb226ab64aeefaec82df217ddd478859b340d Mon Sep 17 00:00:00 2001 From: Tejas <47889755+tejas15802@users.noreply.github.com> Date: Sun, 16 Oct 2022 19:21:42 +0530 Subject: [PATCH 39/67] Removed the commands which were present for LIME --- doc/source/Linux.rst | 14 -------------- 1 file changed, 14 deletions(-) diff --git a/doc/source/Linux.rst b/doc/source/Linux.rst index 9fb5a686e..ea6c2223c 100644 --- a/doc/source/Linux.rst +++ b/doc/source/Linux.rst @@ -11,20 +11,6 @@ You can use any of the following tools to Acquire memory or the ones you are con - `AVML - Acquire Volatile Memory for Linux `_ - `LIME - Linux Memory Extract `_ -.. code-block:: shell-session - - $ tar -xvzf lime-forensics-1.1-r14.tar.gz - $ cd lime-forensics-1.1-r14/src - $ make - .... - CC [M] lime-forensics-1.1-r14/src/tcp.o - CC [M] lime-forensics-1.1-r14/src/disk.o - .... - $ sudo insmod lime-3.2.0-23-generic.ko "path=/tmp/ubuntu.lime format=lime" - $ ls -alh /tmp/ubuntu.lime - -r--r--r-- 1 root root 2.0G Aug 17 19:37 /tmp/ubuntu.lime - -.. note:: The above command required sudo inorder to access the files which are root only. Procedure to create symbol tables for linux -------------------------------------------- From b71e367d387ed13a083bde17cb7a586a1c28cf67 Mon Sep 17 00:00:00 2001 From: Mike Auty Date: Sun, 16 Oct 2022 15:50:22 +0100 Subject: [PATCH 40/67] Documentation: Rename, fix grammar and avoid using personal pronouns --- ...rst => getting-started-linux-tutorial.rst} | 51 +++++++++++-------- ...t => getting-started-windows-tutorial.rst} | 43 +++++++++------- doc/source/index.rst | 18 +++---- doc/source/symbol-tables.rst | 2 +- 4 files changed, 66 insertions(+), 48 deletions(-) rename doc/source/{Linux.rst => getting-started-linux-tutorial.rst} (71%) rename doc/source/{Windows.rst => getting-started-windows-tutorial.rst} (71%) diff --git a/doc/source/Linux.rst b/doc/source/getting-started-linux-tutorial.rst similarity index 71% rename from doc/source/Linux.rst rename to doc/source/getting-started-linux-tutorial.rst index ea6c2223c..15a1f0d1b 100644 --- a/doc/source/Linux.rst +++ b/doc/source/getting-started-linux-tutorial.rst @@ -1,31 +1,34 @@ Linux Tutorial ============== -This guide will give you a brief overview of how volatility3 works as well as a demonstration of several of the plugins available in the suite of plugins. +This guide will give you a brief overview of how volatility3 works as well as a demonstration of several of the plugins available in the suite. Acquiring memory ---------------- -Volatility3 does not provide the ability to acquire memory. -You can use any of the following tools to Acquire memory or the ones you are convenient with: - - `AVML - Acquire Volatile Memory for Linux `_ - - `LIME - Linux Memory Extract `_ +Volatility3 does not provide the ability to acquire memory. Below are some examples of tools that can be used to acquire memory, but more are available: + +* `AVML - Acquire Volatile Memory for Linux `_ +* `LIME - Linux Memory Extract `_ Procedure to create symbol tables for linux -------------------------------------------- -To create a symbol table please refer this :ref:`symbol-tables:Mac or Linux symbol tables`. +To create a symbol table please refer to :ref:`symbol-tables:Mac or Linux symbol tables`. -.. tip:: We can also find some ISF files from `Linux ISF Server `_ , which is built and maintained by `kevthehermit `_. - After creating the file or downloading the file from the ISF server, please place the file under the directory ``volatility3/symbols/linux``. Make a directory linux under symbols. +.. tip:: It may be possible to locate pre-made ISF files from the `Linux ISF Server `_ , + which is built and maintained by `kevthehermit `_. + After creating the file or downloading it from the ISF server, place the file under the directory ``volatility3/symbols/linux``. + If necessary create a linux directory under the symbols directory (this will become unncessary in future versions). Listing plugins --------------- -Following are the sample of linux plugins available for volatility3. More plugins will be available on future releases. -For plugin requests, Please create an issue with description of the plugin. +The following is a sample of the linux plugins available for volatility3, it is not complete and more more plugins may +be added. For a complete reference, please see the volatility 3 :doc:`list of plugins `. +For plugin requests, please create an issue with a description of the requested plugin. .. code-block:: shell-session @@ -36,13 +39,13 @@ For plugin requests, Please create an issue with description of the plugin. linux.check_creds.Check_creds linux.check_idt.Check_idt -.. note:: Here the the command is piped to grep and head in-order to give you sample list of linux plugins. +.. note:: Here the the command is piped to grep and head in-order to provide the start of the list of linux plugins. Using plugins ------------- -The following is the syntax to run volatility tool. +The following is the syntax to run the volatility CLI. .. code-block:: shell-session @@ -52,11 +55,11 @@ The following is the syntax to run volatility tool. Example ------- -Example 1 -~~~~~~~~~ +banners +~~~~~~~ -In this example we will be using memory dump from Insomni'hack teaser 2020 CTF. Challenge name Getdents. We will limit the discussion to memory forensics with volatility3 and not extend to other parts of the challenges. -I'd like to say thanks to `stuxnet `_ for providing this memory dump and `writeup `_. +In this example we will be using a memory dump from the Insomni'hack teaser 2020 CTF Challenge called Getdents. We will limit the discussion to memory forensics with volatility 3 and not extend it to other parts of the challenge. +Thanks go to `stuxnet `_ for providing this memory dump and `writeup `_. .. code-block:: shell-session @@ -75,11 +78,13 @@ I'd like to say thanks to `stuxnet `_ for provid 0x7fde0010 Linux version 4.15.0-72-generic (buildd@lcy01-amd64-026) (gcc version 7.4.0 (Ubuntu 7.4.0-1ubuntu1~18.04.1)) #81-Ubuntu SMP Tue Nov 26 12:20:02 UTC 2019 (Ubuntu 4.15.0-72.81-generic 4.15.18) -This above command helps us to find the memory dump's kernel version and the distribution version. Now using the above banner we can search for ISF file from the ISF server. -If you do not find the ISF file then, please follow the instructions on :ref:`Linux:Procedure to create symbol tables for linux`. After that place the ISF file under ``volatility3/symbols/linux`` directory. +The above command helps us to find the memory dump's kernel version and the distribution version. Now using the above banner we can search for the needed ISF file from the ISF server. +If ISF file cannt be found then, follow the instructions on :ref:`getting-started-linux-tutorial:Procedure to create symbol tables for linux`. After that, place the ISF file under the ``volatility3/symbols/linux`` directory. .. tip:: Use the banner text which is most repeated to search from ISF Server. +linux.pslist +~~~~~~~~~~~~ .. code-block:: shell-session @@ -109,6 +114,9 @@ If you do not find the ISF file then, please follow the instructions on :ref:`Li ``linux.pslist`` helps us to list the processes which are running, their PIDs and PPIDs. +linux.pstree +~~~~~~~~~~~~ + .. code-block:: shell-session $ python3 vol.py -f memory.vmem linux.pstree @@ -148,9 +156,12 @@ If you do not find the ISF file then, please follow the instructions on :ref:`Li ***** 1548 1266 gsd-keyboard ***** 1550 1266 gsd-media-keys -``linux.pstree`` helps us to display the parent child relation of processes. +``linux.pstree`` helps us to display the parent child relationships between processes. -Now to find the commands ran in bash shell. Lets use ``linux.bash``. +linux.bash +~~~~~~~~~~ + +Now to find the commands that were run in the bash shell by using ``linux.bash``. .. code-block:: shell-session diff --git a/doc/source/Windows.rst b/doc/source/getting-started-windows-tutorial.rst similarity index 71% rename from doc/source/Windows.rst rename to doc/source/getting-started-windows-tutorial.rst index 80bc6ddc2..c89b065f5 100644 --- a/doc/source/Windows.rst +++ b/doc/source/getting-started-windows-tutorial.rst @@ -1,21 +1,23 @@ Windows Tutorial ================ -This guide gives you a brief introduction to how volatility3 works and some demonstration on suite of plugins available from +This guide provides a brief introduction to how volatility3 works as a demonstration of several of the plugins available in the suite. Acquiring memory ---------------- Volatility does not provide the ability to acquire memory. -You can use any of the following tools to Acquire memory or the ones you are convenient with: - - `WinPmem `_ - - `FTK Imager `_ +Memory can be acquired using a number of tools, below are some examples but others exist: + +* `WinPmem `_ +* `FTK Imager `_ Listing Plugins --------------- -Following are the sample of linux plugins available for volatility3. More plugins will be available on future releases. -For plugin requests, Please create an issue with description of the plugin. +The following is a sample of the windows plugins available for volatility3, it is not complete and more more plugins may +be added. For a complete reference, please see the volatility 3 :doc:`list of plugins `. +For plugin requests, please create an issue with a description of the requested plugin. .. code-block:: shell-session @@ -24,14 +26,13 @@ For plugin requests, Please create an issue with description of the plugin. windows.cmdline.CmdLine windows.crashinfo.Crashinfo windows.dlllist.DllList - Lists the loaded modules in a particular windows -.. note:: Here the the command is piped to grep and head in-order to give you sample list of windows plugins. +.. note:: Here the the command is piped to grep and head in-order to provide the start of a list of the available windows plugins. Using plugins ------------- -The following is the syntax to run volatility tool. +The following is the syntax to run the volatility CLI. .. code-block:: shell-session @@ -41,13 +42,14 @@ The following is the syntax to run volatility tool. Example ------- -Example 1 -~~~~~~~~~ +windows.pslist +~~~~~~~~~~~~~~ -In this example we will be using memory dump from PragyanCTF'22. -We will limit the discussion to memory forensics with volatility3 and not extend to other parts of the challenges. +In this example we will be using a memory dump from the PragyanCTF'22. +We will limit the discussion to memory forensics with volatility 3 and not extend it to other parts of the challenges. -In windows memory forensics using volatility3, most of the times we do not require creating a ISF file. +When using windows plugins in volatility 3, the required ISF file can often be generated from PDB files automatically +downloaded from Microsoft servers, and therefore does not require locating or adding specific ISF files to the volatility 3 symbols directory. .. code-block:: shell-session @@ -64,7 +66,10 @@ In windows memory forensics using volatility3, most of the times we do not requi 412 396 csrss.exe 0xfa80021c5b30 9 224 1 False 2022-02-07 16:30:13.000000 N/A Disabled 468 396 winlogon.exe 0xfa8002284060 5 113 1 False 2022-02-07 16:30:14.000000 N/A Disabled -``windows.pslist`` helps us list the processes running while the memory dump was taken. +``windows.pslist`` helps list the processes running while the memory dump was taken. + +windows.pstree +~~~~~~~~~~~~~~ .. code-block:: shell-session @@ -90,10 +95,12 @@ In windows memory forensics using volatility3, most of the times we do not requi ** 616 504 svchost.exe 0xfa8002b86ab0 13 314 0 False 2022-02-07 16:32:16.000000 N/A ** 624 504 svchost.exe 0xfa8002410630 10 350 0 False 2022-02-07 16:30:14.000000 N/A -``windows.pstree`` helps us to display the parent child relation of processes. +``windows.pstree`` helps to display the parent child relationships between processes. -.. note:: Here the the command is piped to head in-order to give you smaller output of process here top 20. +.. note:: Here the the command is piped to head in-order to provide smaller output, here listing only the first 20. +windows.hashdump +~~~~~~~~~~~~~~~~ .. code-block:: shell-session @@ -108,7 +115,7 @@ In windows memory forensics using volatility3, most of the times we do not requi HomeGroupUser$ 1002 aad3b435b51404eeaad3b435b51404ee af10ecac6ea817d2bb56e3e5c33ce1cd Dennis 1003 aad3b435b51404eeaad3b435b51404ee cf96684bbc7877920adaa9663698bf54 -``windows.hashdump`` helps us to list the hashes of the users in the system. +``windows.hashdump`` helps to list the hashes of the users in the system. diff --git a/doc/source/index.rst b/doc/source/index.rst index 0d35b02ba..9b1d05858 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -7,7 +7,7 @@ Volatility 3 is Open Source. :doc:`List of plugins ` -Here are some guidelines for using Volatility 3 effectively: +Below is the main documentation regarding volatility 3: .. toctree:: :caption: Documentation @@ -19,6 +19,14 @@ Here are some guidelines for using Volatility 3 effectively: volshell glossary +There is also some information to get you started quickly: + +.. toctree:: + :caption: Getting Started + + getting-started-linux-tutorial + getting-started-windows-tutorial + .. toctree:: :caption: Python Packages @@ -26,14 +34,6 @@ Here are some guidelines for using Volatility 3 effectively: volatility3 -.. toctree:: - :caption: Getting Started - - FAQ - Installation - Linux - Windows - Indices and tables ================== diff --git a/doc/source/symbol-tables.rst b/doc/source/symbol-tables.rst index d912d4906..b7c26e046 100644 --- a/doc/source/symbol-tables.rst +++ b/doc/source/symbol-tables.rst @@ -41,7 +41,7 @@ The :envvar:`PYTHONPATH` environment variable is not required if the Volatility or a virtual environment. Mac or Linux symbol tables ------------------------ +-------------------------- For Mac/Linux systems, both use the same mechanism for identification. The generated files contain an identifying string (the operating system banner), which Volatility's automagic can detect. Volatility caches the mapping between the strings and the symbol From 537f6a6a55b830534af5715fd8bd659111188b54 Mon Sep 17 00:00:00 2001 From: Mike Auty Date: Sun, 16 Oct 2022 15:55:45 +0100 Subject: [PATCH 41/67] Documentation: Fix minor typo --- doc/source/getting-started-linux-tutorial.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/source/getting-started-linux-tutorial.rst b/doc/source/getting-started-linux-tutorial.rst index 15a1f0d1b..e1c671c36 100644 --- a/doc/source/getting-started-linux-tutorial.rst +++ b/doc/source/getting-started-linux-tutorial.rst @@ -9,7 +9,7 @@ Acquiring memory Volatility3 does not provide the ability to acquire memory. Below are some examples of tools that can be used to acquire memory, but more are available: * `AVML - Acquire Volatile Memory for Linux `_ -* `LIME - Linux Memory Extract `_ +* `LiME - Linux Memory Extract `_ Procedure to create symbol tables for linux From 439835a61d4ba3abaec3b94048350ce85585872f Mon Sep 17 00:00:00 2001 From: Donghyun Kim Date: Mon, 17 Oct 2022 04:50:58 +0900 Subject: [PATCH 42/67] Fix: typo for linux tutorial --- doc/source/getting-started-linux-tutorial.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/source/getting-started-linux-tutorial.rst b/doc/source/getting-started-linux-tutorial.rst index e1c671c36..6fd06bcf9 100644 --- a/doc/source/getting-started-linux-tutorial.rst +++ b/doc/source/getting-started-linux-tutorial.rst @@ -20,7 +20,7 @@ To create a symbol table please refer to :ref:`symbol-tables:Mac or Linux symbol .. tip:: It may be possible to locate pre-made ISF files from the `Linux ISF Server `_ , which is built and maintained by `kevthehermit `_. After creating the file or downloading it from the ISF server, place the file under the directory ``volatility3/symbols/linux``. - If necessary create a linux directory under the symbols directory (this will become unncessary in future versions). + If necessary create a linux directory under the symbols directory (this will become unnecessary in future versions). Listing plugins From 88e944192093281c833b1404c4592b51ac364c9f Mon Sep 17 00:00:00 2001 From: Donghyun Kim Date: Sat, 22 Oct 2022 18:20:22 +0900 Subject: [PATCH 43/67] Fix: typo for linux tutorial --- doc/source/getting-started-linux-tutorial.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/source/getting-started-linux-tutorial.rst b/doc/source/getting-started-linux-tutorial.rst index 6fd06bcf9..26ad2c2e4 100644 --- a/doc/source/getting-started-linux-tutorial.rst +++ b/doc/source/getting-started-linux-tutorial.rst @@ -79,7 +79,7 @@ Thanks go to `stuxnet `_ for providing this memo The above command helps us to find the memory dump's kernel version and the distribution version. Now using the above banner we can search for the needed ISF file from the ISF server. -If ISF file cannt be found then, follow the instructions on :ref:`getting-started-linux-tutorial:Procedure to create symbol tables for linux`. After that, place the ISF file under the ``volatility3/symbols/linux`` directory. +If ISF file cannot be found then, follow the instructions on :ref:`getting-started-linux-tutorial:Procedure to create symbol tables for linux`. After that, place the ISF file under the ``volatility3/symbols/linux`` directory. .. tip:: Use the banner text which is most repeated to search from ISF Server. From 94bb22d4bcc35cd355b31d873c8d54f42457f2ae Mon Sep 17 00:00:00 2001 From: Mike Auty Date: Sun, 23 Oct 2022 22:49:06 +0100 Subject: [PATCH 44/67] Automagic: Make cache period longer and configurable --- volatility3/framework/automagic/symbol_cache.py | 2 +- volatility3/framework/constants/__init__.py | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/volatility3/framework/automagic/symbol_cache.py b/volatility3/framework/automagic/symbol_cache.py index 1e0bba86e..30a4068b6 100644 --- a/volatility3/framework/automagic/symbol_cache.py +++ b/volatility3/framework/automagic/symbol_cache.py @@ -157,10 +157,10 @@ class SqliteCache(CacheManagerInterface): _required_framework_version = (2, 0, 0) _version = (1, 0, 0) - cache_period = '-3 days' def __init__(self, filename: str): super().__init__(filename) + self.cache_period = constants.SQLITE_CACHE_PERIOD try: self._database = self._connect_storage(filename) except sqlite3.DatabaseError: diff --git a/volatility3/framework/constants/__init__.py b/volatility3/framework/constants/__init__.py index e0083a539..4fd53a3eb 100644 --- a/volatility3/framework/constants/__init__.py +++ b/volatility3/framework/constants/__init__.py @@ -63,6 +63,9 @@ LOGLEVEL_VVVV = 6 CACHE_PATH = os.path.join(os.path.expanduser("~"), ".cache", "volatility3") """Default path to store cached data""" +SQLITE_CACHE_PERIOD = '-1 month' +"""SQLite time modifier for how long each item is valid in the cache for""" + if sys.platform == 'win32': CACHE_PATH = os.path.realpath(os.path.join(os.environ.get("APPDATA", os.path.expanduser("~")), "volatility3")) os.makedirs(CACHE_PATH, exist_ok = True) From aa0c2b6c744486bbb7135e754b47bf1dc60e7360 Mon Sep 17 00:00:00 2001 From: Mike Auty Date: Wed, 2 Nov 2022 20:53:00 +0000 Subject: [PATCH 45/67] Mac: Fix bug found by buildbot/npetroni due refactoring --- volatility3/framework/symbols/mac/extensions/__init__.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/volatility3/framework/symbols/mac/extensions/__init__.py b/volatility3/framework/symbols/mac/extensions/__init__.py index a66bfb534..92410704f 100644 --- a/volatility3/framework/symbols/mac/extensions/__init__.py +++ b/volatility3/framework/symbols/mac/extensions/__init__.py @@ -28,8 +28,11 @@ class proc(generic.GenericIntelProcess): if not isinstance(parent_layer, interfaces.layers.TranslationLayerInterface): raise TypeError("Parent layer is not a translation layer, unable to construct process layer") - with contextlib.suppress(exceptions.InvalidAddressException): + try: dtb = self.get_task().map.pmap.pm_cr3 + except exceptions.InvalidAddressException: + # Bail out because we couldn't find the DTB + return None if preferred_name is None: preferred_name = self.vol.layer_name + f"_Process{self.p_pid}" From d09f23a7d7a791c6e846f401de7f1168326e34ee Mon Sep 17 00:00:00 2001 From: Mike Auty Date: Wed, 2 Nov 2022 20:55:24 +0000 Subject: [PATCH 46/67] Mac: Fix additional possibility of failure from refactoring --- volatility3/framework/symbols/mac/extensions/__init__.py | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/volatility3/framework/symbols/mac/extensions/__init__.py b/volatility3/framework/symbols/mac/extensions/__init__.py index 92410704f..45dc1db70 100644 --- a/volatility3/framework/symbols/mac/extensions/__init__.py +++ b/volatility3/framework/symbols/mac/extensions/__init__.py @@ -41,10 +41,8 @@ class proc(generic.GenericIntelProcess): return self._add_process_layer(self._context, dtb, config_prefix, preferred_name) def get_map_iter(self) -> Iterable[interfaces.objects.ObjectInterface]: - with contextlib.suppress(exceptions.InvalidAddressException): - task = self.get_task() - try: + task = self.get_task() current_map = task.map.hdr.links.next except exceptions.InvalidAddressException: return From 0c80ae4f816281541e177017f9e2e1e518a78b3e Mon Sep 17 00:00:00 2001 From: Mike Auty Date: Wed, 2 Nov 2022 21:39:27 +0000 Subject: [PATCH 47/67] Automagic: Check file datetime to determine whether to recache --- .../framework/automagic/symbol_cache.py | 27 +++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-) diff --git a/volatility3/framework/automagic/symbol_cache.py b/volatility3/framework/automagic/symbol_cache.py index 30a4068b6..fe5dfac52 100644 --- a/volatility3/framework/automagic/symbol_cache.py +++ b/volatility3/framework/automagic/symbol_cache.py @@ -2,6 +2,7 @@ # which is available at https://www.volatilityfoundation.org/license/vsl-v1.0 # import base64 +import datetime import json import logging import os @@ -170,6 +171,7 @@ class SqliteCache(CacheManagerInterface): def _connect_storage(self, path: str) -> sqlite3.Connection: database = sqlite3.connect(path) database.row_factory = sqlite3.Row + database.cursor().execute( f'CREATE TABLE IF NOT EXISTS database_info (schema_version INT DEFAULT {constants.CACHE_SQLITE_SCHEMA_VERSION})') schema_version = database.cursor().execute('SELECT schema_version FROM database_info').fetchone() @@ -259,10 +261,31 @@ class SqliteCache(CacheManagerInterface): cache_update = set() files_to_timestamp = on_disk_locations.intersection(cached_locations) if files_to_timestamp: - result = self._database.cursor().execute("SELECT location FROM cache WHERE local = 1 " + result = self._database.cursor().execute("SELECT location, cached FROM cache WHERE local = 1 " f"AND cached < date('now', '{self.cache_period}');") for row in result: - if row['location'] in files_to_timestamp: + location = row['location'] + stored_timestamp = datetime.datetime.fromisoformat(row['cached']) + timestamp = stored_timestamp # Default to requiring update + + # See if the file is a local URL type we can handle: + parsed = urllib.parse.urlparse(location) + pathname = None + if parsed.scheme == 'file': + pathname = urllib.request.url2pathname(parsed.path) + if parsed.scheme == 'jar': + inner_url = urllib.parse.urlparse(parsed.path) + if inner_url.scheme == 'file': + pathname = inner_url.path.split('!')[0] + + if pathname: + timestamp = datetime.datetime.fromtimestamp(os.stat(pathname).st_mtime) + else: + vollog.log(constants.LOGLEVEL_VVVV, + "File location in database classed as local but not file/jar URL") + + # If we're supposed to include it, and our last check is older than (or equal to) the file timestamp + if row['location'] in files_to_timestamp and stored_timestamp < timestamp: cache_update.add(row['location']) idextractors = list(framework.class_subclasses(IdentifierProcessor)) From 5ac191b31008a1e678a77839cb2aed489310691a Mon Sep 17 00:00:00 2001 From: Mike Auty Date: Wed, 2 Nov 2022 21:43:18 +0000 Subject: [PATCH 48/67] Automagic: Set the cache period back to 3 days --- volatility3/framework/constants/__init__.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/volatility3/framework/constants/__init__.py b/volatility3/framework/constants/__init__.py index 4fd53a3eb..b19e80472 100644 --- a/volatility3/framework/constants/__init__.py +++ b/volatility3/framework/constants/__init__.py @@ -63,7 +63,7 @@ LOGLEVEL_VVVV = 6 CACHE_PATH = os.path.join(os.path.expanduser("~"), ".cache", "volatility3") """Default path to store cached data""" -SQLITE_CACHE_PERIOD = '-1 month' +SQLITE_CACHE_PERIOD = '-3 days' """SQLite time modifier for how long each item is valid in the cache for""" if sys.platform == 'win32': From 5bbec2d7c4e7330d065e2b3a6e08df964856945b Mon Sep 17 00:00:00 2001 From: Mike Auty Date: Mon, 7 Nov 2022 20:46:30 +0000 Subject: [PATCH 49/67] Core: Bump to 2.4.1 in preparation for 2.4.0 release --- volatility3/framework/constants/__init__.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/volatility3/framework/constants/__init__.py b/volatility3/framework/constants/__init__.py index b19e80472..95b365609 100644 --- a/volatility3/framework/constants/__init__.py +++ b/volatility3/framework/constants/__init__.py @@ -40,7 +40,7 @@ BANG = "!" # We use the SemVer 2.0.0 versioning scheme VERSION_MAJOR = 2 # Number of releases of the library with a breaking change VERSION_MINOR = 4 # Number of changes that only add to the interface -VERSION_PATCH = 0 # Number of changes that do not change the interface +VERSION_PATCH = 1 # Number of changes that do not change the interface VERSION_SUFFIX = "" # TODO: At version 2.0.0, remove the symbol_shift feature From e3c548686c59abadddfc135ba5c9f3ecd32adc9d Mon Sep 17 00:00:00 2001 From: ikelos Date: Sun, 13 Nov 2022 12:16:27 +0000 Subject: [PATCH 50/67] Create codeql.yml Shift from LGTM.com over to built-in github codeql analysis. --- .github/workflows/codeql.yml | 74 ++++++++++++++++++++++++++++++++++++ 1 file changed, 74 insertions(+) create mode 100644 .github/workflows/codeql.yml diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml new file mode 100644 index 000000000..fcefcfa96 --- /dev/null +++ b/.github/workflows/codeql.yml @@ -0,0 +1,74 @@ +# For most projects, this workflow file will not need changing; you simply need +# to commit it to your repository. +# +# You may wish to alter this file to override the set of languages analyzed, +# or to provide custom queries or build logic. +# +# ******** NOTE ******** +# We have attempted to detect the languages in your repository. Please check +# the `language` matrix defined below to confirm you have the correct set of +# supported CodeQL languages. +# +name: "CodeQL" + +on: + push: + branches: [ "develop" ] + pull_request: + # The branches below must be a subset of the branches above + branches: [ "develop" ] + schedule: + - cron: '16 8 * * 0' + +jobs: + analyze: + name: Analyze + runs-on: ubuntu-latest + permissions: + actions: read + contents: read + security-events: write + + strategy: + fail-fast: false + matrix: + language: [ 'python' ] + # CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ] + # Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support + + steps: + - name: Checkout repository + uses: actions/checkout@v3 + + # Initializes the CodeQL tools for scanning. + - name: Initialize CodeQL + uses: github/codeql-action/init@v2 + with: + languages: ${{ matrix.language }} + # If you wish to specify custom queries, you can do so here or in a config file. + # By default, queries listed here will override any specified in a config file. + # Prefix the list here with "+" to use these queries and those in the config file. + + # Details on CodeQL's query packs refer to : https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs + # queries: security-extended,security-and-quality + + + # Autobuild attempts to build any compiled languages (C/C++, C#, Go, or Java). + # If this step fails, then you should remove it and run the build manually (see below) + - name: Autobuild + uses: github/codeql-action/autobuild@v2 + + # ℹ️ Command-line programs to run using the OS shell. + # 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun + + # If the Autobuild fails above, remove it and uncomment the following three lines. + # modify them (or add more) to build your code if your project, please refer to the EXAMPLE below for guidance. + + # - run: | + # echo "Run, Build Application using script" + # ./location_of_script_within_repo/buildscript.sh + + - name: Perform CodeQL Analysis + uses: github/codeql-action/analyze@v2 + with: + category: "/language:${{matrix.language}}" From 297e1c9e81b8e7f415ce370e2fff148e281955a6 Mon Sep 17 00:00:00 2001 From: Mike Auty Date: Sun, 13 Nov 2022 12:56:30 +0000 Subject: [PATCH 51/67] Include code quality alerts in CodeQL scans --- .github/workflows/codeql.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index fcefcfa96..72bba07aa 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -50,7 +50,7 @@ jobs: # Prefix the list here with "+" to use these queries and those in the config file. # Details on CodeQL's query packs refer to : https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs - # queries: security-extended,security-and-quality + queries: security-and-quality # ,security-extended # Autobuild attempts to build any compiled languages (C/C++, C#, Go, or Java). From 3f5fd3502d8cfcd97816cd058049b04a9951a54a Mon Sep 17 00:00:00 2001 From: Mike Auty Date: Sun, 13 Nov 2022 13:34:39 +0000 Subject: [PATCH 52/67] Infra: Update the bug_report template to favour text over screenshots --- .github/ISSUE_TEMPLATE/bug_report.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 3a0cce8cc..2ccd4713c 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -23,8 +23,10 @@ Steps to reproduce the behavior: **Expected behavior** A clear and concise description of what you expected to happen. -**Screenshots** -If applicable, add screenshots to help explain your problem. +**Example output** +Please copy and paste the text demonstrating the issue, ideally with verbose output turned on (`vol.py -vvv ...`). + +Text is preferred to screenshots for searching and to talk about specific parts of the output. **Additional information** Add any other information about the problem here. From 324df0927534fd48fa61835c499bd0583b60c1cf Mon Sep 17 00:00:00 2001 From: Mike Auty Date: Sun, 13 Nov 2022 13:46:59 +0000 Subject: [PATCH 53/67] Core: Fix code scanning warnings notes --- volatility3/framework/objects/__init__.py | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/volatility3/framework/objects/__init__.py b/volatility3/framework/objects/__init__.py index eedd22bb2..4334f9d74 100644 --- a/volatility3/framework/objects/__init__.py +++ b/volatility3/framework/objects/__init__.py @@ -611,12 +611,10 @@ class Array(interfaces.objects.ObjectInterface, collections.abc.Sequence): @overload - def __getitem__(self, i: int) -> interfaces.objects.Template: - ... + def __getitem__(self, i: int) -> interfaces.objects.Template: ... @overload - def __getitem__(self, s: slice) -> List[interfaces.objects.Template]: - ... + def __getitem__(self, s: slice) -> List[interfaces.objects.Template]: ... def __getitem__(self, i): """Returns the i-th item from the array.""" From 129b92e3eedfbc71b1b1570e0d01c4c22e4a6fc1 Mon Sep 17 00:00:00 2001 From: Mike Auty Date: Sun, 13 Nov 2022 14:48:33 +0000 Subject: [PATCH 54/67] Core: Fix code scanning warnings/notes --- development/mac-kdk/parse_pbzx2.py | 84 +++++++++---------- doc/source/conf.py | 1 + volatility3/cli/volshell/__init__.py | 1 - volatility3/cli/volshell/generic.py | 8 +- volatility3/framework/automagic/pdbscan.py | 1 + .../framework/automagic/symbol_cache.py | 4 +- volatility3/framework/interfaces/automagic.py | 2 +- volatility3/framework/layers/resources.py | 1 + .../framework/plugins/linux/check_modules.py | 2 +- volatility3/framework/plugins/mac/lsmod.py | 5 +- .../framework/plugins/windows/cachedump.py | 14 ++-- .../framework/plugins/windows/netstat.py | 2 +- .../framework/plugins/windows/pslist.py | 4 +- .../framework/plugins/windows/psscan.py | 3 +- volatility3/framework/symbols/intermed.py | 7 +- .../symbols/linux/extensions/__init__.py | 1 + volatility3/framework/symbols/metadata.py | 4 +- .../symbols/windows/extensions/__init__.py | 3 +- .../plugins/windows/registry/certificates.py | 22 ++--- 19 files changed, 86 insertions(+), 83 deletions(-) diff --git a/development/mac-kdk/parse_pbzx2.py b/development/mac-kdk/parse_pbzx2.py index 7ce9090d4..5e56c9933 100644 --- a/development/mac-kdk/parse_pbzx2.py +++ b/development/mac-kdk/parse_pbzx2.py @@ -22,53 +22,49 @@ def seekread(f, offset = None, length = 0, relative = True): def parse_pbzx(pbzx_path): section = 0 xar_out_path = '%s.part%02d.cpio.xz' % (pbzx_path, section) - f = open(pbzx_path, 'rb') - # pbzx = f.read() - # f.close() - magic = seekread(f, length = 4) - if magic != 'pbzx': - raise RuntimeError("Error: Not a pbzx file") - # Read 8 bytes for initial flags - flags = seekread(f, length = 8) - # Interpret the flags as a 64-bit big-endian unsigned int - flags = struct.unpack('>Q', flags)[0] - xar_f = open(xar_out_path, 'wb') - while flags & (1 << 24): - # Read in more flags + with open(pbzx_path, 'rb') as f: + # pbzx = f.read() + # f.close() + magic = seekread(f, length = 4) + if magic != 'pbzx': + raise RuntimeError("Error: Not a pbzx file") + # Read 8 bytes for initial flags flags = seekread(f, length = 8) + # Interpret the flags as a 64-bit big-endian unsigned int flags = struct.unpack('>Q', flags)[0] - # Read in length - f_length = seekread(f, length = 8) - f_length = struct.unpack('>Q', f_length)[0] - xzmagic = seekread(f, length = 6) - if xzmagic != '\xfd7zXZ\x00': - # This isn't xz content, this is actually _raw decompressed cpio_ chunk of 16MB in size... - # Let's back up ... - seekread(f, offset = -6, length = 0) - # ... and split it out ... - f_content = seekread(f, length = f_length) - section += 1 - decomp_out = '%s.part%02d.cpio' % (pbzx_path, section) - g = open(decomp_out, 'wb') - g.write(f_content) - g.close() - # Now to start the next section, which should hopefully be .xz (we'll just assume it is ...) - xar_f.close() - section += 1 - new_out = '%s.part%02d.cpio.xz' % (pbzx_path, section) - xar_f = open(new_out, 'wb') - else: - f_length -= 6 - # This part needs buffering - f_content = seekread(f, length = f_length) - tail = seekread(f, offset = -2, length = 2) - xar_f.write(xzmagic) - xar_f.write(f_content) - if tail != 'YZ': - xar_f.close() - raise RuntimeError("Error: Footer is not xar file footer") + while flags & (1 << 24): + with open(xar_out_path, 'wb') as xar_f: + xar_f.seek(0, os.SEEK_END) + # Read in more flags + flags = seekread(f, length = 8) + flags = struct.unpack('>Q', flags)[0] + # Read in length + f_length = seekread(f, length = 8) + f_length = struct.unpack('>Q', f_length)[0] + xzmagic = seekread(f, length = 6) + if xzmagic != '\xfd7zXZ\x00': + # This isn't xz content, this is actually _raw decompressed cpio_ chunk of 16MB in size... + # Let's back up ... + seekread(f, offset = -6, length = 0) + # ... and split it out ... + f_content = seekread(f, length = f_length) + section += 1 + decomp_out = '%s.part%02d.cpio' % (pbzx_path, section) + with open(decomp_out, 'wb') as g: + g.write(f_content) + # Now to start the next section, which should hopefully be .xz (we'll just assume it is ...) + section += 1 + xar_out_path = '%s.part%02d.cpio.xz' % (pbzx_path, section) + else: + f_length -= 6 + # This part needs buffering + f_content = seekread(f, length = f_length) + tail = seekread(f, offset = -2, length = 2) + xar_f.write(xzmagic) + xar_f.write(f_content) + if tail != 'YZ': + raise RuntimeError("Error: Footer is not xar file footer") try: - f.close() xar_f.close() except IOError: pass diff --git a/doc/source/conf.py b/doc/source/conf.py index cadf6d3f2..895219b25 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -121,6 +121,7 @@ try: extensions.append('sphinx_autodoc_typehints') except ImportError: + # If the autodoc typehints extension isn't available, carry on regardless pass # Add any paths that contain templates here, relative to this directory. diff --git a/volatility3/cli/volshell/__init__.py b/volatility3/cli/volshell/__init__.py index 5eeef77cf..f32a587e0 100644 --- a/volatility3/cli/volshell/__init__.py +++ b/volatility3/cli/volshell/__init__.py @@ -36,7 +36,6 @@ class VolShell(cli.CommandLine): def __init__(self): super().__init__() - self.output_dir = None def run(self): """Executes the command line module, taking the system arguments, diff --git a/volatility3/cli/volshell/generic.py b/volatility3/cli/volshell/generic.py index 19e263a03..274b6ca17 100644 --- a/volatility3/cli/volshell/generic.py +++ b/volatility3/cli/volshell/generic.py @@ -324,11 +324,11 @@ class Volshell(interfaces.plugins.PluginInterface): " " * (longest_member - len_member), " ", member_type.vol.type_name) @classmethod - def _display_value(self, value: Any) -> str: + def _display_value(cls, value: Any) -> str: if isinstance(value, objects.PrimitiveObject): return repr(value) elif isinstance(value, objects.Array): - return repr([self._display_value(val) for val in value]) + return repr([cls._display_value(val) for val in value]) else: return hex(value.vol.offset) @@ -390,8 +390,8 @@ class Volshell(interfaces.plugins.PluginInterface): location = "file:" + request.pathname2url(location) print(f"Running code from {location}\n") accessor = resources.ResourceAccessor() - with io.TextIOWrapper(accessor.open(url = location), encoding = 'utf-8') as fp: - self.__console.runsource(fp.read(), symbol = 'exec') + with accessor.open(url = location) as fp: + self.__console.runsource(io.TextIOWrapper(fp.read(), encoding = 'utf-8'), symbol = 'exec') print("\nCode complete") def load_file(self, location: str): diff --git a/volatility3/framework/automagic/pdbscan.py b/volatility3/framework/automagic/pdbscan.py index 5cbdbfe0e..36288ef90 100644 --- a/volatility3/framework/automagic/pdbscan.py +++ b/volatility3/framework/automagic/pdbscan.py @@ -181,6 +181,7 @@ class KernelPDBScanner(interfaces.automagic.AutomagicInterface): hex(kvo))) except exceptions.InvalidAddressException: vollog.debug(f"Potential kernel_virtual_offset caused a page fault: {hex(kvo)}") + return None vollog.debug("Kernel base determination - testing fixed base address") return self._method_layer_pdb_scan(context, vlayer, test_physical_kernel, False, True, progress_callback) diff --git a/volatility3/framework/automagic/symbol_cache.py b/volatility3/framework/automagic/symbol_cache.py index fe5dfac52..164021340 100644 --- a/volatility3/framework/automagic/symbol_cache.py +++ b/volatility3/framework/automagic/symbol_cache.py @@ -223,8 +223,7 @@ class SqliteCache(CacheManagerInterface): def is_url_local(self, url: str) -> bool: """Determines whether an url is local or not""" parsed = urllib.parse.urlparse(url) - if parsed.scheme in ['file', 'jar']: - return True + return parsed.scheme in ['file', 'jar'] def get_identifier(self, location: str) -> Optional[bytes]: results = self._database.cursor().execute('SELECT identifier FROM cache WHERE location = ?', @@ -246,6 +245,7 @@ class SqliteCache(CacheManagerInterface): (location,)).fetchall() for row in results: return row['hash'] + return None def update(self, progress_callback = None): """Locates all files under the symbol directories. Updates the cache with additions, modifications and removals. diff --git a/volatility3/framework/interfaces/automagic.py b/volatility3/framework/interfaces/automagic.py index 713f91da0..4885645c3 100644 --- a/volatility3/framework/interfaces/automagic.py +++ b/volatility3/framework/interfaces/automagic.py @@ -113,7 +113,7 @@ class StackerLayerInterface(metaclass = ABCMeta): """The list operating systems/first-level plugin hierarchy that should exclude this stacker""" @classmethod - def stack(self, + def stack(cls, context: interfaces.context.ContextInterface, layer_name: str, progress_callback: constants.ProgressCallback = None) -> Optional[interfaces.layers.DataLayerInterface]: diff --git a/volatility3/framework/layers/resources.py b/volatility3/framework/layers/resources.py index dca215c85..73f59bdbd 100644 --- a/volatility3/framework/layers/resources.py +++ b/volatility3/framework/layers/resources.py @@ -31,6 +31,7 @@ try: # Import so that the handler is found by the framework.class_subclasses callc import smb.SMBHandler # lgtm [py/unused-import] except ImportError: + # If we fail to import this, it means that SMB handling won't be available pass vollog = logging.getLogger(__name__) diff --git a/volatility3/framework/plugins/linux/check_modules.py b/volatility3/framework/plugins/linux/check_modules.py index 6af8dec96..2c478cebf 100644 --- a/volatility3/framework/plugins/linux/check_modules.py +++ b/volatility3/framework/plugins/linux/check_modules.py @@ -29,7 +29,7 @@ class Check_modules(plugins.PluginInterface): ] @classmethod - def get_kset_modules(self, context: interfaces.context.ContextInterface, vmlinux_name: str): + def get_kset_modules(cls, context: interfaces.context.ContextInterface, vmlinux_name: str): vmlinux = context.modules[vmlinux_name] diff --git a/volatility3/framework/plugins/mac/lsmod.py b/volatility3/framework/plugins/mac/lsmod.py index 095fbc663..345267fea 100644 --- a/volatility3/framework/plugins/mac/lsmod.py +++ b/volatility3/framework/plugins/mac/lsmod.py @@ -46,14 +46,14 @@ class Lsmod(plugins.PluginInterface): try: kmod = kmod_ptr.dereference().cast("kmod_info") except exceptions.InvalidAddressException: - return [] + return # Generation finished yield kmod try: kmod = kmod.next except exceptions.InvalidAddressException: - return [] + return # Generation finished seen: Set = set() @@ -74,6 +74,7 @@ class Lsmod(plugins.PluginInterface): kmod = kmod.next except exceptions.InvalidAddressException: return + return # Generation finished def _generator(self): for module in self.list_modules(self.context, self.config['kernel']): diff --git a/volatility3/framework/plugins/windows/cachedump.py b/volatility3/framework/plugins/windows/cachedump.py index f77c6257b..59ea656f3 100644 --- a/volatility3/framework/plugins/windows/cachedump.py +++ b/volatility3/framework/plugins/windows/cachedump.py @@ -83,6 +83,13 @@ class Cachedump(interfaces.plugins.PluginInterface): return (username, domain, domain_name, hashh) def _generator(self, syshive, sechive): + if not syshive or not sechive: + if syshive is None: + vollog.warning('Unable to locate SYSTEM hive') + if sechive is None: + vollog.warning('Unable to locate SECURITY hive') + return + bootkey = hashdump.Hashdump.get_bootkey(syshive) if not bootkey: vollog.warning('Unable to find bootkey') @@ -142,12 +149,5 @@ class Cachedump(interfaces.plugins.PluginInterface): if hive.get_name().split('\\')[-1].upper() == 'SECURITY': sechive = hive - if syshive is None or sechive is None: - if syshive is None: - vollog.warning('Unable to locate SYSTEM hive') - if sechive is None: - vollog.warning('Unable to locate SECURITY hive') - return - return renderers.TreeGrid([("Username", str), ("Domain", str), ("Domain name", str), ('Hash', bytes)], self._generator(syshive, sechive)) diff --git a/volatility3/framework/plugins/windows/netstat.py b/volatility3/framework/plugins/windows/netstat.py index 93ac3af93..f7b285f05 100644 --- a/volatility3/framework/plugins/windows/netstat.py +++ b/volatility3/framework/plugins/windows/netstat.py @@ -42,7 +42,7 @@ class NetStat(interfaces.plugins.PluginInterface, timeliner.TimeLinerInterface): ] @classmethod - def _decode_pointer(self, value): + def _decode_pointer(cls, value): """Copied from `windows.handles`. Windows encodes pointers to objects and decodes them on the fly diff --git a/volatility3/framework/plugins/windows/pslist.py b/volatility3/framework/plugins/windows/pslist.py index cadccc5f1..6023d5f04 100644 --- a/volatility3/framework/plugins/windows/pslist.py +++ b/volatility3/framework/plugins/windows/pslist.py @@ -62,7 +62,9 @@ class PsList(interfaces.plugins.PluginInterface, timeliner.TimeLinerInterface): """ file_handle = None + proc_id = 'Invalid process object' try: + proc_id = proc.UniqueProcessId proc_layer_name = proc.add_process_layer() peb = context.object(kernel_table_name + constants.BANG + "_PEB", layer_name = proc_layer_name, @@ -76,7 +78,7 @@ class PsList(interfaces.plugins.PluginInterface, timeliner.TimeLinerInterface): file_handle.seek(offset) file_handle.write(data) except Exception as excp: - vollog.debug(f"Unable to dump PE with pid {proc.UniqueProcessId}: {excp}") + vollog.debug(f"Unable to dump PE with pid {proc_id}: {excp}") return file_handle diff --git a/volatility3/framework/plugins/windows/psscan.py b/volatility3/framework/plugins/windows/psscan.py index 335624672..9e3366dff 100644 --- a/volatility3/framework/plugins/windows/psscan.py +++ b/volatility3/framework/plugins/windows/psscan.py @@ -78,7 +78,7 @@ class PsScan(interfaces.plugins.PluginInterface, timeliner.TimeLinerInterface): layer_name: str, symbol_table: str, proc: interfaces.objects.ObjectInterface) -> \ - Iterable[interfaces.objects.ObjectInterface]: + Optional[interfaces.objects.ObjectInterface]: """ Returns a virtual process from a physical addressed one Args: @@ -124,6 +124,7 @@ class PsScan(interfaces.plugins.PluginInterface, timeliner.TimeLinerInterface): if virtual_process and \ proc.vol.offset == ph_offset: return virtual_process + return None @classmethod def get_osversion(cls, context: interfaces.context.ContextInterface, layer_name: str, diff --git a/volatility3/framework/symbols/intermed.py b/volatility3/framework/symbols/intermed.py index 1fceb1bcc..3fde0978d 100644 --- a/volatility3/framework/symbols/intermed.py +++ b/volatility3/framework/symbols/intermed.py @@ -102,10 +102,9 @@ class IntermediateSymbolTable(interfaces.symbols.SymbolTableInterface): # Check there are no obvious errors # Open the file and test the version self._versions = dict([(x.version, x) for x in class_subclasses(ISFormatTable)]) - fp = resources.ResourceAccessor().open(isf_url) - reader = codecs.getreader("utf-8") - json_object = json.load(reader(fp)) # type: ignore - fp.close() + with resources.ResourceAccessor().open(isf_url) as fp: + reader = codecs.getreader("utf-8") + json_object = json.load(reader(fp)) # type: ignore # Validation is expensive, but we cache to store the hashes of successfully validated json objects if validate and not schemas.validate(json_object): diff --git a/volatility3/framework/symbols/linux/extensions/__init__.py b/volatility3/framework/symbols/linux/extensions/__init__.py index 73f31115a..b47013c5d 100644 --- a/volatility3/framework/symbols/linux/extensions/__init__.py +++ b/volatility3/framework/symbols/linux/extensions/__init__.py @@ -128,6 +128,7 @@ class module(generic.GenericIntelProcess): sym_addr = sym.st_value if wanted_sym_name == sym_name: return sym_addr + return # Generation finished @property def section_symtab(self): diff --git a/volatility3/framework/symbols/metadata.py b/volatility3/framework/symbols/metadata.py index 350bb0a53..f42ac78fe 100644 --- a/volatility3/framework/symbols/metadata.py +++ b/volatility3/framework/symbols/metadata.py @@ -2,7 +2,7 @@ # which is available at https://www.volatilityfoundation.org/license/vsl-v1.0 # -from typing import Optional, Tuple +from typing import Optional, Tuple, Union from volatility3.framework import interfaces @@ -11,7 +11,7 @@ class WindowsMetadata(interfaces.symbols.MetadataInterface): """Class to handle the metadata from a Windows symbol table.""" @property - def pe_version(self) -> Optional[Tuple]: + def pe_version(self) -> Optional[Union[Tuple[int, int, int], Tuple[int, int, int, int]]]: build = self._json_data.get('pe', {}).get('build', None) revision = self._json_data.get('pe', {}).get('revision', None) minor = self._json_data.get('pe', {}).get('minor', None) diff --git a/volatility3/framework/symbols/windows/extensions/__init__.py b/volatility3/framework/symbols/windows/extensions/__init__.py index a86f5b3cb..87e8e0f45 100755 --- a/volatility3/framework/symbols/windows/extensions/__init__.py +++ b/volatility3/framework/symbols/windows/extensions/__init__.py @@ -719,7 +719,7 @@ class EPROCESS(generic.GenericIntelProcess, pool.ExecutiveObject): envars = context.layers[process_space].read(block, block_size).decode("utf-16-le", errors = 'replace').split('\x00')[:-1] except exceptions.InvalidAddressException: - return renderers.UnreadableValue() + return # Generation finished for envar in envars: split_index = envar.find('=') @@ -729,6 +729,7 @@ class EPROCESS(generic.GenericIntelProcess, pool.ExecutiveObject): # Exclude parse problem with some types of env if env and var: yield env, var + return # Generation finished class LIST_ENTRY(objects.StructType, collections.abc.Iterable): diff --git a/volatility3/plugins/windows/registry/certificates.py b/volatility3/plugins/windows/registry/certificates.py index 6029c0a5c..94962bc81 100644 --- a/volatility3/plugins/windows/registry/certificates.py +++ b/volatility3/plugins/windows/registry/certificates.py @@ -3,7 +3,7 @@ import logging import struct from typing import List, Iterator, Optional, Tuple, Type -from volatility3.framework import constants, exceptions, interfaces, renderers +from volatility3.framework import exceptions, interfaces, renderers from volatility3.framework.configuration import requirements from volatility3.framework.symbols.windows.extensions.registry import RegValueTypes from volatility3.plugins.windows.registry import hivelist, printkey @@ -46,14 +46,13 @@ class Certificates(interfaces.plugins.PluginInterface): open_method: Type[interfaces.plugins.FileHandlerInterface]) -> \ Optional[interfaces.plugins.FileHandlerInterface]: try: - if not isinstance(certificate_data, interfaces.renderers.BaseAbsentValue): - dump_name = "{}-{}-{}.crt".format(hive_offset, reg_section, key_hash) - file_handle = open_method(dump_name) - file_handle.write(certificate_data) - return file_handle + dump_name = "{}-{}-{}.crt".format(hive_offset, reg_section, key_hash) + file_handle = open_method(dump_name) + file_handle.write(certificate_data) + return file_handle except exceptions.InvalidAddressException: - vollog.debug(f"Unable to certificate file at {hive_offset:#x}") - return None + vollog.debug(f"Unable to dump certificate file at {hive_offset:#x}") + return None def _generator(self) -> Iterator[Tuple[int, Tuple[str, str, str, str]]]: @@ -79,9 +78,10 @@ class Certificates(interfaces.plugins.PluginInterface): key_hash = key_path[key_path.rindex("\\") + 1:] if self.config['dump']: - file_handle = self.dump_certificate(certificate_data, hive.hive_offset, reg_section, key_hash, self.open) - if file_handle: - file_handle.close() + if not isinstance(certificate_data, interfaces.renderers.BaseAbsentValue): + file_handle = self.dump_certificate(certificate_data, hive.hive_offset, reg_section, key_hash, self.open) + if file_handle: + file_handle.close() yield (0, (top_key, reg_section, key_hash, name)) From 374960f6db64109971964a3e18381016650a3087 Mon Sep 17 00:00:00 2001 From: Mike Auty Date: Sun, 13 Nov 2022 14:53:39 +0000 Subject: [PATCH 55/67] Windows: Fix bad use of strip Close #867 --- volatility3/framework/symbols/windows/pdbutil.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/volatility3/framework/symbols/windows/pdbutil.py b/volatility3/framework/symbols/windows/pdbutil.py index d74b21b60..25911e376 100644 --- a/volatility3/framework/symbols/windows/pdbutil.py +++ b/volatility3/framework/symbols/windows/pdbutil.py @@ -359,7 +359,7 @@ class PDBUtility(interfaces.configuration.VersionableInterface): vollog.debug(f"Found {guid['pdb_name']}: {guid['GUID']}-{guid['age']}") - module_name = guid["pdb_name"].strip('.pdb') + module_name = guid["pdb_name"].replace('.pdb', '') symbol_table_name = cls.load_windows_symbol_table(context, guid["GUID"], From b98a311688741d343a8711d2df657fd87b989352 Mon Sep 17 00:00:00 2001 From: Mike Auty Date: Sun, 13 Nov 2022 15:00:40 +0000 Subject: [PATCH 56/67] Core: Fix up recent typing changes --- volatility3/framework/plugins/windows/psscan.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/volatility3/framework/plugins/windows/psscan.py b/volatility3/framework/plugins/windows/psscan.py index 9e3366dff..00f96ff63 100644 --- a/volatility3/framework/plugins/windows/psscan.py +++ b/volatility3/framework/plugins/windows/psscan.py @@ -4,7 +4,7 @@ import datetime import logging -from typing import Iterable, Callable, Tuple +from typing import Iterable, Callable, Optional, Tuple from volatility3.framework import renderers, interfaces, layers, exceptions from volatility3.framework.configuration import requirements From 9b537678a5e1c252852f4aafe6dc669582c047f9 Mon Sep 17 00:00:00 2001 From: Mike Auty Date: Sun, 13 Nov 2022 15:52:09 +0000 Subject: [PATCH 57/67] Core: Fix up more scanning notes/warnings --- development/mac-kdk/parse_pbzx2.py | 5 +---- volatility3/framework/layers/vmware.py | 3 ++- volatility3/framework/objects/__init__.py | 6 ++---- .../framework/plugins/windows/registry/userassist.py | 6 +++++- volatility3/framework/symbols/linux/extensions/__init__.py | 2 +- volatility3/framework/symbols/windows/pdbconv.py | 1 + 6 files changed, 12 insertions(+), 11 deletions(-) diff --git a/development/mac-kdk/parse_pbzx2.py b/development/mac-kdk/parse_pbzx2.py index 5e56c9933..173a4d648 100644 --- a/development/mac-kdk/parse_pbzx2.py +++ b/development/mac-kdk/parse_pbzx2.py @@ -17,6 +17,7 @@ def seekread(f, offset = None, length = 0, relative = True): f.seek(offset, [0, 1, 2][relative]) if length: return f.read(length) + return None def parse_pbzx(pbzx_path): @@ -64,10 +65,6 @@ def parse_pbzx(pbzx_path): xar_f.write(f_content) if tail != 'YZ': raise RuntimeError("Error: Footer is not xar file footer") - try: - xar_f.close() - except IOError: - pass def main(): diff --git a/volatility3/framework/layers/vmware.py b/volatility3/framework/layers/vmware.py index ae4a7d55e..61b13eb88 100644 --- a/volatility3/framework/layers/vmware.py +++ b/volatility3/framework/layers/vmware.py @@ -154,7 +154,8 @@ class VmwareStacker(interfaces.automagic.StackerLayerInterface): vmss_success = False with contextlib.suppress(IOError): - _ = resources.ResourceAccessor().open(vmss).read(10) + with resources.ResourceAccessor().open(vmss) as fp: + _ = fp.read(10) context.config[interfaces.configuration.path_join(current_config_path, "location")] = vmss context.layers.add_layer(physical.FileLayer(context, current_config_path, current_layer_name)) vmss_success = True diff --git a/volatility3/framework/objects/__init__.py b/volatility3/framework/objects/__init__.py index 4334f9d74..2b026ccd1 100644 --- a/volatility3/framework/objects/__init__.py +++ b/volatility3/framework/objects/__init__.py @@ -747,10 +747,8 @@ class AggregateType(interfaces.objects.ObjectInterface): if isinstance(cls, agg_type): agg_name = agg_type.__name__ - assert isinstance(members, collections.abc.Mapping) - f"{agg_name} members parameter must be a mapping: {type(members)}" - assert all([(isinstance(member, tuple) and len(member) == 2) for member in members.values()]) - f"{agg_name} members must be a tuple of relative_offsets and templates" + assert isinstance(members, collections.abc.Mapping), f"{agg_name} members parameter must be a mapping: {type(members)}" + assert all([(isinstance(member, tuple) and len(member) == 2) for member in members.values()]), f"{agg_name} members must be a tuple of relative_offsets and templates" def member(self, attr: str = 'member') -> object: """Specifically named method for retrieving members.""" diff --git a/volatility3/framework/plugins/windows/registry/userassist.py b/volatility3/framework/plugins/windows/registry/userassist.py index 30b5db695..f31b7832e 100644 --- a/volatility3/framework/plugins/windows/registry/userassist.py +++ b/volatility3/framework/plugins/windows/registry/userassist.py @@ -33,7 +33,11 @@ class UserAssist(interfaces.plugins.PluginInterface): self._reg_table_name = None self._win7 = None # taken from http://msdn.microsoft.com/en-us/library/dd378457%28v=vs.85%29.aspx - self._folder_guids = json.load(open(os.path.join(os.path.dirname(__file__), "userassist.json"), "rb")) + try: + with open(os.path.join(os.path.dirname(__file__), "userassist.json"), "rb") as fp: + self._folder_guids = json.load(fp) + except IOError: + vollog.error("Usersassist data file not found") @classmethod def get_requirements(cls) -> List[interfaces.configuration.RequirementInterface]: diff --git a/volatility3/framework/symbols/linux/extensions/__init__.py b/volatility3/framework/symbols/linux/extensions/__init__.py index b47013c5d..ce002b905 100644 --- a/volatility3/framework/symbols/linux/extensions/__init__.py +++ b/volatility3/framework/symbols/linux/extensions/__init__.py @@ -128,7 +128,7 @@ class module(generic.GenericIntelProcess): sym_addr = sym.st_value if wanted_sym_name == sym_name: return sym_addr - return # Generation finished + return None @property def section_symtab(self): diff --git a/volatility3/framework/symbols/windows/pdbconv.py b/volatility3/framework/symbols/windows/pdbconv.py index 15b5c733a..d7d171ce1 100644 --- a/volatility3/framework/symbols/windows/pdbconv.py +++ b/volatility3/framework/symbols/windows/pdbconv.py @@ -934,6 +934,7 @@ class PdbRetreiver: if progress_callback is not None: progress_callback(100, f"Downloading {url + suffix}") if result is None: + result.close() return None return url + suffix From 8918b385a033c6d3adc80b77f1df1db0540395f7 Mon Sep 17 00:00:00 2001 From: Mike Auty Date: Sun, 13 Nov 2022 15:56:50 +0000 Subject: [PATCH 58/67] Windows: Improve nestat error checking Should partially solve #863 --- volatility3/framework/plugins/windows/netstat.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/volatility3/framework/plugins/windows/netstat.py b/volatility3/framework/plugins/windows/netstat.py index f7b285f05..651ca7696 100644 --- a/volatility3/framework/plugins/windows/netstat.py +++ b/volatility3/framework/plugins/windows/netstat.py @@ -427,6 +427,8 @@ class NetStat(interfaces.plugins.PluginInterface, timeliner.TimeLinerInterface): self.config_path) tcpip_module = self.get_tcpip_module(self.context, kernel.layer_name, kernel.symbol_table_name) + if not tcpip_module: + vollog.error("Unable to locate symbols for the memory image's tcpip module") try: tcpip_symbol_table = pdbutil.PDBUtility.symbol_table_from_pdb( From dd876ae18c376e9091d72679272974cfa19d6a53 Mon Sep 17 00:00:00 2001 From: Mike Auty Date: Sun, 13 Nov 2022 16:27:48 +0000 Subject: [PATCH 59/67] Core: Put the dev requirement back in the dev requirements file --- requirements-dev.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements-dev.txt b/requirements-dev.txt index 3ff7c50b8..7c372da2a 100644 --- a/requirements-dev.txt +++ b/requirements-dev.txt @@ -16,7 +16,7 @@ pycryptodome # This can improve error messages regarding improperly configured ISF files, # but is only recommended for development -# jsonschema>=2.3.0 +jsonschema>=2.3.0 # This is required for memory acquisition via leechcore/pcileech. leechcorepyc>=2.4.0 From 60ec8a39fff6bc53da653adf2c54f87c3a0d20f9 Mon Sep 17 00:00:00 2001 From: Mike Auty Date: Sun, 13 Nov 2022 18:37:19 +0000 Subject: [PATCH 60/67] Core: Fix another github scanning issue. --- volatility3/framework/symbols/windows/pdbconv.py | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/volatility3/framework/symbols/windows/pdbconv.py b/volatility3/framework/symbols/windows/pdbconv.py index d7d171ce1..4809aafcf 100644 --- a/volatility3/framework/symbols/windows/pdbconv.py +++ b/volatility3/framework/symbols/windows/pdbconv.py @@ -926,15 +926,16 @@ class PdbRetreiver: try: vollog.debug(f"Attempting to retrieve {url + suffix}") # We have to cache this because the file is opened by a layer and we can't control whether that caches - result = resources.ResourceAccessor(progress_callback).open(url + suffix) + with resources.ResourceAccessor(progress_callback).open(url + suffix) as fp: + fp.read(10) + result = True except (error.HTTPError, error.URLError) as excp: vollog.debug(f"Failed with {excp}") if result: break if progress_callback is not None: progress_callback(100, f"Downloading {url + suffix}") - if result is None: - result.close() + if not result: return None return url + suffix From 8c41007f0b72e80debce6ad2159ff368ae0dd719 Mon Sep 17 00:00:00 2001 From: Donghyun Kim Date: Tue, 29 Nov 2022 21:16:33 +0900 Subject: [PATCH 61/67] Fix: setup python method --- .github/workflows/build-pypi.yml | 12 +++++++----- .github/workflows/test.yaml | 13 ++++++++----- 2 files changed, 15 insertions(+), 10 deletions(-) diff --git a/.github/workflows/build-pypi.yml b/.github/workflows/build-pypi.yml index 77fe26931..ce69d4768 100644 --- a/.github/workflows/build-pypi.yml +++ b/.github/workflows/build-pypi.yml @@ -16,13 +16,15 @@ jobs: build: runs-on: ubuntu-latest + strategy: + matrix: + python-version: ["3.6"] steps: - - uses: actions/checkout@v2 - - - name: Set up Python 3.x - uses: actions/setup-python@v2 + - uses: actions/checkout@v3 + - name: Set up Python ${{ matrix.python-version }} + uses: actions/setup-python@v4 with: - python-version: '3.x' + python-version: ${{ matrix.python-version }} - name: Install dependencies run: | diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml index cf70b66cd..17117bd79 100644 --- a/.github/workflows/test.yaml +++ b/.github/workflows/test.yaml @@ -4,13 +4,16 @@ jobs: build: runs-on: ubuntu-latest + strategy: + matrix: + python-version: ["3.6"] steps: - - uses: actions/checkout@v2 - - - name: Set up Python 3.6 - uses: actions/setup-python@v2 + - uses: actions/checkout@v3 + - name: Set up Python ${{ matrix.python-version }} + uses: actions/setup-python@v4 with: - python-version: '3.6' + python-version: ${{ matrix.python-version }} + - name: Install dependencies run: | From f7a4d4f1ef0cdc408bcf32bb902c3b3e67c9642c Mon Sep 17 00:00:00 2001 From: Donghyun Kim Date: Tue, 29 Nov 2022 21:18:20 +0900 Subject: [PATCH 62/67] Fix: 3.6.7 versions --- .github/workflows/build-pypi.yml | 2 +- .github/workflows/test.yaml | 3 +-- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/.github/workflows/build-pypi.yml b/.github/workflows/build-pypi.yml index ce69d4768..7433269bf 100644 --- a/.github/workflows/build-pypi.yml +++ b/.github/workflows/build-pypi.yml @@ -18,7 +18,7 @@ jobs: runs-on: ubuntu-latest strategy: matrix: - python-version: ["3.6"] + python-version: ["3.6.7"] steps: - uses: actions/checkout@v3 - name: Set up Python ${{ matrix.python-version }} diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml index 17117bd79..bcc3b79b0 100644 --- a/.github/workflows/test.yaml +++ b/.github/workflows/test.yaml @@ -6,7 +6,7 @@ jobs: runs-on: ubuntu-latest strategy: matrix: - python-version: ["3.6"] + python-version: ["3.6.7"] steps: - uses: actions/checkout@v3 - name: Set up Python ${{ matrix.python-version }} @@ -14,7 +14,6 @@ jobs: with: python-version: ${{ matrix.python-version }} - - name: Install dependencies run: | python -m pip install --upgrade pip From 6e056a7819ba4694b9e643dcc8d73e6779c1a9a2 Mon Sep 17 00:00:00 2001 From: Donghyun Kim Date: Tue, 29 Nov 2022 21:20:25 +0900 Subject: [PATCH 63/67] Add: env value --- .github/workflows/build-pypi.yml | 2 ++ .github/workflows/test.yaml | 4 +++- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/.github/workflows/build-pypi.yml b/.github/workflows/build-pypi.yml index 7433269bf..95145f329 100644 --- a/.github/workflows/build-pypi.yml +++ b/.github/workflows/build-pypi.yml @@ -25,6 +25,8 @@ jobs: uses: actions/setup-python@v4 with: python-version: ${{ matrix.python-version }} + env: + AGENT_TOOLSDIRECTORY: /opt/hostedtoolcache - name: Install dependencies run: | diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml index bcc3b79b0..5d712ce25 100644 --- a/.github/workflows/test.yaml +++ b/.github/workflows/test.yaml @@ -6,13 +6,15 @@ jobs: runs-on: ubuntu-latest strategy: matrix: - python-version: ["3.6.7"] + python-version: ["3.6"] steps: - uses: actions/checkout@v3 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v4 with: python-version: ${{ matrix.python-version }} + env: + AGENT_TOOLSDIRECTORY: /opt/hostedtoolcache - name: Install dependencies run: | From 3daba5dabbdba1670262b48ca466a19328721b65 Mon Sep 17 00:00:00 2001 From: Donghyun Kim Date: Wed, 30 Nov 2022 04:04:07 +0900 Subject: [PATCH 64/67] Fix: more detail version --- .github/workflows/test.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml index 5d712ce25..0f3f10228 100644 --- a/.github/workflows/test.yaml +++ b/.github/workflows/test.yaml @@ -6,7 +6,7 @@ jobs: runs-on: ubuntu-latest strategy: matrix: - python-version: ["3.6"] + python-version: ["3.6.7"] steps: - uses: actions/checkout@v3 - name: Set up Python ${{ matrix.python-version }} From 372003b0e48224fcafa3f3bf0620b04d3ca9cb1e Mon Sep 17 00:00:00 2001 From: Donghyun Kim Date: Wed, 30 Nov 2022 04:07:18 +0900 Subject: [PATCH 65/67] Fix: more detail ubuntu version --- .github/workflows/test.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml index 0f3f10228..5f3c996f3 100644 --- a/.github/workflows/test.yaml +++ b/.github/workflows/test.yaml @@ -3,10 +3,10 @@ on: [push, pull_request] jobs: build: - runs-on: ubuntu-latest + runs-on: ubuntu-20.04 strategy: matrix: - python-version: ["3.6.7"] + python-version: ["3.6"] steps: - uses: actions/checkout@v3 - name: Set up Python ${{ matrix.python-version }} From 7aaf157e07480ccdb88bab33053a045fd8c16277 Mon Sep 17 00:00:00 2001 From: Donghyun Kim Date: Wed, 30 Nov 2022 04:09:28 +0900 Subject: [PATCH 66/67] Remove: env --- .github/workflows/test.yaml | 2 -- 1 file changed, 2 deletions(-) diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml index 5f3c996f3..8f044a365 100644 --- a/.github/workflows/test.yaml +++ b/.github/workflows/test.yaml @@ -13,8 +13,6 @@ jobs: uses: actions/setup-python@v4 with: python-version: ${{ matrix.python-version }} - env: - AGENT_TOOLSDIRECTORY: /opt/hostedtoolcache - name: Install dependencies run: | From 130baa7f34e5430d3dc081f0d8d9e936fe59ffb6 Mon Sep 17 00:00:00 2001 From: Donghyun Kim Date: Wed, 30 Nov 2022 04:10:40 +0900 Subject: [PATCH 67/67] Remove: env for build --- .github/workflows/build-pypi.yml | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/.github/workflows/build-pypi.yml b/.github/workflows/build-pypi.yml index 95145f329..061c8359a 100644 --- a/.github/workflows/build-pypi.yml +++ b/.github/workflows/build-pypi.yml @@ -15,18 +15,16 @@ on: jobs: build: - runs-on: ubuntu-latest + runs-on: ubuntu-20.04 strategy: matrix: - python-version: ["3.6.7"] + python-version: ["3.6"] steps: - uses: actions/checkout@v3 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v4 with: python-version: ${{ matrix.python-version }} - env: - AGENT_TOOLSDIRECTORY: /opt/hostedtoolcache - name: Install dependencies run: |