From 4bff4d94e5771bbf321115018dd86205537a4647 Mon Sep 17 00:00:00 2001 From: Mike Auty Date: Sun, 16 Aug 2020 00:51:02 +0100 Subject: [PATCH 1/3] Documentation: Add an initial glossary --- doc/source/glossary.rst | 179 ++++++++++++++++++++++++++++++++++++++++ doc/source/index.rst | 1 + 2 files changed, 180 insertions(+) create mode 100644 doc/source/glossary.rst diff --git a/doc/source/glossary.rst b/doc/source/glossary.rst new file mode 100644 index 000000000..674b67469 --- /dev/null +++ b/doc/source/glossary.rst @@ -0,0 +1,179 @@ +Glossary +======== +There are many terms when talking about memory forensics, this list hopes to define the common ones and +provide some commonality on how to refer to particular ideas within the field. + +A +- +.. _Address: + An address is another name for an :ref:`offset`, specifically an offset within memory. Offsets can be + both relative or absolute, whereas addresses are almost always absolute. + +.. _Address Space: + +Address Space + This is the name in volatility 2 for what's referred to as a :ref:`Translation Layer`. It + encompasses all values that can be addresses, usually in reference to addresses in memory. + +.. _Alignment: + +Alignment + This value is what all data :ref:`offsets` will typically be a multiple of within a :ref:`type`. + +.. _Array: + +Array + This represents a list of items, which can be access by an index, which is zero-based (meaning the first + element has index 0). Items in arrays are almost always the same size (it is not a generic list, as in python) + even if they are :ref:`pointers` to different sized objects. + +D +- +.. _Data Layer: + +Data Layer + A group of bytes, where each byte can be addressed by a specific offset. Data layers are usually contiguous + chunks of data. + +.. _Dereference: + +Dereference + The act of taking the value of a pointer, and using it as an offset to another object, as a reference. + +.. _Domain: + +Domain + This the grouping for input values for a mapping or mathematical function. + +M +- +.. _Map: + +Map, mapping + A mapping is a relationship between two values, where one value (the :ref:`Domain` maps to the :ref:`Range` value). + Mappings can be seen as a mathematical function, and therefore volatility 3 attempts to use mathematical functional + notation where possible. + +.. _Member: + +Member + The name of subcomponents of a type, similar to attributes of objects in common programming parlance. These + are usually recorded as :ref:`offset` and :ref:`type` pairs within a :ref:`structure`. + +O +- +.. _Object: + +Object + This has a specific meaning within computer programming (as in Object Oriented Programming), but within the world + of Volatility it is used to refer to a type that has been associated with a chunk of data. See all :ref:`Type`. + +.. _Offset: + +Offset + A numeric value that identifies a distance within a group of bytes, to uniquely identify a single byte, or the + start of a run of bytes. This is often relative (offset from another object/item) but can be absolute (offset from + the start of a region of data). + +P +- +.. _Packed: + +Packed + Structures are often :ref:`aligned` meaning that the various members (subtypes) are always aligned at + particular values (usually multiples of 2, 4 or 8). Thus if a particular value is an odd number of bytes, the + next chunk of data containing useful information would start at an even offset, and a single byte of + :ref:`padding` would be used to ensure appropriate :ref:`alignment`. In packed structures, no + padding is used, and offsets may be at odd offsets. + +.. _Padding: + +Padding + Data that (usually) contains no useful information. The typical value used for padding is 0, so should a string + :ref:`object` that has been allocated a particular number of bytes, contain a string of fewer bytes, the remaing bytes + will be padded with null (0) bytes. + +.. _Page: + +Page + A specific chunk of contiguous data. It is an organizational quantity of memory (usually 0x1000, or 4096 bytes). + Pages, like pages in a book, make up the whole, but allow for specific chunks to be allocated and used as necessary. + Operating systems uses pages as a means to have granular control over chunks of memory. This allows them to be + reordered and reused as necessary (without having to move large chunks of data around), and allows them to have + access controls placed upon them, limiting actions such as reading and writing. + +.. _Page Table: + +Page Table + A table that points to a series of :ref:`pages`. Each page table is typically the size of a single page, + and page tables can point to pages that are in fact other page tables. Using tables that point to tables, it's + possible to use them as a way to map a particular address within a (potentially larger, but sparsely populated) + virtual space to a concrete (and usually contiguous) physical space, through the process of :ref:`mapping`. + +.. _Pointer: + +Pointer + A value within memory that points to a different area of memory. This allows objects to contain references to + other objects without containing all the data of the other object. Following a pointer is known as :ref:`dereferencing` + a pointer. Pointers are usually as large as the size of the + +R +- +.. _Range: + +Range + This is the grouping the output values for a mapping or mathematical function. + +S +- +.. _Struct: + +Struct, Structure + A means of containing multiple different :ref:`type` associated together. A struct typically contains + other :ref:`type`, one directly after another (unless :ref:`packing` is involved). In this way + the :ref:`members` of a type can be accessed by finding the data at the relative :ref:`offset` to + the start of the structure. + +.. _Symbol: + +Symbol + This is used in many different contexts, as short term for many things. A symbol is a construct that usually + encompasses a specific :ref:`offset` and a :ref:`type`, representing a specific instance of a type within the memory of a + compiled and running program. + +T +- +.. _Template: + +Template + Within volatility 3, the term template applies to a :ref:`type` that has not yet been instantiated or linked + to any data or a specific location within memory. Once a type has been tied to a particular chunk of data, it is + called an :ref:`object`. + +.. _Translation Layer: + +Translation Layer + This is a specific type of :ref:`data layer`, a non-contiguous group of bytes that can be references by + a unique :ref:`offset` within the layer. In particular, translation layers translates (or :ref:`maps`) + requests made of it to a location within a lower layer. This can be either linear (a one-to-one mapping between bytes) + or non-linear (a group of bytes :ref:`maps` to a larger or smaller group of bytes. + +.. _Type: + +Type + This is a structure definition of multiple elements that expresses how data is laid out. Basic types define how + the data should be interpretted in terms of a run of bits (or more commonly a collection of 8 bits at a time, + called bytes). More complex types can be made up of other types combined together at specific locations known + as :ref:`structs` or repeated, known as :ref:`array`. They can even defined types at the same + location depending on the data itself, known as :ref:`Unions`. Once a type has been linked to a specific + chunk of data, the result is referred to as an :ref:`object`. + +U +- +.. _Union: + +Union + A union is a type that can have can hold multiple different subtypes, which specifically overlap. A union is means + for holding two different types within the same size of data, meaning that not all types within the union will hold + valid data at the same time, more that depending on what the union is holding, a subset of the type will point to + accurate data (assumption no corruption). diff --git a/doc/source/index.rst b/doc/source/index.rst index c61e6e847..2ba85da5a 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -17,6 +17,7 @@ Here are some guidelines for using Volatility 3 effectively: complex-plugin using-as-a-library symbol-tables + glossary Python Packages =============== From 84741c30fda75b43d14499d10b634b6da896f67c Mon Sep 17 00:00:00 2001 From: Mike Auty Date: Mon, 7 Dec 2020 23:13:22 +0000 Subject: [PATCH 2/3] Documentation: Make improvements based on feedback from NiklasBeierl --- doc/source/glossary.rst | 45 ++++++++++++++++++++++++----------------- 1 file changed, 27 insertions(+), 18 deletions(-) diff --git a/doc/source/glossary.rst b/doc/source/glossary.rst index 674b67469..50bc69490 100644 --- a/doc/source/glossary.rst +++ b/doc/source/glossary.rst @@ -50,9 +50,13 @@ M .. _Map: Map, mapping - A mapping is a relationship between two values, where one value (the :ref:`Domain` maps to the :ref:`Range` value). - Mappings can be seen as a mathematical function, and therefore volatility 3 attempts to use mathematical functional - notation where possible. + A mapping is a relationship between two sets (where elements of the :ref:`Domain` map to elements + of the :ref:`Range`). Mappings can be seen as a mathematical function, and therefore volatility 3 + attempts to use mathematical functional notation where possible. Within volatility a mapping is most often + used to refer to the function for translating addresses from a higher layer (domain) to a lower layer (range). + For further information, please see + `Function (mathematics) in wikipedia https://en.wikipedia.org/wiki/Function_(mathematics)` + .. _Member: @@ -66,13 +70,14 @@ O Object This has a specific meaning within computer programming (as in Object Oriented Programming), but within the world - of Volatility it is used to refer to a type that has been associated with a chunk of data. See all :ref:`Type`. + of Volatility it is used to refer to a type that has been associated with a chunk of data, or a specific instance + of a type. See also :ref:`Type`. .. _Offset: Offset A numeric value that identifies a distance within a group of bytes, to uniquely identify a single byte, or the - start of a run of bytes. This is often relative (offset from another object/item) but can be absolute (offset from + start of a run of bytes. An offset is often relative (offset from another object/item) but can be absolute (offset from the start of a region of data). P @@ -81,17 +86,18 @@ P Packed Structures are often :ref:`aligned` meaning that the various members (subtypes) are always aligned at - particular values (usually multiples of 2, 4 or 8). Thus if a particular value is an odd number of bytes, the - next chunk of data containing useful information would start at an even offset, and a single byte of - :ref:`padding` would be used to ensure appropriate :ref:`alignment`. In packed structures, no - padding is used, and offsets may be at odd offsets. + particular values (usually multiples of 2, 4 or 8). Thus if the data used to represent a particular value has + an odd number of bytes, not a multiple of the chosen number, there will be :ref:`padding` between it and + the next member. In packed structs, no padding is used and the offset of the next member depends on the length of + the previous one. .. _Padding: Padding - Data that (usually) contains no useful information. The typical value used for padding is 0, so should a string - :ref:`object` that has been allocated a particular number of bytes, contain a string of fewer bytes, the remaing bytes - will be padded with null (0) bytes. + Data that (usually) contains no useful information. The typical value used for padding is 0 (sometimes called + a null byte). As an example, if a string :ref:`object` that has been allocated a particular number of + bytes, actually contains fewer bytes, the rest of the data (to make up the original length) will be padded with + null (0) bytes. .. _Page: @@ -115,14 +121,15 @@ Page Table Pointer A value within memory that points to a different area of memory. This allows objects to contain references to other objects without containing all the data of the other object. Following a pointer is known as :ref:`dereferencing` - a pointer. Pointers are usually as large as the size of the + a pointer. Pointers are usually the same length as the maximum address of the address space, since they + should be able to point to any address within the space. R - .. _Range: Range - This is the grouping the output values for a mapping or mathematical function. + This is the set of the possible output values for a mapping or mathematical function. S - @@ -130,16 +137,18 @@ S Struct, Structure A means of containing multiple different :ref:`type` associated together. A struct typically contains - other :ref:`type`, one directly after another (unless :ref:`packing` is involved). In this way + other :ref:`type`, usually :ref:`aligned` (unless :ref:`packing` is involved). In this way the :ref:`members` of a type can be accessed by finding the data at the relative :ref:`offset` to the start of the structure. .. _Symbol: Symbol - This is used in many different contexts, as short term for many things. A symbol is a construct that usually - encompasses a specific :ref:`offset` and a :ref:`type`, representing a specific instance of a type within the memory of a - compiled and running program. + This is used in many different contexts, as a short term for many things. Within Volatility, a symbol is a + construct that usually encompasses a specific type :ref:`type` at a specfific :ref:`offset`, + representing a particular instance of that type within the memory of a compiled and running program. An example + would be the location in memory of a list of active tcp endpoints maintained by the networking stack + within an operating system. T - From 92bf92eeff7817edb5e3ae01d75382da6526e599 Mon Sep 17 00:00:00 2001 From: Mike Auty Date: Wed, 13 Jan 2021 01:26:01 +0000 Subject: [PATCH 3/3] Documentation: Make suggested changes from @NiklasBeierl --- doc/source/glossary.rst | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/doc/source/glossary.rst b/doc/source/glossary.rst index 50bc69490..68bc41e4c 100644 --- a/doc/source/glossary.rst +++ b/doc/source/glossary.rst @@ -162,27 +162,31 @@ Template .. _Translation Layer: Translation Layer - This is a specific type of :ref:`data layer`, a non-contiguous group of bytes that can be references by - a unique :ref:`offset` within the layer. In particular, translation layers translates (or :ref:`maps`) - requests made of it to a location within a lower layer. This can be either linear (a one-to-one mapping between bytes) - or non-linear (a group of bytes :ref:`maps` to a larger or smaller group of bytes. + This is a type of data layer which allows accessing data from lower layers using addresses different to those + used by the lower layers themselves. When accessing data in a translation layer, it translates (or :ref:`maps`) + addresses from its own :ref:`address space
` to the address space of the lower layer and returns the + corresponding data from the lower layer. Note that multiple addresses in the higher layer might refer to the same + address in the lower layer. Conversely, some addresses in the higher layer might have no corresponding address in the + lower layer at all. Translation layers most commonly handle the translation from virtual to physical addresses, + but can be used to translate data to and from a compressed form or translate data from a particular file format + into another format. .. _Type: Type This is a structure definition of multiple elements that expresses how data is laid out. Basic types define how - the data should be interpretted in terms of a run of bits (or more commonly a collection of 8 bits at a time, - called bytes). More complex types can be made up of other types combined together at specific locations known - as :ref:`structs` or repeated, known as :ref:`array`. They can even defined types at the same - location depending on the data itself, known as :ref:`Unions`. Once a type has been linked to a specific - chunk of data, the result is referred to as an :ref:`object`. + the data should be interpreted in terms of a run of bits (or more commonly a collection of 8 bits at a time, + called bytes). New types can be constructed by combining other types at specific relative offsets, forming something + called a :ref:`struct`, or by repeating the same type, known as an :ref:`array`. They can even + contain other types at the same offset depending on the data itself, known as :ref:`Unions`. Once a type + has been linked to a specific chunk of data, the result is referred to as an :ref:`object`. U - .. _Union: Union - A union is a type that can have can hold multiple different subtypes, which specifically overlap. A union is means - for holding two different types within the same size of data, meaning that not all types within the union will hold - valid data at the same time, more that depending on what the union is holding, a subset of the type will point to - accurate data (assumption no corruption). + A union is a type that can hold multiple different subtypes, whose relative offsets specifically overlap. + A union is a means for holding multiple different types within the same size of data, the relative offsets of the + types within the union specifically overlap. This means that the data in a union object is interpreted differently + based on the types of the union used to access it.