Commit Graph
142 Commits
Author SHA1 Message Date
Mike Auty 3cb48c19e8 Remove the errant urllib3 import in favour of native python. 2017-12-13 20:48:52 +00:00
Mike Auty 1dcad91126 Add in type annotations for scanners. 2017-12-13 20:48:52 +00:00
Mike Auty ac12098317 Add type-annotations for reamining layer files. 2017-12-13 20:48:52 +00:00
Mike Auty f40fae197d Convert to python3.5 syntax (no local type-annotations). 2017-12-13 20:48:52 +00:00
Mike Auty 736cbff1b6 Start on the layers folder (with a few small fixes). 2017-12-13 20:48:52 +00:00
Mike Auty 0ad6294662 Reuse the cache if we're within the same ResourceAccessor.
This effectively means that the FileTranslationLayer will reuse the
cached file even with multiple opens, but rerunning python, or starting
a new context (with a new FileTranslationLayer but on the same URL) will
cause a redownload whether necessary or not.  This ensures that running
volatility as an engine (inside a long lived python session) will not
prevent a file being checked again later.

Other caching mechanisms (such as last-modified) should be used to
determine if the cached file is still valid.

Note this may cause issues if plugins run concurrently.
2017-11-25 14:00:16 +00:00
Mike Auty 7a40d8128c Add in support for the smb protocol when loading files. 2017-11-24 22:28:14 +00:00
Mike Auty 917829d87f Reduce the number of times a ResourceAccessor is created, and use all handlers. 2017-11-24 22:27:36 +00:00
Mike Auty 30059766b6 Plug the WindowsMixin into the automagic. 2017-11-24 12:02:34 +00:00
Mike Auty b075d8c31c Standardize on nt_symbols for standard symboltable requirements.
It's longer to type, but people shouldn't be typing it directly.
They should be pulling the value from the config and using that, which
can default to 'nt' if necessary.
2017-11-13 01:27:22 +00:00
Mike Auty 3a591688a6 Add in a little more information about signatures. 2017-11-11 15:34:06 +00:00
Mike Auty 35d47b0fb0 Improve checks for invalid values
We determine address_masks using log/ln2, which cannot accept 0.
Therefore we don't support address spaces with a maximum_address of 0.
This can affect registry hives, so we've added a check in registry hives
to prevent creating layers with invalid maximum_addresses.
2017-11-11 15:15:14 +00:00
Mike Auty 527ef44800 Make sure the cell boundaries are checked correctly. 2017-11-09 23:13:38 +00:00
Mike Auty 497f0f4950 Registry node with RI signature are effectively _CM_KEY_INDEX, just with single item elements. 2017-11-08 11:40:17 +00:00
Mike Auty 2db214e2f8 Ensure we appropriately truncate unicode strings. 2017-11-08 01:00:53 +00:00
Mike Auty ba9ed5e3eb Add minor comments about the jar uri scheme. 2017-11-08 00:13:26 +00:00
Mike Auty adfc1f889a Add a comment about caching files that have been unzipped. 2017-11-06 00:48:33 +00:00
Mike Auty c0e2d7499a Ensure that magic isn't a hard dependency.
We rely on file extensions, but if we throw an Exception on a bad file
extension then it's not really our fault.
2017-11-05 22:46:24 +00:00
Mike Auty 71d938d78b Add in support for looking for symbols in zip files.
We use the jar scheme because that's actually registered with the right
bodies, even if the syntax is a bit weird.  The contents is still
processed by the ResourceAccessor meaning it can be compressed with any
of the supported compression methods.
2017-11-05 22:29:46 +00:00
Mike Auty a21d0c174a Fix the magic detection and add logging to the ResourceAccessor. 2017-11-05 22:29:37 +00:00
Mike Auty b6d932f4a5 Refactor the ResourceAccessor because it isn't an interface.
It's not strictly limited to layers either, but I don't really want to
create a whole extra generic file to put this in, so layers will do.
2017-11-05 21:27:56 +00:00
Mike Auty a4b4a8eed3 Improve the error handling with automagics. 2017-11-04 18:45:08 +00:00
Mike Auty d9d67efa3a Add in the capability to stash linux_banners.
Since the stacker and Linux automagic will always be different objects
(even if they're the same class), and we don't want to add OS specific
code to the stacker, this is the simplest way of allowing the Linux
symbol automagic to get information from the dtb finding stacker.
2017-11-04 17:09:59 +00:00
Mike Auty 01043b9f3e Rework isf_filename to isf_url since we can load it from a remote location if required. 2017-11-04 15:47:52 +00:00
Mike Auty 4b7543f2f4 Unify the resource opening. 2017-11-02 21:46:39 +00:00
Mike Auty 5ef65d6d38 Remove the ntkrnlmp name and use ntsymbols instead. 2017-11-01 21:23:44 +00:00
Mike Auty f085c9d2e3 Use a _CELL_DATA directly, rather than the eronious construction of a _CM_CACHED_VALUE_INDEX. 2017-10-09 01:41:08 +01:00
Mike Auty c09ce60f73 Support the new HMAP_ENTRY structure for Win10. 2017-10-09 01:40:37 +01:00
Mike Auty 3f70ee92be Improve the debugging output for unknown registry node signatures. 2017-10-09 00:54:02 +01:00
Mike Auty b8f545633d Ensure we do logging rather than printing. 2017-10-09 00:19:05 +01:00
Mike Auty 6c0d98ba1b Improve registry value handling. 2017-10-09 00:19:05 +01:00
Mike Auty 4fb24985eb Add in standard registry walking to the regtest plugin. 2017-10-09 00:19:05 +01:00
Mike Auty 023598a582 Additional improvements to the registry layer and cell/node handling. 2017-10-09 00:19:05 +01:00
Mike Auty fadb421331 Improve the support for signatures of cells. 2017-10-09 00:19:05 +01:00
Mike Auty 757837633f Make sure we don't lose volatile bit for members. 2017-10-09 00:19:05 +01:00
Mike Auty 19d66bd063 Add in current work in progress. 2017-10-09 00:18:15 +01:00
Mike Auty f538319d7c Add a little more code to the registry layer.
Really need to figure out how to deal with _HBIN and _HCELL for
systems that don't have those structures.  The format seems extremely
unlikely to change, but better not to hard code anything if avoidable.
2017-10-09 00:18:15 +01:00
Mike Auty 5a30ef3708 Start adding in some code for reading the registry. 2017-10-09 00:18:15 +01:00
Mike Auty c6724a538a Initial support for compressed files. 2017-08-11 01:12:50 +01:00
Mike Auty 807b5dcaf0 Ensure we throw an appropriate error if there's no patterns available. 2017-07-16 22:52:56 +01:00
Mike Auty c4fbaf248e Max the number of hash buckets needed precise, and explicitly linked to the hash function. 2017-07-16 01:27:20 +01:00
Mike Auty f9375d0797 Reduce pointless memory usage, and remove old imports and debugging exception handlers. 2017-07-16 01:23:31 +01:00
Mike Auty 0ca78bc890 Add an efficient multi-string search module (based on Wu-Manber 1994).
Empirically this seems far quicker than either Aho-Corasick or
Commentz-Walter for our dataset (very many long search terms).
2017-07-16 01:13:49 +01:00
Mike Auty 2b718f104d Remove imports that will break the code. 2017-07-14 01:15:52 +01:00
Mike Auty dd7c9d777a Refactor the scanners to allow for better organization. 2017-07-13 01:30:01 +01:00
Mike Auty 761777c8ef Add commentary as to why we can't scan faster by ignoring surjective duplicates. 2017-06-03 20:57:19 +01:00
Mike Auty c05599d78a Minor clean-up of segmented layer. 2017-06-03 20:26:45 +01:00
Mike Auty b72c3aaed5 Handle invalid translations lower than the top layer when scanning. 2017-06-03 20:26:21 +01:00
Mike Auty 5e15d522bd Remove automatic IDE-inserted attributions.
They weren't consistent and they show just remind me how slowly
I've been getting vol 3 written!  5:S

There's git commits to track all of that stuff if we need it.
2017-05-14 11:40:49 +01:00
Mike Auty 6da8076050 Add in Regular Expression scanner. 2017-05-06 14:02:09 +01:00