Commit Graph
134 Commits
Author SHA1 Message Date
Mike Auty eaa8391b22 Scanning: switch to ThreadPool for threading. 2019-12-29 23:45:26 +00:00
Mike Auty b0b868d79c Typing: General typing fixes across the tree 2019-11-27 11:30:14 +00:00
Mike Auty d29cd776ec Layers: Fixing multiprocessing typing issue. 2019-11-27 11:30:14 +00:00
Mike Auty 19d46563ae layers: Fix up the maths around scanning chunks 2019-11-13 21:44:13 +00:00
Mike Auty ca8099e7d7 layers: Improve scanning of layered chunks
Previously, for segmented layers, no chunk larger than a full segment
was scanned.  Now, we combine contiguous segments up to
scanner.chunk_size (or until a non-present segment) in order to scan it.

This may be slightly slower (by concatting the data together) but gives
results of hits across page boundaries (the point of the scanning
capability), whereas previously each page was scanned individually.
2019-11-13 21:44:13 +00:00
Mike Auty e0b9d72c84 Update the license URL and LICENSE.txt 2019-09-25 21:44:05 +01:00
Mike Auty 37cb541013 Change the layer naming convention to include the underscores when necessary. 2019-09-25 19:38:39 +01:00
Mike Auty 72567e1c50 Yapf-0.28.0 rerun across the whole codebase. 2019-09-21 21:08:23 +01:00
Mike Auty e922cef316 Improve docstrings for all plugins, and reformat all docstrings. 2019-09-07 22:59:54 +01:00
Mike Auty c88044db5b Update more API documentation. 2019-09-04 00:34:55 +01:00
Mike Auty a362dc109a Update API documentation. 2019-09-03 21:28:19 +01:00
Mike Auty 9cde94bceb Change the license blurb on all files. 2019-08-27 23:11:00 +01:00
Mike Auty 387024fe96 Don't forget to return a default encoded blob. 2019-08-27 21:09:13 +01:00
Mike Auty 79cb7a28f6 Add in support for non-linear mapping. 2019-08-27 21:09:13 +01:00
Mike Auty 52b339d0fa Ensure we only write as much data as is required. 2019-08-25 13:25:01 +01:00
Mike Auty e6125b5df6 Improve LayerException with name field. 2019-08-25 13:20:12 +01:00
Mike Auty c188b3e56a Add in threading parallelism option back in. 2019-08-18 22:01:39 +01:00
Mike Auty 272224a1b0 Refactor all references to Context.memory to Context.layers. 2019-07-07 17:42:51 +01:00
Mike Auty 7c9696def9 Minor fixes for typing and imports. 2019-03-02 22:45:39 +00:00
Mike Auty d4045c9c13 Significant fix for reading from the start of SegmentedLayers (including Lime) when ignoring errors. 2019-03-02 22:44:51 +00:00
Mike Auty 14a9788802 Fix a typo in the parallelism code. 2019-03-02 19:34:31 +00:00
Mike Auty 9fadee1a17 Add locking for threaded operation. 2019-03-02 18:07:48 +00:00
Mike Auty 7b06480d7e Major speed-up for bulk scanning of images. 2019-02-14 22:14:32 +00:00
Mike Auty c64ed6a8dd Remove validity checking in favour of static-type checking.
This removes the assertions that checked plugin devs didn't do anything
overly bad in favour of using mypy to ensure appropriate type-checking.

It also moves the ProgressCallback typing information to constants in
order to avoid circular imports (since constants doesn't import
anything).
2019-01-20 02:35:49 +00:00
Mike Auty 3078141352 Fix typing for python < 3.6.1. 2019-01-03 01:34:04 +00:00
Mike Auty b13fcb186c Improve typing issues. 2019-01-03 01:13:53 +00:00
Mike Auty fdc6efc4ad Add in the LICENSE and the README. 2018-12-19 23:55:03 +00:00
Mike Auty 19572b6e1e Reformat all python files using yapf with custom volatility style. 2018-12-16 16:50:17 +00:00
Mike Auty 7818fba4a4 Fix up ChainMap issue in python-3.5. 2018-12-16 13:46:04 +00:00
Mike Auty 29d41470a4 Mass reformat of typing imports
Relented on the strict import of direct objects/classes for the typing
module only.  Typing module components can be directly imported because
it makes the code really painful to read and write otherwise.

This is still in-line with the python style guide adopted from Google at
http://google.github.io/styleguide/pyguide.html section 2.2.
2018-12-16 13:04:22 +00:00
Mike Auty 34ed11bfc0 Fix up issue with coalescing sections. 2018-12-07 01:47:16 +00:00
Mike Auty aaeb349a70 Remove debugging call. 2018-12-07 01:47:16 +00:00
Mike Auty 4fabb762d2 Make some typing fixes and remove specification of the iterator. 2018-12-07 01:47:16 +00:00
Mike Auty 341b9b8f9e Add in initial changes for scanner sections. 2018-12-07 01:47:16 +00:00
Mike Auty 21384be035 Remove the confusing max/min address values in scan chunk (the iterator should only pass chunks that need scanning). 2018-10-11 11:14:07 +01:00
Mike Auty b74eb68c04 Pull the check on layer names out so that DataLayerInterface types get it as well. 2018-08-22 09:33:47 +01:00
Mike Auty 4897b7ab81 Fix up some typing errors. 2018-07-22 13:19:20 +01:00
Mike Auty 7a52ac9deb Fix a large number of typing issues.
There are several instances where mypy didn't detect

if blah is not None:
    blah = thing

and so were rewritten as:

blah = blah or thing
2018-06-04 01:25:02 +01:00
Mike Auty f72e39d558 Fix up small but significant typo.
When asking for data with padding, the padding was guaranteed to always
be at the end without spaces between pages.
2018-05-29 00:42:52 +01:00
Mike Auty 67239002bc Attempt to fix an intel read issue spotted by imhlv2
The length value inside the read method of the TranslationLayer base class
would potentially get overwritten by the inner loop which also defined a
length value.
2018-05-28 23:25:49 +01:00
Mike Auty 61bf1d40b0 Fix scan_iterator parameter type. 2018-05-15 00:06:43 +01:00
Mike Auty 037ac28534 Move around the scanning functions and create an API for the iterators. 2018-05-14 01:03:43 +01:00
Mike Auty a07691b04f Ensure scanners don't return duplicates in the overlap. 2018-05-13 20:55:54 +01:00
Mike Auty ffe2907fa0 Minor typing fix. 2018-05-01 21:22:26 +01:00
Mike Auty 9512cbe9eb Commit metadata changeset.
Layers now accept metadata dictionaries (and chain/stack them on top of
those from lower layers).  Metadata can only be set at construction
time, and the metadata dictionary is readonly.  The hope is this will
make enumerating metadata keys across the codebase simpler.

The current metadata items that layers hold is:

architecture (Unknown | Intel32 | Intel64)
os (Unknown | Windows | Linux)
pae (bool)
page_map_offset (int)

This patchset may develop further to help enumerate all of these
(through a registration/reporting system).
2018-04-26 12:48:14 +01:00
Mike Auty 556fa29ada Deprecate/remove 'provides' attribute. 2018-04-26 12:31:33 +01:00
Mike Auty 623180ddbd Make many typing fixes, based on mypy-0.590. 2018-04-22 20:45:59 +01:00
Mike Auty d8f497eb71 Allow unpickling of PrimitiveObjects. 2018-04-08 22:25:46 +01:00
Mike Auty ac12098317 Add type-annotations for reamining layer files. 2017-12-13 20:48:52 +00:00
Mike Auty f40fae197d Convert to python3.5 syntax (no local type-annotations). 2017-12-13 20:48:52 +00:00