Mike Auty
663cf3eff8
Just tidy up the multistring searchers slightly.
2018-01-25 11:27:59 +00:00
Mike Auty
ccd00afdb2
Add in new multistring scanner based on re
...
Unfortunately in tests it turned out re was faster than a python
implementation of Wu-Manber. The Wu-Manber code has been left (because
why not) but it's not really advantageous over the re builtin since
it's compiled in C.
2018-01-24 20:22:00 +00:00
Mike Auty
4bb1161490
Make WuManber a callable program.
2017-12-24 12:53:50 +00:00
Mike Auty
1dcad91126
Add in type annotations for scanners.
2017-12-13 20:48:52 +00:00
Mike Auty
807b5dcaf0
Ensure we throw an appropriate error if there's no patterns available.
2017-07-16 22:52:56 +01:00
Mike Auty
c4fbaf248e
Max the number of hash buckets needed precise, and explicitly linked to the hash function.
2017-07-16 01:27:20 +01:00
Mike Auty
f9375d0797
Reduce pointless memory usage, and remove old imports and debugging exception handlers.
2017-07-16 01:23:31 +01:00
Mike Auty
0ca78bc890
Add an efficient multi-string search module (based on Wu-Manber 1994).
...
Empirically this seems far quicker than either Aho-Corasick or
Commentz-Walter for our dataset (very many long search terms).
2017-07-16 01:13:49 +01:00
Mike Auty
2b718f104d
Remove imports that will break the code.
2017-07-14 01:15:52 +01:00
Mike Auty
dd7c9d777a
Refactor the scanners to allow for better organization.
2017-07-13 01:30:01 +01:00
Mike Auty