Merge pull request #2548 from NotoriousRebel/codex/upstream-source-workers-default

fix: restore conservative source-worker default

CHANGELOG.md

@@ -36,7 +36,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Changed
 - Routed discovery sources through immutable source jobs with bounded `TaskGroup` ownership, typed outcomes, and native cancellation propagation instead of queuing live coroutine objects.
-- Discovery now uses a fixed pool of six source workers by default. CLI `-j` or `--source-workers`, REST
+- Discovery now uses a fixed pool of three source workers by default. CLI `-j` or `--source-workers`, REST
   `source_workers`, and HarvestView can set another positive count without skipping sources or limiting their results.
 - Replaced Shodan's synchronous Python SDK with cancellable async Host API requests that honor configured proxies, query every unique resolved IPv4, paginate target-bound hostname and TLS-certificate searches without an adapter-specific result cap, retain successful partial results, and add no source-local deadline. Shodan now stores one canonical `shodan-host` result per IP with every normalized TCP or UDP service and scoped certificate CN/SAN metadata in native JSONL, SQLite, API, and HarvestView details instead of an escaped JSON value.
 - Reworked screenshot scans to use one bounded aiohttp session and one shared browser, with isolated per-target contexts, status-based reachability, and deterministic async cleanup.

README.md

@@ -42,7 +42,7 @@ Query several passive sources:
 uv run theHarvester -d example.com -b crtsh,certspotter,commoncrawl
 ```
 
-Six discovery sources run at once by default. Use `-j` or `--source-workers` with a positive number to change that
+Three discovery sources run at once by default. Use `-j` or `--source-workers` with a positive number to change that
 concurrency. The worker count never skips a selected source or limits its results, and it is automatically reduced when
 fewer sources are selected. REST `source_workers` and HarvestView use the same setting.
 

docs/wiki/Rest-API.md

@@ -66,7 +66,7 @@ run_id="$(curl -s http://127.0.0.1:5000/api/v1/runs \
       "target": "example.com",
       "sources": ["emails", "crtsh"],
       "limit": 500,
-      "source_workers": 6,
+      "source_workers": 3,
       "deadline_seconds": 1800
     }' \
   | jq -r '.run_id')"
@@ -79,7 +79,7 @@ curl -s "http://127.0.0.1:5000/api/v1/runs/$run_id" \
 Run submission is asynchronous. Lifecycle status is `queued`, `running`, `cancelling`, `cancelled`, `completed`, or `failed`. Terminal evidence status is reported separately as `complete`, `partial`, or `failed` when evidence exists.
 
 `source_workers` is the same positive concurrency used by CLI `-j` or `--source-workers` and HarvestView. It defaults
-to six, is reduced when fewer sources are selected, and never skips sources or limits their results.
+to three, is reduced when fewer sources are selected, and never skips sources or limits their results.
 
 P1 DNS and P2 direct options are fields on the same run request. The OpenAPI schema shows their current defaults, limits, and descriptions. The server uses the operator-selected target and does not impose a public-only egress policy.
 

tests/lib/test_enumeration.py

@@ -20,7 +20,7 @@ def test_enumeration_options_fill_the_shared_execution_defaults() -> None:
     assert options.start == 0
     assert options.dns_recursive_query_limit == DEFAULT_DNS_RECURSIVE_QUERY_LIMIT is None
     assert options.dns_recursive_runtime_seconds == DEFAULT_DNS_RECURSIVE_RUNTIME_SECONDS is None
-    assert options.source_workers == DEFAULT_SOURCE_WORKERS == 6
+    assert options.source_workers == DEFAULT_SOURCE_WORKERS == 3
 
 
 def test_enumeration_options_preserve_explicit_transport_values() -> None:

theHarvester/lib/enumeration.py

@@ -13,7 +13,7 @@ from theHarvester.lib.virtual_host import (
 
 DEFAULT_RESULT_LIMIT = 500
 DEFAULT_RESULT_START = 0
-DEFAULT_SOURCE_WORKERS = 6
+DEFAULT_SOURCE_WORKERS = 3
 DEFAULT_DNS_RECURSIVE_QUERY_LIMIT = DEFAULT_RECURSIVE_DNS_QUERY_LIMIT
 DEFAULT_DNS_RECURSIVE_RUNTIME_SECONDS = DEFAULT_RECURSIVE_DNS_RUNTIME_SECONDS