mirror of
https://github.com/yogeshojha/rengine.git
synced 2026-10-18 14:27:17 +02:00
Yogesh OjhaandClaude Opus 5
417ffa4b7d
A rate was applied per process rather than per stage. content_discovery
starts four ffuf processes and nuclei runs its two rate groups at once, so
a ceiling of 150/s was 600/s and rate+rate/divisor. share_rate() splits
evenly and _rate_plan() splits in the WAF divisor's ratio, so the guarded
group stays that many times slower rather than merely slower. A ceiling of
1/s with both nuclei groups live still costs 2/s: a group cannot run at 0.
EGRESS_PROXY_URL replaces RENGINE_PROXY_URL, which shared/http.py read and
nothing could set. Third-party intelligence deliberately does not follow
the scan's proxy — that one is for target traffic, and routing RIPEstat or
RDAP through a tester's Burp instance would fill it with noise and may sit
outside its scope — so the instance gets its own egress proxy instead,
beside EGRESS_TIMEOUT and EGRESS_USER_AGENT.
DnsxService takes the recorder and the run's custom dnsx args, so
seed_resolution and target_enrichment appear in the command log like every
other tool.
wafw00f reads its header file with line.split(':', 2) and keeps a line
only at two parts, so any value holding a colon is dropped — every dated
cookie. unreadable_headers() names them and the stage says so.
naabu -passive does honour proxy_env, measured by pointing HTTPS_PROXY at
a closed port: nothing comes back where the direct call returns ports.
dnsx keeps no rate. It has -rl, most of its traffic goes to resolvers
rather than the target, and nothing here has measured what throttling it
costs in answers. It drops them silently, so that is a measurement to take
before adding the knob, not after.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5roHYDYzSPRNaTkn8hSkJ
32 lines
855 B
Python
32 lines
855 B
Python
import httpx
|
|
|
|
from shared.config import base_settings
|
|
|
|
MAX_RETRIES = 3
|
|
|
|
|
|
def _base_kwargs() -> dict:
|
|
settings = base_settings()
|
|
kwargs: dict = {
|
|
"timeout": httpx.Timeout(settings.EGRESS_TIMEOUT),
|
|
"headers": {"User-Agent": settings.EGRESS_USER_AGENT},
|
|
"follow_redirects": True,
|
|
}
|
|
if settings.EGRESS_PROXY_URL:
|
|
kwargs["proxy"] = settings.EGRESS_PROXY_URL
|
|
return kwargs
|
|
|
|
|
|
def get_async_client(**overrides) -> httpx.AsyncClient:
|
|
kwargs = _base_kwargs()
|
|
kwargs["transport"] = httpx.AsyncHTTPTransport(retries=MAX_RETRIES)
|
|
kwargs.update(overrides)
|
|
return httpx.AsyncClient(**kwargs)
|
|
|
|
|
|
def get_sync_client(**overrides) -> httpx.Client:
|
|
kwargs = _base_kwargs()
|
|
kwargs["transport"] = httpx.HTTPTransport(retries=MAX_RETRIES)
|
|
kwargs.update(overrides)
|
|
return httpx.Client(**kwargs)
|