mirror of
https://github.com/yogeshojha/rengine.git
synced 2026-10-20 23:31:31 +02:00
Yogesh OjhaandClaude Opus 5
5cb5d03a9f
Every place that accepts user-supplied YAML — engine import, an engine's stored yaml_source, nuclei template upload and report theme upload — could be handed a billion-laughs document. Measured on engine import: 379 bytes of nested aliases blocked the event loop for 36 seconds. A request issued alongside it timed out at 20s and the next took 13s, against a 4ms baseline. Any authenticated user could do it, repeatedly. The cost is not in the parser. PyYAML shares an aliased node rather than copying it, so safe_load returns in 2ms; the expansion happens in whatever walks the result as a tree, which for us is pydantic — it has no node cache, so nine levels of nine aliases is 43 million visits. Bounding the parser would have missed this entirely. load_document() now measures the document the way a consumer will walk it and stops at 100,000 values, which is four times the largest thing reNgine ships (a 24,133-node nuclei template, still indexed in 0.67s) and far below the bomb. All four call sites use it and each reports the limit in its own idiom — an HTTP 400, a template rejection reason, a theme error. Same payload after: 0.03s and refused, with API latency during the attack at 36ms. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K6LcH8uUddjJbKrahf34ys
45 lines
1.5 KiB
Python
45 lines
1.5 KiB
Python
"""Parse user-supplied YAML without letting aliases expand into a denial of service."""
|
|
|
|
from typing import Any
|
|
|
|
import yaml
|
|
|
|
# The largest document reNgine ships is a 24k-node nuclei template; this leaves headroom
|
|
# while stopping a payload whose aliases expand to tens of millions of nodes.
|
|
MAX_DOCUMENT_NODES = 100_000
|
|
|
|
|
|
class DocumentTooLargeError(ValueError):
|
|
pass
|
|
|
|
|
|
def document_size(data: Any, *, limit: int = MAX_DOCUMENT_NODES) -> int:
|
|
"""Count nodes as a consumer walking the value would see them, stopping at the limit."""
|
|
stack = [data]
|
|
seen = 0
|
|
while stack:
|
|
node = stack.pop()
|
|
seen += 1
|
|
if seen > limit:
|
|
msg = f"Document expands to more than {limit} values."
|
|
raise DocumentTooLargeError(msg)
|
|
if isinstance(node, dict):
|
|
for key, value in node.items():
|
|
stack.append(key)
|
|
stack.append(value)
|
|
elif isinstance(node, list | tuple):
|
|
stack.extend(node)
|
|
return seen
|
|
|
|
|
|
def load_document(text: str, *, limit: int = MAX_DOCUMENT_NODES) -> Any:
|
|
"""safe_load plus an expanded-size budget.
|
|
|
|
PyYAML shares an aliased node rather than copying it, so parsing a billion-laughs
|
|
payload is fast and small — the cost lands on whatever walks the result as a tree,
|
|
which for us is pydantic. Measure the walk here, once, before handing it on.
|
|
"""
|
|
data = yaml.safe_load(text)
|
|
document_size(data, limit=limit)
|
|
return data
|