theHarvester/docs/wiki/Virtual-Host-Discovery.md

Virtual host discovery

A web server can host several sites on one IP address. The site selected by the server depends on the HTTP Host header and, for HTTPS, the TLS Server Name Indication (SNI). Virtual host discovery checks whether an in-scope hostname produces a response that differs from the endpoint's default or wildcard response.

This is a P2 direct action. It sends requests to harvested IP addresses or to one literal-IP endpoint supplied by the operator. Use it only when the target, addresses, ports, and technique are covered by the assessment authorization.

What gets probed

--vhost uses the run's collected evidence:

• Literal IP addresses become endpoints. The sweep tries HTTPS port 443 across the address set before HTTP port 80.
• Harvested hostnames inside the exact target boundary become candidates.
• DNS brute-force IPs and names are included because virtual host discovery runs after DNS brute force.
• In-scope names found by reverse DNS become candidates. The reverse-DNS /24 range does not become a new set of virtual host endpoints.

This is harvested-candidate testing, not wordlist-based virtual host brute forcing. Repeat --vhost-candidate to add a small authorized list that the selected sources did not return.

For a normal harvested run, --vhost is the only virtual-host option you need. The request, runtime, timeout, and concurrency options are advanced safety overrides; bounded defaults apply when you omit them.

The shared request cap may stop the sweep before it reaches every endpoint. When that happens, the vhost action is partial with the stop reason request-limit; it does not claim complete coverage.

An explicit endpoint replaces the harvested endpoint set:

AUTHORIZED_DOMAIN='replace-with-a-domain-you-control'
AUTHORIZED_IP='replace-with-an-authorized-ip'

uv run theHarvester \
  -d "$AUTHORIZED_DOMAIN" \
  --vhost-endpoint "https://${AUTHORIZED_IP}:443/" \
  --vhost-candidate "admin.${AUTHORIZED_DOMAIN}" \
  -f report

The endpoint must use http or https, a literal IPv4 or IPv6 address, and an optional port. Paths, query strings, fragments, credentials, hostnames, and CIDR ranges are rejected.

Candidate names are never resolved by this feature. Each candidate must be the target hostname or a descendant of it. The boundary is exact: authorization for www.example.com does not authorize admin.example.com.

How the classifier works

For each endpoint, theHarvester makes a literal-IP context request and at least three synthetic unknown-host control requests. Controls match the label shape of the candidate being tested. This matters on servers whose wildcard behavior changes with label depth.

HTTPS candidate and control requests send the same hostname in TLS SNI and the HTTP Host header. HTTP requests send the hostname only in Host. Redirects are recorded as evidence but are not followed.

flowchart TD
    A[Collect hostnames and literal IPs] --> B[Keep candidates inside the exact target scope]
    A --> C[Build HTTPS endpoints, then HTTP endpoints]
    B --> D[Select the next endpoint and candidates that fit the shared cap]
    C --> D
    D --> E[Request the literal-IP context]
    D --> F[Request three unknown controls for each candidate shape]
    D --> G[Request the candidate with aligned SNI and Host]
    E --> H{Are both baselines usable and controls stable?}
    F --> H
    G --> H
    H -->|No| I[Indeterminate]
    H -->|Yes| J{Does the candidate match either baseline?}
    J -->|Yes| K[Default]
    J -->|No, status or redirect differs| L[Distinct]
    J -->|Only the body differs| M[Repeat the candidate request]
    M --> N{Does the body difference repeat?}
    N -->|Yes| L
    N -->|No| I
    L --> O[Store structured virtual host evidence]

Before comparison, the classifier replaces an exact reflection of the current authority in the response body or Location header. A generic error page that merely repeats the requested hostname should not become a discovery.

The comparison can use these signals:

Signal	Meaning
status	The HTTP status differs from both baselines.
location	The normalized redirect location differs.
body_size	The bounded response body has a different size.
body_sha256	The bounded response body has different content.

The classifier returns one of three states:

Classification	Meaning
default	The candidate matches either the literal-IP context response or a stable unknown-host control.
distinct	The candidate differs from both baselines. Body-only differences require a matching second candidate response.
indeterminate	A baseline is unusable, the controls disagree, the candidate matches only one baseline, body evidence is incomplete, or a required confirmation did not repeat.

Completed results retain only confirmed distinct observations. Default and indeterminate responses still consume budget, but they are not reported as discoveries.

Request and runtime limits

The defaults are:

Control	CLI option	Default
Requests across the whole sweep	--vhost-request-limit	100
Runtime across the whole sweep	--vhost-runtime-seconds	30 seconds
Timeout for one request	--vhost-timeout-seconds	5 seconds
Concurrent candidate requests	--vhost-concurrency	5

The request limit covers the context request, unknown controls, candidate requests, and any confirmation request. Unused requests and time from an early endpoint carry forward to later endpoints. A low cap favors breadth: HTTPS is attempted across harvested IPs before the sweep starts HTTP.

The vhost entry in action_executions records one of these action statuses:

Action status	Meaning
completed	Every selected endpoint and candidate completed within the limits.
partial	A request or runtime limit stopped coverage, or a request, scan, or cancellation error occurred after a confirmed hostname was retained.
skipped	No candidate names or literal-IP endpoints were available.
failed	A request, scan, or cancellation error ended the action before it retained a confirmed hostname.

Its stop_reason explains the outcome in more detail:

Stop reason	Meaning
completed	Every selected endpoint and candidate completed within the limits.
no-candidates	The run had no in-scope hostnames to test.
no-endpoints	The run had no harvested literal IP and no endpoint override.
request-limit	The cap omitted an endpoint or candidate, or stopped before confirmation finished.
request-errors	Every candidate was attempted, but one or more requests failed.
runtime-limit	The shared wall-clock deadline expired. Completed partial evidence is retained.
scan-error	An unexpected probing error stopped the action.
cancelled	The operator cancelled the run.

Cancellation requested by the operator is different from a runtime limit. It propagates through the worker lifecycle and closes active connections.

Basic CLI use

Harvest hostnames and IP addresses, then run the bounded sweep:

AUTHORIZED_DOMAIN='replace-with-a-domain-you-control'

uv run theHarvester \
  -d "$AUTHORIZED_DOMAIN" \
  -b rapiddns \
  --vhost \
  -f report

rapiddns is used here because it can return both hostnames and literal IP addresses. If the selected sources return only hostnames, enable a DNS action that contributes IP evidence or supply --vhost-endpoint.

Repeat --vhost-candidate to add names that are already authorized but were not returned by the selected sources:

uv run theHarvester \
  -d "$AUTHORIZED_DOMAIN" \
  -b rapiddns \
  --vhost \
  --vhost-candidate "admin.${AUTHORIZED_DOMAIN}" \
  --vhost-candidate "preview.${AUTHORIZED_DOMAIN}"

Virtual host discovery uses direct transport. It rejects --proxies. HTTPS certificate verification is enabled by default. --vhost-insecure disables verification and records tls_verified: false in the evidence. Use that option only when the engagement requires it and the endpoint is authorized.

Reading terminal output

The summary reports confirmed endpoint observations and coverage:

[*] Virtual hosts: confirmed=1; candidate-endpoints=18/48; endpoints=4/8; requests=100
[!] Coverage stopped at the request limit; raise --vhost-request-limit or narrow the scan.
admin.authorized.example at https://192.0.2.10:443/: status=401; signals=status

A candidate-endpoint is one hostname tested against one IP endpoint. This pair count avoids implying that a hostname tested against one IP was also tested against every other IP. When coverage stops early, the next line explains which limit was reached and which option controls it.

HarvestView and REST API

HarvestView exposes a virtual host discovery checkbox plus optional endpoint and candidate inputs. Advanced safety controls contain the request, runtime, timeout, concurrency, and certificate-verification overrides. Leaving the endpoint blank uses harvested IPs.

The same run can be submitted to POST /api/v1/runs:

{
  "target": "authorized.example",
  "sources": ["rapiddns"],
  "vhost": true
}

Add vhost_endpoint or vhost_candidates only when harvested evidence does not supply them. The request, runtime, timeout, concurrency, and insecure-TLS fields override the same bounded defaults shown above.

Supplying vhost_endpoint or vhost_candidates enables the action even when vhost is omitted. A run with sources: [] must supply both an endpoint and at least one candidate; otherwise the missing side must come from harvested results. The API rejects proxy use, hostname endpoints, out-of-scope candidates, and IP targets before it queues a run.

One finding, multiple endpoint observations

JSONL is unversioned. After the summary line, it stores one canonical finding for each confirmed hostname. The normal value field contains the hostname, actions records vhost provenance, and observations is a native array rather than JSON hidden inside a string.

If a hostname is distinct on two endpoints, both endpoint records stay under that one hostname finding:

{
  "type": "hostname",
  "value": "admin.authorized.example",
  "sources": [],
  "actions": ["vhost"],
  "observations": [
    {
      "endpoint": "https://192.0.2.10:443/",
      "http_host": "admin.authorized.example",
      "tls_server_name": "admin.authorized.example",
      "classification": "distinct",
      "phase": "body",
      "status": 401,
      "location": null,
      "body_sha256": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
      "body_size": 123,
      "body_truncated": false,
      "context_phase": "body",
      "context_status": 200,
      "context_location": null,
      "context_body_sha256": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
      "context_body_size": 123,
      "context_body_truncated": false,
      "control_phase": "body",
      "control_status": 200,
      "control_location": null,
      "control_body_sha256": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
      "control_body_size": 123,
      "control_body_truncated": false,
      "confirmation_body_sha256": null,
      "tls_verified": true,
      "distinct_signals": ["status"],
      "reflection_normalized": false
    },
    {
      "endpoint": "https://192.0.2.11:443/",
      "http_host": "admin.authorized.example",
      "tls_server_name": "admin.authorized.example",
      "classification": "distinct",
      "phase": "body",
      "status": 403,
      "location": null,
      "body_sha256": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
      "body_size": 87,
      "body_truncated": false,
      "context_phase": "body",
      "context_status": 404,
      "context_location": null,
      "context_body_sha256": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
      "context_body_size": 87,
      "context_body_truncated": false,
      "control_phase": "body",
      "control_status": 404,
      "control_location": null,
      "control_body_sha256": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
      "control_body_size": 87,
      "control_body_truncated": false,
      "confirmation_body_sha256": null,
      "tls_verified": true,
      "distinct_signals": ["status"],
      "reflection_normalized": false
    }
  ]
}

The context_* fields contain the literal-IP response, while control_* contains the stable unknown-host response. A distinct candidate must differ from both. confirmation_body_sha256 is present only when a repeated response confirmed a body-only difference. These fields let a reviewer verify the recorded distinct_signals without retaining every synthetic control request. The source list is empty because virtual host discovery is an action, not a passive source.

SQLite uses the same shape without inventing another evidence concept: results holds one hostname row, result_origins links it to the vhost execution, and that result's details hold the endpoint observation array. JSONL export and API run details expose the array as native JSON. vhost is an action name, not a result kind.

Legacy JSON and XML keep their existing virtual host name lists. They do not carry the endpoint and response evidence. Use JSONL or GET /api/v1/runs/{run_id} when automation needs the structured observations.

Troubleshooting

The run says no-candidates

Select a source that returns hostnames or add an authorized name with --vhost-candidate. Check the exact target boundary if the run targets a hostname such as www.example.com.

The run says no-endpoints

Select sources that can return IP addresses, enable a DNS action that contributes literal IP evidence, or supply --vhost-endpoint.

The run says request-limit

The cap was too small for every endpoint, candidate shape, and confirmation request. Narrow the source set or candidate list before raising the cap. The terminal summary shows attempted endpoints as attempted/total.

A candidate is missing from the results

Only confirmed distinct observations are retained. A candidate may have matched the default response, produced unstable controls, failed during transport, returned a truncated body, or failed a body-only confirmation.

HTTPS results are indeterminate

Certificate verification or TLS negotiation may have failed before the server returned HTTP evidence. Check whether the certificate is valid for the candidate SNI. If the assessment permits unverified TLS, rerun the bounded test with --vhost-insecure and keep the recorded verification state with the result.

Related pages

• Responsible Use and Scope
• Operator Workflows
• Results and Local Data
• REST API
• Troubleshooting