mirror of
https://github.com/affaan-m/ECC.git
synced 2026-09-20 16:47:59 +02:00
fix(hooks): stop probing the pytest override, and stop failing on exit 5
Three defects, found by reviewing this branch against a running pytest rather than by reading it. Exit 5 is not a failure. pytest reserves it for NO_TESTS_COLLECTED, and `|| fail "pytest failed"` collapsed it into a blocked push. The `|| fail` predates this branch, but this branch is what makes it reachable: a repository whose pyproject.toml only configures ruff or black, with pytest in its venv and no test files, used to hit the "pytest is not installed" skip and now gets gated. $VIRTUAL_ENV is the first candidate, so merely having a venv activated in the pushing shell drags any requirements.txt repository into this path, and the hook is installed globally. Reproduced with pytest 9.1.1. Exit 5 is now non-blocking but loud -- a bad rootdir, testpaths or an unimportable conftest also collects nothing, and swallowing that silently would reopen the hole this resolver exists to close. Other non-zero codes now carry the code, because 1 (tests failed) and 4 (usage error) call for different responses. The ECC_PYTEST_CMD probe ran the operator's command. Validating the override with `--version` assumed it would answer like pytest. A wrapper that sets an environment variable and execs pytest ignores the flag and runs the whole suite, so the probe executed the tests, then rejected the command for not printing a version, then blocked the push -- with the suite green. That is worse than the silent gate the probe was added to close, so the override is taken as given again: it is a deliberate setting, the hook cannot inspect it without running it, and pointing it at something that is not pytest is the operator's call. `is_pytest` still guards the PATH candidate, which this script composes itself, where `pytest --version` is harmless. An empty override still fails closed. The tests inherited the ambient environment. `runHermeticPythonPrePush` passed process.env through, so an exported ECC_PYTEST_CMD or an activated virtualenv resolved a pytest the fixture never created and the venv test failed for anyone who runs the suite that way. Both variables are now neutralised in the base env. Coverage: the gate had no test proving it blocks. Changing the run line to `|| true` left all three previous tests green. Seven now cover a spaced venv path, a red suite, exit 5, an override invoked exactly once with no probe, an empty override, and the PATH candidate in both directions.
This commit is contained in:
@@ -136,6 +136,11 @@ PYTEST_CMD=()
|
||||
# plenty of programs take it and exit 0 -- so the output has to name pytest. The
|
||||
# version is captured rather than piped: under `set -o pipefail` a `| grep -q` can
|
||||
# report the SIGPIPE of the program it just matched.
|
||||
#
|
||||
# Only ever called on a command this script composed itself. Probing an arbitrary
|
||||
# operator-supplied command is not safe: a wrapper that ignores `--version` and
|
||||
# execs pytest runs the entire suite during the probe, and is then rejected for
|
||||
# not having printed a version.
|
||||
is_pytest() {
|
||||
local version
|
||||
version="$("$@" --version 2>&1)" || return 1
|
||||
@@ -144,17 +149,15 @@ is_pytest() {
|
||||
|
||||
resolve_pytest() {
|
||||
if [[ -n "${ECC_PYTEST_CMD:-}" ]]; then
|
||||
# Word-split, so the override names a command on PATH or an interpreter whose
|
||||
# path has no spaces; a venv with spaces in its name is found by the loop below.
|
||||
# Taken as given. This is a deliberate override, and the hook cannot inspect it
|
||||
# without running it -- a wrapper script may ignore `--version` and run the
|
||||
# suite, so probing costs a duplicate test run and then blocks the push anyway.
|
||||
# Pointing this at something that is not pytest turns the gate off, and that is
|
||||
# the operator's call to make, not a misconfiguration for the hook to second
|
||||
# guess. Word-split, so the command names something on PATH or an interpreter
|
||||
# whose path has no spaces; a venv with spaces is found by the loop below.
|
||||
read -r -a PYTEST_CMD <<<"$ECC_PYTEST_CMD" || true
|
||||
# Checked like every other candidate, and fatally rather than by falling
|
||||
# through: an operator who set this asked for that command, and quietly running
|
||||
# a different one would hide the misconfiguration. `ECC_PYTEST_CMD=true` would
|
||||
# otherwise run `true -q`, pass, and report a Python project verified by
|
||||
# nothing -- the same silent gate this resolver exists to remove.
|
||||
if [[ ${#PYTEST_CMD[@]} -eq 0 ]] || ! is_pytest "${PYTEST_CMD[@]}"; then
|
||||
fail "ECC_PYTEST_CMD is set to '$ECC_PYTEST_CMD', which does not run pytest"
|
||||
fi
|
||||
[[ ${#PYTEST_CMD[@]} -gt 0 ]] || fail "ECC_PYTEST_CMD is set but empty"
|
||||
return 0
|
||||
fi
|
||||
local venv
|
||||
@@ -178,8 +181,8 @@ resolve_pytest() {
|
||||
return 0
|
||||
fi
|
||||
fi
|
||||
# `command -v` proves only that a file of that name exists on PATH, which is why
|
||||
# this candidate is confirmed too before it is accepted.
|
||||
# `command -v` proves only that a file of that name exists on PATH. This one the
|
||||
# script composed itself, so confirming it costs a harmless `pytest --version`.
|
||||
if command -v pytest >/dev/null 2>&1 && is_pytest pytest; then
|
||||
PYTEST_CMD=(pytest)
|
||||
return 0
|
||||
@@ -192,7 +195,25 @@ if [[ -f "pyproject.toml" || -f "requirements.txt" ]]; then
|
||||
if resolve_pytest; then
|
||||
ran_any_check=1
|
||||
log "Python project detected. Running: ${PYTEST_CMD[*]} -q"
|
||||
"${PYTEST_CMD[@]}" -q || fail "pytest failed"
|
||||
pytest_status=0
|
||||
"${PYTEST_CMD[@]}" -q || pytest_status=$?
|
||||
case "$pytest_status" in
|
||||
0) ;;
|
||||
# pytest reserves 5 for NO_TESTS_COLLECTED, which is not a red suite. A
|
||||
# pyproject.toml that only configures ruff or black is still a Python project
|
||||
# by this hook's test, and blocking those pushes would make the gate something
|
||||
# people switch off. Never silent, though: a bad rootdir, testpaths or a
|
||||
# conftest that fails to import also collects nothing, and swallowing that is
|
||||
# the same skip-reads-like-a-pass hole this resolver exists to close.
|
||||
5)
|
||||
log "pytest collected no tests (exit 5). Not gating this push."
|
||||
log " If this repository is supposed to have tests, that is the bug:"
|
||||
log " check rootdir, testpaths, and conftest.py import errors."
|
||||
;;
|
||||
# The code is in the message because 1 (tests failed) and 4 (usage error)
|
||||
# need different responses, and "pytest failed" alone cannot tell them apart.
|
||||
*) fail "pytest failed (exit $pytest_status)" ;;
|
||||
esac
|
||||
else
|
||||
log "Python project detected but no pytest found (checked \$VIRTUAL_ENV, .venv,"
|
||||
log " venv, env, uv, poetry, PATH). Set ECC_PYTEST_CMD to point at it."
|
||||
|
||||
Reference in New Issue
Block a user