Files
algo/.github/workflows/integration-tests.yml
T
ba7297268a Simplify codebase: modernize loops, split templates, improve CI (#14889)
* Simplify codebase: modernize loops, split templates, improve CI

This PR consolidates several simplification phases:

## Ansible Modernization
- Modernize `with_items` to `loop` across ~50 task files
- Add OS detection facts (is_ubuntu, os_family_lowercase)
- Condense inline YAML syntax where appropriate

## Template Splitting
- Split 568-line dnscrypt-proxy.toml.j2 into focused partials:
  - global.toml.j2 (core settings)
  - sources.toml.j2 (resolver sources)
  - filters.toml.j2 (blocking rules)
  - cache.toml.j2 (caching config)

## CI Workflow Improvements
- Create setup-algo composite action for shared CI setup
- Re-enable integration tests with health checks
- Fix smart-tests.yml silent lint failures (remove || true)
- Use env variables for GitHub SHAs (security)

## server.yml Async Simplification
- Reorganize VPN service configuration with clear sections
- Add performance_parallel_services toggle
- Simplify status display from json_query to inline conditionals
- Keep services explicit for readability

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix with_items to loop conversion: preserve list flattening

with_items automatically flattens nested lists, but loop does NOT.
The mechanical conversion broke iteration over list variables.

Wrong:
  loop:
    - "{{ users }}"  # ['alice', 'bob'] treated as ONE item

Fixed:
  loop: "{{ users }}"  # Iterates over alice, bob correctly

For combined lists (users + server):
  loop: "{{ users + [IP_subject_alt_name] }}"

Fixes IPsec certificate generation creating files named literally
'['alice', 'bob'].key' instead of separate alice.key and bob.key.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix integration test: use strongswan-starter service name on Ubuntu 20.04+

The StrongSwan service is named 'strongswan-starter' on Ubuntu 20.04+,
not 'strongswan'. The test was checking the wrong service name, causing
false failures even when StrongSwan was actually running.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix IPsec path issues: remove trailing slashes and fix test paths

1. Remove trailing slashes from ipsec_config_path and ipsec_pki_path
   in roles/strongswan/defaults/main.yml (causes double slashes)

2. Fix integration test to check correct subdirectories:
   - .p12 files are in ipsec/manual/
   - .mobileconfig files are in ipsec/apple/

3. Fix strongswan service name check (strongswan-starter on Ubuntu 20.04+)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-28 04:28:59 -05:00

280 lines
9.5 KiB
YAML

---
name: Integration Tests
'on':
pull_request:
types: [opened, synchronize, reopened]
paths:
- 'main.yml'
- 'roles/**'
- 'playbooks/**'
- 'library/**'
workflow_dispatch:
schedule:
- cron: '0 2 * * 1' # Weekly on Monday at 2 AM
permissions:
contents: read
jobs:
localhost-deployment:
name: Localhost VPN Deployment Test
runs-on: ubuntu-22.04
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
vpn_type: ['wireguard', 'ipsec', 'both']
steps:
- uses: actions/checkout@c2d88d3ecc89a9ef08eebf45d9637801dcee7eb5 # v5.0.1
with:
persist-credentials: false
- uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6.1.0
with:
python-version: '3.11'
cache: 'pip'
- name: Install system dependencies
run: |
sudo apt-get update
sudo apt-get install -y \
wireguard \
wireguard-tools \
strongswan \
libstrongswan-standard-plugins \
dnsmasq \
qrencode \
openssl \
linux-headers-$(uname -r)
- name: Install uv
run: curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Install Python dependencies
run: uv sync
- name: Create test configuration
run: |
cat > integration-test.cfg << EOF
users:
- alice
- bob
cloud_providers:
local:
server: localhost
endpoint: 127.0.0.1
wireguard_enabled: ${{ matrix.vpn_type == 'wireguard' || matrix.vpn_type == 'both' }}
ipsec_enabled: ${{ matrix.vpn_type == 'ipsec' || matrix.vpn_type == 'both' }}
dns_adblocking: true
ssh_tunneling: false
store_pki: true
algo_provider: local
algo_server_name: github-ci-test
server: localhost
algo_ssh_port: 22
CA_password: "test-ca-password-${{ github.run_id }}"
p12_export_password: "test-p12-password-${{ github.run_id }}"
tests: true
no_log: false
ansible_connection: local
dns_encryption: true
algo_dns_adblocking: true
algo_ssh_tunneling: false
BetweenClients_DROP: true
block_smb: true
block_netbios: true
pki_in_tmpfs: true
endpoint: 127.0.0.1
ssh_port: 4160
local_service_ip: 172.16.0.1
local_service_ipv6: "fd00::1"
EOF
- name: Run Algo deployment
run: |
# Run ansible-playbook via uv - become: true in playbook handles root
# GitHub runners have passwordless sudo for become escalation
uv run ansible-playbook main.yml \
-i "localhost," \
-c local \
-e @integration-test.cfg \
-e "provider=local" \
-vv
- name: Verify services are running
run: |
# Check WireGuard
if [[ "${{ matrix.vpn_type }}" == "wireguard" || "${{ matrix.vpn_type }}" == "both" ]]; then
echo "Checking WireGuard..."
sudo wg show
if ! sudo systemctl is-active --quiet wg-quick@wg0; then
echo "✗ WireGuard service not running"
exit 1
fi
echo "✓ WireGuard is running"
fi
# Check StrongSwan (service name is strongswan-starter on Ubuntu 20.04+)
if [[ "${{ matrix.vpn_type }}" == "ipsec" || "${{ matrix.vpn_type }}" == "both" ]]; then
echo "Checking StrongSwan..."
sudo ipsec statusall
if ! sudo systemctl is-active --quiet strongswan-starter; then
echo "✗ StrongSwan service not running"
exit 1
fi
echo "✓ StrongSwan is running"
fi
# Check dnsmasq
if ! sudo systemctl is-active --quiet dnsmasq; then
echo "⚠️ dnsmasq not running (may be expected)"
else
echo "✓ dnsmasq is running"
fi
# Check dnscrypt-proxy
if sudo systemctl is-active --quiet dnscrypt-proxy; then
echo "✓ dnscrypt-proxy is running"
else
echo "⚠️ dnscrypt-proxy not running"
fi
# DNS health check - verify DNS resolution works
echo "Testing DNS resolution via local_service_ip (172.16.0.1)..."
if dig @172.16.0.1 google.com +short +timeout=5 | grep -q .; then
echo "✓ DNS resolution working"
else
echo "⚠️ DNS resolution failed (service may still be starting)"
fi
- name: Verify generated configs
run: |
echo "Checking generated configuration files..."
# WireGuard configs
if [[ "${{ matrix.vpn_type }}" == "wireguard" || "${{ matrix.vpn_type }}" == "both" ]]; then
for user in alice bob; do
if [ ! -f "configs/localhost/wireguard/${user}.conf" ]; then
echo "✗ Missing WireGuard config for ${user}"
exit 1
fi
if [ ! -f "configs/localhost/wireguard/${user}.png" ]; then
echo "✗ Missing WireGuard QR code for ${user}"
exit 1
fi
done
echo "✓ All WireGuard configs generated"
fi
# IPsec configs (p12 in manual/, mobileconfig in apple/)
if [[ "${{ matrix.vpn_type }}" == "ipsec" || "${{ matrix.vpn_type }}" == "both" ]]; then
for user in alice bob; do
if [ ! -f "configs/localhost/ipsec/manual/${user}.p12" ]; then
echo "✗ Missing IPsec certificate for ${user}"
exit 1
fi
if [ ! -f "configs/localhost/ipsec/apple/${user}.mobileconfig" ]; then
echo "✗ Missing IPsec mobile config for ${user}"
exit 1
fi
done
echo "✓ All IPsec configs generated"
fi
- name: Test VPN connectivity
run: |
echo "Testing basic VPN connectivity..."
# Test WireGuard
if [[ "${{ matrix.vpn_type }}" == "wireguard" || "${{ matrix.vpn_type }}" == "both" ]]; then
# Get server's WireGuard public key
SERVER_PUBKEY=$(sudo wg show wg0 public-key)
echo "Server public key: $SERVER_PUBKEY"
# Check if interface has peers
PEER_COUNT=$(sudo wg show wg0 peers | wc -l)
echo "✓ WireGuard has $PEER_COUNT peer(s) configured"
fi
# Test StrongSwan
if [[ "${{ matrix.vpn_type }}" == "ipsec" || "${{ matrix.vpn_type }}" == "both" ]]; then
# Check IPsec policies
sudo ipsec statusall | grep -E "INSTALLED|ESTABLISHED" || echo "No active IPsec connections (expected)"
fi
- name: Upload configs as artifacts
if: always()
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
with:
name: vpn-configs-${{ matrix.vpn_type }}-${{ github.run_id }}
path: configs/
retention-days: 7
- name: Upload logs on failure
if: failure()
run: |
echo "=== Network Interfaces ==="
ip addr || true
echo "=== Listening Ports ==="
sudo ss -tulnp || true
echo "=== WireGuard Status ==="
sudo wg show || true
echo "=== IPsec Status ==="
sudo ipsec statusall || true
echo "=== DNS Services ==="
sudo systemctl status dnscrypt-proxy dnscrypt-proxy.socket dnsmasq --no-pager || true
echo "=== WireGuard Log ==="
sudo journalctl -u wg-quick@wg0 -n 50 --no-pager || true
echo "=== StrongSwan Log ==="
sudo journalctl -u strongswan -n 50 --no-pager || true
echo "=== dnscrypt-proxy Log ==="
sudo journalctl -u dnscrypt-proxy -n 50 --no-pager || true
echo "=== System Log (last 100 lines) ==="
sudo journalctl -n 100 --no-pager || true
docker-build-test:
name: Docker Image Build Test
runs-on: ubuntu-22.04
timeout-minutes: 10
steps:
- uses: actions/checkout@c2d88d3ecc89a9ef08eebf45d9637801dcee7eb5 # v5.0.1
with:
persist-credentials: false
- name: Build Algo Docker image
run: |
docker build -t algo:ci-test .
- name: Test Docker image
run: |
# Test that the image can run and show help
docker run --rm --entrypoint /bin/sh algo:ci-test -c "cd /algo && ./algo --help" || true
# Test that required binaries exist in the virtual environment
docker run --rm --entrypoint /bin/sh algo:ci-test -c "cd /algo && uv run which ansible"
docker run --rm --entrypoint /bin/sh algo:ci-test -c "which python3"
docker run --rm --entrypoint /bin/sh algo:ci-test -c "which rsync"
- name: Test Docker config validation
run: |
# Create a minimal valid config
mkdir -p test-data
cat > test-data/config.cfg << 'EOF'
users:
- test-user
cloud_providers:
ec2:
size: t3.micro
region: us-east-1
wireguard_enabled: true
ipsec_enabled: false
dns_encryption: true
algo_provider: ec2
EOF
# Test that config is readable
docker run --rm --entrypoint cat -v $(pwd)/test-data:/data algo:ci-test /data/config.cfg
echo "✓ Docker image built and basic tests passed"