Skip to main content

GitHub Actions QA Pipeline

Trigger — manual only​

on:
# schedule:
# - cron: "0 0 * * 1-5"
workflow_dispatch:
inputs:
run_e2e: { type: boolean, default: false }
run_axe: { type: boolean, default: false }
run_seo: { type: boolean, default: false }
run_lhci_perfo: { type: boolean, default: false }
run_visual: { type: boolean, default: false }
base_url: { type: string, required: false, default: "" }
groups: { type: string, required: false, default: auto }
  • Only workflow_dispatch is active — nothing runs on its own until someone (or a cron) starts it.
  • Every suite defaults to false — an empty checkbox set runs the shell, not a job.
  • Empty base_url → falls back to the repo/environment variable BASE_URL (Settings → Actions → Variables).
  • groups picks the test groups to run, comma-separated (homepage,auth). auto runs what ENV_GROUPS in qa/tests/playwright/helpers/test-groups.ts lists for the target environment. The workflow exports it as QA_GROUPS, which turbo.json lists in globalEnv so Turborepo passes it to the test task. A suite with no test for the picked groups passes with zero tests.
  • run-name is built from the checked inputs and the groups ([E2E, AXE] QA – https://staging... – auto) — a scannable run list.

Job structure​

  • 5 jobs: e2e, axe, seo, visual, lhci_perfo.
  • Each gated by if: ${{ inputs.run_X }}, no needs: between them → they run in parallel.
  • Unchecked = skipped, not failed.
  • Shared steps: checkout → setup-pnpm → cache/install Playwright → the same pnpm tests:playwright:... command as locally → upload report on failure.

Concurrency​

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true
  • On a PR: push new code → the CI run already going for that PR gets killed, the new one takes over. No point finishing a check on code you just replaced.
  • Manual runs: always independent. Click "Run workflow" twice → both run to the end, neither one cancels the other.

Artifacts on failure​

JobUploadsNote
e2eplaywright-report, playwright-tracestrace = trace.zip, open via playwright show-trace
axeaxe-reporttarball
seoseo-reporttarball
visualvisual-reportonly if a comparison actually ran
lhci_perfoperfo-reportif: always() — even on a green run
  • retention-days: 7, if-no-files-found: warn — nothing to upload ≠ failure.

The visual job's extra step​

  • First looks for committed *.png under visual.spec.ts-snapshots.
  • Found nothing → skip the run ("no baseline yet" ≠ failure).
  • Found some → cache/pull a pinned Playwright Docker image (version must match package.json) → tests:playwright:visual:docker.

Running it on demand​

  1. Actions → QA → Run workflow.
  2. Pick the branch — determines the version of qa.yml and the test code.
  3. Check the suites, fill in base_url, and leave groups on auto or list the groups to run.
  4. Run, watch in parallel, artifacts from the run summary.

Cron schedule (commented out)​

# schedule:
# - cron: "0 0 * * 1-5"
  • 00:00 UTC on weekdays.
  • Uncommenting alone isn't enough — if: inputs.run_X is always false outside workflow_dispatch.
  • Each job additionally needs: if: ${{ inputs.run_e2e || github.event_name == 'schedule' }}.
  • run-name already handles "SCHEDULED" on its own.

Decide before enabling:

  • Which suites run nightly (not the whole Visual+Lighthouse combo — slowest).
  • vars.BASE_URL must point at something stable (no one types it in by hand).
  • How a failure gets noticed (the default email is easy to miss → Slack/Teams).
  • Multiple cron: entries are fine (nightly smoke + weekly full run).
  • Cron always targets the default branch, never a PR branch.

Quick recap​

TriggerStarted byRuns againstNeeds
Manual (workflow_dispatch)Anyone with write accessChosen branch + base_urlWorks today
CronGitHub, unattendedDefault branch, vars.BASE_URLUncomment + update if: in each job