Run Skeg in CI¶

CI replays a tag of stored plans against a device on the CI machine. The runner exits non-zero when any plan fails, so the job fails with it.

What the CI machine needs¶

The runner CLI (skeg-runner). The Skeg Runner app installs a launcher at ~/.config/skeg/skeg-runner. For a headless CI image, ask the Skeg team for the CLI build.
A device: an Android emulator on Linux or macOS, or an iOS Simulator on macOS.
Your app installed on that device, or --app-path pointing at the build.
A dedicated CI account in Skeg, invited to the project. Sign in to the website once as that account and accept the beta terms.

Authenticate without exposing the password¶

Project tokens (Settings → Agent tokens) authenticate the MCP and doctor modes only. Run mode uses an account. Do not pass --password on the command line: any local user can read a process's arguments. Log in once, then hand the runner the session token through SKEG_RUNNER_TOKEN. The token lasts 24 hours, which covers any CI job.

Five logins per minute per account

Skeg allows 5 login attempts per account per minute, successful ones included. A matrix build or a burst of pull requests that all log in as one CI account gets HTTP 429 from the 6th login on. Keep concurrent logins under the limit: log in once per job rather than per step, serialize the Skeg jobs with a concurrency group, or give each concurrent pipeline its own CI account. Each login also replaces the account's refresh token. Tokens already handed to running jobs keep working, but a browser signed in as the CI account can no longer refresh its session, so do not use the CI account in a browser.

# jq reads the exported credentials from its environment and curl reads the body from stdin,
# so the password is in no process's arguments. pipefail makes a refused login fail here,
# not later as a confusing auth error in the runner.
set -o pipefail
export SKEG_CI_EMAIL SKEG_CI_PASSWORD
SKEG_RUNNER_TOKEN=$(jq -n '{email: env.SKEG_CI_EMAIL, password: env.SKEG_CI_PASSWORD}' \
  | curl -fsS https://api.getskeg.dev/auth/login -H 'Content-Type: application/json' --data-binary @- \
  | jq -er .token)
export SKEG_RUNNER_TOKEN

Tag the suite¶

In Skeg, give the plans to replay a tag such as smoke. Only enabled plans run. Every plan in one tag must target the same platform. If their default environments differ, pass --env.

Example: GitHub Actions on Android¶

name: Skeg smoke
on: pull_request

jobs:
  smoke:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build debug APK
        run: ./gradlew assembleDebug
      # Install skeg-runner on the runner image here (download step omitted).
      - name: Log in to Skeg
        env:
          SKEG_CI_EMAIL: ${{ secrets.SKEG_CI_EMAIL }}
          SKEG_CI_PASSWORD: ${{ secrets.SKEG_CI_PASSWORD }}
        run: |
          set -o pipefail
          token=$(jq -n '{email: env.SKEG_CI_EMAIL, password: env.SKEG_CI_PASSWORD}' \
            | curl -fsS https://api.getskeg.dev/auth/login -H 'Content-Type: application/json' --data-binary @- \
            | jq -er .token) || { echo "::error::Skeg login failed. curl's line above has the HTTP status: 401 means wrong credentials, 429 the 5-logins-per-minute limit."; exit 1; }
          echo "::add-mask::$token"
          echo "SKEG_RUNNER_TOKEN=$token" >> "$GITHUB_ENV"
      # Ubuntu runners need KVM access for the emulator (android-emulator-runner README).
      - name: Enable KVM
        run: |
          echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
          sudo udevadm control --reload-rules
          sudo udevadm trigger --name-match=kvm
      - name: Replay the smoke suite on an emulator
        uses: reactivecircus/android-emulator-runner@v2
        with:
          api-level: 34
          script: >
            skeg-runner --mode run
            --server https://api.getskeg.dev
            --project "${{ vars.SKEG_PROJECT_ID }}" --tag smoke
            --platform ANDROID --app-id com.example.app
            --app-path app/build/outputs/apk/debug/app-debug.apk
            --device emulator-5554

The Enable KVM step comes from the emulator action's README. Without it the emulator cannot use hardware acceleration on Ubuntu runners.

The runner prints one line per plan (<planId>: pass (pass=12 fail=0 skip=0)). Each run also appears in Skeg's Runs with origin CI, with every step and screenshot.

Exit codes¶

Code	Meaning
`0`	Every plan passed.
`1`	A plan failed, ended as an infrastructure error, or its report did not reach the server. Also `skeg-runner` called with no arguments (it prints usage).
`2`	Bad arguments the parser rejects, or an unknown mode.
`3`	The runner failed. This includes an invalid tag suite: plans that mix platforms, do not match `--platform`, or mix default environments without `--env`, and `--run-id` or `--requires` combined with `--tag`. Read the `runner failed:` line for which.

Not available yet¶

PR status checks and annotations posted by Skeg.
Automatic retry of flaky plans.