CI/CD Integration

This page covers running Donobu tests in automated pipelines and exporting results in different formats.

GitHub Actions: complete workflow

name: E2E Tests

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest

    # Lets the heal agent push its fix branch and open a pull request.
    permissions:
      contents: write
      pull-requests: write

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: '24'
          cache: 'npm'

      - name: Install dependencies
        env:
          DONOBU_API_KEY: ${{ secrets.DONOBU_API_KEY }}
        run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install --with-deps

      - name: Run tests
        env:
          DONOBU_API_KEY: ${{ secrets.DONOBU_API_KEY }}
          GH_TOKEN: ${{ github.token }}
        run: npx donobu test --auto-heal

      - name: Generate Test Reports
        if: always()
        run: |
          npm exec playwright-json-to-markdown test-results/playwright-report.json >> $GITHUB_STEP_SUMMARY
          npm exec playwright-json-to-html test-results/playwright-report.json

      - name: Save Test Reports as Artifacts
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: test-results
          path: test-results/
          retention-days: 14

DONOBU_API_KEY does two jobs here. The install step uses it to read @donobu/test from Donobu's private registry (see Installation & Setup). The test step uses it for AI calls and for the heal agent.

With --auto-heal, failing tests go to the heal agent. It diagnoses each failure, fixes test-side causes, and Donobu re-runs the affected tests to verify the fix. On a push to a branch, the agent then publishes the verified fix as one pull request from donobu/heal/<branch> into that branch. Runs triggered by a pull request keep the fix in the working tree and open nothing. See Self-Healing Tests for details.

The heal agent publishes with the GitHub CLI (gh), which GitHub's hosted runners already have. It needs:

• GH_TOKEN on the test step. The workflow's own token (github.token) works.
• contents: write and pull-requests: write permissions on the job.

Without heal pull requests

To keep self-healing but never open a pull request, set DONOBU_HEAL_PULL_REQUEST to off and leave out the token and the write permissions. The fix stays in the run's working tree and its artifacts:

- name: Run tests
  env:
    DONOBU_API_KEY: ${{ secrets.DONOBU_API_KEY }}
    DONOBU_HEAL_PULL_REQUEST: 'off'
  run: npx donobu test --auto-heal

The test result files (JSON reports and related data) are saved under test-results/. GitHub summaries can be derived from these using playwright-json-to-markdown. The heal agent's session logs go to the hidden folder test-results/.donobu-heal/, which actions/upload-artifact skips unless you set include-hidden-files: true.

Storing API keys as secrets

Add your AI provider key as a repository or organisation secret in GitHub:

Settings → Secrets and variables → Actions → New repository secret

Reference them in the workflow with ${{ secrets.SECRET_NAME }}.

Exporting results as Markdown

playwright-json-to-markdown converts the Playwright JSON report into a Markdown summary:

npx playwright-json-to-markdown test-results/results.json

Redirect the output to $GITHUB_STEP_SUMMARY (as shown in the workflow above) to display a formatted summary directly in the GitHub Actions run view:

npx playwright-json-to-markdown test-results/results.json >> $GITHUB_STEP_SUMMARY

Or write it to a file for use in a pull request comment:

npx playwright-json-to-markdown test-results/results.json > report.md

Posting results to Slack

playwright-json-to-slack-json converts the JSON report into a Slack Block Kit payload. The --report-url flag attaches a link back to the CI run:

npx playwright-json-to-slack-json --report-url "$WORKFLOW_URL" test-results/results.json > slack-payload.json

Post the result to a Slack channel using a webhook:

- name: Notify Slack
  if: always()
  run: |
    SLACK_PAYLOAD=$(npx playwright-json-to-slack-json --report-url "$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID" test-results/results.json)
    curl --header 'Content-type: application/json' --data "$SLACK_PAYLOAD" "${{ secrets.SLACK_WEBHOOK_URL }}"

Sharded runs

Playwright can split a large suite across CI jobs with --shard. Each shard writes a blob report, and a final job combines them with playwright merge-reports. Run each shard with npx donobu test --auto-heal as usual.

• Merged reports show healed results. Next to each blob report, donobu test writes a Donobu report that includes the auto-heal re-run. Under playwright merge-reports, the Donobu reporters (@donobu/test/reporter/html, @donobu/test/reporter/markdown, and @donobu/test/reporter/slack) read these files, so a test healed in one shard shows as healed in the merged summary. Keep the whole blob report directory when you pass it to the merge job. Playwright's own JSON, JUnit, and HTML reporters still show only the initial run.
• One heal pull request. All jobs that run on one commit publish into one pull request. Each job adds its fix to the heal branch and its verified results to the pull request.

GitLab CI

The heal agent also works on GitLab CI. On a branch pipeline, Donobu pushes the fix with the job's own token (CI_JOB_TOKEN), and GitLab push options open or update the merge request. You install nothing and create no token.

Before you use it:

1. In the project, open Settings → CI/CD → Job token permissions.
2. Turn on Allow Git push requests to the repository.
3. Make sure the pipeline user can push a branch and open a merge request.

Add DONOBU_API_KEY as a masked CI/CD variable, then add a job:

e2e:
  image: node:24
  script:
    - npm ci
    - npx playwright install --with-deps
    - npx donobu test --auto-heal
  artifacts:
    when: always
    paths:
      - test-results/

Pipelines triggered by a merge request keep the fix in the working tree and open nothing. Set DONOBU_HEAL_PULL_REQUEST: 'off' under variables to turn off merge requests on every pipeline. Keep the quotes: GitLab reads a bare off as the boolean false, which is not a valid variable value.

Other CI platforms

The GitHub Actions workflow above demonstrates the general pattern. The same approach works on any CI platform that supports Node.js:

1. Install dependencies (npm ci) with DONOBU_API_KEY set, so npm can read the private registry
2. Install Playwright browsers (npx playwright install --with-deps)
3. Set your AI provider key as a secret environment variable
4. Run npx donobu test (add --auto-heal to heal failures)
5. Upload the test-results/ directory as an artifact

For Jenkins, Azure DevOps, CircleCI, and other platforms, adapt the workflow syntax to your platform's format. The Donobu-specific steps (install, configure secrets, run tests, upload artifacts) remain the same. The heal agent opens pull requests only on GitHub Actions and GitLab CI. On other platforms it leaves the verified fix in the working tree.

Best practices

• Manage secrets securely: Use your CI platform's secret management for API keys. Never hardcode keys in pipeline files.
• Cache Playwright browsers: Use your CI platform's caching to avoid re-downloading browsers on every run.
• Upload artifacts on failure: Always save test-results/ as artifacts so you can diagnose failures after the run.
• Use self-healing in CI: Run npx donobu test --auto-heal so the heal agent fixes tests that break because the UI changed and sends you the fix as a pull request.
• Protect your base branch: The heal agent may edit any file. Branch protection makes sure every fix goes through review.
• Commit cache files: The .cache-lock/ directory should be committed to your repository so all CI runs start with a warm cache.