CI/CD Integration

This page covers running Donobu tests in automated pipelines and exporting results in different formats.

GitHub Actions — complete workflow

name: E2E Tests

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: '24'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install --with-deps

      - name: Run tests
        env:
          DONOBU_API_KEY: ${{ secrets.DONOBU_API_KEY }}
        run: npx donobu test --auto-heal

      - name: Generate Test Reports
        if: always()
        run: |
          npm exec playwright-json-to-markdown test-results/playwright-report.json >> $GITHUB_STEP_SUMMARY
          npm exec playwright-json-to-html test-results/playwright-report.json

      - name: Save Test Reports as Artifacts
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: test-results
          path: test-results/
          retention-days: 14

      - name: Post to Slack
        if: always()
        env:
          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
        run: |
          if [ -n "$SLACK_WEBHOOK_URL" ]; then
            WORKFLOW_URL="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}"
            SLACK_PAYLOAD=$(npx playwright-json-to-slack-json --report-url "$WORKFLOW_URL" test-results/playwright-report.json)
            curl --header 'Content-type: application/json' --data "$SLACK_PAYLOAD" "$SLACK_WEBHOOK_URL"
          else
            echo "SLACK_WEBHOOK_URL secret not present, skipping Slack notification."
          fi

      - name: Check for fixes for tests
        id: check-fixes-for-tests
        run: |
          if [ -n "$(git diff --name-only -- tests/)" ]; then
            echo "has_fixes=true" >> "$GITHUB_OUTPUT"
          fi

      # Create a self-healing PR if there are changes and this workflow was not triggered by a pull-request.
      - name: Automatically create a pull request for fixing failed tests (if any)
        if: ${{ github.event_name != 'pull_request' && steps.check-fixes-for-tests.outputs.has_fixes == 'true'  }}
        uses: peter-evans/create-pull-request@v5
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          commit-message: 'Fix failing Playwright tests'
          title: '[Auto-healed] Playwright tests'
          body: 'Automatically healed failing Playwright tests with updated cache.'
          branch: fix-playwright-tests-for-${{ github.ref_name }}
          base: ${{ github.ref_name }}

The SELF_HEAL_TESTS_ENABLED environment variable enables automatic test healing: if a test fails due to a UI change, Donobu re-runs it with the AI to regenerate the cached action sequence. The last step creates a PR if the cache was updated, so that the tests will execute faster on subsequent runs. See Self-Healing Tests for details.

The test result files (JSON reports and related data) are saved under test-results/. GitHub summaries can be derived from these using playwright-json-to-markdown.

Storing API keys as secrets

Add your AI provider key as a repository or organisation secret in GitHub:

Settings → Secrets and variables → Actions → New repository secret

Reference them in the workflow with ${{ secrets.SECRET_NAME }}.

Exporting results as Markdown

playwright-json-to-markdown converts the Playwright JSON report into a Markdown summary:

npx playwright-json-to-markdown test-results/results.json

Redirect the output to $GITHUB_STEP_SUMMARY (as shown in the workflow above) to display a formatted summary directly in the GitHub Actions run view:

npx playwright-json-to-markdown test-results/results.json >> $GITHUB_STEP_SUMMARY

Or write it to a file for use in a pull request comment:

npx playwright-json-to-markdown test-results/results.json > report.md

Posting results to Slack

playwright-json-to-slack-json converts the JSON report into a Slack Block Kit payload. The --report-url flag attaches a link back to the CI run:

npx playwright-json-to-slack-json --report-url "$WORKFLOW_URL" test-results/results.json > slack-payload.json

Post the result to a Slack channel using a webhook:

- name: Notify Slack
  if: always()
  run: |
    SLACK_PAYLOAD=$(npx playwright-json-to-slack-json --report-url "$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID" test-results/results.json)
    curl --header 'Content-type: application/json' --data "$SLACK_PAYLOAD" "${{ secrets.SLACK_WEBHOOK_URL }}"

Other CI platforms

The GitHub Actions workflow above demonstrates the general pattern. The same approach works on any CI platform that supports Node.js:

1. Install dependencies (npm ci)
2. Install Playwright browsers (npx playwright install --with-deps)
3. Set your AI provider key as a secret environment variable
4. Run npx donobu test
5. Upload the test-results/ directory as an artifact

For GitLab CI, Jenkins, Azure DevOps, CircleCI, and other platforms, adapt the workflow syntax to your platform's format. The Donobu-specific steps (install, configure secrets, run tests, upload artifacts) remain the same.

Best practices

• Manage secrets securely — Use your CI platform's secret management for API keys. Never hardcode keys in pipeline files.
• Cache Playwright browsers — Use your CI platform's caching to avoid re-downloading browsers on every run.
• Upload artifacts on failure — Always save test-results/ as artifacts so you can diagnose failures after the run.
• Use self-healing in CI — Set SELF_HEAL_TESTS_ENABLED=true to automatically fix tests that break due to UI changes.
• Commit cache files — The .cache-lock/ directory should be committed to your repository so all CI runs start with a warm cache.