CI/CD Integration

This page covers running Donobu tests in automated pipelines and exporting results in different formats.

GitHub Actions — complete workflow

name: E2E Tests

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: '24'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install --with-deps

      - name: Run tests
        env:
          SELF_HEAL_TESTS_ENABLED: true
          DONOBU_API_KEY: ${{ secrets.DONOBU_API_KEY }}
        run: npx donobu test

      - name: Generate Test Reports
        if: always()
        run: |
          npm exec playwright-json-to-markdown test-results/playwright-report.json >> $GITHUB_STEP_SUMMARY
          npm exec playwright-json-to-html test-results/playwright-report.json

      - name: Save Test Reports as Artifacts
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: test-results
          path: test-results/
          retention-days: 14

      - name: Post summary to GitHub
        if: always()
        run: npx playwright-json-to-markdown test-results/results.json >> $GITHUB_STEP_SUMMARY

      - name: Post to Slack
        if: always()
        env:
          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
        run: |
          if [ -n "$SLACK_WEBHOOK_URL" ]; then
            WORKFLOW_URL="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}"
            SLACK_PAYLOAD=$(npx playwright-json-to-slack-json --report-url "$WORKFLOW_URL" test-results/results.json)
            curl --header 'Content-type: application/json' --data "$SLACK_PAYLOAD" "$SLACK_WEBHOOK_URL"
          else
            echo "SLACK_WEBHOOK_URL secret not present, skipping Slack notification."
          fi

      # Create a self-healing PR only when this workflow was not triggered by a pull-request.
      - name: Automatically create a pull request for fixing failed tests (if any)
        if: ${{ github.event_name != 'pull_request' }}
        uses: peter-evans/create-pull-request@v5
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          commit-message: 'Fix failing Playwright tests'
          title: '[Fixed] Playwright tests'
          body: 'Fix failing Playwright tests'
          branch: fix-playwright-tests-for-${{ github.ref_name }}
          base: ${{ github.ref_name }}

The SELF_HEAL_TESTS_ENABLED environment variable enables automatic test healing: if a test fails due to a UI change, Donobu re-runs it with the AI to regenerate the cached action sequence. The last step creates a PR if the cache was updated, so that the tests will execute faster on subsequent runs. See Self-Healing Tests for details.

The test result files (JSON reports and related data) are saved under test-results/. GitHub summaries can be derived from these using playwright-json-to-markdown.

Storing API keys as secrets

Add your AI provider key as a repository or organisation secret in GitHub:

Settings → Secrets and variables → Actions → New repository secret

Reference them in the workflow with ${{ secrets.SECRET_NAME }}.

Exporting results as Markdown

playwright-json-to-markdown converts the Playwright JSON report into a Markdown summary:

npx playwright-json-to-markdown test-results/results.json

Redirect the output to $GITHUB_STEP_SUMMARY (as shown in the workflow above) to display a formatted summary directly in the GitHub Actions run view:

npx playwright-json-to-markdown test-results/results.json >> $GITHUB_STEP_SUMMARY

Or write it to a file for use in a pull request comment:

npx playwright-json-to-markdown test-results/results.json > report.md

Posting results to Slack

playwright-json-to-slack-json converts the JSON report into a Slack Block Kit payload. The --report-url flag attaches a link back to the CI run:

npx playwright-json-to-slack-json --report-url "$WORKFLOW_URL" test-results/results.json > slack-payload.json

Post the result to a Slack channel using a webhook:

- name: Notify Slack
  if: always()
  run: |
    SLACK_PAYLOAD=$(npx playwright-json-to-slack-json --report-url "$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID" test-results/results.json)
    curl --header 'Content-type: application/json' --data "$SLACK_PAYLOAD" "${{ secrets.SLACK_WEBHOOK_URL }}"