Keep tests out of auto-heal by tag, and keep every fix

Tag a test to keep auto-heal away from it, and fixes made in CI are saved as a patch even when no pull request opens.

Some tests are supposed to fail: they check the right behavior against a product bug that is still open. Tag them, and auto-heal leaves them alone. Their failures are still diagnosed, so the report says why they failed.

Fixes made in CI are also saved now, even when no pull request is opened. Donobu writes them to auto-heal.patch next to the reports, so you can apply them by hand.

  • Set metadata.autoHealExcludeTags in the config or per project; a project's list overrides the config's
  • An excluded test stays as written, even when auto-heal repairs another test in the same file
  • The heal agent keeps the behavior under test and leaves product defects failing
  • auto-heal.patch is written before publishing, so a failed publish never loses the fix