Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature]: add --fail-on-flaky-tests in config file #34397

Open
jfgreffier opened this issue Jan 19, 2025 · 0 comments
Open

[Feature]: add --fail-on-flaky-tests in config file #34397

jfgreffier opened this issue Jan 19, 2025 · 0 comments
Labels
open-to-a-pull-request The feature request looks good, we are open to reviewing a PR

Comments

@jfgreffier
Copy link
Contributor

🚀 Feature Request

I would like to have an failOnFlakyTests option in playwright.config.ts rather than only in CLI. Its default value should be false.

Example

export default defineConfig({
  testDir: './tests',
  /* Run tests in files in parallel */
  fullyParallel: true,
  /* Fail the build on CI if you accidentally left test.only in the source code. */
  forbidOnly: !!process.env.CI,
  /* Retry on CI only */
  retries: process.env.CI ? 2 : 0,
+++  /* Don't accept flaky tests */
+++  failOnFlakyTests: process.env.CI ? true : false,

Motivation

We probably don't want the same settings in CI and locally.
I feel it's more a configuration point than something that you would use occasionally. Really similar to forbidOnly

@pavelfeldman pavelfeldman added the open-to-a-pull-request The feature request looks good, we are open to reviewing a PR label Jan 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
open-to-a-pull-request The feature request looks good, we are open to reviewing a PR
Projects
None yet
Development

No branches or pull requests

2 participants