Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Subset of E2E models for accuracy tests #2246

Open
pbchekin opened this issue Sep 13, 2024 · 1 comment · Fixed by #2272
Open

Subset of E2E models for accuracy tests #2246

pbchekin opened this issue Sep 13, 2024 · 1 comment · Fixed by #2272

Comments

@pbchekin
Copy link
Contributor

pbchekin commented Sep 13, 2024

We currently have a subset of models to test E2E performance. We need similar subset for E2E accuracy tests. If the E2E accuracy workflow started with the subset, it should check that all models in the subset passed the accuracy check.

Periodic E2E jobs (E2E accuracy, Try latest PyTorch) need to use the subset mode to catch regressions early.

@vlad-penkin
Copy link
Contributor

@pbchekin this ticket partially duplicates

Let's align scope between two tickets.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants