You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If the community is comfortable with files that need special permissions to maintain, we could continue using that storage account
We could see if the Linux Foundation has storage we could use somehow, and grant access to that using iree-org GitHub membership or some other open process
Test flags
The tests at https://github.com/iree-org/iree/tree/main/experimental/regression_suite/shark-test-suite-models in particular have long lists of flags and hardware requirements since most model development activities have been focusing on getting great performance on a small number of devices (e.g. AMD MI300 GPUs). For a common test suite, I would prefer to use default or generic flags, not long lists of flags, transform scripts, or tuning specs.
The text was updated successfully, but these errors were encountered:
These two locations have full model integration tests:
Those tests are both currently run as part of the https://github.com/iree-org/iree/blob/main/.github/workflows/pkgci_regression_test.yml workflow.
Together with #48, it should be clear where new tests should go in source code and within workflows in this repository and in iree-org/iree.
The documentation at https://iree.dev/developers/general/testing-guide/#external-test-suites should also be kept up to date.
File storage
Both of those test suites currently store files in the 'sharkpublic' Azure storage account.
Test flags
The tests at https://github.com/iree-org/iree/tree/main/experimental/regression_suite/shark-test-suite-models in particular have long lists of flags and hardware requirements since most model development activities have been focusing on getting great performance on a small number of devices (e.g. AMD MI300 GPUs). For a common test suite, I would prefer to use default or generic flags, not long lists of flags, transform scripts, or tuning specs.
The text was updated successfully, but these errors were encountered: