-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add tests via TestDriverAI #1957
base: main
Are you sure you want to change the base?
Conversation
@Frooodle We need your approval to run the workflow. |
I believe I need the As well? |
Here is a preview of the test execution: Watch testdriverai/Stirling-PDF/refs/heads/main on Dashcam |
Hey @Frooodle could you have another look at this. It's a go from our side! Cheers! |
Sorry been very busy should be able to properly check it end of week |
Added key but job still failed, does account need upgraded? |
The API is reporting the key a |
Hey @Frooodle could please make sure you have got the API set up properly. |
Hey @Frooodle, so I tried to recreate the issue by forking and making a PR. Go to dashcam.io/team and click on the cycle button next to Try to re-run the failed job over here If that doesn't work, make sure the right permissions are set here for PRs to access the secrets, or you can just merge this PR if you do not wish to change the permission. |
All seems okay my side from what I can see, to the job just to see if it can grab it or if its being lost in the call to the github action |
yes, lets try that. |
Hmm printing nope |
It printed yes for me when I tried on my fork |
Is it because of this code ?? |
Description
This PR contains the test cases to test the basic functionality of the stirling.io webpage.
The tests are written using testdriverai a next gen automation AI agent for end-to-end testing of web and desktop application.
Overview
This PR adds TestDriver tests that I generated using the
testdriverai
npm package. Instead of using selectors, TestDriver uses prompts and AI vision to complete tasks. This means the tests will still work even when code and design changes.Another huge benefit is we can test PDF input and output because tests operate at the OS level.
You can see the schema of supported commands here.
Test Cases
Setup
In order to complete this PR you’ll need to add a API key as a GitHub action secret. Follow the directions here for more info.
Once you sign up, let us know your email and we’ll enroll you in the open source program for free Pro account.
Here's how to add more tests :
prompts
field in the test file for motivation )What’s next
We can set this up to build feature branches using our prerun scripts.
Checklist