This project is deprecated in favour of https://github.com/Subito-it/Cachi please use that instead
If you're running UI Tests using xcodebuild you probably already know how hard it is to pin point an error when it occurs. The number of log information written in the various folder is so large that it's often easier to run the failing test again in Xcode and check from there what went wrong. This process is tedious and time consuming and often doesn't solve the problem because the test might mysteriously pass when manually run in Xcode.
sbtuitestbrowser parses xcodebuild's logs for you presenting them in a simple web interface to help visualize errors faster (some say even better than Xcode).
brew install Subito-it/made/sbtuitestbrowser
Then see usage.
sbtuitestbrowser is written in Swift 5, so if you're on macOS Mojave 10.14.3 or earlier, you may need to install an optional Swift library package that can be downloaded from "More Downloads" for Apple Developers at https://developer.apple.com/download/more/
Alternatively you can build manually using swift build.
git clone https://github.com/Subito-it/sbtuitestbrowser.git
cd sbtuitesttunnel
swift build
open sbtuitestbrowser.xcodeproj
Hit run and open your browser at http://localhost:8090, a very short test sample session will be loaded.
To launch sbtuitestbrowser you only need to specify a base path that contains test session logs (typically xcodebuild's -derivedDataPath) of your ui tests sessions.
sbtuitestbrowser [basepath]
This starts a web server reachable at http://localhost:8090 with a very simple and self explanatory interface.
When launched the tool will parse all the test session found inside the basepath specified at launch. As your tests sessions get executed and complete you can force the new results to be parsed by making a GET request to http://localhost:8090/parse
You can request the current running version by making a GET request to http://localhost:8090/version.
You can request the current status (ready/parsing) by making a GET request to http://localhost:8090/status.
You can request the average and last execution time of a test case by making a GET request to http://localhost:8090/teststats?id=testsuite/testcase.
There are scenarios (e.g. when running ui tests in parallarel over multiple machines) where you want different results to be merged together. When parsing all the TestSummaries.plist sbtuitestbrowser will look and group by the GroupingIdentifier key, presenting results as it was a single test session.
You can easily set this key after your xcobuild test is completed as follows:
plutil -insert GroupingIdentifier -string '<GROUPING_IDENTIFIER>' <PATHTOTHETESTSUMMARY.plist>
Besides the aforementioned GroupingIdentifier key there are additional custom keys that are parsed in the TestSummaries.plist.
RepoPath: The path where source code of the project being tested is located.BranchName: The branch name from where tests were launched. This information will be shown in the test runs homepage.CustomDescription: A custom text that will be shown in the test runs homepage.CommitHash: The hash of the commit from where tests were launched. This information will be shown in the test runs homepage.CommitMessage: The message of the commit from where tests were launched. This information will be shown in the test runs homepage when hovering over a test run.CodeCoverageFile: The path to the code coverage file to show code coverage information of a particular test run. RequiresRepoPathto be set.DiagnosticReports: The path to the folder containing crash reports (the logs found under ~/Library/Logs/DiagnosticReports) which will be automatically linked to the test that failed. For those tests a link to the crash report will be added to the test detailStandardOutputAndStandardErrorFile: The path to the StandardOutputAndStandardError-<app bundle id>.txt file which are used to enrich the ui testing actions with the apps console output.
This is a JSON file (OUTPUT_FILE.json) that was generated using xcrun llvm-cov export -instr-profile #{coverage_file}.profdata #{product_path} > #{output_filename}.json
Where product_path = "#{derived_data_base}/Build/Products/#{testing_configuration}-iphonesimulator/#{buildable_name}/#{executable_name}"
To get the most out of sbtuitestbrowser it's highly recommended to specify a unique -derivedDataPath for every test session you run. This will allow to show a complete history of your tests which can be useful to compare tests over time
The following command will launch your tests on an iPhone 6s simulator storing the results to ~/Desktop/iPhone_6s-*CURRENT_UNIXTIMESTAMP*:
xcodebuild \
-project YOURPROJECT.xcodeproj \ *or* -workspace YOURWORKSPACE.xcworkspace \
-scheme YOURUITESTSCHEME \
-sdk iphonesimulator \
-destination 'platform=iOS Simulator,name=iPhone 6s,OS=10.2' \
-derivedDataPath ~/Desktop/iPhone_6s-$(date +%s) \
test
Code isn't probably the cleanest I ever wrote but given the usefulness of the tool I decided to publish it nevertheless.
Kudos to the developers of Perfect-HTTP, a wonderful web framework/server written in Swift.
Contributions are welcome! If you have a bug to report, feel free to help out by opening a new issue or sending a pull request.
sbtuitestbrowser is available under the Apache License, Version 2.0. See the LICENSE file for more info.