From b830bda352c522583238c80bc777e959ad1f02b6 Mon Sep 17 00:00:00 2001 From: XiangRongLin <41164160+XiangRongLin@users.noreply.github.com> Date: Sun, 30 May 2021 21:28:58 +0200 Subject: [PATCH 1/5] Add doc for extractor mocks --- docs/05_Mock_tests.md | 57 +++++++++++++++++++ docs/{05_releasing.md => 06_releasing.md} | 0 ...6_documentation.md => 07_documentation.md} | 0 ...tainers_view.md => 08_maintainers_view.md} | 0 4 files changed, 57 insertions(+) create mode 100644 docs/05_Mock_tests.md rename docs/{05_releasing.md => 06_releasing.md} (100%) rename docs/{06_documentation.md => 07_documentation.md} (100%) rename docs/{07_maintainers_view.md => 08_maintainers_view.md} (100%) diff --git a/docs/05_Mock_tests.md b/docs/05_Mock_tests.md new file mode 100644 index 0000000..cb43525 --- /dev/null +++ b/docs/05_Mock_tests.md @@ -0,0 +1,57 @@ +# Mock Tests + +A web crawler is by nature dependant on the external service which it is crawling. +In order to have a reliable CI pipeline, this external dependency needs to be removed. +For this there is a system in place to automatically save the requests made to a service and their responses. +These can then be used in CI to reliably test changes made to the Extractor and not have test failures due to API changes on the side of the service. + +## Multiple downloader implementations + +There are multiple implementations of the abstract class `Downloader` + +1. `DownloaderTestImpl` is used for running the test against the actual service. +2. `RecordingDownloader` is used the save the request and response to the disk, thus creating the mock. +3. `MockDownloader` is used to answer requests with the saved mocks. + +### Usage + +There are 2 ways to specify which downloader should be used. + +First one is passing the `-Ddownloader=` argument from the command line, where `value` can be one of +[DownloaderType](https://github.com/TeamNewPipe/NewPipeExtractor/blob/dev/extractor/src/test/java/org/schabi/newpipe/downloader/DownloaderType.java) +. The main usecase is in the CI pipeline like this: `./gradlew check --stacktrace -Ddownloader=MOCK`. +Other than that it can also be used to mass generate mocks by specifying which package should be tested. As example if +one wanted to update all YouTube mocks: +`gradle clean test --tests 'org.schabi.newpipe.extractor.services.youtube.*' -Ddownloader=RECORDING` + +The second way is changing the field `DownloaderFactory.DEFAULT_DOWNLOADER`. +The default value is `DownloaderType.REAL` which should be changed on the master branch. +Locally one can change this to `DownloaderType.RECORDING`, run the tests and commit +the generated mocks. +This is the main use case for when developing locally. + +### Mock only tests + +There are some things which cannot ever be tested reliably against an actual service. +For example tests for an upcoming livestream would fail, after the livestream was started. + +For this there is a marker interface `MockOnly` and a custom TestRule `MockOnlyRule`. +It skips the tests in the CI pipeline if they are not run with mocks. + +See `MockOnlyRule` for further details. + +Example usage: + +``` java +public static class TestClass { + + @Rule + public MockOnlyRule rule = new MockOnlyRule(); + + @MockOnly + @Test + public void myTest() throws Exception { + //assertions + } +} +``` \ No newline at end of file diff --git a/docs/05_releasing.md b/docs/06_releasing.md similarity index 100% rename from docs/05_releasing.md rename to docs/06_releasing.md diff --git a/docs/06_documentation.md b/docs/07_documentation.md similarity index 100% rename from docs/06_documentation.md rename to docs/07_documentation.md diff --git a/docs/07_maintainers_view.md b/docs/08_maintainers_view.md similarity index 100% rename from docs/07_maintainers_view.md rename to docs/08_maintainers_view.md From 27a08429938192b67f2c0aa3dbd2205e83d98db0 Mon Sep 17 00:00:00 2001 From: XiangRongLin <41164160+XiangRongLin@users.noreply.github.com> Date: Mon, 22 Nov 2021 14:35:03 +0100 Subject: [PATCH 2/5] Apply suggestions from code review Co-authored-by: opusforlife2 <53176348+opusforlife2@users.noreply.github.com> --- docs/05_Mock_tests.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/05_Mock_tests.md b/docs/05_Mock_tests.md index cb43525..9cb3afc 100644 --- a/docs/05_Mock_tests.md +++ b/docs/05_Mock_tests.md @@ -10,7 +10,7 @@ These can then be used in CI to reliably test changes made to the Extractor and There are multiple implementations of the abstract class `Downloader` 1. `DownloaderTestImpl` is used for running the test against the actual service. -2. `RecordingDownloader` is used the save the request and response to the disk, thus creating the mock. +2. `RecordingDownloader` is used to save the request and response to the disk, thus creating the mock. 3. `MockDownloader` is used to answer requests with the saved mocks. ### Usage @@ -20,7 +20,7 @@ There are 2 ways to specify which downloader should be used. First one is passing the `-Ddownloader=` argument from the command line, where `value` can be one of [DownloaderType](https://github.com/TeamNewPipe/NewPipeExtractor/blob/dev/extractor/src/test/java/org/schabi/newpipe/downloader/DownloaderType.java) . The main usecase is in the CI pipeline like this: `./gradlew check --stacktrace -Ddownloader=MOCK`. -Other than that it can also be used to mass generate mocks by specifying which package should be tested. As example if +Other than that it can also be used to mass generate mocks by specifying which package should be tested. For example, if one wanted to update all YouTube mocks: `gradle clean test --tests 'org.schabi.newpipe.extractor.services.youtube.*' -Ddownloader=RECORDING` From 81195776d7a4c8668497e9ffacb76db00e32e9bd Mon Sep 17 00:00:00 2001 From: XiangRongLin <41164160+XiangRongLin@users.noreply.github.com> Date: Wed, 24 Nov 2021 17:58:17 +0100 Subject: [PATCH 3/5] Update docs/05_Mock_tests.md --- docs/05_Mock_tests.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/05_Mock_tests.md b/docs/05_Mock_tests.md index 9cb3afc..ac2325f 100644 --- a/docs/05_Mock_tests.md +++ b/docs/05_Mock_tests.md @@ -3,7 +3,7 @@ A web crawler is by nature dependant on the external service which it is crawling. In order to have a reliable CI pipeline, this external dependency needs to be removed. For this there is a system in place to automatically save the requests made to a service and their responses. -These can then be used in CI to reliably test changes made to the Extractor and not have test failures due to API changes on the side of the service. +These can then be used in the CI pipeline to reliably test changes made to the Extractor and not have test failures due to API changes on the side of the service. ## Multiple downloader implementations From c688e88ca6a742cfe41276e94978c11915a389dd Mon Sep 17 00:00:00 2001 From: opusforlife2 <53176348+opusforlife2@users.noreply.github.com> Date: Sat, 27 Nov 2021 15:23:42 +0000 Subject: [PATCH 4/5] Apply suggestions from code review - 2 --- docs/05_Mock_tests.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/05_Mock_tests.md b/docs/05_Mock_tests.md index ac2325f..778417e 100644 --- a/docs/05_Mock_tests.md +++ b/docs/05_Mock_tests.md @@ -1,17 +1,17 @@ # Mock Tests -A web crawler is by nature dependant on the external service which it is crawling. +A web crawler is, by its very nature, dependent on the external service which it is crawling. In order to have a reliable CI pipeline, this external dependency needs to be removed. -For this there is a system in place to automatically save the requests made to a service and their responses. -These can then be used in the CI pipeline to reliably test changes made to the Extractor and not have test failures due to API changes on the side of the service. +For this, there is a system in place to automatically save the requests made to a service, as well as the responses received. +These can then be used in the CI pipeline to reliably test changes made to the Extractor, and avoid test failures due to API changes on the side of the service. ## Multiple downloader implementations There are multiple implementations of the abstract class `Downloader` 1. `DownloaderTestImpl` is used for running the test against the actual service. -2. `RecordingDownloader` is used to save the request and response to the disk, thus creating the mock. -3. `MockDownloader` is used to answer requests with the saved mocks. +2. `RecordingDownloader` is used to save the request and response to a file, thus creating the mock. +3. `MockDownloader` is used to answer requests using the saved mocks. ### Usage @@ -19,7 +19,7 @@ There are 2 ways to specify which downloader should be used. First one is passing the `-Ddownloader=` argument from the command line, where `value` can be one of [DownloaderType](https://github.com/TeamNewPipe/NewPipeExtractor/blob/dev/extractor/src/test/java/org/schabi/newpipe/downloader/DownloaderType.java) -. The main usecase is in the CI pipeline like this: `./gradlew check --stacktrace -Ddownloader=MOCK`. +. The main use case is in the CI pipeline, like this: `./gradlew check --stacktrace -Ddownloader=MOCK`. Other than that it can also be used to mass generate mocks by specifying which package should be tested. For example, if one wanted to update all YouTube mocks: `gradle clean test --tests 'org.schabi.newpipe.extractor.services.youtube.*' -Ddownloader=RECORDING` @@ -33,9 +33,9 @@ This is the main use case for when developing locally. ### Mock only tests There are some things which cannot ever be tested reliably against an actual service. -For example tests for an upcoming livestream would fail, after the livestream was started. +For example, tests for an upcoming livestream would fail after the livestream starts. -For this there is a marker interface `MockOnly` and a custom TestRule `MockOnlyRule`. +For this, there is a marker interface `MockOnly`, and a custom TestRule `MockOnlyRule`. It skips the tests in the CI pipeline if they are not run with mocks. See `MockOnlyRule` for further details. From b3a36686adb7134249e9fe1940c51c24baf9f35f Mon Sep 17 00:00:00 2001 From: XiangRongLin <41164160+XiangRongLin@users.noreply.github.com> Date: Sun, 28 Nov 2021 13:47:31 +0100 Subject: [PATCH 5/5] Update docs/05_Mock_tests.md --- docs/05_Mock_tests.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/05_Mock_tests.md b/docs/05_Mock_tests.md index 778417e..763151e 100644 --- a/docs/05_Mock_tests.md +++ b/docs/05_Mock_tests.md @@ -25,7 +25,7 @@ one wanted to update all YouTube mocks: `gradle clean test --tests 'org.schabi.newpipe.extractor.services.youtube.*' -Ddownloader=RECORDING` The second way is changing the field `DownloaderFactory.DEFAULT_DOWNLOADER`. -The default value is `DownloaderType.REAL` which should be changed on the master branch. +The default value is `DownloaderType.REAL` which should not be changed on the master branch. Locally one can change this to `DownloaderType.RECORDING`, run the tests and commit the generated mocks. This is the main use case for when developing locally.