newpipe-documentation/docs/05_Mock_tests.md

Mock Tests

A web crawler is, by its very nature, dependent on the external service which it is crawling. In order to have a reliable CI pipeline, this external dependency needs to be removed. For this, there is a system in place to automatically save the requests made to a service, as well as the responses received. These can then be used in the CI pipeline to reliably test changes made to the Extractor, and avoid test failures due to API changes on the side of the service.

Multiple downloader implementations

There are multiple implementations of the abstract class Downloader

1. DownloaderTestImpl is used for running the test against the actual service.
2. RecordingDownloader is used to save the request and response to a file, thus creating the mock.
3. MockDownloader is used to answer requests using the saved mocks.

Usage

There are 2 ways to specify which downloader should be used.

First one is passing the -Ddownloader=<value> argument from the command line, where value can be one of DownloaderType . The main use case is in the CI pipeline, like this: ./gradlew check --stacktrace -Ddownloader=MOCK.
Other than that it can also be used to mass generate mocks by specifying which package should be tested. For example, if one wanted to update all YouTube mocks: gradle clean test --tests 'org.schabi.newpipe.extractor.services.youtube.*' -Ddownloader=RECORDING

The second way is changing the field DownloaderFactory.DEFAULT_DOWNLOADER. The default value is DownloaderType.REAL which should not be changed on the master branch. Locally one can change this to DownloaderType.RECORDING, run the tests and commit the generated mocks.
This is the main use case for when developing locally.

Mock only tests

There are some things which cannot ever be tested reliably against an actual service. For example, tests for an upcoming livestream would fail after the livestream starts.

For this, there is a marker interface MockOnly, and a custom TestRule MockOnlyRule. It skips the tests in the CI pipeline if they are not run with mocks.

See MockOnlyRule for further details.

Example usage:

public static class TestClass {

     @Rule
     public MockOnlyRule rule = new MockOnlyRule();

     @MockOnly
     @Test
     public void myTest() throws Exception {
         //assertions
     }
}