1556adbb2d
webCommandMetadata object is contained inside a commandMetadata one, so it is not accessible from the root of the navigationEndpoint object. The corresponding statement has been moved at the bottom of the specific endpoints parsing, as the webCommandMetadata object is present almost everywhere, otherwise URLs of some endpoints would have be changed, such as uploader URLs (from channel IDs to handles). As no ParsingException is now thrown by getUrlFromNavigationEndpoint, and so by getTextFromObject, getUrlFromObject and getTextAtKey, the methods which were catching ParsingExceptions thrown by these methods had to be updated. URLs got in the HTML version of getTextFromObject are now escaped properly to provide valid HTML to clients. This has been also done for attribute descriptions, with the description text for this type of descriptions. As YouTube descriptions are in HTML format (except for the fallback on the JSON player response, which is plain text and only happens when there is no visual metadata or a breaking change), all URLs returned are escaped, so tests which are testing presence of URLs with escaped characters had to be updated (it was only the case for YoutubeStreamExtractorDefaultTest.DescriptionTestUnboxing). |
||
|---|---|---|
| .github | ||
| checkstyle | ||
| extractor | ||
| gradle/wrapper | ||
| timeago-parser | ||
| .gitignore | ||
| LICENSE | ||
| README.md | ||
| build.gradle | ||
| gradlew | ||
| gradlew.bat | ||
| jitpack.yml | ||
| settings.gradle | ||
README.md
NewPipe Extractor
NewPipe Extractor is a library for extracting things from streaming sites. It is a core component of NewPipe, but could be used independently.
Usage
NewPipe Extractor is available at JitPack's Maven repo.
If you're using Gradle, you could add NewPipe Extractor as a dependency with the following steps:
- Add
maven { url 'https://jitpack.io' }to therepositoriesin yourbuild.gradle. - Add
implementation 'com.github.TeamNewPipe:NewPipeExtractor:INSERT_VERSION_HERE'to thedependenciesin yourbuild.gradle. ReplaceINSERT_VERSION_HEREwith the latest release.
Note: To use NewPipe Extractor in projects with a minSdk below 26, API desugaring is required. If the minSdk is below 19, the desugar_jdk_libs_nio artifact is required, which requires Android Gradle Plugin (AGP) version 7.4.0.
Testing changes
To test changes quickly you can build the library locally. A good approach would be to add something like the following to your settings.gradle:
includeBuild('../NewPipeExtractor') {
dependencySubstitution {
substitute module('com.github.TeamNewPipe:NewPipeExtractor') with project(':extractor')
}
}
Another approach would be to use the local Maven repository, here's a gist of how to use it:
- Add
mavenLocal()in your projectrepositorieslist (usually as the first entry to give priority above the others). - It's recommended that you change the
versionof this library (e.g.LOCAL_SNAPSHOT). - Run gradle's
ìnstalltask to deploy this library to your local repository (using the wrapper, present in the root of this project:./gradlew install) - Change the dependency version used in your project to match the one you chose in step 2 (
implementation 'com.github.TeamNewPipe:NewPipeExtractor:LOCAL_SNAPSHOT')
Tip for Android Studio users: After you make changes and run the
installtask, use the menu optionFile → "Sync with File System"to refresh the library in your project.
Supported sites
The following sites are currently supported:
- YouTube
- SoundCloud
- media.ccc.de
- PeerTube (no P2P)
- Bandcamp
License
NewPipe is Free Software: You can use, study share and improve it at your will. Specifically you can redistribute and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.