diff --git a/docs/00_Prepare_everything.md b/docs/00_Prepare_everything.md index 164f4c1..d4fc57a 100644 --- a/docs/00_Prepare_everything.md +++ b/docs/00_Prepare_everything.md @@ -1,7 +1,7 @@ # Prepare everything Welcome to the NewPipe tutorial. This tutorial will guide you through the process of creating your own NewPipeExtractor -service with which NewPipe will gain support for a dedicated streaming service like YouTube, Vimeo or SournCloud. Let's +service with which NewPipe will gain support for a dedicated streaming service like YouTube, Vimeo or SoundCloud. Let's dive right in. ;D ## Setup your dev environment @@ -23,14 +23,14 @@ First and foremost you need to meet certain conditions in order to write your ow - __[java 8](https://www.java.com/en/download/faq/java8.xml)__ - __[gradle](https://gradle.org/)__ - __[unit testing](https://junit.org/junit5/)__ - - I highly recomend [IDEA Community](https://www.jetbrains.com/idea/) since it has everything we need. -- A __[github](https://github.com/)__ account -- A loot of patience and excitement ;D + - I highly recommend [IDEA Community](https://www.jetbrains.com/idea/) since it has everything we need. +- A __[Github](https://github.com/)__ account +- A lot of patience and excitement ;D After making sure all these conditions are provided fork the [NewPipeExtractor](https://github.com/TeamNewPipe/NewPipeExtractor), using the [fork button](https://github.com/TeamNewPipe/NewPipeExtractor#fork-destination-box). This way you have your own working repository. Now clone this repository into your local folder in which you want to work in. -Next import the cloned project into your [ide](https://www.jetbrains.com/help/idea/configuring-projects.html#importing-project) +Next import the cloned project into your [IDE](https://www.jetbrains.com/help/idea/configuring-projects.html#importing-project) and [run](https://www.jetbrains.com/help/idea/performing-tests.html) it. If all the checks are green you did everything right, and you are good to go to move on to the next chapter. diff --git a/docs/01_Concept_of_the_extractor.md b/docs/01_Concept_of_the_extractor.md index 3a29021..aea7bc0 100644 --- a/docs/01_Concept_of_the_extractor.md +++ b/docs/01_Concept_of_the_extractor.md @@ -5,7 +5,7 @@ Before we can start coding our own service we need to understand the basic concept of the extractor. There is a pattern you will find all over the code. It is called the __extractor/collector__ pattern. The idea behind it is that the [extractor](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/Extractor.html) -would produce single peaces of data, and the collector would take it and form usable data for the front end out of it. +would produce single pieces of data, and the collector would take it and form usable data for the front end out of it. The collector also controls the parsing process, and takes care about error handling. So if the extractor fails at any point the collector will decide whether it should continue parsing or not. This requires the extractor to be made out of many small methods. One method for every data field the collector wants to have. The collectors are provided by NewPipe. @@ -99,7 +99,7 @@ When a website shows a long list of items it usually does not load the whole lis This is why a list in NewPipe lists are chopped down into smaller lists called [InfoItemsPage](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/ListExtractor.InfoItemsPage.html)s. Each page has its own URL, and needs to be extracted separately. -Additional metainformation about the list such as it's title a thumbnail +Additional metainformation about the list such as its title a thumbnail or its creator, and extracting multiple pages can be handled by a [ListExtractor](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/ListExtractor.html), and it's [ListExtractor.InfoItemsPage](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/ListExtractor.InfoItemsPage.html). @@ -115,7 +115,7 @@ such as: returns a ListExtractor.InfoItemsPage by its URL which was retrieved by the `getNextPageUrl()` method of the previous page. -The reason why the first page is handled speciall is because many Websites such as Youtube will load the first page of +The reason why the first page is handled special is because many Websites such as YouTube will load the first page of items like a regular webpage, but all the others as AJAX request. diff --git a/docs/02_Concept_of_LinkHandler.md b/docs/02_Concept_of_LinkHandler.md index a35852e..230573c 100644 --- a/docs/02_Concept_of_LinkHandler.md +++ b/docs/02_Concept_of_LinkHandler.md @@ -2,12 +2,12 @@ [LinkHandler](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/LinkHandler.html) represent Links to resources like videos, search requests, channels, etc. -The idea behind them is that a video can have multiple links pointig to it, but it has -one unique id that represents it, like this example: +The idea behind them is that a video can have multiple links pointing to it, but it has +one unique ID that represents it, like this example: [oHg5SJYRHA0](https://www.youtube.com/watch?v=oHg5SJYRHA0) can be represented as: -- https://www.youtube.com/watch?v=oHg5SJYRHA0 (default url for youtube) +- https://www.youtube.com/watch?v=oHg5SJYRHA0 (default URL for YouTube) - https://youtu.be/oHg5SJYRHA0 - https://m.youtube.com/watch?v=oHg5SJYRHA0 @@ -16,12 +16,12 @@ one unique id that represents it, like this example: - `LinkHandler` are ReadOnly - LinkHandler are also used to determine which part of the extractor can handle a certain link. - In order to get one you must either call -[fromUrl()](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/LinkHandlerFactory.html#fromUrl-java.lang.String-) or [fromId()](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/LinkHandlerFactory.html#fromId-java.lang.String-) of the the coresponding `LinkHandlerFactory`. +[fromUrl()](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/LinkHandlerFactory.html#fromUrl-java.lang.String-) or [fromId()](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/LinkHandlerFactory.html#fromId-java.lang.String-) of the the corresponding `LinkHandlerFactory`. - Every type of Type of Resource has its own LinkHandlerFactory. Eg. YoutubeStreamLinkHandler, YoutubeChannelLinkHandler, etc. ### Usage -So the typical Usage for getting a LinkHandler would look like this. +So the typical usage for getting a LinkHandler would look like this. ```java LinkHandlerFactory myLinkHandlerFactory = new MyStreamLinkHandlerFactory(); LinkHandler myVideo = myLinkHandlerFactory.fromUrl("https://my.service.com/the_video"); @@ -29,19 +29,19 @@ LinkHandler myVideo = myLinkHandlerFactory.fromUrl("https://my.service.com/the_v ### Implementation -In order to Use LinkHandler for your service you must override the apropriate LinkHandlerFactory. eg: +In order to Use LinkHandler for your service you must override the appropriate LinkHandlerFactory. eg: ```java class MyStreamLinkHandlerFactory extends LinkHandlerFactory { @Override public String getId(String url) throws ParsingException { - // Return the ID based on the url. + // Return the ID based on the URL. } @Override public String getUrl(String id) throws ParsingException { - // Return the url based on the id given. + // Return the URL based on the ID given. } @Override @@ -70,7 +70,7 @@ Through these you can tell the front end which kind of filter your service suppo #### SearchQueryHandler -You can not point to a search request with an ID like you point to a playlist or a channel, simply because one and the +You cannot point to a search request with an ID like you point to a playlist or a channel, simply because one and the same search request might have a changing outcome deepening on the country or the time you send the request. This is why the idea of an "ID" is replaced by a "SearchString" in the [SearchQueryHandler](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/SearchQueryHandler.html) These work like regular ListLinkHandler, accept that you don't have to implement the methodes `onAcceptUrl()` diff --git a/docs/03_Implement_a_service.md b/docs/03_Implement_a_service.md index 2572599..3b4db78 100644 --- a/docs/03_Implement_a_service.md +++ b/docs/03_Implement_a_service.md @@ -1,11 +1,20 @@ # Implement a service Services or better service connectors are the parts of NewPipe which communicative with an actual service like YouTube. -This Page will describe how you can implement and add your own. Please make sure you red and understand the +This Page will describe how you can implement and add your own. Please make sure you read and understand the [Concept of Extractors](https://teamnewpipe.github.io/documentation/01_Concept_of_the_extractor/) and the [Concept of LinkHandler](https://teamnewpipe.github.io/documentation/02_Concept_of_LinkHandler/) before implementing your own Service. +__The parts of a service:__ + +- [Head of Service](#head-of-service) +- [Stream](#stream) +- [Search](#search) +- [Channel](#channel) _(optional)_ +- [Playlist](#playlist) _(optional)_ +- [Kiosk](#kiosk) _(optional)_ + ### Allowed Libraries The NewPipe Extractor already comes a long with a lot of usable tools and external libraries that should make extracting easy. @@ -46,15 +55,16 @@ of NewPipe. This way the will become an official part of the NewPipe Extractor. Every service has an ID, which will be set when this list gets created. You set this Id by entering it in the constructor. So when adding your service just give it the ID of the previously last service in the list incremented by one. -### Search -- [SearchExtractor](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/search/SearchExtractor.html) -- [SearchQueryHandlerFactory](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/SearchQueryHandlerFactory.html) - ### Stream - [StreamExtractor](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/stream/StreamExtractor.html) - [StreamInfoItemExtractor](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/stream/StreamInfoItemExtractor.html) - [LinkHandlerFactory](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/LinkHandlerFactory.html) +### Search +- [SearchExtractor](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/search/SearchExtractor.html) +- [SearchQueryHandlerFactory](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/SearchQueryHandlerFactory.html) +- [SuggestionExtractor](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/SuggestionExtractor.html) _(optional)_ + ### Channel - [ChannelExtractor](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/channel/ChannelExtractor.html) - [ChannelInfoItemExtractor](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/channel/ChannelExtractor.html) diff --git a/docs/04_Run_changes_in_App.md b/docs/04_Run_changes_in_App.md index b9cb94b..a0d91d0 100644 --- a/docs/04_Run_changes_in_App.md +++ b/docs/04_Run_changes_in_App.md @@ -28,7 +28,7 @@ marven *.jar packages for android and java based on a github or gitlab repositor To the extractor through jitpack, you need to push them to your online repository of your copy that you host either on [github](https://github.com) or [gitlab](https://gitlab.com). It's important to host it on one of both. Now copy your repository url in Http format, go to [jitpack](https://jitpack.io/), and past it there -From here you can grap the latest commit via `GET IT` button. +From here you can grab the latest commit via `GET IT` button. I recomend not to use SNAPSHOT, since I am not sure when snapshot is build. An "implementation" string will be generated for you. Copy this string and replace the `implementation 'com.github.TeamNewPipe:NewPipeExtractor:'` line in the file [/app/build.gradle](https://github.com/TeamNewPipe/NewPipe/blob/dev/app/build.gradle#L58) with it. diff --git a/docs/index.md b/docs/index.md index 8039499..476176d 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,4 +1,4 @@ -# Welcome to the NewPipe Documentaiton. +# Welcome to the NewPipe Documentation. This side is/should be a beginner friendly tutorial and documentation for people who want to use, or write services for the [NewPipe Extractor](https://github.com/TeamNewPipe/NewPipeExtractor). It is an addition to our auto generated [jdoc documentation](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/). @@ -8,7 +8,7 @@ Please be aware that it is also in an early state, so help and [feedback](https: ## Introduction -The NewPipeExtractor is a Java framework for scraping video platform websites in a way that they can be accedes like a normal API. The extractor is the core of the popular Youtube and streaming App [NewPipe](https://newpipe.schabi.org) for android, however it's system independent and also available for other platforms. +The NewPipeExtractor is a Java framework for scraping video platform websites in a way that they can be accedes like a normal API. The extractor is the core of the popular YouTube and streaming App [NewPipe](https://newpipe.schabi.org) for android, however it's system independent and also available for other platforms. The beauty behind this framework is it takes care about the extraction process, error handling etc., so you can take care about what is important: Scraping the website. It focuses on making it possible for the creator of a scraper for a streaming service to create best outcome by least amount of written code.