diff --git a/docs/00_Prepare_everything.md b/docs/00_Prepare_everything.md index 161cb24..c122f07 100644 --- a/docs/00_Prepare_everything.md +++ b/docs/00_Prepare_everything.md @@ -5,9 +5,9 @@ service with which NewPipe will gain support for a dedicated streaming service l The whole documentation consists of this page, which explains the general concept of the NewPipeExtractor, as well as our [Jdoc](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/) setup. -__IMPORTANT!!!__ this is likely to be the worst documentation you have ever red, so do not hesitate to +__IMPORTANT!!!__ this is likely to be the worst documentation you have ever read, so do not hesitate to [report](https://github.com/teamnewpipe/documentation/issues) if -you find any (spelling)errors, incomplete parts or you simply don't understand something. We are an open community +you find any spelling errors, incomplete parts or you simply don't understand something. We are an open community and are open for everyone to help :) ## Setup your dev environment @@ -19,7 +19,7 @@ First and foremost you need to meet certain conditions in order to write your ow - Basic understanding of __[git](https://try.github.io)__ - Good __[Java](https://whatpixel.com/best-java-books/)__ knowledge - Good understanding of __[web technology](https://www.w3schools.com/)__ -- Basic understanding about __[unit testing](https://www.vogella.com/tutorials/JUnit/article.html)__ and __[JUnit](https://junit.org/)__ +- Basic understanding of __[unit testing](https://www.vogella.com/tutorials/JUnit/article.html)__ and __[JUnit](https://junit.org/)__ - Flawless understanding of how to [contribute](https://github.com/TeamNewPipe/NewPipe/blob/dev/.github/CONTRIBUTING.md#code-contribution) to the __NewPipe project__ ### What you need to have @@ -61,11 +61,11 @@ After creating you own service you will need to submit it to our [NewPipeExtract - Basically anything except [NOT allowed content](#not-allowed-content). - Any kind of porn/NSFW that is allowed according to the [US Porn act](https://www.justice.gov/archive/opa/pr/2003/April/03_ag_266.htm). -- Advertisement (may be handled specially tho) +- Advertisement (may be handled specially though) ## NOT allowed Content -- NSFL +- NSFL (Not Safe For Life) - Porn that is not allowed according to [US Porn act](https://www.justice.gov/archive/opa/pr/2003/April/03_ag_266.htm). - Any form of violence - Child pornography diff --git a/docs/01_Concept_of_the_extractor.md b/docs/01_Concept_of_the_extractor.md index aea7bc0..11e970f 100644 --- a/docs/01_Concept_of_the_extractor.md +++ b/docs/01_Concept_of_the_extractor.md @@ -5,9 +5,9 @@ Before we can start coding our own service we need to understand the basic concept of the extractor. There is a pattern you will find all over the code. It is called the __extractor/collector__ pattern. The idea behind it is that the [extractor](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/Extractor.html) -would produce single pieces of data, and the collector would take it and form usable data for the front end out of it. -The collector also controls the parsing process, and takes care about error handling. So if the extractor fails at any -point the collector will decide whether it should continue parsing or not. This requires the extractor to be made out of +would produce single pieces of data, and the collector would collect it to form usable data for the front end. +The collector also controls the parsing process, and takes care of error handling. So if the extractor fails at any +point, the collector will decide whether or not it should continue parsing. This requires the extractor to be made out of many small methods. One method for every data field the collector wants to have. The collectors are provided by NewPipe. You need to take care of the extractors. @@ -92,7 +92,7 @@ private MyInfoItemCollector collectInfoItemsFromElement(Element e) { ## InfoItems encapsulated in pages -When a streaming site shows a list of items it usually offers some additional information about that list, like it's title a thumbnail +When a streaming site shows a list of items it usually offers some additional information about that list, like it's title, a thumbnail, or its creator. Such info can be called __list header__. When a website shows a long list of items it usually does not load the whole list, but only a part of it. In order to get more items you may have to click on a next page button, or scroll down. @@ -116,7 +116,7 @@ such as: The reason why the first page is handled special is because many Websites such as YouTube will load the first page of -items like a regular webpage, but all the others as AJAX request. +items like a regular webpage, but all the others as an AJAX request. diff --git a/docs/02_Concept_of_LinkHandler.md b/docs/02_Concept_of_LinkHandler.md index 230573c..dba0a1a 100644 --- a/docs/02_Concept_of_LinkHandler.md +++ b/docs/02_Concept_of_LinkHandler.md @@ -12,12 +12,12 @@ one unique ID that represents it, like this example: - https://m.youtube.com/watch?v=oHg5SJYRHA0 ### Importand notes about LinkHandler: -- A simple `LinkHandler` will contain the default URL, the ID and the original url. -- `LinkHandler` are ReadOnly -- LinkHandler are also used to determine which part of the extractor can handle a certain link. +- A simple `LinkHandler` will contain the default URL, the ID and the original URL. +- `LinkHandler`s are ReadOnly +- `LinkHandler`s are also used to determine which part of the extractor can handle a certain link. - In order to get one you must either call [fromUrl()](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/LinkHandlerFactory.html#fromUrl-java.lang.String-) or [fromId()](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/LinkHandlerFactory.html#fromId-java.lang.String-) of the the corresponding `LinkHandlerFactory`. -- Every type of Type of Resource has its own LinkHandlerFactory. Eg. YoutubeStreamLinkHandler, YoutubeChannelLinkHandler, etc. +- Every type of Type of Resource has its own `LinkHandlerFactory`. Eg. YoutubeStreamLinkHandler, YoutubeChannelLinkHandler, etc. ### Usage @@ -65,15 +65,15 @@ ListLinkHandler are also created by overriding the [ListLinkHandlerFactory](http additionally to the abstract methods this factory inherits from the LinkHandlerFactory you can override [getAvailableContentFilter()](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/ListLinkHandlerFactory.html#getAvailableContentFilter--) and [getAvailableSortFilter()](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/ListLinkHandlerFactory.html#getAvailableSortFilter--). -Through these you can tell the front end which kind of filter your service support. +Through these you can tell the front end which kind of filter your service supports. #### SearchQueryHandler You cannot point to a search request with an ID like you point to a playlist or a channel, simply because one and the -same search request might have a changing outcome deepening on the country or the time you send the request. This is +same search request might have a different outcome depending on the country or the time you send the request. This is why the idea of an "ID" is replaced by a "SearchString" in the [SearchQueryHandler](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/SearchQueryHandler.html) -These work like regular ListLinkHandler, accept that you don't have to implement the methodes `onAcceptUrl()` +These work like regular ListLinkHandler, except that you don't have to implement the methods `onAcceptUrl()` and `getId()` when overriding [SearchQueryHandlerFactory](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/SearchQueryHandlerFactory.html). diff --git a/docs/03_Implement_a_service.md b/docs/03_Implement_a_service.md index 998a22b..bdb72a1 100644 --- a/docs/03_Implement_a_service.md +++ b/docs/03_Implement_a_service.md @@ -10,7 +10,7 @@ before doing this. Your service does not have to implement everything. Some parts are optional. This is because not all services support every feature other services support. For example, it might be that a certain service does not support channels. If so, you can leave out the implementation of channels, and make the corresponding -factory methode of the your __StreamingService__ implementation return __null__. The forntend will handle the lack of +factory method of the your __StreamingService__ implementation return __null__. The frontend will handle the lack of having channels then. However if you start to implement one of the optional parts of the list below, you have to implement all parts/classes @@ -34,9 +34,9 @@ which will give you a little help with that. __Use Regex with care!!!__ Avoid it ask us to introduce a new library than start using regex to often. - Html/XML Parsing: [jsoup](https://jsoup.org/apidocs/overview-summary.html) -- JSON Parsiong: [nanojson](https://github.com/mmastrac/nanojson#parser-example) +- JSON Parsing: [nanojson](https://github.com/mmastrac/nanojson#parser-example) - JavaScript Parsing/Execution: [Mozilla Rhino](https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Rhino/Documentation) -- Link dectection in strings: [AutoLink](https://github.com/robinst/autolink-java) +- Link detection in strings: [AutoLink](https://github.com/robinst/autolink-java) If you need to introduce new libraries please tell us before you do it. @@ -69,8 +69,8 @@ So when adding your service just give it the ID of the previously last service i ### Stream -Streams are considered single entities of video or audio, they come along with metainformation like a title, a description, -next/related videos, thumbnail and commends. For getting the url to the actual stream data as well as this metainformation +Streams are considered single entities of video or audio. They come along with metainformation like a title, a description, +next/related videos, a thumbnail and comments. For getting the URL to the actual stream data as well as this metainformation StreamExtractor is used. The LinkHandlerFactory will represent a link to such a stream. StreamInfoItemExtractor will extract one item in a list of items representing such Streams, like a search result or a playlist. Since every Streaming service (obviously) provides streams this is required to implement. Otherwise your service was @@ -105,7 +105,7 @@ __Parts required to be implemented:__ - [ListLinkHandlerFactory](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/ListLinkHandlerFactory.html) ### Channel -A Channel is mostly a [Playlist](#playlist), the only diferens is that it does not represent a simple list of streams, but a +A Channel is mostly a [Playlist](#playlist), the only difference is that it does not represent a simple list of streams, but a user, a channel, or any entity that could be represented as a user. This is why the metadata supported by the channel extractor differs form the one of a playlist. @@ -123,7 +123,7 @@ Your service would look pretty empty if you select it and no video is being disp since users of NewPipe can decide by the settings weather they want to see the kiosk page or not. #### Multiple Kiosks -Most services will implement more than one Kiosk, so a service might have a "Top 20" for different categories like "Country Music", "Techno" ets. +Most services will implement more than one Kiosk, so a service might have a "Top 20" for different categories like "Country Music", "Techno", etc. This is why the extractor will let you implement multiple __KioskExtractors__. Since different kiosk pages might also differ with their HTML structure every page you want to support has to be implemented as its own __KioskExtractor__. However if the pages are similar you can use the same Implementation, but set the page type when you instantiate your __KioskExtractor__ diff --git a/docs/04_Run_changes_in_App.md b/docs/04_Run_changes_in_App.md index 9fa765f..c5a55ec 100644 --- a/docs/04_Run_changes_in_App.md +++ b/docs/04_Run_changes_in_App.md @@ -23,11 +23,11 @@ sometimes have to adjust the udev rules in order to ### Run your changes on the Extractor In order to use the extractor in our app we use [jitpack](https://jitpack.io). This is a build service that can build -marven *.jar packages for android and java based on a github or gitlab repositories. +maven *.jar packages for android and java based on a github or gitlab repositories. To the extractor through jitpack, you need to push them to your online repository of your copy that you host either on [github](https://github.com) or [gitlab](https://gitlab.com). It's important to host -it on one of both. Now copy your repository url in Http format, go to [jitpack](https://jitpack.io/), and past it there +it on one of both. Now copy your repository url in Http format, go to [jitpack](https://jitpack.io/), and past it there. From here you can grab the latest commit via `GET IT` button. I recomend not to use SNAPSHOT, since I am not sure when snapshot is build. An "implementation" string will be generated for you. Copy this string and replace the `implementation 'com.github.TeamNewPipe:NewPipeExtractor:'` line in @@ -43,7 +43,7 @@ with the new extractor. ![image_sync_ok](img/sync_ok.png) -### Trouble shoot +### Troubleshooting If something went wrong on jitpack site, you can check their build log, by selecting the commit you tried to build and click on that little paper symbol next to the `GET IT` button. If it is red it already shows that the build failed. diff --git a/docs/index.md b/docs/index.md index b81427b..f97093f 100644 --- a/docs/index.md +++ b/docs/index.md @@ -3,7 +3,7 @@ -This side is/should be a beginner friendly tutorial and documentation for people who want to use, or write services for the [NewPipe Extractor](https://github.com/TeamNewPipe/NewPipeExtractor). +This site is/should be a beginner friendly tutorial and documentation for people who want to use, or write services for the [NewPipe Extractor](https://github.com/TeamNewPipe/NewPipeExtractor). It is an addition to our auto generated [jdoc documentation](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/). Please be aware that it is also in an early state, so help and [feedback](https://github.com/TeamNewPipe/documentation/issues) is always welcome :D @@ -11,7 +11,7 @@ Please be aware that it is also in an early state, so help and [feedback](https: ## Introduction -The NewPipeExtractor is a Java framework for scraping video platform websites in a way that they can be accedes like a normal API. The extractor is the core of the popular YouTube and streaming App [NewPipe](https://newpipe.schabi.org) for android, however it's system independent and also available for other platforms. +The NewPipeExtractor is a Java framework for scraping video platform websites in a way that they can be accessed like a normal API. The extractor is the core of the popular YouTube and streaming App [NewPipe](https://newpipe.schabi.org) for android, however it's system independent and also available for other platforms. -The beauty behind this framework is it takes care about the extraction process, error handling etc., so you can take care about what is important: Scraping the website. -It focuses on making it possible for the creator of a scraper for a streaming service to create best outcome by least amount of written code. +The beauty behind this framework is it takes care of the extracting process, error handling etc., so you can take care about what is important: Scraping the website. +It focuses on making it possible for the creator of a scraper for a streaming service to create the best outcome with the least amount of written code.