be-gay-do-crimes
/
newpipe-documentation
mirror of https://github.com/TeamNewPipe/documentation
1
0
Fork 0
Code Issues Releases Wiki Activity

Deployed 32e3e06 with MkDocs version: 1.0.3

This commit is contained in:
Christian Schabesberger 2018-09-21 21:40:35 +01:00
parent 2829547b80
commit 416ea17182
11 changed files with 90 additions and 49 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
00_Prepare_everything/index.html
View File

@ -53,7 +53,7 @@
<li class="toctree-l1">
<a class="" href="..">Welcome to the NewPipe Documentaiton.</a>
<a class="" href="..">Welcome to the NewPipe Documentation.</a>
</li>
<li class="toctree-l1 current">
@ -138,7 +138,7 @@
<h1 id="prepare-everything">Prepare everything</h1>
<p>Welcome to the NewPipe tutorial. This tutorial will guide you through the process of creating your own NewPipeExtractor
service with which NewPipe will gain support for a dedicated streaming service like YouTube, Vimeo or SournCloud. Let's
service with which NewPipe will gain support for a dedicated streaming service like YouTube, Vimeo or SoundCloud. Let's
dive right in. ;D</p>
<h2 id="setup-your-dev-environment">Setup your dev environment</h2>
<p>First and foremost you need to meet certain conditions in order to write your own service.</p>
@ -157,16 +157,16 @@ dive right in. ;D</p>
<li><strong><a href="https://www.java.com/en/download/faq/java8.xml">java 8</a></strong></li>
<li><strong><a href="https://gradle.org/">gradle</a></strong></li>
<li><strong><a href="https://junit.org/junit5/">unit testing</a></strong></li>
<li>I highly recomend <a href="https://www.jetbrains.com/idea/">IDEA Community</a> since it has everything we need.</li>
<li>I highly recommend <a href="https://www.jetbrains.com/idea/">IDEA Community</a> since it has everything we need.</li>
</ul>
</li>
<li>A <strong><a href="https://github.com/">github</a></strong> account</li>
<li>A loot of patience and excitement ;D</li>
<li>A <strong><a href="https://github.com/">Github</a></strong> account</li>
<li>A lot of patience and excitement ;D</li>
</ul>
<p>After making sure all these conditions are provided fork the <a href="https://github.com/TeamNewPipe/NewPipeExtractor">NewPipeExtractor</a>,
using the <a href="https://github.com/TeamNewPipe/NewPipeExtractor#fork-destination-box">fork button</a>.
This way you have your own working repository. Now clone this repository into your local folder in which you want to work in.
Next import the cloned project into your <a href="https://www.jetbrains.com/help/idea/configuring-projects.html#importing-project">ide</a>
Next import the cloned project into your <a href="https://www.jetbrains.com/help/idea/configuring-projects.html#importing-project">IDE</a>
and <a href="https://www.jetbrains.com/help/idea/performing-tests.html">run</a> it.
If all the checks are green you did everything right, and you are good to go to move on to the next chapter.</p>
<h4 id="running-test-in-android-studio">Running test in Android Studio</h4>
@ -208,7 +208,7 @@ As Gradle Project, select NewPipeExtractor. As task add "test". Now save and you
<a href="../01_Concept_of_the_extractor/" class="btn btn-neutral float-right" title="Concept of the Extractor">Next <span class="icon icon-circle-arrow-right"></span></a>
<a href=".." class="btn btn-neutral" title="Welcome to the NewPipe Documentaiton."><span class="icon icon-circle-arrow-left"></span> Previous</a>
<a href=".." class="btn btn-neutral" title="Welcome to the NewPipe Documentation."><span class="icon icon-circle-arrow-left"></span> Previous</a>
</div>

8
01_Concept_of_the_extractor/index.html
View File

@ -53,7 +53,7 @@
<li class="toctree-l1">
<a class="" href="..">Welcome to the NewPipe Documentaiton.</a>
<a class="" href="..">Welcome to the NewPipe Documentation.</a>
</li>
<li class="toctree-l1">
@ -134,7 +134,7 @@
<p>Before we can start coding our own service we need to understand the basic concept of the extractor. There is a pattern
you will find all over the code. It is called the <strong>extractor/collector</strong> pattern. The idea behind it is that
the <a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/Extractor.html">extractor</a>
would produce single peaces of data, and the collector would take it and form usable data for the front end out of it.
would produce single pieces of data, and the collector would take it and form usable data for the front end out of it.
The collector also controls the parsing process, and takes care about error handling. So if the extractor fails at any
point the collector will decide whether it should continue parsing or not. This requires the extractor to be made out of
many small methods. One method for every data field the collector wants to have. The collectors are provided by NewPipe.
@ -214,7 +214,7 @@ that will return the data of one InfoItem.</p>
or its creator. Such info can be called <strong>list header</strong>.</p>
<p>When a website shows a long list of items it usually does not load the whole list, but only a part of it. In order to get more items you may have to click on a next page button, or scroll down. </p>
<p>This is why a list in NewPipe lists are chopped down into smaller lists called <a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/ListExtractor.InfoItemsPage.html">InfoItemsPage</a>s. Each page has its own URL, and needs to be extracted separately.</p>
<p>Additional metainformation about the list such as it's title a thumbnail
<p>Additional metainformation about the list such as its title a thumbnail
or its creator, and extracting multiple pages can be handled by a
<a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/ListExtractor.html">ListExtractor</a>,
and it's <a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/ListExtractor.InfoItemsPage.html">ListExtractor.InfoItemsPage</a>.</p>
@ -228,7 +228,7 @@ such as:</p>
<li><a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/ListExtractor.html#getPage-java.lang.String-">getPage()</a>
returns a ListExtractor.InfoItemsPage by its URL which was retrieved by the <code>getNextPageUrl()</code> method of the previous page.</li>
</ul>
<p>The reason why the first page is handled speciall is because many Websites such as Youtube will load the first page of
<p>The reason why the first page is handled special is because many Websites such as YouTube will load the first page of
items like a regular webpage, but all the others as AJAX request.</p>
</div>

20
02_Concept_of_LinkHandler/index.html
View File

@ -53,7 +53,7 @@
<li class="toctree-l1">
<a class="" href="..">Welcome to the NewPipe Documentaiton.</a>
<a class="" href="..">Welcome to the NewPipe Documentation.</a>
</li>
<li class="toctree-l1">
@ -134,11 +134,11 @@
<h1 id="concept-of-linkhandler">Concept of LinkHandler</h1>
<p><a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/LinkHandler.html">LinkHandler</a>
represent Links to resources like videos, search requests, channels, etc.
The idea behind them is that a video can have multiple links pointig to it, but it has
one unique id that represents it, like this example:</p>
The idea behind them is that a video can have multiple links pointing to it, but it has
one unique ID that represents it, like this example:</p>
<p><a href="https://www.youtube.com/watch?v=oHg5SJYRHA0">oHg5SJYRHA0</a> can be represented as:</p>
<ul>
<li>https://www.youtube.com/watch?v=oHg5SJYRHA0 (default url for youtube)</li>
<li>https://www.youtube.com/watch?v=oHg5SJYRHA0 (default URL for YouTube)</li>
<li>https://youtu.be/oHg5SJYRHA0</li>
<li>https://m.youtube.com/watch?v=oHg5SJYRHA0</li>
</ul>
@ -148,27 +148,27 @@ one unique id that represents it, like this example:</p>
<li><code>LinkHandler</code> are ReadOnly</li>
<li>LinkHandler are also used to determine which part of the extractor can handle a certain link.</li>
<li>In order to get one you must either call
<a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/LinkHandlerFactory.html#fromUrl-java.lang.String-">fromUrl()</a> or <a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/LinkHandlerFactory.html#fromId-java.lang.String-">fromId()</a> of the the coresponding <code>LinkHandlerFactory</code>.</li>
<a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/LinkHandlerFactory.html#fromUrl-java.lang.String-">fromUrl()</a> or <a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/LinkHandlerFactory.html#fromId-java.lang.String-">fromId()</a> of the the corresponding <code>LinkHandlerFactory</code>.</li>
<li>Every type of Type of Resource has its own LinkHandlerFactory. Eg. YoutubeStreamLinkHandler, YoutubeChannelLinkHandler, etc.</li>
</ul>
<h3 id="usage">Usage</h3>
<p>So the typical Usage for getting a LinkHandler would look like this.</p>
<p>So the typical usage for getting a LinkHandler would look like this.</p>
<pre><code class="java">LinkHandlerFactory myLinkHandlerFactory = new MyStreamLinkHandlerFactory();
LinkHandler myVideo = myLinkHandlerFactory.fromUrl(&quot;https://my.service.com/the_video&quot;);
</code></pre>
<h3 id="implementation">Implementation</h3>
<p>In order to Use LinkHandler for your service you must override the apropriate LinkHandlerFactory. eg:</p>
<p>In order to Use LinkHandler for your service you must override the appropriate LinkHandlerFactory. eg:</p>
<pre><code class="java">class MyStreamLinkHandlerFactory extends LinkHandlerFactory {
@Override
public String getId(String url) throws ParsingException {
// Return the ID based on the url.
// Return the ID based on the URL.
}
@Override
public String getUrl(String id) throws ParsingException {
// Return the url based on the id given.
// Return the URL based on the ID given.
}
@Override
@ -192,7 +192,7 @@ additionally to the abstract methods this factory inherits from the LinkHandlerF
and <a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/ListLinkHandlerFactory.html#getAvailableSortFilter--">getAvailableSortFilter()</a>.
Through these you can tell the front end which kind of filter your service support.</p>
<h4 id="searchqueryhandler">SearchQueryHandler</h4>
<p>You can not point to a search request with an ID like you point to a playlist or a channel, simply because one and the
<p>You cannot point to a search request with an ID like you point to a playlist or a channel, simply because one and the
same search request might have a changing outcome deepening on the country or the time you send the request. This is
why the idea of an "ID" is replaced by a "SearchString" in the <a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/SearchQueryHandler.html">SearchQueryHandler</a>
These work like regular ListLinkHandler, accept that you don't have to implement the methodes <code>onAcceptUrl()</code>

59
03_Implement_a_service/index.html
View File

@ -53,7 +53,7 @@
<li class="toctree-l1">
<a class="" href="..">Welcome to the NewPipe Documentaiton.</a>
<a class="" href="..">Welcome to the NewPipe Documentation.</a>
</li>
<li class="toctree-l1">
@ -80,7 +80,19 @@
<ul>
<li><a class="toctree-l3" href="#the-service">The service</a></li>
<li><a class="toctree-l3" href="#allowed-libraries">Allowed Libraries</a></li>
<li><a class="toctree-l3" href="#head-of-service">Head of Service</a></li>
<li><a class="toctree-l3" href="#stream">Stream</a></li>
<li><a class="toctree-l3" href="#search">Search</a></li>
<li><a class="toctree-l3" href="#channel">Channel</a></li>
<li><a class="toctree-l3" href="#playlist">Playlist</a></li>
<li><a class="toctree-l3" href="#kiosk">Kiosk</a></li>
</ul>
@ -127,24 +139,53 @@
<h1 id="implement-a-service">Implement a service</h1>
<p>Services or better service connectors are the parts of NewPipe which communicative with an actual service like YouTube.
This Page will describe how you can implement and add your own. Please make sure you red and understand the
This Page will describe how you can implement and add your own. Please make sure you read and understand the
<a href="https://teamnewpipe.github.io/documentation/01_Concept_of_the_extractor/">Concept of Extractors</a>
and the <a href="https://teamnewpipe.github.io/documentation/02_Concept_of_LinkHandler/">Concept of LinkHandler</a>
before implementing your own Service.</p>
<h2 id="the-service">The service</h2>
<ul>
<li><a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/ServiceList.html">ServiceList</a></li>
<li><a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/StreamingService.html">StreamingService</a></li>
<li><a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/StreamingService.ServiceInfo.html">ServiceInfo</a></li>
</ul>
<p><strong>The parts of a service:</strong></p>
<ul>
<li><a href="#head-of-service">Head of Service</a></li>
<li><a href="#stream">Stream</a></li>
<li><a href="#search">Search</a></li>
<li><a href="#channel">Channel</a> <em>(optional)</em></li>
<li><a href="#playlist">Playlist</a> <em>(optional)</em></li>
<li><a href="#kiosk">Kiosk</a> <em>(optional)</em></li>
</ul>
<h3 id="allowed-libraries">Allowed Libraries</h3>
<p>The NewPipe Extractor already comes a long with a lot of usable tools and external libraries that should make extracting easy.
For some specific (tiny) tasks regex is allowed. Here you can take a look at the
<a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/utils/Parser.html">Parser</a>,
which will give you a little help with that. <strong>Use Regex with care!!!</strong> Avoid it as often as possible. It's better to
ask us to introduce a new library than start using regex to often.</p>
<ul>
<li>Html/XML Parsing: <a href="https://jsoup.org/apidocs/overview-summary.html">jsoup</a></li>
<li>JSON Parsiong: <a href="https://github.com/mmastrac/nanojson#parser-example">nanojson</a></li>
<li>JavaScript Parsing/Execution: <a href="https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Rhino/Documentation">Mozilla Rhino</a></li>
<li>Link dectection in strings: <a href="https://github.com/robinst/autolink-java">AutoLink</a></li>
</ul>
<p>If you need to introduce new libraries please tell us before you do it.</p>
<h3 id="head-of-service">Head of Service</h3>
<p>First of all if you want to create a new service you should create a new package below <code>org.schabi.newpipe.services</code>
, with the name of your service as package name. Here you must put an implementation of these two classes:</p>
<ul>
<li><a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/StreamingService.html">StreamingService</a></li>
<li><a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/StreamingService.ServiceInfo.html">ServiceInfo</a></li>
</ul>
<p><a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/StreamingService.html">StreamingService</a>
is a factory class that will return objects of all important parts of your service.
Every extractor Kisok, Info type you add, and which should be part of your implementation must be instantiated using an
instance of this class.</p>
<p><a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/StreamingService.ServiceInfo.html">ServiceInfo</a>
will return some meta information about your service. Such as the name, the capabilities, and your name as well as your
email address for further notice and maintenance issues. Remember, after extending this class you need to return an
instance of it by through your implementation of
<a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/StreamingService.html#getServiceInfo--"><code>StreamingService.getServiceInfo()</code></a>.</p>
<p>When these two classes are extended by you, you need to add them to the
<a href="https://github.com/TeamNewPipe/NewPipeExtractor/blob/49c2eb51859a58e4bb5ead2d9d0771408f7d59d6/extractor/src/main/java/org/schabi/newpipe/extractor/ServiceList.java#L23">ServiceList</a>
of NewPipe. This way the will become an official part of the NewPipe Extractor.
Every service has an ID, which will be set when this list gets created. You set this Id by entering it in the constructor.
So when adding your service just give it the ID of the previously last service in the list incremented by one.</p>
<h3 id="stream">Stream</h3>
<ul>
<li><a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/stream/StreamExtractor.html">StreamExtractor</a></li>

4
04_Run_changes_in_App/index.html
View File

@ -53,7 +53,7 @@
<li class="toctree-l1">
<a class="" href="..">Welcome to the NewPipe Documentaiton.</a>
<a class="" href="..">Welcome to the NewPipe Documentation.</a>
</li>
<li class="toctree-l1">
@ -153,7 +153,7 @@ marven *.jar packages for android and java based on a github or gitlab repositor
<p>To the extractor through jitpack, you need to push them to your online repository of
your copy that you host either on <a href="https://github.com">github</a> or <a href="https://gitlab.com">gitlab</a>. It's important to host
it on one of both. Now copy your repository url in Http format, go to <a href="https://jitpack.io/">jitpack</a>, and past it there
From here you can grap the latest commit via <code>GET IT</code> button.
From here you can grab the latest commit via <code>GET IT</code> button.
I recomend not to use SNAPSHOT, since I am not sure when snapshot is build. An "implementation" string will be generated
for you. Copy this string and replace the <code>implementation 'com.github.TeamNewPipe:NewPipeExtractor:&lt;commit&gt;'</code> line in
the file <a href="https://github.com/TeamNewPipe/NewPipe/blob/dev/app/build.gradle#L58">/app/build.gradle</a> with it.</p>

2
404.html
View File

@ -43,7 +43,7 @@
<li class="toctree-l1">
<a class="" href="/.">Welcome to the NewPipe Documentaiton.</a>
<a class="" href="/.">Welcome to the NewPipe Documentation.</a>
</li>
<li class="toctree-l1">

16
index.html
View File

@ -8,7 +8,7 @@
<meta name="description" content="None">
<link rel="shortcut icon" href="img/favicon.ico">
<title>Welcome to the NewPipe Documentaiton. - NewPipe Documentation</title>
<title>Welcome to the NewPipe Documentation. - NewPipe Documentation</title>
<!-- local fonts -->
<link rel="stylesheet" href="./css/local_fonts.css" type="text/css" />
@ -20,7 +20,7 @@
<script>
// Current page data
var mkdocs_page_name = "Welcome to the NewPipe Documentaiton.";
var mkdocs_page_name = "Welcome to the NewPipe Documentation.";
var mkdocs_page_input_path = "index.md";
var mkdocs_page_url = null;
</script>
@ -53,10 +53,10 @@
<li class="toctree-l1 current">
<a class="current" href=".">Welcome to the NewPipe Documentaiton.</a>
<a class="current" href=".">Welcome to the NewPipe Documentation.</a>
<ul class="subnav">
<li class="toctree-l2"><a href="#welcome-to-the-newpipe-documentaiton">Welcome to the NewPipe Documentaiton.</a></li>
<li class="toctree-l2"><a href="#welcome-to-the-newpipe-documentation">Welcome to the NewPipe Documentation.</a></li>
<ul>
@ -115,7 +115,7 @@
<li>Welcome to the NewPipe Documentaiton.</li>
<li>Welcome to the NewPipe Documentation.</li>
<li class="wy-breadcrumbs-aside">
</li>
@ -125,12 +125,12 @@
<div role="main">
<div class="section">
<h1 id="welcome-to-the-newpipe-documentaiton">Welcome to the NewPipe Documentaiton.</h1>
<h1 id="welcome-to-the-newpipe-documentation">Welcome to the NewPipe Documentation.</h1>
<p>This side is/should be a beginner friendly tutorial and documentation for people who want to use, or write services for the <a href="https://github.com/TeamNewPipe/NewPipeExtractor">NewPipe Extractor</a>.
It is an addition to our auto generated <a href="https://teamnewpipe.github.io/NewPipeExtractor/javadoc/">jdoc documentation</a>.</p>
<p>Please be aware that it is also in an early state, so help and <a href="https://github.com/TeamNewPipe/documentation/issues">feedback</a> is always welcome :D</p>
<h2 id="introduction">Introduction</h2>
<p>The NewPipeExtractor is a Java framework for scraping video platform websites in a way that they can be accedes like a normal API. The extractor is the core of the popular Youtube and streaming App <a href="https://newpipe.schabi.org">NewPipe</a> for android, however it's system independent and also available for other platforms. </p>
<p>The NewPipeExtractor is a Java framework for scraping video platform websites in a way that they can be accedes like a normal API. The extractor is the core of the popular YouTube and streaming App <a href="https://newpipe.schabi.org">NewPipe</a> for android, however it's system independent and also available for other platforms. </p>
<p>The beauty behind this framework is it takes care about the extraction process, error handling etc., so you can take care about what is important: Scraping the website.
It focuses on making it possible for the creator of a scraper for a streaming service to create best outcome by least amount of written code.</p>
@ -181,5 +181,5 @@ It focuses on making it possible for the creator of a scraper for a streaming se
<!--
MkDocs version : 1.0.3
Build Date UTC : 2018-09-15 18:59:36
Build Date UTC : 2018-09-21 20:40:35
-->

2
search.html
View File

@ -43,7 +43,7 @@
<li class="toctree-l1">
<a class="" href="./.">Welcome to the NewPipe Documentaiton.</a>
<a class="" href="./.">Welcome to the NewPipe Documentation.</a>
</li>
<li class="toctree-l1">

2
search/search_index.json
View File

File diff suppressed because one or more lines are too long

12
sitemap.xml
View File

@ -2,32 +2,32 @@
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>None</loc>
<lastmod>2018-09-15</lastmod>
<lastmod>2018-09-21</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2018-09-15</lastmod>
<lastmod>2018-09-21</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2018-09-15</lastmod>
<lastmod>2018-09-21</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2018-09-15</lastmod>
<lastmod>2018-09-21</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2018-09-15</lastmod>
<lastmod>2018-09-21</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2018-09-15</lastmod>
<lastmod>2018-09-21</lastmod>
<changefreq>daily</changefreq>
</url>
</urlset>

BIN
sitemap.xml.gz
View File

Binary file not shown.