snappyapple632
GitHub
commit
3e81f09bc4
|
|
@ -1,9 +1,8 @@
|
|||
# Before You Start
|
||||
|
||||
These documents will guide you through the process of creating your own Extractor
|
||||
service of which will enable NewPipe to access additional streaming services, such as the currently supported YouTube and SoundCloud.
|
||||
The whole documentation consists of this page, which explains the general concept of the NewPipeExtractor, as well as our
|
||||
[Jdoc](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/) setup.
|
||||
These documents will guide you through the process of understanding or creating your own Extractor
|
||||
service of which will enable NewPipe to access additional streaming services, such as the currently supported YouTube, SoundCloud and MediaCCC.
|
||||
The whole documentation consists of this page and [Jdoc](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/) setup, which explains the general concept of the NewPipeExtractor.
|
||||
|
||||
__IMPORTANT!!!__ This is likely to be the worst documentation you have ever read, so do not hesitate to
|
||||
[report](https://github.com/teamnewpipe/documentation/issues) if
|
||||
|
|
|
|||
|
|
@ -57,19 +57,20 @@ class MyExtractor extends FutureExtractor {
|
|||
|
||||
Information can be represented as a list. In NewPipe, a list is represented by a
|
||||
[InfoItemsCollector](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/InfoItemsCollector.html).
|
||||
A InfoItemCollector will collect and assemble a list of [InfoItem](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/InfoItem.html).
|
||||
For each item that should be extracted, a new Extractor must be created, and given to the InfoItemCollector via [commit()](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/InfoItemsCollector.html#commit-E-).
|
||||
A InfoItemsCollector will collect and assemble a list of [InfoItem](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/InfoItem.html).
|
||||
For each item that should be extracted, a new Extractor must be created, and given to the InfoItemsCollector via [commit()](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/InfoItemsCollector.html#commit-E-).
|
||||
|
||||
|
||||
|
||||
If you are implementing a list for your service you need to extend InfoItem containing the extracted information
|
||||
and implement an [InfoItemExtractor](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/Extractor.html),
|
||||
that will return the data of one InfoItem.
|
||||
If you are implementing a list in your service you need to implement an [InfoItemExtractor](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/Extractor.html),
|
||||
that will be able to retreve data for one and only one InfoItem. This extractor will then be _comitted_ to the __InfoItemsCollector__ that can collect the type of InfoItems you want to generate.
|
||||
|
||||
A common implementation would look like this:
|
||||
```
|
||||
private MyInfoItemCollector collectInfoItemsFromElement(Element e) {
|
||||
MyInfoItemCollector collector = new MyInfoItemCollector(getServiceId());
|
||||
private SomeInfoItemCollector collectInfoItemsFromElement(Element e) {
|
||||
// See *Some* as something like Stream or Channel
|
||||
// e.g. StreamInfoItemsCollector, and ChannelInfoItemsCollector are provided by NP
|
||||
SomeInfoItemCollector collector = new SomeInfoItemCollector(getServiceId());
|
||||
|
||||
for(final Element li : element.children()) {
|
||||
collector.commit(new InfoItemExtractor() {
|
||||
|
|
@ -90,20 +91,21 @@ private MyInfoItemCollector collectInfoItemsFromElement(Element e) {
|
|||
|
||||
```
|
||||
|
||||
## InfoItems Encapsulated in Pages
|
||||
## ListExtractor
|
||||
|
||||
When a streaming site shows a list of items, it usually offers some additional information about that list like its title, a thumbnail,
|
||||
There is more to know about lists:
|
||||
|
||||
1. When a streaming site shows a list of items, it usually offers some additional information about that list like its title, a thumbnail,
|
||||
and its creator. Such info can be called __list header__.
|
||||
|
||||
When a website shows a long list of items it usually does not load the whole list, but only a part of it. In order to get more items you may have to click on a next page button, or scroll down.
|
||||
2. When a website shows a long list of items it usually does not load the whole list, but only a part of it. In order to get more items you may have to click on a next page button, or scroll down.
|
||||
|
||||
This is why a list in NewPipe lists are chopped down into smaller lists called [InfoItemsPage](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/ListExtractor.InfoItemsPage.html)s. Each page has its own URL, and needs to be extracted separately.
|
||||
Both of these Problems are fixed by the [ListExtractor](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/ListExtractor.html) which takes care about extracting additional metadata about the liast,
|
||||
and by chopping down lists into several pages, so called [InfoItemsPage](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/ListExtractor.InfoItemsPage.html)s.
|
||||
Each page has its own URL, and needs to be extracted separately.
|
||||
|
||||
Additional metadata about the list and extracting multiple pages can be handled by a
|
||||
[ListExtractor](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/ListExtractor.html),
|
||||
and its [ListExtractor.InfoItemsPage](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/ListExtractor.InfoItemsPage.html).
|
||||
|
||||
For extracting list header information it behaves like a regular extractor. For handling `InfoItemsPages` it adds methods
|
||||
For extracting list header information a `ListExtractor` behaves like a regular extractor. For handling `InfoItemsPages` it adds methods
|
||||
such as:
|
||||
|
||||
- [getInitialPage()](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/ListExtractor.html#getInitialPage--)
|
||||
|
|
@ -117,5 +119,46 @@ such as:
|
|||
The reason why the first page is handled special is because many Websites such as YouTube will load the first page of
|
||||
items like a regular web page, but all the others as an AJAX request.
|
||||
|
||||
An InfoItemsPage itself has two constructors which take these parameters:
|
||||
- The __InfoitemsCollector__ of the list that the page should represent
|
||||
- A __nextPageUrl__ which represents the url of the following page (may be null if not page follows).
|
||||
- Optionally __errors__ which is a list of Exceptions that may have happened during extracton.
|
||||
|
||||
Here is a simplified reference implementation of a list extractor that only extracts pages, but not metadata:
|
||||
|
||||
```
|
||||
class MyListExtractor extends ListExtractor {
|
||||
...
|
||||
private Document document;
|
||||
|
||||
...
|
||||
|
||||
public InfoItemsPage<SomeInfoItem> getPage(pageUrl)
|
||||
throws ExtractionException {
|
||||
SomeInfoItemCollector collector = new SomeInfoItemCollector(getServiceId());
|
||||
document = myFunctionToGetThePageHTMLWhatever(pageUrl);
|
||||
|
||||
//remember this part from the simple list extraction
|
||||
for(final Element li : document.children()) {
|
||||
collector.commit(new InfoItemExtractor() {
|
||||
@Override
|
||||
public String getName() throws ParsingException {
|
||||
...
|
||||
}
|
||||
|
||||
@Override
|
||||
public String getUrl() throws ParsingException {
|
||||
...
|
||||
}
|
||||
...
|
||||
}
|
||||
return new InfoItemsPage<SomeInfoItem>(collector, myFunctionToGetTheNextPageUrl(document));
|
||||
}
|
||||
|
||||
public InfoItemsPage<SomeInfoItem> getInitialPage() {
|
||||
//document here got initialzied by the fetch() function.
|
||||
return getPage(getTheCurrentPageUrl(document));
|
||||
}
|
||||
...
|
||||
}
|
||||
```
|
||||
|
|
|
|||
|
|
@ -0,0 +1,78 @@
|
|||
# Maintainers View
|
||||
|
||||
So I want to document some of the views i have when maintaining NewPipe.
|
||||
|
||||
|
||||
### Keep it Streamlined
|
||||
NewPipe is a Player for online videos on a smart phone, by means it is used
|
||||
for entertainment reason. This means it does not have to be some professional
|
||||
application, and it does not have to be complicated to be used.
|
||||
However NewPipe might not focus on the casual user completely as there are
|
||||
many features that are a bit more "tecki" and may require some knowledge about
|
||||
technology, however all in all NewPipe should be easy to use, even for not teck
|
||||
guys.
|
||||
|
||||
1. NewPipe does not have to be a air plane cockpit: __Don't add to much special
|
||||
features__. If people want to do professionally things with Videos they
|
||||
might use professional tools.
|
||||
2. __Design the UI so it does make sense to the user__. Try to make it comply with
|
||||
[material design guidelines](https://material.io/design/guidelines-overview/).
|
||||
3. __Don't add to much features__: Think about the Betamax vs. VHS phenomena
|
||||
or the Unix principle of having one program designed for one specific task:
|
||||
If you add to much functionality you add complexity and this is not appealing
|
||||
to the user. Focus on what NewPipe should be, and make it be only that.
|
||||
|
||||
### Bugfixes
|
||||
|
||||
]
|
||||
|
||||
*Disclaimer: This is a meme maybe in real live it is different. Pleas no shit storm.*
|
||||
|
||||
__Always go for Bugfixes__, as the best application with the best features
|
||||
does not help much if it is broken, or annoying to use. Now if a program
|
||||
is in an early stage it is quite understandable that many things brake. This
|
||||
is one reason why NewPipe still has no 1 in the beginning of its version
|
||||
number.
|
||||
However by now NewPipe is in a stage where there should be a strong focus on
|
||||
stability.
|
||||
|
||||
1. If there are multiple Pull requests open, check the ones with the bugfixes first.
|
||||
2. Do not add to much features every version, as every feature will inevitable
|
||||
introduce more bugs. It is quite ok, if PRs stay open for a while (not to long though).
|
||||
3. If there are bugs that are stale, or open for a while bump them from time
|
||||
to time, so devs know that there is still something left to fix.
|
||||
4. Never accept bugs. From my perception the community does not like to fix bugs, this is why you as a maintainer should
|
||||
especially focus on perusing bugs.
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
Well features are also something that can cause a headache. You should always see adding features critical and question
|
||||
whether that features does make sense, is useful and would actually be an advantage for the app. You should not blindly
|
||||
say yes to features even if they are small, however you should also not directly say no as well. Think about it, may
|
||||
be even for days before deciding whether you want to accept a feature or not. If you are not sure, try it, look into the
|
||||
code, speak with the developer, and then make a decision and justify it. The criteria whether to add a feature or not
|
||||
should be:
|
||||
|
||||
- Is the features just requested by one or two people or was the feature requested by multiple people?
|
||||
- Is the code of the feature written well?
|
||||
- Is it a quick and hacky solution and could a proper solution be implemented later on?
|
||||
- Does the amount of code justify the outcome?
|
||||
|
||||
Maybe people will send a pull request that will add a frequently requested feature, but is implemented in a hacky way,
|
||||
than don't add it, as you might get into trouble with that solution later on. Either through problems of extending the
|
||||
feature, by introducing to much bugs or simply by braking the architecture or the philosophy of NewPipe. If so don't add it.
|
||||
|
||||
### PRs
|
||||
|
||||
If a PR contains one or more features/bugs be curious. The more stuff a PR changes the longer it will take to be added.
|
||||
Also there might be things you are ok with, but then there are other parts that are not ok with and because of these you
|
||||
can't merge it. This is why you should insist to make the dev chop down the PR into multiple smaller PRs if it's possible.
|
||||
|
||||
### Community
|
||||
|
||||
When you talk to the community stay friendly and respectful, and make sure a friendly and respectful tone will stay.
|
||||
When you have a bad day just don't go to GitHub (an advice from my experience ;D ).
|
||||
|
||||
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 14 KiB |
|
|
@ -0,0 +1,3 @@
|
|||
.wy-nav-top i {
|
||||
line-height: 50px;
|
||||
}
|
||||
|
|
@ -16,6 +16,7 @@
|
|||
|
||||
<link rel="stylesheet" href="{{ base_url }}/css/theme.css" type="text/css" />
|
||||
<link rel="stylesheet" href="{{ base_url }}/css/theme_extra.css" type="text/css" />
|
||||
<link rel="stylesheet" href="{{ base_url }}/css/theme_child.css" type="text/css" />
|
||||
<!-- local code syntax highlighting -->
|
||||
<link rel="stylesheet" href="{{ base_url }}/css/github.min.css" type="text/css" />
|
||||
<link rel="stylesheet" href="{{ base_url }}/css/highlight.css" type="text/css" />
|
||||
|
|
|
|||
Loading…
Reference in New Issue