Browse Source

add linkcheck script

add travis control file

test travis

try fix travis

try fix python version

try fix dependency issue

another try to fix travis

another try to fix travis

fuck travis bullshit

try to fix test
pull/8/head
Christian Schabesberger 3 years ago
parent
commit
77999d2252
  1. 19
      .travis.yml
  2. 2
      README.md
  3. 9
      docs/00_Prepare_everything.md
  4. 2
      docs/01_Concept_of_the_extractor.md
  5. 3
      docs/03_Implement_a_service.md
  6. 2
      docs/04_Run_changes_in_App.md
  7. 39
      linkcheck.py
  8. 2
      requirements.txt

19
.travis.yml

@ -0,0 +1,19 @@
language: python
python:
- "3.6"
branches:
only:
- master
before_install:
- "pip3 install -U pip"
- "export PYTHONPATH=$PYTHONPATH:$(pwd)"
cache: pip
install:
- pip install -r requirements.txt
script: python linkcheck.py

2
README.md

@ -1,6 +1,8 @@
NewPipe Tutorial
================
[![travis_build_state](https://api.travis-ci.org/TeamNewPipe/documentation.svg?branch=master)](https://travis-ci.org/TeamNewPipe/documentation)
This is the [tutorial](https://teamnewpipe.github.io/documentation/) for the [NewPipeExtractor](https://github.com/TeamNewPipeExtractor).
It's thought for thous who want to write their own service, or use NewPipeExtractor in their own projects.

9
docs/00_Prepare_everything.md

@ -11,9 +11,9 @@ First and foremost you need to meet certain conditions in order to write your ow
### What you need to know
- Basic understanding of __[git](https://try.github.io)__
- Good __[Java](http://whatpixel.com/best-java-books/)__ knowledge
- Good __[Java](https://whatpixel.com/best-java-books/)__ knowledge
- Good understanding of __[web technology](https://www.w3schools.com/)__
- Basic understanding about __[unit testing](http://www.vogella.com/tutorials/JUnit/article.html)__ and __[JUnit](https://junit.org/)__
- Basic understanding about __[unit testing](https://www.vogella.com/tutorials/JUnit/article.html)__ and __[JUnit](https://junit.org/)__
- Flawless understanding of how to [contribute](https://github.com/TeamNewPipe/NewPipe/blob/dev/.github/CONTRIBUTING.md#code-contribution) to the __NewPipe project__
### What you need to have
@ -39,7 +39,7 @@ If all the checks are green you did everything right, and you are good to go to
Go to _run_ > _edit configurations_ > _add new configuration_ and select "Gradle".
As Gradle Project, select NewPipeExtractor. As task add "test". Now save and you should be able to run.
![tests passed on idea](img/prepare_tests_passed.png)
![tests passed on idea](/img/prepare_tests_passed.png)
# Inclusion criteria for services
@ -67,6 +67,3 @@ After creating you own service you will need to submit it to our [NewPipeExtract
- Media that violates human rights
- Copyright infringement/pirated media

2
docs/01_Concept_of_the_extractor.md

@ -60,7 +60,7 @@ Sometimes information can be represented as a list. In NewPipe a list is represe
A InfoItemCollector will collect and assemble a list of [InfoItem](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/InfoItem.html).
For each item that should be extracted a new Extractor must be created, and given to the InfoItemCollector via [commit()](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/InfoItemsCollector.html#commit-E-).
![InfoItemsCollector_objectdiagram.svg](img/InfoItemsCollector_objectdiagram.svg)
![InfoItemsCollector_objectdiagram.svg](/img/InfoItemsCollector_objectdiagram.svg)
If you are implementing a list for your service you need to extend InfoItem containing the extracted information,
and implement an [InfoItemExtractor](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/Extractor.html)

3
docs/03_Implement_a_service.md

@ -83,7 +83,8 @@ __Parts required to be implemented:__
- [LinkHandlerFactory](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/LinkHandlerFactory.html)
### Search
The SearchExtractor is also required to be implemented. It will take a search query represented as [SearchQueryHandler](link here),
The SearchExtractor is also required to be implemented. It will take a search query represented as
[SearchQueryHandler](https://teamnewpipe.github.io/NewPipeExtractor/javadoc/org/schabi/newpipe/extractor/linkhandler/SearchQueryHandler.html)
and return a list of search results. Since many services support a suggestion popup while you type you will also want to implement
a __SuggestionExtractor__. This will make it possible for the frontend to as well display a suggestion while typing.

2
docs/04_Run_changes_in_App.md

@ -18,7 +18,7 @@ You can then build and run it following [these instructions](https://developer.a
Please also make sure you are comfortable with [adb](https://en.droidwiki.org/wiki/Android_Debug_Bridge) since
you might experience some trouble running your compiled app on a real device, especially under linux where you
sometimes have to adjust the udev rules in order to
[make your device be accessible](http://www.janosgyerik.com/adding-udev-rules-for-usb-debugging-android-devices/).
[make your device be accessible](https://www.janosgyerik.com/adding-udev-rules-for-usb-debugging-android-devices/).
### Run your changes on the Extractor

39
linkcheck.py

@ -0,0 +1,39 @@
#!/usr/bin/python3
import sys
import os
import re
import requests
BASE_URL = "https://teamnewpipe.github.io/documentation"
RETURN_VALUE = 0
os.chdir("docs")
for filename in os.listdir():
#print(filename + ":")
if ".md" in filename:
with open(filename) as file:
filedata = file.read()
for link in re.findall('\[.*\]\(([^\)]*)\)', filedata):
if link.startswith("#"):
checkstring = "# " + link.replace("#", "").replace("-", " ")
if not checkstring in filedata.lower():
RETURN_VALUE = 1
print(filename + ": Could not find target for" + link)
else:
if link.startswith("/"):
link = BASE_URL + link
if not link.startswith("http"):
RETURN_VALUE = 1
print(filename + ": " + link + " is not filled out or not http")
elif not link.startswith("https"):
RETURN_VALUE = 1
print(filename + ": " + link + " is not https")
else:
res = requests.get(link)
if res.status_code != 200:
RETURN_VALUE = 1
print(filename + ": " + link + " returns " + str(res.status_code))
sys.exit(RETURN_VALUE)

2
requirements.txt

@ -0,0 +1,2 @@
App
Requests>=2.19.1
Loading…
Cancel
Save