5292cc1636
[peertube] test: remove usage of peertube.mastodon.host
...
It's down for months
2021-01-11 19:46:27 +01:00
f3c22da618
comments: add heartedByCreator and extract it for youtube
2021-01-11 18:48:08 +01:00
63c237db41
Only create directories if they do not exist
2021-01-11 07:57:11 +01:00
e82cb1efea
Remove dependency to commons-io
2021-01-10 20:50:38 +01:00
1ea6c6ce54
Prefix mock file from RecordingDownloader with "generated_mock_"
...
Only read those files in MockDownloader
2021-01-10 20:46:42 +01:00
f91916c017
Remove unnecessary file object creation
2021-01-10 20:39:22 +01:00
255c726f20
Add resource path base to DownloaderFactory
2021-01-10 20:27:34 +01:00
35e299759e
Add method to allow resetting youtube client version and key
...
This is needed so that a request is made for each test class when running multiple at once. This way RecordingDownloader records all necessary requests.
This works as long as tests are run sequentially and not in parallel.
2021-01-10 20:24:50 +01:00
f447a7a450
fix import due to downloader package move
2021-01-10 19:49:14 +01:00
f8aa989c42
Add generated json and manually copy client_version.json
2021-01-10 19:38:47 +01:00
285c26eafa
Adjust YoutubeMixPlaylistExtractorTest to use DownloaderFactory
2021-01-10 19:38:41 +01:00
e6e8e39def
Add DownloaderFactory to return a specific downloader based on 2 variables.
...
If the system property 'downloader' is set that use that specific downloader. This is used from gradle by appending `-Ddownloader=ABCD to the command.
ABCD is one of DownloaderType.
The other variable is the static property `DEFAULT_DOWNLOADER` in DownloaderFactory, which can be easily changed as needed inside the IDE according to development needs`.
Normal workflow would be to first use the recording downloader and afterwards only use mocks, if the requests are always staying the same.
2021-01-10 19:36:24 +01:00
7c40fb8bf7
Add additional downloader implementations
...
RecordingDownloader relies on the real downloader and saves the request/response pair into a json file.
MockDownloader uses json files from above and mocks responses for specific requests.
2021-01-10 19:36:24 +01:00
1bcb9c76a7
Generate equals and hashCode for Request
2021-01-10 19:36:24 +01:00
4dad3d60d6
Move DownloaderTestImpl into downloader package
2021-01-10 19:36:21 +01:00
fe3902c669
Un-comment out 360p webm
2021-01-08 19:43:03 +01:00
2d93b23723
[media.ccc.de] Show conference title in search results instead of slug
2021-01-08 14:15:25 +01:00
52f3bd15f2
[media.ccc.de] Add conference title to "recent" kiosk
...
The "conference_title" field was added upstream recently (see https://github.com/voc/voctoweb/issues/495 )
2021-01-08 12:04:39 +01:00
ab62464e86
Merge pull request #495 from B0pol/sepiasearch
...
[PeerTube] Implement sepia search
2021-01-06 11:06:36 +01:00
0bcea9c2cd
add JDoc
2021-01-01 14:37:32 +01:00
0e45c25ea9
Update extractor/src/main/java/org/schabi/newpipe/extractor/services/peertube/extractors/PeertubeStreamInfoItemExtractor.java
...
Co-authored-by: Tobias Groza <TobiGr@users.noreply.github.com>
2021-01-01 14:20:47 +01:00
fae67fbd45
Merge remote-tracking branch 'origin/master' into dev
2021-01-01 13:06:11 +01:00
d63d20f943
Fix JDoc
2020-12-31 18:05:40 +01:00
2cbc3ccb1a
[media.ccc.de] Fix live stream test if there are no current live streams
2020-12-31 17:03:23 +01:00
a896ec41b9
[media.ccc.de] Use Pattern for getting the ive stream id
2020-12-31 17:03:23 +01:00
5dc9a76e3c
[media.ccc.de] Recent kiosk: order streams by upload date
2020-12-30 18:38:09 +01:00
edf8dd0e92
[media.ccc.de] Fix NPE in search results if they contain a future talk
2020-12-30 18:34:28 +01:00
3c8c8e7307
Merge pull request #497 from TiA4f8R/fix-player-js-url-redirect
...
Fix YouTube player JavaScript URL redirection
2020-12-27 18:04:40 +01:00
d30caeb091
Fix YouTube player JavaScript URL redirection
2020-12-27 15:55:03 +01:00
deb9af7bf5
[media.ccc.de] live kiosk - display info of the current talk
2020-12-27 15:52:09 +01:00
b4e1913971
[media.ccc.de] Play live streams
2020-12-27 13:25:48 +01:00
80f4d42226
[Draft] Add live extractor
2020-12-27 11:01:22 +01:00
ed9402c002
[media.ccc.de] Escape "." in id pattern
2020-12-27 11:01:10 +01:00
674f2227f3
make url pattern static
2020-12-27 01:27:37 +01:00
090dade5b3
[media.ccc.de] Add short test for recent kiosk
2020-12-27 01:03:08 +01:00
0cfefe222a
Revert "[media.ccc.de] Get conference name from API URL"
...
This reverts commit d1a0686d2f
2020-12-26 16:55:44 +01:00
d1a0686d2f
[media.ccc.de] Get conference name from API URL
2020-12-26 13:15:03 +01:00
41699798fe
[media.ccc.de] Use web URL for uploaderUrl instead of API URL in recent
2020-12-26 12:47:32 +01:00
b2154c6e87
[media.ccc.de] Fix conferences kiosk test
2020-12-25 23:55:45 +01:00
ed6ae329b3
[media.ccc.de] Add "recent" kiosk
2020-12-25 22:26:58 +01:00
b5e50cc9fb
Merge pull request #484 from TeamNewPipe/yt_,music_search
...
Fix YouTube Music search
2020-12-25 18:30:25 +01:00
d9e2da53c3
Replace explicit string checks whether a playlist íd is a certain YouTube Mix type with calling the dedicated methods
2020-12-25 15:00:34 +01:00
abee0a8df1
Fix typos in YoutubePlaylistLinkHandlerFactory
2020-12-24 10:34:21 +01:00
3033c0b993
[YouTube] Music Mix: Fix playlist not being accepted
...
Regression introduced by YouTube Mix support (#280 )
2020-12-23 21:08:36 +01:00
b62144b49d
[YouTube] Music: Fix search extraction caused by changed JSON response
2020-12-23 21:08:36 +01:00
c9d9bd1e24
Fix typo in ListLinkHandlerFactory#getUrl
2020-12-23 21:05:58 +01:00
3ae924a7f1
Implement sepia search
2020-12-23 15:07:55 +01:00
942d840624
Fix typo in AffiliateService: rename "NO_AFILIATE" to "NO_AFFILIATE"
2020-12-23 12:50:45 +01:00
865c42e273
Fix line break
2020-12-23 12:49:50 +01:00
41a8ed625d
Make some vars final and add annotations to methods
2020-12-22 18:19:26 +01:00
bc6de14952
Extract stream and search meta info for YouTube
...
Add method to extract Google webcache URLs.
2020-12-22 18:19:26 +01:00
838dc1ab54
Fix vulnerability in junit
2020-12-21 23:22:16 +01:00
8d3436565f
[peertube] fix related streams api url when no tags
...
fixes #489
2020-12-19 23:25:37 +01:00
22a415156f
Merge pull request #479 from vkay94/stream-segments
...
Extract stream segments for YouTube
2020-12-19 11:12:22 +01:00
cdcb66b93e
Clean up stuff from mix pl
...
Coming from unclean merge and missing hints from IDE
2020-12-16 08:35:28 +01:00
ba8782a9ed
Use lowercase string for extracting cookies
...
The map in a real response happens to ignore upper-/lowercase differences. Other maps used in unit test may not have that behaviour.
2020-12-16 08:32:04 +01:00
85fa006214
Merge pull request #280 from XiangRongLin/mixPL
...
Extractor for youtube mix (auto-generated playlist)
2020-12-14 18:11:07 +01:00
f90f6fcf92
[YouTube] Don't escape & in getUrlFromNavigationEndpoint for playlists
2020-12-12 20:40:13 +01:00
a338e4e08e
[Youtube] Apply review suggestions and avoid channel mix edge case
2020-12-12 20:32:43 +01:00
22d2f7e400
[Youtube] Add cookies to youtube mix request
...
This way youtube wont return duplicates when getting more items of the mix (but youtube can also track us)
2020-12-12 20:32:40 +01:00
421935401f
[Youtube] Add subchannel functions to mix and fix imports
2020-12-12 20:30:57 +01:00
3ff8619bcc
[Youtube] apply wb9688 suggestion (mix)
...
Channel mix adjusments and test
Don't accept youtube music mix urls as playlist
Don't override playlistData to keep getInitialPage()
Remove json constants
Indentation
2020-12-12 20:30:57 +01:00
822cf307f7
[Youtube] Add _ITEMS constants and improve code style
...
Move thumbnail id exctraction code to getThumbnailUrlFromId
Add test for "My mix" detection to service tests
Use ITEM_COUNT_UNKNOWN everywhere instead of -1 and add some tests
2020-12-12 20:30:57 +01:00
df38b1926c
[Youtube] Add tests and take thumbnail image always from first video of mix
...
Also fix getThumbnailUrl for "My Mix"
2020-12-12 20:30:57 +01:00
68a3948af6
[Youtube] Fix get banner url
2020-12-12 20:30:57 +01:00
ec6b99c082
[Youtube] Adjust mix extractor to new user agent
...
Also extract continuation now
2020-12-12 20:30:57 +01:00
0ff054acb4
[Youtube] Extract initial playlist info
2020-12-12 20:30:57 +01:00
a376792a5d
[Youtube] Handle case where url is in "youtube.com/playlist?list=listID" format.
...
This occurs when sharing a mix from the official youtube app.
2020-12-12 20:30:57 +01:00
327a5730a8
[Youtube] Add some comments to mix
2020-12-12 20:30:57 +01:00
d74265c846
[Youtube] Extract getThumbnailUrl into method and change getUploaderName
2020-12-12 20:30:57 +01:00
0efb854d27
[Youtube] Implement mix extractor for auto-generated playlists.
...
-New YoutubeMixPlaylistExtractor, that extracts from a mix (auto-generated playlist).
-The url has the format of "youtube.com/watch?v=videoID&playlistID",
where playlistID always starts with "RD" and usually followed by the videoID.
-Change YoutubePlaylistLinkHandlerFactory to create a linkhandler with the given url if it is a mix.
-Change YoutubeService to return YoutubeMixPlaylistExtractor if the url is a mix.
2020-12-12 20:30:57 +01:00
2ba27b39af
Stream segments: Remove replaceAll from setPreviewUrl
2020-12-12 15:00:45 +01:00
9a7a224a54
Add tests for stream segments
2020-12-12 10:24:55 +01:00
d3eea4f4be
Add stream segments to StreamInfo
2020-12-12 10:24:29 +01:00
57be1f1123
Use Objects.requireNonNull().
2020-12-11 06:09:57 +05:30
4c19a88612
Use Objects.toString().
2020-12-11 06:09:50 +05:30
b8f64595a2
Use Objects' static equals() and hashCode() methods.
2020-12-11 06:09:44 +05:30
9dbacbc618
Fix comments parsing
2020-12-09 14:57:38 +01:00
ba3e2302bc
Fix name of media.ccc.de service
2020-12-01 23:54:27 +01:00
b7a995187f
Merge pull request #463 from opusforlife2/update_invidious_URLs
...
Update Invidious URL List
2020-11-28 13:30:32 +01:00
650f0920fe
Merge pull request #465 from XiangRongLin/playlist_continuation
...
[YouTube] Fix playlist continuations extraction
2020-11-20 20:48:20 +01:00
334e1e9b53
Fix YouTube comments test
2020-11-20 19:53:53 +01:00
5bceff0083
[YouTube] Fix extraction of next page url for the last page of playlist
2020-11-18 19:03:12 +01:00
8347e14952
[YouTube] Fix playlist continuations extraction
2020-11-17 21:04:53 +01:00
2174055ef8
Update Invidious URL list in Parsing Helper
2020-11-11 15:12:31 +00:00
c0afd5213a
Update Invidious URL list in Link Handler Factory
2020-11-11 14:54:16 +00:00
89a77ae74a
[YouTube] Fix detection of ended livestreams and parse livestream upload date
2020-11-04 16:03:08 +01:00
827f7bd137
[YouTube] Cache deobfuscation and improve requests made
...
Fix age restriction extraction
Automatically fixes more things
2020-11-04 16:02:12 +01:00
4fe28d7e3a
Fix YouTube parse error when only a date is present.
2020-11-04 05:49:42 +05:30
9cf9e7e980
Call existing constructor in DateWrapper.
2020-11-04 05:49:41 +05:30
fe31a90cb3
Remove DateTimeFormatter.ISO_OFFSET_DATE_TIME usage.
2020-11-04 05:49:41 +05:30
b13c7e1c1e
Merge pull request #452 from Stypox/yt-import
...
Implement YouTube subscription import from Google takeout
2020-11-03 20:32:17 +01:00
345e136f6c
create YouTubeCommentsLinkHandlerFactoryTest
...
and remove invidious test from YouTubeCommentsExtractorTest, because it was just testing if the URL is accepted, then the extractor does the same thing, we don't need to test the same thing twice
2020-11-03 19:10:10 +01:00
501ec30152
Implement youtube subscription import from Google takeout
2020-11-02 11:04:48 +01:00
82746d172f
Fix typo in DonationLinkHelper and rewrote swtich statement
2020-11-01 17:34:34 -05:00
2f02c0e6a4
Merge pull request #450 from B0pol/fileutils
...
create FileUtils
2020-11-01 18:13:57 +01:00
f69b3ef05d
create FileUtils
2020-11-01 17:15:34 +01:00
be9e160333
Fix build
2020-11-01 17:04:02 +01:00
4f04cfccca
Switch from Calendar to OffsetDateTime in DateWrapper.
2020-11-01 06:28:54 +05:30
ee3af63c04
Switch to ChronoUnit.
2020-11-01 06:28:53 +05:30
6cc50b57e3
Merge pull request #410 from Scrxtchy/sets-track-title
...
Fix parsing Soundcloud tracks that contain the term 'sets'
2020-10-31 21:43:21 +01:00
b242e1d113
Merge pull request #362 from Stypox/google-search-urls
...
Add support for Google search redirect url
2020-10-31 21:03:48 +01:00
9e53cf0b56
Fix parameter reassignment and other style issues
...
Also remove left-behind debug statement
2020-10-31 20:58:28 +01:00
3fe55b30ba
Add support for Google search redirect url
2020-10-31 20:58:28 +01:00
c190a3029b
Consider protocol as base url when it is a custom one (e.g. vnd.youtube)
2020-10-31 20:58:28 +01:00
b21e59925d
[PeerTube] fix account and channel extractors
2020-10-29 19:52:29 +01:00
30ed4f2d63
Remove any reference to decrypt and improve error message
2020-10-28 20:26:00 +01:00
62912ee834
Merge branch 'master' into dev
2020-10-28 14:28:06 +01:00
947ce3ee10
Fix parsing Soundcloud tracks that contain the term 'sets'
2020-10-28 07:55:02 +11:00
6dc5ab4015
find playerUrl in another place when assetsPattern regex fails
2020-10-27 13:48:58 +01:00
db0ef83d6b
fix youtube decryption and three attemps bug
...
fixes teamnewpipe/newpipe#4572 fixes #439
2020-10-27 13:40:24 +01:00
c1e9857960
fix subscriber count when subscribe is disabled
...
fixes #305
2020-10-26 19:06:01 +01:00
a39a2cca82
fix redirect channels
2020-10-26 19:05:19 +01:00
0a12300c5e
polish tests
2020-10-26 16:57:37 +01:00
01f49e8f66
polish strings
2020-10-26 16:32:39 +01:00
14c0c37d64
Merge pull request #309 from Stypox/improve-stream-tests
...
Improve stream extractor tests & various fixes
2020-10-25 18:49:53 +01:00
366ed71523
[YouTube] Ignore leading characters in video id
2020-10-25 14:34:20 +01:00
8026304a0a
Merge pull request #409 from Bartoshr/bugfix/short-channel-urls
...
Support short custom YouTube channel URLs
2020-10-25 11:43:50 +01:00
57e7994c9e
Add some missing finals, nullables and comments
2020-10-25 08:12:41 +01:00
f11fe87688
[YouTube] Replace outdated PewDiePie video test with another one
...
The old video was made private, and this video (wedding) is probably never going to be removed.
2020-10-24 18:41:41 +02:00
3c55ea9321
[PeerTube] Change age restricted video in tests
...
The old one wasn't available anymore
2020-10-24 18:41:41 +02:00
880ff04a5c
[Test] Add stream dash mpd url test
2020-10-24 18:41:41 +02:00
d0b14644bb
[YouTube/MediaCCC] Consider dates as GMT and not as local
2020-10-24 18:41:41 +02:00
8ce711f40f
[Test] Add sub channel name, url and thumbnail tests
2020-10-24 18:41:41 +02:00
6127826571
[Test] Add stream metadata tests
2020-10-24 18:41:41 +02:00
a087b092b4
[Test] Improve code style and add final
2020-10-24 18:39:55 +02:00
a4097d8d01
[MediaCCC] Return empty list of video-only streams instead of null
2020-10-24 18:39:55 +02:00
8dc3f28618
[PeerTube] Test one channel url with api and one without
2020-10-24 18:39:55 +02:00
68d23defba
[YouTube] Do not catch every exception on getErrorMessage
...
@B0pol suggestion
2020-10-24 18:39:55 +02:00
3191bd6c70
[YouTube] Use final when possible
2020-10-24 18:39:55 +02:00
55bc01d1ce
[SoundCloud] Use final when possible, ide refactorings
2020-10-24 18:39:55 +02:00
06430c4749
[PeerTube] Use final when possible, ide refactorings
2020-10-24 18:39:06 +02:00
fcb9b6f855
[MediaCCC] Use final when possible, ide refactorings
...
Refactorings on `throws` clause
2020-10-24 18:36:22 +02:00
af5b8b1915
[MediaCCC] Add tests for stream and conference link handlers
2020-10-24 18:36:21 +02:00
07a90d116a
[MediaCCC] Use regex to parse stream and conference urls
2020-10-24 18:36:21 +02:00
0c980b2d64
[PeerTube] Improve channel and stream link handler tests
2020-10-24 18:36:21 +02:00
d130fd79c3
[PeerTube] Prepend "accounts/" to channel id for backward compatibility
2020-10-24 18:36:21 +02:00
492db83ccf
[MediaCCC] Return null instead of empty items collector
...
As per the documentation in the base getRelatedStreams()
2020-10-24 18:34:57 +02:00
3b2a1829d4
[MediaCCC] Extract tags
2020-10-24 18:34:57 +02:00
b461da792f
[MediaCCC] Fix link handler inconsistency providing API links
2020-10-24 18:34:34 +02:00
aeeae87641
[PeerTube] Parse timestamp from url (previously unimplemented)
2020-10-24 18:33:42 +02:00
7ae3cb6d07
[PeerTube] Fix link handler inconsistency providing API links
2020-10-24 18:33:11 +02:00
4349be13af
[PeerTube] Return empty audio stream list instead of null
2020-10-24 18:25:56 +02:00
3b2cfb4ca2
[SoundCloud] Return empty video stream list instead of null
...
Also replace every instance of `return new ArrayList<>();` with `return Collections.emptyList();`
2020-10-24 18:23:46 +02:00
072bae321f
[YouTube] Fix frame extraction for livestreams
...
Use saved playerResponse instead of parsing json every time
2020-10-24 18:23:46 +02:00
7cd410f3fc
[YouTube] Return 0 when there is no timestamp, not -2, as per javadoc
2020-10-24 18:23:45 +02:00
7fb867c166
[YouTube] Fix error message obtaining when there is none
2020-10-24 18:22:44 +02:00
1d7a86e664
[Test] Add base classes for stream extractor tests
...
Refactor all stream extractor tests to use new base class.
Remove check if upload date is in the past: this does not have to hold true: youtube premieres turn up in search results even though they are in the future
2020-10-24 18:22:44 +02:00
29695aed0a
Small field refactor
2020-10-23 16:42:30 +02:00
0e67d820bc
Use static regex pattern for excluded path segments
2020-10-22 20:13:29 +02:00
bopol
XiangRongLin
TobiGr
TiA4f8R
Stypox
vkay94
Isira Seneviratne
TheAssassin
opusforlife2
Bri@n
Scratch
Kiril Isakov
Bartosz Rumiński