8dc3f28618
[PeerTube] Test one channel url with api and one without
2020-10-24 18:39:55 +02:00
68d23defba
[YouTube] Do not catch every exception on getErrorMessage
...
@B0pol suggestion
2020-10-24 18:39:55 +02:00
3191bd6c70
[YouTube] Use final when possible
2020-10-24 18:39:55 +02:00
55bc01d1ce
[SoundCloud] Use final when possible, ide refactorings
2020-10-24 18:39:55 +02:00
06430c4749
[PeerTube] Use final when possible, ide refactorings
2020-10-24 18:39:06 +02:00
fcb9b6f855
[MediaCCC] Use final when possible, ide refactorings
...
Refactorings on `throws` clause
2020-10-24 18:36:22 +02:00
af5b8b1915
[MediaCCC] Add tests for stream and conference link handlers
2020-10-24 18:36:21 +02:00
07a90d116a
[MediaCCC] Use regex to parse stream and conference urls
2020-10-24 18:36:21 +02:00
0c980b2d64
[PeerTube] Improve channel and stream link handler tests
2020-10-24 18:36:21 +02:00
d130fd79c3
[PeerTube] Prepend "accounts/" to channel id for backward compatibility
2020-10-24 18:36:21 +02:00
492db83ccf
[MediaCCC] Return null instead of empty items collector
...
As per the documentation in the base getRelatedStreams()
2020-10-24 18:34:57 +02:00
3b2a1829d4
[MediaCCC] Extract tags
2020-10-24 18:34:57 +02:00
b461da792f
[MediaCCC] Fix link handler inconsistency providing API links
2020-10-24 18:34:34 +02:00
aeeae87641
[PeerTube] Parse timestamp from url (previously unimplemented)
2020-10-24 18:33:42 +02:00
7ae3cb6d07
[PeerTube] Fix link handler inconsistency providing API links
2020-10-24 18:33:11 +02:00
4349be13af
[PeerTube] Return empty audio stream list instead of null
2020-10-24 18:25:56 +02:00
3b2cfb4ca2
[SoundCloud] Return empty video stream list instead of null
...
Also replace every instance of `return new ArrayList<>();` with `return Collections.emptyList();`
2020-10-24 18:23:46 +02:00
072bae321f
[YouTube] Fix frame extraction for livestreams
...
Use saved playerResponse instead of parsing json every time
2020-10-24 18:23:46 +02:00
7cd410f3fc
[YouTube] Return 0 when there is no timestamp, not -2, as per javadoc
2020-10-24 18:23:45 +02:00
7fb867c166
[YouTube] Fix error message obtaining when there is none
2020-10-24 18:22:44 +02:00
1d7a86e664
[Test] Add base classes for stream extractor tests
...
Refactor all stream extractor tests to use new base class.
Remove check if upload date is in the past: this does not have to hold true: youtube premieres turn up in search results even though they are in the future
2020-10-24 18:22:44 +02:00
29695aed0a
Small field refactor
2020-10-23 16:42:30 +02:00
0e67d820bc
Use static regex pattern for excluded path segments
2020-10-22 20:13:29 +02:00
d3f80d1538
Exlude links which are not channels
2020-10-20 20:06:06 +02:00
6887d59570
[YouTube] Handle urls for Shorts
2020-10-18 12:03:01 +02:00
e945f711c1
Fix SoundCloud test artist account name ( #416 )
2020-10-16 23:42:46 +02:00
be9a6f931c
Fix parsing new ytInitialData
2020-10-16 20:27:40 +02:00
1a9bc8caf1
Merge pull request #414 from Scrxtchy/soundcloud-slash
...
Remove tailing slash from SoundCloud URLs
2020-10-16 09:48:44 +02:00
6a70cb9d50
Remove tailing slash from SoundCloud URLs
...
Fixes #412
2020-10-16 17:17:40 +11:00
9a785f9a7e
Merge pull request #413 from Scrxtchy/soundcloud-test-update
...
Fix SoundCloud test artist account name
2020-10-16 08:05:44 +02:00
538f5d3973
Fix SoundCloud test artist account name
2020-10-16 13:45:05 +11:00
19e862657a
[YouTube] Fix some decryption exceptions by retrying
2020-10-15 20:16:44 +02:00
e3f996e014
Exlude links which are not channels
2020-10-12 20:59:56 +02:00
5ab1b053d2
Update youtube channel link handler tests
...
Co-authored-by: Tobias Groza <TobiGr@users.noreply.github.com>
2020-10-12 20:11:28 +02:00
7abb4b3713
Update extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/linkHandler/YoutubeChannelLinkHandlerFactory.java
...
Fix typos
Co-authored-by: Tobias Groza <TobiGr@users.noreply.github.com>
2020-10-12 19:57:45 +02:00
9d63211a66
Fix typos
...
Co-authored-by: Tobias Groza <TobiGr@users.noreply.github.com>
2020-10-12 19:56:53 +02:00
8c38a5509e
Prevent attribution_link urls to be accepted by channel links handler
2020-10-09 19:07:38 +02:00
4e04991762
Support short custom youtube channel urls
2020-10-09 17:06:59 +02:00
81b5e7cf3d
Fix extractor
2020-10-08 17:56:03 +02:00
ebbfe7f6d4
Skip YouTube's OTF streams
2020-09-29 10:48:02 +02:00
6633f26ec5
Merge pull request #386 from mhmdanas/remove-subtitles-stream-get-url
...
Remove SubtitlesStream#getURL()
2020-08-26 21:32:36 +02:00
7657c2ed1a
Use initSafeStandardsObjects()
2020-08-15 17:08:07 +02:00
1a63dcb355
Remove overriden field `url`
2020-08-12 22:37:36 +03:00
f0f1c009b2
Remove SubtitlesStream#getURL()
2020-08-10 19:28:56 +03:00
cfe88a74c1
Throw ContentNotSupportedException when opening radio uploader channel
2020-08-09 10:07:49 +02:00
0fb73301e3
[YouTube] Fix crash on empty comment
...
Closes #380
2020-08-05 18:25:40 +02:00
932d094d6a
Upgrade featured playlist urls to https
2020-08-02 16:55:18 +02:00
c12ef3a02d
Merge TNP/dev into fynngodau/dev
2020-08-02 16:43:04 +02:00
5ac80624a4
Fix detection of YouTube's decryption function
...
Closes TeamNewPipe/NewPipe#3951
2020-07-28 01:01:10 +02:00
32dff1541f
Merge pull request #370 from wb9688/fix-yt-age-restricted
...
Fix age restricted YouTube videos
2020-07-26 14:17:19 +02:00
ec778200d9
Add comments to the code
2020-07-26 14:15:13 +02:00
667dce034c
Only use fallback methods for clientVersion when clientVersion hasn't been successfully extracted yet
2020-07-26 13:14:25 +02:00
f345f667e2
Extract YouTube's key
2020-07-26 12:01:59 +02:00
4c987a5302
Support YouTube's new continuations for search
2020-07-26 10:01:03 +02:00
81459e289f
Fix age restricted YouTube videos
2020-07-18 19:52:54 +02:00
fc3a63fec5
Update Javadoc for getRelatedStreams()
2020-07-15 18:48:37 +02:00
ff560e907a
Remove getNextStream() from StreamExtractor
2020-07-15 18:48:37 +02:00
a70cb0283f
Merge pull request #314 from wb9688/remove-getnextpageurl
...
Next page stuff
2020-07-07 20:49:38 +02:00
0a5a905bc7
Add final at more places
2020-07-07 20:45:47 +02:00
17ba8a57fa
Clean up the code
2020-07-06 20:19:31 +02:00
9b6fe1dea6
Throw IllegalArgumentException when Page is invalid
2020-07-06 20:19:31 +02:00
4cc312086a
Introduce Page class
2020-07-06 20:19:31 +02:00
e3bfdba135
Remove getNextPageUrl() function from ListExtractor
2020-07-06 20:11:40 +02:00
156a26b64b
Support start= timestamps for YouTube
2020-07-06 14:08:40 +02:00
261471e7f4
remove invidiou.sh from tests
...
dead instance
2020-07-02 21:33:45 +02:00
eb3901acee
update invidious instances
2020-07-02 21:31:05 +02:00
c0ceb5cb27
Merge pull request #340 from wb9688/peertube-deleted-comments
...
Handle isDeleted for PeerTube comments
2020-06-28 10:21:37 +02:00
339f1d9c43
Merge pull request #349 from wb9688/okhttp
...
Use OkHttp for tests like in NewPipe
2020-06-27 20:08:02 +02:00
970bc7f69d
Merge pull request #359 from mauriciocolli/fix-client-id-bug
...
[SoundCloud] Fix concurrency issue when getting the client id
2020-06-27 17:36:24 +02:00
576754982e
Update extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/YoutubeParsingHelper.java
...
Co-authored-by: Tobias Groza <TobiGr@users.noreply.github.com>
2020-06-27 11:35:01 +02:00
7505e828cc
[SoundCloud] Update client id
2020-06-24 01:50:09 -03:00
d2f1c0f40d
[SoundCloud] Fix concurrency issue when getting the client id
2020-06-24 01:50:08 -03:00
b63ae93495
Added a comment
...
and fixed a "typo" in the method parseDurationString replacing Long with Integer.
These are not the droids you are looking for wb9688 😁
2020-06-15 11:27:44 +02:00
92f6754f0f
Merge pull request #338 from wb9688/parse-duration-string-test
...
Add tests for parseDurationString()
2020-06-15 11:01:21 +02:00
5808439c3f
[PeerTube] Change age restricted video in tests
...
The old one wasn't available anymore
2020-06-14 19:57:39 +02:00
3239aa84f2
Handle isDeleted for PeerTube comments
2020-06-14 19:27:20 +02:00
a65e46e4b1
Use OkHttp for tests like in NewPipe
2020-06-11 15:06:08 +02:00
96de834b67
Bandcamp: parse date from radio info item
2020-06-04 19:36:58 +02:00
39b55b5b42
Bandcamp radio stream extractor: test uploader url
2020-06-04 19:32:19 +02:00
9a555d97e5
Remove useless bandcamp tests
2020-06-04 19:31:01 +02:00
e13f341a45
Bandcamp stream extractor: test service id
2020-06-04 19:29:37 +02:00
4dd9540782
Bandcamp license switch order
2020-06-04 19:27:39 +02:00
52103ac61f
Remove "useless" comments
2020-06-04 19:27:13 +02:00
8fa81537c0
Better bandcamp stream extractor description test
2020-06-04 19:17:51 +02:00
0950a95577
Fix for testing wrong object for null or empty
2020-06-04 19:16:08 +02:00
e6ecd91929
Bandcamp: Test full date
2020-06-04 19:12:10 +02:00
27e7aad159
Bandcamp: Improve code style
2020-06-04 19:04:25 +02:00
ea49202f64
Don't display internal license id
...
Co-authored-by: bopol <bopol@e.email>
2020-06-04 18:26:58 +02:00
063f4be766
For #342 : support PeerTube embeds
2020-06-04 14:26:42 +02:00
34b6928124
Don't print stacktrace before throwing new exception
2020-06-04 14:25:38 +02:00
1be20ceeec
Bandcamp: parse date
2020-06-04 14:09:21 +02:00
6822fe3dc8
Bandcamp: Get Stream upload date
2020-06-04 13:03:16 +02:00
e98bff8ae9
Improved Bandcamp radio stream extractor tests
2020-06-04 12:30:53 +02:00
8f6c00f8d8
Use default test in BandcampSearchExtractorTest
2020-06-04 12:12:20 +02:00
de776561b0
Bandcamp: fix loading uploader from streams in search
2020-06-04 12:02:18 +02:00
3940138fc5
Fix Bandcamp capitalization in one more spot
2020-06-03 22:41:27 +02:00
9fa9d920a9
Bandcamp: Implement new methods required due to interface changes
2020-06-03 21:35:11 +02:00
dd955c7f0c
Merge branch 'origin/dev' into dev
2020-06-03 21:13:40 +02:00
33b1121fc7
Add tests for parseDurationString()
2020-05-30 17:20:54 +02:00
e39147202c
Fix YouTube video duration parsing
2020-05-30 09:31:08 +02:00
98055a3c3c
Revert "Add more decryption function regexes"
...
This reverts commit 384d6acd5d
2020-05-29 18:41:40 +02:00
bdb0f2ae6b
Merge pull request #332 from wb9688/learning-playlist
...
Support YouTube's learning playlists
2020-05-29 10:11:08 +02:00
cf18cdb2f5
Merge pull request #329 from wb9688/decryption-regexes
...
Add more decryption function regexes
2020-05-29 10:10:50 +02:00
ab77961b0a
Extract trailer thumbnail in YT learning playlists
2020-05-29 09:55:24 +02:00
4d683e7655
Support YouTube's learning playlists
2020-05-29 09:17:03 +02:00
0b4977bb0c
Merge pull request #333 from wb9688/playerresponse-dashmanifesturl
...
Extract dashManifestUrl in playerResponse
2020-05-28 21:17:07 +02:00
384d6acd5d
Add more decryption function regexes
2020-05-28 11:40:38 +02:00
8f71bde776
Fix bug for YT videos longer than 1000h
2020-05-28 11:34:42 +02:00
aa277e1b6a
Upgrade other dependencies
2020-05-28 11:34:42 +02:00
036cc7e26e
Upgrade jsoup
2020-05-28 11:34:41 +02:00
89b0639faa
New Bandcamp tests for channel extractor
2020-05-25 20:22:23 +02:00
e08256ebc6
Remove "url cleanup" in Bandcamp link handlers
2020-05-25 19:17:27 +02:00
692b2d06f4
Bandcamp: read info item data in place and not in advance
2020-05-25 18:53:15 +02:00
fff21caa87
Fix Bandcamp no avatar test
...
I own the bandcamp artist account called "NewPipeExtractorTest", so it
shouldn't break again.
2020-05-25 18:53:10 +02:00
7ef3e80493
Extract dashManifestUrl in playerResponse
2020-05-15 12:07:26 +02:00
efafbbb88a
Merge pull request #331 from wb9688/playlist-subchannel
...
Add sub-channel stuff to PlaylistInfo
2020-05-12 10:49:13 +02:00
baf5dd3e8f
Merge pull request #311 from B0pol/fixSearchSuggestion
...
Search: add isCorrectedSearch() and fix YoutubeSearchExtractor#getSea…
2020-05-12 09:29:15 +02:00
102f04e297
Add sub-channel stuff to PlaylistInfo
2020-05-12 09:14:29 +02:00
b40ccb5075
Merge pull request #307 from B0pol/playlists_peertube
...
[PeerTube] playlist support
2020-05-11 20:53:03 +02:00
adaf196c99
remove unused import
2020-05-11 12:54:33 +02:00
de26e00079
changed all == null || isEmpty() to isNullOrEmpty()
2020-05-11 11:40:24 +02:00
202a73516c
refactor: add Utils.isNullOrEmpty()
2020-05-11 11:26:18 +02:00
1de0876fb4
Fix #327
2020-05-09 20:06:50 +02:00
7cae95bf09
add tests for subchannel stuff in playlists
2020-05-09 10:15:35 +02:00
1a6e92ebf6
add getSubChannel Name, Url and AvatarUrl for playlists
2020-05-09 09:52:24 +02:00
cf99107745
address yausername change
2020-05-09 00:52:04 +02:00
5bab9d9fc0
[PeerTube] playlist support & refactoring
2020-05-09 00:52:04 +02:00
f3913e241e
Merge pull request #324 from Stypox/music-detector-url
...
[YouTube] Support stream urls in `vnd.youtube://ID` form
2020-05-08 17:43:04 +02:00
7320108c66
Rename SubChannel in channels back to ParentChannel
2020-05-08 16:02:59 +02:00
a408661771
Fix PR review
...
Rename "ParentChannel" to "SubChannel"
Add Tests for sub-channel info fetching
2020-05-07 20:20:12 +02:00
1de1f97cf1
Add parent channel info to StreamExtractor: name, url & avatar url
2020-05-07 20:20:12 +02:00
4234740baa
Add parent channel info to ChannelExtractor: name, url & avatar url
2020-05-07 20:20:12 +02:00
6cff5de400
renaming methods for consistency
2020-05-07 15:40:41 +02:00
5760366d70
use JsonObject#isEmpty instead of JsonObject#equals(JsonUtils.EMPTY_OBJECT)
2020-05-07 15:40:41 +02:00
93e868634e
clarified javadoc for isCorrectedSearch and getSearchSuggestions in SearchExtractor
2020-05-07 15:40:41 +02:00
3757541a9c
adapt new nanojson version
2020-05-07 15:40:41 +02:00
79483a6dc0
«Use this in SearchInfo, for consistency's sake» @Stypox order xD
2020-05-07 15:40:41 +02:00
e8d58e09c9
getSearchSuggestion now returns the corrected query if isCorrectedSearch() is true
2020-05-07 15:40:41 +02:00
5d1c3b3fa2
Use setters in SearchInfo
2020-05-07 15:39:57 +02:00
29b639b454
Search: add isCorrectedSearch() and fix YoutubeSearchExtractor#getSearchSuggestion()
...
isCorrectedSearch: see the javadoc
getSearchSuggestion: in YoutubeSearchExtractor, it was giving the corrected search query. It now gives the suggested query, as it should
2020-05-07 15:39:57 +02:00
a02bfbeae5
[YouTube] Support stream urls in `vnd.youtube://ID` form
2020-05-07 15:31:31 +02:00
bc13e0c616
Merge remote-tracking branch 'upstream/dev' into soundcloudComments
2020-05-05 14:49:21 +02:00
6aae99f832
Fix NPE in defaultAlertsCheck()
2020-05-03 10:28:45 +02:00
a1eabc7589
Return null instead of "" in getTextFromObject()
2020-05-02 08:02:25 +02:00
e7b046db9d
Merge remote-tracking branch 'origin/dev' into dev
2020-04-26 23:12:43 +02:00
a3e8e1c70e
Bandcamp: move own methods to helper class
2020-04-26 23:05:56 +02:00
9bc7a470ab
Replace fori with foreach loop
2020-04-26 23:02:14 +02:00
49157fcf0d
Merge pull request #317 from wb9688/tnp-nanojson
...
Use our fork of nanojson
2020-04-24 21:54:35 +02:00
2564bcf399
refactor comments
...
rename methods in CommentsInfoItemExtractor interface to match the other method names across NewPipeExtractor
remove getName in (Youtube|SoundCloud|Peertube)CommentsExtractor and move it up in CommentsExtractor, return "Comments" instead
2020-04-21 20:05:54 +02:00
f71bd29d58
Bandcamp playlist extractor: add more tests
2020-04-20 23:52:12 +02:00
5009d9f53b
Bandcamp tests: remove certain tests
2020-04-20 23:25:00 +02:00
b5e251c82f
Bandcamp channel extractor: package-private access for getImageUrl(…)
2020-04-20 23:12:43 +02:00
8c70dab8c5
Bandcamp search query handler factory: inline variable
2020-04-20 23:11:16 +02:00
10ae3db118
Bandcamp javadoc: Replace br tags with p tags
2020-04-20 23:03:12 +02:00
9201e0d3ea
Don't declare exceptions that can't actually be thrown
2020-04-20 22:46:54 +02:00
965bce00cf
Bandcamp stream extractor: return NO_AGE_LIMIT
2020-04-20 22:44:45 +02:00
82099592c7
Bandcamp: Move code from SearchExtractor to InfoItemExtractors
2020-04-20 22:44:35 +02:00
00c0333059
Bandcamp: Fetch channelInfo in onFetchPage
2020-04-20 22:41:44 +02:00
67de0285e1
Bandcamp: Generate query JSON in a proper way
2020-04-20 22:03:29 +02:00
c133190c53
Bandcamp capitalization
2020-04-20 21:55:35 +02:00
b100b9873f
Bandcmap: Use nanojson fork instead of org.json
2020-04-20 21:53:04 +02:00
b51699a20e
Implement @TobiGr's suggestions
2020-04-20 14:27:33 +02:00
74f0ab1792
Merge pull request #301 from wb9688/media-ccc-non-api-urls
...
Use media.ccc.de instead of api.media.ccc.de
2020-04-19 17:06:57 +02:00
9c239371f4
Change nanojson dependency to @wb9688's fork
2020-04-19 14:29:39 +02:00
5c0a03328a
Merge 'origin/dev' into dev
2020-04-19 14:26:51 +02:00
0cbbc2a1f9
remove unused import
2020-04-19 14:16:38 +02:00
979c5a7502
Use our fork of nanojson
2020-04-17 19:36:42 +02:00
d4352f9b84
support comments for SoundCloud
2020-04-10 20:41:41 +02:00
4afe657f6f
Add tests for getUrlFromId and test for video-channels in acceptUrlTest
2020-04-10 19:05:38 +03:00
2c9f1260eb
Fix wb9688 review comments
...
* Rename PeertubeUserExtractor to PeertubeAccountExtractor
* Add test for video-channels in PeertubeChannelLinkHandlerFactoryTest
* Compatibility support for older versions (use "accounts/" as default)
2020-04-10 16:40:53 +03:00
636c430743
refactor: create extractors and linkHandler packages for SoundCloud, move YoutubeParsingHelper to youtube package (not linkhandler anymore)
2020-04-10 10:51:05 +02:00
b6e6f403a8
add support for PeerTube channels extraction
2020-04-09 20:37:49 +03:00
a20d53dd98
remove the mention of Opus on wb9688 advise when
...
ContentNotSupportedException is thrown
2020-04-08 15:31:11 +02:00
9a7c6b7ab0
use ContentNotSupportedException for channels without tabs
2020-04-07 14:33:43 +02:00
a22104cbda
add tests for ContentNotSupportedException
2020-04-07 14:33:43 +02:00
093762e793
throw ContentNotSupportedException when content is know to be unsupported
2020-04-07 14:33:43 +02:00
094b87c537
Merge pull request #299 from B0pol/sc_countries
...
add supported countries for soundcloud
2020-04-03 19:12:33 +02:00
bce27a0e22
Rename getValidResponseBody() to getValidJsonResponseBody()
2020-04-03 17:23:18 +02:00
1eb3deb7fd
add supported countries for soundcloud
2020-04-02 11:26:19 +02:00
8a9e137385
Extract some code to getValidResponseBody()
2020-04-01 16:01:21 +02:00
bf24caa5f2
Use media.ccc.de instead of api.media.ccc.de
2020-04-01 13:25:10 +02:00
2af610e3e7
Add tests for other YT Music search types
2020-04-01 10:30:51 +02:00
c7f7bd2442
Fix error when YT Music videos has no uploader URL
2020-04-01 10:30:51 +02:00
ac15df4548
Move YouTube Music search to its own class
2020-04-01 10:30:51 +02:00
cf0f2aff3e
Extract uploader url from certain YouTube Music videos
2020-04-01 10:30:51 +02:00
aa8cea47f3
Refactor YouTube Music search tests
2020-04-01 10:30:51 +02:00
dd434cca01
Fix issue when there is no didYouMeanRenderer in itemSectionRenderer
2020-04-01 10:30:51 +02:00
5a775a4bbe
Use new way of specifying stream count
2020-04-01 10:30:51 +02:00
d58c0f230d
Improve code for YouTube Music search
2020-04-01 10:30:51 +02:00
dc29d87962
Extract YouTube search suggestions
2020-04-01 10:30:51 +02:00
c852b13d5a
Add Referer header so that it also works with HttpsUrlConnection
2020-04-01 10:30:51 +02:00
eb48524411
Add tests for YouTube Music search
2020-04-01 10:30:51 +02:00
2b9b2a78e8
Handle 100+ items in playlist
2020-04-01 10:30:51 +02:00
fb9b9691b7
Improve getYoutubeMusicKeys()
2020-04-01 10:30:51 +02:00
4ddbdf0aee
Disable artist search for now
2020-04-01 10:30:51 +02:00
1762a527c9
Add support for YouTube Music search
2020-04-01 10:30:51 +02:00
69e0624e35
Merge pull request #298 from B0pol/feed_peertube
...
[peertube] add feed url in channels
2020-03-28 12:53:47 +01:00
0ae53c1e50
[peertube] adapt test for channel feed url
2020-03-28 11:48:24 +01:00
bc43f8977d
[peertube] add feed url in channels
2020-03-28 10:22:47 +01:00
647e7cd450
Accept YouTube Music playlists but not YouTube Mixes
2020-03-25 22:24:18 +01:00
bcd2a1092b
Don't accept YouTube Mix playlists
2020-03-25 11:07:28 +01:00
e4874d554e
Merge pull request #296 from mauriciocolli/fix-search-errors-detection
...
Fix search errors detection and refactor search tests
2020-03-24 21:53:32 +01:00
8a9ae32e6b
Merge pull request #292 from wb9688/fix-yt-no-views
...
Fix YouTube videos with no views
2020-03-24 20:27:08 +01:00
0a20c53f1a
Use easier-to-read asserts statements in tests
2020-03-23 18:08:59 -03:00
c921e5d6a5
Remove unnecessary collector creation function from search extractor
2020-03-23 18:08:51 -03:00
6fd9b38ad9
Merge pull request #287 from mauriciocolli/fix-channel-redirect
...
[YouTube] Fix channel with redirects directly in the response
2020-03-23 18:41:56 +01:00
04bfa0ec6a
[PeerTube] Update video used in age limit test (was 404 not found)
2020-03-21 15:55:40 -03:00
921bf30bb7
[YouTube] Add check for channel items with no video count in search
2020-03-21 15:55:39 -03:00
b7f8001a49
[YouTube] Add check for channel items without description in search
2020-03-21 15:55:38 -03:00
9b7999fe54
[YouTube] Check if channel item has subscription count in search
2020-03-21 15:55:37 -03:00
9704fc9952
Improve search extractor tests for services
2020-03-21 15:55:36 -03:00
d72130edae
Always return a new item collector in the search extractor
2020-03-21 15:55:34 -03:00
0c27198ba1
Include errors from child collectors in the search collector
2020-03-21 15:55:29 -03:00
2cf8cbf0df
Merge branch 'master' into dev
2020-03-18 22:29:10 +01:00
222d659d9e
[SoundCloud] Don't make separate request for getAudioStreams() in StreamExtractor
...
Signed-off-by: Stypox <stypox@pm.me>
2020-03-18 11:30:15 +01:00
904c9d681f
Fix YouTube videos with no views
2020-03-18 10:07:49 +01:00
265cfb61f3
[SoundCloud] Add test for small playlist, use defaultTestGetPageInNewExtractor
2020-03-17 21:03:50 +01:00
45bb646480
[SoundCloud] Do not overwrite nextPageUrl in PlaylistExtractor
...
Consistent with YouTube and the documentation
2020-03-17 20:56:47 +01:00
a1523eb293
Update Bandcamp service to latest interface changes
2020-03-17 20:40:25 +01:00
c505d4e2b7
[SoundCloud] Remove trailing , in playlist page urls
2020-03-17 20:36:59 +01:00
ae47c9587c
[SoundCloud] Optimize imports in edited files
2020-03-17 20:32:04 +01:00
4389fd3b7b
[SoundCloud] Migrate StreamExtractor to api-v2
2020-03-17 18:06:13 +01:00
5e4ddb368f
[SoundCloud] Fix extractors built from next playlist pages
...
They didn't have the information to calculate another next page url. So now `nextPageUrl` contains a full link with all video ids, and `getPage` takes the first part of the url (containing 15 streams) and produces another `nextPageUrl` with the remaining streams.
Also add a test for this.
2020-03-17 18:04:40 +01:00
b78f788017
Merge branch 'dev' into dev
2020-03-17 18:03:14 +01:00
0e1b4bbf17
[SoundCloud] Test playlists banner: it should not exist
2020-03-17 15:53:25 +01:00
65bdb3bc9d
[SoundCloud] Unignore ignored playlist tests
2020-03-17 15:49:58 +01:00
1558da6f6b
[SoundCloud] Fix playlist next page generation
2020-03-17 15:46:35 +01:00
ca8bf53b61
[SoundCloud] Fix playlist test: number of streams changed
2020-03-17 15:35:33 +01:00
d4aa4a0763
[SoundCloud] Fix typo in ChartsExtractor
2020-03-17 15:25:11 +01:00
4b1121aac7
[SoundCloud] Add tests for api-v2 channel urls
2020-03-17 15:23:13 +01:00
c3d811fde5
[SoundCloud] Use api-v2 in SubscriptionExtractor
...
Also added
2020-03-17 15:22:25 +01:00
d0e66cc600
[SoundCloud] Improve thumbnail url extraction in playlists
...
Prevent NullPointerExceptions and remove duplicate code
2020-03-17 15:13:28 +01:00
f3095713f9
[SoundCloud] Use api-v2 in PlaylistExtractor
...
Rewrote methods to calculate next page url and to get items from it. `api-v2` is different from `api` since the initial playlist page contains (usually) the full info of the first 3 streams and only the id of the other. Then the single tracks can be requested in batch using `/tracks?ids=id1,id2,...`.
2020-03-17 15:12:13 +01:00
9eca7df947
Forget to fetch the page
2020-03-16 20:50:08 +01:00
edb57840d7
[SoundCloud] Update hard-coded client_id
2020-03-16 19:43:36 +01:00
efad0be99f
Test if the extractor can get AudiStreams with the hard-coded cliend_id
...
SoundCloud client_id are changed on a regular basis now. We might want to remove the hardcoded id completely on a later point of time when client_id is changed to often
2020-03-16 19:42:35 +01:00
5f39bc4e99
[SoundCloud] Update hardcoded client id
2020-03-16 17:26:30 +01:00
65a7eda446
Merge pull request #256 from B0pol/mediaccc
...
Support for non-api mediaccc links, invidious shortened links, refactor mediaccc tests
2020-03-15 17:04:09 +01:00
40b3bc138f
Remove hack to fix YoutubeStreamLinkHandlerFactoryTest of vnd.youtube videos
2020-03-14 21:54:54 +01:00
1c954fbded
Merge pull request #277 from XiangRongLin/vnd
...
Handle youtube stream urls in "vnd.youtube:videoId" format.
2020-03-14 21:52:49 +01:00
9edd0f2675
Merge pull request #288 from Stypox/fix-unlisted-video
...
[YouTube] Fix videos without next stream throwing error and add test for unlisted videos
2020-03-14 21:36:13 +01:00
957b78ed9b
[YouTube] Add test for an unlisted video, which has no next stream
2020-03-14 14:04:01 +01:00
325a51f73a
[YouTube] Do not crash if there is no next stream, but return null
...
Like every other extractor does if there is no next stream
2020-03-14 14:02:48 +01:00
aea0962cfe
[SoundCloud] Update hardcoded client id
2020-03-14 12:41:44 +01:00
b086e9db3f
[YouTube] Fix id extraction for some channels
...
Some channels had no reliable way to get the redirected id in the response,
so saving it for later was a valid alternative.
2020-03-14 02:39:49 -03:00
00d1ed439b
[YouTube] Fix channel extraction when redirects are in the response
...
Some redirects were embed directly into the response as instructions
for the page, instead of the usual http redirects.
2020-03-14 02:39:48 -03:00
66518ec444
check wether mediaccc streams url are secure
2020-03-13 20:07:36 +01:00
0cd5e05b7b
MediaCCCLH: use substring instead of replace; improve a bit tests, return Collections.emptyList(); instead of null where it's annotated @NonNull
2020-03-13 20:00:49 +01:00
f742a6bd3e
code optimization
2020-03-13 20:00:49 +01:00
9701c7d800
invidious shortened links
2020-03-13 20:00:49 +01:00
e8e535b815
mediaccc: update linkhandlers & refactor tests
2020-03-13 20:00:49 +01:00
70abd57852
[YouTube] Cache commonly used stream type result
2020-03-07 16:48:32 -03:00
1ef706f567
[YouTube] Detect LIVE videos in the trending page
2020-03-07 16:48:31 -03:00
cc8fb486ee
Handle youtube stream urls in "vnd.youtube:videoId" format.
...
The official YouTube app accept intents in this format, which causes other developers to use it.
2020-03-02 20:40:33 +01:00
6aa4d59b91
[PeerTube] Update video used in age limit test (the previous was 404)
2020-03-01 10:29:30 -03:00
cfc278317d
[YouTube] Fail-fast if status exist and is anything other than "OK"
2020-03-01 10:29:29 -03:00
90ae5fbea2
Improve kiosk tests across services
2020-03-01 12:57:18 +01:00
e590417cc4
Test if services recognizes their own items urls
2020-03-01 12:57:18 +01:00
35252235b0
Go through all badges when checking if a StreamInfoItem is a live stream
2020-03-01 12:52:37 +01:00
5686a6f562
[YouTube] Detect when a stream is deleted or doesn't exist
...
Added a test case as well.
2020-03-01 12:49:13 +01:00
e65333c3ce
[YouTube] Detect deleted/nonexistent/invalid channels and playlists
...
- Added tests for these cases.
2020-03-01 12:49:13 +01:00
98e359438a
[YouTube] Detect simple 404s in the standard fetch method
2020-03-01 12:49:13 +01:00
408f042127
[YouTube] Fix bug when url isn't present in the browseEndpoint object
2020-03-01 12:49:13 +01:00
342bdbb852
[YouTube] Avoid crashing by letting exceptions bubble up
2020-03-01 12:49:13 +01:00
e9644e6216
[YouTube] Handle video premiere's date and duration
2020-03-01 12:49:13 +01:00
2a470ac4f7
[YouTube] Handle videos with no views or with "Recommended to you" text
2020-03-01 12:49:13 +01:00
da2c562eb0
[YouTube] Take into account videos that have their views hidden
...
Also remove catch because it may hide bugs when the page changes.
2020-03-01 12:49:13 +01:00
f66c20de54
Ignore null-keyed entries when iterating through the response headers
2020-03-01 12:49:13 +01:00
5edd774fc4
Add latest url to the response to make detection of a redirect possible
...
Will be latest one in this commit because there's need to check the
history of redirects as of now.
2020-03-01 12:49:13 +01:00
3441946bea
Make test downloader return a response instead of throwing an exception
...
The test implementation was throwing an exception instead of just
returning the response and letting the caller handle it.
2020-03-01 12:49:13 +01:00
df219a9453
Don't use StandardCharsets class
2020-03-01 09:41:06 +01:00
2807079531
Catch NumberFormatException instead of checking whether viewCount equals "Recommended for you"
2020-02-29 21:28:38 +01:00
3e4d2fd64e
Improve testChannelUrl()
2020-02-29 18:20:35 +01:00
bfe3eb1409
Use suggested try-if code style
2020-02-29 17:18:50 +01:00
a6a63e9570
Let isHardcodedClientVersionValid() throw IOException
2020-02-29 17:01:36 +01:00
45df8248bf
Use Localization in getJsonResponse()
2020-02-29 16:55:07 +01:00
157055fbfd
Create getJsonResponse() function
2020-02-29 16:42:04 +01:00
4f406512c2
Use YoutubeChannelLinkHandlerFactory
2020-02-29 16:26:28 +01:00
4ff655352d
Reimplement getAgeLimit()
2020-02-28 17:14:26 +01:00
6ba03e542b
Cache getVideoTab()
2020-02-28 17:05:31 +01:00
8bfbae904d
Cache getVideo(Primary|Secondary)InfoRenderer()
2020-02-28 17:03:21 +01:00
0b940f5cf4
Use "getAgeLimit() != NO_AGE_LIMIT" instead of "getAgeLimit() == 18"
2020-02-28 16:40:50 +01:00
beb07a4852
Add test to check whether hardcoded client version is still valid
2020-02-28 16:36:14 +01:00
b89b6b1db2
Check if hard-coded client version is valid in separate function
2020-02-28 16:35:24 +01:00
7898ba230d
Use pbj in YoutubeStreamExtractor
2020-02-28 15:17:47 +01:00
612ec06839
Actually fix thumbnail URLs starting with //
2020-02-28 09:36:33 +01:00
f946310af5
Fix search queries with a promotedSparklesTextSearchRenderer returning 0 results
2020-02-28 08:57:44 +01:00
94a2d5be46
Fix thumbnail URLs starting with //
2020-02-27 19:08:46 +01:00
838d834ce2
Get Uploads view of Videos tab
2020-02-27 18:54:27 +01:00
365b0329f3
Implement getTextFromObject() function
2020-02-27 17:39:23 +01:00
0798bdd5cd
Fix getSubscriberCount() for 0 subscribers
2020-02-27 16:26:22 +01:00
8ebd971648
Fix channel descriptions consisting of multiple parts
2020-02-27 10:06:35 +01:00
880b951088
Fix getUploaderAvatarUrl() in YoutubeStreamExtractor
2020-02-27 09:53:59 +01:00
8aa560bef4
Fix video titles consisting of multiple parts due to @
2020-02-27 09:42:22 +01:00
951ed9f0d8
Don't call getUploaderInfo() in onFetchPage() for playlists
...
This way YouTube Music playlists will keep working in NewPipe
2020-02-27 09:27:56 +01:00
2399d46ab0
Fix YoutubeChannelExtractor for channels without videos
2020-02-27 09:21:25 +01:00
cdbc751b87
Fix getUploaderUrl() in YoutubeStreamInfoItemExtractor
2020-02-27 09:05:41 +01:00
db305408e9
Get all expandedShelfContentsRenderers in YoutubeTrendingExtractor
2020-02-26 20:37:20 +01:00
a2d8489e16
Parse timestamps in description
2020-02-26 20:27:30 +01:00
8d503407ef
Parse browseEndpoint and watchEndpoint in descriptions
2020-02-26 19:37:58 +01:00
8dc39d517a
Fix tests
...
Except testGetFullLinksInDescription() in DescriptionTestUnboxing, since that requires other changes first.
2020-02-26 17:31:01 +01:00
88282761f4
Fix YoutubeStreamExtractor not being able to getAgeLimit() in onFetchPage()
2020-02-26 16:19:42 +01:00
bb3a3d70bf
Use pbj in YoutubeTrendingExtractor
2020-02-26 16:10:24 +01:00
1c3a4e710b
Check if there are no related streams in getAgeLimit()
2020-02-26 16:03:06 +01:00
985c3ec877
Improve getClientVersion()
2020-02-26 15:27:50 +01:00
15afbea3e1
Use pbj in YoutubePlaylistExtractor
2020-02-26 15:27:50 +01:00
89a41a7f69
Use pbj in YoutubeChannelExtractor
2020-02-26 15:27:50 +01:00
0973263aab
Use pbj in YoutubeSearchExtractor
2020-02-26 15:27:46 +01:00
fc465c8bf7
Fix getUploadDate() for premiered videos with a given date
...
Detect publishDate and uploadDate
2020-02-25 22:54:37 +01:00
96285e0815
Fix getUploadDate() for premiered videos with a given date
2020-02-25 21:51:30 +01:00
5a35300a5e
Fix parsing upload date for premiered streams
2020-02-25 21:51:30 +01:00
729fd2eaee
Fix parsing description for navigationEndpoint without urlEndpoint
2020-02-25 21:51:30 +01:00
26fb44595f
Fix parsing description
2020-02-25 21:51:30 +01:00
0fff03038e
Get description with correct links
2020-02-25 21:51:30 +01:00
5d08c34322
Fix parsing upload date in YoutubeStreamExtractor
2020-02-25 21:51:30 +01:00
bbe1a3cd62
Use getVideo(Primary|Secondary)InfoRenderer()
2020-02-25 21:51:30 +01:00
8e6be880d7
Simplify getId() in YoutubeChannelExtractor
2020-02-25 21:51:30 +01:00
ecfc27a92c
Implement fallback methods for getDescription() and getTextualUploadDate() in YouTubeStreamExtractor
2020-02-25 21:51:30 +01:00
eed29ea734
Return null for related streams at age restricted videos
2020-02-25 21:51:30 +01:00
9efcc61ca6
Adapt some tests to match the new URLs
2020-02-25 21:51:30 +01:00
f39603f6ef
Implement getUploadDate() in YouTubeStreamInfoItemExtractor
2020-02-25 21:51:30 +01:00
26ea3dceb6
Fix getViewCount() in YoutubeStreamExtractor
2020-02-25 21:51:30 +01:00
76d54abdbf
Reimplement more methods in YoutubeStreamExtractor
2020-02-25 21:51:30 +01:00
f13c0288cc
Reimplement some methods in YoutubeStreamExtractor
2020-02-25 21:51:30 +01:00
02b59903fa
Remove useless code
2020-02-25 21:51:30 +01:00
dd4dd849dc
Get client version dynamically in YouTubeSearchExtractor
2020-02-25 21:51:30 +01:00
583e9c1571
Fix getPage()
...
Someone should check if there are unstaged changes before committing...
2020-02-25 21:51:30 +01:00
5842b9ad37
Add getClientVersion() and HARDCODED_CLIENT_VERSION to YouTubeParsingHelper
...
Prefer hardcoded client version above the current one when making requests to retrieve the same JSON structure for each request.
2020-02-25 21:51:30 +01:00
5d883d100c
Implement pagination in YoutubeSearchExtractor
2020-02-25 21:51:30 +01:00
c0a8e01889
Implement pagination in YoutubePlaylistExtractor
2020-02-25 21:51:30 +01:00
4039409820
Fix getLikeCount() for certain streams
2020-02-25 21:51:30 +01:00
627ab2459f
Uncomment fetchPage()
2020-02-25 21:51:30 +01:00
2dfa2187ff
Implement pagination in YoutubeChannelExtractor
2020-02-25 21:51:30 +01:00
a38ab9b791
Return null if channel has no banner
2020-02-25 21:51:30 +01:00
1eca63089b
Count private and deleted videos as ad
2020-02-25 21:51:30 +01:00
c7360e4a46
Fix getThumbnailUrl() in YoutubeChannelInfoItemExtractor
2020-02-25 21:51:30 +01:00
51fb26625a
Implement getSubscriberCount() in YoutubeChannelInfoItemExtractor
2020-02-25 21:51:30 +01:00
6d20b2b1d0
Fix NPE on collecting page items
2020-02-25 21:51:30 +01:00
4462cbe3f1
Reimplement YoutubePlaylistInfoItemExtractor
2020-02-25 21:51:30 +01:00
8aea4d445b
Reimplement YoutubeChannelInfoItemExtractor
2020-02-25 21:51:30 +01:00
3187116a63
Handle premium videos
2020-02-25 21:51:30 +01:00
7442f91aa6
Don't assume which channel tab is videos
2020-02-25 21:51:30 +01:00
316fe0109d
Fix first playlist page
2020-02-25 21:51:30 +01:00
4e57e589ce
Fix first channel page
2020-02-25 21:51:30 +01:00
5816202cc7
Move getInitialData() method to YouTubeParsingHelper
...
Rename ytInitialData to initialData
2020-02-25 21:51:30 +01:00
38aabc6aca
Use plain YoutubeStreamInfoItemExtractor in YoutubeTrendingExtractor
2020-02-25 21:51:30 +01:00
957db062ef
Fix getting uploader for videoRenderer
2020-02-25 21:51:30 +01:00
b88188d419
Move stuff from extractVideoPreviewInfo() into YoutubeStreamInfoItemExtractor and partially fix search
2020-02-25 21:51:30 +01:00
af49b3c487
Fix live streams in extractVideoPreviewInfo()
2020-02-25 21:51:30 +01:00
f33f9466ce
Fix testDescription() in ChannelExtractor
2020-02-25 21:51:30 +01:00
Stypox
Bartosz Rumiński
Scratch
wb9688
Tobias Groza
Bartosz Rumiński
Fynn Godau
mhmdanas
bopol
Robin
Mauricio Colli
wb9688
Roy Yosef
Xiang Rong Lin