be-gay-do-crimes
/
NewPipeExtractor
mirror of https://github.com/TeamNewPipe/NewPipeExtractor
1
0
Fork 0
Code Issues Releases Wiki Activity
845 Commits 10 Branches 75 Tags 153 MiB
Commit Graph

547 Commits

Author SHA1 Message Date
TobiGr 9eca7df947 Forget to fetch the page 2020-03-16 20:50:08 +01:00
TobiGr edb57840d7 [SoundCloud] Update hard-coded client_id 2020-03-16 19:43:36 +01:00
TobiGr efad0be99f Test if the extractor can get AudiStreams with the hard-coded cliend_id 
SoundCloud client_id are changed on a regular basis now. We might want to remove the hardcoded id completely on a later point of time when client_id is changed to often
 2020-03-16 19:42:35 +01:00
TobiGr 5f39bc4e99 [SoundCloud] Update hardcoded client id 2020-03-16 17:26:30 +01:00
Tobias Groza 65a7eda446
Merge pull request #256 from B0pol/mediaccc 
Support for non-api mediaccc links, invidious shortened links, refactor mediaccc tests
 2020-03-15 17:04:09 +01:00
TobiGr 40b3bc138f Remove hack to fix YoutubeStreamLinkHandlerFactoryTest of vnd.youtube videos 2020-03-14 21:54:54 +01:00
Tobias Groza 1c954fbded
Merge pull request #277 from XiangRongLin/vnd 
Handle youtube stream urls in "vnd.youtube:videoId" format.
 2020-03-14 21:52:49 +01:00
Tobias Groza 9edd0f2675
Merge pull request #288 from Stypox/fix-unlisted-video 
[YouTube] Fix videos without next stream throwing error and add test for unlisted videos
 2020-03-14 21:36:13 +01:00
Stypox 957b78ed9b
[YouTube] Add test for an unlisted video, which has no next stream 2020-03-14 14:04:01 +01:00
Stypox 325a51f73a
[YouTube] Do not crash if there is no next stream, but return null 
Like every other extractor does if there is no next stream
 2020-03-14 14:02:48 +01:00
TobiGr aea0962cfe [SoundCloud] Update hardcoded client id 2020-03-14 12:41:44 +01:00
Mauricio Colli b086e9db3f
[YouTube] Fix id extraction for some channels 
Some channels had no reliable way to get the redirected id in the response,
so saving it for later was a valid alternative.
 2020-03-14 02:39:49 -03:00
Mauricio Colli 00d1ed439b
[YouTube] Fix channel extraction when redirects are in the response 
Some redirects were embed directly into the response as instructions
for the page, instead of the usual http redirects.
 2020-03-14 02:39:48 -03:00
bopol 66518ec444 check wether mediaccc streams url are secure 2020-03-13 20:07:36 +01:00
bopol 0cd5e05b7b MediaCCCLH: use substring instead of replace; improve a bit tests, return Collections.emptyList(); instead of null where it's annotated @NonNull 2020-03-13 20:00:49 +01:00
bopol f742a6bd3e code optimization 2020-03-13 20:00:49 +01:00
bopol 9701c7d800 invidious shortened links 2020-03-13 20:00:49 +01:00
bopol e8e535b815 mediaccc: update linkhandlers & refactor tests 2020-03-13 20:00:49 +01:00
Mauricio Colli 70abd57852
[YouTube] Cache commonly used stream type result 2020-03-07 16:48:32 -03:00
Mauricio Colli 1ef706f567
[YouTube] Detect LIVE videos in the trending page 2020-03-07 16:48:31 -03:00
Xiang Rong Lin cc8fb486ee Handle youtube stream urls in "vnd.youtube:videoId" format. 
The official YouTube app accept intents in this format, which causes other developers to use it.
 2020-03-02 20:40:33 +01:00
Mauricio Colli 6aa4d59b91
[PeerTube] Update video used in age limit test (the previous was 404) 2020-03-01 10:29:30 -03:00
Mauricio Colli cfc278317d
[YouTube] Fail-fast if status exist and is anything other than "OK" 2020-03-01 10:29:29 -03:00
Mauricio Colli 90ae5fbea2 Improve kiosk tests across services 2020-03-01 12:57:18 +01:00
Mauricio Colli e590417cc4 Test if services recognizes their own items urls 2020-03-01 12:57:18 +01:00
TobiGr 35252235b0 Go through all badges when checking if a StreamInfoItem is a live stream 2020-03-01 12:52:37 +01:00
Mauricio Colli 5686a6f562 [YouTube] Detect when a stream is deleted or doesn't exist 
Added a test case as well.
 2020-03-01 12:49:13 +01:00
Mauricio Colli e65333c3ce [YouTube] Detect deleted/nonexistent/invalid channels and playlists 
- Added tests for these cases.
 2020-03-01 12:49:13 +01:00
Mauricio Colli 98e359438a [YouTube] Detect simple 404s in the standard fetch method 2020-03-01 12:49:13 +01:00
Mauricio Colli 408f042127 [YouTube] Fix bug when url isn't present in the browseEndpoint object 2020-03-01 12:49:13 +01:00
Mauricio Colli 342bdbb852 [YouTube] Avoid crashing by letting exceptions bubble up 2020-03-01 12:49:13 +01:00
Mauricio Colli e9644e6216 [YouTube] Handle video premiere's date and duration 2020-03-01 12:49:13 +01:00
Mauricio Colli 2a470ac4f7 [YouTube] Handle videos with no views or with "Recommended to you" text 2020-03-01 12:49:13 +01:00
Mauricio Colli da2c562eb0 [YouTube] Take into account videos that have their views hidden 
Also remove catch because it may hide bugs when the page changes.
 2020-03-01 12:49:13 +01:00
Mauricio Colli f66c20de54 Ignore null-keyed entries when iterating through the response headers 2020-03-01 12:49:13 +01:00
Mauricio Colli 5edd774fc4 Add latest url to the response to make detection of a redirect possible 
Will be latest one in this commit because there's need to check the
history of redirects as of now.
 2020-03-01 12:49:13 +01:00
Mauricio Colli 3441946bea Make test downloader return a response instead of throwing an exception 
The test implementation was throwing an exception instead of just
returning the response and letting the caller handle it.
 2020-03-01 12:49:13 +01:00
wb9688 df219a9453 Don't use StandardCharsets class 2020-03-01 09:41:06 +01:00
wb9688 2807079531 Catch NumberFormatException instead of checking whether viewCount equals "Recommended for you" 2020-02-29 21:28:38 +01:00
wb9688 3e4d2fd64e Improve testChannelUrl() 2020-02-29 18:20:35 +01:00
wb9688 bfe3eb1409 Use suggested try-if code style 2020-02-29 17:18:50 +01:00
wb9688 a6a63e9570 Let isHardcodedClientVersionValid() throw IOException 2020-02-29 17:01:36 +01:00
wb9688 45df8248bf Use Localization in getJsonResponse() 2020-02-29 16:55:07 +01:00
wb9688 157055fbfd Create getJsonResponse() function 2020-02-29 16:42:04 +01:00
wb9688 4f406512c2 Use YoutubeChannelLinkHandlerFactory 2020-02-29 16:26:28 +01:00
wb9688 4ff655352d Reimplement getAgeLimit() 2020-02-28 17:14:26 +01:00
wb9688 6ba03e542b Cache getVideoTab() 2020-02-28 17:05:31 +01:00
wb9688 8bfbae904d Cache getVideo(Primary|Secondary)InfoRenderer() 2020-02-28 17:03:21 +01:00
wb9688 0b940f5cf4 Use "getAgeLimit() != NO_AGE_LIMIT" instead of "getAgeLimit() == 18" 2020-02-28 16:40:50 +01:00
wb9688 beb07a4852 Add test to check whether hardcoded client version is still valid 2020-02-28 16:36:14 +01:00
wb9688 b89b6b1db2 Check if hard-coded client version is valid in separate function 2020-02-28 16:35:24 +01:00
wb9688 7898ba230d Use pbj in YoutubeStreamExtractor 2020-02-28 15:17:47 +01:00
wb9688 612ec06839 Actually fix thumbnail URLs starting with // 2020-02-28 09:36:33 +01:00
wb9688 f946310af5 Fix search queries with a promotedSparklesTextSearchRenderer returning 0 results 2020-02-28 08:57:44 +01:00
wb9688 94a2d5be46 Fix thumbnail URLs starting with // 2020-02-27 19:08:46 +01:00
wb9688 838d834ce2 Get Uploads view of Videos tab 2020-02-27 18:54:27 +01:00
wb9688 365b0329f3 Implement getTextFromObject() function 2020-02-27 17:39:23 +01:00
wb9688 0798bdd5cd Fix getSubscriberCount() for 0 subscribers 2020-02-27 16:26:22 +01:00
wb9688 8ebd971648 Fix channel descriptions consisting of multiple parts 2020-02-27 10:06:35 +01:00
wb9688 880b951088 Fix getUploaderAvatarUrl() in YoutubeStreamExtractor 2020-02-27 09:53:59 +01:00
wb9688 8aa560bef4 Fix video titles consisting of multiple parts due to @ 2020-02-27 09:42:22 +01:00
wb9688 951ed9f0d8 Don't call getUploaderInfo() in onFetchPage() for playlists 
This way YouTube Music playlists will keep working in NewPipe
 2020-02-27 09:27:56 +01:00
wb9688 2399d46ab0 Fix YoutubeChannelExtractor for channels without videos 2020-02-27 09:21:25 +01:00
wb9688 cdbc751b87 Fix getUploaderUrl() in YoutubeStreamInfoItemExtractor 2020-02-27 09:05:41 +01:00
wb9688 db305408e9 Get all expandedShelfContentsRenderers in YoutubeTrendingExtractor 2020-02-26 20:37:20 +01:00
wb9688 a2d8489e16 Parse timestamps in description 2020-02-26 20:27:30 +01:00
wb9688 8d503407ef Parse browseEndpoint and watchEndpoint in descriptions 2020-02-26 19:37:58 +01:00
wb9688 8dc39d517a Fix tests 
Except testGetFullLinksInDescription() in DescriptionTestUnboxing, since that requires other changes first.
 2020-02-26 17:31:01 +01:00
wb9688 88282761f4 Fix YoutubeStreamExtractor not being able to getAgeLimit() in onFetchPage() 2020-02-26 16:19:42 +01:00
wb9688 bb3a3d70bf Use pbj in YoutubeTrendingExtractor 2020-02-26 16:10:24 +01:00
wb9688 1c3a4e710b Check if there are no related streams in getAgeLimit() 2020-02-26 16:03:06 +01:00
wb9688 985c3ec877 Improve getClientVersion() 2020-02-26 15:27:50 +01:00
wb9688 15afbea3e1 Use pbj in YoutubePlaylistExtractor 2020-02-26 15:27:50 +01:00
wb9688 89a41a7f69 Use pbj in YoutubeChannelExtractor 2020-02-26 15:27:50 +01:00
wb9688 0973263aab Use pbj in YoutubeSearchExtractor 2020-02-26 15:27:46 +01:00
TobiGr fc465c8bf7 Fix getUploadDate() for premiered videos with a given date 
Detect publishDate and uploadDate
 2020-02-25 22:54:37 +01:00
TobiGr 96285e0815 Fix getUploadDate() for premiered videos with a given date 2020-02-25 21:51:30 +01:00
wb9688 5a35300a5e Fix parsing upload date for premiered streams 2020-02-25 21:51:30 +01:00
wb9688 729fd2eaee Fix parsing description for navigationEndpoint without urlEndpoint 2020-02-25 21:51:30 +01:00
wb9688 26fb44595f Fix parsing description 2020-02-25 21:51:30 +01:00
TobiGr 0fff03038e Get description with correct links 2020-02-25 21:51:30 +01:00
wb9688 5d08c34322 Fix parsing upload date in YoutubeStreamExtractor 2020-02-25 21:51:30 +01:00
wb9688 bbe1a3cd62 Use getVideo(Primary|Secondary)InfoRenderer() 2020-02-25 21:51:30 +01:00
wb9688 8e6be880d7 Simplify getId() in YoutubeChannelExtractor 2020-02-25 21:51:30 +01:00
TobiGr ecfc27a92c Implement fallback methods for getDescription() and getTextualUploadDate() in YouTubeStreamExtractor 2020-02-25 21:51:30 +01:00
wb9688 eed29ea734 Return null for related streams at age restricted videos 2020-02-25 21:51:30 +01:00
TobiGr 9efcc61ca6 Adapt some tests to match the new URLs 2020-02-25 21:51:30 +01:00
TobiGr f39603f6ef Implement getUploadDate() in YouTubeStreamInfoItemExtractor 2020-02-25 21:51:30 +01:00
wb9688 26ea3dceb6 Fix getViewCount() in YoutubeStreamExtractor 2020-02-25 21:51:30 +01:00
wb9688 76d54abdbf Reimplement more methods in YoutubeStreamExtractor 2020-02-25 21:51:30 +01:00
wb9688 f13c0288cc Reimplement some methods in YoutubeStreamExtractor 2020-02-25 21:51:30 +01:00
wb9688 02b59903fa Remove useless code 2020-02-25 21:51:30 +01:00
TobiGr dd4dd849dc Get client version dynamically in YouTubeSearchExtractor 2020-02-25 21:51:30 +01:00
TobiGr 583e9c1571 Fix getPage() 
Someone should check if there are unstaged changes before committing...
 2020-02-25 21:51:30 +01:00
TobiGr 5842b9ad37 Add getClientVersion() and HARDCODED_CLIENT_VERSION to YouTubeParsingHelper 
Prefer hardcoded client version above the current one when making requests to retrieve the same JSON structure for each request.
 2020-02-25 21:51:30 +01:00
wb9688 5d883d100c Implement pagination in YoutubeSearchExtractor 2020-02-25 21:51:30 +01:00
wb9688 c0a8e01889 Implement pagination in YoutubePlaylistExtractor 2020-02-25 21:51:30 +01:00
wb9688 4039409820 Fix getLikeCount() for certain streams 2020-02-25 21:51:30 +01:00
wb9688 627ab2459f Uncomment fetchPage() 2020-02-25 21:51:30 +01:00
wb9688 2dfa2187ff Implement pagination in YoutubeChannelExtractor 2020-02-25 21:51:30 +01:00