adbbdc7a5b
[YouTube] Fix regex warning: use ' {2}' instead of ' '
2022-03-26 22:07:14 +01:00
24e83997b4
[Bandcamp] Add Java 8 streams
2022-03-26 22:07:12 +01:00
349990fd48
Fix redundant escape \\ in regex in Utils
2022-03-26 22:01:30 +01:00
3bf7aa3762
Cleanup ``TimeAgoParser``
2022-03-26 21:09:31 +01:00
af82edf9dc
Fix checkstyle problems
2022-03-26 20:54:20 +01:00
29408f9356
``YoutubeMusicSearchExtractorTest$Suggestion`` has no mock data
2022-03-26 20:52:28 +01:00
fcee247f9f
Switched to mockonly tests
...
* Made the reason why the tests are mockonly mandatory
2022-03-26 20:52:28 +01:00
0fceb4686b
Added missing mock data
2022-03-26 20:52:27 +01:00
70b20f0d6b
Updated mock data after conflicts
2022-03-26 20:52:27 +01:00
6a6c9359af
Fixed kurzgesagt test - They changed their description...
2022-03-26 20:52:27 +01:00
26596215fa
Refactored ``MediaCCCRecentListExtractorTest``
2022-03-26 20:52:26 +01:00
53962bfd7d
Disabled suggestion test for now
...
Has to be done once YT's backed works again...
2022-03-26 20:52:26 +01:00
66dc5e8bb8
API hardening against changes
2022-03-26 20:52:26 +01:00
5d58156cde
Update mock data
2022-03-26 20:52:25 +01:00
7598b40957
Workaround for incorrect duration for "YT shorts" videos in channels
...
As a workaround 0 is returned as duration for such videos.
See also https://github.com/TeamNewPipe/NewPipe/issues/8034
2022-03-26 20:52:24 +01:00
164e21b5af
Fixed ``MediaCCCRecentKiosk``
...
Ignore faulty data/items (with duration <= 0)
2022-03-26 20:52:23 +01:00
49dfdae5d2
Removed useless throw declaration
2022-03-26 20:51:38 +01:00
69a58fd3d1
Don't use sysout
2022-03-26 20:51:37 +01:00
639be7adda
Minimized some code
2022-03-26 20:51:37 +01:00
2bd4299563
Fixed test: Peertube - Use new comments and videos
...
Old comments/videos got deleted
2022-03-26 20:49:39 +01:00
606a386a98
Fix tests: What's peertube has less comments now
2022-03-26 20:49:39 +01:00
4d1a1c8fb8
Better test for ``MediaCCCRecentListExtractorTest``
...
* Use assertAll
* Show which item is affected
2022-03-26 20:49:38 +01:00
ba43dbaa28
Fix Tests: The channel lost abos
2022-03-26 20:49:38 +01:00
ff436e5740
Fixed test: YT mix playlist
2022-03-26 20:49:38 +01:00
9c07e8a664
Fix useage of wrong object
2022-03-26 20:17:50 +01:00
804e57004f
Fixed new checkstyle problems from dev
2022-03-26 19:46:10 +01:00
33347ac18b
Removed unused methods
...
``contentFilters`` and ``sortfilter`` are get inside the ``ListLinkHandler`` and not the ``ListLinkHandlerFactory``
``ListLinkHandlerFactory`` only passes these values through when ``fromQuery`` is called
2022-03-26 19:43:11 +01:00
ec5b54c38b
Removed unused class
2022-03-26 19:43:10 +01:00
8771af7ba5
Restored original naming
2022-03-26 19:43:09 +01:00
bdadcfa1f7
Legitimately suppress remaining checkstyle warnings
2022-03-26 19:43:08 +01:00
740a37a2de
[YouTube] Fix checkstyle issues
2022-03-26 19:42:40 +01:00
9dc17cd1ca
[Soundcloud] Fix checkstyle issues
2022-03-26 19:40:20 +01:00
9ab32cb2e7
[Peertube] Fix checkstyle issues
2022-03-26 19:40:19 +01:00
9f7e06c817
[MediaCCC] Fix checkstyle issues
2022-03-26 19:40:18 +01:00
3a94839359
[Bandcamp] Fix checkstyle issues
2022-03-26 19:40:17 +01:00
08dff33002
Use Java 8 streams in NewPipe class
2022-03-26 19:40:15 +01:00
c2446ecff0
Use Java 8 streams and deduplicate code in MediaFormat class
2022-03-26 19:40:15 +01:00
d79e20340c
Fix checkstyle issues in root package extractor/
...
Note: not all issues were fixed because MediaFormat and ServiceList use a specific formatting that makes sense for them
2022-03-26 19:40:14 +01:00
ca7c63f273
Fix remaining checkstyle issues in utils/ subpackage
2022-03-26 19:40:13 +01:00
1d5f22e41f
Fix checkstyle issues & more in JsonUtils
...
Also use Java 8 streams and extract duplicate code to getInstanceOf function
2022-03-26 19:40:13 +01:00
87d2834986
Fix checkstyle issues & more in DonationLinkHelper
...
Also add comment about the class being unused and replace the fixLink function with Utils.stringToUrl()
2022-03-26 19:40:12 +01:00
bd7b362040
Fix checkstyle issues & more in DashMpdParser
...
Also remove useless null check on ItagItem.getItag() as that function already throws an exception if there is no itag
2022-03-26 19:40:11 +01:00
8aba2b47b0
Fix checkstyle issues in subpackages with abstract classes
2022-03-26 19:40:10 +01:00
e4951a0623
Refactor code handling http headers in downloader.Request
2022-03-26 19:37:47 +01:00
37690058d2
Add checkstyle to extractor gradle project
...
With respect to NewPipe's checkstyle.xml, checkstyle is disabled for javadoc comments. There is no need for strict rules over comments here in the extractor, as sometimes javadocs are just needed to clarify a small thing and having empty/meaningless @param or @throws is useless.
2022-03-26 19:37:46 +01:00
9284569c84
Merge pull request #774 from TeamNewPipe/dependabot/gradle/org.mozilla-rhino-1.7.14
...
Bump rhino from 1.7.13 to 1.7.14
2022-03-26 17:25:55 +01:00
aa6b7272a4
Merge pull request #804 from Stypox/fix-yt-music-mix
...
[YouTube] Fix music mixes in some countries
2022-03-20 08:35:56 +01:00
09ddb6adbb
[YouTube] Add MockOnly to method testing mixes in related items
2022-03-19 10:54:38 +01:00
8201b3b90e
[YouTube] Parse any playlist (including music mixes) in related items
2022-03-19 10:48:13 +01:00
13f7900816
[YouTube] Add test for genre mix
2022-03-19 10:48:13 +01:00
279f3a20fe
[YouTube] Fix mix tests with invalid video ids
...
Replaces mix tests based on a strange mix type RDQM{videoId} (only reference I could find is https://github.com/ytdl-org/youtube-dl/issues/26228 ) and with an invalid video id of 13 characters (the first two characters were QM, but even after removing QM there still wasn't a video available at that id).
Also updates mocks.
2022-03-19 10:48:13 +01:00
d660c04838
[YouTube] Also test playlist type in playlist tests
2022-03-19 10:48:13 +01:00
401082abe4
[YouTube] Extract playlist type in playlist extractor
2022-03-19 10:48:12 +01:00
63ed06a710
[YouTube] Differentiate genre mixes from normal mixes
...
Note: genre mixes already worked, now they are just considered as such in various video id extraction and in related items
Note 2: now extracting a mix id from a *normal* youtube mix id will fail if the video id wouldn't be exactly 11 characters long
2022-03-19 10:46:31 +01:00
f19660e7d9
[YouTube] Deduplicate code extracting video id from mix id
2022-03-19 10:46:30 +01:00
8f9d5b858e
[YouTube] Remove useless comments about mixes
2022-03-19 10:44:06 +01:00
34a4484c72
[YouTube] Add test for a video with a mix in related items
2022-03-19 10:44:06 +01:00
50db871d89
[YouTube] Extract mixes from streams related items
2022-03-19 10:44:06 +01:00
638da1756c
[Mix] Create MultiInfoItemsCollector
...
It is a collector that can handle many extractor types, to be used when a list contains items of different types (e.g. search). It was renamed from InfoItemsSearchCollector so that it can now be used not just for search but for any extractor needing it. It supports, streams, channels, playlists and *mixes*.
2022-03-19 10:44:06 +01:00
53673d64c6
[Mix] Add type to playlists & playlist items, to distinguish mixes
2022-03-19 10:44:06 +01:00
d8f2031619
Merge pull request #816 from Stypox/mock-only-extension
...
Add `@MockOnly` Junit 5 extension
2022-03-19 10:40:38 +01:00
cc2e4d7104
Merge pull request #815 from litetex/fix-soundcloud-id-once-and-for-all
...
Removed hardcoded soundcloud HARDCODED_CLIENT_ID
2022-03-17 13:54:08 +01:00
c7757c0994
Apply requested changes
2022-03-16 20:14:08 +01:00
35e082248e
Fix YouTube and SoundCloud playlists tests
2022-03-16 19:40:30 +01:00
8b3f90eb7e
[YouTube] Fix extraction of series playlists and don't return the view count as the stream count for learning playlists
...
ITEM_COUNT_UNKNOWN is returned when the JSON array which contains usally the number of videos is less than 3 items.
Also apply the same type of optimizations done in other PlaylistExtractors in YoutubePlaylistExtractor.
2022-03-16 19:18:58 +01:00
58a247907e
Apply changes in all playlist extractors except YoutubePlaylistExtractor
...
Also fix some issues in the extractors, remove uneeded overrides, use the Java 8 Stream API where possible and replace usages of Utils.UTF_8 with StandardCharsets.UTF_8 in these classes.
2022-03-16 19:18:57 +01:00
fc6b45ee36
Implement some methods in PlaylistExtractor
...
This will prevent their override in each child class where the values corresponding to the methods could not be extracted.
2022-03-16 19:18:36 +01:00
73d1fd472f
Add MockOnly junit 5 test extension
2022-03-16 19:03:08 +01:00
ef71a5fa0f
static final instead of final static
2022-03-16 17:24:33 +01:00
0c37c75981
Make getDownloader static & extract getDownloaderType
2022-03-16 17:22:42 +01:00
40aa5104b1
Merge pull request #786 from XiangRongLin/throttling_resilience
...
[Youtube] Make throttling decryption more resilient to api change
2022-03-16 11:03:16 +01:00
ba56be8ef1
Removed hardcoded soundcloud id
...
It never works (long enough) so let's simply remove it...
2022-03-15 21:19:19 +01:00
e726437da3
Update extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/extractors/YoutubeStreamExtractor.java
...
Co-authored-by: Stypox <stypox@pm.me>
2022-03-15 17:10:05 +01:00
e7aee0ca57
Merge pull request #807 from FireMasterK/no-commentsinfo-instance
...
Remove the need for a CommentsInfo instance in CommentsInfo.getMoreItems and fix PeertubeCommentsExtractorTest.Default test
2022-03-15 15:06:56 +01:00
d806984aa8
Merge pull request #800 from TeamNewPipe/dependabot/gradle/com.google.code.gson-gson-2.9.0
...
Bump gson from 2.8.9 to 2.9.0
2022-03-14 19:47:54 +01:00
60cc71e944
Remove the need for a CommentsInfo instance.
2022-03-03 11:48:41 +00:00
dd8687f9fe
[YouTube] Fix music mixes in some countries
2022-03-01 23:02:56 +01:00
e6d334765d
Apply requested codestyle improvements.
2022-02-26 17:12:51 +00:00
6950b362f2
Add test values for failing tests.
2022-02-24 12:55:13 +00:00
5b0ec694a6
Add requested changes.
2022-02-24 12:50:51 +00:00
d290d2e393
Move variable to super Extractor classes.
2022-02-24 12:50:51 +00:00
ab49cb6e18
Add requested changes.
2022-02-24 12:50:51 +00:00
4bd59f65f4
Update exception message.
...
Co-authored-by: Tobi <TobiGr@users.noreply.github.com>
2022-02-24 12:50:51 +00:00
6f3f608ab6
Add support for extracting channel subscriber count in StreamInfo.
2022-02-24 12:50:51 +00:00
1a67ea100a
Refactored code
...
according to review
2022-02-21 19:56:12 +01:00
f79ce1f52a
Refactored YoutubeCommentsExtractor
...
* Use Java Streaming API
* Use StandardCharsets
* Prevented several NPEs/ArrayIndexOutOfBound
* Reformatted some code so that it's easier readable
2022-02-21 19:56:09 +01:00
d337e537dc
[YouTube] Add a comment clarifying why the panelIdentfier can be null
2022-02-18 20:26:19 +01:00
5c8fbeb166
Fix NullPointerException in YouTubeStreamExtractor.getStreamSegments() ( #797 )
...
Fix NullPointerException in YouTubeStreamExtractor.getStreamSegments()
Co-authored-by: litetex <40789489+litetex@users.noreply.github.com>
2022-02-16 23:39:26 +01:00
89900431ab
Bump gson from 2.8.9 to 2.9.0
...
Bumps [gson](https://github.com/google/gson ) from 2.8.9 to 2.9.0.
- [Release notes](https://github.com/google/gson/releases )
- [Changelog](https://github.com/google/gson/blob/master/CHANGELOG.md )
- [Commits](https://github.com/google/gson/compare/gson-parent-2.8.9...gson-parent-2.9.0 )
---
updated-dependencies:
- dependency-name: com.google.code.gson:gson
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-02-14 09:15:09 +00:00
0d6aebce25
[SoundCloud] Update clientId ( #798 )
2022-02-11 17:16:32 +01:00
a2a41e3bf2
Bump rhino from 1.7.13 to 1.7.14
...
Bumps [rhino](https://github.com/mozilla/rhino ) from 1.7.13 to 1.7.14.
- [Release notes](https://github.com/mozilla/rhino/releases )
- [Changelog](https://github.com/mozilla/rhino/blob/master/RELEASE-NOTES.md )
- [Commits](https://github.com/mozilla/rhino/commits )
---
updated-dependencies:
- dependency-name: org.mozilla:rhino
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-02-07 20:29:51 +00:00
545522b80f
[Youtube] Make throttling decryption more resilient to api change
2022-02-01 19:57:34 +01:00
047d75a4c9
Improve code style
2022-02-01 12:56:59 +01:00
50a4b026f8
Adding removal of the braces
2022-01-31 23:52:31 -08:00
a9174f6b9f
Changed the regex to account for
...
nonword characters
2022-01-31 22:29:00 -08:00
59e0aeba88
Merge pull request #768 from litetex/junit-5
...
JUnit 5
2022-01-24 16:56:16 +01:00
65df39b3e3
[YouTube] parse timestamps with >3 digits correctly ( #775 )
...
* [YouTube] parse timestamps with >3 digits correctly
Fixes https://github.com/TeamNewPipe/NewPipe/issues/7530 ; check the
issue for details.
* Remove outdated comment
2022-01-12 10:45:06 +01:00
a579ef2651
Fixed Soundcloud tests
2022-01-07 14:27:24 +01:00
a32edce750
Removed deprecation because AndroidSDK/API < 19
2022-01-07 14:15:55 +01:00
602068937d
Code cleanup
...
Co-authored-by: Tobi <TobiGr@users.noreply.github.com>
2022-01-06 21:04:16 +01:00
dd3f3da828
Kurzgesagt mock data is outdated
2022-01-04 17:44:12 +01:00
a8ebd2cead
Update mock resources for YT Mix playlist
2022-01-04 17:39:14 +01:00
066afd8629
Restored test
2022-01-04 17:33:08 +01:00
3712a669b1
Code cleanup / improvements
...
* Deprecated Utils#UTF-8; see StandardCharsets
* Added more helpful methods to ``ExtractorAsserts``
* Use parameterized (cool new) tests
* Restore functionality of some tests + updated mockdata
* Other code cleanups + Sonarlint improvements
2022-01-04 17:28:31 +01:00
4291a90251
Fixed SoundCloud's search(for tests)
2021-12-27 21:42:43 +01:00
4995709871
Fixed SoundCloud's search(for tests)
...
Getting the initial page was not returning initial page
2021-12-27 21:18:39 +01:00
a2cbdf0991
Updated to JUnit 5
2021-12-27 21:08:08 +01:00
148ba6641d
Updated mock data
2021-12-27 17:08:14 +01:00
4d0a689cb6
Disabled dislike count in tests
2021-12-27 17:04:46 +01:00
a6961c5efa
Fixed NumberFormatException when processing likes
...
See also #7525
2021-12-27 16:54:47 +01:00
65687f3b9b
Removed YT dislike count as it no longer works
2021-12-27 16:54:46 +01:00
1e2e0029fc
[StreamExtractor] Deduplicated a ton of code by using default methods
2021-12-27 16:54:45 +01:00
15b98ffdb4
Remove old ways of getting YT dis/likes
...
* Added additional check for averageRating (in dislikes)
2021-12-27 16:54:44 +01:00
7489502cc8
Disabled ``testRelatedItems`` as it's unreliable
2021-12-11 15:23:11 +01:00
d9b2a2042a
Updated mock data for YoutubeCommentsExtractorTest
2021-12-08 22:11:45 +01:00
a3a0c1ee85
Fixed YoutubeCommentsExtractorTest
...
Old video was deleted / set to private
2021-12-08 22:11:34 +01:00
5c288eb8d7
Updated mock data for YoutubeChannelExtractorTest
2021-12-08 21:51:38 +01:00
75bb83a1b8
Fixed YoutubeChannelExtractorTest
...
Channel was restored, picked a new one
2021-12-08 21:51:15 +01:00
f276bacce5
Fixed SoundcloudStreamExtractorTest
2021-12-08 21:41:27 +01:00
7f99c1c193
Fixed YoutubeStreamExtractorLivestreamTest
2021-12-08 21:16:39 +01:00
652fdf2c36
Updated mock-data for YoutubeStreamExtractorAgeRestrictedTest
2021-12-08 21:08:25 +01:00
3c669d3cb5
Fixed YoutubeStreamExtractorAgeRestrictedTest
...
Looks like a lot of stuff (description, tags, name) got changed on the existing video.
Use a better suited video.
2021-12-08 21:08:03 +01:00
f735788a39
Fixed Soundcloud hardcoded client id
2021-12-08 21:00:53 +01:00
56c8af710e
Code refactoring
2021-11-21 18:16:31 +01:00
c4eca91be9
Fix YouTube likes + dislikes
2021-11-19 21:36:03 +01:00
5028396405
Improve names and formatting
2021-11-14 13:44:13 +01:00
68d7e4e20a
Merge pull request #742 from TeamNewPipe/dependabot/gradle/com.google.code.gson-gson-2.8.9
...
Bump gson from 2.8.8 to 2.8.9
2021-11-08 19:37:09 +01:00
cb80a646d9
Update hardcoded SoundCloud client_id
...
Hah! I got here first, @TobiGr!
2021-11-04 15:28:26 +00:00
396aecef19
Make sure playlists aren't accepted by PeertubeStreamLinkHandlerFactory
2021-11-03 14:41:39 +01:00
38ad1eaac5
[PeerTube] Support /w/ short video links part 2
...
We also need to support it for comments
2021-11-03 13:54:37 +01:00
026751624f
[PeerTube] Support /w/p/ short playlist links
2021-11-03 13:35:30 +01:00
e0b8e142fc
[PeerTube] Support /a/ and /c/ short links
2021-11-03 13:26:27 +01:00
599a91c88c
[PeerTube] Support /w/ short video links
2021-11-03 11:51:58 +01:00
198d37090b
Bump gson from 2.8.8 to 2.8.9
...
Bumps [gson](https://github.com/google/gson ) from 2.8.8 to 2.8.9.
- [Release notes](https://github.com/google/gson/releases )
- [Changelog](https://github.com/google/gson/blob/master/CHANGELOG.md )
- [Commits](https://github.com/google/gson/compare/gson-parent-2.8.8...gson-parent-2.8.9 )
---
updated-dependencies:
- dependency-name: com.google.code.gson:gson
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-01 09:12:53 +00:00
3e8e2a1532
Add support for y2u.be links
2021-10-22 22:48:18 +03:00
b425394e7a
Merge pull request #731 from FireMasterK/short-description
...
Extract Video Short Description in YouTube.
2021-10-15 23:10:00 +02:00
62313962a0
Add default implementation for getShortDescription.
2021-10-04 01:42:29 +01:00
d1add05bc1
Add requested changes.
...
Co-authored-by: TiA4f8R <74829229+TiA4f8R@users.noreply.github.com>
Add final.
Co-authored-by: TiA4f8R <74829229+TiA4f8R@users.noreply.github.com>
2021-10-04 01:36:51 +01:00
3ca9f51b30
Bump jsoup from 1.14.2 to 1.14.3
...
Bumps [jsoup](https://github.com/jhy/jsoup ) from 1.14.2 to 1.14.3.
- [Release notes](https://github.com/jhy/jsoup/releases )
- [Changelog](https://github.com/jhy/jsoup/blob/master/CHANGES )
- [Commits](https://github.com/jhy/jsoup/compare/jsoup-1.14.2...jsoup-1.14.3 )
---
updated-dependencies:
- dependency-name: org.jsoup:jsoup
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-02 08:20:38 +00:00
d8ec3ce370
[SoundCloud] Update client id
2021-10-01 02:51:38 +02:00
94efe86c71
Extract Video Short Description in YouTube.
...
In Trending, and Search results.
2021-09-22 16:26:16 +01:00
a9d214478d
Merge pull request #703 from FireMasterK/comment-replies
...
Add support for extracting comment replies continuation
2021-09-14 23:58:14 +02:00
6aabdc6d16
Fix for requested changes.
2021-09-12 01:15:19 +05:30
0aad09fa22
Update JavaDocs for comment replies continuation.
...
Co-authored-by: Stypox <stypox@pm.me>
2021-09-12 01:05:20 +05:30
feb09e9997
Extract Channel Avatar in related videos.
2021-09-04 01:09:40 +05:30
f0aa46b008
Merge pull request #724 from FireMasterK/streams-uploader-verified
...
Extract uploaderVerified to StreamInfo.
2021-09-03 18:18:04 +02:00
857b2f39e8
Extract uploaderVerified to StreamInfo.
2021-09-03 21:27:58 +05:30
1db463b55f
Add a default test for the Uploader's Avatar.
2021-09-03 21:05:36 +05:30
db6b3b2c29
Extract uploader's avatar in peertube.
...
Co-authored-by: Tobi <TobiGr@users.noreply.github.com>
2021-09-02 16:14:45 +05:30
a0c1dcc8d8
Remove throws parsing exception.
2021-09-02 16:14:45 +05:30
Stypox
litetex
XiangRongLin
TiA4f8R
FireMasterK
TobiGr
dependabot[bot]
Abdu Ameen
Mohammed Anas
bopol
opusforlife2