Commit Graph

2686 Commits

Author SHA1 Message Date
AudricV a13510b962
[YouTube] Update clients info 2024-07-23 20:43:53 +02:00
AudricV f4931d8bbd
[YouTube] Workaround 403s on streaming URLs of WEB client after some time
These changes work around an anti-bot token, for which its requirement is A/B
tested on the WEB client. In this test, streaming URLs of this client return
HTTP errors 403 if the token is not provided after some time.

It also allows to not fetch the JavaScript player for non-age restricted
videos, reducing data usage.

The TVHTML5 embed client is now only fetched in the case of age-restricted
videos.

The methods forceFetchAndroidClient and forceFetchIosClient of
YoutubeStreamExtractor have been removed, as they are now not needed anymore.

These changes also break the extraction of appropriate error states for private
and deleted videos and invalid video IDs.
2024-07-23 20:43:48 +02:00
Tobi 312e91048c
Merge pull request #1177 from TeamNewPipe/bandcamp-upgrade-to-https
[Bandcamp] Upgrade incoming links to HTTPS
2024-07-22 11:52:52 +02:00
Fynn Godau f441036ed2 [Bandcamp] Upgrade incoming links to HTTPS 2024-07-22 11:48:02 +02:00
Marco Sirabella 02e14b8931
Add support for on.soundcloud.com urls (#1179)
Add support for on.soundcloud.com urls

Fixes #1178

Co-authored-by: TobiGr <tobigr@users.noreply.github.com>
2024-07-22 11:22:47 +02:00
Tobi 0e15f9ac1b
Merge pull request #1201 from TeamNewPipe/bandcamp-radio-v3
[Bandcamp] Show additional info in radio kiosk
2024-07-22 08:27:27 +02:00
Tobi 87af6bb223
Merge pull request #1200 from TeamNewPipe/bandcamp-fix-null-url-catenation
[Bandcamp] Null-safe url catenation in track playlist
2024-07-22 08:26:16 +02:00
Fynn Godau 227c6894a7 [Bandcamp] Show additional info in radio kiosk
A newer version of the radio list API delivers additional fields. Thanks
@crisp5.
2024-07-21 23:59:10 +02:00
Fynn Godau 97955e5e41 [Bandcamp] Null-safe url catenation in track playlist
Previously, if no URL was provided due to the track being a preorder or
unpaid track that doesn't have its own public page, we would catenate
the artist URL and null to a nonsensical string.

This invalid URL would also be used to try fetching a thumbnail URL.

The downstream behavior of the NewPipe client is only marginally
improved, as it now no longer throws visible exceptions due to the
missing thumbnails, but still gives an unfriendly error upon clicking
the item that has a `null` URL.
2024-07-21 23:35:57 +02:00
Tobi 4aaab63c12
Merge pull request #1199 from TeamNewPipe/bandcamp-update-artist-detection
[Bandcamp] Update artist page detection
2024-07-21 22:04:53 +02:00
Isira Seneviratne 9a29f9ee2d
Use the new URL encode/decode methods introduced in Java 10 (#1196)
* Use Java 10 URLDecoder/URLEncoder methods

* Simplify compatParseMap
2024-07-21 19:45:28 +05:30
Fynn Godau de9fb7cb28 [Bandcamp] Update artist page detection
Bandcamp changed the way the footer is rendered. Therefore, we check for
a link to the cart page instead.
2024-07-21 15:37:21 +02:00
XiangRongLin d39fc43282
[Youtube] Adjust throttling function extraction to changes (#1191)
* [Youtube] Adjust throttling function extraction to changes

---------

Co-authored-by: Stypox <stypox@pm.me>
2024-07-11 11:23:53 +02:00
XiangRongLin 592f1596e6
[Youtube] Adjust throttling function extraction to changes (#1191)
* [Youtube] Adjust throttling function extraction to changes

---------

Co-authored-by: Stypox <stypox@pm.me>
2024-07-11 11:20:33 +02:00
Tobi c3c6de85bc
Merge pull request #1180 from TeamNewPipe/dependabot/gradle/com.google.code.gson-gson-2.11.0
Bump com.google.code.gson:gson from 2.10.1 to 2.11.0
2024-06-29 11:27:13 +02:00
dependabot[bot] 383000f10d
Bump com.google.code.gson:gson from 2.10.1 to 2.11.0
Bumps [com.google.code.gson:gson](https://github.com/google/gson) from 2.10.1 to 2.11.0.
- [Release notes](https://github.com/google/gson/releases)
- [Changelog](https://github.com/google/gson/blob/main/CHANGELOG.md)
- [Commits](https://github.com/google/gson/compare/gson-parent-2.10.1...gson-parent-2.11.0)

---
updated-dependencies:
- dependency-name: com.google.code.gson:gson
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-29 09:22:41 +00:00
Tobi 2c7076930c
Merge pull request #1186 from TeamNewPipe/dependabot/gradle/org.junit-junit-bom-5.10.3
Bump org.junit:junit-bom from 5.10.2 to 5.10.3
2024-06-29 11:20:50 +02:00
dependabot[bot] 11a31721c5
Bump org.junit:junit-bom from 5.10.2 to 5.10.3
Bumps [org.junit:junit-bom](https://github.com/junit-team/junit5) from 5.10.2 to 5.10.3.
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.10.2...r5.10.3)

---
updated-dependencies:
- dependency-name: org.junit:junit-bom
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-28 09:50:44 +00:00
Tobi 90183056b5
Merge pull request #1183 from TeamNewPipe/dependabot/gradle/com.github.spotbugs-spotbugs-annotations-4.8.6
Bump com.github.spotbugs:spotbugs-annotations from 4.8.5 to 4.8.6
2024-06-22 23:35:46 +02:00
dependabot[bot] 2efea787d2
Bump com.github.spotbugs:spotbugs-annotations from 4.8.5 to 4.8.6
Bumps [com.github.spotbugs:spotbugs-annotations](https://github.com/spotbugs/spotbugs) from 4.8.5 to 4.8.6.
- [Release notes](https://github.com/spotbugs/spotbugs/releases)
- [Changelog](https://github.com/spotbugs/spotbugs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/spotbugs/spotbugs/compare/4.8.5...4.8.6)

---
updated-dependencies:
- dependency-name: com.github.spotbugs:spotbugs-annotations
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-18 09:35:16 +00:00
TobiGr fafd471606 [PeerTube] Fix test for like count
Number changed
2024-05-08 19:25:12 +02:00
TobiGr 4f477ad72b [PeerTube] Fix testing comment content
The comment is not available anymore.
2024-05-08 19:25:12 +02:00
Tobi 964e429978
Merge pull request #1166 from TeamNewPipe/dependabot/github_actions/peaceiris/actions-gh-pages-4
Bump peaceiris/actions-gh-pages from 3 to 4
2024-05-08 19:24:40 +02:00
dependabot[bot] 10c6965a28
Bump peaceiris/actions-gh-pages from 3 to 4
Bumps [peaceiris/actions-gh-pages](https://github.com/peaceiris/actions-gh-pages) from 3 to 4.
- [Release notes](https://github.com/peaceiris/actions-gh-pages/releases)
- [Changelog](https://github.com/peaceiris/actions-gh-pages/blob/main/CHANGELOG.md)
- [Commits](https://github.com/peaceiris/actions-gh-pages/compare/v3...v4)

---
updated-dependencies:
- dependency-name: peaceiris/actions-gh-pages
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-08 17:16:08 +00:00
TobiGr e54f38f5e7 [PeerTube] Fix test
UploaderName was changed by user
2024-05-08 19:13:49 +02:00
Tobi 5dd5c7a65b
Merge pull request #1173 from TeamNewPipe/dependabot/gradle/com.github.spotbugs-spotbugs-annotations-4.8.5
Bump com.github.spotbugs:spotbugs-annotations from 4.8.3 to 4.8.5
2024-05-08 19:12:40 +02:00
Tobi 37438ff82f
Merge pull request #1174 from TeamNewPipe/dependabot/gradle/org.mozilla-rhino-1.7.15
Bump org.mozilla:rhino from 1.7.13 to 1.7.15
2024-05-08 19:02:57 +02:00
TobiGr bba3b6c69b version 0.24.0 2024-05-08 13:01:06 +02:00
TobiGr b40a5784ed Fix JDoc
YoutubePostLiveStreamDvrDashManifestCreator.java:90: error: unexpected end tag: </p>
2024-05-08 12:41:36 +02:00
dependabot[bot] c6da4004e2
Bump org.mozilla:rhino from 1.7.13 to 1.7.15
Bumps [org.mozilla:rhino](https://github.com/mozilla/rhino) from 1.7.13 to 1.7.15.
- [Release notes](https://github.com/mozilla/rhino/releases)
- [Changelog](https://github.com/mozilla/rhino/blob/master/RELEASE-NOTES.md)
- [Commits](https://github.com/mozilla/rhino/commits)

---
updated-dependencies:
- dependency-name: org.mozilla:rhino
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-06 09:43:05 +00:00
dependabot[bot] f26e84d39f
Bump com.github.spotbugs:spotbugs-annotations from 4.8.3 to 4.8.5
Bumps [com.github.spotbugs:spotbugs-annotations](https://github.com/spotbugs/spotbugs) from 4.8.3 to 4.8.5.
- [Release notes](https://github.com/spotbugs/spotbugs/releases)
- [Changelog](https://github.com/spotbugs/spotbugs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/spotbugs/spotbugs/compare/4.8.3...4.8.5)

---
updated-dependencies:
- dependency-name: com.github.spotbugs:spotbugs-annotations
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-06 09:43:03 +00:00
Tobi ec3e8378c6
Merge pull request #1171 from TeamNewPipe/fix/jdoc
Fix JavaDocs and add Checkstyle exception for line length in JavaDocs' links
2024-04-23 21:22:12 +02:00
TobiGr 8d2a7a5281 [PeerTube] Fix test 2024-04-23 20:02:29 +02:00
TobiGr 7c29dbc965 Fix JDoc
Add Checkstyle exception for LineLength in JDoc links.

See https://github.com/TeamNewPipe/NewPipeExtractor/actions/runs/8804403691/job/24164703883
2024-04-23 19:55:51 +02:00
Stypox fbe9e6223a
Merge pull request #1168 from AudricV/yt_upd-cver-rm-keys-and-do-fixes
[YouTube] Update clients versions, restore access to some streams and more
2024-04-20 11:54:24 +02:00
Stypox 4e9e7cb29c
Improve assertTabsContain() to also check size 2024-04-20 11:48:36 +02:00
Stypox 9d0dd36034
[YouTube] Create constants for client names/versions 2024-04-20 11:43:54 +02:00
Stypox d4e6d22e64
[YouTube] Improve meta info code for review 2024-04-20 11:43:08 +02:00
AudricV 74bf000473
[YouTube] Update mocks 2024-04-11 18:59:11 +02:00
AudricV f9792cf3a9
[YouTube] Fix InteractiveTabbedHeader.testTabs test
YouTube now returns a Shorts tab for InteractiveTabbedHeader gaming channels,
which contains Shorts about the game of the topic channel but are not uploaded
on the game topic channel.

As this tab is already supported by the extractor, fetching a gaming topic
channel now returns a tab instead of none.

Channel name and channel URL of these Shorts needs to be set to null in a
separate commit, as Shorts on this tab do not have the topic channel as their
uploader.
2024-04-11 18:59:10 +02:00
AudricV f40fc0aa4f
[YouTube] Add support for new crisis meta info action data
The new action data can return multiple contact actions instead of only one,
which will be concatenated by a new line return.

This commit fixes tests of the CrisisResources test class of
YoutubeSearchExtractorTest.
2024-04-11 18:59:09 +02:00
AudricV 2a3c6f80d2
[YouTube] Fix YoutubeStreamExtractorRelatedMixTest 2024-04-10 21:19:03 +02:00
AudricV 657b4377aa
[YouTube] Fix YoutubeStreamExtractorDefaultTest tests 2024-04-10 21:19:03 +02:00
AudricV 7bf50bf1cb
[YouTube] Update Android client player parameters
YouTube disabled the effectiveness of the parameters which were used (the
player response we get redirects to another video), but new parameters which
work around Android's client integrity checks have been found.
2024-04-10 21:19:03 +02:00
AudricV 27dc1b1f50
[YouTube] Remove usage of API keys for InnerTube requests, bump versions
The API keys are not used anymore by official clients in almost all cases
(still used by the Android app until it gets a configuration) for all requests
we made.

Clients and device OS versions have been bumped to their latest stable version
known.

Methods and fields related to API keys have been renamed or deleted if they're
no longer relevant.
2024-04-10 21:19:02 +02:00
AudricV e380bb4bc3
[YouTube] Add missing prettyPrint query parameter to mixes continuations 2024-04-10 19:06:36 +02:00
Audric V 6c3c2e25d7
Merge pull request #1163 from AudricV/yt-fix_comments_extraction
[YouTube] Support new comments data
2024-04-10 18:19:59 +02:00
Stypox 02274d5395
[YouTube] Avoid XSS attacks in description or comments 2024-04-08 11:21:31 +02:00
Stypox 3f7b2653e3
[YouTube] Add YoutubeDescriptionHelperTest 2024-04-08 11:21:31 +02:00
Stypox a90237816a
[YouTube] Cleanup description helper
Remove unneeded isClose field, and make constants private
2024-04-08 11:21:31 +02:00