Common code in WEB client version HTML extraction has been deduplicated, usage of the Java 8 Stream API has been made and initial data fallback has been used as a last resort.
This means that the client version extraction from regexes will be used before this fallback, as it doesn't contain the full client version.
This can be used as a way to fingerprint the extractor, even if it seems to be not the case.
This will prevent any future extractor break due to decryption failure, like it was excepted to be the case before.
Some documentation about the throttling decryption has been also improved.
The ANDROID client was only fetched for video contents, where it can be useful on ended/post livestreams, if the n parameter of the WEB client cannot be decrypted, to avoid throttling issues (because the WEB client was only used before for ended/post livestreams).
It also provides an exclusive 48kbps M4A audio format in the adaptiveFormats array of the JSON player response, like other mobile clients (which can be also extracted from the response of the DASH manifest URL returned into the WEB client player's response, but the DASH manifest is not used by the extractor).
A note about non-fatality of fetching or parsing issues of the ANDROID and IOS clients has been added.
SoundCloud is currently removing this workaround completely, so there is no need to keep it, because it impacts the loading time (a HLS playlist was downloaded and parsed).
These values were only set before for video streams.
A fallback for the audio channels count has been added, in order to prevent exceptions when generating DASH manifests of audio streams: the fallback value is 2, because most audio streams on YouTube have 2 audio channels.
SoundCloud broke the workaround used to get a single file from HLS manifests for Opus manifests, but it still works for MP3 ones.
The code has been adapted to prevent an unneeded request (the one to the Opus HLS manifest) and the HLS delivery method is now used for SoundCloud MP3 and Opus streams, plus the progressive one (for tracks which have a progressive stream (MP3) and for the ones which doesn't have one, it is still used by trying to get a progressive stream, using the workaround).
Streams extraction has been also moved to Java 8 Stream's API and the relevant test has been also updated.
Move DASH manifests creation into a new subpackage of the YouTube package, dashmanifestcreators.
This subpackage contains:
- CreationException, exception extending Java's RuntimeException, thrown by manifest creators when something goes wrong;
- YoutubeDashManifestCreatorsUtils, class which contains all common methods and constants of all or a part of the manifest creators;
- a manifest creator has been added per delivery type of YouTube streams:
- YoutubeProgressiveDashManifestCreator, for progressive streams;
- YoutubeOtfDashManifestCreator, for OTF streams;
- YoutubePostLiveStreamDvrDashManifestCreator, for post-live DVR streams (which use the live delivery method).
Every DASH manifest creator has a getCache() static method, which returns the ManifestCreatorCache instance used to cache results.
DeliveryType has been also extracted from the YouTube DASH manifest creators part of the extractor and moved to the YouTube package.
YoutubeDashManifestCreatorTest has been updated and renamed to YoutubeDashManifestCreatorsTest, and YoutubeDashManifestCreator has been removed.
Finally, several documentation and exception messages fixes and improvements have been made.
This change allows to build DASH manifests using YoutubeDashManifestCreator with the real duration of streams and prevent potential cuts of the end of progressive streams, because the duration in YouTube's player response is in seconds and not milliseconds.
When moving opus-lo into a constant, opus-lo was renamed to opus_lo and was only used if no MP3 stream was available (which was not the case before the changes in BandcampRadioStreamExtractor related to the addition of the support of all delivery methods), so these changes removed the ability to get Opus streams of Bandcamp radios.
This commit reverts this unwanted change.
AudricV
ThetaDev
Stypox
Isira Seneviratne
dependabot[bot]
Mohammed Anas
litetex
Gábor Lipták
Kavin
TiA4f8R