Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delivery methods without design flaws #862

Draft
wants to merge 43 commits into
base: dev
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
c2b9537
Created new basic stream architecture
litetex May 28, 2022
c129d7b
Removed old streaming API
litetex Jun 4, 2022
8b94206
Added Utility
litetex Jun 4, 2022
73bd33d
Rewrote extractors part 1
litetex Jun 4, 2022
d0bae9e
Added VideoQualityData
litetex Jun 5, 2022
9055f55
Reordered code inside StreamInfo + Added dashMpdUrl and hlsMasterPlay…
litetex Jun 5, 2022
489b0cd
Added more TODOs
litetex Jun 5, 2022
ca5a141
Reworked VideoQualityData
litetex Jun 5, 2022
f632fc9
Re-Implemented Itags
litetex Jun 5, 2022
a77865b
More refactoring + started implementing ItagInfo
litetex Jun 6, 2022
cd42e19
More reworks
litetex Jun 9, 2022
32e85d3
More refactoring...
litetex Jun 10, 2022
a2df180
Fix StreamInfoItem
litetex Jun 10, 2022
474c3cd
Fix more compile problems
litetex Jun 10, 2022
cf77bf5
Use better name for meethod
litetex Jun 12, 2022
862ed87
Added validation for ``MediaFormat``s
litetex Jun 13, 2022
1d0a27c
Abstracted YT Dash manifest creators and made them instantiable
litetex Jun 13, 2022
411b6c1
YT: Use new DASHManifestCreators
litetex Jun 14, 2022
2ef215a
Fixed checkstyle problems
litetex Jun 14, 2022
7363669
Fixed API
litetex Jun 14, 2022
9da8dd8
Started fixing tests...
litetex Jun 14, 2022
943b46c
Fixed most tests
litetex Jun 16, 2022
95581b2
Started fixing YoutubeDashManifestCreatorsTest
litetex Jun 16, 2022
6f45e5c
Fixed YoutubeDashManifestCreatorsTest
litetex Jun 18, 2022
7f9e895
Fixed more tests
litetex Jun 18, 2022
34d11af
Made code non-nullable (MediaFormatRegistry)
litetex Jun 18, 2022
9c715d7
Fixed Soundcloud test: Return EVERYTHING that is found
litetex Jun 18, 2022
579de7b
Fix merge conflicts
litetex Jun 18, 2022
8c71edf
Fixed merge conflicts from https://github.com/TeamNewPipe/NewPipeExtr…
litetex Jun 18, 2022
eb74ae3
Fixed ``YoutubeDashManifestCreatorsTest``
litetex Jun 19, 2022
d2e82bd
Removed unused ``equalsStream``
litetex Jun 20, 2022
3f1f2b2
Naming changes
litetex Jun 21, 2022
5390982
Cleanup ``MediaCCCLiveStreamExtractor``
litetex Jun 24, 2022
0732bef
[Peertube] No special handling of live video streams, use ``getHlsMas…
litetex Jun 24, 2022
3ce7f2e
Code cleanup
litetex Jun 24, 2022
e3c989e
Backported fixes from #864
litetex Jun 24, 2022
889b5f2
Improvements for NewPipe
litetex Jul 6, 2022
ff8eb68
Modified type hierarchy for downloads
litetex Aug 26, 2022
fa57ec0
Code improvements
litetex Aug 26, 2022
c9e911e
Implemented ``StreamResolvingStrategy``
litetex Aug 26, 2022
95a8b8b
Fixed compile problems after rebase
litetex Aug 26, 2022
8b7cb62
Remove useless checkstyle off
litetex Aug 26, 2022
c2f3c1a
Added changes from (missing) rebase commit 301a795ed388b84cdad030e18a…
litetex Aug 26, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions extractor/src/main/java/org/schabi/newpipe/extractor/Info.java
Original file line number Diff line number Diff line change
Expand Up @@ -42,19 +42,19 @@ public void addAllErrors(final Collection<Throwable> throwables) {
this.errors.addAll(throwables);
}

public Info(final int serviceId,
final String id,
final String url,
final String originalUrl,
final String name) {
protected Info(final int serviceId,
final String id,
final String url,
final String originalUrl,
final String name) {
this.serviceId = serviceId;
this.id = id;
this.url = url;
this.originalUrl = originalUrl;
this.name = name;
}

public Info(final int serviceId, final LinkHandler linkHandler, final String name) {
protected Info(final int serviceId, final LinkHandler linkHandler, final String name) {
this(serviceId,
linkHandler.getId(),
linkHandler.getUrl(),
Expand Down
168 changes: 0 additions & 168 deletions extractor/src/main/java/org/schabi/newpipe/extractor/MediaFormat.java

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,13 @@
import org.schabi.newpipe.extractor.exceptions.ReCaptchaException;
import org.schabi.newpipe.extractor.localization.Localization;

import javax.annotation.Nonnull;
import javax.annotation.Nullable;
import java.io.IOException;
import java.util.List;
import java.util.Map;

import javax.annotation.Nonnull;
import javax.annotation.Nullable;

/**
* A base for downloader implementations that NewPipe will use
* to download needed resources during extraction.
Expand Down Expand Up @@ -148,8 +149,27 @@ public Response post(final String url,
/**
* Do a request using the specified {@link Request} object.
*
* @param request The request to process
* @return the result of the request
*/
public abstract Response execute(@Nonnull Request request)
throws IOException, ReCaptchaException;

/**
* Get the size of the content that the url is pointing by firing a HEAD request.
*
* @param url an url pointing to the content
* @return the size of the content, in bytes or -1 if unknown
*/
public long getContentLength(final String url) {
try {
final String contentLengthHeader = head(url).getHeader("Content-Length");
if (contentLengthHeader == null) {
return -1;
}
return Long.parseLong(contentLengthHeader);
} catch (final Exception e) {
return -1;
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,17 @@

package org.schabi.newpipe.extractor.services.bandcamp.extractors;

import static org.schabi.newpipe.extractor.services.bandcamp.extractors.BandcampExtractorHelper.BASE_URL;
import static org.schabi.newpipe.extractor.services.bandcamp.extractors.BandcampExtractorHelper.getImageUrl;

import com.grack.nanojson.JsonObject;

import org.schabi.newpipe.extractor.exceptions.ParsingException;
import org.schabi.newpipe.extractor.localization.DateWrapper;
import org.schabi.newpipe.extractor.stream.StreamInfoItemExtractor;
import org.schabi.newpipe.extractor.stream.StreamType;

import javax.annotation.Nullable;

import static org.schabi.newpipe.extractor.services.bandcamp.extractors.BandcampExtractorHelper.BASE_URL;
import static org.schabi.newpipe.extractor.services.bandcamp.extractors.BandcampExtractorHelper.getImageUrl;

public class BandcampRadioInfoItemExtractor implements StreamInfoItemExtractor {

private final JsonObject show;
Expand Down Expand Up @@ -58,8 +58,8 @@ public String getThumbnailUrl() {
}

@Override
public StreamType getStreamType() {
return StreamType.AUDIO_STREAM;
public boolean isAudioOnly() {
return true;
}

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@

import org.jsoup.Jsoup;
import org.jsoup.nodes.Element;
import org.schabi.newpipe.extractor.MediaFormat;
import org.schabi.newpipe.extractor.NewPipe;
import org.schabi.newpipe.extractor.StreamingService;
import org.schabi.newpipe.extractor.downloader.Downloader;
Expand All @@ -21,9 +20,12 @@
import org.schabi.newpipe.extractor.exceptions.ReCaptchaException;
import org.schabi.newpipe.extractor.linkhandler.LinkHandler;
import org.schabi.newpipe.extractor.playlist.PlaylistInfoItemsCollector;
import org.schabi.newpipe.extractor.stream.AudioStream;
import org.schabi.newpipe.extractor.stream.Description;
import org.schabi.newpipe.extractor.stream.StreamSegment;
import org.schabi.newpipe.extractor.streamdata.delivery.simpleimpl.SimpleProgressiveHTTPDeliveryDataImpl;
import org.schabi.newpipe.extractor.streamdata.format.registry.AudioFormatRegistry;
import org.schabi.newpipe.extractor.streamdata.stream.AudioStream;
import org.schabi.newpipe.extractor.streamdata.stream.simpleimpl.SimpleAudioStreamImpl;

import java.io.IOException;
import java.util.ArrayList;
Expand Down Expand Up @@ -120,24 +122,23 @@ public long getLength() {

@Override
public List<AudioStream> getAudioStreams() {
final List<AudioStream> audioStreams = new ArrayList<>();
final JsonObject streams = showInfo.getObject("audio_stream");

final List<AudioStream> audioStreams = new ArrayList<>();
if (streams.has(MP3_128)) {
audioStreams.add(new AudioStream.Builder()
.setId(MP3_128)
.setContent(streams.getString(MP3_128), true)
.setMediaFormat(MediaFormat.MP3)
.setAverageBitrate(128)
.build());
audioStreams.add(new SimpleAudioStreamImpl(
AudioFormatRegistry.MP3,
new SimpleProgressiveHTTPDeliveryDataImpl(streams.getString(MP3_128)),
128
));
}

if (streams.has(OPUS_LO)) {
audioStreams.add(new AudioStream.Builder()
.setId(OPUS_LO)
.setContent(streams.getString(OPUS_LO), true)
.setMediaFormat(MediaFormat.OPUS)
.setAverageBitrate(100).build());
audioStreams.add(new SimpleAudioStreamImpl(
AudioFormatRegistry.OPUS,
new SimpleProgressiveHTTPDeliveryDataImpl(streams.getString(OPUS_LO)),
100
));
}

return audioStreams;
Expand Down
Loading