Skip to content

Commit

Permalink
Merge branch 'dev' into Merge-dev
Browse files Browse the repository at this point in the history
# Conflicts:
#	app/build.gradle
#	app/src/main/java/org/schabi/newpipe/App.java
#	app/src/main/java/org/schabi/newpipe/about/AboutActivity.kt
#	app/src/main/java/org/schabi/newpipe/fragments/detail/VideoDetailFragment.java
#	app/src/main/java/org/schabi/newpipe/fragments/list/channel/ChannelFragment.java
#	app/src/main/java/org/schabi/newpipe/fragments/list/comments/CommentRepliesFragment.java
#	app/src/main/java/org/schabi/newpipe/fragments/list/videos/RelatedItemsFragment.java
#	app/src/main/java/org/schabi/newpipe/local/bookmark/BookmarkFragment.java
#	app/src/main/res/values-bg/strings.xml
#	app/src/main/res/values-da/strings.xml
#	app/src/main/res/values-is/strings.xml
#	app/src/main/res/values-lv/strings.xml
#	app/src/main/res/values-zh-rTW/strings.xml
#	build.gradle
  • Loading branch information
Isira-Seneviratne committed Dec 21, 2024
2 parents fbafdeb + 3fc0147 commit d4ed18b
Show file tree
Hide file tree
Showing 442 changed files with 5,756 additions and 1,313 deletions.
2 changes: 1 addition & 1 deletion .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ NewPipe contribution guidelines
## Crash reporting

Report crashes through the **automated crash report system** of NewPipe.
This way all the data needed for debugging is included in your bugreport for GitHub.
This way all the data needed for debugging is included in your bug report for GitHub.
You'll see *exactly* what is sent, be able to add **your comments**, and then send it.

## Issue reporting/feature requests
Expand Down
38 changes: 38 additions & 0 deletions .github/workflows/build-release-apk.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: "Build unsigned release APK on master"

on:
workflow_dispatch:

jobs:
release:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: 'master'

- uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: '21'
cache: 'gradle'

- name: "Build release APK"
run: ./gradlew assembleRelease --stacktrace

- name: "Rename APK"
run: |
VERSION_NAME="$(jq -r ".elements[0].versionName" "app/build/outputs/apk/release/output-metadata.json")"
echo "Version name: $VERSION_NAME" >> "$GITHUB_STEP_SUMMARY"
echo '```json' >> "$GITHUB_STEP_SUMMARY"
cat "app/build/outputs/apk/release/output-metadata.json" >> "$GITHUB_STEP_SUMMARY"
echo >> "$GITHUB_STEP_SUMMARY"
echo '```' >> "$GITHUB_STEP_SUMMARY"
# assume there is only one APK in that folder
mv app/build/outputs/apk/release/*.apk "app/build/outputs/apk/release/NewPipe_v$VERSION_NAME.apk"
- name: "Upload APK"
uses: actions/upload-artifact@v4
with:
name: app
path: app/build/outputs/apk/release/*.apk
14 changes: 11 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
<p align="center"><a href="https://newpipe.net">Website</a> &bull; <a href="https://newpipe.net/blog/">Blog</a> &bull; <a href="https://newpipe.net/FAQ/">FAQ</a> &bull; <a href="https://newpipe.net/press/">Press</a></p>
<hr>

*Read this document in other languages: [Deutsch](doc/README.de.md), [English](README.md), [Español](doc/README.es.md), [Français](doc/README.fr.md), [हिन्दी](doc/README.hi.md), [Italiano](doc/README.it.md), [한국어](doc/README.ko.md), [Português Brasil](doc/README.pt_BR.md), [Polski](doc/README.pl.md), [ਪੰਜਾਬੀ ](doc/README.pa.md), [日本語](doc/README.ja.md), [Română](doc/README.ro.md), [Soomaali](doc/README.so.md), [Türkçe](doc/README.tr.md), [正體中文](doc/README.zh_TW.md), [অসমীয়া](doc/README.asm.md), [Српски](doc/README.sr.md)*
*Read this document in other languages: [Deutsch](doc/README.de.md), [English](README.md), [Español](doc/README.es.md), [Français](doc/README.fr.md), [हिन्दी](doc/README.hi.md), [Italiano](doc/README.it.md), [한국어](doc/README.ko.md), [Português Brasil](doc/README.pt_BR.md), [Polski](doc/README.pl.md), [ਪੰਜਾਬੀ ](doc/README.pa.md), [日本語](doc/README.ja.md), [Română](doc/README.ro.md), [Soomaali](doc/README.so.md), [Türkçe](doc/README.tr.md), [正體中文](doc/README.zh_TW.md), [অসমীয়া](doc/README.asm.md), [Српски](doc/README.sr.md), [العربية](README.ar.md)*

> [!warning]
> <b>THIS APP IS IN BETA, SO YOU MAY ENCOUNTER BUGS. IF YOU DO, OPEN AN ISSUE IN OUR GITHUB REPOSITORY BY FILLING OUT THE ISSUE TEMPLATE.</b>
Expand Down Expand Up @@ -96,7 +96,7 @@ Also, since they are free and open source software, neither the app nor the Extr
## Installation and updates
You can install NewPipe using one of the following methods:
1. Add our custom repo to F-Droid and install it from there. The instructions are here: https://newpipe.net/FAQ/tutorials/install-add-fdroid-repo/
2. Download the APK from [GitHub Releases](https://github.com/TeamNewPipe/NewPipe/releases) and install it.
2. Download the APK from [GitHub Releases](https://github.com/TeamNewPipe/NewPipe/releases), [compare the signing key](#apk-info) and install it.
3. Update via F-Droid. This is the slowest method of getting updates, as F-Droid must recognize changes, build the APK itself, sign it, and then push the update to users.
4. Build a debug APK yourself. This is the fastest way to get new features on your device, but is much more complicated, so we recommend using one of the other methods.
5. If you're interested in a specific feature or bugfix provided in a Pull Request in this repo, you can also download its APK from within the PR. Read the PR description for instructions. The great thing about PR-specific APKs is that they're installed side-by-side the official app, so you don't have to worry about losing your data or messing anything up.
Expand All @@ -109,7 +109,15 @@ In the meanwhile, if you want to switch sources for some reason (e.g. NewPipe's
3. Download the APK from the new source and install it
4. Import the data from step 1 via Settings > Backup and Restore > Import Database

<b>Note: when you're importing a database into the official app, always make sure that it is the one you exported _from_ the official app. If you import a database exported from an APK other than the official app, it may break things. Such an action is unsupported, and you should only do so when you're absolutely certain you know what you're doing.</b>
> [!Note]
> When you're importing a database into the official app, always make sure that it is the one you exported _from_ the official app. If you import a database exported from an APK other than the official app, it may break things. Such an action is unsupported, and you should only do so when you're absolutely certain you know what you're doing.
### APK Info

This is the SHA fingerprint of NewPipe's signing key to verify downloaded APKs which are signed by us. The fingerprint is also available on [NewPipe's website](https://newpipe.net#download). This is relevant for method 2.
```
CB:84:06:9B:D6:81:16:BA:FA:E5:EE:4E:E5:B0:8A:56:7A:A6:D8:98:40:4E:7C:B1:2F:9E:75:6D:F5:CF:5C:AB
```

## Contribution
Whether you have ideas, translations, design changes, code cleaning, or even major code changes, help is always welcome. The app gets better and better with each contribution, no matter how big or small! If you'd like to get involved, check our [contribution notes](.github/CONTRIBUTING.md).
Expand Down
11 changes: 9 additions & 2 deletions app/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,15 @@ android {
resValue "string", "app_name", "NewPipe"
minSdk 21
targetSdk 33
versionCode 999
versionName "0.27.2"
if (System.properties.containsKey('versionCodeOverride')) {
versionCode System.getProperty('versionCodeOverride') as Integer
} else {
versionCode 1001
}
versionName "0.27.4"
if (System.properties.containsKey('versionNameSuffix')) {
versionNameSuffix System.getProperty('versionNameSuffix')
}

testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"

Expand Down
2 changes: 1 addition & 1 deletion app/src/main/java/org/schabi/newpipe/DownloaderImpl.java
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@

public final class DownloaderImpl extends Downloader {
public static final String USER_AGENT =
"Mozilla/5.0 (Windows NT 10.0; rv:91.0) Gecko/20100101 Firefox/91.0";
"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:128.0) Gecko/20100101 Firefox/128.0";
public static final String YOUTUBE_RESTRICTED_MODE_COOKIE_KEY =
"youtube_restricted_mode_key";
public static final String YOUTUBE_RESTRICTED_MODE_COOKIE = "PREF=f2=8000000";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,6 @@ default Flowable<List<PlaylistStreamEntity>> listByService(final int serviceId)
+ " AND :streamUrl = :streamUrl"

+ " GROUP BY " + JOIN_PLAYLIST_ID
+ " ORDER BY " + PLAYLIST_DISPLAY_INDEX)
+ " ORDER BY " + PLAYLIST_DISPLAY_INDEX + ", " + PLAYLIST_NAME)
Flowable<List<PlaylistDuplicatesEntry>> getPlaylistDuplicatesMetadata(String streamUrl);
}
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,10 @@ public enum StreamDialogDefaultEntry {
DOWNLOAD(R.string.download, (fragment, item) ->
fetchStreamInfoAndSaveToDatabase(fragment.requireContext(), item.getServiceId(),
item.getUrl(), info -> {
if (fragment.getContext() != null) {
// Ensure the fragment is attached and its state hasn't been saved to avoid
// showing dialog during lifecycle changes or when the activity is paused,
// e.g. by selecting the download option and opening a different fragment.
if (fragment.isAdded() && !fragment.isStateSaved()) {
final DownloadDialog downloadDialog =
new DownloadDialog(fragment.requireContext(), info);
downloadDialog.show(fragment.getChildFragmentManager(),
Expand Down
7 changes: 7 additions & 0 deletions app/src/main/java/org/schabi/newpipe/ktx/SharedPreferences.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
package org.schabi.newpipe.ktx

import android.content.SharedPreferences

fun SharedPreferences.getStringSafe(key: String, defValue: String): String {
return getString(key, null) ?: defValue
}
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,8 @@ private void onPlaylistSelected(@NonNull final LocalPlaylistManager manager,
.subscribe(ignored -> {
successToast.show();

if (playlist.thumbnailUrl.equals(PlaylistEntity.DEFAULT_THUMBNAIL)) {
if (playlist.thumbnailUrl != null
&& playlist.thumbnailUrl.equals(PlaylistEntity.DEFAULT_THUMBNAIL)) {
playlistDisposables.add(manager
.changePlaylistThumbnail(playlist.getUid(), streams.get(0).getUid(),
false)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,10 @@ import org.schabi.newpipe.database.subscription.NotificationMode
import org.schabi.newpipe.database.subscription.SubscriptionEntity
import org.schabi.newpipe.extractor.Info
import org.schabi.newpipe.extractor.NewPipe
import org.schabi.newpipe.extractor.ServiceList
import org.schabi.newpipe.extractor.feed.FeedInfo
import org.schabi.newpipe.extractor.stream.StreamInfoItem
import org.schabi.newpipe.ktx.getStringSafe
import org.schabi.newpipe.local.feed.FeedDatabaseManager
import org.schabi.newpipe.local.subscription.SubscriptionManager
import org.schabi.newpipe.util.ChannelTabHelper
Expand Down Expand Up @@ -69,12 +71,10 @@ class FeedLoadManager(private val context: Context) {
val outdatedThreshold = if (ignoreOutdatedThreshold) {
OffsetDateTime.now(ZoneOffset.UTC)
} else {
val thresholdOutdatedSeconds = (
defaultSharedPreferences.getString(
context.getString(R.string.feed_update_threshold_key),
context.getString(R.string.feed_update_threshold_default_value)
) ?: context.getString(R.string.feed_update_threshold_default_value)
).toInt()
val thresholdOutdatedSeconds = defaultSharedPreferences.getStringSafe(
context.getString(R.string.feed_update_threshold_key),
context.getString(R.string.feed_update_threshold_default_value)
).toInt()
OffsetDateTime.now(ZoneOffset.UTC).minusSeconds(thresholdOutdatedSeconds.toLong())
}

Expand All @@ -91,6 +91,10 @@ class FeedLoadManager(private val context: Context) {
else -> feedDatabaseManager.outdatedSubscriptionsForGroup(groupId, outdatedThreshold)
}

// like `currentProgress`, but counts the number of YouTube extractions that have begun, so
// they can be properly throttled every once in a while (see doOnNext below)
val youtubeExtractionCount = AtomicInteger()

return outdatedSubscriptions
.take(1)
.doOnNext {
Expand All @@ -106,6 +110,15 @@ class FeedLoadManager(private val context: Context) {
.observeOn(Schedulers.io())
.flatMap { Flowable.fromIterable(it) }
.takeWhile { !cancelSignal.get() }
.doOnNext { subscriptionEntity ->
// throttle YouTube extractions once every BATCH_SIZE to avoid being rate limited
if (subscriptionEntity.serviceId == ServiceList.YouTube.serviceId) {
val previousCount = youtubeExtractionCount.getAndIncrement()
if (previousCount != 0 && previousCount % BATCH_SIZE == 0) {
Thread.sleep(DELAY_BETWEEN_BATCHES_MILLIS.random())
}
}
}
.parallel(PARALLEL_EXTRACTIONS, PARALLEL_EXTRACTIONS * 2)
.runOn(Schedulers.io(), PARALLEL_EXTRACTIONS * 2)
.filter { !cancelSignal.get() }
Expand Down Expand Up @@ -329,7 +342,19 @@ class FeedLoadManager(private val context: Context) {
/**
* How many extractions will be running in parallel.
*/
private const val PARALLEL_EXTRACTIONS = 6
private const val PARALLEL_EXTRACTIONS = 3

/**
* How many YouTube extractions to perform before waiting [DELAY_BETWEEN_BATCHES_MILLIS]
* to avoid being rate limited
*/
private const val BATCH_SIZE = 50

/**
* Wait a random delay in this range once every [BATCH_SIZE] YouTube extractions to avoid
* being rate limited
*/
private val DELAY_BETWEEN_BATCHES_MILLIS = (6000L..12000L)

/**
* Number of items to buffer to mass-insert in the database.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -332,10 +332,6 @@ private void showInfoItemDialog(final StreamStatisticsEntry item) {
StreamDialogDefaultEntry.DELETE,
(f, i) -> deleteEntry(
Math.max(itemListAdapter.getItemsList().indexOf(item), 0)))
.setAction(
StreamDialogDefaultEntry.START_HERE_ON_BACKGROUND,
(f, i) -> NavigationHelper.playOnBackgroundPlayer(
context, getPlayQueueStartingAt(item), true))
.create()
.show();
} catch (final IllegalArgumentException e) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,10 @@ public int onStartCommand(final Intent intent, final int flags, final int startI

try {
outFile = new StoredFileHelper(this, path, "application/json");
outputStream = new SharpOutputStream(outFile.getStream());
// truncate the file before writing to it, otherwise if the new content is smaller than
// the previous file size, the file will retain part of the previous content and be
// corrupted
outputStream = new SharpOutputStream(outFile.openAndTruncateStream());
} catch (final IOException e) {
handleError(e);
return START_NOT_STICKY;
Expand Down
1 change: 0 additions & 1 deletion app/src/main/java/org/schabi/newpipe/player/Player.java
Original file line number Diff line number Diff line change
Expand Up @@ -454,7 +454,6 @@ public void handleIntent(@NonNull final Intent intent) {
if (oldPlayerType != playerType && playQueue != null) {
// If playerType changes from one to another we should reload the player
// (to disable/enable video stream or to set quality)
setRecovery();
reloadPlayQueueManager();
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -133,17 +133,8 @@ private void generateDataFrom(final Frameset frameset, final UUID updateRequestI

// Get the bounds where the frame is found
final int[] bounds = frameset.getFrameBoundsAt(currentPosMs);
generatedDataForUrl.put(currentPosMs, () -> {
// It can happen, that the original bitmap could not be downloaded
// In such a case - we don't want a NullPointer - simply return null
if (srcBitMap == null) {
return null;
}

// Cut out the corresponding bitmap form the "srcBitMap"
return Bitmap.createBitmap(srcBitMap, bounds[1], bounds[2],
frameset.getFrameWidth(), frameset.getFrameHeight());
});
generatedDataForUrl.put(currentPosMs,
createBitmapSupplier(srcBitMap, bounds, frameset));

currentPosMs += frameset.getDurationPerFrame();
pos++;
Expand All @@ -166,6 +157,45 @@ private void generateDataFrom(final Frameset frameset, final UUID updateRequestI
}
}

private Supplier<Bitmap> createBitmapSupplier(final Bitmap srcBitMap,
final int[] bounds,
final Frameset frameset) {
return () -> {
// It can happen, that the original bitmap could not be downloaded
// (or it was recycled though that should not happen)
// In such a case - we don't want a NullPointer/
// "cannot use a recycled source in createBitmap" Exception -> simply return null
if (srcBitMap == null || srcBitMap.isRecycled()) {
return null;
}

// Under some rare circumstances the YouTube API returns slightly too small storyboards,
// (or not the matching frame width/height)
// This would lead to createBitmap cutting out a bitmap that is out of bounds,
// so we need to adjust the bounds accordingly
if (srcBitMap.getWidth() < bounds[1] + frameset.getFrameWidth()) {
bounds[1] = srcBitMap.getWidth() - frameset.getFrameWidth();
}

if (srcBitMap.getHeight() < bounds[2] + frameset.getFrameHeight()) {
bounds[2] = srcBitMap.getHeight() - frameset.getFrameHeight();
}

// Cut out the corresponding bitmap form the "srcBitMap"
final Bitmap cutOutBitmap = Bitmap.createBitmap(srcBitMap, bounds[1], bounds[2],
frameset.getFrameWidth(), frameset.getFrameHeight());

// If the cut out bitmap is identical to its source,
// we need to copy the bitmap to create a new instance.
// createBitmap allows itself to return the original object that is was created with
// this leads to recycled bitmaps being returned (if they are identical)
// Reference: https://stackoverflow.com/a/23683075 + first comment
// Fixes: https://github.com/TeamNewPipe/NewPipe/issues/11461
return cutOutBitmap == srcBitMap
? cutOutBitmap.copy(cutOutBitmap.getConfig(), true) : cutOutBitmap;
};
}

@Nullable
private Bitmap getBitMapFrom(final String url) {
if (url == null) {
Expand Down
Loading

0 comments on commit d4ed18b

Please sign in to comment.