Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

F-droid availability? #7

Closed
6 tasks done
Mr-Bajs opened this issue Jan 27, 2024 · 44 comments
Closed
6 tasks done

F-droid availability? #7

Mr-Bajs opened this issue Jan 27, 2024 · 44 comments
Labels
👍 planned Officially planned for a future release

Comments

@Mr-Bajs
Copy link
Contributor

Mr-Bajs commented Jan 27, 2024

Checklist

  • I made sure that there are no existing issues - open or closed - which I could contribute my information to.
  • I have read the FAQ and my problem isn't listed.
  • I'm aware that this is a request for NewPipe itself and that requests for adding a new service need to be made at NewPipeExtractor.
  • I have taken the time to fill in all the required details. I understand that the feature request will be dismissed otherwise.
  • This issue contains only one feature request.
  • I have read and understood the contribution guidelines.

Feature description

Great to se the new app. I hope that the new app will be on f-droid. Maybe it's even in the pipeline due to the delay in f-droid, but it would be nice to know if it's planned to be there.

Why do you want this feature?

.

Additional information

No response

@polymorphicshade
Copy link
Owner

I have no idea how to do this (someone else did this with my old fork).

I'll do this whenever I find the time, unless someone else wants to do this... 😅

@IzzySoft
Copy link

@polymorphicshade inclusion was just requested with my repo. I'm fine with adding it, but have a few questions a.o. from my scanner reports, so let me copy those over here for you:

Offending libs:
---------------
* ACRA (/org/acra): Tracking

1 offenders.

Permissions:
------------
* android.permission.INTERNET
* android.permission.WAKE_LOCK
* android.permission.ACCESS_NETWORK_STATE
* android.permission.WRITE_EXTERNAL_STORAGE
* android.permission.SYSTEM_ALERT_WINDOW
* android.permission.FOREGROUND_SERVICE
* android.permission.POST_NOTIFICATIONS
* android.permission.RECEIVE_BOOT_COMPLETED
* org.polymorphicshade.tubular.DYNAMIC_RECEIVER_NOT_EXPORTED_PERMISSION
* android.permission.READ_EXTERNAL_STORAGE*

SigningBlock blobs:
-------------------
0x504b4453 (DEPENDENCY_INFO_BLOCK; GOOGLE)

Top-down:

  • is ACRA opt-in, opt-out or mandatory? If it's used the same way as with its predecessor (Newpipe x SponsorBlock), it's only used via email the users have to send themselves, and thus opt-in, but that needs to be confirmed.
  • what does android.permission.SYSTEM_ALERT_WINDOW apply to (educated guess: floating player?)
  • android.permission.READ_EXTERNAL_STORAGE is granted implicitly (hence the asterisk) due to WRITE_EXTERNAL_STORAGE. So if the two are indeed needed, it would be good to know what for – and maybe to have READ_EXTERNAL_STORAGE be requested explicitly.

SigningBlock: I guess Android Studio (or IntelliJ IDEA) is used for signing? In that case, the following should be integrated with build.gradle?

android {
    dependenciesInfo {
        // Disables dependency metadata when building APKs.
        includeInApk = false
        // Disables dependency metadata when building Android App Bundles.
        includeInBundle = false
    }
}

For some background: that BLOB is supposed to be just a binary representation of your app's dependency tree. But as it's encrypted with a public key belonging to Google, only Google can read it – and nobody else can even verify what it contains.

The entire Fastlane tree is untouched from NewPipe. Could it please be updated to match the app? Especially some screenshots would be good to have, and the proper shortdesc/fulldesc.

Thanks for checking (and implementing what's needed 😉)

@polymorphicshade
Copy link
Owner

@IzzySoft

  • ARCA is only used for sending crash reports
  • the SYSTEM_ALERT_WINDOW permission is only used for the app to draw over other apps
  • code related to the WRITE_EXTERNAL_STORAGE permission hasn't changed from vanilla NewPipe, so I'm opting not to change anything related to that
  • I'll work on the the other changes soon 👍

@IzzySoft
Copy link

ARCA is only used for sending crash reports

The one asking the user to confirm? Fine, then I can remove the Tracking flag (done, effective with the next sync).

the SYSTEM_ALERT_WINDOW permission is only used for the app to draw over other apps

Yeah, that's what the permission SYSTEM_ALERT_WINDOW is described as 😉 I mean, how does the app make use of that? A floating video player?

code related to the WRITE_EXTERNAL_STORAGE permission hasn't changed from vanilla NewPipe, so I'm opting not to change anything related to that

Vanilla NewPipe is not in my repo, so I cannot tell. But with the latest updates to my APK checks those permissions turn up "red" as being sensitive, unless an explanation is added what for they are necessary. I have no issues with READ_EXTERNAL_STORAGE being granted implicitly as long as I know what it's needed for 😉

I'll work on the the other changes soon

Thanks a lot!

@ColorfulRhino
Copy link

Yeah, that's what the permission SYSTEM_ALERT_WINDOW is described as 😉 I mean, how does the app make use of that? A floating video player?

Yes, Newpipe has a floating mini-player.

Btw, I'm sure you @IzzySoft are aware that @polymorphicshade did have another Newpipe fork based on the previous Newpipe version in your repo (org.polymorphicshade.newpipe). Since this one is basically an updated version of this fork and the other one is out-of-development (https://github.com/polymorphicshade/NewPipe), consider removing the old fork from the repo :)

@IzzySoft
Copy link

@ColorfulShire thanks! And yes, I'm aware of that – and yes again, it's already "marked for removal" here. I'll just need to prepare the transition (was busy with other things), like adding a comment to the description of the "old one" that it's continued as "new one", to give those currently using that a chance to find there way "over".

@polymorphicshade
Copy link
Owner

polymorphicshade commented Jan 31, 2024

@IzzySoft
Oh, sorry... yes the SYSTEM_ALERT_WINDOW permission is used for the floating popup player.

The WRITE_EXTERNAL_STORAGE permission is used for downloading videos.
The app explicitly asks for permission, see here

@IzzySoft
Copy link

Thanks, added them all to the allow-list here then:

image

As for the signing BLOB, please see above. Easy to get rid of that – which would also remove it from the screenshot. Err, from where the screenshot was taken from of course…

@kaoneko
Copy link

kaoneko commented Feb 2, 2024

like adding a comment to the description of the "old one" that it's continued as "new one", to give those currently using that a chance to find there way "over".

@IzzySoft seems the actual app name went MIA, it currently reads:

Note: the app's repository has been archived 2023-12-29, so there won't be updates anymore. The app is however continued as

Major cliffhanger! 😄

Also, might I suggest putting the note above the regular description instead of below? Most F-Droid clients don't show the full description of an app until you tap a Read more button, which someone who is already using the app and just trying to find out if there are updates isn't likely to do.

@IzzySoft
Copy link

IzzySoft commented Feb 2, 2024

@kaoneko thanks for the pointers! Fixed both, should go live within the next half hour.

@IzzySoft
Copy link

Hmpf, one more thing:

image

What is this? Someone complained to me:

Showed an alert and tap opened the .apk download in browser. No explanation or nothing.

That's violating the inclusion criteria concerning self-updaters. Those must be strictly opt-in, with all details explained – as such side-loaded updates bypass the scans performed by the repo (F-Droid's and mine). Can you please remove that?

@IzzySoft
Copy link

IzzySoft commented Mar 6, 2024

So no word, @polymorphicshade? You're still there?

@polymorphicshade
Copy link
Owner

So no word, @polymorphicshade? You're still there?

OH! Sorry... I acknowledged your comment with a thumbs-up, not realizing that wasn't enough 😅
Yes next version the auto update prompt will be disabled by default.

@IzzySoft
Copy link

IzzySoft commented Mar 7, 2024

np – and thanks for the explicit statement. I must admit I saw 4 thumbs when I first checked, but didn't check whom they came from 🙈

@polymorphicshade polymorphicshade added the 👍 planned Officially planned for a future release label Mar 24, 2024
@shuvashish76
Copy link

shuvashish76 commented Apr 4, 2024

TeamNewPipe/NewPipe#10785 (comment) | TeamNewPipe/NewPipe#10790

@polymorphicshade This is already fixed in upstream for upcoming vv0.27.0
Here for Tubular, does it connect to official NewPipe domain? Then disabling makes sense but if checks from this repository then you may consider the same behavior as NewPipe.

@shuvashish76
Copy link

Requested at F-Droid: https://gitlab.com/fdroid/rfp/-/issues/2707

@shuvashish76
Copy link

Why closed? Official F-Droid repo inclusion not planned? @polymorphicshade

@polymorphicshade
Copy link
Owner

Why closed? Official F-Droid repo inclusion not planned? @polymorphicshade

It is on F-Droid under the IzzyOnDroid repo.

@shuvashish76
Copy link

Yes. I assume originally the issue was created for official F-Droid repo. I've requested for packaging at F-Droid. So I'm asking if you intend to publish there or no interest for it.

@polymorphicshade
Copy link
Owner

@shuvashish76 oh! I understand what you're saying now.
I have little knowledge of F-Droid and how to publish things, and I don't have the time to figure it out, so I let IzzyOnDroid deal with it.

@shuvashish76
Copy link

shuvashish76 commented Apr 26, 2024

https://gitlab.com/fdroid/rfp/-/issues/2707

@licaon-kter & @eighthave from F-Droid team working on it do have a look on #52 Thanks.
See their instructions here.

@polymorphicshade
Copy link
Owner

@shuvashish76 I recently pushed a fix to master that should fix that Git clone issue they mention (hopefully...)

@IzzySoft
Copy link

IzzySoft commented Apr 26, 2024

F-Droid team working on it

And what solution did the two find? That issue is closed with no reference to a solution.

I recently pushed a fix to master that should fix that Git clone issue

Ah, you found it yourself? Glad to read it's solved then! And of course Tubular is welcome to stay in the IzzyOnDroid repo if you want it to.

so I let IzzyOnDroid deal with it.

Deal with what?

@licaon-kter
Copy link

And what solution did the two find? That issue is closed with no reference to a solution.

I couldn't even clone the repo locally, since there's some olden corruption or whatever :(

I recently pushed a fix to master that should fix that Git clone issue they mention (hopefully...)

but indeed, now it is fixed 🎉

@polymorphicshade

It is on F-Droid under the...

There's no Izzy repo unless you add it.

this issue was closed because of a misunderstanding I guess?!

the OP asks for the app being built by f-droid.org (like NewPipe is built too)

Will take a look asap, now that we can clone :)

@IzzySoft
Copy link

@licaon-kter

I couldn't even clone the repo locally, since there's some olden corruption or whatever :(

Yes, that was described. I was wondering about the solution. Meanwhile I know it: @obfusk has provided a fix, see #60 – as that's implemented, you should now be able to clone.

but indeed, now it is fixed 🎉

Yupp 😃

@obfusk
Copy link

obfusk commented Apr 26, 2024

I couldn't even clone the repo locally, since there's some olden corruption or whatever :(

Being unable to clone was because of a file with wrong mode bits (see #60 for my explanation). There was never any kind of corruption; you mistook a transient network error ("fetch-pack: unexpected disconnect while reading sideband packet") for an fsck failure.

@shuvashish76
Copy link

@polymorphicshade any updates?
planned label but issue closed ‽...please reopen the issue.

@licaon-kter
Copy link

This draft recipe metadata/org.polymorphicshade.tubular.yml

AntiFeatures:
  NonFreeNet:
    en-US: Depends on Youtube for videos.
Categories:
  - Internet
  - Multimedia
License: GPL-3.0-or-later
SourceCode: https://github.com/polymorphicshade/Tubular

RepoType: git
Repo: https://github.com/polymorphicshade/Tubular.git
Binaries: https://github.com/polymorphicshade/Tubular/releases/download/v%v/tubular_v%v.apk

Builds:
  - versionName: 0.27.2
    versionCode: 999
    commit: c2765316542743c2a4fc02d75ccf44bc22f53878
    subdir: app
    gradle:
      - yes
    rm:
      - doc
      - app/src/test

AllowedAPKSigningKeys: 8ad7025a8c911454e2a7b4515e360c52ca63ec0410a042ff46e9ad05b509e187

AutoUpdateMode: None
UpdateCheckMode: Tags .*[0-9]$

builds, but it's not build reproducible (ref: https://f-droid.org/docs/Inclusion_How-To/#reproducible-builds)

difflog: tub999.log

was the APK in https://github.com/polymorphicshade/Tubular/releases/tag/v0.27.2 built from c276531 ? Or maybe from some local dirty tree?

@asandikci
Copy link

what is the reason that release.yml could build the apk but f-droid reproducible build cannot?

steps:
- name: Checkout branch "${{ github.ref_name }}"
run: |
git clone --no-checkout https://github.com/polymorphicshade/Tubular.git .
git config core.symlinks false
git checkout --progress --force ${{ github.ref_name }}
- name: Set up JDK 17
uses: actions/setup-java@v4
with:
java-version: 17
distribution: "temurin"
cache: 'gradle'
- name: Build release APK
run: ./gradlew assembleRelease
- name: Sign APK
env:
KEYSTORE: ${{ secrets.KEYSTORE }}
SIGNING_STORE_PASSWORD: ${{ secrets.SIGNING_STORE_PASSWORD }}
run: |
version=$( grep "versionName" app/build.gradle | awk -F'"' '{print $2}' )
echo "${KEYSTORE}" | base64 -d > apksign.keystore
${ANDROID_HOME}/build-tools/34.0.0/apksigner sign --ks apksign.keystore --ks-pass env:SIGNING_STORE_PASSWORD "app/build/outputs/apk/release/app-release-unsigned.apk"
mv app/build/outputs/apk/release/app-release-unsigned.apk app/build/outputs/apk/release/"tubular_v${version}.apk"
- name: Create release and upload
run: |
version=$( grep "versionName" app/build.gradle | awk -F'"' '{print $2}' )
gh auth login --with-token <<<"${{ secrets.GITHUB_TOKEN }}"
gh release create "v${version}" --title "${{ inputs.title }}" --notes-file ".github/changelog.md" --prerelease=${{ inputs.is_pre_release }} --repo polymorphicshade/Tubular
gh release upload "v${version}" app/build/outputs/apk/release/*.apk --repo polymorphicshade/Tubular

@licaon-kter
Copy link

Not sure I follow.

As said, F-Droid can build it just fine.

But, as you can read in the attached log, there are differences compared to the released tagged apk.

Somebody familiar with the codebase can read and maybe figure out why this happens.

@YoshiRulz
Copy link

It looks like a commit updating the Hebrew locale hasn't been pushed.

@IzzySoft
Copy link

It's RB at IzzyOnDroid btw. Both last versions, as you can see:

image

Green shield = verified. See here for details on the shields.

@licaon-kter
Copy link

OK, so interesting, 0.27.1 has the same issue, same affected files, IW translation

If we dissemble and compare the res/values-iw folders we see that upstream APK files look incomplete, hence the diff log that appears to show that the F-Droid version has the missing bits.

Tubular 0.27.2 IW vs HE from the release page https://github.com/polymorphicshade/Tubular/releases/tag/v0.27.2

tubular_v0.27.2/res/values-iw $ ls -latr
-rw-r--r--   1 fdroid fdroid   907 aug  1 09:57 plurals.xml
-rw-r--r--   1 fdroid fdroid 13510 aug  1 09:57 strings.xml
-rw-r--r--   1 fdroid fdroid   328 aug  1 09:57 arrays.xml
tubular_v0.27.2/res/values-he $ ls -latr
-rw-r--r--   1 fdroid fdroid  3734 aug  1 09:57 plurals.xml
-rw-r--r--   1 fdroid fdroid 70708 aug  1 09:57 strings.xml

comparing this with Newpipe 0.27.2 IW vs HE (this is reproducible in F-Droid)

org.schabi.newpipe_999/res/values-iw $ ls -latr
-rw-r--r--   1 fdroid fdroid  4577 aug  1 15:31 plurals.xml
-rw-r--r--   1 fdroid fdroid 84154 aug  1 15:31 strings.xml
-rw-r--r--   1 fdroid fdroid   328 aug  1 15:31 arrays.xml
org.schabi.newpipe_999/res/values-he $ ls -latr
-rw-r--r--   1 fdroid fdroid  3734 aug  1 15:31 plurals.xml
-rw-r--r--   1 fdroid fdroid 70708 aug  1 15:31 strings.xml

Did you change the way IW strings are generated compared to NewPipe?

compare with my local builds of Tubular 0.27.2 IW

org.polymorphicshade.tubular_999/res/values-iw $ ls -latr
-rw-r--r--   1 fdroid fdroid  4577 aug  1 09:54 plurals.xml
-rw-r--r--   1 fdroid fdroid 84154 aug  1 09:54 strings.xml
-rw-r--r--   1 fdroid fdroid   328 aug  1 09:54 arrays.xml
org.polymorphicshade.tubular_999/res/values-he $ ls -latr
-rw-r--r--   1 fdroid fdroid  3734 aug  1 09:54 plurals.xml
-rw-r--r--   1 fdroid fdroid 70708 aug  1 09:54 strings.xml

...which seem to match sizes to those of NewPipe

Can somebody speaking IW confirm that the Github/Izzy APK shows all the strings ok and compare with NewPipe?

@licaon-kter
Copy link

So looks like the issue is symlinks treatment, looking at the comment above #7 (comment)

and doing that

    prebuild:
      - cd ../..
      - rm -fr org.polymorphicshade.tubular
      - git clone --no-checkout https://github.com/polymorphicshade/Tubular org.polymorphicshade.tubular
      - cd org.polymorphicshade.tubular
      - git config core.symlinks false
      - git checkout --progress --force v0.27.2

and now it's repro

two issues here:

@YoshiRulz
Copy link

https://github.com/polymorphicshade/Tubular/blob/master/app/src/main/res/values-iw
It's the only symlink in the repo.

@obfusk
Copy link

obfusk commented Aug 2, 2024

NewPipe isn't built on Windows or with core.symlinks=false so the values-iw -> values-he symlink works as expected.

Back in February, I just did rm app/src/main/res/values-iw to get a reproducible build (with both builds then having broken iw locale of course).

Before #60 was fixed I used the same workaround as the CI still does -- core.symlinks=false -- to get a reproducible build. As the CI still uses that even though it is no longer needed for the git clone failure, it's still in place here too.

My recommendation would be to stop building with core.symlinks=false in CI now that #60 is fixed. That way the iw locale is included properly and no workaround is needed for reproducible builds.

@rugk
Copy link

rugk commented Aug 18, 2024

As Youtube breaks the app so often, do you also consider setting up your own F-Droid repository so people can add it?

https://newpipe.net/FAQ/tutorials/install-add-fdroid-repo/

@marek22k
Copy link

An alternative to this is IzzyOnDroid (since he doesn't build the apps himself, but takes the APKs from GitHub and co) or Obtainium.

@IzzySoft
Copy link

@rugk scroll up a little, it is already available at IzzyOnDroid: https://apt.izzysoft.de/packages/org.polymorphicshade.tubular 😉

@rugk
Copy link

rugk commented Aug 22, 2024

Sure, but I don't want to include that as it contains many unrelated apps and I basically just want an up-to-date NewPipe fork here. Officially from the devs is another plus (although I trust @IzzySoft of course, but you know).
Just as the old NewPipe repo I want one simple repo that I can rely on, which does not come with yet totally unrelated hundreds or so apps, which I do not really want in F-Droid (the reasons should not matter, but e.g. one could be because IIRC the Izzysoft apps do not need to be 100% FLOSS etc/did not were built through F-Droid, which is a quality feature, after all). I just want an exception for NewPipe(fork) for obvious reasons (aka it needs to stay more up-to-date due to YouTube breaking stuff at all times 🙃).

@IzzySoft
Copy link

although I trust @IzzySoft of course

thanks 😍

did not were built through F-Droid

No, luckily we do not depend on F-Droid builds. We couldn't provide the timely updates then 🙈 and that would be a problem with things like…

it needs to stay more up-to-date

right? So IzzyOnDroid takes the APKs signed by the devs your trust, and ships those – but not before having scanned them left and right, see Ramping up security: additional APK checks are in place with the IzzyOnDroid repo.

which is a quality feature

Ah. You mean it's only quality if a certain person or org builds? Or did you mean something like… Reproducible Builds, special client support and more in our repo maybe? More than 20% of the apps at IoD are covered by RB meanwhile, and that number is still increasing.

And Tubular is one of those RB apps. So one of our verification builders built it from source (in this case it was Fay's, see above). There will probably soon be a verification builder verificator, if you want to call it that – one that runs all the recipes from the existing builders to see if it can reproduce their verification…

So the only point left from your wishlist, @rugk, is to want a bunch of "1 app only repos". Which then lack all the extra checks (and extra trust). Those are pretty good for development of course, to provide test builds to team and testers. But it also puts extra work on the shoulders of the dev team, which then has to maintain that repo as well…

@rugk
Copy link

rugk commented Aug 22, 2024

Yeah nothing against your repo, but my point stays, that I don't want my F-Droid cluttered with apps I do not need, which may or may not be (fully) open-source and whatever they may be scanned or nor, it's more about tracking issues (yeah you may also scan that, and we have exodus for that, but really, if I were to install such apps, I could use Google Play/Aurora Store) and that the app comes from the devs itself.

It was mostly inspired by the old Newpipe thing, which was convenient and I thought F-Droid also had https://f-droid.org/repomaker/ for setting up such a repo relatively simple.

@je-vv
Copy link

je-vv commented Aug 22, 2024

With latest F-Droid, one can choose the repo source for the app, per app. One can as well select which repo to pick by default by the order of the repos in the settings, the repo listed first with the app available will be the on picked by default.

So with current F-Droid there are several options to select packages from multiple repositories, particularly if one prefers packages from one over the other. Some prefer apks signed by the developer (IzzyOnDroid does this for us), some prefer to have apks from official F-Droid repo if possible, and some other would prefer the most up to date packages. For the latest, IzsyOnDroid is pretty close to the latest release on the developer releases, so that's a great option to grab latest, and still using F-Droid instead of grabbing the apk directly from the developer releases, and if not wanting the rest of the packages, keep IzzyOnDroid after the official F-Droid repo in the list of repos, or even place it as the last one, and when searching for apps, make sure to select them from other repos...

I really see no need for additional repos, if unofficial ones like IzzyOnDroid grab the apks pretty close to their developers release.

I still like for most apps to have the option between the official F-Droid repo, which might not be as up to date, but is built and signed by the F-Droid folks, and the IzzyOnDroid one, or other alternatives if available.

BTW, newPipe is a good example of a multi-repo app. When you search for it (I don't have it installed, since I prefer Tubular), and you select it, you'll see in the repositories field: F-Droid (preferred). That's because I have the F-Droid official repo before the newPipe repo (for some reason I haven't deleted that repo yet). But if I wanted to change it for the newPipe repo, I can repo source of the apk to newPipe upstream and it'll be marked as the preferred one at once. So that's the per app way. I'm using F-Droid version 1.21.0-alpha0 BTW. So there you go...

The only down side of multiple repos as of now, is that the more amount of packages from the repo, the longer it takes for the F-Droid app to update the metadata for each repo, jeje.

@obfusk
Copy link

obfusk commented Aug 22, 2024

Yeah nothing against your repo, but my point stays, that I don't want my F-Droid cluttered with apps I do not need, which may or may not be (fully) open-source and whatever they may be scanned or nor, it's more about tracking issues (yeah you may also scan that, and we have exodus for that, but really, if I were to install such apps, I could use Google Play/Aurora Store) and that the app comes from the devs itself.

Google Play/Aurora Store doesn't allow developers to sign their own apps any more. And they certainly don't have a way to check the provided APK matches the source code. Or have any kind of checks for non-free or tracking libraries like IzzyOnDroid does.

With Reproducible Builds, both IzzyOnDroid and F-Droid provide the exact same APK the developer does, verified to correspond to the published source code, with additional checks (IzzyOnDroid has several checks F-Droid doesn't, and vice versa). And IzzyOnDroid updates faster, which as you yourself pointed out is very useful for apps like Tubular :)

Edit: and any non-free components in IzzyOnDroid are clearly labelled with anti-features.

@in-plaintext in-plaintext mentioned this issue Dec 22, 2024
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
👍 planned Officially planned for a future release
Projects
None yet
Development

No branches or pull requests