APK files missing at releases sinve v3.8.1 #4053

IzzySoft · 2024-11-04T18:43:40Z

Could you please add the APK files again to your releases (starting with the current one would be fine, no need to fill up the older ones)? Else Kiwix cannot be kept updated at e.g. IzzyOnDroid (which currently is stuck at 3.8.0 due to this).

Thanks in advance!

kelson42 · 2024-11-05T03:26:34Z

I don't understand the big report, all APK are here https://download.kiwix.org/release/kiwix-android/

MohitMaliFtechiz · 2024-11-05T06:07:56Z

@kelson42 I think @IzzySoft is talking about our release section.

Release 3.11.1	Release 3.8.0

MohitMaliFtechiz · 2024-11-05T06:26:16Z

@IzzySoft We have updated the our release 3.11.1 with APK's files https://github.com/kiwix/kiwix-android/releases/tag/3.11.1.

kelson42 · 2024-11-05T06:27:00Z

@MohitMaliFtechiz Can you easily do that? Do we know why this is not there anymore?

MohitMaliFtechiz · 2024-11-05T06:52:47Z

@MohitMaliFtechiz Can you easily do that? Do we know why this is not there anymore?

Yes, I was checking why it was not there and I found, in those releases which we self made these APKs are not available because these APKs are uploaded by our release.yml file after generating the APK

kiwix-android/.github/workflows/release.yml

Lines 49 to 55 in 1576095

    
           - name: Publish to GitHub 
        
             uses: ncipollo/release-action@v1 
        
             with: 
        
               artifacts: "app/build/outputs/apk/release/**" 
        
               token: ${{ secrets.GITHUB_TOKEN }} 
        
               draft: true 
        
               prerelease: true

which are available in our release section but currently in draft.

It is not occurring after the release 3.8.1 if we see the releases before it which is self-made (not pushed by the Github action) these APKs are not available there as shown in the below screenshot.

MohitMaliFtechiz · 2024-11-05T09:26:49Z

@MohitMaliFtechiz Can you easily do that?

It needed the manual steps to do it:

Download all artifacts from the draft release.
Upload them in the existing release.
Delete the draft release.

So to reduce the manual steps, we have now directly uploaded the APKs to the existing release instead of creating the new draft release in #4054.

IzzySoft · 2024-11-05T11:43:34Z

I don't understand the big report, all APK are here https://download.kiwix.org/release/kiwix-android/

@kelson42 the updater at the IzzyOnDroid repo looks for them at the releases of the repo hosting the source code.

@MohitMaliFtechiz thanks for fixing! 🤩 Funnily the updater here now picks 3.8.1 as latest one (most likely because the Github API returns the releases by date, and 3.8.1 has the newest timestamp), but that should "fix itself" with the next release then hopefully:

$ iod repo get org.kiwix.kiwixmobile
org.kiwix.kiwixmobile: looking for 'https://api.github.com/repos/kiwix/kiwix-android/releases'
org.kiwix.kiwixmobile: checking tag '3.8.1'
org.kiwix.kiwixmobile: lastRelNo set to '3.8.1', checking for files
org.kiwix.kiwixmobile: Upstream file date (2023-12-03 16:14) is newer than ours (2023-11-25 19:40).
org.kiwix.kiwixmobile: returning ['3.8.1','https://github.com/kiwix/kiwix-android/releases/download/3.8.1/kiwix-armeabi-v7a-release.apk',1701616476]
org.kiwix.kiwixmobile: 3.8.0/3.8.1, https://github.com/kiwix/kiwix-android/releases: https://github.com/kiwix/kiwix-android/releases/download/3.8.1/kiwix-armeabi-v7a-release.apk
- Grabbing update for org.kiwix.kiwixmobile: OK
- Checking 'repo/org.kiwix.kiwixmobile_5230801.apk' for libraries and malware …
- Checking the app's AndroidManifest.xml …
! repo/org.kiwix.kiwixmobile_5230801.apk declares flag(s): usesCleartextTraffic
! repo/org.kiwix.kiwixmobile_5230801.apk declares sensitive permission(s): android.permission.ACCESS_FINE_LOCATION android.permission.MANAGE_EXTERNAL_STORAGE android.permission.READ_EXTERNAL_STORAGE android.permission.NEARBY_WIFI_DEVICES android.permission.ACCESS_COARSE_LOCATION
! repo/org.kiwix.kiwixmobile_5230801.apk contains signature block blobs: 0x504b4453 (DEPENDENCY_INFO_BLOCK; GOOGLE)
org.kiwix.kiwixmobile: check if repo contains FUNDING.yml
org.kiwix.kiwixmobile: looking for 'https://api.github.com/repos/kiwix/kiwix-android/contents/.github'
org.kiwix.kiwixmobile: FUNDING.yml already up-to-date.
org.kiwix.kiwixmobile: calling 'getFastlaneMeta(github,[host:github.com,owner:kiwix,repo:kiwix-android,path:/fastlane/metadata/android])'
org.kiwix.kiwixmobile: FastlaneFeatures shortdesc,fulldescMD,icon,screenshotsJPG
org.kiwix.kiwixmobile: looking for 'https://api.github.com/repos/kiwix/kiwix-android/contents/fastlane%2Fmetadata%2Fandroid'
org.kiwix.kiwixmobile: looking for 'https://api.github.com/repos/kiwix/kiwix-android/contents/fastlane%2Fmetadata%2Fandroid%2Fde'
...

Oops, some exclamation marks 😱 Let's see… I cannot find details on permissions in the repo (or in its wiki), so:

what for are location permissions (especially ACCESS_FINE_LOCATION) needed?
similarly, NEARBY_WIFI_DEVICES?
does it really need MANAGE_EXTERNAL_STORAGE (I have not checked where files are stored, but shouldn't that be possible via SAF?)
what kind of cleartext (unencrypted) network connections are made?

As for DEPENDENCY_INFO_BLOCK, that can easily be avoided with a minor addition to your build.gradle:

android {
    dependenciesInfo {
        // Disables dependency metadata when building APKs.
        includeInApk = false
        // Disables dependency metadata when building Android App Bundles.
        includeInBundle = false
    }
}

For some background: that BLOB is supposed to be just a binary representation of your app's dependency tree. But as it's encrypted with a public key belonging to Google, only Google can read it – and nobody else can even verify what it really contains. More details can be found e.g. here: Ramping up security: additional APK checks are in place with the IzzyOnDroid repo.

Thanks in advance!

IzzySoft · 2024-11-05T14:39:43Z

PS: I've just checked why 3.8.1 is considered the newest release: the publish date (published_at) is the newest there.

Pulled in 3.11.1 now manually, so that's the one going live here with the next sync around 7 pm UTC today.

kelson42 · 2024-11-05T15:10:18Z

We have a mess here (also to many workflow files). We have to clean and republish the APK (but manual release triggering). I will have to look

MohitMaliFtechiz · 2024-11-05T15:25:07Z

what for are location permissions (especially ACCESS_FINE_LOCATION) needed?
similarly, NEARBY_WIFI_DEVICES?

The ACCESS_FINE_LOCATION permission is used to discover peer-to-peer (P2P) devices for transferring ZIM files to nearby devices. For Android 13 and above, we request the NEARBY_WIFI_DEVICES permission, as recommended by the WifiP2pManager API, which allows us to directly find nearby devices without needing ACCESS_FINE_LOCATION. Since the NEARBY_WIFI_DEVICES permission was introduced in Android 13, we continue to request ACCESS_FINE_LOCATION for devices running versions before Android 13.

does it really need MANAGE_EXTERNAL_STORAGE (I have not checked where files are stored, but shouldn't that be possible via SAF?)

@IzzySoft We require the MANAGE_EXTERNAL_STORAGE to load the sideloaded ZIM files in our application, no currently, it is not possible to load the sideloaded ZIM files with SAF. Currently, libkiwix needs direct filePath to properly work with the Xapain index(We have an issue open for this openzim/libzim#852). But as of now, we need this permission.

what kind of cleartext (unencrypted) network connections are made?

For this, we have opened a ticket #4055.

As for DEPENDENCY_INFO_BLOCK, that can easily be avoided with a minor addition to your build.gradle:

Thanks for this, we will fix this.

PS: I've just checked why 3.8.1 is considered the newest release: the publish date (published_at) is the newest there.

@IzzySoft, @kelson42 It was published by mistake by github action while fixing and testing this issue. My bad 😥

Pulled in 3.11.1 now manually, so that's the one going live here with the next sync around 7 pm UTC today.

Thanks for doing this ❤️

IzzySoft · 2024-11-05T16:43:57Z

Thanks @MohitMaliFtechiz 😍 Permissions are added to Kiwix' "green list" now:

android.permission.ACCESS_COARSE_LOCATION: used on Android < 13 to discover peer-to-peer (P2P) devices for transferring ZIM files to nearby devices
android.permission.ACCESS_FINE_LOCATION: used on Android < 13 to discover peer-to-peer (P2P) devices for transferring ZIM files to nearby devices
android.permission.NEARBY_WIFI_DEVICES: used on Android ≥ 13 to discover peer-to-peer (P2P) devices for transferring ZIM files to nearby devices
android.permission.MANAGE_EXTERNAL_STORAGE: used to load the sideloaded ZIM files on Android ≥ 11
android.permission.READ_EXTERNAL_STORAGE: used to load the sideloaded ZIM files on Android < 11

Already visible if you visit Kiwix in the IzzyOnDroid repo browser. Please correct if I abbreviated/deduced wrongly. Thanks for splitting off the cleartextTraffic to follow it up – and for taking care for the DEPENDENCY_INFO_BLOCK!

It was published by mistake by github action while fixing and testing this issue. My bad 😥

No worries, it's fixed here by now. And as I wrote, the next release will "close the gap" again.

Thanks for doing this ❤️

Gladly – thanks for your support and swift action!

IzzySoft · 2024-11-20T20:25:15Z

Thanks again! Btw, the F-Droid badge on your Readme links to a 404. Maybe you want to pick an IzzyOnDroid badge and link it to your app there instead, @MohitMaliFtechiz?

MohitMaliFtechiz · 2024-11-21T06:23:10Z

Thanks again! Btw, the F-Droid badge on your Readme links to a 404. Maybe you want to pick an IzzyOnDroid badge and link it to your app there instead, @MohitMaliFtechiz?

@IzzySoft Thanks. We have updated our badge in #4100.

IzzySoft · 2024-11-21T08:06:40Z

Thanks!

MohitMaliFtechiz · 2024-11-27T13:15:41Z

@IzzySoft We have just released the version 3.12.0 and it has all the APKs artifact in the release https://github.com/kiwix/kiwix-android/releases/tag/3.12.0. So it could be now directly updated to the IzzyOnDroid without needing manual pulling?

Edited
@IzzySoft One question, do you need output-metadata.json file for uploading the application on IzzyOnDroid? Since this file have the information about the all APKs.

IzzySoft · 2024-11-27T13:44:33Z

So it could be now directly updated to the IzzyOnDroid without needing manual pulling?

Triggering manually what would have run automatically later today:

$ iod repo get org.kiwix.kiwixmobile
org.kiwix.kiwixmobile: looking for 'https://api.github.com/repos/kiwix/kiwix-android/releases'
org.kiwix.kiwixmobile: checking tag '3.12.0'
org.kiwix.kiwixmobile: lastRelNo set to '3.12.0', checking for files
org.kiwix.kiwixmobile: Upstream file date (2024-11-27 12:28) is newer than ours (2024-11-05 19:44).
org.kiwix.kiwixmobile: returning ['3.12.0','https://github.com/kiwix/kiwix-android/releases/download/3.12.0/3.12.0-armeabi-v7a-standalone.apk',1732706891]
org.kiwix.kiwixmobile: 3.11.1/3.12.0, https://github.com/kiwix/kiwix-android/releases: https://github.com/kiwix/kiwix-android/releases/download/3.12.0/3.12.0-armeabi-v7a-standalone.apk
- Grabbing update for org.kiwix.kiwixmobile: OK
- Checking 'repo/org.kiwix.kiwixmobile_5231234.apk' for libraries and malware …
- Checking the app's AndroidManifest.xml …
? repo/org.kiwix.kiwixmobile_5231234.apk no longer carries sensitive permission(s): android.permission.ACCESS_COARSE_LOCATION
! repo/org.kiwix.kiwixmobile_5231234.apk contains signature block blobs: 0x504b4453 (DEPENDENCY_INFO_BLOCK; GOOGLE)
org.kiwix.kiwixmobile: check if repo contains FUNDING.yml
org.kiwix.kiwixmobile: looking for 'https://api.github.com/repos/kiwix/kiwix-android/contents/.github'
org.kiwix.kiwixmobile: FUNDING.yml already up-to-date.
org.kiwix.kiwixmobile: calling 'getFastlaneMeta(github,[host:github.com,owner:kiwix,repo:kiwix-android,path:/fastlane/metadata/android])'
org.kiwix.kiwixmobile: FastlaneFeatures shortdesc,fulldescMD,icon,screenshotsJPG
org.kiwix.kiwixmobile: looking for 'https://api.github.com/repos/kiwix/kiwix-android/contents/fastlane%2Fmetadata%2Fandroid'
org.kiwix.kiwixmobile: looking for 'https://api.github.com/repos/kiwix/kiwix-android/contents/fastlane%2Fmetadata%2Fandroid%2Fde'
...

Looks good so far. But, we don't have that app:

2024-11-27 14:35:32,250 WARNING: org.kiwix.kiwixmobile_5231234.apk (org.kiwix.kiwixmobile.standalone) has no metadata!
	Use `fdroid update -c` to create it.

What we have here is org.kiwix.kiwixmobile – which is not available at your releases, so the update will fail 🤷‍♂ Any reason those APKs have a different packageName/applicationId? To avoid the non-matching APK being downloaded over and again, I've told the updater to ignore /standalone/ for now:

$ iod repo get org.kiwix.kiwixmobile
org.kiwix.kiwixmobile: looking for 'https://api.github.com/repos/kiwix/kiwix-android/releases'
org.kiwix.kiwixmobile: checking tag '3.12.0'
org.kiwix.kiwixmobile: lastRelNo set to '3.12.0', checking for files
No true match found for org.kiwix.kiwixmobile
org.kiwix.kiwixmobile: Upstream file date (2024-11-05 19:44) not newer than ours (2024-11-05 19:44), skipping.
org.kiwix.kiwixmobile: returning local appdata as ['3.11.1','',0]
org.kiwix.kiwixmobile: 3.11.1 up-to-date.

(and btw, there's no direct update path from org.kiwix.kiwixmobile to org.kiwix.kiwixmobile.standalone – a different packageName means it's a different app)

One question, do you need output-metadata.json file for uploading the application on IzzyOnDroid? Since this file have the information about the all APKs.

No, the updater here checks the releases for files matching the defined pattern, For Kiwix, that pattern is /armeabi-v7a/(and of course restricted to *.apk, and ignoring /debug/ (hopefully) – and now also ignores /standalone.

MohitMaliFtechiz · 2024-11-28T05:48:47Z

@IzzySoft Thank you for your response.

What we have here is org.kiwix.kiwixmobile – which is not available at your releases, so the update will fail 🤷‍♂ Any reason those APKs have a different packageName/applicationId? To avoid the non-matching APK being downloaded over and again, I've told the updater to ignore /standalone/ for now:
(and btw, there's no direct update path from org.kiwix.kiwixmobile to org.kiwix.kiwixmobile.standalone – a different packageName means it's a different app)

It was due to #3933 and I have opened a separate ticket for this problem #4114.

MohitMaliFtechiz · 2024-11-28T07:28:19Z

@IzzySoft We have updated our release with org.kiwix.kiwixmobile package name https://github.com/kiwix/kiwix-android/releases/tag/3.12.0, and from the next release it will automatically publish these APKs to release.

No, the updater here checks the releases for files matching the defined pattern, For Kiwix, that pattern is /armeabi-v7a/(and of course restricted to *.apk, and ignoring /debug/ (hopefully) – and now also ignores /standalone.

@IzzySoft Why it is only taking the /armeabi-v7a/ APK, this APK is only for the armeabi-v7a architecture devices. For example, I have an arm-64-v8a device and I have downloaded the kiwix application from IzzyOnDroid https://apt.izzysoft.de/fdroid/index/apk/org.kiwix.kiwixmobile, but it is not compatible with my device.

So either it should upload all abi APKs on IzzyOnDroid or should take the universal APK from github release so that it can be installed on different architecture devices.

Like it is uploading different abi APKs for the ADB over Network application. But for Kiwix it is only uploading the armeabi-v7a APK.

IzzySoft · 2024-11-28T09:06:42Z

We have updated our release with org.kiwix.kiwixmobile package name

Thanks! That will then be picked up with the next run of the updater later today.

Why it is only taking the /armeabi-v7a/ APK

We have a size limit of 30 MB per app, so we had to decide for one. Back then it was decided for armeabi because it offers the widest compatibilities (excluding only very few devices like the Pixel 7a plus, where vendors decided to make the OS "arm64 only". At IzzyOnDroid, we want to support longevity, to enable the use of older devices (e.g. with custom ROMs) for as long as they last (I e.g. still use a 2015 Fairphone 2, which is 32bit and thus cannot support 64bit apps).

So either it should upload all abi APKs on IzzyOnDroid or should take the universal APK

Multiple APKs per release are not supported (even at F-Droid they'd then need a unique versionCode to be distinguishable). Apart from that, see what I just wrote about the size limit: even the armeabi is already scratching that. All ABIs together would be much beyond it, and the universal one is factor 3+ beyond it – so no, unfortunately not.

Like it is uploading different abi APKs for the ADB over Network application.

That's a single APK covering all ABIs ("universal APK"). You've cut off the screenshot before the APK size, but that APK there is much smaller: just 2 MB per APK.

MohitMaliFtechiz · 2024-11-28T09:15:54Z

Multiple APKs per release are not supported (even at F-Droid they'd then need a unique versionCode to be distinguishable).

It is already there in our code, all the APKs are generated with unique versionCode.

We have a size limit of 30 MB per app

Ohh!! I was not aware of that, Thanks for the explanation. Now I can understand why only armeabi-v7a is published on the IzzyOnDroid. Closing the issue.

IzzySoft · 2024-11-28T10:05:42Z

With 32bit devices slowly aging out (and in case of Kiwix, I'm not even sure if they're still capable of running the app with an acceptable "fluidity"), we can of course switch to the arm64 build later.

Btw, as you might be aware of, it's the native libs which per APK make up ~18 MB, and are stored uncompressed. I've no idea if any size optimization is possible there, but you've probably checked that already.

MohitMaliFtechiz · 2024-11-28T11:08:00Z

Btw, as you might be aware of, it's the native libs which per APK make up ~18 MB, and are stored uncompressed. I've no idea if any size optimization is possible there, but you've probably checked that already.

Yes, We have done the all optimization to minimize the APK size.

kelson42 self-assigned this Nov 5, 2024

kelson42 added the question label Nov 5, 2024

kelson42 added this to the 3.12.0 milestone Nov 5, 2024

MohitMaliFtechiz mentioned this issue Nov 5, 2024

Fixed: APK files missing at releases since v3.8.1 #4054

Merged

MohitMaliFtechiz added bug enhancement and removed question labels Nov 5, 2024

rgaudin assigned rgaudin and unassigned rgaudin Nov 5, 2024

MohitMaliFtechiz mentioned this issue Nov 6, 2024

Disabled the dependencies metadata from Singed APK for kiwix app. #4057

Open

kelson42 closed this as completed in #4054 Nov 20, 2024

MohitMaliFtechiz mentioned this issue Nov 28, 2024

The release job is publishing the standalone APK in the GitHub release, which can not be uploaded to IzzyOnDroid due to a different package name. #4114

Closed

MohitMaliFtechiz reopened this Nov 28, 2024

MohitMaliFtechiz modified the milestones: 3.12.0, 3.12.1 Nov 28, 2024

MohitMaliFtechiz closed this as completed Nov 28, 2024

MohitMaliFtechiz modified the milestones: 3.12.1, 3.12.0 Nov 28, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

APK files missing at releases sinve v3.8.1 #4053

APK files missing at releases sinve v3.8.1 #4053

IzzySoft commented Nov 4, 2024

kelson42 commented Nov 5, 2024

MohitMaliFtechiz commented Nov 5, 2024

MohitMaliFtechiz commented Nov 5, 2024

kelson42 commented Nov 5, 2024

MohitMaliFtechiz commented Nov 5, 2024

MohitMaliFtechiz commented Nov 5, 2024 •

edited

Loading

IzzySoft commented Nov 5, 2024

IzzySoft commented Nov 5, 2024 •

edited

Loading

kelson42 commented Nov 5, 2024

MohitMaliFtechiz commented Nov 5, 2024 •

edited

Loading

IzzySoft commented Nov 5, 2024

IzzySoft commented Nov 20, 2024

MohitMaliFtechiz commented Nov 21, 2024

IzzySoft commented Nov 21, 2024

MohitMaliFtechiz commented Nov 27, 2024 •

edited

Loading

IzzySoft commented Nov 27, 2024 •

edited

Loading

MohitMaliFtechiz commented Nov 28, 2024

MohitMaliFtechiz commented Nov 28, 2024 •

edited

Loading

IzzySoft commented Nov 28, 2024

MohitMaliFtechiz commented Nov 28, 2024

IzzySoft commented Nov 28, 2024

MohitMaliFtechiz commented Nov 28, 2024

APK files missing at releases sinve v3.8.1 #4053

APK files missing at releases sinve v3.8.1 #4053

Comments

IzzySoft commented Nov 4, 2024

kelson42 commented Nov 5, 2024

MohitMaliFtechiz commented Nov 5, 2024

MohitMaliFtechiz commented Nov 5, 2024

kelson42 commented Nov 5, 2024

MohitMaliFtechiz commented Nov 5, 2024

MohitMaliFtechiz commented Nov 5, 2024 • edited Loading

IzzySoft commented Nov 5, 2024

IzzySoft commented Nov 5, 2024 • edited Loading

kelson42 commented Nov 5, 2024

MohitMaliFtechiz commented Nov 5, 2024 • edited Loading

IzzySoft commented Nov 5, 2024

IzzySoft commented Nov 20, 2024

MohitMaliFtechiz commented Nov 21, 2024

IzzySoft commented Nov 21, 2024

MohitMaliFtechiz commented Nov 27, 2024 • edited Loading

IzzySoft commented Nov 27, 2024 • edited Loading

MohitMaliFtechiz commented Nov 28, 2024

MohitMaliFtechiz commented Nov 28, 2024 • edited Loading

IzzySoft commented Nov 28, 2024

MohitMaliFtechiz commented Nov 28, 2024

IzzySoft commented Nov 28, 2024

MohitMaliFtechiz commented Nov 28, 2024

MohitMaliFtechiz commented Nov 5, 2024 •

edited

Loading

IzzySoft commented Nov 5, 2024 •

edited

Loading

MohitMaliFtechiz commented Nov 5, 2024 •

edited

Loading

MohitMaliFtechiz commented Nov 27, 2024 •

edited

Loading

IzzySoft commented Nov 27, 2024 •

edited

Loading

MohitMaliFtechiz commented Nov 28, 2024 •

edited

Loading