Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved the scanning of ZIM files. #3674

Merged
merged 6 commits into from
Feb 5, 2024
Merged

Improved the scanning of ZIM files. #3674

merged 6 commits into from
Feb 5, 2024

Conversation

MohitMaliFtechiz
Copy link
Collaborator

@MohitMaliFtechiz MohitMaliFtechiz commented Jan 17, 2024

Fixes #3646

  • Excluding the "data," "obb," and "Trash" folders from scanning is justified for several reasons. The "Trash" folder contains deleted files, making it unnecessary for scanning. Additionally, the "data" and "obb" folders are specifically designed for the app's private directory, and users usually do not store ZIM files there. Most file managers prohibit direct copying of files into these directories. Therefore, scanning these folders is not essential. Moreover, such scans consume time, given the presence of numerous files written by other apps, which are irrelevant to our application.

    • Without this fix
      Which takes 2.3 Seconds to scan for the ZIM files.

    • After place fix
      It takes 1.6 seconds to scan the storage for the same ZIM files. However, in obb, and the data folder has only a few files but still reduces the .7 second

  • Now, scanning the storage only occurs when the user explicitly requests it. When the user attempts to refresh the list, the storage is scanned to identify ZIM files. This approach prevents unnecessary scanning when users frequently navigate to other screens.

  • We have enhanced our scanning process to provide real-time progress updates, allowing users to track how much storage has been scanned.

XRecorder_26012024_163432.mp4
XRecorder_26012024_163518.mp4

@MohitMaliFtechiz MohitMaliFtechiz marked this pull request as draft January 17, 2024 10:32
Copy link

codecov bot commented Jan 17, 2024

Codecov Report

Attention: 10 lines in your changes are missing coverage. Please review.

Comparison is base (0fafa85) 48.54% compared to head (ddbb11c) 48.69%.
Report is 2 commits behind head on main.

❗ Current head ddbb11c differs from pull request most recent head e3b597b. Consider uploading reports for the commit e3b597b to get more accurate results

Files Patch % Lines
...le/nav/destination/library/LocalLibraryFragment.kt 52.63% 4 Missing and 5 partials ⚠️
...g/kiwix/kiwixmobile/core/utils/files/FileSearch.kt 95.45% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #3674      +/-   ##
============================================
+ Coverage     48.54%   48.69%   +0.15%     
- Complexity     1089     1090       +1     
============================================
  Files           285      285              
  Lines         10591    10622      +31     
  Branches       1422     1429       +7     
============================================
+ Hits           5141     5172      +31     
+ Misses         4606     4603       -3     
- Partials        844      847       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@kelson42
Copy link
Collaborator

@gouri-panda Any chance you can review this PR?
@MohitMaliFtechiz From your video I have no clue how to request a scan works and how the progressbar looks like. Do I'm just blind?

@MohitMaliFtechiz
Copy link
Collaborator Author

MohitMaliFtechiz commented Jan 26, 2024

I have no clue how to request a scan works

@kelson42 When the user attempts to refresh the list then the scan will work.

How the progress bar looks like.

@kelson42 We have implemented the progress bar as we have in the reader screen when loading the article. I think in the video progress bar does not look proper due to the small height. Let me increase the height of the progress bar so that it will look proper in the UI, and the user clearly see it.

@MohitMaliFtechiz MohitMaliFtechiz force-pushed the Fix#3646 branch 2 times, most recently from 608c97f to 6ad6e6f Compare January 26, 2024 11:09
@MohitMaliFtechiz
Copy link
Collaborator Author

I have changed the UI to show the progress properly and updated the video on the PR description.

@kelson42
Copy link
Collaborator

@MohitMaliFtechiz So you need to swap down in the local library to request a new scan. This is the only way to request a new scan. When a new scan happen then what happen exactly. Can you please provide a screencast because this is unclear to me.

@MohitMaliFtechiz
Copy link
Collaborator Author

@MohitMaliFtechiz So you need to swap down in the local library to request a new scan. This is the only way to request a new scan. When a new scan happen then what happen exactly. Can you please provide a screencast because this is unclear to me.

@kelson42 Yes, this is the proper and easy way to refresh any data, SwipeRefreshLayout (Swap down) is specially designed for refreshing the data.

When a new scan is triggered then a horizontal progress bar starts between the ZIM files list and the NavBar which shows the progress of how much storage has been scanned.

RefreshData.mp4

@kelson42
Copy link
Collaborator

@gouri-panda Can you please review?

Copy link
Collaborator

@kelson42 kelson42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From a user POV LGTM.

@gouri-panda
Copy link
Collaborator

@kelson42 Sure :) Give me some time.

Copy link
Collaborator

@gouri-panda gouri-panda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kelson42 Both the UI and code looks good! @MohitMaliDeveloper Thanks :)

* Excluding the "data," "obb," and "Trash" folders from scanning is justified for several reasons. The "Trash" folder contains deleted files, making it unnecessary for scanning. Additionally,
the "data" and "obb" folders are specifically designed for the
app's private directory, and users usually do not store ZIM files there.
Most file managers prohibit direct copying of files into these directories. Therefore, scanning these folders is not essential. Moreover, such scans consume time, given the presence of numerous files written by other apps, which are irrelevant to our application.
…yFragment`.

* Now, scanning the storage only occurs when the user explicitly requests it. When the user attempts to refresh the list, the storage is scanned to identify ZIM files. This approach prevents unnecessary scanning when users frequently navigate to other screens.
* We have enhanced our scanning process to provide real-time progress updates, allowing users to track how much time is required to complete the operation.
…lTest` test that are failing after this change.
@kelson42 kelson42 merged commit ba327da into main Feb 5, 2024
7 of 8 checks passed
@kelson42 kelson42 deleted the Fix#3646 branch February 5, 2024 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Finding local .zim files takes a very long time ~15 minutes
4 participants