Adding browser for zim library

[Rishabhg71]

closes #846

kiwix.mp4

This kind of setup seems best to me if it a easily visible if he wants to open the library + a new user doesn't have to look anywhere

[Jaifroid]

Yes, this looks good to me! It might be a good idea for a new tab to appear at the top when the user presses the button, if that's possible (i.e. add one programmatically, that would disappear when leaving the page), but that's a detail that could be left till after implementation.

[Rishabhg71]

@Jaifroid I don't think it's required at least not right now until this #523 gets completed
or do i add it and should be removed later

[Jaifroid]

You're right, no point adding it till #523, unless it's really clear that's going to take a very long time.

[Rishabhg71]

@Jaifroid could you take a quick look I have added a library with working app theme
one question how exactly a person can download a zim file (cant find download button)

[Jaifroid]

It's looking good!

Regarding download, on each tile there is a small, narrow strip that says "Download" -- easy to miss. You have to click it, and not anywhere else on the tile, which is a bit counterintuitive IMHO, but it is what it is. It'a an accessibility nightmare, but not our problem directly...

I also find it problematic that the filename of the archive is only visible (in some browsers) by hovering over the tile and looking at any link preview the browser might provide. It really should be on each tile, and I've argued for this, but for some reason I really cannot fathom there is resistance to the idea... It's really difficult to distinguish amongst the many different Wikipedia offerings for this reason. Again, not our problem, directly, and patching the incoming code would create a maintenance issue.

[Jaifroid]

Two small things:

• the button for browsing the library should just be in singular (it's only one library, even if it has many "books" in it);
• the button is also misaligned compared to the button next to it, by a few pixels. You might need to use Bootstrap columns (within a row) to fix that. Or it might be fixable with small style adjustments or applying bootstrap styles more carefully, or putting each button inside a div...

[Jaifroid]

Ah, and clicking the download button leads to content being blocked by the sandbox, see below. To fix that, you'll have to add an onclick event to the iframe and check what's being clicked, redirect it to top-level scope so the download occurs outside the iframe. This is the same trick we use to download external links from inside a ZIM. Look for that code and if possible re-use it.

[Jaifroid]

Look for code params.openExternalLinksInNewTabs round about 1601 in app.js to see how it's handled for the other iframe.

Hopefully the server permits CORS, so you can inspect the clicked link. If it's blocked, we'll need to ask the library.kiwix.org devs to add a CORS permissive header (I had to ask for this in the case of download.kiwix.org).

[Jaifroid]

Final thing before I forget is that library.kiwix.org doesn't support IE11 or Edge Legacy. So I suggest when the user clicks the open library button, you should sniff for a function that only those two browsers support -- I use if ('MSBlobBuilder' in window) -- and use download.kiwix.org as the iframe source instead of library.kiwix.org.

I suggest defining a parameter in init.js: params.downloadLibrary = 'https://library.kiwix.org and another one params.altDownloadLibrary = 'https://download.kiwix.org/zim/, and use those params in your code (sorry, I haven't looked at your code yet, so don't know if you already define the variables in a similar way).

[Rishabhg71]

I don't think we can add an event listener to any iframe content due to the same origin policy
the reason it's working on articleContent is because their origin is the same
we have to set a connection in the library using window.parent.postMessage() or maybe you could provide any idea?

[Jaifroid]

Yes, that's why I said we might have to ask library.kiwix.org to set a CORS-permissive header. It looks like it.

[Rishabhg71]

Can you tell me what exactly i have to do now?
Should i wait until library work is done

[Jaifroid]

Need to check what header it is returning currently, and compare to the header returned by download.kiwix.org. You should be able to check this in the network tab of browser. Check for the CORS headers in download.kiwix.org.

It's possible even CORS permissions won't fix it, because in my code for download.kiwix.org, I get the raw HTML and then manipulate that before injecting it into a DIV. I was hoping we could avoid that with library.kiwix.org... You could try switching to https://download.kiwix.org/zim/ just for a test to see whether you can access the links in JS. I think you should be able to. If you can, then this is fixable by asking for the CORS headers... I'll let you know how to do that when we've got the info (what CORS headers to set, etc.).

[Rishabhg71]

Okay it fixed it, but do you mind telling me how can I get all the mirrors

[Jaifroid]

How did you fix it on your side?

Regarding mirrors, this should be handled automatically when you click a download link in library.kiwix.org (or in download.kiwix.org). They use mirrorbrain software server-side, so it should redirect to a mirror automatically. But it depends what you're doing with the download link of course!

[Rishabhg71]

mirror-sites-in.mblibrary.info by adding this to the CSP meta tag under frame-src
if it's managed automatically then maybe adding *.mblibrary.info should work

[Jaifroid]

OK, well done! So it was our sandbox rather than server-side lack of CORS policy. Well spotted.

[Jaifroid]

On mirror source, I guess you'd have to experiment to see what works, it's not very transparent exactly how it works.

[Rishabhg71]

I think you can review PR now

[Jaifroid]

I'll need a bit of time -- I've got some very urgent deadlines...

[Rishabhg71]

I'll need a bit of time -- I've got some very urgent deadlines...

ok no worries

On mirror source, I guess you'd have to experiment to see what works, it's not very transparent exactly how it works.

I have pushed a solution and I am 90% confident this will be it. I have allowed every subdomain for the mirror domain
*.mblibrary.info so unless it's under the same domain I don't it its gonna break

[Jaifroid]

Just quickly (without reviewing properly), I notice you haven't internationalized the new button text. Please add:

Spanish: "Biblioteca ZIM"
French: "Bibliothèque ZIM"

(I don't think we need the word for "browse" in each case, as the buttons will become too big).

Look at how it's done for the adjacent button (data-i18n attribute), and then add the appropriate strings in en.jsonp.js, es.jsonp.js and fr.jsonp.js.

Then test buttons in all three languages and on narrow screen sizes to ensure responsiveness.

Thanks!

[Jaifroid]

Happy to confirm going with the suggested path, and thanks for further explanation. So, I suggest that if it is Chrome in a local extension, which basically means Chrome in jQuery mode OR in localServiceWorker mode, we should open library.kiwix.org in a new browser tab. If it's Chrome using the browser-extension.kiwix.org code (which we strongly encourage users to do), then you should have no problems, because no code is blocked in that mode (eval, etc. all run fine).

Let me know if you don't follow that hasty explanation, or if I got anything wrong.

[Jaifroid]

PS I don't think you need to be too elaborate: if it's easier just to detect that library.kiwix.org won't work, and in all such cases just open a new tab with download.kiwix.org (whether it's IE11, Edge Legacy, or Chrome in a local extension), then that would be fine too.

[Rishabhg71]

@Jaifroid,

Update: I've implemented the XHR method, but unfortunately, it's not functioning correctly in IE11. When clicking on a link, IE opens the file as text instead of downloading it. I've tried various approaches, like opening a new tab for the URL and setting the download URL from the app root, but none of them seem to work.

I'm starting to think it might be best to take a different approach: when the platform is not supported, we could simply open the link in a new tab.

Additionally, I've been working on this PR for a while, and I'm concerned that it might not be adding significant value to the organization. If it's acceptable, we could proceed with the above approach for now and revisit this PR later to add support for more versions.

[Jaifroid]

I'm very happy for you to open in a new tab when library.kiwix.org is not supported. In fact it's what I though you'd decided to do in your response to #1127 (comment).

XMLHttpRequest does work in IE11 for most mirrors, but not all mirrors, because some do not send the right headers to force a download in the browser. Maybe you were stuck testing a mirror that doesn't work.

In any case, that comment is not implying you should try to fix it. As we discussed above, opening download.kiwix.org in a new tab when you detect that ´library.kiwix.org´ is not supported (that includes in a Chromium extension) seems a good compromise to me.

[Rishabhg71]

@Jaifroid I feel this should be the final Review and one interesting thing I found is library.kiwix.org is supported in jQuery mode (new browsers)
Also, I think I should start working on #1032

[Jaifroid]

@Jaifroid I feel this should be the final Review and one interesting thing I found is library.kiwix.org is supported in jQuery mode (new browsers) Also, I think I should start working on #1032

Interesting! I'll take a look. And yes, please feel free to start work on #1032. I'll add some thoughts in that issue.

[Jaifroid]

This is looking excellent to me! Well done, it makes a big difference to UX. You'll see I have some very minor suggestions that will probably take 5 or 10 minutes. While you're doing that, I'd like to check the code in an old Firefox OS IDE, which is also a good way of checking that there aren't any regressions with old browsers. So let me know when you've done the minor changes, so I can confirm I've completed that test.

[Jaifroid]

OK, the Firefox OS test has thrown up a small issue, which is that the file selectors are hidden in that implementation (see screenshot), and unfortunately that includes the library button. It's because in Firefox OS there is access to the FS, so the app scans the FS instead of relying on file pickers. The library button is currently hidden along with the standard file picker.

[Jaifroid]

Now I'm not sure what the best way to deal with this is. We could just forcibly unhide the buttons, but it makes for a cluttered UI (see below).

[Jaifroid]

Or else, we could say that some reworking of the UI will be necessary for #656, so we could deal with it there.

Any thoughts?

[Jaifroid]

When clicking on the button, I get the screen below. Any idea why it's untrusted? Are you using https? In any case, the user can click on "I understand the risks", and then the library opens, so I think that's acceptable. But we should just check that it's not redirecting to http: instead of https:...

[Jaifroid]

And the good news is that it's possible to download a ZIM, though the user has to go through another warning screen. Tbh, I think it's just the age of the device (it's kind of a miracle it works at all, really). NB I support this because Kiwix JS was originally built for Firefox OS, and the previous main developer was very attached to it. It's a good proxy for possible regressions. But I don't expect it to function perfectly any more.

[Rishabhg71]

Or else, we could say that some reworking of the UI will be necessary for #656, so we could deal with it there.

Yes, let's merge it here first I will figure it out later

When clicking on the button, I get the screen below. Any idea why it's untrusted? Are you using https? In any case, the user can click on "I understand the risks", and then the library opens, so I think that's acceptable. But we should just check that it's not redirecting to http: instead of https:...

All the URLs are https, not sure why it's giving a warning

[Jaifroid]

OK, so I think it's just the age of the device, probably doesn't handle https very well. At least it gives the option to go ahead anyway.

So, when you're happy with your final fixes (let me know if you want me to check anything), go ahead and squash/merge.