You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
quite old (especially regarding TypeScript/JS coding conventions)
not well suited for larger collections (no search, no tagging etc.)
barely updated (dependencies are out of date, some of them no longer supported etc)
I think this server doesn't work with newer versions of the website-scraper node module, so it's still on an older version
While it still works (I've had it running for 4+ years), it feels like it's time for an overhaul. The following could be good steps:
Creating a better frontend (the code in browser.js works, but is a mess)
Creating a docker (compose) configuration that just works to simplify installation
Maybe switch to a database instead of just storing everything in a JSON file (not sure I'll do this, as it complicates setup etc.)
Updating the downloader code to use newer website-scraper versions
2 & 4 should be possible without major rewrites, the others are somewhat more involved.
Maybe it also makes sense to rewrite the server in Go because I like it better, but I'm not sure if there's a good website-scraper-like module for Go. I actually tried writing something like that that started a chromium browser and used SingleFile to download pages, but it didn't work that well.
I'm not sure when (or if) I'll work on this.
The text was updated successfully, but these errors were encountered:
This software is
website-scraper
node module, so it's still on an older versionWhile it still works (I've had it running for 4+ years), it feels like it's time for an overhaul. The following could be good steps:
browser.js
works, but is a mess)2 & 4 should be possible without major rewrites, the others are somewhat more involved.
Maybe it also makes sense to rewrite the server in Go because I like it better, but I'm not sure if there's a good
website-scraper
-like module for Go. I actually tried writing something like that that started a chromium browser and used SingleFile to download pages, but it didn't work that well.I'm not sure when (or if) I'll work on this.
The text was updated successfully, but these errors were encountered: