Post-1.0 Roadmap

[thekevinscott]

As we gear up to release the 1.0 release, I'd like to start a discussion into what comes after. What are some of the things you'd like to see next in UpscalerJS?

Here's a non-exhaustive list of ideas of mine, in no particular order:

New Models

We could continue adding new models to the UpscalerJS ecosystem. Some ones in particular I'd like to add:

• DDNM
• Mirnet

New Input Mechanisms

We could explore additional input mechanisms in UpscalerJS, including:

• Video
• 4D Tensors (e.g., support for multiple images)

We could also investigate support for text and other supplementary parameters such as masks for something like inpainting with DDN.

Performance Improvements

We could continue exploring performance improvements.

There's two general buckets:

• Improvements to the models, such as pruning
• Improvements to the core algorithms, like optimizing the patch size calculations
• First class support for transmitting memory to web workers (maybe leveraging the shared memory model)

Platform Support

Better support for other platforms, like bun or deno.

Is Node ESM support something worth exploring?

DX Improvements

We could explore things that make developers' lives easier, like:

• Adding the ability to load models in the browser from just a string, no installation necessary
• Adding the ability to load models from arbitrary endpoints, such as hugging face or GitHub.

---

If there are particular things you'd like to see in the next version of UpscalerJS, leave a comment below!

[AlenaTih]

Hello,

Could you please tell me if it is possible to integrate several different models in the same app, so that you can have a dropdown menu and a user can choose which model to choose? Like ersgan super resolution, deblurring, dehazing, denoising, deraining, low light enhancement, and retouching, all in one.
(For an app that works in a browser)

Thank you so much!

[thekevinscott]

Certainly, you would need to create separate UpscalerJS instances for each, like:

import Upscaler from 'upscaler';
import x2Model from '@upscalerjs/esrgan-thick/2x'
import deblurringModel from '@upscalerjs/maxim-deblurring'
import denoisingModel from '@upscalerjs/maxim-denoising'

const upscaler = new Upscaler({
  model: x2Model,
})

const deblurrer = new Upscaler({
  model: deblurringModel,
})

const denoiser = new Upscaler({
  model: denoisingModel,
})

I haven't tried this myself, so I'm not sure what the performance implications will be. When switching between instances, you can dispose of old instances with:

upscaler.dispose().then(() => {
  console.log("All cleaned up!");
})

Which might help if on device memory becomes a concern.

[AlenaTih]

Thank you so much!