v6 Roadmap

[jeanlescure]

This adopted library is taking wings of its own. In 2020-2021, weekly downloads briefly surpassed the 10k 16k milestone. So much so, that large open source projects have trusted short-unique-id as generator in mission critical functionalities of production-ready solutions.

Over the past 3+ years we have focused on improving quality of the tool specifically for the Javascript (and more recently Typescript) ecosystem.

Now that maturity has been reached, we have begun to hit some walls in regards to progress (#31), mainly due to our choice of being "Deno first". Also there is visible interest (#40) in having this tool ported to other languages.

With the previous in mind, we have done our best in regards of proper research and have committed to once again re-write this library, this time in Rust, in order to be able to compile native modules that can be wrapped-around by easily distributable packages in Javascript/Typescript, WebAssembly, C/C++, Python, etc.

Another significant change we will be introducing is a complete re-brand of short-unique-id, to become:

Add suid alias to CLI

FOR THE RECORD The main reason behind this decision is one of convenience when using the CLI version. At the end of the day, we truly believe that shorter is better!

Note: we are scrapping the idea of re-naming/re-branding since the name and domain have been effective and we have a large enough user-base that we do not want to cause confusion or frustration for existing users.

Minimum Viable Product (MVP)

In order to better serve the existing users of our library and CLI tool we will initially focus on building:

• a port to Rust which compiles a native module consumable by Node.js
• a WebAssembly module for modern browser usage
• and a transpilation from WebAssembly to vanilla Javascript for legacy browser usage

Also we need to figure out the best way to offer the CLI version of this tool through mainstream package managers (both in terms of OS package managers, as in apt-get, homebrew, etc, and programming language specific package managers, as in npm, pip, etc).

This sets an expectation that our core team will be focusing mainly on Javascript/Typescript backwards compatibility. This is not to say that work related to other programming languages will not be done. Any and all help in this regard will be welcome with open arms 🙌🏼

Roadmap

• Refactor short-unique-id to be a more traditional Typescript package (a.k.a. less Deno centric)
• Add alias suid for shorter CLI usage
• Implement a Proof Of Concept (POC) on Rust (at least the random_uuid function)
• Create a v5 branch
• Setup initial Rust project
• Port all tests over to the newly created Rust project
• Port all functionality to an appropriately named struct
• Test replacement of short-unique-id with resulting package (Node.js compatible native module) on a fork of a back-end project that uses it
• Test replacement of short-unique-id with resulting package (WebAssembly module) on a fork of a front-end project that uses it
• Test replacement of short-unique-id with resulting package (vanilla JS transpiled function) on a fork of a front-end project that uses it
• Port CLI functionality to Rust
• Solve widespread distribution of CLI functionality
• Publish beta of new package
• Publish release candidate (RC) of new package
• Publish new v5 package

[jeanlescure]

I've already begun work on the Rust POC, here's many useful resources I've encountered so far:

• https://levelup.gitconnected.com/rust-with-visual-studio-code-46404befed8
• https://github.com/rust-lang/vscode-rust
• https://github.com/rust-lang/rust-clippy
• https://github.com/rust-lang/rustfmt
• https://github.com/rustwasm/wasm-pack
• https://rustwasm.github.io/wasm-pack/book/quickstart.html
• https://github.com/WebAssembly/binaryen#wasm2js
• https://doc.rust-lang.org/rust-by-example/index.html
• https://doc.rust-lang.org/cargo/guide/creating-a-new-project.html
• https://doc.rust-lang.org/std/
• https://rust-lang-nursery.github.io/rust-cookbook/algorithms/randomness.html
• https://rust-random.github.io/book/guide-rngs.html
• https://rust-random.github.io/book/guide-seeding.html
• https://rust-lang.github.io/rfcs/2169-euclidean-modulo.html

FOR FUTURE REFERENCE Regarding WASM, I tried setting up an environment using Emscripten but failed hard due to LLVM incompatibilities in Arch Linux. Best way to go is to use the listed wasm-pack by official rustwasm team.

[jeanlescure]

This is my initial try to implement random_uuid in Rust:

use std::time::{SystemTime, UNIX_EPOCH};
use rand_core::{RngCore};
use rand_pcg::Pcg32;

const DEFAULT_DICT: &[u8] = b"dx4sqy6W8vX5fJMwa9icNgm7QPl32BhTVEutR1U0rbGeoYAKZSLDjpkInOzHCF";
const MAX_RGN: f32 = u32::MAX as f32;

pub fn random_uuid(uuid_length: i32) -> String {
  if uuid_length <= 0 {
    panic!("Invalid UUID Length Provided");
  }
  const DICT_SIZE: usize = DEFAULT_DICT.len();

  let mut id = String::with_capacity(uuid_length as usize);
  let mut random_fragment_idx: usize;

  let mut rng = Pcg32::new(
    SystemTime::now()
      .duration_since(UNIX_EPOCH)
      .expect("Time went backwards").subsec_nanos() as u64,
    0xa02bdbf7bb3c0a7
  );
  let mut rgf: f32;

  for _ in 0..uuid_length {
    rgf = rng.next_u32() as f32;

    random_fragment_idx = (
      ((rgf / MAX_RGN) * DICT_SIZE as f32) as i32
    ).rem_euclid(DICT_SIZE as i32) as usize;

    id.push(DEFAULT_DICT[random_fragment_idx] as char);
  }

  println!("> {}", id);
  id
}

#[cfg(test)]
mod tests {
  use super::*;
  
  #[test]
  fn it_works() {
    assert_eq!(random_uuid(1).len(), 1);
    assert_eq!(random_uuid(3).len(), 3);
    assert_eq!(random_uuid(6).len(), 6);
  }

  #[test]
  #[should_panic(expected = "Invalid UUID Length Provided")]
  fn it_panics() {
    random_uuid(0);
  }
}

OUTPUT:

⮞ cargo test -- --nocapture
    Finished test [unoptimized + debuginfo] target(s) in 0.00s
     Running target/debug/deps/mod_test-5a655cdd6405616e

running 2 tests
thread 'tests::it_panics' panicked at 'Invalid UUID Length Provided', src/lib.rs:10:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
test tests::it_panics ... ok
> e
> oZ9
> ErkP5l
test tests::it_works ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

   Doc-tests mod_test

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

There's two decisions I'd like to highlight:

1. The usage of rand_pcg (random PCG) crate rather than the more commonly used rand: as this is a POC I wanted to dip my hands in chasing better performance rather than true "randomness", mainly because our usage of random integers is just one in five layers of inherent randomness. In any case, once we start implementing the proper port of the code, there will have to be deeper research into how using rand vs rand_pcg will affect the probabilities of collision (if at all).
2. Using rem_euclid rather than %: again I was chasing better performance here, and modulo is the most computationally expensive of the algebraic functions we use, so this euclidean implementation being part of Rust out of the box was a nice surprise. So far I have not perceived any difference (as in anything other than better speed) from the % we've been using (and there probably isn't any since euclidean modulo vs non-euclidean only affect results where negative numbers are involved).

[jeanlescure]

This repository has been fully refactored to be a more traditional Typescript/JS package. The Rust rework has been pushed to 2022 as v5.

v4 is now live 🥳

[jeanlescure]

Going over the notes regarding rand vs rand_pcg, using either of these in the output for any other language would add the responsibility for the performance of said pseudo-random generator(s) on other languages.

Each language should use their default pseudo-random generator (i.e. Math.random() for JS/TS).

In order to achieve this we should consider implementing the ability to use your own pseudo-random generator as it has been aluded to on #49 . Then we can use the feature's logic to set the default based on the language you are compiling for.

[jeanlescure]

As of v4.4.0 (d260860) we can now use the cli as the much shorter command suid 🎉

[jeanlescure]

We're going to put this one to rest, there has been virtually zero interest from the community for ports of this repo to other languages, we've narrowed our scope at Simply Hexagonal to be creators and maintainers of Typescript libraries, and the features we wanted to see included in short-unique-id such as the CLI binary have been delivered in v5.