joliGEN v4.0.0

This main version adds many improvements as well as video generation with diffusion and super resolution with supervised metrics, including for consistency models.

Features

• adding separate control of vertical and horizontal flips as augmentation (a7a6109)
• aligned crops for super-resolution (8418470)
• allow tf32 on cudnn (367cd91)
• better Canny for cond image with background (c3c7de6)
• consistency models with supervised losses (ed701ad)
• data: random bbox for inpainting (764646d)
• input and output multiple and different channels (6bcd64c)
• load models without stricness (073d57c)
• max number of visualized images from train/test set (24f0e81)
• ml: add option for vid inference (c3f83b7)
• ml: add supervised loss with GANs with aligned datasets (d7f5119)
• ml: added LPIPS supervised loss with GANs (70e8ee4)
• ml: adding example of CM+discriminator (b6b8b64)
• ml: batched prompts for turbo (023dd54)
• ml: Canny can use a range of dropout probabilities (7b4c860)
• ml: canny dropout for vid (06ce7d7)
• ml: CM with added discriminator (10516e0)
• ml: consistency models for pix2pix (cd92712)
• ml: CUT turbo (cdd508f)
• ml: debug args (a11172b)
• ml: debug crop (51c9fd6)
• ml: debug for canny inference (930f3ce)
• ml: debug for canny threshold (dca0bfa)
• ml: debug for vid metrics (7c57471)
• ml: debug inference_vid for canny (17b9a29)
• ml: debug vid for frame limit (ff97c03)
• ml: debug vid metric (ba43725)
• ml: DISTS supervised loss for aligned data (56273ef)
• ml: FID,KID,MSID for multiple test sets and non 8 bit images (74b0e65)
• ml: fix canny range option (c102ee0)
• ml: fix inference regeneration and crop canny (f75196f)
• ml: HDiT for GANs (58bedff)
• ml: HDiT generator (9a95f1f)
• ml: jenkins test inference print (b68ab53)
• ml: L1 or MSE for diffusion multiscale loss (06e3d6a)
• ml: metric fvd for video (6d458a3)
• ml: min-SNR loss weight for diffusion, 2303.09556 (c802119)
• ml: modif for horse2zebra prompt (b66a954)
• ml: multiple test sets (6db745c)
• ml: option for max_sequence_lenght of video generation (12cfc1b)
• ml: prompt for inference horze2zebra (b8e9929)
• ml: random canny inside batch (70919cd)
• ml: rename dataloader for video generation (98b1315)
• ml: The implementation of UNetVid for generating video with temporal consistency and inference (43b7018)
• ml: unchange fill_img_with_canny with random drop canny (a2ed3fc)
• ml: UNetVid for generating video with bs > 1 (00f11bc)
• ml: vid try autoregressive inference (5b92031)
• multi-prompt local works (b98746a)
• multiprompt (2bffc8b)
• multistep lr scheduler (01c3558)
• train_finetune for finetuning gans/others and removing / adding losses and networks (2f26503)
• unet_vid motion module fine-grained configuration (813e435)

Bug Fixes

• aligne dataset, resize domain A only if necessary (4127571)
• allowing for no NCE with cut (9d8ff9b)
• clamp bbox to image size during inference (fc3874d)
• cm at test time (706356b)
• cm with conditioning (0fd2d14)
• consistency model schedule upon resume (88d03f9)
• consistency models with input/output different channels (db61821)
• crash in inference script, errors in documentation (f99dd34)
• cut options at test time (dcd2438)
• D input is G output size with gans (194f42b)
• diff across input/output channels in gans (6845816)
• diff real/fake not needed + cleanup (5cbd1f0)
• diffusion inference for images > 8bits (aefdc38)
• diffusion with input and output of different channel size (cd264de)
• disable hdit flop count (8c449f8)
• fix pytest rootdir (1fe0e80)
• further lowering the input test size of cut-turbo (6914731)
• gan inference script with prompts ([cef7681](https://github.com/jolibrain/j...

joliGEN v3.0.0

This release adds Consistency Models and REST API for GAN and diffusion models inference.

Features

• api inference for gan and diffusion (6fd43d8)
• inference server option returns base64 image (5bc8f44)
• ml: consistency models as diffusion models for inpainting, with conditioning (de9d725)
• scripts: canny thresholds grid plots (6d94ffa)
• specialized helpers for each model (f361414)

Bug Fixes

• correct help for image generation scripts (76ce6b4)
• enable model export during training (79525b8)
• not force reloading dinov2 models (800967d)
• ref_in option (2927574)
• resize image to test base64 api diffusion (2708a6f)

Docker images:

• GPU (CUDA only): docker pull docker.jolibrain.com/joligen_server:v3.0.0
• All images available from https://docker.jolibrain.com/#!/taglist/joligen_server

joliGEN 2.0.0

Features

• ml: dinov2 discriminator with registers (7fcf790)
• ml: DinoV2 feature-based projected discriminator (c67ffa8)
• ml: SigLIP based projected discriminators (5e10a86)
• optimization eps value control (0556987)
• pix2pix task for palette (7e47139)
• scripts: adding a video generation script for gans (85d1922)

Bug Fixes

• amp for discriminators (811ba3d)
• APA augmentation on multiple discriminators (becb3eb)
• docker release script (f1c56de)
• end of training metrics computation (e1f213c)
• init_metrics directory and metrics on CPU (0b77943)
• load size for rectangular images, resize ref image for inference (965e1bf)
• ml: inference for diffusion with reference image (df8c504)

Docker images:

• GPU (CUDA only): `docker pull docker.jolibrain.com/joligen_server:v$tag`
• All images available from https://docker.jolibrain.com/#!/taglist/joligen_server

v1.0.0

joliGEN: Generative AI Toolset (Changelog)

1.0.0 (2023-10-06)

Docker

• GPU (CUDA only): `docker pull docker.jolibrain.com/joligen_server:v1.0.0`
• All images available from https://docker.jolibrain.com/#!/taglist/joligen_server

Features

• add a server endpoint to delete files (30b2143)
• add choices for all options (ed43b82)
• add ddim inference (0196134)
• add DDPM tutorial on the VITON-HD dataset (c932d73)
• add FastAPI server to run training (f517462)
• add lambda for semantic losses (aab53fe)
• add LPIPS metric (f1e0526)
• add miou compute to tests (c0033ef)
• add new metrics (f3c84cd)
• add palette model (b7db294)
• add psnr metric (7135458)
• add sampling options to test (a2958dc)
• add SRC and hDCE losses (ddfcc97)
• add test for doc generation (41526f8)
• add test on cycle_gan_semantic_mask (3eeff76)
• add tests for reference image dataloaders (ae6405e)
• added D noise to CUT with semantics (31aa4a3)
• added optimizers and options (505cac2)
• allow control of projected discriminator interpolation (dbffec5)
• allow ViT custom resolution at D projector init (82e6e83)
• api: display current commit at startup (6f90be8)
• aug: affine transforms for semantics (170b0f8)
• aug: configurable online mask delta augmentation by x and y axis (dfa6459)
• aug: select bbox category through the path sanitization functionality (a8d3f48)
• auto download segformer weights (083cc5e)
• backward while computing MSE criterion loss (1b87906)
• bbox as sam prompt (a39c5bd)
• bbox prompt for sam (1fa9cae)
• bilinear interpolation of attention heads when dimension does not match, useful for segformer G (eed9494)
• bw model export (8e43efa)
• check code format when PR (eeb56cb)
• choices for canny random thresholds (9573fc1)
• class weights for semantic segmentation network with cross entropy (4274f1e)
• classifier training on domain B (fa343c0)
• commandline saving (6eb503e)
• commandline script for joligan server calls (48ae23b)
• compute_feats for unet G (9f1109e)
• conditioning for palette (b9854ee)
• config json for client script (174dce9)
• context for D (b0d3c7b)
• contrastive classifier noise (7193e0e)
• contrastive loss for D (deb2ec4)
• cut_semantic model (b20a943)
• D accuracy (26ead91)
• data: random noise in images for object insertion (42cf13d)
• DDP (68f24da)
• deceiving D for GAN training (2e2113f)
• depth model as projector (10ffc28)
• depth prediction and depth discriminator (01bc62b)
• diff augment (054509c)
• diffusion inference with old and new models (9c4c5a9)
• display augmented images (2126253)
• display test images (a1de083)
• doc options auto update (1b08f92)
• doc: add JSON config examples (5332213)
• doc: basic server REST API (a757d17)
• doc: datasets (dfe2343)
• doc: DDPM conditioning training and inference examples (e694a29)
• doc: models (be1fe34)
• doc: refactored README with links to documentation (b5bf121)
• doc: reference image conditioning (70aeb32)
• doc: remove overview (2360527)
• doc: server, client, docker (68a5b96)
• doc: tips (3fea9ca)
• doc: training (a4b720d)
• doc: update inference models and examples (3c43a7b)
• doc: updated FAQ (88b417c)
• doc: updated model export (e692f78)
• edge detection techniques (78202ea)
• export for unet_mha (b4c3cfd)
• extract bbox from img (fb64ef0)
• first recut model (aaa4069)
• first test (4ac8cd9)
• fixed bbox size for online creation and bbox size randomization ([5cd6227]...