Skip to content

LiBai v0.1.0 Release
Compare
Choose a tag to compare
@L1aoXingyu L1aoXingyu released this 22 Mar 07:37
· 209 commits to main since this release
v0.1.0
ba20850
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

Some major features:

  • Data Parallelism
  • 1D Tensor Parallelism
  • Pipeline Parallelism
  • Unified Distributed Layers for single-GPU and multi-GPU
  • "LazyConfig" system for more flexible syntax and no predefined structures
  • Easy-to-use trainer and engine
  • CV & NLP Data Preprocessing
  • Mixed Precision Training
  • Activation Checkpointing
  • Gradient Accumulation
  • Gradient Clipping
  • Zero Redundancy Optimizer (ZeRO)

Model_Zoo

  • Bert (3D Parallelism)
  • GPT-2 (3D Parallelism)
  • ViT (3D Parallelism)
  • Swin-Transformer (data parallelism)
  • Add finetune task in projects/
  • Add text classification in projects
Assets 2
Loading