Update on the development branch

[kaiyux]

Hi,

The TensorRT-LLM team is pleased to announce that we have pushed an update to the development branch (and the Triton backend) this Aug 7th, 2024.

This update includes:

• Model Support
  • Support GLM4 models, see examples/chatglm/README.md.
• API
  • [BREAKING CHANGE] The build time argument context_fmha_fp32_acc is moved to runtime for decoder models.
  • Supported Falcon models in the Python high level API.
• Bug fixes
  • Fixed segmentation fault in TopP sampling layer, thanks to the contribution from @akhoroshev in Fix segfault in TopP sampling layer #2039. (Segfault on main branch (problem in TopP layer) #2040)
  • Fixed the failure when converting the checkpoint for Mistral Nemo model. (Support for Mistral Nemo #1985)
  • Propagate exclude_modules to weight only quantization, thanks to the contribution from @fjosw in [Fix] Propagate QuantConfig.exclude_modules to weight only quantization #2056.
• Documentation
  • Fixed wrong links in README, thanks to the contribution from @Tayef-Shah in update links in overview section of README #2028.
  • Fixed some typos in the documentation, thanks to the contribution from @lfz941 in chore(docs): fix typos #1939.

Thanks,
The TensorRT-LLM Engineering Team

[MARD1NO]

The update link is wrong(skip to pr2053 :D

[kaiyux]

Sorry for the mistake, I've fixed it. Thanks a lot for your attention and support!