Support QoS in api_server

[sallyjunjun]

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily receiving feedbacks. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

The purpose of this commit is to introduce Quality of Service (QoS) functionality. When the system is in an overloaded state, this feature ensures prioritization based on user priority, allowing high-priority users to have precedence. For users with the same priority, resources can be utilized in accordance with the specified ratio.

Modification

1. Added the QoS (Quality of Service) module along with configuration file parsing.
2. Introduced three rate-limiting interfaces: chat_completions_v1_qos, completions_v1_qos, and chat_interactive_v1_qos.
3. Included test files for this module.
4. No impact on the functionality of existing interfaces.

BC-breaking (Optional)

Does the modification introduce changes that break the backward-compatibility of the downstream repositories?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.

Use cases (Optional)

If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.

Checklist

1. Pre-commit or other linting tools are used to fix the potential lint issues.
2. The modification is covered by complete unit tests. If not, please add more unit tests to ensure the correctness.
3. If the modification has a dependency on downstream projects of a newer version, this PR should be tested with all supported versions of downstream projects.
4. The documentation has been modified accordingly, like docstring or example tutorials.

[lvhan028]

May provide user guide

[lvhan028]

Please resolve the linting error

[lvhan028]

图像不建议用文件的方式放入repo，这样会导致项目太大了。请使用github图床方式，加入图像的链接

[sallyjunjun]

May provide user guide

[awslshadowstar]

图像不建议用文件的方式放入repo，这样会导致项目太大了。请使用github图床方式，加入图像的链接

现在github 图床获取到的链接都是 private-user-images.githubusercontent.com 域名的带过期jwt的图片链接，不适合用于文档，请问是否可以使用第三方图床工具

[lvhan028]

图像不建议用文件的方式放入repo，这样会导致项目太大了。请使用github图床方式，加入图像的链接

现在github 图床获取到的链接都是 private-user-images.githubusercontent.com 域名的带过期jwt的图片链接，不适合用于文档，请问是否可以使用第三方图床工具

把图像拽到这个PR下的comment栏中，会自动生成一条 url，用这个url就可以了

[lvhan028]

linting的错误还在：

pip install pre-commit
cd lmdeploy
pre-commit install .
pre-commit run --all-files

用上面这条命令，能够检查commit之前所有的文件格式规范上的错误，可以在本地修改好一并提交

[lvhan028]

May improve docstring coverage of qos_engine module

[lvhan028]

May use key=value to initiate UsageStats in order to clarify what 60, 6, 0 are

[awslshadowstar]

okk

[lvhan028]

When will stop be called?

[awslshadowstar]

it will be deleted

[lvhan028]

self.lock is defined but not used

[Dofgal]

It'll be deleted

[AllentDan]

LGTM after left comments resolved.