Skip to content

Commit

Permalink
Update ascend readme (#2756)
Browse files Browse the repository at this point in the history
* Update get_started.md of ascend

* Update get_started.md of ascend
  • Loading branch information
jinminxi104 authored Nov 14, 2024
1 parent fd8906c commit 59c1c63
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 4 deletions.
4 changes: 2 additions & 2 deletions docs/en/get_started/ascend/get_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ For more information about running the Docker client on Ascend devices, please r
## Offline batch inference

> \[!TIP\]
> Graph mode has been supported on Atlas 800T A2. Currently, LLaMa3-8B/LLaMa2-7B/Qwen2-7B are tested on graph mode.
> Graph mode has been supported on Atlas 800T A2.
> Users can set `eager_mode=False` to enable graph mode, or, set `eager_mode=True` to disable graph mode.
> (Please source `/usr/local/Ascend/nnal/atb/set_env.sh` before enabling graph mode)
Expand Down Expand Up @@ -86,7 +86,7 @@ if __name__ == "__main__":
## Online serving

> \[!TIP\]
> Graph mode has been supported on Atlas 800T A2. Currently, InternLM2-7B/LLaMa2-7B/Qwen2-7B are tested on graph mode.
> Graph mode has been supported on Atlas 800T A2.
> Graph mode is default enabled in online serving. Users can add `--eager-mode` to disable graph mode.
> (Please source `/usr/local/Ascend/nnal/atb/set_env.sh` before enabling graph mode)
Expand Down
4 changes: 2 additions & 2 deletions docs/zh_cn/get_started/ascend/get_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ docker run -e ASCEND_VISIBLE_DEVICES=0 --rm --name lmdeploy -t lmdeploy-aarch64-
## 离线批处理

> \[!TIP\]
> 图模式已经支持了Atlas 800T A2。目前,单卡下的LLaMa3-8B/LLaMa2-7B/Qwen2-7B已经通过测试。用户可以设定`eager_mode=False`来开启图模式,或者设定`eager_mode=True`来关闭图模式。(启动图模式需要事先source `/usr/local/Ascend/nnal/atb/set_env.sh`)
> 图模式已经支持了Atlas 800T A2。用户可以设定`eager_mode=False`来开启图模式,或者设定`eager_mode=True`来关闭图模式。(启动图模式需要事先source `/usr/local/Ascend/nnal/atb/set_env.sh`)
### LLM 推理

Expand Down Expand Up @@ -84,7 +84,7 @@ if __name__ == "__main__":
## 在线服务

> \[!TIP\]
> 图模式已经支持Atlas 800T A2。目前,单卡下的InternLM2-7B/LLaMa2-7B/Qwen2-7B已经通过测试。
> 图模式已经支持Atlas 800T A2。
> 在线服务时,图模式默认开启,用户可以添加`--eager-mode`来关闭图模式。(启动图模式需要事先source `/usr/local/Ascend/nnal/atb/set_env.sh`)
### LLM 模型服务
Expand Down

0 comments on commit 59c1c63

Please sign in to comment.