Code

Refactor torch inference engine#871

Code

Merged

lvhan028 merged 68 commits intomainInternLM/lmdeploy:mainfrom pytorch-pocInternLM/lmdeploy:pytorch-pocCopy head branch name to clipboardDec 28, 2023

+15,562-303

Commits

Commits on Aug 17, 2023

• WIP

  grimoire

  committed
  4f0b674

Commits on Aug 18, 2023

• cache engine wip

  grimoire

  committed
  3a4fabc

Commits on Aug 20, 2023

• finish cache engine

  grimoire

  committed
  4d8383b

Commits on Aug 21, 2023

• fix cache and scheduler

  grimoire

  committed
  b56b2cc

Commits on Aug 22, 2023

• add paged attention

  grimoire

  committed
  1266bd9

• step and stop

  grimoire

  committed
  244f1f9

• add infer

  grimoire

  committed
  0a359e4

Commits on Aug 23, 2023

• add request process

  grimoire

  committed
  f55da8f

Commits on Aug 24, 2023

• fix end

  grimoire

  committed
  887e449

• request without schedulersession

  grimoire

  committed
  06eb7a9

Commits on Aug 29, 2023

• add logits processor

  grimoire

  committed
  35b70c3

• better context

  grimoire

  committed
  8d403ba

Commits on Sep 5, 2023

• update patch

  grimoire

  committed
  8f1be2c

• [Improve] Use 4d input in pytorch poc (#371)

  Show description for 105070f
  wangruohui

  authored
  105070f

Commits on Sep 6, 2023

• solve conflict

  grimoire

  committed
  fb894d0

• Merge branch 'main' into pytorch-poc

  AllentDan

  committed
  cbb5a6c

• tp wip

  grimoire

  committed
  8bca5a4

Commits on Sep 7, 2023

• almost

  grimoire

  committed
  1f68f05

• Merge branch 'pytorch-poc' of github.com:InternLM/lmdeploy into pytorch-poc

  grimoire

  committed
  3e15e36

• update logger

  grimoire

  committed
  046249b

• run_check=false

  grimoire

  committed
  8d0355b

Commits on Sep 13, 2023

• little optimize

  Show description for e57fe5d
  grimoire

  committed
  e57fe5d

• share attention forward

  grimoire

  committed
  a9f165b

• fix end

  grimoire

  committed
  4333c81

Commits on Sep 14, 2023

• Support Baichuan (#382)

  Show description for bd3e26e
  AllentDan

  authored
  bd3e26e

• Move q_seq_info into context (#398)

  Show description for a3fbe93
  wangruohui

  authored
  a3fbe93

• merge upstream

  grimoire

  committed
  7d1b213

Commits on Sep 15, 2023

• alibi wip

  grimoire

  committed
  0489a4f

Commits on Sep 19, 2023

• add alibi

  grimoire

  committed
  e533d5d

Commits on Sep 20, 2023

• reduce logic block (#435)

  wangruohui

  authored
  6eb6aa1

• add docstring

  grimoire

  committed
  1231302

• add baichuan lint (#445)

  AllentDan

  authored
  b478b31

Commits on Sep 21, 2023

• add fill cache back

  grimoire

  committed
  18768d2

Commits on Sep 22, 2023

• support internlm

  grimoire

  committed
  6252b99

• fix path of weight index

  grimoire

  committed
  cdbea77

Commits on Sep 25, 2023

• Support chatglm2 in pytorch_poc (#360)

  Show description for 8123c8e
  wangruohui

  authored
  8123c8e

Commits on Oct 9, 2023

• rename modules (#504)

  Show description for 8085fbc
  

  q.yao

  and

  grimoire

  authored
  8085fbc

• optimize fill kv cache (#523)

  Show description for 668e30d
  

  q.yao

  authored
  668e30d

• Make trust_remote_code as cli argument (#434)

  Show description for bf7008a
  wangruohui

  authored
  bf7008a

Commits on Oct 10, 2023

• update tokenizer

  grimoire

  committed
  fd25cdc

Commits on Oct 11, 2023

• optimize rotary

  grimoire

  committed
  373ff4d

Commits on Oct 12, 2023

• wtf

  grimoire

  committed
  ec9ba00

Commits on Oct 17, 2023

• merge main

  grimoire

  committed
  bd5ca4c

Commits on Oct 18, 2023

• Support Falcon models (#406)

  Show description for 0c8e633
  wangruohui

  authored
  0c8e633

• fix batch>2

  grimoire

  committed
  1bc99a1

Commits on Oct 19, 2023

• Refactor scheduler (#551)

  Show description for 7d76feb
  

  q.yao

  authored
  7d76feb

Commits on Oct 20, 2023

• profile pytorch poc (#455)

  Show description for 21a6657
  AllentDan

  authored
  21a6657

Commits on Oct 25, 2023

• Support Tensor parallel on Falcon models (#582)

  Show description for c2cc49e
  wangruohui

  authored
  c2cc49e

Commits on Oct 26, 2023

• fix remain len

  grimoire

  committed
  8595cbd

Commits on Oct 27, 2023

• Apply rotary kernel (#572)

  Show description for 2ba8867
  

  q.yao

  and

  grimoire

  authored
  2ba8867

Commits on Oct 30, 2023

• fix(pytorch_poc): memory cal (#606)

  Show description for ef11f5a
  tpoisonooo

  authored
  ef11f5a

Commits on Oct 31, 2023

• Optimize attention (#597)

  Show description for f8b1afd
  grimoire

  authored
  f8b1afd

Commits on Nov 8, 2023

• feat(pytorch_poc): implement ReRoPE (#625)

  Show description for 3618c0d
  tpoisonooo

  authored
  3618c0d

Commits on Nov 10, 2023

• Refactor engine (#623)

  Show description for fe0d957
  grimoire

  authored
  fe0d957

• [Feature] w8a8 based on pytorch poc (#595)

  Show description for 8c5f6bb
  

  HIT-cwh

  and

  grimoire

  authored
  8c5f6bb

Commits on Nov 27, 2023

• feat(lmdeploy): add rerope quantization (#718)

  Show description for b1ff483
  tpoisonooo

  authored
  b1ff483

Commits on Nov 29, 2023

• [Refactor & Doc] Improve w8a8 and add docstring (#768)

  Show description for 6efbd63
  HIT-cwh

  authored
  6efbd63

Commits on Dec 11, 2023

• rename pytorch poc (#764)

  Show description for 99be323
  grimoire

  authored
  99be323

Commits on Dec 18, 2023

• Unified paging (#860)

  Show description for 32cc6ce
  

  16 peopleauthored
  32cc6ce

Commits on Dec 19, 2023

• [Fix] Adapt to the pyTorch poc branch (#863)

  Show description for 1e8fd4f
  HIT-cwh

  authored
  1e8fd4f

• merge main

  grimoire

  committed
  38dac46

• update model

  grimoire

  committed
  4cdc357

Commits on Dec 21, 2023

• update benchmark

  grimoire

  committed
  10a8912

• [Fix] Fix conflicts in lite (#878)

  Show description for 32bc114
  

  HIT-cwh

  and

  pppppM

  authored
  32bc114

Commits on Dec 22, 2023

• merge main

  grimoire

  committed
  afa328b

Commits on Dec 26, 2023

• [Feature] Support w8a8 tp (#888)

  Show description for ed85b00
  HIT-cwh

  authored
  ed85b00

Commits on Dec 27, 2023

• fix message step update

  grimoire

  committed
  37708a7

Commits on Dec 28, 2023

• update docs

  grimoire

  committed
  fed6e8e