Skip to content

Commit

Permalink
Bugfix: wrongly assign gen_config with True (#1594)
Browse files Browse the repository at this point in the history
* Bugfix: fix bugs with error info:"
 if gen_config.stop_words is None:

AttributeError: 'bool' object has no attribute 'stop_words'",which is caused by assign gen_config with value 'True'.

* Style: Fix flake8 line length issue
  • Loading branch information
thelongestusernameofall authored May 16, 2024
1 parent cff5634 commit ec1a39e
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions lmdeploy/serve/qos_engine/qos_engine.py
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ async def generate(self, request):
result_generator = self.engine.generate(
request.prompt[i],
request.session_id + i,
True, # always use stream to enable batching
stream_response=True, # always use stream for batching
sequence_start=True,
sequence_end=True,
request_output_len=request.max_tokens
Expand Down Expand Up @@ -119,7 +119,7 @@ async def generate(self, request):
result_generator = self.engine.generate(
request.messages,
request.session_id,
True, # always use stream to enable batching
stream_response=True, # always use stream to enable batching
sequence_start=True,
sequence_end=True,
request_output_len=request.max_tokens
Expand Down

0 comments on commit ec1a39e

Please sign in to comment.