[Question]: vllm-tp generate can't stop

[unicorneeee]

Describe the issue

I hope this message find you well.
when i use the hjiang branch to use Qwen2.5-7B-instruct, it's running on the InfinitBench. The issue is as follow:
{"id": 0, "prediction": " The pass key is 71432. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass is green. The sun is yellow. The sky is blue. The grass", "ground_truth": ["71432"]}
Although it answer the question correctly and it will be judge as True because the compute_score in Infinitbench will strip the unusful tokens, however I print the all generate tokens, there isn't the <im_end> token, which means there may be some mistakes?

[unicorneeee]

I use the vllm-tp with minference in A100, and the case come from the task passkey in Infinitbench

[iofu728]

Hi @unicorneeee, this issue seems to be related to the inputs dataset. The passkey in InfiniteBench contains duplicate entries in the dataset.