Question about expert_mask

[xiaojunjie]

System Info / 系統信息

According to Data Building

• input_ids contains：bos + boi + image + eoi + text
• token_type_ids contains: L V ... V L L ..L,
  where the quantity of V equals vision_token_num + 2

According to the function get_expert_mask， boi + image will be fed to vision expert.
My question: Why not boi + image + eoi or only image ?

Who can help? / 谁可以帮助到您？

@Sleepychord @cenyk1230 @Btlmd @huangshiyu13 @1049451037

Information / 问题信息

• The official example scripts / 官方的示例脚本
• My own modified scripts / 我自己修改的脚本和任务

Reproduction / 复现过程

.

Expected behavior / 期待表现

..............

[wsxwd]

did you got it？