Unable to export to torchscript and int8 model

[vishnukarthik-1994]

Hi,

When I try to export my trained model into int8 by calling convert_and_export_predictor_method, I run into the following issue: TypeError: 'bool' object is not iterable

Reference code:

`import copy
from detectron2.data import build_detection_test_loader
from d2go.export.api import convert_and_export_predictor
from d2go.export.d2_meta_arch import patch_d2_meta_arch

import logging

previous_level = logging.root.manager.disable
logging.disable(logging.INFO)

patch_d2_meta_arch()

pytorch_model = runner.build_model(cfg, eval_only=True)
pytorch_model.cpu()

datasets = cfg.DATASETS.TRAIN[0]
data_loader = runner.build_detection_test_loader(cfg, datasets)

predictor_path = convert_and_export_predictor(
copy.deepcopy(cfg),
copy.deepcopy(pytorch_model),
"torchscript_int8@tracing",
'./',
data_loader
)

logging.disable(previous_level)`

Issue seen:

File ~\anaconda3\envs\capstone_3\lib\site-packages\torch\ao\quantization\fuser_method_mappings.py:129, in get_fuser_method(op_list, additional_fuser_method_mapping)
127 if additional_fuser_method_mapping is None:
128 additional_fuser_method_mapping = dict()
--> 129 all_mappings = get_combined_dict(DEFAULT_OP_LIST_TO_FUSER_METHOD,
130 additional_fuser_method_mapping)
131 fuser_method = all_mappings.get(op_list, None)
132 assert fuser_method is not None, "did not find fuser method for: {} ".format(op_list)

File ~\anaconda3\envs\capstone_3\lib\site-packages\torch\ao\quantization\utils.py:12, in get_combined_dict(default_dict, additional_dict)
10 def get_combined_dict(default_dict, additional_dict):
11 d = default_dict.copy()
---> 12 d.update(additional_dict)
13 return d

TypeError: 'bool' object is not iterable

Before running into this issue, I had also faced another problem in fuse_utils.py which I overcame by making this change:

    if fuser_method is None or mod_list[0].training:
        #return fuse_known_modules(mod_list, is_qat=is_qat) -> old code
        return fuse_known_modules(mod_list, is_qat)

After this change, I run into the bool object not iterable issue.

My analysis:

1. fuse_known_modules(mod_list, is_qat) -> here is_qat is false

2. def fuse_known_modules(mod_list, additional_fuser_method_mapping=None):

3. fuser_method = get_fuser_method(types, additional_fuser_method_mapping)

4. additional_fuser_method_mapping is false now as it is is_qat from the call stack.

5. get_fuser_method calls this function:
   all_mappings = get_combined_dict(DEFAULT_OP_LIST_TO_FUSER_METHOD,
   additional_fuser_method_mapping)

6. def get_combined_dict(default_dict, additional_dict):
   d = default_dict.copy()
   d.update(additional_dict)
   return d

Since additional_dict is additional_fuser_method_mapping which is false (a boolean), I ran into this problem.

[zhanghang1989]

Could you use the command at https://github.com/facebookresearch/d2go/tree/main/demo#export-to-torchscript--int8-model

[wat3rBro]

Please pull the latest mobile-vision repo as it has the latest fuse_utils.py, it should solve the issue.

[yuzhuhua]

您可以使用https://github.com/facebookresearch/d2go/tree/main/demo#export-to-torchscript--int8-model上的命令吗？

I used the above command, but because I was using a custom data set, I kept getting errors. KeyError showed an unregistered data set.