Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update vi_layoutxlm_ser to load ckpt, fix bug in predict_ser #24

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

hongziqi
Copy link
Collaborator

Thank you for your contribution to the MindOCR repo.
Before submitting this PR, please make sure:

Motivation

  1. update register_model: vi_layoutxlm_ser to use pretrain ckpt
  2. fix bug by inferring ser in layoutxlm_ser:
    [ERROR] ANALYZER(2921322,ffffa6185010,python):2024-12-30-08:59:24.113.844 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/evaluator.cc:713] Run] Primitive: <StandardPrimEvaluator_list_getitem> infer failed, failed info: list_getitem evaluator index should be in range[-4, 4), but got 4.

Before Update(would not load ckpt in mindocr/mindocr/models/kie_layoutxlm.py

  1. infer with vi_layoutxlm:
python tools/infer/text/predict_ser.py --rec_algorithm CRNN_CH --det_algorithm DB_PPOCRv3 --image_dir configs/kie/vi_layoutxlm/example.jpg

image

  1. infer with layoutxlm, the following bugs have occurred:
python tools/infer/text/predict_ser.py --rec_algorithm CRNN_CH --det_algorithm DB_PPOCRv3 --image_dir configs/kie/vi_layoutxlm/example.jpg --ser_algorithm LAYOUTXLM

image

After Update(use the best ckpt link)

  1. infer with vi_layoutxlm:
python tools/infer/text/predict_ser.py --rec_algorithm CRNN_CH --det_algorithm DB_PPOCRv3 --image_dir configs/kie/vi_layoutxlm/example.jpg

image

  1. infer with layoutxlm:
python tools/infer/text/predict_ser.py --rec_algorithm CRNN_CH --det_algorithm DB_PPOCRv3 --image_dir configs/kie/vi_layoutxlm/example.jpg --ser_algorithm LAYOUTXLM

image

  1. eval with vi_layoutxlm:
python tools/eval.py --config configs/kie/vi_layoutxlm/ser_vi_layoutxlm_xfund_zh.yaml

image

  1. eval with layoutxlm:
python tools/eval.py --config configs/kie/layoutxlm/ser_layoutxlm_xfund_zh.yaml

image

  1. train with vi_layoutxlm:
python tools/train.py --config configs/kie/vi_layoutxlm/ser_vi_layoutxlm_xfund_zh.yaml

image
image

  1. train with layoutxlm:
python tools/train.py --config configs/kie/layoutxlm/ser_layoutxlm_xfund_zh.yaml

image
image

Test Plan

test for

# download mindocr
git clone -b feat-update-kie-ckpt-link https://github.com/hongziqi/mindocr.git
# build mindocr wheel package and install mindocr
python setup.py bdist_wheel
pip install dist/mindocr-0.4.0-py3-none-any.whl

# infer with VI_LAYOUTXLM
python tools/infer/text/predict_ser.py --rec_algorithm CRNN_CH --det_algorithm DB_PPOCRv3 --image_dir configs/kie/vi_layoutxlm/example.jpg

# infer with LAYOUTXLM
python tools/infer/text/predict_ser.py --rec_algorithm CRNN_CH --det_algorithm DB_PPOCRv3 --image_dir configs/kie/vi_layoutxlm/example.jpg --ser_algorithm LAYOUTXLM

# download dataset
wget https://download.mindspore.cn/toolkits/mindocr/vi-layoutxlm/XFUND.tar && tar -xf XFUND.tar
# train or eval (need to update the path in yaml, please refer to the file: configs/kie/vi_layoutxlm/README_CN.md)
python tools/train.py(eval.py) --config configs/kie/vi_layoutxlm/ser_vi_layoutxlm_xfund_zh.yaml
python tools/train.py(eval.py) --config configs/kie/layoutxlm/ser_layoutxlm_xfund_zh.yaml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant