[Python] [Pytesseract] [Urdu] [Segmentation fault] [Deserialize header failed]

[IrtazaIjaz]

Hi All,

I'm having trouble executing the fine-tunning on this repository. Below is my code which I run on my Jupyter notebook:

**Step1:**
!git clone https://github.com/tesseract-ocr/tesstrain.git

Step-2:
%cd tesstrain
!make tesseract-langdata

**Step-3:**
import zipfile
with zipfile.ZipFile('/content/tesstrain/irt-ground-truth.zip', 'r') as zip_ref:
    zip_ref.extractall('/content/tesstrain/data')

**Step-4:**
# Create the directory 'usr/share/tessdata'
!mkdir -p usr/share/tessdata

# Download the trained data file and save it to 'usr/share/tessdata'
!wget -P usr/share/tessdata https://github.com/tesseract-ocr/tessdata_best/raw/main/urd.traineddata

**Step-5:**
!pip install Pillow>=6.2.1
!pip install python-bidi>=0.4
!pip install matplotlib
!pip install pandas
!pip install pytesseract
!apt-get install tesseract-ocr-urd
!apt-get install tesseract-ocr
!make leptonica tesseract

Step-6:
I have replaced /content/tesstrain/data/irt/list.train folder with my file which contains below text:

/content/tesstrain/data/irt-ground-truth/page_10_line_1.png نقش فریادی ہے کس کی شوخیٔ تحریر کا
/content/tesstrain/data/irt-ground-truth/page_10_line_2.png کاغذی ہے پیرہن ہر پیکر تصویر کا
/content/tesstrain/data/irt-ground-truth/page_10_line_3.png کاو کاو سخت جانی ہائے تنہائی نہ پوچھ
/content/tesstrain/data/irt-ground-truth/page_10_line_4.png صبح کرنا شام کا لانا ہے جوئے شیر کا
/content/tesstrain/data/irt-ground-truth/page_10_line_5.png جذبۂ بے اختیار شوق دیکھا چاہیے
/content/tesstrain/data/irt-ground-truth/page_10_line_6.png سینۂ شمشیر سے باہر ہے دم شمشیر کا
/content/tesstrain/data/irt-ground-truth/page_10_line_7.png آگہی دام شنیدن جس قدر چاہے بچھائے
/content/tesstrain/data/irt-ground-truth/page_10_line_8.png مدعا عنقا ہے اپنے عالم تقریر کا
/content/tesstrain/data/irt-ground-truth/page_10_line_9.png نبسکہ ہوں غالبؔ اسیری میں بھی آتش زیر پا
/content/tesstrain/data/irt-ground-truth/page_10_line_10.png موئے آتش دیدہ ہے حلقہ مری زنجیر کا

**Step-7:**
# Giving Read/Write rights on tesstrain folder

import os
import subprocess
folder_path = '/content/tesstrain'

# Define the chmod command as a list of arguments
chmod_command = ['chmod', '-R', '777', folder_path]

# Execute the chmod command
try:
    subprocess.run(chmod_command, check=True)
    print(f"Permissions changed for {folder_path}")
except subprocess.CalledProcessError as e:
    print(f"Error: {e}")

Step8:
# /content/tesstrain Path to run the below code
!make training MODEL_NAME=irt START_MODEL=urd FINETUNE_TYPE=Impact

Step8 OutCome:
You are using make version: 4.3
lstmtraining
--debug_interval 0
--traineddata data/irt/irt.traineddata
--old_traineddata /content/tesstrain/usr/share/tessdata/urd.traineddata
--continue_from data/urd/irt.lstm
--learning_rate 0.0001
--model_output data/irt/checkpoints/irt
--train_listfile data/irt/list.train
--eval_listfile data/irt/list.eval
--max_iterations 10000
--target_error_rate 0.01
Loaded file data/urd/irt.lstm, unpacking...
Warning: LSTMTrainer deserialized an LSTMRecognizer!
Code range changed from 129 to 129!
Num (Extended) outputs,weights in Series:
1,48,0,1:1, 0
Num (Extended) outputs,weights in Series:
C3,3:9, 0
Ft16:16, 160
Total weights = 160
[C3,3Ft16]:16, 160
Mp3,3:16, 0
Lfys64:64, 20736
Lfx96:96, 61824
Lrx96:96, 74112
Lfx384:384, 738816
Fc129:129, 49665
Total weights = 945313
Previous null char=2 mapped to 128
Continuing from data/urd/irt.lstm
Deserialize header failed: /content/tesstrain/data/irt-ground-truth/page_10_line_1.png نقش فریادی ہے کس کی شوخیٔ تحریر کا
Deserialize header failed: /content/tesstrain/data/irt-ground-truth/page_10_line_2.png کاغذی ہے پیرہن ہر پیکر تصویر کا
Deserialize header failed: /content/tesstrain/data/irt-ground-truth/page_10_line_5.png جذبۂ بے اختیار شوق دیکھا چاہیے
Load of page 0 failed!
Load of images failed!!
make: *** [Makefile:327: data/irt/checkpoints/irt_checkpoint] Segmentation fault (core dumped)

Please help me how to proceed further. I'm stuck.

Thanks you

[stefan6419846]

How is this related to Python and pytesseract? By the way: GitHub allows formatting code sections as code to improve readability (just use the <> button after marking the corresponding lines).

[zdenop]

Also, it seems you try to run training on some platform (kaggle?) - run it on your local computer Linux/WSL or Mac.
Next do not report problems with your data - first, make sure that example data training works (e.g. you install and set training env correctly )

[IrtazaIjaz]

Hi @zdenop,

I'm running it on Jupyter Notebook. I started with a single page that contained 10 lines only.

[IrtazaIjaz]

Hi @stefan6419846,

I'm working on Jupyter notebook for python and writing the code in it. Moreover, I have also made the code more readable as you suggested.

Thanks

[zdenop]

Follow readme instruction - only supported training process. Jupyter notebook is not there.
Otherwise you will not get support and issue will be closed.