Skip to content

Latest commit

 

History

History
84 lines (56 loc) · 5.46 KB

README.eng.md

File metadata and controls

84 lines (56 loc) · 5.46 KB
Raw

HED-tutorial-for-document-scanning

Code for blog 手机端运行卷积神经网络实现文档检测功能(二) -- 从 VGG 到 MobileNetV2 知识梳理

get code

git clone https://github.com/fengjian0106/hed-tutorial-for-document-scanning

how to run

1 Prepare image resources, synthesize training samples

1.1 background image is downloaded to the ./sample_images/background_images directory.

1.2 The foreground image is downloaded to the ./sample_images/rect_images directory.

2 Synthesize training samples using the iPhone simulator

2.1 Open the ./generate_training_dataset/generate_training_dataset.xcodeproj project.

  • Check the loadImagePaths function of ViewController.m: make sure that self.backgroundImagesPath and self.rectImagesPath point to the directories mentioned in 1.1 and 1.2 respectively, then run The program (_dataset.xcodeproj project) by pressing Run button on the xcode. Choose an iphone simulator to run this code (Please dont run this code on apple device) , and based on the printed log information, find the directory corresponding to self.imageSaveFolder on the Mac, the generated sample image will be saved in this directory.

  • Basically you will observe two types of images : image1_color.jpg, image1_annotation.png. image2_color.jpg, image2_annotation.png ... etc. (I have attached few images in >> dataset->generate_sample_by_ios_image_size_256_256_thickness_0.2 (folder) for my reference)

2.2 Move all the images generated in 2.1 to the ./dataset/generate_sample_by_ios_image_size_256_256_thickness_0.2 directory.

2.3 The white rectangle border drawn on the UIView is smoothed. The white Point corresponds to a pixel value other than 255. Therefore, you need to binarize these white Points and run the following program. :

python preprocess_generate_training_dataset.py \
			--dataset_root_dir dataset \
			--dataset_folder_name generate_sample_by_ios_image_size_256_256_thickness_0.2

After this program is executed, you will get the ./dataset/generate_sample_by_ios_image_size_256_256_thickness_0.2.csv file.

2.4 Use the gshuf tool to randomly mess up the contents of the ./dataset/generate_sample_by_ios_image_size_256_256_thickness_0.2.csv file and execute the following command:

gshuf ./dataset/generate_sample_by_ios_image_size_256_256_thickness_0.2.csv > ./dataset/temp.txt
gshuf ./dataset/temp.txt > ./dataset/generate_sample_by_ios_image_size_256_256_thickness_0.2.csv

After performing this step, a batch of synthetic training sample images is obtained.

The process of preparing the training samples should be customized according to the specific needs. Here is just a reference method.

3 Training Network

Run the following program:

python train_hed.py --dataset_root_dir dataset \
                    --csv_path dataset/generate_sample_by_ios_image_size_256_256_thickness_0.2.csv \
                    --display_step 5

After the training is done you get additional two types of images in the dataset->generate_sample_by_ios_image_size_256_256_thickness_0.2 folder these new images have names image1_annotation_gray.png and image1_annotation_thresh_gray.png. image2_annotation_gray.png and image2_annotation_thresh_gray.png ... etc

4 Testing HED networks in a python environment

Run the following program to process a picture:

python evaluate_hed.py --checkpoint_dir checkpoint \
                       --image test_image/test27.jpg \
                       --output_dir test_image

5 Run a complete process in the iPhone real-world environment, including running the HED network and executing an OpenCV-based point-finding algorithm

5.1 Export the model file in pb format and run the following program:

python freeze_model.py --checkpoint_dir checkpoint

After running successfully, you can see a model file named hed_graph.pb in the ./checkpoint directory, which will be loaded in the iOS program.

5.2 Running the iOS demo program

./ios_demo/DemoWithStaticLib/DemoWithStaticLib.xcodeproj is a demo program that contains compiled static libraries of various dependencies and can be run directly. There is a complete process in the demo. The first step is to call the HED network to get the edge detection map. The second step is to execute the algorithm for finding the quadrilateral vertices.

5.3 Compile FMHEDNet static library

./ios_demo/FMHEDNet/FMHEDNet.xcodeproj is a static library project that encapsulates the call to the HED network to avoid introducing TensorFlow source files into the project files of the business layer app. If you want to compile this FMHEDNet static library, you need to compile TensorFlow Mobile first. For how to compile TensorFlow Mobile, please see 5.4 below. For the process of compiling FMHEDNet, please see [here] (https://github.com/fengjian0106/hed-tutorial-for-document-scanning/blob/master/ios_demo/FMHEDNet/FMHEDNet/FMHEDNet.mm).

5.4 Compile TensorFlow Mobile TensorFlow's [official documentation] (https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/makefile) has instructions for compiling. I am using a manually cropped version and have modified the namespace in the Protobuf source. For details, see [here] (https://github.com/fengjian0106/hed-tutorial-for-document-scanning/blob/master /how_to_build_tensorflow_and_change_namespace_of_protobuf.txt).