diff --git a/CITATION.cff b/CITATION.cff new file mode 100644 index 00000000..cd9c47bf --- /dev/null +++ b/CITATION.cff @@ -0,0 +1,38 @@ +# This CITATION.cff file was generated with cffinit. +# Visit https://bit.ly/cffinit to generate yours today! + +cff-version: 1.2.0 +title: Hailo Model Zoo +message: >- + If you use Hailo in your research, please cite this + repository using these metadata. +type: software +authors: + - given-names: Tamir + family-names: Tapuhi + email: tamirt@hailo.ai + - given-names: Shachar + family-names: Gluska + email: shacharg@hailo.ai + - given-names: Amit + family-names: Klinger + email: amitk@hailo.ai + - given-names: Nadiv + family-names: Dharan + email: nadivd@hailo.ai + - given-names: Omer + family-names: Sholev + email: omersh@hailo.ai + - given-names: Yuval + family-names: Zelig + email: yuvalz@hailo.ai + - given-names: Niv + family-names: Vosco + email: nivv@hailo.ai +repository-code: 'https://github.com/hailo-ai/hailo_model_zoo' +url: 'https://hailo.ai/products/hailo-software-suite/model-zoo/' +abstract: >- + The Hailo Model Zoo includes pre-trained models and a full + building and evaluation environment +license: MIT +date-released: '2021-07-01' diff --git a/README.rst b/README.rst index 27a4c533..348e9d6f 100644 --- a/README.rst +++ b/README.rst @@ -1,45 +1,51 @@ Hailo Model Zoo =============== -.. |python| image:: https://img.shields.io/badge/python-3.8-blue.svg +.. |python| image:: https://img.shields.io/badge/python-3.8%20%7C%203.9%20%7C%203.10-blue.svg :target: https://www.python.org/downloads/release/python-380/ :alt: Python 3.8 - :width: 70 + :width: 150 + :height: 20 .. |tensorflow| image:: https://img.shields.io/badge/Tensorflow-2.9.2-blue.svg :target: https://github.com/tensorflow/tensorflow/releases/tag/v2.9.2 - :alt: Generic badge - :width: 90 + :alt: Tensorflow + :width: 110 + :height: 20 .. |cuda| image:: https://img.shields.io/badge/CUDA-11.2-blue.svg :target: https://developer.nvidia.com/cuda-toolkit - :alt: Generic badge - :width: 70 + :alt: Cuda + :width: 80 + :height: 20 -.. |compiler| image:: https://img.shields.io/badge/HailoDataflow;Compiler-3.22.1-.svg +.. |compiler| image:: https://img.shields.io/badge/Hailo%20Dataflow%20Compiler-3.23.0-brightgreen.svg :target: https://hailo.ai/contact-us/ - :alt: Generic badge + :alt: Hailo Dataflow Compiler :width: 180 + :height: 20 -.. |badge| image:: https://img.shields.io/badge/(optional)HailoRT-4.12.1-.svg +.. |runtime| image:: https://img.shields.io/badge/HailoRT%20(optional)-4.13.0-brightgreen.svg :target: https://hailo.ai/contact-us/ - :alt: Generic badge - :width: 100 + :alt: HailoRT + :width: 170 + :height: 20 .. |license| image:: https://img.shields.io/badge/License-MIT-yellow.svg :target: https://github.com/hailo-ai/hailo_model_zoo/blob/master/LICENSE :alt: License: MIT - :width: 70 + :width: 80 + :height: 20 .. image:: docs/images/logo.png -|python| |tensorflow| |cuda| |compiler| |badge| |license| +|python| |tensorflow| |cuda| |compiler| |runtime| |license| The Hailo Model Zoo provides pre-trained models for high-performance deep learning applications. Using the Hailo Model Zoo you can measure the full precision accuracy of each model, the quantized accuracy using the Hailo Emulator and measure the accuracy on the Hailo-8 device. Finally, you will be able to generate the Hailo Executable Format (HEF) binary file to speed-up development and generate high quality applications accelerated with Hailo-8. The Hailo Model Zoo also provides re-training instructions to train the models on custom datasets and models that were trained for specific use-cases on internal datasets. @@ -127,4 +133,4 @@ Please visit `hailo.ai `_ for support / requests / issues. Changelog --------- -For further information please see our `CHANGLOGE `_ page. +For further information please see our `CHANGELOG `_ page. diff --git a/docs/CHANGELOG.rst b/docs/CHANGELOG.rst index b1365b29..dcea620c 100644 --- a/docs/CHANGELOG.rst +++ b/docs/CHANGELOG.rst @@ -1,33 +1,55 @@ Changelog ========= +**v2.7** + +* Update to use Dataflow Compiler v3.23.0 (`developer-zone `_) +* Updated to use HailoRT 4.13.0 (`developer-zone `_) +* Inference flow was moved to new high-level APIs +* New object detection variants: + + * yolov8: yolov8n / yolov8s / yolov8m / yolov8l / yolov8x + * damoyolo: damoyolo_tinynasL20_T / damoyolo_tinynasL25_S / damoyolo_tinynasL35_M + +* New transformers based models: + + * vit_base - classification model + * yolov5s_c3tr - object detection model with a self-attention block + +* Examples for using HailoRT-pp - support for seamless integration of models and their corresponding postprocessing + + * yolov5m_hpp + +* Configuration YAMLs and model-scripts for networks with YUY2 input format +* DAMO-YOLO retraining docker +* Bug fixes + **v2.6.1** * Bug fixes -* Update to use Dataflow Compiler v3.22.1 (`developer-zone `_) -* Updated to use HailoRT 4.12.1 (`developer-zone `_) **v2.6** * Update to use Dataflow Compiler v3.22.0 (`developer-zone `_) * Updated to use HailoRT 4.12.0 (`developer-zone `_) -* ViT (`Vision Transformer `_) - new classification network with transformers-encoder based architecture +* ViT (`Vision Transformer `_) - new classification network with transformers-encoder based architecture * New instance segmentation variants: * yolov5n_seg * yolov5s_seg * yolov5m_seg - + * yolov5l_seg + * New object detecion variants for high resolution images: * yolov7e6 * yolov5n6_6.1 * yolov5s6_6.1 * yolov5m6_6.1 - + * New flag ``--performance`` to reproduce highest performance for a subset of networks * Hailo model-zoo log is now written into ``sdk_virtualenv/etc/hailo/modelzoo/hailo_examples.log`` -* Bug fixes +* Bug fixes **v2.5** @@ -37,8 +59,8 @@ Changelog * yolact_regnetx_800mf * yolact_regnetx_1.6gf - -* Bug fixes + +* Bug fixes **v2.4** @@ -48,8 +70,8 @@ Changelog * New models: * Face Detection - scrfd_500m / scrfd_2.5g / scrfd_10g - -* New tasks: + +* New tasks: 1. Super-Resolution @@ -64,7 +86,7 @@ Changelog #. arcface_mobilefacenet * Retraining docker for arcface architecture -* Added support for new hw-arch - hailo8l +* Added support for new hw-arch - hailo8l **v2.3** @@ -75,7 +97,7 @@ Changelog * yolov7 / yolov7-tiny * nanodet_repvgg_a1_640 * efficientdet_lite0 / efficientdet_lite1 / efficientdet_lite2 - + * New tasks: * mspn_regnetx_800mf - single person pose estimation @@ -94,13 +116,13 @@ Changelog * Hailo model zoo data directory by default will be ``~/.hailomz`` * New models: - + * yolov5xs_wo_spp_nms - a model which contains bbox decoding and confidence thresholding on Hailo-8 * osnet_x1_0 - person ReID network * yolov5m_6.1 - yolov5m network from the latest tag of the repo (6.1) including silu activation * New tasks: - + * person_attr_resnet_v1_18 - person attribute recognition * ReID training docker for the Hailo model repvgg_a0_person_reid_512/2048 @@ -121,7 +143,7 @@ Changelog * | repvgg_a0_person_reid_512 / repvgg_a0_person_reid_2048 - ReID networks which outputs a person embedding | These models were trained in-house as part of our upcoming new application * | stdc1 - Segmentation architecture for Cityscapes - + **v2.0** * Updated to use Dataflow Compiler v3.16 (`developer-zone `_) with TF version 2.5 which require CUDA11.2 @@ -144,7 +166,7 @@ Changelog * Added new documentation to the `YAML structure `_ - + **v1.5** * Remove HailoRT installation dependency. @@ -173,7 +195,7 @@ Changelog * | hand_landmark_lite from MediaPipe * | palm_detection_lite from MediaPipe - + | Both tasks are without evaluation module. @@ -221,7 +243,7 @@ Changelog * Model Optimization API Changes * Model Optimization parameters can be updated using the networks' model script files (\*.alls) - + * Deprecated: quantization params in YAMLs * Training Guide: new training guide for yolov5 with compatible Dockerfile @@ -261,25 +283,25 @@ Changelog **v1.1** -* Support VisDrone Dataset -* New task: pose estimation +* Support VisDrone Dataset +* New task: pose estimation - * centerpose_regnetx_200mf_fpn - * centerpose_regnetx_800mf - * centerpose_regnetx_1.6gf_fpn + * centerpose_regnetx_200mf_fpn + * centerpose_regnetx_800mf + * centerpose_regnetx_1.6gf_fpn -* New task: face detection +* New task: face detection - * lightfaceslim - * retinaface_mobilenet_v1 + * lightfaceslim + * retinaface_mobilenet_v1 -* New models: +* New models: - * hardnet39ds - * hardnet68 - * yolox_tiny_leaky - * yolox_s_leaky - * deeplab_v3_mobilenet_v2 + * hardnet39ds + * hardnet68 + * yolox_tiny_leaky + * yolox_s_leaky + * deeplab_v3_mobilenet_v2 * Use your own network manual for YOLOv3, YOLOv4_leaky and YOLOv5. diff --git a/docs/GETTING_STARTED.rst b/docs/GETTING_STARTED.rst index f7d78751..5515972c 100644 --- a/docs/GETTING_STARTED.rst +++ b/docs/GETTING_STARTED.rst @@ -9,8 +9,8 @@ System Requirements * Ubuntu 20.04/22.04, 64 bit (supported also on Windows, under WSL2) * Python 3.8/3.9/3.10, including ``pip`` and ``virtualenv`` -* Hailo Dataflow Compiler v3.22.1 (Obtain from `hailo.ai `_\ ) -* HailoRT 4.12.1 (Obtain from `hailo.ai `_\ ) - required only for inference on Hailo-8. +* Hailo Dataflow Compiler v3.23.0 (Obtain from `hailo.ai `_\ ) +* HailoRT 4.13.0 (Obtain from `hailo.ai `_\ ) - required only for inference on Hailo-8. * The Hailo Model Zoo supports Hailo-8 connected via PCIe only. * Nvidia’s Pascal/Turing/Ampere GPU architecture (such as Titan X Pascal, GTX 1080 Ti, RTX 2080 Ti, or RTX A4000) * GPU driver version 470 diff --git a/docs/HAILO_MODELS.rst b/docs/HAILO_MODELS.rst index a777ebc3..eb51e24b 100644 --- a/docs/HAILO_MODELS.rst +++ b/docs/HAILO_MODELS.rst @@ -7,6 +7,7 @@ Each model is accompanied with its own README, retraining docker and retraining * FLOPs in the table are counted as MAC operations. +* All models were compiled using Hailo Dataflow Compiler v3.23.0 * Supported tasks: * `Object Detection`_ diff --git a/docs/PUBLIC_MODELS.rst b/docs/PUBLIC_MODELS.rst index 5617979d..588303f6 100644 --- a/docs/PUBLIC_MODELS.rst +++ b/docs/PUBLIC_MODELS.rst @@ -8,15 +8,12 @@ Public Pre-Trained Models .. |star| image:: images/star.png :width: 18 -.. |bomb| image:: images/bomb.png - :width: 18 - Here, we give the full list of publicly pre-trained models supported by the Hailo Model Zoo. * Network available in `Hailo Benchmark `_ are marked with |rocket| * Networks available in `TAPPAS `_ are marked with |star| -* Networks available in Hailo Recommended are marked with |bomb| * Benchmark, TAPPAS and Recommended networks run in performance mode +* All models were compiled using Hailo Dataflow Compiler v3.23.0 * Supported tasks: * `Classification`_ @@ -64,8 +61,8 @@ ImageNet - 9.70 - `link `_ - `link `_ - - `link `_ - * - efficientnet_lite0 |bomb| + - `link `_ + * - efficientnet_lite0 - 74.99 - 73.91 - 224x224x3 @@ -73,8 +70,8 @@ ImageNet - 0.39 - `link `_ - `link `_ - - `link `_ - * - efficientnet_lite1 |bomb| + - `link `_ + * - efficientnet_lite1 - 76.68 - 76.21 - 240x240x3 @@ -82,8 +79,8 @@ ImageNet - 0.61 - `link `_ - `link `_ - - `link `_ - * - efficientnet_lite2 |bomb| + - `link `_ + * - efficientnet_lite2 - 77.45 - 76.74 - 260x260x3 @@ -91,8 +88,8 @@ ImageNet - 0.87 - `link `_ - `link `_ - - `link `_ - * - efficientnet_lite3 |bomb| + - `link `_ + * - efficientnet_lite3 - 79.29 - 78.33 - 280x280x3 @@ -100,8 +97,8 @@ ImageNet - 1.40 - `link `_ - `link `_ - - `link `_ - * - efficientnet_lite4 |bomb| + - `link `_ + * - efficientnet_lite4 - 80.79 - 80.47 - 300x300x3 @@ -109,7 +106,7 @@ ImageNet - 2.58 - `link `_ - `link `_ - - `link `_ + - `link `_ * - efficientnet_m |rocket| - 78.91 - 78.63 @@ -118,7 +115,7 @@ ImageNet - 3.68 - `link `_ - `link `_ - - `link `_ + - `link `_ * - efficientnet_s - 77.64 - 77.32 @@ -127,16 +124,16 @@ ImageNet - 2.36 - `link `_ - `link `_ - - `link `_ + - `link `_ * - hardnet39ds - 73.43 - - 72.23 + - 72.33 - 224x224x3 - 3.48 - 0.43 - `link `_ - `link `_ - - `link `_ + - `link `_ * - hardnet68 - 75.47 - 75.04 @@ -145,16 +142,16 @@ ImageNet - 4.25 - `link `_ - `link `_ - - `link `_ + - `link `_ * - inception_v1 - 69.74 - - 69.3 + - 69.54 - 224x224x3 - 6.62 - 1.50 - `link `_ - `link `_ - - `link `_ + - `link `_ * - mobilenet_v1 - 70.97 - 70.25 @@ -163,7 +160,7 @@ ImageNet - 0.57 - `link `_ - `link `_ - - `link `_ + - `link `_ * - mobilenet_v2_1.0 |rocket| - 71.78 - 71.08 @@ -172,7 +169,7 @@ ImageNet - 0.31 - `link `_ - `link `_ - - `link `_ + - `link `_ * - mobilenet_v2_1.4 - 74.18 - 73.07 @@ -181,7 +178,7 @@ ImageNet - 0.59 - `link `_ - `link `_ - - `link `_ + - `link `_ * - mobilenet_v3 - 72.21 - 71.73 @@ -190,7 +187,7 @@ ImageNet - 1.00 - `link `_ - `link `_ - - `link `_ + - `link `_ * - mobilenet_v3_large_minimalistic - 72.11 - 71.07 @@ -199,8 +196,8 @@ ImageNet - 0.21 - `link `_ - `link `_ - - `link `_ - * - regnetx_1.6gf |bomb| + - `link `_ + * - regnetx_1.6gf - 77.05 - 76.75 - 224x224x3 @@ -208,8 +205,8 @@ ImageNet - 1.61 - `link `_ - `link `_ - - `link `_ - * - regnetx_800mf |bomb| + - `link `_ + * - regnetx_800mf - 75.16 - 74.84 - 224x224x3 @@ -217,7 +214,7 @@ ImageNet - 0.80 - `link `_ - `link `_ - - `link `_ + - `link `_ * - regnety_200mf - 70.38 - 69.91 @@ -226,25 +223,25 @@ ImageNet - 0.20 - `link `_ - `link `_ - - `link `_ - * - resmlp12_relu |bomb| + - `link `_ + * - resmlp12_relu - 75.26 - - 74.06 + - 74.16 - 224x224x3 - 15.77 - 3.02 - `link `_ - `link `_ - - `link `_ + - `link `_ * - resnet_v1_18 - 71.26 - - 70.64 + - 71.06 - 224x224x3 - 11.68 - 1.82 - `link `_ - `link `_ - - `link `_ + - `link `_ * - resnet_v1_34 - 72.7 - 72.14 @@ -253,7 +250,7 @@ ImageNet - 3.67 - `link `_ - `link `_ - - `link `_ + - `link `_ * - resnet_v1_50 |rocket| |star| - 75.12 - 74.47 @@ -262,7 +259,7 @@ ImageNet - 3.49 - `link `_ - `link `_ - - `link `_ + - `link `_ * - resnet_v2_18 - 69.57 - 69.1 @@ -271,7 +268,7 @@ ImageNet - 1.82 - `link `_ - `link `_ - - `link `_ + - `link `_ * - resnet_v2_34 - 73.07 - 72.72 @@ -280,7 +277,7 @@ ImageNet - 3.67 - `link `_ - `link `_ - - `link `_ + - `link `_ * - resnext26_32x4d - 76.18 - 75.78 @@ -289,7 +286,7 @@ ImageNet - 2.48 - `link `_ - `link `_ - - `link `_ + - `link `_ * - resnext50_32x4d - 79.31 - 78.39 @@ -298,16 +295,16 @@ ImageNet - 4.24 - `link `_ - `link `_ - - `link `_ + - `link `_ * - shufflenet_g8_w1 - 66.3 - - 65.44 + - 65.5 - 224x224x3 - 2.46 - 0.18 - `link `_ - `link `_ - - `link `_ + - `link `_ * - squeezenet_v1.1 - 59.85 - 59.4 @@ -316,7 +313,16 @@ ImageNet - 0.39 - `link `_ - `link `_ - - `link `_ + - `link `_ + * - vit_base + - 79.98 + - 77.25 + - 224x224x3 + - 86.5 + - 17.1 + - `link `_ + - `link `_ + - `link `_ * - vit_tiny - 68.02 - 65.42 @@ -325,7 +331,7 @@ ImageNet - 2.36 - `link `_ - `link `_ - - `link `_ + - `link `_ .. _Object Detection: @@ -356,7 +362,7 @@ COCO - 15.63 - `link `_ - `link `_ - - `link `_ + - `link `_ * - centernet_resnet_v1_50_postprocess - 31.78 - 29.64 @@ -365,7 +371,34 @@ COCO - 28.46 - `link `_ - `link `_ - - `link `_ + - `link `_ + * - damoyolo_tinynasL20_T + - 42.8 + - 42.0 + - 640x640x3 + - 11.35 + - 9.03 + - `link `_ + - `link `_ + - `link `_ + * - damoyolo_tinynasL25_S + - 46.53 + - 46.04 + - 640x640x3 + - 16.25 + - 18.85 + - `link `_ + - `link `_ + - `link `_ + * - damoyolo_tinynasL35_M + - 49.7 + - 47.23 + - 640x640x3 + - 33.98 + - 30.87 + - `link `_ + - `link `_ + - `link `_ * - efficientdet_lite0 - 27.43 - 26.6 @@ -374,7 +407,7 @@ COCO - 0.99 - `link `_ - `link `_ - - `link `_ + - `link `_ * - efficientdet_lite1 - 32.46 - 31.91 @@ -383,7 +416,7 @@ COCO - 2 - `link `_ - `link `_ - - `link `_ + - `link `_ * - efficientdet_lite2 - 36.16 - 34.88 @@ -392,7 +425,7 @@ COCO - 3.42 - `link `_ - `link `_ - - `link `_ + - `link `_ * - nanodet_repvgg - 29.3 - 28.53 @@ -401,7 +434,7 @@ COCO - 5.64 - `link `_ - `link `_ - - `link `_ + - `link `_ * - nanodet_repvgg_a1_640 - 33.28 - 32.88 @@ -410,16 +443,16 @@ COCO - 21.4 - `link `_ - `link `_ - - `link `_ + - `link `_ * - ssd_mobilenet_v1 |rocket| |star| - 23.17 - - 21.47 + - 22.37 - 300x300x3 - 6.79 - 1.25 - `link `_ - `link `_ - - `link `_ + - `link `_ * - ssd_mobilenet_v1_hd - 17.66 - 15.73 @@ -428,7 +461,7 @@ COCO - 12.26 - `link `_ - `link `_ - - `link `_ + - `link `_ * - ssd_mobilenet_v2 - 24.15 - 23.07 @@ -437,17 +470,17 @@ COCO - 0.76 - `link `_ - `link `_ - - `link `_ + - `link `_ * - tiny_yolov3 - 14.36 - - 13.45 + - 14.16 - 416x416x3 - 8.85 - 2.79 - `link `_ - `link `_ - - `link `_ - * - tiny_yolov4 |bomb| + - `link `_ + * - tiny_yolov4 - 19.18 - 17.73 - 416x416x3 @@ -455,7 +488,7 @@ COCO - 3.46 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolov3 |star| - 38.42 - 37.32 @@ -464,16 +497,16 @@ COCO - 79.17 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolov3_416 - 37.73 - - 35.86 + - 36.08 - 416x416x3 - 61.92 - 32.97 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolov3_gluon |rocket| |star| - 37.28 - 35.64 @@ -482,7 +515,7 @@ COCO - 79.17 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolov3_gluon_416 |star| - 36.27 - 34.92 @@ -491,7 +524,7 @@ COCO - 32.97 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolov4_leaky |star| - 42.37 - 41.47 @@ -500,7 +533,7 @@ COCO - 45.60 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolov5l - 46.01 - 44.01 @@ -509,7 +542,7 @@ COCO - 60.78 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolov5m - 42.59 - 41.19 @@ -518,7 +551,7 @@ COCO - 26.14 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolov5m6_6.1 - 50.68 - 48.74 @@ -527,7 +560,7 @@ COCO - 100.02 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolov5m_6.1 - 44.81 - 43.38 @@ -536,7 +569,16 @@ COCO - 24.48 - `link `_ - `link `_ - - `link `_ + - `link `_ + * - yolov5m_hpp + - 42.59 + - 41.19 + - 640x640x3 + - 21.78 + - 26.14 + - `link `_ + - `link `_ + - `link `_ * - yolov5m_wo_spp |rocket| - 42.46 - 40.43 @@ -545,7 +587,7 @@ COCO - 26.49 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolov5n6_6.1 - 35.63 - 33.68 @@ -554,7 +596,7 @@ COCO - 9.17 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolov5s |star| - 35.33 - 34.25 @@ -563,7 +605,7 @@ COCO - 8.72 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolov5s6_6.1 - 44.17 - 41.74 @@ -572,16 +614,16 @@ COCO - 33.70 - `link `_ - `link `_ - - `link `_ - * - yolov5s_personface |star| - - 47.7 - - 45.75 + - `link `_ + * - yolov5s_c3tr + - 37.13 + - 35.33 - 640x640x3 - - 7.25 - - 8.38 - - `link `_ - - `link `_ - - `link `_ + - 10.29 + - 8.51 + - `link `_ + - `link `_ + - `link `_ * - yolov5xs_wo_spp - 32.78 - 31.8 @@ -590,17 +632,17 @@ COCO - 5.68 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolov5xs_wo_spp_nms - 32.57 - - 30.7 + - 31.06 - 512x512x3 - 7.85 - 5.68 - `link `_ - `link `_ - - `link `_ - * - yolov6n |bomb| + - `link `_ + * - yolov6n - 34.29 - 32.19 - 640x640x3 @@ -608,8 +650,8 @@ COCO - 5.57 - `link `_ - `link `_ - - `link `_ - * - yolov7 |bomb| + - `link `_ + * - yolov7 - 49.72 - 46.92 - 640x640x3 @@ -617,8 +659,8 @@ COCO - 52.34 - `link `_ - `link `_ - - `link `_ - * - yolov7_tiny |bomb| + - `link `_ + * - yolov7_tiny - 36.49 - 35.39 - 640x640x3 @@ -626,7 +668,7 @@ COCO - 6.87 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolov7e6 - 55.37 - 53.17 @@ -635,34 +677,79 @@ COCO - 257.56 - `link `_ - `link `_ - - `link `_ - * - yolox_l_leaky |star| |bomb| + - `link `_ + * - yolov8l + - 52.61 + - 51.95 + - 640x640x3 + - 43.7 + - 82.65 + - `link `_ + - `link `_ + - `link `_ + * - yolov8m + - 50.08 + - 49.28 + - 640x640x3 + - 25.9 + - 39.5 + - `link `_ + - `link `_ + - `link `_ + * - yolov8n + - 37.23 + - 36.23 + - 640x640x3 + - 3.2 + - 4.4 + - `link `_ + - `link `_ + - `link `_ + * - yolov8s + - 44.75 + - 44.15 + - 640x640x3 + - 11.2 + - 14.3 + - `link `_ + - `link `_ + - `link `_ + * - yolov8x + - 53.61 + - 52.21 + - 640x640x3 + - 68.2 + - 129 + - `link `_ + - `link `_ + - `link `_ + * - yolox_l_leaky |star| - 48.68 - - 47.18 + - 47.08 - 640x640x3 - 54.17 - 77.74 - `link `_ - `link `_ - - `link `_ - * - yolox_s_leaky |bomb| + - `link `_ + * - yolox_s_leaky - 38.13 - - 37.33 + - 37.51 - 640x640x3 - 8.96 - 13.37 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolox_s_wide_leaky - 42.4 - - 41.01 + - 41.38 - 640x640x3 - 20.12 - 29.73 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolox_tiny - 32.64 - 31.32 @@ -671,7 +758,7 @@ COCO - 3.22 - `link `_ - `link `_ - - `link `_ + - `link `_ VisDrone ^^^^^^^^ @@ -697,7 +784,7 @@ VisDrone - 1.15 - `link `_ - `link `_ - - `link `_ + - `link `_ .. _Semantic Segmentation: @@ -728,7 +815,7 @@ Cityscapes - 71.26 - `link `_ - `link `_ - - `link `_ + - `link `_ * - fcn8_resnet_v1_18 - 68.75 - 67.97 @@ -737,17 +824,17 @@ Cityscapes - 71.51 - `link `_ - `link `_ - - `link `_ + - `link `_ * - fcn8_resnet_v1_22 - 67.55 - 67.39 - - 1920x1024x3 + - 1024x1920x3 - 15.12 - 150.04 - `link `_ - `link `_ - - `link `_ - * - stdc1 |bomb| + - `link `_ + * - stdc1 |rocket| - 74.57 - 73.47 - 1024x1920x3 @@ -755,7 +842,7 @@ Cityscapes - 63.34 - `link `_ - `link `_ - - `link `_ + - `link `_ Oxford-IIIT Pet ^^^^^^^^^^^^^^^ @@ -775,13 +862,13 @@ Oxford-IIIT Pet - Compiled * - unet_mobilenet_v2 - 77.32 - - 76.82 + - 77.02 - 256x256x3 - 10.08 - 14.44 - `link `_ - `link `_ - - `link `_ + - `link `_ Pascal VOC ^^^^^^^^^^ @@ -807,7 +894,7 @@ Pascal VOC - 8.91 - `link `_ - `link `_ - - `link `_ + - `link `_ * - deeplab_v3_mobilenet_v2_wo_dilation - 71.46 - 71.08 @@ -816,7 +903,7 @@ Pascal VOC - 1.64 - `link `_ - `link `_ - - `link `_ + - `link `_ .. _Pose Estimation: @@ -847,25 +934,25 @@ COCO - 32.38 - `link `_ - `link `_ - - `link `_ + - `link `_ * - centerpose_regnetx_800mf - 44.07 - - 41.66 + - 41.9 - 512x512x3 - 12.31 - 43.06 - `link `_ - `link `_ - - `link `_ + - `link `_ * - centerpose_repvgg_a0 |star| - 39.17 - - 37.22 + - 37.09 - 416x416x3 - 11.71 - 14.15 - `link `_ - `link `_ - - `link `_ + - `link `_ .. _Single Person Pose Estimation: @@ -896,7 +983,7 @@ COCO - 1.47 - `link `_ - `link `_ - - `link `_ + - `link `_ .. _Face Detection: @@ -921,22 +1008,22 @@ WiderFace - Compiled * - lightface_slim |star| - 39.7 - - 39.24 + - 39.41 - 240x320x3 - 0.26 - 0.08 - `link `_ - `link `_ - - `link `_ + - `link `_ * - retinaface_mobilenet_v1 |star| - 81.27 - - 81.03 + - 81.17 - 736x1280x3 - 3.49 - 12.57 - `link `_ - `link `_ - - `link `_ + - `link `_ * - scrfd_10g - 82.13 - 82.03 @@ -945,7 +1032,7 @@ WiderFace - 13.37 - `link `_ - `link `_ - - `link `_ + - `link `_ * - scrfd_2.5g - 76.59 - 76.32 @@ -954,16 +1041,16 @@ WiderFace - 3.44 - `link `_ - `link `_ - - `link `_ + - `link `_ * - scrfd_500m - 68.98 - - 68.45 + - 68.88 - 640x640x3 - 0.63 - 0.75 - `link `_ - `link `_ - - `link `_ + - `link `_ .. _Instance Segmentation: @@ -978,7 +1065,7 @@ COCO :header-rows: 1 * - Network Name - - mAP + - mAP-segmentation - Quantized - Input Resolution (HxWxC) - Params (M) @@ -994,7 +1081,7 @@ COCO - 51.92 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolact_regnetx_1.6gf - 27.57 - 27.27 @@ -1003,7 +1090,7 @@ COCO - 62.67 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolact_regnetx_800mf - 25.61 - 25.5 @@ -1012,7 +1099,7 @@ COCO - 58.375 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolact_regnetx_800mf_20classes |star| - 20.23 - 20.22 @@ -1021,7 +1108,7 @@ COCO - 51.47 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolov5l_seg - 39.78 - 39.09 @@ -1030,7 +1117,7 @@ COCO - 73.94 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolov5m_seg - 37.05 - 36.32 @@ -1039,7 +1126,7 @@ COCO - 35.47 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolov5n_seg - 23.35 - 22.24 @@ -1048,7 +1135,7 @@ COCO - 3.55 - `link `_ - `link `_ - - `link `_ + - `link `_ * - yolov5s_seg - 31.57 - 30.49 @@ -1057,7 +1144,7 @@ COCO - 13.21 - `link `_ - `link `_ - - `link `_ + - `link `_ D2S ^^^ @@ -1077,13 +1164,13 @@ D2S - Compiled * - yolact_regnetx_600mf_d2s_31classes - 62.48 - - 63.98 + - 63.36 - 512x512x3 - 22.14 - 51.62 - `link `_ - `link `_ - - `link `_ + - `link `_ .. _Depth Estimation: @@ -1114,7 +1201,7 @@ NYU - 0.37 - `link `_ - `link `_ - - `link `_ + - `link `_ .. _Facial Landmark Detection: @@ -1139,13 +1226,13 @@ AFLW2k3d - Compiled * - tddfa_mobilenet_v1 |star| - 3.68 - - 4.06 + - 3.78 - 120x120x3 - 3.26 - 0.18 - `link `_ - `link `_ - - `link `_ + - `link `_ .. _Person Re-ID: @@ -1176,25 +1263,25 @@ Market1501 - 0.99 - `link `_ - `link `_ - - `link `_ + - `link `_ * - repvgg_a0_person_reid_2048 - 90.02 - - 89.47 + - 89.12 - 256x128x3 - 9.65 - 0.89 - `link `_ - `link `_ - - `link `_ + - `link `_ * - repvgg_a0_person_reid_512 |star| - 89.9 - - 89.6 + - 89.4 - 256x128x3 - 7.68 - 0.89 - `link `_ - `link `_ - - `link `_ + - `link `_ .. _Super Resolution: @@ -1219,31 +1306,31 @@ BSD100 - Compiled * - espcn_x2 - 31.4 - - 29.5 + - 30.3 - 156x240x1 - 0.02 - 0.8 - `link `_ - `link `_ - - `link `_ + - `link `_ * - espcn_x3 - 28.41 - - 27.86 + - 28.06 - 104x160x1 - 0.02 - 0.38 - `link `_ - `link `_ - - `link `_ + - `link `_ * - espcn_x4 - 26.83 - - 26.43 + - 26.58 - 78x120x1 - 0.02 - 0.23 - `link `_ - `link `_ - - `link `_ + - `link `_ .. _Face Recognition: @@ -1274,7 +1361,7 @@ LFW - 0.44 - `link `_ - `link `_ - - `link `_ + - `link `_ * - arcface_r50 - 99.72 - 99.71 @@ -1283,7 +1370,7 @@ LFW - 6.30 - `link `_ - `link `_ - - `link `_ + - `link `_ .. _Person Attribute: @@ -1314,7 +1401,7 @@ PETA - 1.82 - `link `_ - `link `_ - - `link `_ + - `link `_ .. _Hand Landmark detection: @@ -1340,4 +1427,4 @@ Hand Landmark - 0.15 - `link `_ - `link `_ - - `link `_ + - `link `_ diff --git a/hailo_model_zoo/cfg/alls/base/centernet_resnet_v1_50_postprocess.alls b/hailo_model_zoo/cfg/alls/base/centernet_resnet_v1_50_postprocess.alls index 69abe02d..4328f48f 100644 --- a/hailo_model_zoo/cfg/alls/base/centernet_resnet_v1_50_postprocess.alls +++ b/hailo_model_zoo/cfg/alls/base/centernet_resnet_v1_50_postprocess.alls @@ -1,16 +1,5 @@ normalization1 = normalization([123.675, 116.28, 103.53], [58.395, 57.12, 57.375]) -model_optimization_config(calibration, batch_size=4, calibset_size=64) +model_optimization_config(calibration, batch_size=2, calibset_size=64) post_quantization_optimization(bias_correction, policy=disabled) pre_quantization_optimization(equalization, policy=enabled) -post_quantization_optimization(finetune, policy=enabled, dataset_size=4000, epochs=8, learning_rate=0.0001, loss_factors=[0.2, 0.1, 30.0], loss_layer_names=[ew_add1, conv60, conv62], loss_types=[l2, l2, l2]) -quantization_param([conv1, conv14, conv27, conv46], bias_mode=single_scale_decomposition) -quantization_param([conv2, conv3, conv4, conv5, conv6, conv7, conv8, conv9, conv10, conv11, conv12, conv13, conv15, - conv16, conv17, conv18, conv19, conv20, conv21, conv22, conv23, conv24, conv25, conv26, conv28, - conv29, conv30, conv31, conv32, conv33, conv34, conv35, conv36, conv37, conv38, conv39, conv40, - conv41, conv42, conv43, conv44, conv45, conv47, conv48, conv49, conv50, conv51, conv52, conv53, - conv54, deconv1, conv55, deconv2, conv56, deconv3, conv57, conv58, conv59, conv60, conv61, conv62, - conv63], bias_mode=double_scale_initialization) -quantization_param(conv54, precision_mode=a8_w4) -quantization_param(conv49, precision_mode=a8_w4) -quantization_param(conv52, precision_mode=a8_w4) -quantization_param(conv44, precision_mode=a8_w4) +post_quantization_optimization(finetune, policy=enabled, dataset_size=4000, epochs=8, learning_rate=0.0001, loss_factors=[0.2, 0.1, 30.0], loss_layer_names=[ew_add18, conv61, conv63], loss_types=[l2, l2, l2]) diff --git a/hailo_model_zoo/cfg/alls/base/centerpose_regnetx_1.6gf_fpn.alls b/hailo_model_zoo/cfg/alls/base/centerpose_regnetx_1.6gf_fpn.alls index 4c7f9f89..92af88ab 100644 --- a/hailo_model_zoo/cfg/alls/base/centerpose_regnetx_1.6gf_fpn.alls +++ b/hailo_model_zoo/cfg/alls/base/centerpose_regnetx_1.6gf_fpn.alls @@ -1,3 +1,6 @@ centerpose_regnetx_1_6gf_fpn/normalization1 = normalization([119.85, 113.985, 104.04], [70.89, 69.87, 73.695], centerpose_regnetx_1_6gf_fpn/input_layer1) model_optimization_config(calibration, batch_size=4, calibset_size=32) pre_quantization_optimization(equalization, policy=disabled) +change_output_activation(centerpose_regnetx_1_6gf_fpn/conv79, sigmoid) +change_output_activation(centerpose_regnetx_1_6gf_fpn/conv75, sigmoid) +quantization_param({*output_layer*}, precision_mode=a8_w8) \ No newline at end of file diff --git a/hailo_model_zoo/cfg/alls/base/centerpose_regnetx_800mf.alls b/hailo_model_zoo/cfg/alls/base/centerpose_regnetx_800mf.alls index 3b208a0f..98a94f27 100644 --- a/hailo_model_zoo/cfg/alls/base/centerpose_regnetx_800mf.alls +++ b/hailo_model_zoo/cfg/alls/base/centerpose_regnetx_800mf.alls @@ -2,6 +2,6 @@ normalization1 = normalization([104.04, 113.985, 119.85], [73.695, 69.87, 70.89] change_output_activation(centerpose_regnetx_800mf/conv64, sigmoid) change_output_activation(centerpose_regnetx_800mf/conv60, sigmoid) input_conversion = input_conversion(bgr_to_rgb) -model_optimization_config(calibration, batch_size=8, calibset_size=2048) -post_quantization_optimization(finetune, policy=enabled, loss_factors=[1.0, 0.25, 0.25, 0.25, 0.25, 0.25], dataset_size=4000, epochs=8, learning_rate=0.0001, loss_types=[l2, l2, l2, l2, l2, l2]) +model_optimization_config(calibration, batch_size=4, calibset_size=2048) +post_quantization_optimization(finetune, policy=enabled, loss_factors=[1.0, 0.25, 0.25, 0.25, 0.25, 0.25], dataset_size=4000, epochs=8, learning_rate=0.0001, loss_types=[l2, l2, l2, l2, l2, l2], loss_layer_names=[conv60, conv61, conv62, conv63, conv64, conv65]) quantization_param(output_layer1, precision_mode=a16_w16) diff --git a/hailo_model_zoo/cfg/alls/base/centerpose_repvgg_a0.alls b/hailo_model_zoo/cfg/alls/base/centerpose_repvgg_a0.alls index 5fb52e8b..b00c5cbf 100644 --- a/hailo_model_zoo/cfg/alls/base/centerpose_repvgg_a0.alls +++ b/hailo_model_zoo/cfg/alls/base/centerpose_repvgg_a0.alls @@ -1,2 +1,4 @@ centerpose_repvgg_a0/normalization1 = normalization([119.85, 113.985, 104.04], [70.89, 69.87, 73.695], centerpose_repvgg_a0/input_layer1) model_optimization_config(calibration, batch_size=8, calibset_size=2048) +change_output_activation(centerpose_repvgg_a0/conv40, sigmoid) +change_output_activation(centerpose_repvgg_a0/conv36, sigmoid) diff --git a/hailo_model_zoo/cfg/alls/base/damoyolo_tinynasL20_T.alls b/hailo_model_zoo/cfg/alls/base/damoyolo_tinynasL20_T.alls new file mode 100644 index 00000000..96cc5330 --- /dev/null +++ b/hailo_model_zoo/cfg/alls/base/damoyolo_tinynasL20_T.alls @@ -0,0 +1,2 @@ +pre_quantization_optimization(equalization, policy=disabled) +post_quantization_optimization(finetune, policy=enabled, loss_layer_names=[conv53, conv54, conv65, conv66, conv67, conv77, conv51, conv63, conv75], loss_factors=[1, 1, 1, 1, 1, 1, 52, 16, 4]) \ No newline at end of file diff --git a/hailo_model_zoo/cfg/alls/base/damoyolo_tinynasL25_S.alls b/hailo_model_zoo/cfg/alls/base/damoyolo_tinynasL25_S.alls new file mode 100644 index 00000000..d3e17c39 --- /dev/null +++ b/hailo_model_zoo/cfg/alls/base/damoyolo_tinynasL25_S.alls @@ -0,0 +1,6 @@ +pre_quantization_optimization(equalization, policy=disabled) +post_quantization_optimization(finetune, policy=enabled, batch_size=4, loss_layer_names = [conv57, conv69, conv81, output_layer1, output_layer2, output_layer3, output_layer4, output_layer5, output_layer6], + loss_types = [l2rel,l2rel,l2rel,l2rel,l2rel,l2rel,l2rel,l2rel,l2rel], + loss_factors=[1,1,1,2,2,2,2,2,2], epochs = 4) + + diff --git a/hailo_model_zoo/cfg/alls/base/damoyolo_tinynasL35_M.alls b/hailo_model_zoo/cfg/alls/base/damoyolo_tinynasL35_M.alls new file mode 100644 index 00000000..43815b98 --- /dev/null +++ b/hailo_model_zoo/cfg/alls/base/damoyolo_tinynasL35_M.alls @@ -0,0 +1,2 @@ +pre_quantization_optimization(equalization, policy=disabled) +post_quantization_optimization(finetune, policy=enabled, batch_size=4, loss_layer_names = [conv81, conv95, conv109, output_layer1, output_layer2, output_layer3, output_layer4, output_layer5, output_layer6], loss_factors = [1,1,1,2,2,2,2,2,2]) \ No newline at end of file diff --git a/hailo_model_zoo/cfg/alls/base/efficientdet_lite2.alls b/hailo_model_zoo/cfg/alls/base/efficientdet_lite2.alls index 3b70552f..9715db32 100644 --- a/hailo_model_zoo/cfg/alls/base/efficientdet_lite2.alls +++ b/hailo_model_zoo/cfg/alls/base/efficientdet_lite2.alls @@ -1,3 +1,4 @@ normalization1 = normalization([127.0, 127.0, 127.0], [128.0, 128.0, 128.0]) post_quantization_optimization(bias_correction, policy=enabled) +model_optimization_config(calibration, batch_size=4) post_quantization_optimization(finetune, policy=disabled) diff --git a/hailo_model_zoo/cfg/alls/base/efficientnet_s.alls b/hailo_model_zoo/cfg/alls/base/efficientnet_s.alls index 1abd27c6..9261dfce 100644 --- a/hailo_model_zoo/cfg/alls/base/efficientnet_s.alls +++ b/hailo_model_zoo/cfg/alls/base/efficientnet_s.alls @@ -1 +1 @@ -model_optimization_flavor(optimization_level=1, compression_level=1) +model_optimization_flavor(optimization_level=2, compression_level=1) diff --git a/hailo_model_zoo/cfg/alls/base/face_attr_resnet_v1_18.alls b/hailo_model_zoo/cfg/alls/base/face_attr_resnet_v1_18.alls index 5aec9d54..fe24b4f6 100644 --- a/hailo_model_zoo/cfg/alls/base/face_attr_resnet_v1_18.alls +++ b/hailo_model_zoo/cfg/alls/base/face_attr_resnet_v1_18.alls @@ -1,2 +1,2 @@ normalization1 = normalization([123.675, 116.28, 103.53], [58.395, 57.12, 57.375]) -model_optimization_flavor(optimization_level=1, compression_level=2) +model_optimization_flavor(optimization_level=2, compression_level=2) diff --git a/hailo_model_zoo/cfg/alls/base/lightface_slim.alls b/hailo_model_zoo/cfg/alls/base/lightface_slim.alls index d381521a..a2114734 100644 --- a/hailo_model_zoo/cfg/alls/base/lightface_slim.alls +++ b/hailo_model_zoo/cfg/alls/base/lightface_slim.alls @@ -1,3 +1,4 @@ normalization1 = normalization([127.0, 127.0, 127.0], [128.0, 128.0, 128.0]) model_optimization_config(calibration, batch_size=8, calibset_size=16) post_quantization_optimization(finetune, policy=disabled) +quantization_param({output_layer*}, precision_mode=a8_w8) diff --git a/hailo_model_zoo/cfg/alls/base/lprnet.alls b/hailo_model_zoo/cfg/alls/base/lprnet.alls index da86e058..bd4a9a7e 100644 --- a/hailo_model_zoo/cfg/alls/base/lprnet.alls +++ b/hailo_model_zoo/cfg/alls/base/lprnet.alls @@ -1,2 +1,2 @@ lprnet/normalization1 = normalization([127.5, 127.5, 127.5], [128.0, 128.0, 128.0]) -input_conversion = input_conversion(bgr_to_rgb) +input_conversion = input_conversion(bgr_to_rgb) \ No newline at end of file diff --git a/hailo_model_zoo/cfg/alls/base/lprnet_yuy2.alls b/hailo_model_zoo/cfg/alls/base/lprnet_yuy2.alls new file mode 100644 index 00000000..c08db2f4 --- /dev/null +++ b/hailo_model_zoo/cfg/alls/base/lprnet_yuy2.alls @@ -0,0 +1,6 @@ +input_conversion = input_conversion(bgr_to_rgb) +normalization1 = normalization([127.5, 127.5, 127.5], [128.0, 128.0, 128.0]) +yuv_to_rgb1 = yuv_to_rgb() +reshape_yuy2 = input_conversion(lprnet_yuy2/input_layer1, yuy2_to_hailo_yuv, emulator_support=True) +model_optimization_config(calibration, batch_size=4, calibset_size=64) +pre_quantization_optimization(equalization, policy=disabled) \ No newline at end of file diff --git a/hailo_model_zoo/cfg/alls/base/mspn_regnetx_800mf.alls b/hailo_model_zoo/cfg/alls/base/mspn_regnetx_800mf.alls index 6284eb35..055a6f72 100644 --- a/hailo_model_zoo/cfg/alls/base/mspn_regnetx_800mf.alls +++ b/hailo_model_zoo/cfg/alls/base/mspn_regnetx_800mf.alls @@ -1,2 +1,3 @@ mspn_regnetx_800mf/normalization1 = normalization([123.675, 116.28, 103.53], [58.395, 57.12, 57.375]) model_optimization_flavor(optimization_level=2, compression_level=1) +quantization_param({output_layer*}, precision_mode=a8_w8) \ No newline at end of file diff --git a/hailo_model_zoo/cfg/alls/base/nanodet_repvgg.alls b/hailo_model_zoo/cfg/alls/base/nanodet_repvgg.alls index db0d6e87..52ac0809 100644 --- a/hailo_model_zoo/cfg/alls/base/nanodet_repvgg.alls +++ b/hailo_model_zoo/cfg/alls/base/nanodet_repvgg.alls @@ -1,3 +1,4 @@ normalization1 = normalization([103.53, 116.28, 123.675], [57.375, 57.12, 58.395]) model_optimization_config(calibration, batch_size=4, calibset_size=64) -input_conversion = input_conversion(bgr_to_rgb) +quantization_param({output_layer*}, precision_mode=a8_w8) +input_conversion = input_conversion(bgr_to_rgb) \ No newline at end of file diff --git a/hailo_model_zoo/cfg/alls/base/repvgg_a0_person_reid_2048.alls b/hailo_model_zoo/cfg/alls/base/repvgg_a0_person_reid_2048.alls index 7440939c..7e46547d 100644 --- a/hailo_model_zoo/cfg/alls/base/repvgg_a0_person_reid_2048.alls +++ b/hailo_model_zoo/cfg/alls/base/repvgg_a0_person_reid_2048.alls @@ -2,4 +2,4 @@ normalization1 = normalization([111.296 , 104.263, 104.396], [64.193 , 62.547, 6 pre_quantization_optimization(activation_clipping, layers=[conv2], mode=percentile, clipping_values=[0, 99.25]) pre_quantization_optimization(activation_clipping, layers=[conv9], mode=percentile, clipping_values=[0, 99.25]) pre_quantization_optimization(activation_clipping, layers=[conv20], mode=percentile, clipping_values=[0, 99.45]) -post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=8, dataset_size=8000, loss_factors=[1.0], loss_types=[L2REL]) +post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=8, dataset_size=8000, loss_factors=[1.0], loss_types=[L2REL], loss_layer_names=[fc1]) diff --git a/hailo_model_zoo/cfg/alls/base/repvgg_a0_person_reid_512.alls b/hailo_model_zoo/cfg/alls/base/repvgg_a0_person_reid_512.alls index 7440939c..7e46547d 100644 --- a/hailo_model_zoo/cfg/alls/base/repvgg_a0_person_reid_512.alls +++ b/hailo_model_zoo/cfg/alls/base/repvgg_a0_person_reid_512.alls @@ -2,4 +2,4 @@ normalization1 = normalization([111.296 , 104.263, 104.396], [64.193 , 62.547, 6 pre_quantization_optimization(activation_clipping, layers=[conv2], mode=percentile, clipping_values=[0, 99.25]) pre_quantization_optimization(activation_clipping, layers=[conv9], mode=percentile, clipping_values=[0, 99.25]) pre_quantization_optimization(activation_clipping, layers=[conv20], mode=percentile, clipping_values=[0, 99.45]) -post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=8, dataset_size=8000, loss_factors=[1.0], loss_types=[L2REL]) +post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=8, dataset_size=8000, loss_factors=[1.0], loss_types=[L2REL], loss_layer_names=[fc1]) diff --git a/hailo_model_zoo/cfg/alls/base/resmlp12_relu.alls b/hailo_model_zoo/cfg/alls/base/resmlp12_relu.alls index 95c7e48e..e8875a7b 100644 --- a/hailo_model_zoo/cfg/alls/base/resmlp12_relu.alls +++ b/hailo_model_zoo/cfg/alls/base/resmlp12_relu.alls @@ -8,38 +8,38 @@ buffers(resmlp12_relu/space_to_depth1, resmlp12_relu/conv1, 0, PARTIAL_ROW) buffers(resmlp12_relu/mul_and_add1, resmlp12_relu/conv3, 0, PARTIAL_ROW) buffers(resmlp12_relu/mul_and_add1, resmlp12_relu/conv4, 0, PARTIAL_ROW) buffers(resmlp12_relu/conv3, resmlp12_relu/conv4, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add2, resmlp12_relu/conv6, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add2, resmlp12_relu/conv7, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add3, resmlp12_relu/conv6, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add3, resmlp12_relu/conv7, 0, PARTIAL_ROW) buffers(resmlp12_relu/conv6, resmlp12_relu/conv7, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add3, resmlp12_relu/conv9, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add3, resmlp12_relu/conv10, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add5, resmlp12_relu/conv9, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add5, resmlp12_relu/conv10, 0, PARTIAL_ROW) buffers(resmlp12_relu/conv9, resmlp12_relu/conv10, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add4, resmlp12_relu/conv12, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add4, resmlp12_relu/conv13, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add7, resmlp12_relu/conv12, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add7, resmlp12_relu/conv13, 0, PARTIAL_ROW) buffers(resmlp12_relu/conv12, resmlp12_relu/conv13, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add5, resmlp12_relu/conv15, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add5, resmlp12_relu/conv16, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add9, resmlp12_relu/conv15, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add9, resmlp12_relu/conv16, 0, PARTIAL_ROW) buffers(resmlp12_relu/conv15, resmlp12_relu/conv16, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add6, resmlp12_relu/conv18, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add6, resmlp12_relu/conv19, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add11, resmlp12_relu/conv18, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add11, resmlp12_relu/conv19, 0, PARTIAL_ROW) buffers(resmlp12_relu/conv18, resmlp12_relu/conv19, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add7, resmlp12_relu/conv21, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add7, resmlp12_relu/conv22, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add13, resmlp12_relu/conv21, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add13, resmlp12_relu/conv22, 0, PARTIAL_ROW) buffers(resmlp12_relu/conv21, resmlp12_relu/conv22, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add8, resmlp12_relu/conv24, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add8, resmlp12_relu/conv25, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add15, resmlp12_relu/conv24, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add15, resmlp12_relu/conv25, 0, PARTIAL_ROW) buffers(resmlp12_relu/conv24, resmlp12_relu/conv25, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add9, resmlp12_relu/conv27, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add9, resmlp12_relu/conv28, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add17, resmlp12_relu/conv27, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add17, resmlp12_relu/conv28, 0, PARTIAL_ROW) buffers(resmlp12_relu/conv27, resmlp12_relu/conv28, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add10, resmlp12_relu/conv30, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add10, resmlp12_relu/conv31, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add19, resmlp12_relu/conv30, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add19, resmlp12_relu/conv31, 0, PARTIAL_ROW) buffers(resmlp12_relu/conv30, resmlp12_relu/conv31, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add11, resmlp12_relu/conv33, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add11, resmlp12_relu/conv34, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add21, resmlp12_relu/conv33, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add21, resmlp12_relu/conv34, 0, PARTIAL_ROW) buffers(resmlp12_relu/conv33, resmlp12_relu/conv34, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add12, resmlp12_relu/conv36, 0, PARTIAL_ROW) -buffers(resmlp12_relu/mul_and_add12, resmlp12_relu/conv37, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add23, resmlp12_relu/conv36, 0, PARTIAL_ROW) +buffers(resmlp12_relu/mul_and_add23, resmlp12_relu/conv37, 0, PARTIAL_ROW) buffers(resmlp12_relu/conv36, resmlp12_relu/conv37, 0, PARTIAL_ROW) performance_param(fps=1431) diff --git a/hailo_model_zoo/cfg/alls/base/resnet_v1_50.alls b/hailo_model_zoo/cfg/alls/base/resnet_v1_50.alls index 83c9c271..6ab64962 100644 --- a/hailo_model_zoo/cfg/alls/base/resnet_v1_50.alls +++ b/hailo_model_zoo/cfg/alls/base/resnet_v1_50.alls @@ -1,4 +1,4 @@ normalization1 = normalization([123.68, 116.78, 103.94], [1.0, 1.0, 1.0]) -# quantization_param({conv*}, max_elementwise_feed_repeat=1) -quantization_param([conv5, conv8, conv11, conv15, conv18, conv21, conv24, conv28, conv31, conv34, conv37, conv40, conv43, conv47, conv50, conv53], max_elementwise_feed_repeat=1) +quantization_param({conv*}, max_elementwise_feed_repeat=1) +# quantization_param([conv5, conv8, conv11, conv15, conv18, conv21, conv24, conv28, conv31, conv34, conv37, conv40, conv43, conv47, conv50, conv53], max_elementwise_feed_repeat=1) post_quantization_optimization(finetune, policy=disabled) diff --git a/hailo_model_zoo/cfg/alls/base/retinaface_mobilenet_v1.alls b/hailo_model_zoo/cfg/alls/base/retinaface_mobilenet_v1.alls index 53177822..06a106fb 100644 --- a/hailo_model_zoo/cfg/alls/base/retinaface_mobilenet_v1.alls +++ b/hailo_model_zoo/cfg/alls/base/retinaface_mobilenet_v1.alls @@ -1,7 +1,8 @@ normalization1 = normalization([104, 117, 123], [1.0, 1.0, 1.0]) input_conversion = input_conversion(bgr_to_rgb) -model_optimization_config(calibration, batch_size=4, calibset_size=16) +model_optimization_config(calibration, batch_size=2, calibset_size=16) quantization_param([conv1, conv2, conv3, conv4, conv5, conv6, conv7, conv8, conv9, conv10, conv11, conv12, conv13, conv14, conv15, conv16, conv17, conv18, conv19, conv20, conv21, conv22, conv23, conv24, conv25, conv26, conv27, conv28, conv29, conv30, conv31, conv32, conv33, conv34, conv35, conv36, conv37, conv38, conv39, conv40, conv41, conv42, conv43], bias_mode=single_scale_decomposition) post_quantization_optimization(finetune, policy=disabled) post_quantization_optimization(bias_correction, policy=enabled) +quantization_param({output_layer*}, precision_mode=a8_w8) diff --git a/hailo_model_zoo/cfg/alls/base/scrfd_10g.alls b/hailo_model_zoo/cfg/alls/base/scrfd_10g.alls index 5842b050..b872cf95 100644 --- a/hailo_model_zoo/cfg/alls/base/scrfd_10g.alls +++ b/hailo_model_zoo/cfg/alls/base/scrfd_10g.alls @@ -1,4 +1,4 @@ normalization1 = normalization([127.5, 127.5, 127.5], [128.0, 128.0, 128.0]) -change_output_activation(conv47, sigmoid) -change_output_activation(conv53, sigmoid) +change_output_activation(conv41, sigmoid) +change_output_activation(conv49, sigmoid) change_output_activation(conv56, sigmoid) diff --git a/hailo_model_zoo/cfg/alls/base/scrfd_2.5g.alls b/hailo_model_zoo/cfg/alls/base/scrfd_2.5g.alls index 5bdd7c39..295cd4b1 100644 --- a/hailo_model_zoo/cfg/alls/base/scrfd_2.5g.alls +++ b/hailo_model_zoo/cfg/alls/base/scrfd_2.5g.alls @@ -1,4 +1,4 @@ normalization1 = normalization([127.5, 127.5, 127.5], [128.0, 128.0, 128.0]) -change_output_activation(conv46, sigmoid) -change_output_activation(conv52, sigmoid) +change_output_activation(conv42, sigmoid) +change_output_activation(conv49, sigmoid) change_output_activation(conv55, sigmoid) diff --git a/hailo_model_zoo/cfg/alls/base/scrfd_500m.alls b/hailo_model_zoo/cfg/alls/base/scrfd_500m.alls index c497ed29..f0dc0aed 100644 --- a/hailo_model_zoo/cfg/alls/base/scrfd_500m.alls +++ b/hailo_model_zoo/cfg/alls/base/scrfd_500m.alls @@ -1,4 +1,4 @@ normalization1 = normalization([127.5, 127.5, 127.5], [128.0, 128.0, 128.0]) -change_output_activation(conv30, sigmoid) -change_output_activation(conv34, sigmoid) +change_output_activation(conv26, sigmoid) +change_output_activation(conv32, sigmoid) change_output_activation(conv38, sigmoid) diff --git a/hailo_model_zoo/cfg/alls/base/ssd_mobilenet_v1.alls b/hailo_model_zoo/cfg/alls/base/ssd_mobilenet_v1.alls index af9e3e28..f11caa73 100644 --- a/hailo_model_zoo/cfg/alls/base/ssd_mobilenet_v1.alls +++ b/hailo_model_zoo/cfg/alls/base/ssd_mobilenet_v1.alls @@ -2,4 +2,7 @@ normalization1 = normalization([127.5, 127.5, 127.5], [127.5, 127.5, 127.5]) nms_postprocess("$HMZ_DATA/models_files/ObjectDetection/Detection-COCO/ssd/ssd_mobilenet_v1/pretrained/2021-07-11/mobilenet_ssd_nms_postprocess_config.json", ssd) context_switch_param(mode=disabled) quantization_param({conv*}, bias_mode=single_scale_decomposition) -post_quantization_optimization(finetune, policy=enabled, loss_factors=[0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1], dataset_size=4000, epochs=8, learning_rate=0.0001, loss_types=[l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2]) +post_quantization_optimization(finetune, policy=enabled, dataset_size=4000, epochs=8, learning_rate=0.0001, + loss_layer_names=[bbox_decoder13, conv14, bbox_decoder17, conv18, bbox_decoder21, conv22, bbox_decoder25, conv26, bbox_decoder29, conv30, bbox_decoder33, conv34, conv32, conv24, conv28, conv12, conv20, conv16], + loss_factors=[0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 1.0, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1], + loss_types=[l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2]) diff --git a/hailo_model_zoo/cfg/alls/base/ssd_mobilenet_v1_no_alls.alls b/hailo_model_zoo/cfg/alls/base/ssd_mobilenet_v1_no_alls.alls index f96e4c24..d3ba6ffc 100644 --- a/hailo_model_zoo/cfg/alls/base/ssd_mobilenet_v1_no_alls.alls +++ b/hailo_model_zoo/cfg/alls/base/ssd_mobilenet_v1_no_alls.alls @@ -1,4 +1,4 @@ normalization1 = normalization([127.5, 127.5, 127.5], [127.5, 127.5, 127.5]) nms_postprocess("$HMZ_DATA/models_files/ObjectDetection/Detection-COCO/ssd/ssd_mobilenet_v1/pretrained/2021-07-11/mobilenet_ssd_nms_postprocess_config.json", ssd) context_switch_param(mode=disabled) -post_quantization_optimization(finetune, policy=enabled, loss_factors=[0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 1.0, 0.1, 0.1], dataset_size=4000, epochs=8, learning_rate=0.0001, loss_types=[l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2]) +post_quantization_optimization(finetune, policy=enabled, loss_factors=[0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 1.0, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1], dataset_size=4000, epochs=8, learning_rate=0.0001, loss_types=[l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2], loss_layer_names=[bbox_decoder13, conv14, bbox_decoder17, conv18, bbox_decoder21, conv22, bbox_decoder25, conv26, bbox_decoder29, conv30, bbox_decoder33, conv34, conv32, conv24, conv28, conv12, conv20, conv16]) diff --git a/hailo_model_zoo/cfg/alls/base/tddfa_mobilenet_v1.alls b/hailo_model_zoo/cfg/alls/base/tddfa_mobilenet_v1.alls index 360c6ebf..7e942782 100644 --- a/hailo_model_zoo/cfg/alls/base/tddfa_mobilenet_v1.alls +++ b/hailo_model_zoo/cfg/alls/base/tddfa_mobilenet_v1.alls @@ -2,3 +2,4 @@ model_optimization_config(calibration, batch_size=2, calibset_size=64) post_quantization_optimization(finetune, policy=disabled) post_quantization_optimization(bias_correction, policy=enabled) input_conversion = input_conversion(bgr_to_rgb) +quantization_param({output_layer*}, precision_mode=a8_w8) diff --git a/hailo_model_zoo/cfg/alls/base/tiny_yolov3.alls b/hailo_model_zoo/cfg/alls/base/tiny_yolov3.alls index 17685cda..c0ad72eb 100644 --- a/hailo_model_zoo/cfg/alls/base/tiny_yolov3.alls +++ b/hailo_model_zoo/cfg/alls/base/tiny_yolov3.alls @@ -1,3 +1,3 @@ normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) model_optimization_config(calibration, batch_size=4, calibset_size=64) -post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=8, dataset_size=4000, loss_factors=[1.0, 1.0], loss_types=[l2, l2]) +post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=8, dataset_size=4000, loss_factors=[1.0, 1.0], loss_types=[l2, l2], loss_layer_names=[conv10, conv13]) diff --git a/hailo_model_zoo/cfg/alls/base/tiny_yolov4_license_plates_yuy2.alls b/hailo_model_zoo/cfg/alls/base/tiny_yolov4_license_plates_yuy2.alls new file mode 100644 index 00000000..495dbe69 --- /dev/null +++ b/hailo_model_zoo/cfg/alls/base/tiny_yolov4_license_plates_yuy2.alls @@ -0,0 +1,5 @@ +normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) +quantization_param(tiny_yolov4_license_plates_yuy2/output_layer1, precision_mode=a16_w16) +quantization_param(tiny_yolov4_license_plates_yuy2/output_layer2, precision_mode=a16_w16) +yuv_to_rgb1 = yuv_to_rgb() +reshape_yuy2 = input_conversion(tiny_yolov4_license_plates_yuy2/input_layer1, yuy2_to_hailo_yuv, emulator_support=True) diff --git a/hailo_model_zoo/cfg/alls/base/vit_base.alls b/hailo_model_zoo/cfg/alls/base/vit_base.alls new file mode 100644 index 00000000..0aa5f6d2 --- /dev/null +++ b/hailo_model_zoo/cfg/alls/base/vit_base.alls @@ -0,0 +1,41 @@ +allocator_param(disable_row_per_cut=True) +conv49_d0, conv49_d1, conv49_d2, conv49_d3, conv49_dc = defuse(conv49, 4) + +# model_optimization_config commands +model_optimization_config(calibration, batch_size=16, calibset_size=64) +post_quantization_optimization(finetune, policy=disabled) +pre_quantization_optimization(equalization, policy=disabled) + +pre_quantization_optimization(activation_clipping, layers=[conv49], mode=percentile, clipping_values=[0.01, 99.9]) +pre_quantization_optimization(activation_clipping, layers=[conv47], mode=percentile, clipping_values=[0.01, 99.5]) +pre_quantization_optimization(activation_clipping, layers=[conv43], mode=percentile, clipping_values=[0.01, 99.5]) +pre_quantization_optimization(activation_clipping, layers=[conv39], mode=percentile, clipping_values=[0.01, 99.7]) +pre_quantization_optimization(activation_clipping, layers=[conv37], mode=percentile, clipping_values=[0.01, 99.7]) +pre_quantization_optimization(activation_clipping, layers=[conv35], mode=percentile, clipping_values=[0.01, 99.7]) +pre_quantization_optimization(activation_clipping, layers=[conv33], mode=percentile, clipping_values=[0.01, 99.7]) +pre_quantization_optimization(activation_clipping, layers=[conv31], mode=percentile, clipping_values=[0.01, 99.7]) +pre_quantization_optimization(activation_clipping, layers=[conv29], mode=percentile, clipping_values=[0.01, 99.7]) +pre_quantization_optimization(activation_clipping, layers=[conv27], mode=percentile, clipping_values=[0.01, 99.7]) +pre_quantization_optimization(activation_clipping, layers=[conv25], mode=percentile, clipping_values=[0.01, 99.7]) +pre_quantization_optimization(activation_clipping, layers=[conv23], mode=percentile, clipping_values=[0.01, 99.7]) +pre_quantization_optimization(activation_clipping, layers=[conv21], mode=percentile, clipping_values=[0.01, 99.7]) +pre_quantization_optimization(activation_clipping, layers=[conv19], mode=percentile, clipping_values=[0.01, 99.7]) +pre_quantization_optimization(activation_clipping, layers=[conv17], mode=percentile, clipping_values=[0.01, 99.7]) +pre_quantization_optimization(activation_clipping, layers=[conv15], mode=percentile, clipping_values=[0.01, 99.7]) +pre_quantization_optimization(activation_clipping, layers=[conv13], mode=percentile, clipping_values=[0.01, 99.7]) +pre_quantization_optimization(activation_clipping, layers=[conv11], mode=percentile, clipping_values=[0.01, 99.7]) +pre_quantization_optimization(activation_clipping, layers=[conv9], mode=percentile, clipping_values=[0.01, 99.7]) +pre_quantization_optimization(activation_clipping, layers=[conv7], mode=percentile, clipping_values=[0.01, 99.7]) +pre_quantization_optimization(activation_clipping, layers=[conv5], mode=percentile, clipping_values=[0.01, 99.7]) +pre_quantization_optimization(activation_clipping, layers=[conv3], mode=percentile, clipping_values=[0.01, 99.7]) + +pre_quantization_optimization(activation_clipping, layers=[conv4], mode=percentile, clipping_values=[0.01, 99.99]) +pre_quantization_optimization(activation_clipping, layers=[conv8], mode=percentile, clipping_values=[0.01, 99.99]) +pre_quantization_optimization(activation_clipping, layers=[conv12], mode=percentile, clipping_values=[0.01, 99.99]) +pre_quantization_optimization(activation_clipping, layers=[conv16], mode=percentile, clipping_values=[0.01, 99.99]) +pre_quantization_optimization(activation_clipping, layers=[conv20], mode=percentile, clipping_values=[0.01, 99.99]) +pre_quantization_optimization(activation_clipping, layers=[conv24], mode=percentile, clipping_values=[0.01, 99.99]) +pre_quantization_optimization(activation_clipping, layers=[conv28], mode=percentile, clipping_values=[0.01, 99.99]) +pre_quantization_optimization(activation_clipping, layers=[conv40], mode=percentile, clipping_values=[0.01, 99.99]) +pre_quantization_optimization(activation_clipping, layers=[conv44], mode=percentile, clipping_values=[0.01, 99.99]) +pre_quantization_optimization(activation_clipping, layers=[conv48], mode=percentile, clipping_values=[0.01, 99.99]) diff --git a/hailo_model_zoo/cfg/alls/base/vit_tiny.alls b/hailo_model_zoo/cfg/alls/base/vit_tiny.alls index 22b64a0b..1d959e06 100644 --- a/hailo_model_zoo/cfg/alls/base/vit_tiny.alls +++ b/hailo_model_zoo/cfg/alls/base/vit_tiny.alls @@ -1,6 +1,6 @@ context_switch_param(mode=enabled) allocator_param(enable_partial_row_buffers=disabled) -reshape1 = format_conversion(ew_add1, conv2, conv3, conv4, conv5, reshape_1xw0_to_hxw) +reshape1 = format_conversion(ew_add1, conv_slice1, conv_slice2, conv_slice3, conv3, reshape_1xw0_to_hxw) allocator_param(automatic_reshapes=disabled) buffers(space_to_depth1, conv1, 0, PARTIAL_ROW) resources_param(strategy=greedy, max_compute_utilization=0.7, max_control_utilization=0.7, max_memory_utilization=0.7) @@ -12,5 +12,5 @@ post_quantization_optimization(finetune, policy=disabled) pre_quantization_optimization(equalization, policy=disabled) context_0 = context([input_layer1]) -context_1 = context([conv26, conv27, conv28]) -context_2 = context([conv50, conv51, conv52]) +context_1 = context([conv_slice13, conv_slice14, conv_slice15]) +context_2 = context([conv_slice25, conv_slice26, conv_slice27]) diff --git a/hailo_model_zoo/cfg/alls/base/yolact_mobilenet_v1.alls b/hailo_model_zoo/cfg/alls/base/yolact_mobilenet_v1.alls index df67dd9d..8d1ecc84 100644 --- a/hailo_model_zoo/cfg/alls/base/yolact_mobilenet_v1.alls +++ b/hailo_model_zoo/cfg/alls/base/yolact_mobilenet_v1.alls @@ -1,10 +1,10 @@ normalization1 = normalization([127.5, 127.5, 127.5], [127.5, 127.5, 127.5]) change_output_activation(yolact_mobilenet_v1/conv48, tanh) -change_output_activation(yolact_mobilenet_v1/conv41, tanh) -change_output_activation(yolact_mobilenet_v1/conv34, tanh) -change_output_activation(yolact_mobilenet_v1/conv29, tanh) -change_output_activation(yolact_mobilenet_v1/conv24, tanh) +change_output_activation(yolact_mobilenet_v1/conv42, tanh) +change_output_activation(yolact_mobilenet_v1/conv36, tanh) +change_output_activation(yolact_mobilenet_v1/conv31, tanh) +change_output_activation(yolact_mobilenet_v1/conv26, tanh) model_optimization_config(calibration, batch_size=1, calibset_size=64) -quantization_param([conv1, conv18, conv19, deconv1], bias_mode=single_scale_decomposition) +quantization_param([conv1, conv20, conv21, deconv1], bias_mode=single_scale_decomposition) post_quantization_optimization(finetune, policy=disabled) post_quantization_optimization(bias_correction, policy=enabled) diff --git a/hailo_model_zoo/cfg/alls/base/yolact_regnetx_1.6gf.alls b/hailo_model_zoo/cfg/alls/base/yolact_regnetx_1.6gf.alls index 6750207a..d9bcdc40 100644 --- a/hailo_model_zoo/cfg/alls/base/yolact_regnetx_1.6gf.alls +++ b/hailo_model_zoo/cfg/alls/base/yolact_regnetx_1.6gf.alls @@ -1,7 +1,7 @@ normalization1 = normalization([123.68, 116.78, 103.94], [58.4, 57.12, 57.38]) change_output_activation(yolact_regnetx_1_6gf/conv89, tanh) -change_output_activation(yolact_regnetx_1_6gf/conv81, tanh) -change_output_activation(yolact_regnetx_1_6gf/conv69, tanh) -change_output_activation(yolact_regnetx_1_6gf/conv72, tanh) -change_output_activation(yolact_regnetx_1_6gf/conv75, tanh) +change_output_activation(yolact_regnetx_1_6gf/conv82, tanh) +change_output_activation(yolact_regnetx_1_6gf/conv71, tanh) +change_output_activation(yolact_regnetx_1_6gf/conv74, tanh) +change_output_activation(yolact_regnetx_1_6gf/conv77, tanh) model_optimization_config(calibration, batch_size=4, calibset_size=64) diff --git a/hailo_model_zoo/cfg/alls/base/yolact_regnetx_600mf_d2s_31classes.alls b/hailo_model_zoo/cfg/alls/base/yolact_regnetx_600mf_d2s_31classes.alls index 75455332..2f44c93d 100644 --- a/hailo_model_zoo/cfg/alls/base/yolact_regnetx_600mf_d2s_31classes.alls +++ b/hailo_model_zoo/cfg/alls/base/yolact_regnetx_600mf_d2s_31classes.alls @@ -1,7 +1,7 @@ normalization1 = normalization([123.68, 116.78, 103.94], [58.4, 57.12, 57.38]) change_output_activation(yolact_regnetx_600mf_d2s_31classes/conv83, tanh) -change_output_activation(yolact_regnetx_600mf_d2s_31classes/conv75, tanh) -change_output_activation(yolact_regnetx_600mf_d2s_31classes/conv63, tanh) -change_output_activation(yolact_regnetx_600mf_d2s_31classes/conv66, tanh) -change_output_activation(yolact_regnetx_600mf_d2s_31classes/conv69, tanh) +change_output_activation(yolact_regnetx_600mf_d2s_31classes/conv76, tanh) +change_output_activation(yolact_regnetx_600mf_d2s_31classes/conv65, tanh) +change_output_activation(yolact_regnetx_600mf_d2s_31classes/conv68, tanh) +change_output_activation(yolact_regnetx_600mf_d2s_31classes/conv71, tanh) model_optimization_config(calibration, batch_size=4, calibset_size=256) diff --git a/hailo_model_zoo/cfg/alls/base/yolact_regnetx_800mf.alls b/hailo_model_zoo/cfg/alls/base/yolact_regnetx_800mf.alls index 75b6280e..cb3ad708 100644 --- a/hailo_model_zoo/cfg/alls/base/yolact_regnetx_800mf.alls +++ b/hailo_model_zoo/cfg/alls/base/yolact_regnetx_800mf.alls @@ -1,7 +1,7 @@ normalization1 = normalization([123.68, 116.78, 103.94], [58.4, 57.12, 57.38]) change_output_activation(yolact_regnetx_800mf/conv83, tanh) -change_output_activation(yolact_regnetx_800mf/conv75, tanh) -change_output_activation(yolact_regnetx_800mf/conv63, tanh) -change_output_activation(yolact_regnetx_800mf/conv66, tanh) -change_output_activation(yolact_regnetx_800mf/conv69, tanh) +change_output_activation(yolact_regnetx_800mf/conv76, tanh) +change_output_activation(yolact_regnetx_800mf/conv65, tanh) +change_output_activation(yolact_regnetx_800mf/conv68, tanh) +change_output_activation(yolact_regnetx_800mf/conv71, tanh) model_optimization_config(calibration, batch_size=4, calibset_size=64) diff --git a/hailo_model_zoo/cfg/alls/base/yolact_regnetx_800mf_20classes.alls b/hailo_model_zoo/cfg/alls/base/yolact_regnetx_800mf_20classes.alls index 5001bc2e..fbebd677 100644 --- a/hailo_model_zoo/cfg/alls/base/yolact_regnetx_800mf_20classes.alls +++ b/hailo_model_zoo/cfg/alls/base/yolact_regnetx_800mf_20classes.alls @@ -1,7 +1,8 @@ normalization1 = normalization([123.68, 116.78, 103.94], [58.4, 57.12, 57.38]) change_output_activation(yolact_regnetx_800mf_20classes/conv83, tanh) -change_output_activation(yolact_regnetx_800mf_20classes/conv75, tanh) -change_output_activation(yolact_regnetx_800mf_20classes/conv63, tanh) -change_output_activation(yolact_regnetx_800mf_20classes/conv66, tanh) -change_output_activation(yolact_regnetx_800mf_20classes/conv69, tanh) +change_output_activation(yolact_regnetx_800mf_20classes/conv76, tanh) +change_output_activation(yolact_regnetx_800mf_20classes/conv65, tanh) +change_output_activation(yolact_regnetx_800mf_20classes/conv68, tanh) +change_output_activation(yolact_regnetx_800mf_20classes/conv71, tanh) model_optimization_config(calibration, batch_size=4, calibset_size=64) +quantization_param({output_layer*}, precision_mode=a8_w8) \ No newline at end of file diff --git a/hailo_model_zoo/cfg/alls/base/yolov3.alls b/hailo_model_zoo/cfg/alls/base/yolov3.alls index eac63ee6..d6116b30 100644 --- a/hailo_model_zoo/cfg/alls/base/yolov3.alls +++ b/hailo_model_zoo/cfg/alls/base/yolov3.alls @@ -1,6 +1,7 @@ normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) model_optimization_config(calibration, batch_size=2, calibset_size=64) model_optimization_flavor(compression_level=2) +pre_quantization_optimization(ew_add_fusing, policy=enabled, infusible_ew_add_type=conv) post_quantization_optimization(finetune, policy=enabled, dataset_size=8192, epochs=8) quantization_param(output_layer1, precision_mode=a16_w16) quantization_param(output_layer2, precision_mode=a16_w16) diff --git a/hailo_model_zoo/cfg/alls/base/yolov3_gluon.alls b/hailo_model_zoo/cfg/alls/base/yolov3_gluon.alls index ed043d29..e96e320d 100644 --- a/hailo_model_zoo/cfg/alls/base/yolov3_gluon.alls +++ b/hailo_model_zoo/cfg/alls/base/yolov3_gluon.alls @@ -1,6 +1,7 @@ normalization1 = normalization([123.675, 116.28, 103.53], [58.395, 57.12, 57.375]) model_optimization_config(calibration, batch_size=2, calibset_size=64) model_optimization_flavor(compression_level=2) +pre_quantization_optimization(ew_add_fusing, policy=enabled, infusible_ew_add_type=conv) post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=8, dataset_size=4000) quantization_param(output_layer1, precision_mode=a16_w16) quantization_param(output_layer2, precision_mode=a16_w16) diff --git a/hailo_model_zoo/cfg/alls/base/yolov5l.alls b/hailo_model_zoo/cfg/alls/base/yolov5l.alls index 39910355..5a868b19 100644 --- a/hailo_model_zoo/cfg/alls/base/yolov5l.alls +++ b/hailo_model_zoo/cfg/alls/base/yolov5l.alls @@ -1,4 +1,5 @@ normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) change_output_activation(sigmoid) model_optimization_config(calibration, batch_size=2, calibset_size=64) -post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=2, dataset_size=1000, loss_factors=[1.0, 1.0, 1.0], loss_types=[l2rel, l2rel, l2rel], loss_layer_names=[conv139, conv128, conv116]) +pre_quantization_optimization(ew_add_fusing, policy=enabled, infusible_ew_add_type=conv) +post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=2, dataset_size=1000, loss_factors=[1.0, 1.0, 1.0], loss_types=[l2rel, l2rel, l2rel], loss_layer_names=[conv118, conv107, conv95]) diff --git a/hailo_model_zoo/cfg/alls/base/yolov5m.alls b/hailo_model_zoo/cfg/alls/base/yolov5m.alls index 84095118..50e18c66 100644 --- a/hailo_model_zoo/cfg/alls/base/yolov5m.alls +++ b/hailo_model_zoo/cfg/alls/base/yolov5m.alls @@ -8,4 +8,8 @@ quantization_param(conv53, precision_mode=a8_w4) quantization_param(conv84, precision_mode=a8_w4) quantization_param(conv89, precision_mode=a8_w4) quantization_param(conv91, precision_mode=a8_w4) + +# post_quantization_optimization commands post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=4, dataset_size=4000, loss_factors=[1.0, 1.0, 1.0], loss_types=[l2rel, l2rel, l2rel], loss_layer_names=[conv94, conv85, conv75]) +allocator_param(merge_min_layer_utilization=0.1) +performance_param(fps=150) \ No newline at end of file diff --git a/hailo_model_zoo/cfg/alls/base/yolov5m_wo_spp_yuv.alls b/hailo_model_zoo/cfg/alls/base/yolov5m_hpp.alls similarity index 52% rename from hailo_model_zoo/cfg/alls/base/yolov5m_wo_spp_yuv.alls rename to hailo_model_zoo/cfg/alls/base/yolov5m_hpp.alls index 287fc36e..5fc80d41 100644 --- a/hailo_model_zoo/cfg/alls/base/yolov5m_wo_spp_yuv.alls +++ b/hailo_model_zoo/cfg/alls/base/yolov5m_hpp.alls @@ -1,13 +1,12 @@ normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) change_output_activation(sigmoid) -yuv_to_rgb1 = yuv_to_rgb(input_layer1) -resize_input1 = resize_input([720, 1280], input_layer1) +nms_postprocess("$HMZ_DATA/models_files/ObjectDetection/Detection-COCO/yolo/yolov5m_spp/pretrained/2022-01-02/yolov5m_nms_config.json", yolov5, engine=cpu) model_optimization_config(calibration, batch_size=4, calibset_size=64) quantization_param(conv45, precision_mode=a8_w4) quantization_param(conv46, precision_mode=a8_w4) -quantization_param(conv50, precision_mode=a8_w4) -quantization_param(conv52, precision_mode=a8_w4) -quantization_param(conv83, precision_mode=a8_w4) -quantization_param(conv88, precision_mode=a8_w4) -quantization_param(conv90, precision_mode=a8_w4) -post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=4, dataset_size=4000, loss_factors=[1.0, 1.0, 1.0], loss_types=[l2rel, l2rel, l2rel], loss_layer_names=[conv93, conv84, conv74]) +quantization_param(conv51, precision_mode=a8_w4) +quantization_param(conv53, precision_mode=a8_w4) +quantization_param(conv84, precision_mode=a8_w4) +quantization_param(conv89, precision_mode=a8_w4) +quantization_param(conv91, precision_mode=a8_w4) +post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=4, dataset_size=4000, loss_factors=[1.0, 1.0, 1.0], loss_types=[l2rel, l2rel, l2rel], loss_layer_names=[conv94, conv85, conv75]) diff --git a/hailo_model_zoo/cfg/alls/base/yolov5m_vehicles.alls b/hailo_model_zoo/cfg/alls/base/yolov5m_vehicles.alls index 6fcf7566..9f24ae5b 100644 --- a/hailo_model_zoo/cfg/alls/base/yolov5m_vehicles.alls +++ b/hailo_model_zoo/cfg/alls/base/yolov5m_vehicles.alls @@ -9,4 +9,5 @@ quantization_param(conv53, precision_mode=a8_w4) quantization_param(conv84, precision_mode=a8_w4) quantization_param(conv89, precision_mode=a8_w4) quantization_param(conv91, precision_mode=a8_w4) +quantization_param({output_layer*}, precision_mode=a8_w8) post_quantization_optimization(finetune, policy=enabled, use_acceleras=disabled, learning_rate=0.0001, epochs=4, dataset_size=4000, loss_factors=[1.0, 1.0, 1.0], loss_types=[l2rel, l2rel, l2rel], loss_layer_names=[conv94, conv85, conv75]) diff --git a/hailo_model_zoo/cfg/alls/base/yolov5m_vehicles_yuy2.alls b/hailo_model_zoo/cfg/alls/base/yolov5m_vehicles_yuy2.alls new file mode 100644 index 00000000..6ef48d0f --- /dev/null +++ b/hailo_model_zoo/cfg/alls/base/yolov5m_vehicles_yuy2.alls @@ -0,0 +1,15 @@ +normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) +change_output_activation(sigmoid) +resize_input1 = resize_input([1080, 1920], input_layer1) +model_optimization_config(calibration, batch_size=4, calibset_size=64) +quantization_param(conv45, precision_mode=a8_w4) +quantization_param(conv46, precision_mode=a8_w4) +quantization_param(conv51, precision_mode=a8_w4) +quantization_param(conv53, precision_mode=a8_w4) +quantization_param(conv84, precision_mode=a8_w4) +quantization_param(conv89, precision_mode=a8_w4) +quantization_param(conv91, precision_mode=a8_w4) +quantization_param({output_layer*}, precision_mode=a8_w8) +post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=4, dataset_size=4000, loss_factors=[1.0, 1.0, 1.0], loss_types=[l2rel, l2rel, l2rel], loss_layer_names=[conv94, conv85, conv75]) +yuv_to_rgb1 = yuv_to_rgb() +reshape_yuy2 = input_conversion(input_layer1, yuy2_to_hailo_yuv, emulator_support=True) diff --git a/hailo_model_zoo/cfg/alls/performance/yolov5m_wo_spp_yuv.alls b/hailo_model_zoo/cfg/alls/base/yolov5m_wo_spp_yuy2.alls similarity index 87% rename from hailo_model_zoo/cfg/alls/performance/yolov5m_wo_spp_yuv.alls rename to hailo_model_zoo/cfg/alls/base/yolov5m_wo_spp_yuy2.alls index 37f24824..3c0be92c 100644 --- a/hailo_model_zoo/cfg/alls/performance/yolov5m_wo_spp_yuv.alls +++ b/hailo_model_zoo/cfg/alls/base/yolov5m_wo_spp_yuy2.alls @@ -2,6 +2,8 @@ normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) change_output_activation(sigmoid) yuv_to_rgb1 = yuv_to_rgb(input_layer1) resize_input1 = resize_input([720, 1280], input_layer1) +reshape_yuy2 = input_conversion(input_layer1, yuy2_to_hailo_yuv, emulator_support=True) + model_optimization_config(calibration, batch_size=4, calibset_size=64) quantization_param(conv45, precision_mode=a8_w4) quantization_param(conv46, precision_mode=a8_w4) @@ -14,11 +16,7 @@ quantization_param(conv90, precision_mode=a8_w4) # post_quantization_optimization commands post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=4, dataset_size=4000, loss_factors=[1.0, 1.0, 1.0], loss_types=[l2rel, l2rel, l2rel], loss_layer_names=[conv93, conv84, conv74]) -allocator_param(timeout=100000, cluster_timeout=100000, automatic_ddr=disabled) -allocator_param(merge_min_layer_utilization=0.1) - conv45_d0, conv45_d1, conv45_dc = defuse(conv45, 2) conv46_d0, conv46_d1, conv46_dc = defuse(conv46, 2) compilation_param(resize_input1, resize_bilinear_streaming=True) -performance_param(fps=100) diff --git a/hailo_model_zoo/cfg/alls/base/yolov5n6_6.1.alls b/hailo_model_zoo/cfg/alls/base/yolov5n6_6.1.alls index a9e1e4c5..e03e662a 100644 --- a/hailo_model_zoo/cfg/alls/base/yolov5n6_6.1.alls +++ b/hailo_model_zoo/cfg/alls/base/yolov5n6_6.1.alls @@ -1,3 +1,4 @@ normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) change_output_activation(sigmoid) -post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=4, dataset_size=4000, loss_factors=[1.0, 1.0, 1.0, 1.0], loss_types=[l2rel, l2rel, l2rel, l2rel], loss_layer_names=[conv79, conv73, conv66, conv59]) +model_optimization_config(calibration, batch_size=2) +post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=4, batch_size=2, dataset_size=4000, loss_factors=[1.0, 1.0, 1.0, 1.0], loss_types=[l2rel, l2rel, l2rel, l2rel], loss_layer_names=[conv79, conv73, conv66, conv59]) diff --git a/hailo_model_zoo/cfg/alls/base/yolov5s.alls b/hailo_model_zoo/cfg/alls/base/yolov5s.alls index f78d3939..c2473039 100644 --- a/hailo_model_zoo/cfg/alls/base/yolov5s.alls +++ b/hailo_model_zoo/cfg/alls/base/yolov5s.alls @@ -2,3 +2,4 @@ normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) change_output_activation(sigmoid) model_optimization_config(calibration, batch_size=4, calibset_size=64) pre_quantization_optimization(equalization, policy=disabled) +quantization_param({output_layer*}, precision_mode=a8_w8) \ No newline at end of file diff --git a/hailo_model_zoo/cfg/alls/base/yolov5s6_6.1.alls b/hailo_model_zoo/cfg/alls/base/yolov5s6_6.1.alls index 09f6f8ee..820fe135 100644 --- a/hailo_model_zoo/cfg/alls/base/yolov5s6_6.1.alls +++ b/hailo_model_zoo/cfg/alls/base/yolov5s6_6.1.alls @@ -1,4 +1,4 @@ normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) change_output_activation(sigmoid) -model_optimization_config(calibration, batch_size=4, calibset_size=64) +model_optimization_config(calibration, batch_size=2, calibset_size=64) post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=4, dataset_size=4000, loss_factors=[1.0, 1.0, 1.0, 1.0], loss_types=[l2rel, l2rel, l2rel, l2rel], loss_layer_names=[conv79, conv73, conv66, conv59]) diff --git a/hailo_model_zoo/cfg/alls/base/yolov5s_c3tr.alls b/hailo_model_zoo/cfg/alls/base/yolov5s_c3tr.alls new file mode 100644 index 00000000..732b5df5 --- /dev/null +++ b/hailo_model_zoo/cfg/alls/base/yolov5s_c3tr.alls @@ -0,0 +1,3 @@ +normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) +change_output_activation(sigmoid) +model_optimization_config(calibration, batch_size=4, calibset_size=64) diff --git a/hailo_model_zoo/cfg/alls/base/yolov5xs_wo_spp_nms.alls b/hailo_model_zoo/cfg/alls/base/yolov5xs_wo_spp_nms.alls index 5f94170e..80c7ff62 100644 --- a/hailo_model_zoo/cfg/alls/base/yolov5xs_wo_spp_nms.alls +++ b/hailo_model_zoo/cfg/alls/base/yolov5xs_wo_spp_nms.alls @@ -1,6 +1,6 @@ normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) change_output_activation(sigmoid) -nms_postprocess("$HMZ_DATA/models_files/ObjectDetection/Detection-COCO/yolo/yolov5xs/pretrained/2022-05-10/yolov5xs_wo_spp_nms_config.json", yolo) +nms_postprocess("$HMZ_DATA/models_files/ObjectDetection/Detection-COCO/yolo/yolov5xs/pretrained/2022-05-10/yolov5xs_wo_spp_nms_config.json", yolov5) model_optimization_config(calibration, batch_size=2, calibset_size=64) post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=4, dataset_size=4000, loss_types=[l2rel, l2rel, l2rel], loss_layer_names=[conv69, conv61, conv53]) context_switch_param(mode=disabled) diff --git a/hailo_model_zoo/cfg/alls/base/yolov7.alls b/hailo_model_zoo/cfg/alls/base/yolov7.alls index 20623706..e42353ef 100644 --- a/hailo_model_zoo/cfg/alls/base/yolov7.alls +++ b/hailo_model_zoo/cfg/alls/base/yolov7.alls @@ -1,7 +1,7 @@ normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) change_output_activation(sigmoid) -model_optimization_config(calibration, batch_size=4, calibset_size=512) +model_optimization_config(calibration, batch_size=2, calibset_size=512) pre_quantization_optimization(activation_clipping, layers={*}, mode=percentile, clipping_values=[0, 99.99]) pre_quantization_optimization(activation_clipping, layers=[yolov7/maxpool1, yolov7/maxpool2, yolov7/maxpool3, yolov7/maxpool4, yolov7/maxpool5, yolov7/maxpool6, yolov7/maxpool7, yolov7/maxpool8, yolov7/maxpool8], mode=disabled) pre_quantization_optimization(activation_clipping, layers=[yolov7/resize1,yolov7/resize2], mode=disabled) -post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=6, dataset_size=4000) +post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=6, dataset_size=4000, batch_size=4) diff --git a/hailo_model_zoo/cfg/alls/base/yolov7e6.alls b/hailo_model_zoo/cfg/alls/base/yolov7e6.alls index c89f8741..b51369a8 100644 --- a/hailo_model_zoo/cfg/alls/base/yolov7e6.alls +++ b/hailo_model_zoo/cfg/alls/base/yolov7e6.alls @@ -1,4 +1,5 @@ normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) change_output_activation(sigmoid) -model_optimization_config(calibration, batch_size=2, calibset_size=64) +model_optimization_config(calibration, batch_size=1, calibset_size=64) post_quantization_optimization(finetune, policy=enabled, batch_size=1) +model_optimization_config(checker_cfg, policy=disabled) \ No newline at end of file diff --git a/hailo_model_zoo/cfg/alls/base/yolov8l.alls b/hailo_model_zoo/cfg/alls/base/yolov8l.alls new file mode 100644 index 00000000..67623067 --- /dev/null +++ b/hailo_model_zoo/cfg/alls/base/yolov8l.alls @@ -0,0 +1,15 @@ +normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) +model_optimization_config(calibration, batch_size=2) +post_quantization_optimization(finetune, policy=enabled, loss_layer_names = [conv97, conv82, conv67, conv25, conv100, conv88, conv73, conv103, conv89, conv74], + loss_types = [l2rel,l2rel,l2rel,l2rel,l2rel,l2rel,l2rel,ce,ce,ce], + loss_factors=[1,1,1,1,2,2,2,2,2,2], epochs = 4, batch_size=2) +model_optimization_flavor(compression_level=0) +quantization_param(output_layer1, precision_mode=a16_w16) +quantization_param(output_layer2, precision_mode=a16_w16) +quantization_param(output_layer3, precision_mode=a16_w16) +quantization_param(output_layer4, precision_mode=a16_w16) +quantization_param(output_layer5, precision_mode=a16_w16) +quantization_param(output_layer6, precision_mode=a16_w16) +change_output_activation(conv74, sigmoid) +change_output_activation(conv89, sigmoid) +change_output_activation(conv103, sigmoid) diff --git a/hailo_model_zoo/cfg/alls/base/yolov8m.alls b/hailo_model_zoo/cfg/alls/base/yolov8m.alls new file mode 100644 index 00000000..ae1ab478 --- /dev/null +++ b/hailo_model_zoo/cfg/alls/base/yolov8m.alls @@ -0,0 +1,5 @@ +normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) +model_optimization_config(calibration, batch_size=2) +change_output_activation(conv58, sigmoid) +change_output_activation(conv71, sigmoid) +change_output_activation(conv83, sigmoid) diff --git a/hailo_model_zoo/cfg/alls/base/yolov8n.alls b/hailo_model_zoo/cfg/alls/base/yolov8n.alls new file mode 100644 index 00000000..10db7ab7 --- /dev/null +++ b/hailo_model_zoo/cfg/alls/base/yolov8n.alls @@ -0,0 +1,4 @@ +normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) +change_output_activation(conv42, sigmoid) +change_output_activation(conv53, sigmoid) +change_output_activation(conv63, sigmoid) diff --git a/hailo_model_zoo/cfg/alls/base/yolov8s.alls b/hailo_model_zoo/cfg/alls/base/yolov8s.alls new file mode 100644 index 00000000..10db7ab7 --- /dev/null +++ b/hailo_model_zoo/cfg/alls/base/yolov8s.alls @@ -0,0 +1,4 @@ +normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) +change_output_activation(conv42, sigmoid) +change_output_activation(conv53, sigmoid) +change_output_activation(conv63, sigmoid) diff --git a/hailo_model_zoo/cfg/alls/base/yolov8x.alls b/hailo_model_zoo/cfg/alls/base/yolov8x.alls new file mode 100644 index 00000000..67623067 --- /dev/null +++ b/hailo_model_zoo/cfg/alls/base/yolov8x.alls @@ -0,0 +1,15 @@ +normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) +model_optimization_config(calibration, batch_size=2) +post_quantization_optimization(finetune, policy=enabled, loss_layer_names = [conv97, conv82, conv67, conv25, conv100, conv88, conv73, conv103, conv89, conv74], + loss_types = [l2rel,l2rel,l2rel,l2rel,l2rel,l2rel,l2rel,ce,ce,ce], + loss_factors=[1,1,1,1,2,2,2,2,2,2], epochs = 4, batch_size=2) +model_optimization_flavor(compression_level=0) +quantization_param(output_layer1, precision_mode=a16_w16) +quantization_param(output_layer2, precision_mode=a16_w16) +quantization_param(output_layer3, precision_mode=a16_w16) +quantization_param(output_layer4, precision_mode=a16_w16) +quantization_param(output_layer5, precision_mode=a16_w16) +quantization_param(output_layer6, precision_mode=a16_w16) +change_output_activation(conv74, sigmoid) +change_output_activation(conv89, sigmoid) +change_output_activation(conv103, sigmoid) diff --git a/hailo_model_zoo/cfg/alls/base/yolox_l_leaky.alls b/hailo_model_zoo/cfg/alls/base/yolox_l_leaky.alls index d055d301..6e95f666 100644 --- a/hailo_model_zoo/cfg/alls/base/yolox_l_leaky.alls +++ b/hailo_model_zoo/cfg/alls/base/yolox_l_leaky.alls @@ -1,5 +1,7 @@ model_optimization_config(calibration, use_acceleras=disabled, batch_size=2, calibset_size=64) + input_conversion = input_conversion(bgr_to_rgb) + pre_quantization_optimization(activation_clipping, layers=[conv93], mode=percentile_force, clipping_values=[0.0001,99.6]) pre_quantization_optimization(activation_clipping, layers=[conv95], mode=percentile_force, clipping_values=[0.15,99.4]) pre_quantization_optimization(activation_clipping, layers=[conv111], mode=percentile_force, clipping_values=[0.0001,99.5]) @@ -7,3 +9,4 @@ pre_quantization_optimization(activation_clipping, layers=[conv113], mode=percen pre_quantization_optimization(activation_clipping, layers=[conv128], mode=percentile_force, clipping_values=[0.0001,99.5]) pre_quantization_optimization(activation_clipping, layers=[conv130], mode=percentile_force, clipping_values=[0.15,99.4]) post_quantization_optimization(finetune, policy=enabled, use_acceleras=disabled, loss_factors=[0.125, 2, 0.25, 0.125, 2, 0.25, 0.125, 2, 0.25], dataset_size=4000, epochs=8, learning_rate=1e-05, loss_layer_names=[conv94, conv95, conv96, conv112, conv113, conv114, conv129, conv130, conv131], loss_types=[l2, l2, l2, l2, l2, l2, l2, l2, l2]) +quantization_param({output_layer*}, precision_mode=a8_w8) \ No newline at end of file diff --git a/hailo_model_zoo/cfg/alls/performance/centerpose_regnetx_1.6gf_fpn.alls b/hailo_model_zoo/cfg/alls/performance/centerpose_regnetx_1.6gf_fpn.alls index 98025d7a..7bbb4314 100644 --- a/hailo_model_zoo/cfg/alls/performance/centerpose_regnetx_1.6gf_fpn.alls +++ b/hailo_model_zoo/cfg/alls/performance/centerpose_regnetx_1.6gf_fpn.alls @@ -2,4 +2,7 @@ centerpose_regnetx_1_6gf_fpn/normalization1 = normalization([119.85, 113.985, 10 model_optimization_config(calibration, batch_size=4, calibset_size=32) # pre_quantization_optimization commands pre_quantization_optimization(equalization, policy=disabled) +quantization_param({*output_layer*}, precision_mode=a8_w8) performance_param(fps=130) +change_output_activation(centerpose_regnetx_1_6gf_fpn/conv79, sigmoid) +change_output_activation(centerpose_regnetx_1_6gf_fpn/conv75, sigmoid) \ No newline at end of file diff --git a/hailo_model_zoo/cfg/alls/performance/centerpose_regnetx_200mf_fpn_common.alls b/hailo_model_zoo/cfg/alls/performance/centerpose_regnetx_200mf_fpn_common.alls deleted file mode 100644 index df67963b..00000000 --- a/hailo_model_zoo/cfg/alls/performance/centerpose_regnetx_200mf_fpn_common.alls +++ /dev/null @@ -1,2 +0,0 @@ -model_optimization_config(calibration, batch_size=8, calibset_size=2048) -input_conversion = input_conversion(bgr_to_rgb) diff --git a/hailo_model_zoo/cfg/alls/performance/centerpose_repvgg_a0.alls b/hailo_model_zoo/cfg/alls/performance/centerpose_repvgg_a0.alls index eb2623d3..ff7ab594 100644 --- a/hailo_model_zoo/cfg/alls/performance/centerpose_repvgg_a0.alls +++ b/hailo_model_zoo/cfg/alls/performance/centerpose_repvgg_a0.alls @@ -1,5 +1,7 @@ centerpose_repvgg_a0/normalization1 = normalization([119.85, 113.985, 104.04], [70.89, 69.87, 73.695], centerpose_repvgg_a0/input_layer1) model_optimization_config(calibration, batch_size=8, calibset_size=2048) +change_output_activation(centerpose_repvgg_a0/conv40, sigmoid) +change_output_activation(centerpose_repvgg_a0/conv36, sigmoid) allocator_param(merge_min_layer_utilization=0.1) allocator_param(automatic_ddr=disabled) performance_param(fps=500) diff --git a/hailo_model_zoo/cfg/alls/performance/lightface_slim.alls b/hailo_model_zoo/cfg/alls/performance/lightface_slim.alls index a2cff089..e120a8b7 100644 --- a/hailo_model_zoo/cfg/alls/performance/lightface_slim.alls +++ b/hailo_model_zoo/cfg/alls/performance/lightface_slim.alls @@ -2,4 +2,4 @@ normalization1 = normalization([127.0, 127.0, 127.0], [128.0, 128.0, 128.0]) model_optimization_config(calibration, batch_size=8, calibset_size=16) post_quantization_optimization(finetune, policy=disabled) allocator_param(automatic_ddr=disabled) - +quantization_param({output_layer*}, precision_mode=a8_w8) diff --git a/hailo_model_zoo/cfg/alls/performance/lprnet.alls b/hailo_model_zoo/cfg/alls/performance/lprnet.alls index 004452f7..a8652eea 100644 --- a/hailo_model_zoo/cfg/alls/performance/lprnet.alls +++ b/hailo_model_zoo/cfg/alls/performance/lprnet.alls @@ -1,4 +1,3 @@ lprnet/normalization1 = normalization([127.5, 127.5, 127.5], [128.0, 128.0, 128.0]) input_conversion = input_conversion(bgr_to_rgb) - performance_param(fps=330) diff --git a/hailo_model_zoo/cfg/alls/performance/lprnet_yuy2.alls b/hailo_model_zoo/cfg/alls/performance/lprnet_yuy2.alls new file mode 100644 index 00000000..0d8f14b8 --- /dev/null +++ b/hailo_model_zoo/cfg/alls/performance/lprnet_yuy2.alls @@ -0,0 +1,8 @@ +input_conversion = input_conversion(bgr_to_rgb) +normalization1 = normalization([127.5, 127.5, 127.5], [128.0, 128.0, 128.0]) +yuv_to_rgb1 = yuv_to_rgb() +reshape_yuy2 = input_conversion(lprnet_yuy2/input_layer1, yuy2_to_hailo_yuv, emulator_support=True) +model_optimization_config(calibration, batch_size=4, calibset_size=64) +pre_quantization_optimization(equalization, policy=disabled) +allocator_param(automatic_ddr=disabled) +performance_param(fps=330) diff --git a/hailo_model_zoo/cfg/alls/performance/resnet_v1_50.alls b/hailo_model_zoo/cfg/alls/performance/resnet_v1_50.alls index d1c66a73..eb25210a 100644 --- a/hailo_model_zoo/cfg/alls/performance/resnet_v1_50.alls +++ b/hailo_model_zoo/cfg/alls/performance/resnet_v1_50.alls @@ -1,225 +1,5 @@ normalization1 = normalization([123.68, 116.78, 103.94], [1.0, 1.0, 1.0]) - -# quantization_param({conv*}, max_elementwise_feed_repeat=1) -quantization_param([conv5, conv8, conv11, conv15, conv18, conv21, conv24, conv28, conv31, conv34, conv37, conv40, conv43, conv47, conv50, conv53], max_elementwise_feed_repeat=1) - +quantization_param({conv*}, max_elementwise_feed_repeat=1) post_quantization_optimization(finetune, policy=disabled) - -allocator_param(automatic_ddr=disabled, automatic_ddr_l3_cuts_threshold=25, automatic_l4_portals=False, automatic_l4_portals_l3_cuts_threshold=15, cluster_timeout=300, defuse_by_validator=False, enable_axis_upsize=True, enable_collapser=False, enable_defuse_with_slack=False, enable_fixer=False, enable_swapper=False, max_auto_defuse=1, max_auto_merge_depth=1, merge_min_layer_utilization=1.0, seed=0, timeout=3600, validate=True) -buffer_calc_param(buffer_calc_fps=0.0, optimize_buffers=True, split_aware_optimize_buffers=True) -context_switch_param(max_compute_utilization=0.75, max_control_utilization=0.75, max_memory_utilization=0.75, max_utilization=1.0, mode=allowed) - -conv45_d0, conv45_d1, conv45_d2, conv45_dc = defuse(conv45, 3) -conv49_d0, conv49_d1, conv49_d2, conv49_dc = defuse(conv49, 3) -conv1_sd0, conv1_sd1, conv1_sdc = defuse(conv1, 2, defuse_type=SPATIAL) -maxpool1_d0, maxpool1_d1, maxpool1_dc = defuse(maxpool1, 2) -conv5_d0, conv5_d1, conv5_dc = defuse(conv5, 2) -conv8_d0, conv8_d1, conv8_dc = defuse(conv8, 2) -conv46_d0, conv46_d1, conv46_dc = defuse(conv46, 2) -conv52_d0, conv52_d1, conv52_dc = defuse(conv52, 2) -fc1_d0, fc1_d1, fc1_dc = defuse(fc1, 2) - -compilation_param(avgpool1, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=2, number_of_subclusters=2, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv10, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=3, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv11, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=2, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv12, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=3, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv13, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv14, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv15, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv16, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=6, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv17, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv18, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv19, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=6, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv1_sd0, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=True, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=2, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=enabled) -compilation_param(conv1_sd1, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=True, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=2, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=enabled) -compilation_param(conv1_sdc, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=True, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=1, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv2, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=1, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv20, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv21, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv22, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=6, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv23, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv24, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=5, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv25, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv26, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv27, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv28, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv29, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv3, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=2, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=enabled) -compilation_param(conv30, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv31, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv32, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv33, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv34, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv35, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv36, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv37, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv38, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv39, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv4, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=6, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv40, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv41, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv42, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv43, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv44, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv45_d0, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv45_d1, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv45_d2, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv45_dc, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=1, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv46_d0, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv46_d1, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv46_dc, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=1, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv47, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv48, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv49_d0, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv49_d1, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv49_d2, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv49_dc, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=1, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv50, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv51, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv52_d0, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv52_d1, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv52_dc, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=1, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv53, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv5_d0, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=4, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv5_d1, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=4, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv5_dc, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=1, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv6, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=4, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv7, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=2, number_of_subclusters=8, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=enabled) -compilation_param(conv8_d0, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=4, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv8_d1, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=4, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv8_dc, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=1, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(conv9, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=4, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(fc1_d0, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=3, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(fc1_d1, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=3, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(fc1_dc, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=1, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(maxpool1_d0, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=1, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(maxpool1_d1, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=1, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(maxpool1_dc, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=1, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(maxpool2, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=1, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(maxpool3, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=1, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(maxpool4, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=1, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(normalization1, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=1, number_of_input_aligners=1, number_of_subclusters=1, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) -compilation_param(softmax1, balance_output_multisplit=enabled, buffer_in_l4=False, defuse_num_layers=1, defuse_spatial_w=False, layer_output_buffers_format=consecutive, mixed_mem=True, no_contexts=allowed, number_of_apus=0, number_of_input_aligners=0, number_of_subclusters=0, resize_bilinear_streaming=disabled, resources_allocation_strategy=manual_scs_selection, use_16x4_sc=disabled) - -buffers(avgpool1, fc1_d0, 2, FULL_ROW) -buffers(avgpool1, fc1_d1, 2, 2, FULL_ROW) -buffers(conv10, conv11, 2, FULL_ROW) -buffers(conv11, conv12, 2, FULL_ROW) -buffers(conv11, conv14, 2, FULL_ROW) -buffers(conv12, conv13, 2, FULL_ROW) -buffers(conv13, conv15, 2, 2, FULL_ROW) -buffers(conv14, conv15, 2, 3, FULL_ROW) -buffers(conv15, conv16, 5, FULL_ROW) -buffers(conv15, conv18, 5, FULL_ROW) -buffers(conv16, conv17, 1, 2, FULL_ROW) -buffers(conv17, conv18, 2, 2, FULL_ROW) -buffers(conv18, conv19, 2, FULL_ROW) -buffers(conv18, conv21, 2, 4, FULL_ROW) -buffers(conv19, conv20, 1, 2, FULL_ROW) -buffers(conv1_sd0, conv1_sdc, 1, 2, FULL_ROW) -buffers(conv1_sd1, conv1_sdc, 1, 2, FULL_ROW) -buffers(conv1_sdc, maxpool1_d0, 5, FULL_ROW) -buffers(conv1_sdc, maxpool1_d1, 5, FULL_ROW) -buffers(conv2, conv3, 2, 2, FULL_ROW) -buffers(conv20, conv21, 2, 2, FULL_ROW) -buffers(conv21, conv22, 2, FULL_ROW) -buffers(conv21, maxpool3, 2, 2, FULL_ROW) -buffers(conv22, conv23, 1, 2, FULL_ROW) -buffers(conv23, conv24, 1, FULL_ROW) -buffers(conv24, conv25, 2, FULL_ROW) -buffers(conv24, conv27, 2, FULL_ROW) -buffers(conv25, conv26, 1, 2, FULL_ROW) -buffers(conv26, conv28, 2, FULL_ROW) -buffers(conv27, conv28, 1, 3, FULL_ROW) -buffers(conv28, conv29, 2, 2, FULL_ROW) -buffers(conv28, conv31, 2, 4, FULL_ROW) -buffers(conv29, conv30, 1, 2, FULL_ROW) -buffers(conv3, conv5_d0, 2, 2, FULL_ROW) -buffers(conv3, conv5_d1, 2, FULL_ROW) -buffers(conv30, conv31, 2, FULL_ROW) -buffers(conv31, conv32, 5, FULL_ROW) -buffers(conv31, conv34, 5, FULL_ROW) -buffers(conv32, conv33, 1, 2, FULL_ROW) -buffers(conv33, conv34, 2, 2, FULL_ROW) -buffers(conv34, conv35, 2, FULL_ROW) -buffers(conv34, conv37, 2, 4, FULL_ROW) -buffers(conv35, conv36, 2, FULL_ROW) -buffers(conv36, conv37, 2, 2, FULL_ROW) -buffers(conv37, conv38, 2, FULL_ROW) -buffers(conv37, conv40, 2, 4, FULL_ROW) -buffers(conv38, conv39, 2, FULL_ROW) -buffers(conv39, conv40, 2, 2, FULL_ROW) -buffers(conv4, conv5_d0, 5, FULL_ROW) -buffers(conv4, conv5_d1, 5, 4, FULL_ROW) -buffers(conv40, conv41, 2, 2, FULL_ROW) -buffers(conv40, maxpool4, 2, 2, FULL_ROW) -buffers(conv41, conv42, 1, FULL_ROW) -buffers(conv42, conv43, 1, FULL_ROW) -buffers(conv43, conv44, 1, 2, FULL_ROW) -buffers(conv43, conv46_d0, 1, 2, FULL_ROW) -buffers(conv43, conv46_d1, 1, 2, FULL_ROW) -buffers(conv44, conv45_d0, 1, 2, FULL_ROW) -buffers(conv44, conv45_d1, 1, FULL_ROW) -buffers(conv44, conv45_d2, 1, FULL_ROW) -buffers(conv45_d0, conv45_dc, 2, 2, FULL_ROW) -buffers(conv45_d1, conv45_dc, 2, FULL_ROW) -buffers(conv45_d2, conv45_dc, 2, FULL_ROW) -buffers(conv45_dc, conv47, 2, 2, FULL_ROW) -buffers(conv46_d0, conv46_dc, 1, 2, FULL_ROW) -buffers(conv46_d1, conv46_dc, 1, 2, FULL_ROW) -buffers(conv46_dc, conv47, 1, 2, FULL_ROW) -buffers(conv47, conv48, 2, FULL_ROW) -buffers(conv47, conv50, 2, 3, FULL_ROW) -buffers(conv48, conv49_d0, 1, 2, FULL_ROW) -buffers(conv48, conv49_d1, 1, 2, FULL_ROW) -buffers(conv48, conv49_d2, 1, 2, FULL_ROW) -buffers(conv49_d0, conv49_dc, 2, FULL_ROW) -buffers(conv49_d1, conv49_dc, 2, FULL_ROW) -buffers(conv49_d2, conv49_dc, 2, FULL_ROW) -buffers(conv49_dc, conv50, 2, FULL_ROW) -buffers(conv50, conv51, 1, 2, FULL_ROW) -buffers(conv50, conv53, 1, 4, FULL_ROW) -buffers(conv51, conv52_d0, 1, 2, FULL_ROW) -buffers(conv51, conv52_d1, 1, 2, FULL_ROW) -buffers(conv52_d0, conv52_dc, 2, 2, FULL_ROW) -buffers(conv52_d1, conv52_dc, 2, 2, FULL_ROW) -buffers(conv52_dc, conv53, 2, FULL_ROW) -buffers(conv53, avgpool1, 2, FULL_ROW) -buffers(conv5_d0, conv5_dc, 2, 2, FULL_ROW) -buffers(conv5_d1, conv5_dc, 2, FULL_ROW) -buffers(conv5_dc, conv6, 5, FULL_ROW) -buffers(conv5_dc, conv8_d0, 5, FULL_ROW) -buffers(conv5_dc, conv8_d1, 5, FULL_ROW) -buffers(conv6, conv7, 2, FULL_ROW) -buffers(conv7, conv8_d0, 2, FULL_ROW) -buffers(conv7, conv8_d1, 2, FULL_ROW) -buffers(conv8_d0, conv8_dc, 2, 2, FULL_ROW) -buffers(conv8_d1, conv8_dc, 2, 2, FULL_ROW) -buffers(conv8_dc, conv9, 2, FULL_ROW) -buffers(conv8_dc, maxpool2, 2, 2, FULL_ROW) -buffers(conv9, conv10, 1, 2, FULL_ROW) -buffers(fc1_d0, fc1_dc, 1, 2, FULL_ROW) -buffers(fc1_d1, fc1_dc, 1, 2, FULL_ROW) -buffers(fc1_dc, softmax1, 1, 2, FULL_ROW) -buffers(maxpool1_d0, maxpool1_dc, 2, FULL_ROW) -buffers(maxpool1_d1, maxpool1_dc, 2, FULL_ROW) -buffers(maxpool1_dc, conv2, 2, FULL_ROW) -buffers(maxpool1_dc, conv4, 2, FULL_ROW) -buffers(maxpool2, conv11, 2, FULL_ROW) -buffers(maxpool3, conv24, 2, FULL_ROW) -buffers(maxpool4, conv43, 2, FULL_ROW) -buffers(normalization1, conv1_sd0, 5, 5, FULL_ROW) -buffers(normalization1, conv1_sd1, 5, FULL_ROW) -buffers(softmax1, output_layer1, 1, FULL_ROW) - -context_0 = context([avgpool1, conv10, conv11, conv12, conv13, conv14, conv15, conv16, conv17, conv18, conv19, conv1_sd0, conv1_sd1, conv1_sdc, conv2, conv20, conv21, conv22, conv23, conv24, conv25, conv26, conv27, conv28, conv29, conv3, conv30, conv31, conv32, conv33, conv34, conv35, conv36, conv37, conv38, conv39, conv4, conv40, conv41, conv42, conv43, conv44, conv45_d0, conv45_d1, conv45_d2, conv45_dc, conv46_d0, conv46_d1, conv46_dc, conv47, conv48, conv49_d0, conv49_d1, conv49_d2, conv49_dc, conv50, conv51, conv52_d0, conv52_d1, conv52_dc, conv53, conv5_d0, conv5_d1, conv5_dc, conv6, conv7, conv8_d0, conv8_d1, conv8_dc, conv9, fc1_d0, fc1_d1, fc1_dc, maxpool1_d0, maxpool1_d1, maxpool1_dc, maxpool2, maxpool3, maxpool4, normalization1, softmax1]) - -place(0, [conv3, conv47, conv48, conv5_d1, conv5_dc, conv6, conv7, conv8_d0, conv8_d1, fc1_d1]) -place(1, [conv21, conv22, conv49_d0, conv49_d1, conv49_d2, conv49_dc, conv50, conv8_dc, conv9]) -place(2, [conv1_sdc, conv2, conv37, conv38, conv39, conv4, conv41, conv42, conv43, conv5_d0, maxpool1_d0, maxpool1_d1, maxpool1_dc, maxpool4]) -place(3, [conv30, conv31, conv32, conv34, conv35, conv36, conv40, conv46_d1]) -place(4, [avgpool1, conv15, conv16, conv18, conv19, conv1_sd0, conv46_d0, conv52_dc, conv53, fc1_d0]) -place(5, [conv1_sd1, conv20, conv44, conv45_d1, conv45_d2, conv45_dc, conv52_d0, normalization1]) -place(6, [conv10, conv11, conv12, conv13, conv14, conv26, conv28, conv45_d0, conv52_d1, fc1_dc, maxpool2]) -place(7, [conv17, conv23, conv24, conv25, conv27, conv29, conv33, conv46_dc, conv51, maxpool3]) performance_param(fps=1331) + diff --git a/hailo_model_zoo/cfg/alls/performance/ssd_mobilenet_v1.alls b/hailo_model_zoo/cfg/alls/performance/ssd_mobilenet_v1.alls index 780345c0..2ebd77cd 100644 --- a/hailo_model_zoo/cfg/alls/performance/ssd_mobilenet_v1.alls +++ b/hailo_model_zoo/cfg/alls/performance/ssd_mobilenet_v1.alls @@ -7,7 +7,10 @@ context_switch_param(mode=disabled) quantization_param({conv*}, bias_mode=single_scale_decomposition) # post_quantization_optimization commands -post_quantization_optimization(finetune, policy=enabled, loss_factors=[0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1], dataset_size=4000, epochs=8, learning_rate=0.0001, loss_types=[l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2]) +post_quantization_optimization(finetune, policy=enabled, dataset_size=4000, epochs=8, learning_rate=0.0001, + loss_layer_names=[bbox_decoder13, conv14, bbox_decoder17, conv18, bbox_decoder21, conv22, bbox_decoder25, conv26, bbox_decoder29, conv30, bbox_decoder33, conv34, conv32, conv24, conv28, conv12, conv20, conv16], + loss_factors=[0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 1.0, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1], + loss_types=[l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2, l2]) allocator_param(agent=melg, automatic_ddr=False, automatic_ddr_l3_cuts_threshold=25, builder_exit_point=post-compilation, clk_freq=400000000.0, cluster_timeout=300, defuse_by_validator=True, enable_axis_upsize=True, enable_fixer=True, enable_loose_mode=False, enable_macros=True, enable_swapper=True, max_auto_defuse=8, max_cluster_util=1.0, network_fps=1150.0, seed=74, strategy=blind_search, timeout=30000, validate=True, automatic_l4_portals=False, optimize_buffers=False) diff --git a/hailo_model_zoo/cfg/alls/performance/tiny_yolov4_license_plates_yuy2.alls b/hailo_model_zoo/cfg/alls/performance/tiny_yolov4_license_plates_yuy2.alls new file mode 100644 index 00000000..d0b04d52 --- /dev/null +++ b/hailo_model_zoo/cfg/alls/performance/tiny_yolov4_license_plates_yuy2.alls @@ -0,0 +1,10 @@ +normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) + +quantization_param(tiny_yolov4_license_plates_yuy2/output_layer1, precision_mode=a16_w16) +quantization_param(tiny_yolov4_license_plates_yuy2/output_layer2, precision_mode=a16_w16) + +yuv_to_rgb1 = yuv_to_rgb() +reshape_yuy2 = input_conversion(tiny_yolov4_license_plates_yuy2/input_layer1, yuy2_to_hailo_yuv, emulator_support=True) +allocator_param(automatic_ddr=disabled, enable_partial_row_buffers=disabled) + +performance_param(fps=1385) diff --git a/hailo_model_zoo/cfg/alls/performance/yolov3_gluon.alls b/hailo_model_zoo/cfg/alls/performance/yolov3_gluon.alls index 353c7690..03d83016 100644 --- a/hailo_model_zoo/cfg/alls/performance/yolov3_gluon.alls +++ b/hailo_model_zoo/cfg/alls/performance/yolov3_gluon.alls @@ -6,9 +6,11 @@ post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, e quantization_param(output_layer1, precision_mode=a16_w16) quantization_param(output_layer2, precision_mode=a16_w16) -quantization_param(output_layer3, precision_mode=a16_w16) allocator_param(automatic_ddr=False) buffer_calc_param(buffer_calc_fps=0.0, optimize_buffers=True, split_aware_optimize_buffers=True) context_switch_param(goal_network_compute_utilization=0.75, goal_network_control_utilization=1.0, goal_network_memory_utilization=0.75, goal_network_weights_utilization=0.75, max_compute_utilization=0.3, max_control_utilization=0.3, max_memory_utilization=0.5, max_utilization=1.0, mode=allowed) - +allocator_param(automatic_ddr=False) +allocator_param(automatic_reshapes=False) +context_switch_param(max_utilization=0.5) +resources_param(max_control_utilization=0.8, max_compute_utilization=0.8, max_memory_utilization=0.8) diff --git a/hailo_model_zoo/cfg/alls/performance/yolov5m_hpp.alls b/hailo_model_zoo/cfg/alls/performance/yolov5m_hpp.alls new file mode 100644 index 00000000..efbb078e --- /dev/null +++ b/hailo_model_zoo/cfg/alls/performance/yolov5m_hpp.alls @@ -0,0 +1,14 @@ +normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) +change_output_activation(sigmoid) +nms_postprocess("$HMZ_DATA/models_files/ObjectDetection/Detection-COCO/yolo/yolov5m_spp/pretrained/2022-01-02/yolov5m_nms_config.json", yolov5, engine=cpu) +model_optimization_config(calibration, batch_size=4, calibset_size=64) +quantization_param(conv45, precision_mode=a8_w4) +quantization_param(conv46, precision_mode=a8_w4) +quantization_param(conv51, precision_mode=a8_w4) +quantization_param(conv53, precision_mode=a8_w4) +quantization_param(conv84, precision_mode=a8_w4) +quantization_param(conv89, precision_mode=a8_w4) +quantization_param(conv91, precision_mode=a8_w4) +post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=4, dataset_size=4000, loss_factors=[1.0, 1.0, 1.0], loss_types=[l2rel, l2rel, l2rel], loss_layer_names=[conv94, conv85, conv75]) +allocator_param(merge_min_layer_utilization=0.1) +performance_param(fps=150) diff --git a/hailo_model_zoo/cfg/alls/performance/yolov5m_vehicles.alls b/hailo_model_zoo/cfg/alls/performance/yolov5m_vehicles.alls index 4bef20d2..b3dac3ef 100644 --- a/hailo_model_zoo/cfg/alls/performance/yolov5m_vehicles.alls +++ b/hailo_model_zoo/cfg/alls/performance/yolov5m_vehicles.alls @@ -9,7 +9,7 @@ quantization_param(conv53, precision_mode=a8_w4) quantization_param(conv84, precision_mode=a8_w4) quantization_param(conv89, precision_mode=a8_w4) quantization_param(conv91, precision_mode=a8_w4) - +quantization_param({output_layer*}, precision_mode=a8_w8) post_quantization_optimization(finetune, policy=enabled, use_acceleras=disabled, learning_rate=0.0001, epochs=4, dataset_size=4000, loss_factors=[1.0, 1.0, 1.0], loss_types=[l2rel, l2rel, l2rel], loss_layer_names=[conv94, conv85, conv75]) allocator_param(timeout=100000, cluster_timeout=100000, automatic_ddr=disabled) diff --git a/hailo_model_zoo/cfg/alls/performance/yolov5m_vehicles_yuy2.alls b/hailo_model_zoo/cfg/alls/performance/yolov5m_vehicles_yuy2.alls new file mode 100644 index 00000000..59556926 --- /dev/null +++ b/hailo_model_zoo/cfg/alls/performance/yolov5m_vehicles_yuy2.alls @@ -0,0 +1,23 @@ +normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) +change_output_activation(sigmoid) +resize_input1 = resize_input([1080, 1920], input_layer1) +model_optimization_config(calibration, batch_size=4, calibset_size=64) +quantization_param(conv45, precision_mode=a8_w4) +quantization_param(conv46, precision_mode=a8_w4) +quantization_param(conv51, precision_mode=a8_w4) +quantization_param(conv53, precision_mode=a8_w4) +quantization_param(conv84, precision_mode=a8_w4) +quantization_param(conv89, precision_mode=a8_w4) +quantization_param(conv91, precision_mode=a8_w4) +quantization_param({output_layer*}, precision_mode=a8_w8) +post_quantization_optimization(finetune, policy=enabled, learning_rate=0.0001, epochs=4, dataset_size=4000, loss_factors=[1.0, 1.0, 1.0], loss_types=[l2rel, l2rel, l2rel], loss_layer_names=[conv94, conv85, conv75]) + +yuv_to_rgb1 = yuv_to_rgb() +reshape_yuy2 = input_conversion(input_layer1, yuy2_to_hailo_yuv, emulator_support=True) + +allocator_param(timeout=100000, cluster_timeout=100000, automatic_ddr=disabled) +allocator_param(merge_min_layer_utilization=0.1) + +compilation_param(yolov5m_vehicles_yuy2/resize_input1, resize_bilinear_streaming=enabled) +allocator_param(automatic_reshapes=False) +performance_param(fps=80) diff --git a/hailo_model_zoo/cfg/alls/performance/yolov5s.alls b/hailo_model_zoo/cfg/alls/performance/yolov5s.alls index badba391..67f6bfb6 100644 --- a/hailo_model_zoo/cfg/alls/performance/yolov5s.alls +++ b/hailo_model_zoo/cfg/alls/performance/yolov5s.alls @@ -2,4 +2,5 @@ normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0]) change_output_activation(sigmoid) model_optimization_config(calibration, batch_size=4, calibset_size=64) pre_quantization_optimization(equalization, policy=disabled) +quantization_param({output_layer*}, precision_mode=a8_w8) performance_param(fps=375) \ No newline at end of file diff --git a/hailo_model_zoo/cfg/base/base.yaml b/hailo_model_zoo/cfg/base/base.yaml index 87abc095..1ab1f299 100644 --- a/hailo_model_zoo/cfg/base/base.yaml +++ b/hailo_model_zoo/cfg/base/base.yaml @@ -6,6 +6,7 @@ quantization: calib_batch_size: null hn_editor: yuv2rgb: false + yuy2: false input_resize: enabled: false channels_remove: @@ -32,7 +33,6 @@ allocation: allocation_timeout: null legacy: false parser: - ew_add_policy: fused_conv_and_add nodes: - null - null @@ -55,6 +55,8 @@ postprocessing: meta_arch: null score_threshold: null nms_iou_thresh: null + nms_max_output_per_class: null + mask_threshold: null ext_upsample: null device_pre_post_layers: softmax: false @@ -63,17 +65,19 @@ postprocessing: nms: false max_finder: false sigmoid: false + hpp: false postprocess_config_json: null evaluation: data_set: null labels_offset: 0 limit_num_images: 1000000000.0 - infer_type: tf_infer + infer_type: model_infer classes: 1 two_stage_arch: false network_type: null gt_json_path: null labels_map: null + meta_arch: null info: client_or_rnd: r&d standard_or_customized: standard diff --git a/hailo_model_zoo/cfg/base/damoyolo.yaml b/hailo_model_zoo/cfg/base/damoyolo.yaml new file mode 100644 index 00000000..0017d68a --- /dev/null +++ b/hailo_model_zoo/cfg/base/damoyolo.yaml @@ -0,0 +1,46 @@ +base: +- base/coco.yaml +quantization: + calib_set: + - models_files/coco/2021-06-18/coco_calib2017.tfrecord +preprocessing: + network_type: detection + meta_arch: yolo_v5 + padding_color: 0 + centered: false +postprocessing: + device_pre_post_layers: + sigmoid: true + nms_iou_thresh: 0.7 + score_threshold: 0.05 + meta_arch: nanodet_split + anchors: + scale_factors: + - 0.0 + - 0.0 + regression_length: 16 + strides: + - 8 + - 16 + - 32 +info: + source: https://github.com/tinyvision/DAMO-YOLO + framework: pytorch + training_data: coco train2017 + validation_data: coco val2017 + eval_metric: mAP + license_url: https://github.com/tinyvision/DAMO-YOLO/blob/master/LICENSE +parser: + normalization_params: + normalize_in_net: true + mean_list: + - 0.0 + - 0.0 + - 0.0 + std_list: + - 1.0 + - 1.0 + - 1.0 +evaluation: + classes: 80 + dataset_name: coco_2017_detection diff --git a/hailo_model_zoo/cfg/base/yolact.yaml b/hailo_model_zoo/cfg/base/yolact.yaml index ee5eee52..356d6532 100644 --- a/hailo_model_zoo/cfg/base/yolact.yaml +++ b/hailo_model_zoo/cfg/base/yolact.yaml @@ -10,6 +10,7 @@ postprocessing: meta_arch: yolact nms_iou_thresh: 0.5 score_threshold: 0.05 + mask_threshold: 0.5 anchors: aspect_ratios: - 1.0 @@ -54,7 +55,8 @@ parser: - 57.12 - 57.38 evaluation: - infer_type: np_infer + meta_arch: yolact + infer_type: np_infer_lite labels_offset: 0 classes: 81 gt_json_path: models_files/coco/2021-06-18/instances_val2017.json diff --git a/hailo_model_zoo/cfg/base/yolov5_seg.yaml b/hailo_model_zoo/cfg/base/yolov5_seg.yaml index b51e3f0e..79bb4dd6 100644 --- a/hailo_model_zoo/cfg/base/yolov5_seg.yaml +++ b/hailo_model_zoo/cfg/base/yolov5_seg.yaml @@ -4,8 +4,10 @@ preprocessing: network_type: instance_segmentation postprocessing: meta_arch: yolov5_seg + mask_threshold: 0.5 evaluation: - infer_type: np_infer + meta_arch: yolov5_seg + infer_type: np_infer_lite dataset_name: coco_detection gt_json_path: models_files/coco/2021-06-18/instances_val2017.json inference: diff --git a/hailo_model_zoo/cfg/base/yolov8.yaml b/hailo_model_zoo/cfg/base/yolov8.yaml new file mode 100644 index 00000000..dfd5bb61 --- /dev/null +++ b/hailo_model_zoo/cfg/base/yolov8.yaml @@ -0,0 +1,15 @@ +base: +- base/yolo.yaml +postprocessing: + meta_arch: nanodet_v8 + anchors: + scale_factors: + - 0.0 + - 0.0 + regression_length: 15 + device_pre_post_layers: + sigmoid: true + nms_max_output_per_class: 300 + post_nms_topk: 300 + nms_iou_thresh: 0.7 + score_threshold: 0.001 diff --git a/hailo_model_zoo/cfg/networks/centernet_resnet_v1_18_postprocess.yaml b/hailo_model_zoo/cfg/networks/centernet_resnet_v1_18_postprocess.yaml index 28460069..ca13f0f8 100644 --- a/hailo_model_zoo/cfg/networks/centernet_resnet_v1_18_postprocess.yaml +++ b/hailo_model_zoo/cfg/networks/centernet_resnet_v1_18_postprocess.yaml @@ -18,9 +18,9 @@ paths: parser: nodes: - zero_padding2d/Pad - - - threshold_confidence/threshold_activation/threshold_confidence/re_lu/Relu - - centernet0_conv3/BiasAdd + - - centernet0_conv3/BiasAdd - centernet0_conv5/BiasAdd + - threshold_confidence/threshold_activation/threshold_confidence/re_lu/Relu info: task: object detection input_shape: 512x512x3 diff --git a/hailo_model_zoo/cfg/networks/centernet_resnet_v1_50_postprocess.yaml b/hailo_model_zoo/cfg/networks/centernet_resnet_v1_50_postprocess.yaml index 78dae0fa..0f3850a0 100644 --- a/hailo_model_zoo/cfg/networks/centernet_resnet_v1_50_postprocess.yaml +++ b/hailo_model_zoo/cfg/networks/centernet_resnet_v1_50_postprocess.yaml @@ -21,9 +21,9 @@ paths: parser: nodes: - zero_padding2d/Pad - - - threshold_confidence/threshold_activation/threshold_confidence/re_lu/Relu - - centernet0_conv3/BiasAdd + - - centernet0_conv3/BiasAdd - centernet0_conv5/BiasAdd + - threshold_confidence/threshold_activation/threshold_confidence/re_lu/Relu info: task: object detection input_shape: 512x512x3 diff --git a/hailo_model_zoo/cfg/networks/centerpose_regnetx_1.6gf_fpn.yaml b/hailo_model_zoo/cfg/networks/centerpose_regnetx_1.6gf_fpn.yaml index 3a955537..456ec90a 100644 --- a/hailo_model_zoo/cfg/networks/centerpose_regnetx_1.6gf_fpn.yaml +++ b/hailo_model_zoo/cfg/networks/centerpose_regnetx_1.6gf_fpn.yaml @@ -3,13 +3,6 @@ base: hn_editor: bgr2rgb: true output_scheme: - change_activations: - enabled: true - changes: - - layer_name: conv79 - activation: sigmoid - - layer_name: conv75 - activation: sigmoid integrated_postprocessing: enabled: true chains: diff --git a/hailo_model_zoo/cfg/networks/centerpose_repvgg_a0.yaml b/hailo_model_zoo/cfg/networks/centerpose_repvgg_a0.yaml index dc4aa975..42dde1be 100644 --- a/hailo_model_zoo/cfg/networks/centerpose_repvgg_a0.yaml +++ b/hailo_model_zoo/cfg/networks/centerpose_repvgg_a0.yaml @@ -3,13 +3,6 @@ base: hn_editor: bgr2rgb: true output_scheme: - change_activations: - enabled: true - changes: - - layer_name: conv40 - activation: sigmoid - - layer_name: conv36 - activation: sigmoid integrated_postprocessing: enabled: true chains: diff --git a/hailo_model_zoo/cfg/networks/damoyolo_tinynasL20_T.yaml b/hailo_model_zoo/cfg/networks/damoyolo_tinynasL20_T.yaml new file mode 100644 index 00000000..d061ff58 --- /dev/null +++ b/hailo_model_zoo/cfg/networks/damoyolo_tinynasL20_T.yaml @@ -0,0 +1,25 @@ +base: +- base/damoyolo.yaml +network: + network_name: damoyolo_tinynasL20_T +paths: + alls_script: damoyolo_tinynasL20_T.alls + network_path: + - models_files/ObjectDetection/Detection-COCO/yolo/damoyolo_tinynasL20_T/pretrained/2022-12-19/damoyolo_tinynasL20_T.onnx + url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ObjectDetection/Detection-COCO/yolo/damoyolo_tinynasL20_T/pretrained/2022-12-19/damoyolo_tinynasL20_T.zip +parser: + nodes: + - null + - - Mul_224 + - Sigmoid_244 + - Mul_264 + - Sigmoid_284 + - Mul_304 + - Sigmoid_324 +info: + task: object detection + input_shape: 640x640x3 + output_shape: 80x80x68, 80x80x81, 40X40X68, 40X40X81, 20X20X68, 20X20X81 + operations: 9.03G + parameters: 11.35M + full_precision_result: 42.8 diff --git a/hailo_model_zoo/cfg/networks/damoyolo_tinynasL25_S.yaml b/hailo_model_zoo/cfg/networks/damoyolo_tinynasL25_S.yaml new file mode 100644 index 00000000..3189445b --- /dev/null +++ b/hailo_model_zoo/cfg/networks/damoyolo_tinynasL25_S.yaml @@ -0,0 +1,25 @@ +base: +- base/damoyolo.yaml +network: + network_name: damoyolo_tinynasL25_S +paths: + alls_script: damoyolo_tinynasL25_S.alls + network_path: + - models_files/ObjectDetection/Detection-COCO/yolo/damoyolo_tinynasL25_S/pretrained/2022-12-19/damoyolo_tinynasL25_S.onnx + url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ObjectDetection/Detection-COCO/yolo/damoyolo_tinynasL25_S/pretrained/2022-12-19/damoyolo_tinynasL25_S.zip +parser: + nodes: + - null + - - Mul_239 + - Sigmoid_259 + - Mul_279 + - Sigmoid_299 + - Mul_319 + - Sigmoid_339 +info: + task: object detection + input_shape: 640x640x3 + output_shape: 80x80x68, 80x80x81, 40X40X68, 40X40X81, 20X20X68, 20X20X81 + operations: 18.85G + parameters: 16.25M + full_precision_result: 46.53 diff --git a/hailo_model_zoo/cfg/networks/damoyolo_tinynasL35_M.yaml b/hailo_model_zoo/cfg/networks/damoyolo_tinynasL35_M.yaml new file mode 100644 index 00000000..37907d5e --- /dev/null +++ b/hailo_model_zoo/cfg/networks/damoyolo_tinynasL35_M.yaml @@ -0,0 +1,25 @@ +base: +- base/damoyolo.yaml +network: + network_name: damoyolo_tinynasL35_M +paths: + alls_script: damoyolo_tinynasL35_M.alls + network_path: + - models_files/ObjectDetection/Detection-COCO/yolo/damoyolo_tinynasL35_M/pretrained/2022-12-19/damoyolo_tinynasL35_M.onnx + url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ObjectDetection/Detection-COCO/yolo/damoyolo_tinynasL35_M/pretrained/2022-12-19/damoyolo_tinynasL35_M.zip +parser: + nodes: + - null + - - Mul_396 + - Sigmoid_416 + - Mul_436 + - Sigmoid_456 + - Mul_476 + - Sigmoid_496 +info: + task: object detection + input_shape: 640x640x3 + output_shape: 80x80x68, 80x80x81, 40X40X68, 40X40X81, 20X20X68, 20X20X81 + operations: 30.87G + parameters: 33.98M + full_precision_result: 49.7 diff --git a/hailo_model_zoo/cfg/networks/lprnet.yaml b/hailo_model_zoo/cfg/networks/lprnet.yaml index 887bc6f2..adf8faf9 100644 --- a/hailo_model_zoo/cfg/networks/lprnet.yaml +++ b/hailo_model_zoo/cfg/networks/lprnet.yaml @@ -34,6 +34,7 @@ evaluation: data_set: null validation_data_set: null info: + task: pipeline input_shape: 75x300x3 output_shape: 5x19x11 operations: 18.29G diff --git a/hailo_model_zoo/cfg/networks/lprnet_yuy2.yaml b/hailo_model_zoo/cfg/networks/lprnet_yuy2.yaml new file mode 100644 index 00000000..f40cce47 --- /dev/null +++ b/hailo_model_zoo/cfg/networks/lprnet_yuy2.yaml @@ -0,0 +1,47 @@ +base: +- base/base.yaml +hn_editor: + yuv2rgb: true + yuy2: true +preprocessing: + network_type: ocr + meta_arch: lprnet +quantization: + calib_set: null +network: + network_name: lprnet_yuy2 +paths: + alls_script: lprnet_yuy2.alls + network_path: + - models_files/HailoNets/LPR/ocr/lprnet_304x75/2022-05-01/lprnet_304x75.onnx + url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/HailoNets/LPR/ocr/lprnet_304x75/2022-05-01/lprnet_304x75.zip +parser: + nodes: + - input + - - Conv_61 + normalization_params: + normalize_in_net: true + mean_list: + - 127.5 + - 127.5 + - 127.5 + std_list: + - 128.0 + - 128.0 + - 128.0 +evaluation: + infer_type: np_infer + labels_offset: 0 + classes: 10 + dataset_name: lp_ocr + data_set: null + validation_data_set: null +info: + task: pipeline + input_shape: 75x304x2 + output_shape: 5x19x11 + operations: 18.53G + parameters: 7.14M + framework: pytorch + eval_metric: Accuracy + full_precision_result: 54.075 diff --git a/hailo_model_zoo/cfg/networks/tiny_yolov4_license_plates.yaml b/hailo_model_zoo/cfg/networks/tiny_yolov4_license_plates.yaml index b3677cc2..f05c1d5b 100644 --- a/hailo_model_zoo/cfg/networks/tiny_yolov4_license_plates.yaml +++ b/hailo_model_zoo/cfg/networks/tiny_yolov4_license_plates.yaml @@ -45,6 +45,7 @@ evaluation: data_set: null validation_data_set: null info: + task: pipeline input_shape: 416x416x3 output_shape: 13x13x18, 26x26x18 operations: 3.4G diff --git a/hailo_model_zoo/cfg/networks/tiny_yolov4_license_plates_yuy2.yaml b/hailo_model_zoo/cfg/networks/tiny_yolov4_license_plates_yuy2.yaml new file mode 100644 index 00000000..a5eb0bfb --- /dev/null +++ b/hailo_model_zoo/cfg/networks/tiny_yolov4_license_plates_yuy2.yaml @@ -0,0 +1,58 @@ +base: +- base/yolo.yaml +preprocessing: + input_shape: null + meta_arch: mobilenet_ssd_ar +postprocessing: + nms_iou_thresh: 0.3 + score_threshold: 0.1 + meta_arch: yolo_v4 + anchors: + strides: + - 16 + - 32 + sizes: + - - 10 + - 14 + - 23 + - 27 + - 37 + - 58 + - - 81 + - 82 + - 135 + - 169 + - 344 + - 319 +hn_editor: + yuv2rgb: true + yuy2: true +quantization: + calib_set: null +network: + network_name: tiny_yolov4_license_plates_yuy2 +paths: + alls_script: tiny_yolov4_license_plates_yuy2.alls + network_path: + - models_files/HailoNets/LPR/lp_detector/tiny_yolov4_license_plates/2021-12-23/tiny_yolov4_license_plates.onnx + url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/HailoNets/LPR/lp_detector/tiny_yolov4_license_plates/2021-12-23/tiny_yolov4_license_plates.zip +parser: + nodes: + - input + - - Conv_100 + - Conv_453 +evaluation: + labels_offset: 0 + classes: 1 + dataset_name: license_plates + data_set: null + validation_data_set: null +info: + task: pipeline + input_shape: 416x416x2 + output_shape: 13x13x18, 26x26x18 + operations: 3.4G + parameters: 5.87M + framework: darknet + eval_metric: mAP + full_precision_result: 74.083 diff --git a/hailo_model_zoo/cfg/networks/vit_base.yaml b/hailo_model_zoo/cfg/networks/vit_base.yaml new file mode 100644 index 00000000..fc5931fd --- /dev/null +++ b/hailo_model_zoo/cfg/networks/vit_base.yaml @@ -0,0 +1,25 @@ +base: +- base/vit.yaml +network: + network_name: vit_base +paths: + network_path: + - models_files/Classification/vit_base/pretrained/2023-01-25/vit_base_patch16_224_unfilt.onnx + url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/Classification/vit_base/pretrained/2023-01-25/vit_base.zip + alls_script: vit_base.alls +parser: + nodes: + - input.1 + - '1353' +info: + task: classification + input_shape: 224x224x3 + output_shape: '1000' + operations: 17.1G + parameters: 86.5M + framework: pytorch + training_data: imagenet train + validation_data: imagenet val + eval_metric: Accuracy (top1) + source: https://github.com/rwightman/pytorch-image-models + full_precision_result: 80.16 diff --git a/hailo_model_zoo/cfg/networks/yolov3.yaml b/hailo_model_zoo/cfg/networks/yolov3.yaml index 1843c04d..ae0af89a 100644 --- a/hailo_model_zoo/cfg/networks/yolov3.yaml +++ b/hailo_model_zoo/cfg/networks/yolov3.yaml @@ -36,7 +36,6 @@ paths: - models_files/ObjectDetection/Detection-COCO/yolo/yolov3/pretrained/2021-08-16/yolov3_1_3_608_608_static.onnx url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ObjectDetection/Detection-COCO/yolo/yolov3/pretrained/2021-08-16/yolov3.zip parser: - ew_add_policy: fused_conv_and_add_identity_fallback nodes: - input - - Conv_139 diff --git a/hailo_model_zoo/cfg/networks/yolov3_gluon.yaml b/hailo_model_zoo/cfg/networks/yolov3_gluon.yaml index 56ddf461..452cacab 100644 --- a/hailo_model_zoo/cfg/networks/yolov3_gluon.yaml +++ b/hailo_model_zoo/cfg/networks/yolov3_gluon.yaml @@ -38,7 +38,6 @@ paths: - models_files/ObjectDetection/Detection-COCO/yolo/yolov3_gluon/pretrained/2021-07-11/yolo3_darknet53_coco.ckpt.data-00000-of-00001 url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ObjectDetection/Detection-COCO/yolo/yolov3_gluon/pretrained/2021-07-11/yolov3_gluon.zip parser: - ew_add_policy: fused_conv_and_add_identity_fallback nodes: - zero_padding2d/Pad - - yolov30_yolooutputv30_conv0/BiasAdd diff --git a/hailo_model_zoo/cfg/networks/yolov5l.yaml b/hailo_model_zoo/cfg/networks/yolov5l.yaml index 2a2f001d..6aa62840 100644 --- a/hailo_model_zoo/cfg/networks/yolov5l.yaml +++ b/hailo_model_zoo/cfg/networks/yolov5l.yaml @@ -11,7 +11,6 @@ paths: url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ObjectDetection/Detection-COCO/yolo/yolov5l_spp/pretrained/2022-02-03/yolov5l.zip alls_script: yolov5l.alls parser: - ew_add_policy: fused_conv_and_add_identity_fallback nodes: - null - - Conv_406 diff --git a/hailo_model_zoo/cfg/networks/yolov5m_hpp.yaml b/hailo_model_zoo/cfg/networks/yolov5m_hpp.yaml new file mode 100644 index 00000000..7aff30e7 --- /dev/null +++ b/hailo_model_zoo/cfg/networks/yolov5m_hpp.yaml @@ -0,0 +1,37 @@ +base: +- base/yolov5.yaml +postprocessing: + device_pre_post_layers: + nms: true + postprocess_config_json: models_files/ObjectDetection/Detection-COCO/yolo/yolov5m_spp/pretrained/2022-01-02/yolov5m_nms_config.json + meta_arch: yolo + hpp: true +quantization: + calib_set: + - models_files/coco/2021-06-18/coco_calib2017.tfrecord +network: + network_name: yolov5m_hpp +paths: + alls_script: yolov5m_hpp.alls + network_path: + - models_files/ObjectDetection/Detection-COCO/yolo/yolov5m_spp/pretrained/2022-01-02/yolov5m.onnx + url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ObjectDetection/Detection-COCO/yolo/yolov5m_spp/pretrained/2022-01-02/yolov5m.zip +parser: + nodes: + - images + - - Conv_289 + - Conv_273 + - Conv_257 +info: + task: object detection + input_shape: 640x640x3 + output_shape: 20x20x255, 40x40x255, 80x80x255 + operations: 26.14G + parameters: 21.78M + framework: pytorch + training_data: coco train2017 + validation_data: coco val2017 + eval_metric: mAP + full_precision_result: 42.59 + source: https://github.com/ultralytics/yolov5/releases/tag/v2.0 + license_url: https://github.com/ultralytics/yolov5/blob/master/LICENSE diff --git a/hailo_model_zoo/cfg/networks/yolov5m_vehicles.yaml b/hailo_model_zoo/cfg/networks/yolov5m_vehicles.yaml index 39648f2b..a0d80c9d 100644 --- a/hailo_model_zoo/cfg/networks/yolov5m_vehicles.yaml +++ b/hailo_model_zoo/cfg/networks/yolov5m_vehicles.yaml @@ -28,6 +28,7 @@ evaluation: data_set: null validation_data_set: null info: + task: pipeline input_shape: 1080x1920x3 output_shape: 20x20x18, 40x40x18, 80x80x18 operations: 25.63G diff --git a/hailo_model_zoo/cfg/networks/yolov5m_vehicles_yuy2.yaml b/hailo_model_zoo/cfg/networks/yolov5m_vehicles_yuy2.yaml new file mode 100644 index 00000000..753b775f --- /dev/null +++ b/hailo_model_zoo/cfg/networks/yolov5m_vehicles_yuy2.yaml @@ -0,0 +1,40 @@ +base: +- base/yolov5.yaml +quantization: + calib_set: null +hn_editor: + yuv2rgb: true + yuy2: true + input_resize: + enabled: true + input_shape: + - 1080 + - 1920 +network: + network_name: yolov5m_vehicles_yuy2 +paths: + alls_script: yolov5m_vehicles_yuy2.alls + network_path: + - models_files/HailoNets/LPR/vehicle_detector/yolov5m_vehicles/2022-02-23/yolov5m_vehicles.onnx + url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/HailoNets/LPR/vehicle_detector/yolov5m_vehicles/2022-02-23/yolov5m_vehicles.zip +parser: + nodes: + - images + - - Conv_289 + - Conv_273 + - Conv_257 +evaluation: + labels_offset: 0 + classes: 1 + dataset_name: vehicle_detection + data_set: null + validation_data_set: null +info: + task: pipeline + input_shape: 1080x1920x2 + output_shape: 20x20x18, 40x40x18, 80x80x18 + operations: 25.63G + parameters: 21.47M + framework: pytorch + eval_metric: mAP + full_precision_result: 44.69 diff --git a/hailo_model_zoo/cfg/networks/yolov5m_wo_spp_yuv.yaml b/hailo_model_zoo/cfg/networks/yolov5m_wo_spp_yuy2.yaml similarity index 65% rename from hailo_model_zoo/cfg/networks/yolov5m_wo_spp_yuv.yaml rename to hailo_model_zoo/cfg/networks/yolov5m_wo_spp_yuy2.yaml index e900a28c..71a2011d 100644 --- a/hailo_model_zoo/cfg/networks/yolov5m_wo_spp_yuv.yaml +++ b/hailo_model_zoo/cfg/networks/yolov5m_wo_spp_yuy2.yaml @@ -2,17 +2,19 @@ base: - networks/yolov5m_wo_spp.yaml hn_editor: yuv2rgb: true + yuy2: true input_resize: enabled: true input_shape: - 720 - 1280 network: - network_name: yolov5m_wo_spp_yuv + network_name: yolov5m_wo_spp_yuy2 paths: - alls_script: yolov5m_wo_spp_yuv.alls + alls_script: yolov5m_wo_spp_yuy2.alls info: - input_shape: 720x1280x3 + task: pipeline + input_shape: 720x1280x2 output_shape: 20x20x255, 40x40x255, 80x80x255 operations: 26.50G parameters: 22.67M diff --git a/hailo_model_zoo/cfg/networks/yolov5s_c3tr.yaml b/hailo_model_zoo/cfg/networks/yolov5s_c3tr.yaml new file mode 100644 index 00000000..8d47ad4a --- /dev/null +++ b/hailo_model_zoo/cfg/networks/yolov5s_c3tr.yaml @@ -0,0 +1,31 @@ +base: +- base/yolov5.yaml +inference: + full_precision_batch_size: 2 + emulator_batch_size: 2 +network: + network_name: yolov5s_c3tr +paths: + alls_script: yolov5s_c3tr.alls + network_path: + - models_files/ObjectDetection/Detection-COCO/yolo/yolov5s_c3tr/pretrained/2023-02-07/yolov5s_c3tr_simp.onnx + url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ObjectDetection/Detection-COCO/yolo/yolov5s_c3tr/pretrained/2023-02-07/yolov5s_c3tr.zip +parser: + nodes: + - images + - - Conv_383 + - Conv_332 + - Conv_281 +info: + task: object detection + input_shape: 640x640x3 + output_shape: 20x20x255, 40x40x255, 80x80x255 + operations: 8.51G + parameters: 10.29M + framework: pytorch + training_data: coco train2017 + validation_data: coco val2017 + eval_metric: mAP + full_precision_result: 37.134 + source: https://github.com/ultralytics/yolov5/tree/v6.0 + license_url: https://github.com/ultralytics/yolov5/blob/v6.0/LICENSE diff --git a/hailo_model_zoo/cfg/networks/yolov5s_personface.yaml b/hailo_model_zoo/cfg/networks/yolov5s_personface.yaml index 3176efb8..0b4894b4 100644 --- a/hailo_model_zoo/cfg/networks/yolov5s_personface.yaml +++ b/hailo_model_zoo/cfg/networks/yolov5s_personface.yaml @@ -26,7 +26,7 @@ evaluation: data_set: null validation_data_set: null info: - task: object detection + task: pipeline input_shape: 640x640x3 output_shape: 80x80x21, 40x40x21, 20x20x21 operations: 8.38G diff --git a/hailo_model_zoo/cfg/networks/yolov8l.yaml b/hailo_model_zoo/cfg/networks/yolov8l.yaml new file mode 100644 index 00000000..89a3d1ad --- /dev/null +++ b/hailo_model_zoo/cfg/networks/yolov8l.yaml @@ -0,0 +1,31 @@ +base: +- base/yolov8.yaml +network: + network_name: yolov8l +paths: + network_path: + - models_files/ObjectDetection/Detection-COCO/yolo/yolov8l/2023-02-02/yolov8l.onnx + alls_script: yolov8l.alls + url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ObjectDetection/Detection-COCO/yolo/yolov8l/2023-02-02/yolov8l.zip +parser: + nodes: + - null + - - /model.22/cv2.0/cv2.0.2/Conv + - /model.22/cv3.0/cv3.0.2/Conv + - /model.22/cv2.1/cv2.1.2/Conv + - /model.22/cv3.1/cv3.1.2/Conv + - /model.22/cv2.2/cv2.2.2/Conv + - /model.22/cv3.2/cv3.2.2/Conv +info: + task: object detection + input_shape: 640x640x3 + output_shape: 80x80x80, 80X80X64, 40X40X80, 40X40X64, 20X20X80, 20X20X64 + operations: 82.65G + parameters: 43.7M + framework: pytorch + training_data: coco train2017 + validation_data: coco val2017 + eval_metric: mAP + full_precision_result: 52.61 + source: https://github.com/ultralytics/ultralytics + license_url: https://github.com/ultralytics/ultralytics/blob/main/LICENSE diff --git a/hailo_model_zoo/cfg/networks/yolov8m.yaml b/hailo_model_zoo/cfg/networks/yolov8m.yaml new file mode 100644 index 00000000..d3bce09b --- /dev/null +++ b/hailo_model_zoo/cfg/networks/yolov8m.yaml @@ -0,0 +1,31 @@ +base: +- base/yolov8.yaml +network: + network_name: yolov8m +paths: + network_path: + - models_files/ObjectDetection/Detection-COCO/yolo/yolov8m/2023-02-02/yolov8m.onnx + alls_script: yolov8m.alls + url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ObjectDetection/Detection-COCO/yolo/yolov8m/2023-02-02/yolov8m.zip +parser: + nodes: + - null + - - /model.22/cv2.0/cv2.0.2/Conv + - /model.22/cv3.0/cv3.0.2/Conv + - /model.22/cv2.1/cv2.1.2/Conv + - /model.22/cv3.1/cv3.1.2/Conv + - /model.22/cv2.2/cv2.2.2/Conv + - /model.22/cv3.2/cv3.2.2/Conv +info: + task: object detection + input_shape: 640x640x3 + output_shape: 80x80x80, 80X80X64, 40X40X80, 40X40X64, 20X20X80, 20X20X64 + operations: 39.5G + parameters: 25.9M + framework: pytorch + training_data: coco train2017 + validation_data: coco val2017 + eval_metric: mAP + full_precision_result: 50.08 + source: https://github.com/ultralytics/ultralytics + license_url: https://github.com/ultralytics/ultralytics/blob/main/LICENSE diff --git a/hailo_model_zoo/cfg/networks/yolov8n.yaml b/hailo_model_zoo/cfg/networks/yolov8n.yaml new file mode 100644 index 00000000..fc1c9112 --- /dev/null +++ b/hailo_model_zoo/cfg/networks/yolov8n.yaml @@ -0,0 +1,31 @@ +base: +- base/yolov8.yaml +network: + network_name: yolov8n +paths: + network_path: + - models_files/ObjectDetection/Detection-COCO/yolo/yolov8n/2023-01-30/yolov8n.onnx + alls_script: yolov8n.alls + url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ObjectDetection/Detection-COCO/yolo/yolov8n/2023-01-30/yolov8n.zip +parser: + nodes: + - null + - - /model.22/cv2.0/cv2.0.2/Conv + - /model.22/cv3.0/cv3.0.2/Conv + - /model.22/cv2.1/cv2.1.2/Conv + - /model.22/cv3.1/cv3.1.2/Conv + - /model.22/cv2.2/cv2.2.2/Conv + - /model.22/cv3.2/cv3.2.2/Conv +info: + task: object detection + input_shape: 640x640x3 + output_shape: 80x80x80, 80X80X64, 40X40X80, 40X40X64, 20X20X80, 20X20X64 + operations: 4.4G + parameters: 3.2M + framework: pytorch + training_data: coco train2017 + validation_data: coco val2017 + eval_metric: mAP + full_precision_result: 37.23 + source: https://github.com/ultralytics/ultralytics + license_url: https://github.com/ultralytics/ultralytics/blob/main/LICENSE diff --git a/hailo_model_zoo/cfg/networks/yolov8s.yaml b/hailo_model_zoo/cfg/networks/yolov8s.yaml new file mode 100644 index 00000000..88abc778 --- /dev/null +++ b/hailo_model_zoo/cfg/networks/yolov8s.yaml @@ -0,0 +1,31 @@ +base: +- base/yolov8.yaml +network: + network_name: yolov8s +paths: + network_path: + - models_files/ObjectDetection/Detection-COCO/yolo/yolov8s/2023-02-02/yolov8s.onnx + alls_script: yolov8s.alls + url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ObjectDetection/Detection-COCO/yolo/yolov8s/2023-02-02/yolov8s.zip +parser: + nodes: + - null + - - /model.22/cv2.0/cv2.0.2/Conv + - /model.22/cv3.0/cv3.0.2/Conv + - /model.22/cv2.1/cv2.1.2/Conv + - /model.22/cv3.1/cv3.1.2/Conv + - /model.22/cv2.2/cv2.2.2/Conv + - /model.22/cv3.2/cv3.2.2/Conv +info: + task: object detection + input_shape: 640x640x3 + output_shape: 80x80x80, 80X80X64, 40X40X80, 40X40X64, 20X20X80, 20X20X64 + operations: 14.3G + parameters: 11.2M + framework: pytorch + training_data: coco train2017 + validation_data: coco val2017 + eval_metric: mAP + full_precision_result: 44.75 + source: https://github.com/ultralytics/ultralytics + license_url: https://github.com/ultralytics/ultralytics/blob/main/LICENSE diff --git a/hailo_model_zoo/cfg/networks/yolov8x.yaml b/hailo_model_zoo/cfg/networks/yolov8x.yaml new file mode 100644 index 00000000..17fd8971 --- /dev/null +++ b/hailo_model_zoo/cfg/networks/yolov8x.yaml @@ -0,0 +1,31 @@ +base: +- base/yolov8.yaml +network: + network_name: yolov8x +paths: + network_path: + - models_files/ObjectDetection/Detection-COCO/yolo/yolov8x/2023-02-02/yolov8x.onnx + alls_script: yolov8x.alls + url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ObjectDetection/Detection-COCO/yolo/yolov8x/2023-02-02/yolov8x.zip +parser: + nodes: + - null + - - /model.22/cv2.0/cv2.0.2/Conv + - /model.22/cv3.0/cv3.0.2/Conv + - /model.22/cv2.1/cv2.1.2/Conv + - /model.22/cv3.1/cv3.1.2/Conv + - /model.22/cv2.2/cv2.2.2/Conv + - /model.22/cv3.2/cv3.2.2/Conv +info: + task: object detection + input_shape: 640x640x3 + output_shape: 80x80x80, 80X80X64, 40X40X80, 40X40X64, 20X20X80, 20X20X64 + operations: 129G + parameters: 68.2M + framework: pytorch + training_data: coco train2017 + validation_data: coco val2017 + eval_metric: mAP + full_precision_result: 53.61 + source: https://github.com/ultralytics/ultralytics + license_url: https://github.com/ultralytics/ultralytics/blob/main/LICENSE diff --git a/hailo_model_zoo/core/augmentations/__init__.py b/hailo_model_zoo/core/augmentations/__init__.py new file mode 100644 index 00000000..f575e100 --- /dev/null +++ b/hailo_model_zoo/core/augmentations/__init__.py @@ -0,0 +1,8 @@ +from .left_to_right import AugmentedLRModel + + +def make_model_callback(network_info): + if network_info.evaluation.infer_type == 'facenet_infer': + return lambda m: AugmentedLRModel(m) + + return lambda m: m diff --git a/hailo_model_zoo/core/augmentations/left_to_right.py b/hailo_model_zoo/core/augmentations/left_to_right.py new file mode 100644 index 00000000..ae5ac6f9 --- /dev/null +++ b/hailo_model_zoo/core/augmentations/left_to_right.py @@ -0,0 +1,11 @@ +import tensorflow as tf + + +class AugmentedLRModel(tf.keras.Model): + def __init__(self, m): + super().__init__() + self._m = m + + def call(self, x): + flipped = tf.image.flip_left_right(x) + return tf.concat([self._m(x), self._m(flipped)], axis=1) diff --git a/hailo_model_zoo/core/eval/face_verification_evaluation.py b/hailo_model_zoo/core/eval/face_verification_evaluation.py index 204f53e5..0a238981 100644 --- a/hailo_model_zoo/core/eval/face_verification_evaluation.py +++ b/hailo_model_zoo/core/eval/face_verification_evaluation.py @@ -33,9 +33,15 @@ def __init__(self, **kwargs): def _parse_net_output(self, net_output): return net_output['predictions'] + def _parse_gt_data(self, gt_data): + # facenet_infer tf1 legacy support + if isinstance(gt_data, tuple): + return gt_data + return gt_data['image_name'], gt_data['is_same'] + def update_op(self, logits_batch, gt_data): logits_batch = self._parse_net_output(logits_batch) - image_name, is_same = gt_data + image_name, is_same = self._parse_gt_data(gt_data) idx = 0 for logits, name, same in zip(logits_batch, image_name, is_same): if idx % 2 == 0: diff --git a/hailo_model_zoo/core/eval/head_pose_estimation_evaluation.py b/hailo_model_zoo/core/eval/head_pose_estimation_evaluation.py index 0cfe9abe..ec895e5e 100644 --- a/hailo_model_zoo/core/eval/head_pose_estimation_evaluation.py +++ b/hailo_model_zoo/core/eval/head_pose_estimation_evaluation.py @@ -11,13 +11,9 @@ def __init__(self, **kwargs): self._normalize_results = kwargs.get('normalize_results', True) self.reset() - def _parse_net_output(self, net_output): - return net_output['predictions'] - def update_op(self, net_output, img_info): - net_output = self._parse_net_output(net_output) _pitch, _yaw, _roll = img_info['angles'][:, 0], img_info['angles'][:, 1], img_info['angles'][:, 2] - _pitch_predicted, _roll_predicted, _yaw_predicted = net_output + _pitch_predicted, _roll_predicted, _yaw_predicted = (net_output['pitch'], net_output['roll'], net_output['yaw']) self.yaw_err += list(np.abs(_yaw_predicted - _yaw)) self.pitch_err += list(np.abs(_pitch_predicted - _pitch)) self.roll_err += list(np.abs(_roll_predicted - _roll)) diff --git a/hailo_model_zoo/core/eval/instance_segmentation_evaluation.py b/hailo_model_zoo/core/eval/instance_segmentation_evaluation.py index 40cd4ab0..69643a8d 100644 --- a/hailo_model_zoo/core/eval/instance_segmentation_evaluation.py +++ b/hailo_model_zoo/core/eval/instance_segmentation_evaluation.py @@ -1,389 +1,52 @@ import numpy as np -import cv2 from collections import OrderedDict from pycocotools.coco import COCO from pycocotools.cocoeval import COCOeval from pycocotools import mask as maskUtils from hailo_model_zoo.core.eval.eval_base_class import Eval -from hailo_model_zoo.core.postprocessing.instance_segmentation_postprocessing import postprocess, _spraseinst_post +from hailo_model_zoo.core.eval.instance_segmentation_evaluation_utils import (YolactEval, + Yolov5SegEval, + SparseInstEval) from hailo_model_zoo.core.datasets.datasets_info import get_dataset_info -class InstanceSegmentationEval(Eval): - def __init__(self, **kwargs): - super().__init__(**kwargs) - net_name = kwargs['net_name'] - eval_class = self._get_eval_class(net_name)(**kwargs) - self.evaluate = eval_class.evaluate - self.update_op = eval_class.update_op - self._get_accuracy = eval_class._get_accuracy - self.reset = eval_class.reset - - def _get_eval_class(self, net_name): - if 'sparseinst' in net_name: - return SparseInstEval - elif 'yolact' in net_name: - return YolactEval - elif 'yolov5' in net_name: - return Yolov5SegEval - else: - raise NotImplementedError(f"Evaluation for {net_name} not supported") - - def evaluate(self): - pass - - def update_op(self): - pass - - def _get_accuracy(self): - pass - - def reset(self): - pass - - -class SparseInstEval(Eval): - """COCO evaluation metric class.""" - - def __init__(self, **kwargs): - """Constructs COCO evaluation class. - """ - dataset_name = kwargs.get('dataset_name', None) - dataset_info = get_dataset_info(dataset_name=dataset_name) - self._label_map = dataset_info.label_map - self._label_inv_map = {v: k for k, v in self._label_map.items()} - self._metric_names = ['mask AP', 'mask AP50', 'mask AP75', - 'mask APs', 'mask APm', 'mask APl', - 'mask ARmax1', 'mask ARmax10', 'mask ARmax100', - 'mask ARs', 'mask ARm', 'mask ARl'] - self._metrics_vals = [0] * len(self._metric_names) - - self._mask_data = [] - self._gt_ann_file = kwargs['gt_json_path'] - self.reset() - - def instances_to_coco_json(self, instances, img_id): - """ - Dump an "Instances" object to a COCO-format json that's used for evaluation. - - Args: - instances (Instances): - img_id (int): the image id - - Returns: - list[dict]: list of json annotations in COCO format. - """ - scores = instances['scores'] - classes = instances['pred_classes'] - assert scores.shape[0] == classes.shape[0] == instances['pred_masks'].shape[0] - - num_instance = scores.shape[0] - if num_instance == 0: - return [] - - rles = [ - maskUtils.encode(np.array(mask[:, :, None], order="F", dtype="uint8"))[0] - for mask in instances['pred_masks'] - ] - for rle in rles: - rle["counts"] = rle["counts"].decode("utf-8") - - results = [] - for k in range(num_instance): - result = { - "image_id": img_id, - "category_id": self._label_inv_map[classes[k] + 1], - "segmentation": rles[k], - "bbox": list(), - "score": scores[k] - } - results.append(result) - return results - - def reset(self): - self._images = set() - - def _parse_net_output(self, net_output): - return net_output['predictions'] - - def update_op(self, net_output, img_info): - gt_data_image_id = img_info["image_id"] - input_shape = img_info['height'], img_info['width'] - output_shapes = img_info['orig_height'], img_info['orig_width'] - - net_output = self._parse_net_output(net_output) - detections = _spraseinst_post(net_output, input_shape, output_shapes) - - for batch_idx, results in enumerate(detections): - img_id = gt_data_image_id[batch_idx] - self._images.add(img_id) - coco_results = self.instances_to_coco_json(results, img_id) - self._mask_data.extend(coco_results) - - def evaluate(self): - """Evaluates with detections from all images in our data set with COCO API. - - Returns: - coco_metric: float numpy array with shape [12] representing the - coco-style evaluation metrics. - """ - coco_gt = COCO(self._gt_ann_file) - coco_gt.createIndex() - coco_dt = coco_gt.loadRes(self._mask_data) - seg_eval = COCOeval(coco_gt, coco_dt, 'segm') - seg_eval.params.imgIds = list(self._images) - seg_eval.evaluate() - seg_eval.accumulate() - seg_eval.summarize() - self._metrics_vals = list(np.array(seg_eval.stats, dtype=np.float32)) - - def _get_accuracy(self): - return OrderedDict([(self._metric_names[0], self._metrics_vals[0]), - (self._metric_names[1], self._metrics_vals[1])]) - - -class Yolov5SegEval(Eval): - """COCO evaluation metric class.""" - - def __init__(self, **kwargs): - """Constructs COCO evaluation class. - """ - self.centered = kwargs['centered'] - dataset_name = kwargs.get('dataset_name', None) - dataset_info = get_dataset_info(dataset_name=dataset_name) - self._label_map = dataset_info.label_map - self._label_inv_map = {v: k for k, v in self._label_map.items()} - self._metric_names = ['bbox AP', 'bbox AP50', 'bbox AP75', 'bbox APs', 'bbox APm', - 'bbox APl', 'bbox ARmax1', 'bbox ARmax10', 'bbox ARmax100', - 'bbox ARs', 'bbox ARm', 'bbox ARl', 'mask AP', 'mask AP50', - 'mask AP75', 'mask APs', 'mask APm', 'mask APl', 'mask ARmax1', - 'mask ARmax10', 'mask ARmax100', 'mask ARs', 'mask ARm', 'mask ARl'] - self._metrics_vals = [0] * len(self._metric_names) - self._mask_data = [] - self._gt_ann_file = kwargs['gt_json_path'] - self.reset() - - def reset(self): - """Reset COCO API object.""" - self._coco_gt = COCO() - self._detections = np.empty(shape=(0, 7)) - self._dataset = { - 'images': [], - 'annotations': [], - 'categories': [] - } - self._images = set() - self._annotation_id = 1 - self._category_ids = set() - - def _get_ratio_and_padding(self, batch_idx, img_info): - orig_shape = img_info['height'][batch_idx], img_info['width'][batch_idx] - ratio = (img_info['letterbox_height'][batch_idx] / orig_shape[0], - img_info['letterbox_width'][batch_idx] / orig_shape[1]) - padding = (img_info['horizontal_pad'][batch_idx] / 2, - img_info['vertical_pad'][batch_idx] / 2) - return ratio, padding - - def scale_boxes(self, img1_shape, boxes, img0_shape, ratio_pad=None): - # Rescale boxes (xyxy) from img1_shape to img0_shape - if ratio_pad is None: # calculate from img0_shape - gain = min(img1_shape[0] / img0_shape[0], img1_shape[1] / img0_shape[1]) # gain = old / new - pad = (img1_shape[1] - img0_shape[1] * gain) / 2, (img1_shape[0] - img0_shape[0] * gain) / 2 # wh padding - else: - gain = ratio_pad[0][0] - pad = ratio_pad[1] - - boxes[:, [0, 2]] -= pad[0] # x padding - boxes[:, [1, 3]] -= pad[1] # y padding - boxes[:, :4] /= gain - - # Clip boxes (xyxy) to image shape (height, width) - boxes[:, [0, 2]] = boxes[:, [0, 2]].clip(0, img0_shape[1]) # x1, x2 - boxes[:, [1, 3]] = boxes[:, [1, 3]].clip(0, img0_shape[0]) # y1, y2 - return boxes - - def scale_image(self, im1_shape, masks, im0_shape, ratio_pad=None): - """ - img1_shape: model input shape, [h, w] - img0_shape: origin pic shape, [h, w, 3] - masks: [h, w, num] - """ - # Rescale coordinates (xyxy) from im1_shape to im0_shape - if ratio_pad is None: # calculate from im0_shape - gain = min(im1_shape[0] / im0_shape[0], im1_shape[1] / im0_shape[1]) # gain = old / new - pad = (im1_shape[1] - im0_shape[1] * gain) / 2, (im1_shape[0] - im0_shape[0] * gain) / 2 # wh padding - else: - pad = ratio_pad[1] - top, left = int(pad[1]), int(pad[0]) # y, x - bottom, right = int(im1_shape[0] - pad[1]), int(im1_shape[1] - pad[0]) - - if len(masks.shape) < 2: - raise ValueError(f'"len of masks shape" should be 2 or 3, but got {len(masks.shape)}') - masks = masks[top:bottom, left:right] - masks = cv2.resize(masks, (im0_shape[1], im0_shape[0])) - - if len(masks.shape) == 2: - masks = masks[:, :, None] - return masks - - def _parse_net_output(self, net_output): - return net_output['predictions'] +EVAL_CLASS_MAP = { + 'yolact': YolactEval, + 'yolov5_seg': Yolov5SegEval, + 'sparseinst': SparseInstEval, +} - def update_op(self, net_output, img_info): - net_output = self._parse_net_output(net_output) - gt_data_height = img_info["height"] - gt_data_width = img_info["width"] - gt_data_image_id = img_info["image_id"] - gt_data_num_boxes = img_info["num_boxes"] - gt_data_bbox = img_info["bbox"] - gt_data_area = img_info["area"] - gt_data_category_id = img_info["category_id"] - gt_data_is_crowd = img_info["is_crowd"] - - for batch_idx, output in enumerate(net_output): - input_shape = img_info['img_orig'].shape[1:3] - boxes = output['detection_boxes'] - detection_masks = output['mask'] - detection_scores = output['detection_scores'] - detection_classes = output['detection_classes'] - ratio_pad = self._get_ratio_and_padding(batch_idx, img_info) - - if detection_classes.shape[0]: - # rescale boxes to GT size - output_shape = gt_data_height[batch_idx], gt_data_width[batch_idx] - boxes = self.scale_boxes(input_shape, boxes, output_shape, ratio_pad=ratio_pad) - # rescale masks to GT size - detection_masks = self.scale_image(input_shape, - detection_masks.transpose(1, 2, 0).astype(np.uint8), - output_shape, - ratio_pad=ratio_pad) - - detection_masks = detection_masks.transpose((2, 0, 1)) - - num_detections = detection_classes.shape[0] - # Rearrange detections bbox to be in the coco format - xmin, ymin, width, height - bbox = np.transpose(np.vstack([boxes[:, 0], - boxes[:, 1], - boxes[:, 2] - boxes[:, 0], - boxes[:, 3] - boxes[:, 1]])) - # Concat image id per detection in the first column - image_id_vec = np.ones(shape=(bbox.shape[0], 1)) * gt_data_image_id[batch_idx] - # Finalize the detections np.array as shape=(num_detections, 7) - # (image_id, xmin, ymin, width, height, score, class) - for i in range(num_detections): - rle = maskUtils.encode(np.asfortranarray(detection_masks[i]).astype(np.uint8)) - rle['counts'] = rle['counts'].decode('ascii') # json.dump doesn't like bytes strings - detection_category_id = detection_classes[i] + 1 - self._mask_data.append({ - 'image_id': int(gt_data_image_id[batch_idx]), - 'category_id': int(self._label_inv_map[detection_category_id]), - 'segmentation': rle, - 'bbox': [bbox[i, 0], bbox[i, 1], bbox[i, 2], bbox[i, 3]], - 'score': float(detection_scores[i]) - }) - - new_detections = np.hstack([image_id_vec, bbox, detection_scores[:, np.newaxis], - detection_classes[:, np.newaxis]])[:num_detections, :] - self._detections = np.vstack([self._detections, new_detections]) - - for num_boxes, image_id, width, height, bboxes, area, category_id, is_crowd in zip(gt_data_num_boxes, - gt_data_image_id, - gt_data_width, - gt_data_height, - gt_data_bbox, - gt_data_area, - gt_data_category_id, - gt_data_is_crowd): - def _add_gt_bbox(annotation_id, image_id, category_id, bbox, is_crowd, area): - self._dataset['annotations'].append({'id': int(annotation_id), - 'image_id': int(image_id), - 'category_id': int(category_id), - 'bbox': bbox, - 'area': area, - 'iscrowd': is_crowd}) - for i in range(num_boxes): - if category_id[i] in self._label_map.keys(): - cat = self._label_map[category_id[i]] - 1 - _add_gt_bbox(annotation_id=self._annotation_id, image_id=image_id, - category_id=cat, - bbox=bboxes[i, :], is_crowd=is_crowd[i], area=area[i]) - self._category_ids.add(cat) - self._annotation_id += 1 - self._images.add(image_id) - self._dataset['images'] = [{'id': int(img_id)} for img_id in self._images] - self._dataset['categories'] = [{'id': int(category_id)} for category_id in self._category_ids] - - def _evaluate_mask(self): - gt_annotations = COCO(self._gt_ann_file) - mask_dets = gt_annotations.loadRes(self._mask_data) - seg_eval = COCOeval(gt_annotations, mask_dets, 'segm') - seg_eval.params.imgIds = list(self._images) - seg_eval.evaluate() - seg_eval.accumulate() - seg_eval.summarize() - self._metrics_vals += list(np.array(seg_eval.stats, dtype=np.float32)) - - def evaluate(self): - """Evaluates with detections from all images in our data set with COCO API. - - Returns: - coco_metric: float numpy array with shape [12] representing the - coco-style evaluation metrics. - """ - self._coco_gt.dataset = self._dataset - - self._coco_gt.createIndex() - detections = self._detections - coco_dt = self._coco_gt.loadRes(detections) - coco_eval = COCOeval(self._coco_gt, coco_dt, iouType='bbox') - coco_eval.params.imgIds = list(self._images) - coco_eval.evaluate() - coco_eval.accumulate() - coco_eval.summarize() - self._metrics_vals = list(np.array(coco_eval.stats, dtype=np.float32)) - self._evaluate_mask() - - def _get_accuracy(self): - return OrderedDict([(self._metric_names[12], self._metrics_vals[12]), - (self._metric_names[13], self._metrics_vals[13]), - (self._metric_names[0], self._metrics_vals[0]), - (self._metric_names[1], self._metrics_vals[1])]) - - -class YolactEval(Eval): +class InstanceSegmentationEval(Eval): """COCO evaluation metric class.""" def __init__(self, **kwargs): """Constructs COCO evaluation class. - - The class provides the interface to metrics_fn in TPUEstimator. The - _update_op() takes detections from each image and push them to - self._detections. The _evaluate() loads a JSON file in COCO annotation format - as the groundtruths and runs COCO evaluation. - """ + self.meta_arch = kwargs.get('meta_arch', None) + self.mask_thresh = kwargs.get('mask_thresh', None) + self.input_shape = kwargs.get('input_shape', None) dataset_name = kwargs.get('dataset_name', None) dataset_info = get_dataset_info(dataset_name=dataset_name) self._label_map = dataset_info.label_map self._label_inv_map = {v: k for k, v in self._label_map.items()} - self._metric_names = ['mask AP', 'mask AP50', 'mask AP75', 'mask APs', 'mask APm', - 'mask APl', 'mask ARmax1', 'mask ARmax10', 'mask ARmax100', - 'mask ARs', 'mask ARm', 'mask ARl' - 'bbox AP', 'bbox AP50', 'bbox AP75', 'bbox APs', 'bbox APm', - 'bbox APl', 'bbox ARmax1', 'bbox ARmax10', 'bbox ARmax100', - 'bbox ARs', 'bbox ARm', 'bbox ARl'] - self._metrics_vals = [0, 0] - self._channels_remove = kwargs["channels_remove"] if kwargs["channels_remove"]["enabled"] else None if self._channels_remove: self.cls_mapping, self.catIds = self._create_class_mapping() else: self._channels_remove = None self._mask_data = [] + + self.eval_config = EVAL_CLASS_MAP[self.meta_arch]() + self._metric_names = self.eval_config._metric_names + self._metrics_vals = [0] * len(self._metric_names) + self.scale_boxes = self.eval_config.scale_boxes + self.scale_masks = self.eval_config.scale_masks + self.eval_mask = self.eval_config.eval_mask + self.eval_bbox = self.eval_config.eval_bbox + self._gt_ann_file = kwargs['gt_json_path'] - self._top_k = 200 self.reset() def _create_class_mapping(self): @@ -415,48 +78,44 @@ def reset(self): def _parse_net_output(self, net_output): return net_output['predictions'] - def update_op(self, net_output, img_info): - net_output = self._parse_net_output(net_output) + def detections_to_coco(self, boxes, detection_scores, detection_classes, detection_masks, img_id): + bbox = np.transpose(np.vstack([boxes[:, 0], + boxes[:, 1], + boxes[:, 2] - boxes[:, 0], + boxes[:, 3] - boxes[:, 1]])) + # Concat image id per detection in the first column + image_id_vec = np.ones(shape=(bbox.shape[0], 1)) * img_id + # Finalize the detections np.array as shape=(num_detections, 7) + # (image_id, xmin, ymin, width, height, score, class) + num_detections = detection_classes.shape[0] + for i in range(num_detections): + rle = maskUtils.encode(np.asfortranarray(detection_masks[i]).astype(np.uint8)) + rle['counts'] = rle['counts'].decode('ascii') # json.dump doesn't like bytes strings + detection_category_id = detection_classes[i] + 1 \ + if not self._channels_remove else self.cls_mapping[detection_classes[i] + 1] + self._mask_data.append({ + 'image_id': int(img_id), + 'category_id': int(self._label_inv_map[detection_category_id]), + 'segmentation': rle, + 'bbox': [bbox[i, 0], bbox[i, 1], bbox[i, 2], bbox[i, 3]], + 'score': float(detection_scores[i]) + }) + if self._channels_remove: + detection_classes[i] = self.cls_mapping[detection_classes[i] + 1] - 1 + + new_detections = np.hstack([image_id_vec, bbox, detection_scores[:, np.newaxis], + detection_classes[:, np.newaxis]])[:num_detections, :] + self._detections = np.vstack([self._detections, new_detections]) + + def arrange_dataset(self, img_info): gt_data_height = img_info["height"] gt_data_width = img_info["width"] gt_data_image_id = img_info["image_id"] gt_data_num_boxes = img_info["num_boxes"] - gt_data_bbox = img_info["bbox"] + gt_data_bbox = img_info.get("bbox", np.zeros((gt_data_width.shape[0], 100, 4))) gt_data_area = img_info["area"] gt_data_category_id = img_info["category_id"] gt_data_is_crowd = img_info["is_crowd"] - for batch_idx in range(len(net_output)): - t = postprocess(net_output, gt_data_width[batch_idx], gt_data_height[batch_idx], batch_idx=batch_idx) - if t[0] is not None: - detection_classes, detection_scores, boxes, detection_masks = [x[:self._top_k] for x in t[:4]] - num_detections = min(self._top_k, detection_classes.shape[0]) - # Rearrange detections bbox to be in the coco format - xmin, ymin, width, height - bbox = np.transpose(np.vstack([boxes[:, 0], boxes[:, 1], - boxes[:, 2] - boxes[:, 0], - boxes[:, 3] - boxes[:, 1]])) - # Concat image id per detection in the first column - image_id_vec = np.ones(shape=(bbox.shape[0], 1)) * gt_data_image_id[batch_idx] - # Finalize the detections np.array as shape=(num_detections, 7) - # (image_id, xmin, ymin, width, height, score, class) - for i in range(num_detections): - rle = maskUtils.encode(np.asfortranarray(detection_masks[i]).astype(np.uint8)) - rle['counts'] = rle['counts'].decode('ascii') # json.dump doesn't like bytes strings - detection_category_id = detection_classes[i] + 1 \ - if not self._channels_remove else self.cls_mapping[detection_classes[i] + 1] - self._mask_data.append({ - 'image_id': int(gt_data_image_id[batch_idx]), - 'category_id': int(self._label_inv_map[detection_category_id]), - 'segmentation': rle, - 'bbox': [bbox[i, 0], bbox[i, 1], bbox[i, 2], bbox[i, 3]], - 'score': float(detection_scores[i]) - }) - if self._channels_remove: - detection_classes[i] = self.cls_mapping[detection_classes[i] + 1] - 1 - - new_detections = np.hstack([image_id_vec, bbox, detection_scores[:, np.newaxis], - detection_classes[:, np.newaxis]])[:num_detections, :] - self._detections = np.vstack([self._detections, new_detections]) - for num_boxes, image_id, width, height, bboxes, area, category_id, is_crowd in zip(gt_data_num_boxes, gt_data_image_id, gt_data_width, @@ -485,6 +144,36 @@ def _add_gt_bbox(annotation_id, image_id, category_id, bbox, is_crowd, area): self._dataset['images'] = [{'id': int(img_id)} for img_id in self._images] self._dataset['categories'] = [{'id': int(category_id)} for category_id in self._category_ids] + def update_op(self, net_output, img_info): + net_output = self._parse_net_output(net_output) + gt_data_height = img_info["height"] + gt_data_width = img_info["width"] + gt_data_image_id = img_info["image_id"] + + for batch_idx, output in enumerate(net_output): + output_shape = gt_data_height[batch_idx], gt_data_width[batch_idx] + + num_detections = output['detection_classes'].shape[0] + detection_boxes = output.get('detection_boxes', np.zeros((num_detections, 4))) + detection_masks = output['mask'] + detection_scores = output['detection_scores'] + detection_classes = output['detection_classes'] + if num_detections: + detection_boxes = self.scale_boxes(detection_boxes, shape_out=output_shape, shape_in=self.input_shape) + detection_masks = self.scale_masks(detection_masks, shape_out=output_shape, shape_in=self.input_shape) + # Binarize masks + detection_masks = detection_masks > self.mask_thresh + + # Rearrange detections bbox to be in the coco format - xmin, ymin, width, height + self.detections_to_coco(detection_boxes, + detection_scores, + detection_classes, + detection_masks, + gt_data_image_id[batch_idx]) + + # Arrange annotations + self.arrange_dataset(img_info) + def _evaluate_mask(self): gt_annotations = COCO(self._gt_ann_file) mask_dets = gt_annotations.loadRes(self._mask_data) @@ -517,11 +206,11 @@ def evaluate(self): coco-style evaluation metrics. """ self._coco_gt.dataset = self._dataset - self._coco_gt.createIndex() - - self._evaluate_mask() - self._evaluate_bbox() + if self.eval_mask: + self._evaluate_mask() + if self.eval_bbox: + self._evaluate_bbox() def _get_accuracy(self): return OrderedDict([(self._metric_names[0], self._metrics_vals[0]), diff --git a/hailo_model_zoo/core/eval/instance_segmentation_evaluation_utils.py b/hailo_model_zoo/core/eval/instance_segmentation_evaluation_utils.py new file mode 100644 index 00000000..253ba7ed --- /dev/null +++ b/hailo_model_zoo/core/eval/instance_segmentation_evaluation_utils.py @@ -0,0 +1,104 @@ +import cv2 +import numpy as np + +from hailo_model_zoo.core.postprocessing.instance_segmentation_postprocessing import _sanitize_coordinates + +BBOX_METRIC_NAMES = ['bbox AP', 'bbox AP50', 'bbox AP75', 'bbox APs', 'bbox APm', + 'bbox APl', 'bbox ARmax1', 'bbox ARmax10', 'bbox ARmax100', + 'bbox ARs', 'bbox ARm', 'bbox ARl'] + +SEGM_METRIC_NAMES = ['mask AP', 'mask AP50', 'mask AP75', 'mask APs', 'mask APm', + 'mask APl', 'mask ARmax1', 'mask ARmax10', 'mask ARmax100', + 'mask ARs', 'mask ARm', 'mask ARl'] + + +class InstSegEvalBase(): + + def __init__(self): + self.eval_bbox = True + self.eval_mask = True + self._metric_names = SEGM_METRIC_NAMES + BBOX_METRIC_NAMES + + def scale_boxes(self, boxes, **kwargs): + return boxes + + def scale_masks(self, masks, **kwargs): + return masks + + +class YolactEval(InstSegEvalBase): + + def __init__(self): + super().__init__() + + def scale_boxes(self, boxes, shape_out, **kwargs): + boxes[:, 0], boxes[:, 2] = _sanitize_coordinates(boxes[:, 0], boxes[:, 2], shape_out[1], cast=False) + boxes[:, 1], boxes[:, 3] = _sanitize_coordinates(boxes[:, 1], boxes[:, 3], shape_out[0], cast=False) + boxes = np.array(boxes, np.float64) + return boxes + + def scale_masks(self, masks, shape_out, **kwargs): + masks = cv2.resize(masks, (shape_out[1], shape_out[0])) + if len(masks.shape) < 3: + masks = np.expand_dims(masks, axis=0) + else: + masks = np.transpose(masks, (2, 0, 1)) + return masks + + +class Yolov5SegEval(InstSegEvalBase): + + def __init__(self): + super().__init__() + + def scale_boxes(self, boxes, shape_out, shape_in=None, **kwargs): + ratio_pad = kwargs.get('ratio_pad', None) + if ratio_pad is None: # calculate from shape_out + if shape_in is None: + raise ValueError('Expected shape_in to be a tuple of size 2 when ratio is not provided but got None') + gain = min(shape_in[0] / shape_out[0], shape_in[1] / shape_out[1]) # gain = old / new + pad = (shape_in[1] - shape_out[1] * gain) / 2,\ + (shape_in[0] - shape_out[0] * gain) / 2 # wh padding + else: + gain = ratio_pad[0][0] + pad = ratio_pad[1] + + boxes[:, [0, 2]] -= pad[0] # x padding + boxes[:, [1, 3]] -= pad[1] # y padding + boxes[:, :4] /= gain + + # Clip boxes (xyxy) to image shape (height, width) + boxes[:, [0, 2]] = boxes[:, [0, 2]].clip(0, shape_out[1]) # x1, x2 + boxes[:, [1, 3]] = boxes[:, [1, 3]].clip(0, shape_out[0]) # y1, y2 + + return boxes + + def scale_masks(self, masks, shape_out, shape_in=None, **kwargs): + ratio_pad = kwargs.get('ratio_pad', None) + if ratio_pad is None: # calculate from shape_out and shape_in + if shape_in is None: + raise ValueError('Expected shape_in to be a tuple of size 2 but got None') + gain = min(shape_in[0] / shape_out[0], shape_in[1] / shape_out[1]) # gain = old / new + pad = (shape_in[1] - shape_out[1] * gain) / 2, (shape_in[0] - shape_out[0] * gain) / 2 # wh padding + else: + pad = ratio_pad[1] + top, left = int(pad[1]), int(pad[0]) # y, x + bottom, right = int(shape_in[0] - pad[1]), int(shape_in[1] - pad[0]) + + masks = masks.transpose((1, 2, 0)) + if len(masks.shape) < 2: + raise ValueError(f'"len of masks shape" should be 2 or 3, but got {len(masks.shape)}') + masks = masks[top:bottom, left:right] + masks = cv2.resize(masks, (shape_out[1], shape_out[0])) + + if len(masks.shape) == 2: + masks = masks[:, :, None] + masks = masks.transpose((2, 0, 1)) + return masks + + +class SparseInstEval(InstSegEvalBase): + def __init__(self): + super().__init__() + self.eval_bbox = False + self._metric_names = SEGM_METRIC_NAMES diff --git a/hailo_model_zoo/core/eval/pose_estimation_evaluation.py b/hailo_model_zoo/core/eval/pose_estimation_evaluation.py index fd48ac0d..7706a86d 100644 --- a/hailo_model_zoo/core/eval/pose_estimation_evaluation.py +++ b/hailo_model_zoo/core/eval/pose_estimation_evaluation.py @@ -33,14 +33,15 @@ def _parse_net_output(self, net_output): return net_output['predictions'] def update_op(self, net_output, img_info): - net_output = self._parse_net_output(net_output) if "center" not in img_info: # OpenPose Evaluation + net_output = self._parse_net_output(net_output) self._coco_result += net_output return # CenterPose evaluation - bboxes, scores, keypoints, joint_scores = net_output + bboxes, scores, keypoints, joint_scores = ( + net_output['bboxes'], net_output['scores'], net_output['keypoints'], net_output['joint_scores']) batch_size = bboxes.shape[0] for batch_index in range(batch_size): diff --git a/hailo_model_zoo/core/eval/tracking_evaluation_external/matching.py b/hailo_model_zoo/core/eval/tracking_evaluation_external/matching.py index c537dc58..cdf3fb02 100644 --- a/hailo_model_zoo/core/eval/tracking_evaluation_external/matching.py +++ b/hailo_model_zoo/core/eval/tracking_evaluation_external/matching.py @@ -49,13 +49,13 @@ def linear_assignment(cost_matrix, thresh): def ious(atlbrs, btlbrs): - ious = np.zeros((len(atlbrs), len(btlbrs)), dtype=np.float) + ious = np.zeros((len(atlbrs), len(btlbrs)), dtype=float) if ious.size == 0: return ious ious = bbox_overlaps( - np.ascontiguousarray(atlbrs, dtype=np.float), - np.ascontiguousarray(btlbrs, dtype=np.float) + np.ascontiguousarray(atlbrs, dtype=float), + np.ascontiguousarray(btlbrs, dtype=float) ) return ious @@ -76,13 +76,13 @@ def iou_distance(atracks, btracks): def embedding_distance(tracks, detections, metric='cosine'): - cost_matrix = np.zeros((len(tracks), len(detections)), dtype=np.float) + cost_matrix = np.zeros((len(tracks), len(detections)), dtype=float) if cost_matrix.size == 0: return cost_matrix - det_features = np.asarray([track.curr_feat for track in detections], dtype=np.float) + det_features = np.asarray([track.curr_feat for track in detections], dtype=float) # for i, track in enumerate(tracks): # cost_matrix[i, :] = np.maximum(0.0, cdist(track.smooth_feat.reshape(1,-1), det_features, metric)) - track_features = np.asarray([track.smooth_feat for track in tracks], dtype=np.float) + track_features = np.asarray([track.smooth_feat for track in tracks], dtype=float) cost_matrix = np.maximum(0.0, cdist(track_features, det_features, metric)) # Nomalized features return cost_matrix diff --git a/hailo_model_zoo/core/eval/tracking_evaluation_external/tracking_classes.py b/hailo_model_zoo/core/eval/tracking_evaluation_external/tracking_classes.py index 874e1e05..b396d2e2 100644 --- a/hailo_model_zoo/core/eval/tracking_evaluation_external/tracking_classes.py +++ b/hailo_model_zoo/core/eval/tracking_evaluation_external/tracking_classes.py @@ -61,7 +61,7 @@ class STrack(BaseTrack): def __init__(self, tlwh, score, temp_feat, buffer_size=30): # wait activate - self._tlwh = np.asarray(tlwh, dtype=np.float) + self._tlwh = np.asarray(tlwh, dtype=float) self.kalman_filter = None self.mean, self.covariance = None, None self.is_activated = False diff --git a/hailo_model_zoo/core/eval/widerface_evaluation_external/box_overlaps.pyx b/hailo_model_zoo/core/eval/widerface_evaluation_external/box_overlaps.pyx index 6365c866..9fc69530 100644 --- a/hailo_model_zoo/core/eval/widerface_evaluation_external/box_overlaps.pyx +++ b/hailo_model_zoo/core/eval/widerface_evaluation_external/box_overlaps.pyx @@ -5,7 +5,7 @@ import numpy as np cimport numpy as np -DTYPE = np.float +DTYPE = float ctypedef np.float_t DTYPE_t def bbox_overlaps( diff --git a/hailo_model_zoo/core/eval/widerface_evaluation_external/python_box_overlaps.py b/hailo_model_zoo/core/eval/widerface_evaluation_external/python_box_overlaps.py index db76292f..dafaa284 100644 --- a/hailo_model_zoo/core/eval/widerface_evaluation_external/python_box_overlaps.py +++ b/hailo_model_zoo/core/eval/widerface_evaluation_external/python_box_overlaps.py @@ -4,7 +4,7 @@ def bbox_overlaps(boxes, query_boxes): boxes_count = boxes.shape[0] secondary_boxes_count = query_boxes.shape[0] - overlaps = np.zeros((boxes_count, secondary_boxes_count), dtype=np.float) + overlaps = np.zeros((boxes_count, secondary_boxes_count), dtype=float) for k in range(secondary_boxes_count): box_area = ((query_boxes[k, 2] - query_boxes[k, 0] + 1) * (query_boxes[k, 3] - query_boxes[k, 1] + 1)) for n in range(boxes_count): diff --git a/hailo_model_zoo/core/hn_editor/channel_remove.py b/hailo_model_zoo/core/hn_editor/channel_remove.py index 4cc44299..a75870be 100644 --- a/hailo_model_zoo/core/hn_editor/channel_remove.py +++ b/hailo_model_zoo/core/hn_editor/channel_remove.py @@ -15,16 +15,16 @@ def _channel_remove(db, layer, mask, orig_shape, npz): if layer in k: if len(npz[k].shape) == 1: # bias - npz[k] = v[np.array(mask, np.bool)] + npz[k] = v[np.array(mask, bool)] elif len(npz[k].shape) == 4: # kernel sh = db['layers'][layer]['params']['kernel_shape'] if sh[2] == orig_shape: - tmp = v[:, :, :, np.array(mask, np.bool)] - npz[k] = tmp[:, :, np.array(mask, np.bool), :] + tmp = v[:, :, :, np.array(mask, bool)] + npz[k] = tmp[:, :, np.array(mask, bool), :] db['layers'][layer]['params']['kernel_shape'] = [sh[0], sh[1], int(sum(mask)), int(sum(mask))] else: - npz[k] = v[:, :, :, np.array(mask, np.bool)] + npz[k] = v[:, :, :, np.array(mask, bool)] db['layers'][layer]['params']['kernel_shape'] = [sh[0], sh[1], sh[2], int(sum(mask))] diff --git a/hailo_model_zoo/core/infer/infer_factory.py b/hailo_model_zoo/core/infer/infer_factory.py index 2d775f36..779861d1 100644 --- a/hailo_model_zoo/core/infer/infer_factory.py +++ b/hailo_model_zoo/core/infer/infer_factory.py @@ -1,20 +1,23 @@ """Contains a factory for network infer.""" from hailo_model_zoo.core.infer.tf_infer import tf_infer -from hailo_model_zoo.core.infer.np_infer import np_infer +from hailo_model_zoo.core.infer.model_infer import model_infer +from hailo_model_zoo.core.infer.model_infer_lite import model_infer_lite try: # THIS CODE IS EXPERIMENTAL AND IN USE ONLY FOR TAPPAS VALIDATION from hailo_model_zoo.core.infer.so_infer import so_infer except ModuleNotFoundError: so_infer = None -from hailo_model_zoo.core.infer.face_infer import facenet_infer from hailo_model_zoo.core.infer.tf_infer_second_stage import tf_infer_second_stage from hailo_model_zoo.core.infer.runner_infer import runner_infer NAME_TO_INFER = { 'tf_infer': tf_infer, 'runner_infer': runner_infer, - 'np_infer': np_infer, - 'facenet_infer': facenet_infer, + 'np_infer': lambda *args, **kwargs: model_infer(*args, **kwargs, np_infer=True), + 'facenet_infer': model_infer, + 'model_infer': model_infer, + 'model_infer_lite': model_infer_lite, + 'np_infer_lite': lambda *args, **kwargs: model_infer_lite(*args, **kwargs, np_infer=True), 'so_infer': so_infer, 'tf_infer_second_stage': tf_infer_second_stage, } diff --git a/hailo_model_zoo/core/infer/infer_utils.py b/hailo_model_zoo/core/infer/infer_utils.py index 68d93cfb..d1843b5b 100644 --- a/hailo_model_zoo/core/infer/infer_utils.py +++ b/hailo_model_zoo/core/infer/infer_utils.py @@ -1,6 +1,12 @@ -import numpy as np import os +import cv2 +import numpy as np + +from PIL import Image + +from hailo_model_zoo.utils.numpy_utils import to_numpy + def save_image(img, image_name): if isinstance(image_name, bytes): @@ -64,14 +70,89 @@ def log_accuracy(logger, num_of_images, accuracies_output): def get_logits_per_image(logits): - if type(logits) is np.ndarray: - # (BATCH, someshape) -> (BATCH, 1, someshape) - return np.expand_dims(logits, axis=1) + assert isinstance(logits, dict), f"Expected logits to be dict but got {type(logits)}" + types = set(type(v) for v in logits.values()) + assert len(types) == 1, f"Assumed dict of lists or dict of arrays but got {types}" + t = list(types)[0] - if type(logits) is list: - return zip(*[get_logits_per_image(logit) for logit in logits]) + if t is np.ndarray: + # (BATCH, someshape) -> (BATCH, 1, someshape) + expanded_vals = [np.expand_dims(v, axis=1) for v in logits.values()] + return [dict(zip(logits, v)) for v in zip(*expanded_vals)] - if type(logits) is dict: - return [dict(zip(logits, t)) for t in get_logits_per_image(list(logits.values()))] + if t is list: + return [dict(zip(logits, v)) for v in zip(*logits.values())] raise ValueError("Unsupported type {} for logits".format(type(logits))) + + +class ImageSaver: + def __enter__(self): + return self + + def __exit__(self, exc_type, exc_value, exc_traceback): + pass + + def write(self, image, image_name): + save_image(Image.fromarray(image), image_name) + + +class VideoWriter: + def __init__(self, width, height, video_outpath): + self.video_writer = cv2.VideoWriter(video_outpath, + cv2.VideoWriter_fourcc(*'mp4v'), + 24, + (width, height)) + + def __enter__(self): + return self + + def __exit__(self, exc_type, exc_value, exc_traceback): + self.video_writer.release() + + def write(self, image, image_name): + self.video_writer.write(image) + + +def _make_writer(info_per_image, video_outpath): + if not video_outpath: + writer = ImageSaver() + else: + ref_image = info_per_image[0]['img_orig'] + width, height = ref_image.shape[2], ref_image.shape[1] + writer = VideoWriter(width, height, video_outpath) + return writer + + +def visualize(logits_batch, info_per_image, visualize_callback, video_outpath): + with _make_writer(info_per_image, video_outpath) as writer: + logits_per_image = get_logits_per_image(logits_batch) + for image_index, (image_logits, image_info) in enumerate(zip(logits_per_image, info_per_image)): + original_image = image_info['img_orig'] + original_image = to_numpy(original_image) + image_name = image_info.get('image_name', f'image{image_index}') + image_name = to_numpy(image_name, decode=True) + # Decode image if needed + if type(original_image) is bytes: + original_image = cv2.imdecode(np.fromstring(original_image, dtype=np.uint8), + cv2.IMREAD_UNCHANGED) + original_image = np.expand_dims(original_image, axis=0) + + image = visualize_callback(image_logits, original_image, img_info=image_info, image_name=image_name) + writer.write(image, image_name) + + +def aggregate(elements): + if not elements: + return elements + e = elements[0] + # we got a list instead of tensor - flatten the list of lists + if isinstance(e, list): + return [item for sublist in elements for item in sublist] + + # we got primitives - collect them to an array + if len(e.shape) == 0: + return np.array(elements) + + # we got multiple numpy arrays, concatenate them + return np.concatenate(elements, axis=0) diff --git a/hailo_model_zoo/core/infer/model_infer.py b/hailo_model_zoo/core/infer/model_infer.py new file mode 100644 index 00000000..a7d2972e --- /dev/null +++ b/hailo_model_zoo/core/infer/model_infer.py @@ -0,0 +1,62 @@ +import tensorflow as tf +from tqdm import tqdm + +from hailo_model_zoo.core.infer.infer_utils import log_accuracy, write_results, aggregate, visualize, to_numpy + + +def model_infer(runner, context, logger, eval_num_examples, print_num_examples, + batch_size, dataset, postprocessing_callback, + eval_callback, visualize_callback, model_augmentation_callback, + video_outpath, dump_results, results_path, *, np_infer=False): + eval_metric = eval_callback() + if not np_infer: + postprocessing_callback = tf.function(postprocessing_callback, reduce_retracing=True) + if eval_num_examples: + dataset = dataset.take(eval_num_examples) + batched_dataset = dataset.batch(batch_size) + logger.info('Running inference...') + with context as ctx, tqdm(total=None, desc="Processed", unit="images", + disable=None if not print_num_examples < 1 else True) as pbar: + model = runner.get_keras_model(ctx) + model = model_augmentation_callback(model) + + @tf.function() + def predict_function(data): + return model(data, training=False) + + num_of_images = 0 + logits = [] + gt = [] + for preprocessed_data, img_info in batched_dataset: + output_tensors = predict_function(preprocessed_data) + if np_infer: + output_tensors = to_numpy(output_tensors) + img_info = to_numpy(img_info) + logits_batch = postprocessing_callback(output_tensors, gt_images=img_info) + current_batch_size = (output_tensors[0].shape[0] if isinstance(output_tensors, list) + else output_tensors.shape[0]) + num_of_images += current_batch_size + pbar.update(current_batch_size) + logits.append(logits_batch) + if not visualize_callback: + if "img_orig" in img_info: + del img_info["img_orig"] + if "img_resized" in img_info: + del img_info["img_resized"] + gt.append(to_numpy(img_info)) + labels_keys = list(gt[0].keys()) + labels = {k: aggregate([p[k] for p in gt]) for k in labels_keys} + probs = {k: aggregate([p[k] for p in logits]) for k in logits[0].keys()} + accuracy = None + if not visualize_callback and not dump_results: + eval_metric.update_op(probs, labels) + eval_metric.evaluate() + accuracy = eval_metric.get_accuracy() + log_accuracy(logger, num_of_images, accuracy) + if dump_results: + write_results(probs, labels, results_path) + + if visualize_callback: + img_info_per_image = [x[1] for x in dataset] + visualize(probs, img_info_per_image, visualize_callback, video_outpath) + return accuracy diff --git a/hailo_model_zoo/core/infer/model_infer_lite.py b/hailo_model_zoo/core/infer/model_infer_lite.py new file mode 100644 index 00000000..cc6f27db --- /dev/null +++ b/hailo_model_zoo/core/infer/model_infer_lite.py @@ -0,0 +1,62 @@ +import tensorflow as tf +from tqdm import tqdm + +from hailo_model_zoo.core.infer.infer_utils import log_accuracy, aggregate, visualize, to_numpy + + +def model_infer_lite(runner, context, logger, eval_num_examples, print_num_examples, + batch_size, dataset, postprocessing_callback, + eval_callback, visualize_callback, model_augmentation_callback, + video_outpath, dump_results, results_path, *, np_infer=False): + eval_metric = eval_callback() + if not np_infer: + postprocessing_callback = tf.function(postprocessing_callback, reduce_retracing=True) + if eval_num_examples: + dataset = dataset.take(eval_num_examples) + batched_dataset = dataset.batch(batch_size) + logger.info('Running inference...') + with context as ctx, tqdm(total=None, desc="Processed", unit="images", + disable=None if not print_num_examples < 1 else True) as pbar: + model = runner.get_keras_model(ctx) + model = model_augmentation_callback(model) + + @tf.function() + def predict_function(data): + return model(data, training=False) + + num_of_images = 0 + logits = [] + gt = [] + for preprocessed_data, img_info in batched_dataset: + output_tensors = predict_function(preprocessed_data) + if np_infer: + output_tensors = to_numpy(output_tensors) + img_info = to_numpy(img_info) + logits_batch = postprocessing_callback(output_tensors, gt_images=img_info) + current_batch_size = (output_tensors[0].shape[0] if isinstance(output_tensors, list) + else output_tensors.shape[0]) + num_of_images += current_batch_size + if not visualize_callback: + if "img_orig" in img_info: + del img_info["img_orig"] + if "img_resized" in img_info: + del img_info["img_resized"] + image_info = to_numpy(img_info) + if not visualize_callback and not dump_results: + eval_metric.update_op(logits_batch, image_info) + if visualize_callback: + logits.append(logits_batch) + gt.append(image_info) + pbar.update(current_batch_size) + + accuracy = None + if not visualize_callback and not dump_results: + eval_metric.evaluate() + accuracy = eval_metric.get_accuracy() + log_accuracy(logger, num_of_images, accuracy) + + if visualize_callback: + probs = {k: aggregate([p[k] for p in logits]) for k in logits[0].keys()} + img_info_per_image = [x[1] for x in dataset] + visualize(probs, img_info_per_image, visualize_callback, video_outpath) + return accuracy diff --git a/hailo_model_zoo/core/infer/np_infer.py b/hailo_model_zoo/core/infer/np_infer.py index 286f51c7..7eab0fc3 100644 --- a/hailo_model_zoo/core/infer/np_infer.py +++ b/hailo_model_zoo/core/infer/np_infer.py @@ -34,7 +34,9 @@ def np_infer(runner, target, logger, eval_num_examples, print_num_examples, break logits_batch, img_info = sdk_export.session.run([sdk_export.output_tensors, image_info]) num_of_images += batch_size - probs = postprocessing_callback(logits_batch, image_info=img_info) + if len(logits_batch) == 1: + logits_batch = logits_batch[0] + probs = postprocessing_callback(logits_batch, gt_images=img_info) if not visualize_callback and not dump_results: eval_metric.update_op(probs, img_info) if num_of_images % print_num_examples == 0: diff --git a/hailo_model_zoo/core/infer/runner_infer.py b/hailo_model_zoo/core/infer/runner_infer.py index ed184cb4..476d12b0 100644 --- a/hailo_model_zoo/core/infer/runner_infer.py +++ b/hailo_model_zoo/core/infer/runner_infer.py @@ -1,86 +1,15 @@ -import cv2 -import numpy as np -import tensorflow as tf -from PIL import Image -from hailo_model_zoo.core.infer.infer_utils import log_accuracy, write_results, save_image, get_logits_per_image +from hailo_model_zoo.core.infer.infer_utils import log_accuracy, write_results, aggregate, visualize -class ImageSaver: - def __enter__(self): - return self - - def __exit__(self, exc_type, exc_value, exc_traceback): - pass - - def write(self, image, image_name): - save_image(Image.fromarray(image), image_name) - - -class VideoWriter: - def __init__(self, width, height, video_outpath): - self.video_writer = cv2.VideoWriter(video_outpath, - cv2.VideoWriter_fourcc(*'mp4v'), - 24, - (width, height)) - - def __enter__(self): - return self - - def __exit__(self, exc_type, exc_value, exc_traceback): - self.video_writer.release() - - def write(self, image, image_name): - self.video_writer.write(image) - - -def _make_writer(info_per_image, video_outpath): - if not video_outpath: - writer = ImageSaver() - else: - ref_image = info_per_image[0]['img_orig'] - width, height = ref_image.shape[2], ref_image.shape[1] - writer = VideoWriter(width, height, video_outpath) - return writer - - -def _visualize(logits_batch, info_per_image, visualize_callback, video_outpath): - with _make_writer(info_per_image, video_outpath) as writer: - logits_per_image = get_logits_per_image(logits_batch) - for image_index, (image_logits, image_info) in enumerate(zip(logits_per_image, info_per_image)): - original_image = image_info['img_orig'] - image_name = image_info.get('image_name', f'image{image_index}') - if hasattr(image_name, 'numpy'): - image_name = image_name.numpy().decode('utf8') - # Decode image if needed - if type(original_image) is bytes: - original_image = cv2.imdecode(np.fromstring(original_image, dtype=np.uint8), - cv2.IMREAD_UNCHANGED) - original_image = np.expand_dims(original_image, axis=0) - - image = visualize_callback(image_logits, original_image, img_info=image_info, image_name=image_name) - writer.write(image, image_name) - - -def aggregate(elements): - if not elements: - return elements - e = elements[0] - assert tf.is_tensor(e) - if e.dtype == tf.string: - return elements - if len(e.shape) == 0: - return np.array(elements) - return np.concatenate(elements, axis=0) - - -def runner_infer(runner, target, logger, eval_num_examples, print_num_examples, +def runner_infer(runner, context, logger, eval_num_examples, print_num_examples, batch_size, dataset, postprocessing_callback, - eval_callback, visualize_callback, video_outpath, dump_results, results_path): + eval_callback, visualize_callback, model_wrapper_callback, + video_outpath, dump_results, results_path): eval_metric = eval_callback() logger.info('Running inference...') - - logits = runner.infer(target, dataset, batch_size=batch_size, data_count=eval_num_examples) + with context as ctx: + logits = runner.infer(context=ctx, dataset=dataset, batch_size=batch_size, data_count=eval_num_examples) num_of_images = logits.shape[0] img_info_per_image = [x[1] for x in dataset.take(num_of_images)] img_info = {k: aggregate([p[k] for p in img_info_per_image]) for k in img_info_per_image[0].keys()} @@ -95,6 +24,6 @@ def runner_infer(runner, target, logger, eval_num_examples, print_num_examples, write_results(logits, img_info, results_path) if visualize_callback: - _visualize(probs, img_info_per_image, visualize_callback, video_outpath) + visualize(probs, img_info_per_image, visualize_callback, video_outpath) return accuracy diff --git a/hailo_model_zoo/core/main_utils.py b/hailo_model_zoo/core/main_utils.py index 26f33d97..109dc908 100644 --- a/hailo_model_zoo/core/main_utils.py +++ b/hailo_model_zoo/core/main_utils.py @@ -3,6 +3,7 @@ from hailo_model_zoo.core.info_utils import get_network_info # noqa (F401) - exports this function for backwards compat +from hailo_model_zoo.core.augmentations import make_model_callback from hailo_model_zoo.core.infer import infer_factory from hailo_model_zoo.core.eval import eval_factory from hailo_model_zoo.core.postprocessing import postprocessing_factory @@ -49,13 +50,13 @@ def _apply_output_scheme(runner, network_info): change_activations(runner, activation_changes) -def _get_integrated_postprocessing(network_info): +def get_integrated_postprocessing(network_info): output_scheme = network_info.hn_editor.output_scheme return output_scheme.integrated_postprocessing def _add_postprocess(runner, network_info): - integrated_postprocessing = _get_integrated_postprocessing(network_info) + integrated_postprocessing = get_integrated_postprocessing(network_info) if integrated_postprocessing and integrated_postprocessing.enabled: integrate_postprocessing(runner, integrated_postprocessing, network_info) @@ -118,6 +119,7 @@ def make_preprocessing(runner, network_info): meta_arch = preprocessing_args.get('meta_arch') hn_editor = network_info.hn_editor yuv2rgb = hn_editor.yuv2rgb + yuy2 = hn_editor.yuy2 input_resize = hn_editor.input_resize normalize_in_net, mean_list, std_list = get_normalization_params(network_info) normalization_params = [mean_list, std_list] if not normalize_in_net else None @@ -125,7 +127,7 @@ def make_preprocessing(runner, network_info): flip = runner.get_hn_model().is_transposed() preproc_callback = preprocessing_factory.get_preprocessing( - meta_arch, height=height, width=width, flip=flip, yuv2rgb=yuv2rgb, + meta_arch, height=height, width=width, flip=flip, yuv2rgb=yuv2rgb, yuy2=yuy2, input_resize=input_resize, normalization_params=normalization_params, **preprocessing_args) @@ -230,12 +232,13 @@ def postprocessing_callback(endnodes, gt_images=None, image_info=None, **kwargs) return postprocessing_callback -def make_eval_callback(network_info): +def make_eval_callback(network_info, runner): network_type = network_info.evaluation.network_type or network_info.preprocessing.network_type net_name = network_info.network.network_name gt_json_path = network_info.evaluation.gt_json_path + meta_arch = network_info.evaluation.meta_arch gt_json_path = path_resolver.resolve_data_path(gt_json_path) if gt_json_path else None - + input_shape = _get_input_shape(runner, network_info) eval_args = dict( net_name=net_name, network_type=network_type, @@ -250,7 +253,9 @@ def make_eval_callback(network_info): centered=network_info.preprocessing.centered, nms_iou_thresh=network_info.postprocessing.nms_iou_thresh, score_threshold=network_info.postprocessing.score_threshold, - input_shape=network_info.preprocessing.input_shape, + input_shape=input_shape, + meta_arch=meta_arch, + mask_thresh=network_info.postprocessing.mask_threshold, ) evaluation_constructor = eval_factory.get_evaluation(network_type) @@ -261,38 +266,25 @@ def eval_callback(): return eval_callback -def infer_model(runner, network_info, target, logger, eval_num_examples, - data_path, batch_size, print_num_examples=256, visualize_results=False, - video_outpath=None, dump_results=False, network_groups=None): - infer_type = network_info.evaluation.infer_type - if infer_type == 'runner_infer': - return infer_model_tf2(runner, network_info, target, logger, eval_num_examples, data_path, - batch_size, print_num_examples, visualize_results, video_outpath, - dump_results, network_groups) - - return infer_model_tf1(runner, network_info, target, logger, eval_num_examples, data_path, - batch_size, print_num_examples, visualize_results, video_outpath, - dump_results, network_groups) - - def infer_model_tf2(runner, network_info, target, logger, eval_num_examples, data_path, batch_size, print_num_examples=256, visualize_results=False, - video_outpath=None, dump_results=False, network_groups=None): + video_outpath=None, dump_results=False): logger.info('Initializing the dataset ...') preproc_callback = make_preprocessing(runner, network_info) # we do not pass batch_size, batching is now done in infer_callback dataset = make_evalset_callback(network_info, None, preproc_callback, data_path, return_dataset=True) # TODO refactor postprocessing_callback = get_postprocessing_callback(runner, network_info) - eval_callback = make_eval_callback(network_info) + eval_callback = make_eval_callback(network_info, runner) visualize_callback = make_visualize_callback(network_info) if visualize_results else None + model_wrapper_callback = make_model_callback(network_info) infer_type = network_info.evaluation.infer_type infer_callback = infer_factory.get_infer(infer_type) return infer_callback(runner, target, logger, eval_num_examples, print_num_examples, batch_size, - dataset, postprocessing_callback, eval_callback, - visualize_callback, video_outpath, dump_results, results_path=None) + dataset, postprocessing_callback, eval_callback, visualize_callback, + model_wrapper_callback, video_outpath, dump_results, results_path=None) def infer_model_tf1(runner, network_info, target, logger, eval_num_examples, @@ -312,7 +304,7 @@ def tf_graph_callback(preprocessed_data, rescale_output=None): return sdk_export postprocessing_callback = get_postprocessing_callback(runner, network_info) - eval_callback = make_eval_callback(network_info) + eval_callback = make_eval_callback(network_info, runner) visualize_callback = make_visualize_callback(network_info) if visualize_results else None infer_type = network_info.evaluation.infer_type @@ -329,7 +321,8 @@ def get_hef_path(results_dir, model_name): def compile_model(runner, network_info, results_dir, allocator_script_filename): model_name = network_info.network.network_name - runner.load_model_script(allocator_script_filename) + if allocator_script_filename is not None: + runner.load_model_script(allocator_script_filename) hef = runner.compile() with open(get_hef_path(results_dir, model_name), "wb") as hef_out_file: diff --git a/hailo_model_zoo/core/postprocessing/centerpose_postprocessing.py b/hailo_model_zoo/core/postprocessing/centerpose_postprocessing.py index ec89a21b..a013f858 100644 --- a/hailo_model_zoo/core/postprocessing/centerpose_postprocessing.py +++ b/hailo_model_zoo/core/postprocessing/centerpose_postprocessing.py @@ -165,4 +165,9 @@ def centerpose_postprocessing(endnodes, device_pre_post_layers=None, gt_images=N gt_images["center"], gt_images["scale"]], [tf.float32, tf.float32, tf.float32, tf.float32], name='centerpose_postprocessing') - return [bboxes, scores, keypoints, joint_scores] + return { + 'bboxes': bboxes, + 'scores': scores, + 'keypoints': keypoints, + 'joint_scores': joint_scores, + } diff --git a/hailo_model_zoo/core/postprocessing/cython_utils/cython_nms.pyx b/hailo_model_zoo/core/postprocessing/cython_utils/cython_nms.pyx index 70f36205..e4efd8fe 100644 --- a/hailo_model_zoo/core/postprocessing/cython_utils/cython_nms.pyx +++ b/hailo_model_zoo/core/postprocessing/cython_utils/cython_nms.pyx @@ -26,7 +26,7 @@ def nms(np.ndarray[np.float32_t, ndim=2] dets, np.float32_t thresh): cdef int ndets = dets.shape[0] cdef np.ndarray[np.int_t, ndim=1] suppressed = \ - np.zeros((ndets), dtype=np.int) + np.zeros((ndets), dtype=int) # nominal indices cdef int _i, _j diff --git a/hailo_model_zoo/core/postprocessing/detection/centernet.py b/hailo_model_zoo/core/postprocessing/detection/centernet.py index f5dae5b9..216b3180 100644 --- a/hailo_model_zoo/core/postprocessing/detection/centernet.py +++ b/hailo_model_zoo/core/postprocessing/detection/centernet.py @@ -43,17 +43,17 @@ def postprocessing(self, endnodes, device_pre_post_layers=None, **kwargs): return tf_postproc_nms_centernet(endnodes, max_detections_per_class=self._nms_topk_perclass) bb_dict = {} if device_pre_post_layers and device_pre_post_layers.get('max_finder', False): - endnodes.append(endnodes[0]) # following code expects prob tensor to be 3rd. + outputs = endnodes else: endnodes0_padded = tf.pad(endnodes[0], [[0, 0], [1, 1], [1, 1], [0, 0]]) maxpooled_probs = tf.nn.max_pool2d(endnodes0_padded, [1, 3, 3, 1], [1, 1, 1, 1], 'VALID') probs_maxima_booleans = tf.cast(tf.math.equal(endnodes[0], maxpooled_probs), 'float32') probs_maxima_values = tf.math.multiply(probs_maxima_booleans, endnodes[0]) - endnodes.append(probs_maxima_values) + outputs = [endnodes[1], endnodes[2], probs_maxima_values] # we discard the 0 element in the endnodes list. This is the probabilities tensor. # Instead we pass the sparse probabilities tensor probs_maxima_values: bb_probs, bb_classes, bb_boxes, num_detections, top_k_indices = tf.numpy_function( - self._centernet_postprocessing, endnodes[1:], ['float32', 'int32', 'float32', 'int32', 'int64'], + self._centernet_postprocessing, outputs, ['float32', 'int32', 'float32', 'int32', 'int64'], name='centernet_postprocessing') bb_dict['detection_scores'] = bb_probs bb_dict['detection_classes'] = bb_classes diff --git a/hailo_model_zoo/core/postprocessing/detection/detection_common.py b/hailo_model_zoo/core/postprocessing/detection/detection_common.py index 571e839f..bb01b5cb 100644 --- a/hailo_model_zoo/core/postprocessing/detection/detection_common.py +++ b/hailo_model_zoo/core/postprocessing/detection/detection_common.py @@ -21,18 +21,28 @@ def translate_coco_2017_to_2014(nmsed_classes): def tf_postproc_nms(endnodes, score_threshold, coco_2017_to_2014=True): def _single_batch_parse(args): frame_detections = args[:, :, :] - indices = tf.where(frame_detections[:, :, 4] > score_threshold)[0:100] - boxes_and_scores = tf.gather_nd(frame_detections, indices) - num_detections = tf.shape(indices)[0] - pad_size = 100 - num_detections - classes_expanded = tf.expand_dims(tf.cast(indices[:, 0], tf.int32), axis=1) + 1 - classes_expanded = tf.squeeze(tf.pad(classes_expanded, paddings=[[0, pad_size], [0, 0]], - mode='CONSTANT', constant_values=0), axis=1) - final_frame_results = boxes_and_scores - final_frame_results_padded = tf.pad(final_frame_results, - paddings=[[0, pad_size], - [0, 0]], mode='CONSTANT', constant_values=0) - return final_frame_results_padded[:, :4], final_frame_results_padded[:, 4], classes_expanded, num_detections + boxes = frame_detections[:, :, :4] + scores = frame_detections[:, :, 4] + indices = tf.where(scores > score_threshold) + boxes_after_tsh = tf.gather_nd(boxes, indices) + scores_after_tsh = tf.gather_nd(scores, indices) + class_after_tsh = tf.cast(indices[:, 0], tf.int32) + 1 + num_detection_after_tsh = tf.shape(indices)[0] + padding_size = 100 - num_detection_after_tsh + if padding_size > 0: + boxes_after_tsh = tf.pad(boxes_after_tsh, + paddings=[[0, padding_size], + [0, 0]], mode='CONSTANT', constant_values=0) + scores_after_tsh = tf.pad(scores_after_tsh, + paddings=[[0, padding_size]], mode='CONSTANT', constant_values=0) + class_after_tsh = tf.pad(class_after_tsh, + paddings=[[0, padding_size]], mode='CONSTANT', constant_values=0) + + top_k_scores, top_k_ind = tf.math.top_k(scores_after_tsh, k=100) + top_k_boxes = tf.gather(boxes_after_tsh, top_k_ind) + top_k_class = tf.gather(class_after_tsh, top_k_ind) + num_detections = tf.shape(top_k_ind)[0] + return top_k_boxes, top_k_scores, top_k_class, num_detections with tf.name_scope('Postprocessor'): detections = tf.transpose(endnodes, [0, 1, 3, 2]) diff --git a/hailo_model_zoo/core/postprocessing/detection/efficientdet.py b/hailo_model_zoo/core/postprocessing/detection/efficientdet.py index c720924f..a79b0d91 100644 --- a/hailo_model_zoo/core/postprocessing/detection/efficientdet.py +++ b/hailo_model_zoo/core/postprocessing/detection/efficientdet.py @@ -3,6 +3,7 @@ from .ssd import collect_box_class_predictions from tensorflow.image import combined_non_max_suppression +from hailo_model_zoo.core.postprocessing.detection.detection_common import tf_postproc_nms class EfficientDetPostProc(object): @@ -25,6 +26,8 @@ def __init__(self, img_dims, nms_iou_thresh, score_threshold, anchors, np.array(anchors["sizes"]), np.array(anchors["strides"])], ['float32'])[0], (1, -1, 4)) + self._nms_on_device = self._nms_on_device = kwargs["device_pre_post_layers"].get( + 'nms', False) if kwargs["device_pre_post_layers"] else False def bbox_transform_inv(self, deltas): cxa = (self._anchors_input[..., 0] + self._anchors_input[..., 2]) / 2 @@ -51,6 +54,8 @@ def clip_boxes(self, boxes): return tf.stack([y1, x1, y2, x2], axis=2) def postprocessing(self, endnodes, **kwargs): + if self._nms_on_device: + return tf_postproc_nms(endnodes, self._score_threshold, False) with tf.name_scope('Postprocessor'): regression, classification = collect_box_class_predictions(endnodes, self._num_classes, self._anchors_type) classification = tf.sigmoid(classification) diff --git a/hailo_model_zoo/core/postprocessing/detection/faster_rcnn_stage1_postprocessing.py b/hailo_model_zoo/core/postprocessing/detection/faster_rcnn_stage1_postprocessing.py index b5b7c6ec..1d65b0c4 100644 --- a/hailo_model_zoo/core/postprocessing/detection/faster_rcnn_stage1_postprocessing.py +++ b/hailo_model_zoo/core/postprocessing/detection/faster_rcnn_stage1_postprocessing.py @@ -62,7 +62,7 @@ def get_all_anchors(stride, sizes, ratios, max_size): def _generate_anchors(base_size, scales, aspect_ratios): - anchor = np.array([1, 1, base_size, base_size], dtype=np.float) - 1 + anchor = np.array([1, 1, base_size, base_size], dtype=float) - 1 anchors = _ratio_enum(anchor, aspect_ratios) anchors = np.vstack( [_scale_enum(anchors[i, :], scales) for i in range(anchors.shape[0])] @@ -157,8 +157,8 @@ def __init__(self, img_dims, nms_iou_thresh, if anchors is None: raise ValueError('Missing detection anchors metadata') self._stride = int(anchors['strides'][0]) - self._aspect_ratios = np.array(anchors['aspect_ratios'], dtype=np.float) - self._sizes = np.array(anchors['sizes'], dtype=np.float) / self._stride + self._aspect_ratios = np.array(anchors['aspect_ratios'], dtype=float) + self._sizes = np.array(anchors['sizes'], dtype=float) / self._stride self._pre_nms_topk = pre_nms_topk self._post_nms_topk = post_nms_topk self._feature_map_height = int(anchors['featuremap_shape'][0]) diff --git a/hailo_model_zoo/core/postprocessing/detection/nanodet.py b/hailo_model_zoo/core/postprocessing/detection/nanodet.py index 9c43f897..61a30a93 100644 --- a/hailo_model_zoo/core/postprocessing/detection/nanodet.py +++ b/hailo_model_zoo/core/postprocessing/detection/nanodet.py @@ -7,7 +7,8 @@ class NanoDetPostProc: def __init__(self, img_dims=(416, 416), nms_iou_thresh=0.6, labels_offset=0, - score_threshold=0.3, anchors=None, classes=80, meta_arch='nanodet', **kwargs): + score_threshold=0.3, anchors=None, classes=80, nms_max_output_per_class=None, post_nms_topk=None, + meta_arch='nanodet', **kwargs): self._num_classes = classes self._image_dims = img_dims self._nms_iou_thresh = nms_iou_thresh @@ -18,9 +19,17 @@ def __init__(self, img_dims=(416, 416), nms_iou_thresh=0.6, labels_offset=0, # since scale factors don't make sense in nanodet we abuse it to store the offsets self._offset_factors = anchors.scale_factors self._network_arch = meta_arch - self._decode = { + self._nms_max_output_per_class = 100 if nms_max_output_per_class is None else nms_max_output_per_class + self._nms_max_output = 100 if post_nms_topk is None else post_nms_topk + self._split = { 'nanodet': self.nanodet_decode, 'nanodet_split': self.split_decode, + 'nanodet_v8': self.split_decode_yolo # nanodet_v8 is for YOLOv8 + } + self._decode = { + 'nanodet': self._box_decoding_nano, + 'nanodet_split': self._box_decoding_nano, + 'nanodet_v8': self._box_decoding_yolo # nanodet_v8 is for YOLOv8 } @staticmethod @@ -49,10 +58,25 @@ def split_decode(endnodes, reg_max, num_classes): return tf.concat(scores, axis=1), boxes + @staticmethod + def split_decode_yolo(endnodes, reg_max, num_classes): + scores, boxes = [], [] + for node in endnodes[::2]: + fm_size_h, fm_size_w = node.shape[1:3] + box = tf.reshape(node, (-1, fm_size_h * fm_size_w, 4, (reg_max + 1))) + boxes.append(box) + for node in endnodes[1::2]: + fm_size_h, fm_size_w = node.shape[1:3] + score = tf.reshape(node, (-1, fm_size_h * fm_size_w, num_classes)) + score = score[:, :, :num_classes] + scores.append(score) + + return tf.concat(scores, axis=1), boxes + def _get_scores_boxes(self, endnodes): - return self._decode[self._network_arch](endnodes, self.reg_max, self._num_classes) + return self._split[self._network_arch](endnodes, self.reg_max, self._num_classes) - def _box_decoding(self, raw_boxes): + def _box_decoding_nano(self, raw_boxes): boxes = None for box_distribute, stride in zip(raw_boxes, self._strides): # create grid @@ -86,6 +110,43 @@ def _box_decoding(self, raw_boxes): boxes = decode_box if boxes is None else tf.concat([boxes, decode_box], axis=1) return tf.expand_dims(boxes, axis=2) + def _box_decoding_yolo(self, raw_boxes): + boxes = None + for box_distribute, stride in zip(raw_boxes, self._strides): + # create grid + shape = [int(x / stride) for x in self._image_dims] + grid_x = np.arange(shape[1]) + 0.5 + grid_y = np.arange(shape[0]) + 0.5 + grid_x, grid_y = np.meshgrid(grid_x, grid_y) + x_offset, y_offset = self._offset_factors + ct_row = (grid_y.flatten() + x_offset) * stride + ct_col = (grid_x.flatten() + y_offset) * stride + center = np.stack((ct_col, ct_row, ct_col, ct_row), axis=1) + + # box distribution to distance + reg_range = np.arange(self.reg_max + 1) + box_distance = tf.nn.softmax(box_distribute, axis=-1) + box_distance = box_distance * np.reshape(reg_range, (1, 1, 1, -1)) + box_distance = tf.reduce_sum(box_distance, axis=-1) + box_distance = box_distance * stride + + # decode box + box_distance = tf.concat([box_distance[:, :, :2] * (-1), box_distance[:, :, 2:]], axis=-1) + decode_box = np.expand_dims(center, axis=0) + box_distance + + # clipping + xmin = tf.maximum(0.0, decode_box[:, :, 0]) / self._image_dims[1] + ymin = tf.maximum(0.0, decode_box[:, :, 1]) / self._image_dims[0] + xmax = tf.minimum(tf.cast(self._image_dims[1], tf.float32), decode_box[:, :, 2]) / self._image_dims[1] + ymax = tf.minimum(tf.cast(self._image_dims[0], tf.float32), decode_box[:, :, 3]) / self._image_dims[0] + decode_box = tf.transpose([ymin, xmin, ymax, xmax], [1, 2, 0]) + + boxes = decode_box if boxes is None else tf.concat([boxes, decode_box], axis=1) + return tf.expand_dims(boxes, axis=2) + + def _box_decoding(self, raw_boxes): + return self._decode[self._network_arch](raw_boxes) + def postprocessing(self, endnodes, *, device_pre_post_layers, **kwargs): scores, raw_boxes = self._get_scores_boxes(endnodes) @@ -102,8 +163,8 @@ def postprocessing(self, endnodes, *, device_pre_post_layers, **kwargs): scores=scores, score_threshold=self._score_threshold, iou_threshold=self._nms_iou_thresh, - max_output_size_per_class=100, - max_total_size=100) + max_output_size_per_class=self._nms_max_output_per_class, + max_total_size=self._nms_max_output) # adding offset to the class prediction and cast to integer def translate_coco_2017_to_2014(nmsed_classes): diff --git a/hailo_model_zoo/core/postprocessing/detection/ssd.py b/hailo_model_zoo/core/postprocessing/detection/ssd.py index e50eae90..282be606 100644 --- a/hailo_model_zoo/core/postprocessing/detection/ssd.py +++ b/hailo_model_zoo/core/postprocessing/detection/ssd.py @@ -119,6 +119,7 @@ def __init__(self, img_dims=(300, 300), nms_iou_thresh=0.6, self._nms_on_device = False if kwargs["device_pre_post_layers"] and kwargs["device_pre_post_layers"].get('nms', False): self._nms_on_device = True + self.hpp = kwargs.get("hpp", False) @staticmethod def expanded_shape(orig_shape, start_dim, num_dims): @@ -155,7 +156,7 @@ def extract_anchors(self, endnodes): # base_anchor_size = np.array([1., 1.]) bboxes_list = [] im_height, im_width = self._image_dims - min_im_shape = np.float(min(im_height, im_width)) + min_im_shape = float(min(im_height, im_width)) scale_height = min_im_shape / im_height scale_width = min_im_shape / im_width box_specs_list = self._anchors.create_box_specs_list() @@ -276,7 +277,7 @@ def tf_postproc(self, endnodes): 'num_detections': num_detections} def postprocessing(self, endnodes, **kwargs): - if self._nms_on_device: + if self._nms_on_device or self.hpp: return tf_postproc_nms(endnodes, self._score_threshold, False) else: return self.tf_postproc(endnodes) diff --git a/hailo_model_zoo/core/postprocessing/detection/yolo.py b/hailo_model_zoo/core/postprocessing/detection/yolo.py index 861df6e3..47ba6e06 100644 --- a/hailo_model_zoo/core/postprocessing/detection/yolo.py +++ b/hailo_model_zoo/core/postprocessing/detection/yolo.py @@ -4,6 +4,7 @@ from .centernet import COCO_2017_TO_2014_TRANSLATION from .detection_common import translate_coco_2017_to_2014 from tensorflow.image import combined_non_max_suppression +from hailo_model_zoo.core.postprocessing.detection.detection_common import tf_postproc_nms def sigmoid(x): @@ -34,6 +35,7 @@ def __init__(self, img_dims=(608, 608), nms_iou_thresh=0.45, score_threshold=0.0 self._nms_on_device = False if kwargs["device_pre_post_layers"] and kwargs["device_pre_post_layers"].get('nms', False): self._nms_on_device = True + self.hpp = kwargs.get("hpp", False) # not needed once SDK change the output shape of emulator @staticmethod def _yolo3_decode(raw_box_centers, raw_box_scales, objness, class_pred, anchors_for_stride, offsets, stride): @@ -262,7 +264,30 @@ def reorganize_split_output_numpy(self, centers, scales, obj, probs): full_concat_array = np.concatenate([full_concat_array, partial_concat], 3) return full_concat_array + def hpp_detection_postprocess(self, endnodes): + ''' + input endnodes from HPP in the following format (Batch, 1, 6, MAX_PROPOSALS) + ''' + endnodes = tf.transpose(endnodes, [0, 1, 3, 2]) + detection_boxes = endnodes[:, 0, :, :4] # (B, 100, 4) + detection_scores = endnodes[:, 0, :, 4] # (B, 100) + detection_classes = endnodes[:, 0, :, 5] # (B, 100) + + def translate_coco_2017_to_2014(nmsed_classes): + return np.vectorize(COCO_2017_TO_2014_TRANSLATION.get)(nmsed_classes).astype(np.int32) + + num_detections = tf.reduce_sum(tf.cast(detection_scores > 0, dtype=tf.int32), axis=1) + nmsed_classes = tf.cast(tf.add(detection_classes, self._labels_offset), tf.int16) + [nmsed_classes] = tf.py_function(translate_coco_2017_to_2014, [nmsed_classes], ['int32']) + + return {'detection_boxes': detection_boxes, + 'detection_scores': detection_scores, + 'detection_classes': nmsed_classes, + 'num_detections': num_detections} + def postprocessing(self, endnodes, **kwargs): + if self.hpp: + return tf_postproc_nms(endnodes, score_threshold=0.0, coco_2017_to_2014=True) if self._nms_on_device: return self.iou_nms(endnodes) else: diff --git a/hailo_model_zoo/core/postprocessing/detection_3d_postprocessing.py b/hailo_model_zoo/core/postprocessing/detection_3d_postprocessing.py index ed7ff8a7..fff6fda0 100644 --- a/hailo_model_zoo/core/postprocessing/detection_3d_postprocessing.py +++ b/hailo_model_zoo/core/postprocessing/detection_3d_postprocessing.py @@ -268,12 +268,12 @@ def _decode_orientation(self, vector_ori, locations, flip_mask=None): if flip_mask is not None: fm = flip_mask.flatten() - rotys_flip = np.float(fm) * rotys + rotys_flip = float(fm) * rotys rotys_flip_pos_idx = rotys_flip > 0 rotys_flip_neg_idx = rotys_flip < 0 rotys_flip[rotys_flip_pos_idx] -= np.pi rotys_flip[rotys_flip_neg_idx] += np.pi - rotys_all = np.float(fm) * rotys_flip + (1 - np.float(fm)) * rotys + rotys_all = float(fm) * rotys_flip + (1 - float(fm)) * rotys return rotys_all else: return rotys, alphas diff --git a/hailo_model_zoo/core/postprocessing/face_landmarks_3d_postprocessing.py b/hailo_model_zoo/core/postprocessing/face_landmarks_3d_postprocessing.py index 5f6216a8..2cbb857d 100644 --- a/hailo_model_zoo/core/postprocessing/face_landmarks_3d_postprocessing.py +++ b/hailo_model_zoo/core/postprocessing/face_landmarks_3d_postprocessing.py @@ -3,6 +3,7 @@ import numpy as np import tensorflow as tf +from hailo_model_zoo.core.infer.infer_utils import to_numpy from hailo_model_zoo.utils.path_resolver import resolve_data_path @@ -68,7 +69,7 @@ def __init__(self, shape_dim=40, exp_dim=10): self._tri = pickle.load(f) # this tri/face is re-built for bfm_noneck_v3 self._tri = _to_ctype(self._tri.T).astype(np.int32) - self._keypoints = bfm.get('keypoints').astype(np.long) # fix bug + self._keypoints = bfm.get('keypoints').astype(int) # fix bug w = np.concatenate((self._w_shp, self._w_exp), axis=1) self._w_norm = np.linalg.norm(w, axis=0) @@ -159,8 +160,8 @@ def face_landmarks_3d_postprocessing(endnodes, device_pre_post_layers=None, *, i def visualize_face_landmarks_3d_result(logits, image, **kwargs): logits = logits['predictions'] - img = kwargs.get('img_info', {}).get('uncropped_image', image[0]) - box = kwargs.get('img_info', {}).get('roi_box') + img = to_numpy(kwargs.get('img_info', {}).get('uncropped_image', image[0])) + box = to_numpy(kwargs.get('img_info', {}).get('roi_box')) for landmark in logits[0, :, :2]: landmark_as_int = tuple(int(x) for x in landmark) diff --git a/hailo_model_zoo/core/postprocessing/head_pose_estimation_postprocessing.py b/hailo_model_zoo/core/postprocessing/head_pose_estimation_postprocessing.py index c98c4a8a..78b40fa1 100644 --- a/hailo_model_zoo/core/postprocessing/head_pose_estimation_postprocessing.py +++ b/hailo_model_zoo/core/postprocessing/head_pose_estimation_postprocessing.py @@ -13,13 +13,16 @@ def head_pose_estimation_postprocessing(endnodes, device_pre_post_layers, **kwar pitch_predicted = tf.reduce_sum(probs[0] * idx_tensor, 1) * 3 - 99 roll_predicted = tf.reduce_sum(probs[1] * idx_tensor, 1) * 3 - 99 yaw_predicted = tf.reduce_sum(probs[2] * idx_tensor, 1) * 3 - 99 - return {'predictions': [pitch_predicted, roll_predicted, yaw_predicted]} + return { + 'pitch': pitch_predicted, + 'roll': roll_predicted, + 'yaw': yaw_predicted, + } def visualize_head_pose_result(net_output, img, **kwargs): - net_output = net_output['predictions'] img = img[0] - pitch, roll, yaw = net_output[0][0], net_output[1][0], net_output[2][0] + pitch, roll, yaw = net_output['pitch'][0], net_output['roll'][0], net_output['yaw'][0] pitch = pitch * np.pi / 180 yaw = -(yaw * np.pi / 180) roll = roll * np.pi / 180 diff --git a/hailo_model_zoo/core/postprocessing/instance_segmentation_postprocessing.py b/hailo_model_zoo/core/postprocessing/instance_segmentation_postprocessing.py index 5096006f..642ca504 100644 --- a/hailo_model_zoo/core/postprocessing/instance_segmentation_postprocessing.py +++ b/hailo_model_zoo/core/postprocessing/instance_segmentation_postprocessing.py @@ -198,62 +198,8 @@ def _sigmoid(x): return 1 / (1 + np.exp(-x)) -def postprocess(det_output, w, h, batch_idx=0, interpolation_mode='bilinear', - visualize_lincomb=False, crop_masks=True, score_threshold=0.2): - dets = det_output[batch_idx] - - if dets is None: - return [None] * 4 # Warning, this is 4 copies of the same thing - - if score_threshold > 0: - keep = dets['detection_scores'] > score_threshold - for k in dets: - if k != 'proto': - dets[k] = dets[k][keep] - - if dets['detection_scores'].shape[0] == 0: - return [None] * 4 - - # im_w and im_h when it concerns bboxes. This is a workaround hack for preserve_aspect_ratio - b_w, b_h = (w, h) - - # Actually extract everything from dets now - classes = dets['detection_classes'] - boxes = dets['detection_boxes'] - scores = dets['detection_scores'] - masks = dets['mask'] - - # At this points masks is only the coefficients - proto_data = dets['proto'] - - # Test flag, do not upvote - - masks = np.matmul(proto_data, masks.transpose()) - masks = _sigmoid(masks) - - # Crop masks before upsampling because you know why - if crop_masks: - masks = _crop(masks, boxes) - - # Scale masks up to the full image - masks = cv2.resize(masks, (w, h)) - if len(masks.shape) < 3: - masks = np.expand_dims(masks, axis=0) - else: - masks = np.transpose(masks, (2, 0, 1)) - - # Binarize the masks - masks = masks > 0.5 - - boxes[:, 0], boxes[:, 2] = _sanitize_coordinates(boxes[:, 0], boxes[:, 2], b_w, cast=False) - boxes[:, 1], boxes[:, 3] = _sanitize_coordinates(boxes[:, 1], boxes[:, 3], b_h, cast=False) - boxes = np.array(boxes, np.float64) - - return classes, scores, boxes, masks - - def prep_display(dets_out, img, score_threshold, class_color=False, mask_alpha=0.45, - channels_remove=None, class_names=None): + channels_remove=None, class_names=None, mask_thresh=0.5): top_k = 5 img_gpu = img / 255.0 h, w, _ = img.shape @@ -264,12 +210,32 @@ def prep_display(dets_out, img, score_threshold, class_color=False, mask_alpha=0 class_names_mask = channels_remove[1:] # Remove background class cats = np.where(np.array(class_names_mask) == 1)[0] visualization_class_names = list(np.array(class_names)[cats]) - t = postprocess(dets_out, w, h, visualize_lincomb=False, - crop_masks=True, score_threshold=score_threshold) - if t[3] is None: + + boxes = dets_out['detection_boxes'] + masks = dets_out['mask'] + classes = dets_out['detection_classes'] + scores = dets_out['detection_scores'] + + # Scale Boxes + boxes[:, 0], boxes[:, 2] = _sanitize_coordinates(boxes[:, 0], boxes[:, 2], w, cast=False) + boxes[:, 1], boxes[:, 3] = _sanitize_coordinates(boxes[:, 1], boxes[:, 3], h, cast=False) + boxes = np.array(boxes, np.float64) + + # Scale Masks + masks = cv2.resize(masks, (w, h)) + if len(masks.shape) < 3: + masks = np.expand_dims(masks, axis=0) + else: + masks = np.transpose(masks, (2, 0, 1)) + # Binarize the masks + masks = masks > mask_thresh + + if not masks.shape[0]: return np.array(img_gpu * 255, np.uint8) - masks = t[3][:top_k] - classes, scores, boxes = [x[:top_k] for x in t[:3]] + boxes = boxes[:top_k] + masks = masks[:top_k] + classes = classes[:top_k] + scores = scores[:top_k] num_dets_to_consider = min(top_k, classes.shape[0]) for j in range(num_dets_to_consider): @@ -279,7 +245,7 @@ def prep_display(dets_out, img, score_threshold, class_color=False, mask_alpha=0 if num_dets_to_consider == 0: # No detections found so just output the original image - return (img_gpu * 255) + return np.array(img_gpu * 255, np.uint8) def get_color(j, on_gpu=None): global color_cache @@ -500,7 +466,7 @@ def process_mask(protos, masks_in, bboxes, shape, upsample=True, if not downsample: masks = crop_mask(masks, downsampled_bboxes) # CHW - return masks > 0.5 + return masks def _yolov5_decoding(branch_idx, output, stride_list, anchor_list, num_classes): @@ -563,6 +529,10 @@ def yolov5_seg_postprocess(endnodes, device_pre_post_layers=None, **kwargs): iou_thres = kwargs['nms_iou_thresh'] outputs = non_max_suppression(outputs, score_thres, iou_thres) + return _finalize_detections_yolov5_seg(outputs, protos, **kwargs) + + +def _finalize_detections_yolov5_seg(outputs, protos, **kwargs): for batch_idx, output in enumerate(outputs): shape = kwargs.get('img_dims', None) boxes = output['detection_boxes'] @@ -592,9 +562,10 @@ def _make_grid(anchors, stride, bs=8, nx=20, ny=20): return grid, anchor_grid -def sparseinst_postprocess(endnodes, device_pre_post_layers=None, scale_factor=2, **kwargs): +def sparseinst_postprocess(endnodes, device_pre_post_layers=None, scale_factor=2, num_groups=4, **kwargs): inst_kernels_path = path_resolver.resolve_data_path(kwargs['postprocess_config_json']) + meta_arch = kwargs.get('meta_arch', 'sparseinst_giam') inst_kernels = np.load(inst_kernels_path, allow_pickle=True)['arr_0'][()] mask_features = endnodes[0].copy() # 80 x 80 x 128 @@ -619,6 +590,20 @@ def sparseinst_postprocess(endnodes, device_pre_post_layers=None, scale_factor=2 normalizer = np.clip(np.sum(iam_prob, axis=1), a_min=1e-6, a_max=None) inst_features /= normalizer[:, :, None] + if 'giam' in meta_arch: + inst_features = inst_features.reshape(B, num_groups, -1, C) + inst_features = inst_features.transpose(0, 2, 1, 3) + inst_features = inst_features.reshape((B, -1, num_groups * C)) + + features = list() + for batch_idx in range(B): + features.append(np.expand_dims(np.matmul(inst_features[batch_idx], + inst_kernels['fc']['weights'].transpose()), axis=0)) + inst_features = np.vstack(features) + + inst_features = inst_features + inst_kernels['fc']['bias'] + inst_features[inst_features < 0.] = 0. + pred_logits = list() pred_kernel = list() pred_scores = list() @@ -639,6 +624,7 @@ def sparseinst_postprocess(endnodes, device_pre_post_layers=None, scale_factor=2 np.matmul(pred_kernel[batch_idx], np.transpose(mask_features.reshape(B, H * W, C), axes=[0, 2, 1])[batch_idx]), axis=0)) + N = pred_kernel.shape[1] pred_masks = np.vstack(pred_masks).reshape(B, N, H, W) pred_masks_tmp = np.zeros((B, N, H * scale_factor, W * scale_factor)) @@ -648,29 +634,25 @@ def sparseinst_postprocess(endnodes, device_pre_post_layers=None, scale_factor=2 interpolation=cv2.INTER_LINEAR), axes=(2, 0, 1)) pred_masks = np.vstack(pred_masks_tmp).reshape(B, N, H * scale_factor, W * scale_factor) + pred_objectness = _sigmoid(pred_scores) pred_scores = _sigmoid(pred_logits) pred_masks = _sigmoid(pred_masks) - pred_objectness = _sigmoid(pred_scores) pred_scores = np.sqrt(pred_scores * pred_objectness) - img_info = kwargs['image_info'] - orig_height, orig_width = img_info['orig_height'], img_info['orig_width'] - output_shapes = (orig_height, orig_width) + return _finalize_detections_sparseinst(pred_masks, pred_scores, **kwargs) - output = {'pred_scores': pred_scores, 'pred_masks': pred_masks, 'output_shapes': output_shapes} - - return output +def _finalize_detections_sparseinst(pred_masks, pred_scores, **kwargs): + img_info = kwargs['gt_images'] + hin, win = img_info['img_orig'].shape[1:3] + cls_threshold = kwargs.get('score_threshold', 0.005) + mask_threshold = kwargs.get('mask_threshold', 0.45) -def _spraseinst_post(output, input_shape, output_shape, - cls_threshold=0.005, mask_threshold=0.45): - pred_scores = output['pred_scores'] - pred_masks = output['pred_masks'] - results = [] + outputs = [] for idx, (scores_per_image, masks_per_image) in enumerate(zip(pred_scores, pred_masks)): - result = {} - hout, wout = output_shape[0][idx], output_shape[1][idx] - hin, win = input_shape[0][idx], input_shape[1][idx] + output = {} + hout, wout = img_info['height'][idx], img_info['width'][idx] + h_resized, w_resized = img_info['resized_height'][idx], img_info['resized_width'][idx] scores = np.max(scores_per_image, axis=-1) labels = np.argmax(scores_per_image, axis=-1) @@ -680,30 +662,30 @@ def _spraseinst_post(output, input_shape, output_shape, masks_per_image = masks_per_image[keep] if not scores.shape[0]: - result['scores'] = scores - result['pred_classes'] = labels - results.append(result) + output['detection_scores'] = scores + output['detection_classes'] = labels + outputs.append(output) continue scores = _rescoring_mask(scores, masks_per_image > mask_threshold, masks_per_image) # (1) upsampling the masks to input size, remove the padding area masks_per_image = np.transpose(cv2.resize(np.transpose(masks_per_image, axes=(2, 1, 0)), - (hin, win), interpolation=cv2.INTER_LINEAR), - axes=(2, 1, 0))[:, :hout, :wout] + (hin, win), interpolation=cv2.INTER_LINEAR), + axes=(2, 1, 0))[:, :h_resized, :w_resized] + # (2) upsampling/downsampling the masks to the original sizes masks_per_image = np.transpose(cv2.resize(np.transpose(masks_per_image, axes=(2, 1, 0)), - (hout, wout), interpolation=cv2.INTER_LINEAR), + (hout, wout), interpolation=cv2.INTER_LINEAR), axes=(2, 1, 0)) - mask_pred = masks_per_image > mask_threshold - - result['pred_masks'] = mask_pred - result['scores'] = scores - result['pred_classes'] = labels + output['mask'] = masks_per_image + output['detection_scores'] = scores + output['detection_classes'] = labels + output['orig_shape'] = img_info['height'][idx], img_info['width'][idx] + output['resized_shape'] = img_info['resized_height'][idx], img_info['resized_width'][idx] + outputs.append(output) - results.append(result) - - return results + return outputs def _rescoring_mask(scores, masks_pred, masks): @@ -727,22 +709,26 @@ def _get_pol_area(x, y): return 0.5 * np.abs(np.dot(x, np.roll(y, 1)) - np.dot(y, np.roll(x, 1))) -def visualize_sparseinst_results(detections, img, class_names=None, alpha=0.5, confidence_threshold=0.5, **kwargs): - - output_shapes = detections['output_shapes'] - input_shapes = ([im.shape[0] for im in img], [im.shape[1] for im in img]) - detections = _spraseinst_post(detections, input_shapes, input_shapes) +def visualize_sparseinst_results(detections, img, class_names=None, alpha=0.5, + confidence_threshold=0.5, mask_thresh=0.45, **kwargs): img_idx = 0 - results = detections[img_idx] - output_shape = output_shapes[0][img_idx], output_shapes[1][img_idx] - scores = results['scores'] - keep = scores > confidence_threshold - - masks = results['pred_masks'][keep] - scores = results['scores'][keep] - classes = results['pred_classes'][keep] img_out = img[img_idx].copy() + + output_shape = detections['orig_shape'] + resized_shape = detections['resized_shape'] + keep = detections['detection_scores'] > confidence_threshold + scores = detections['detection_scores'][keep] + classes = detections['detection_classes'][keep] + masks = detections['mask'][keep] + + # Binarize the masks + masks = masks > mask_thresh + + # remove padding + img_out = img_out[:resized_shape[0], :resized_shape[1], :] + img_out = cv2.resize(img_out, output_shape[::-1], interpolation=cv2.INTER_LINEAR) + for idx, mask in enumerate(masks): label = f"{CLASS_NAMES_COCO[classes[idx]]}" @@ -786,12 +772,11 @@ def visualize_sparseinst_results(detections, img, class_names=None, alpha=0.5, c thickness=2, lineType=cv2.FILLED) - # remove padding - img_out = img_out[:output_shape[0], :output_shape[1], :] return img_out -def yolact_postprocessing(endnodes, device_pre_post_layers=None, **kwargs): +def yolact_postprocessing(endnodes, device_pre_post_layers=None, + score_thresh=0.2, crop_masks=True, **kwargs): channels_remove = kwargs["channels_remove"] if kwargs["channels_remove"]["enabled"] else None if channels_remove: mask_list = list(np.where(np.array(kwargs['channels_remove']['mask'][0]) == 0)[0]) @@ -821,7 +806,62 @@ def yolact_postprocessing(endnodes, device_pre_post_layers=None, **kwargs): mask = np.concatenate([mask0, mask1, mask2, mask3, mask4], axis=-2) detect = Detect(num_classes, bkg_label=0, top_k=200, conf_thresh=kwargs['score_threshold'], nms_thresh=kwargs['nms_iou_thresh']) - return detect(loc, proto, conf, mask, priors) + + det_output = detect(loc, proto, conf, mask, priors) + + return _finalize_detections_yolact(det_output, proto) + + +def _finalize_detections_yolact(det_output, protos, score_thresh=0.2, crop_masks=True, **kwargs): + + outputs = [] + for batch_idx, dets in enumerate(det_output): + proto = protos[batch_idx] + empty_output = {'detection_boxes': np.zeros((0, 4)), + 'mask': np.zeros((proto.shape[0], proto.shape[1], 0)), + 'detection_classes': np.zeros(0), + 'detection_scores': np.zeros(0)} + + if dets is None: + outputs.append(empty_output) + continue + + if score_thresh > 0: + keep = dets['detection_scores'] > score_thresh + for k in dets: + if k != 'proto': + dets[k] = dets[k][keep] + + if dets['detection_scores'].shape[0] == 0: + outputs.append(empty_output) + continue + + # Actually extract everything from dets now + classes = dets['detection_classes'] + boxes = dets['detection_boxes'] + scores = dets['detection_scores'] + masks = dets['mask'] + + # At this points masks is only the coefficients + proto_data = dets['proto'] + + # Test flag, do not upvote + + masks = np.matmul(proto_data, masks.transpose()) + masks = _sigmoid(masks) + + # Crop masks before upsampling because you know why + if crop_masks: + masks = _crop(masks, boxes) + + output = {} + output['detection_boxes'] = boxes + output['mask'] = masks + output['detection_scores'] = scores + output['detection_classes'] = classes + outputs.append(output) + + return outputs def instance_segmentation_postprocessing(endnodes, device_pre_post_layers=None, **kwargs): @@ -843,15 +883,15 @@ def instance_segmentation_postprocessing(endnodes, device_pre_post_layers=None, return {'predictions': predictions} -def visualize_yolov5_seg_results(detections, img, class_names=None, alpha=0.5, score_thres=0.25, **kwargs): +def visualize_yolov5_seg_results(detections, img, class_names=None, alpha=0.5, score_thres=0.25, + mask_thresh=0.5, **kwargs): img_idx = 0 - det = detections[img_idx] img_out = img[img_idx].copy() - boxes = det['detection_boxes'] - masks = det['mask'] - scores = det['detection_scores'] - classes = det['detection_classes'] + boxes = detections['detection_boxes'] + masks = detections['mask'] > mask_thresh + scores = detections['detection_scores'] + classes = detections['detection_classes'] keep = scores > score_thres boxes = boxes[keep] @@ -923,7 +963,7 @@ def visualize_instance_segmentation_result(detections, img, **kwargs): return visualize_yolov5_seg_results(detections, img, class_names=dataset_info.class_names, **kwargs) elif 'yolact' in meta_arch: channels_remove = kwargs["channels_remove"] - return prep_display(dets_out=detections[0:1], img=img[0], score_threshold=0.2, + return prep_display(dets_out=detections, img=img[0], score_threshold=0.2, channels_remove=channels_remove, class_names=dataset_info.class_names) else: raise NotImplementedError(f'Visualization {meta_arch} not found') diff --git a/hailo_model_zoo/core/postprocessing/mspn_postprocessing.py b/hailo_model_zoo/core/postprocessing/mspn_postprocessing.py index ad9b5103..c49071aa 100644 --- a/hailo_model_zoo/core/postprocessing/mspn_postprocessing.py +++ b/hailo_model_zoo/core/postprocessing/mspn_postprocessing.py @@ -157,14 +157,14 @@ def _get_default_bbox(batch_size): def mspn_postprocessing(endnodes, device_pre_post_layers=None, **kwargs): - image_info = kwargs['image_info'] + image_info = kwargs['gt_images'] height, width = image_info['img_resized'].shape[1:3] aspect_ratio = width / height # Get box info if exists, otherwise assume box spans the entire image - bbox = image_info.get('bbox', _get_default_bbox(endnodes[0].shape[0])) + bbox = image_info.get('bbox', _get_default_bbox(endnodes.shape[0])) center, scale = bbox_xyxy2cs(bbox, image_info['orig_height'], image_info['orig_width'], aspect_ratio) - heatmaps = np.transpose(endnodes[0], axes=[0, 3, 1, 2]) + heatmaps = np.transpose(endnodes, axes=[0, 3, 1, 2]) heatmaps = _gaussian_blur(heatmaps, kernel=5) N, K, H, W = heatmaps.shape @@ -199,9 +199,7 @@ def visualize_single_person_pose_estimation_result(probs, image, kpt_score_thr=0 radius=8, thickness=2, **kwargs): idx = 0 - # decode image - img = cv2.imdecode(np.fromstring(image[idx], dtype=np.uint8), cv2.IMREAD_UNCHANGED) - img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) + img = cv2.cvtColor(image[idx], cv2.COLOR_BGR2RGB) img_h, img_w, _ = img.shape probs = probs['predictions'][idx] kpts = np.array(probs, copy=False) diff --git a/hailo_model_zoo/core/postprocessing/ocr_postprocessing.py b/hailo_model_zoo/core/postprocessing/ocr_postprocessing.py index ac5f6050..6cdf1930 100644 --- a/hailo_model_zoo/core/postprocessing/ocr_postprocessing.py +++ b/hailo_model_zoo/core/postprocessing/ocr_postprocessing.py @@ -9,7 +9,7 @@ def ocr_postprocessing(endnodes, device_pre_post_layers=None, **kwargs): - logits = np.mean(endnodes[0], axis=1) + logits = np.mean(endnodes, axis=1) # probs = _softmax(logits) return {'predictions': np.argmax(logits, axis=2)} diff --git a/hailo_model_zoo/core/postprocessing/pose_estimation_postprocessing.py b/hailo_model_zoo/core/postprocessing/pose_estimation_postprocessing.py index 6377fda5..e0f62324 100644 --- a/hailo_model_zoo/core/postprocessing/pose_estimation_postprocessing.py +++ b/hailo_model_zoo/core/postprocessing/pose_estimation_postprocessing.py @@ -21,12 +21,12 @@ def pose_estimation_postprocessing(endnodes, device_pre_post_layers=None, **kwargs): if kwargs.get('meta_arch') == 'centerpose': - return {'predictions': centerpose_postprocessing(endnodes, - device_pre_post_layers=None, - **kwargs)} + return centerpose_postprocessing(endnodes, + device_pre_post_layers=None, + **kwargs) coco_result_list = [] for i in range(len(endnodes)): - image_info = kwargs['image_info'] + image_info = kwargs['gt_images'] heatmaps, pafs, image_id, pad, orig_shape = endnodes[0][i], endnodes[1][i], \ int(image_info['image_id'][i]), image_info['pad'][i], image_info['orig_shape'][i][:2] total_keypoints_num = 0 @@ -57,15 +57,19 @@ def pose_estimation_postprocessing(endnodes, device_pre_post_layers=None, **kwar def visualize_pose_estimation_result(results, img, dataset_name, *, detection_threshold=0.5, joint_threshold=0.5, **kwargs): - results = results['predictions'] assert dataset_name == 'cocopose' - bboxes, scores, keypoints, joint_scores = results + if 'predictions' in results: + results = results['predictions'] + bboxes, scores, keypoints, joint_scores = results + else: + bboxes, scores, keypoints, joint_scores = ( + results['bboxes'], results['scores'], results['keypoints'], results['joint_scores']) batch_size = bboxes.shape[0] assert batch_size == 1 box, score, keypoint, keypoint_score = bboxes[0], scores[0], keypoints[0], joint_scores[0] - image = img[0] + image = cv2.cvtColor(img[0], cv2.COLOR_BGR2RGB) for detection_box, detection_score, detection_keypoints, detection_keypoints_score in ( zip(box, score, keypoint, keypoint_score)): diff --git a/hailo_model_zoo/core/postprocessing/postprocessing_factory.py b/hailo_model_zoo/core/postprocessing/postprocessing_factory.py index d07683b2..fc8ac232 100644 --- a/hailo_model_zoo/core/postprocessing/postprocessing_factory.py +++ b/hailo_model_zoo/core/postprocessing/postprocessing_factory.py @@ -59,6 +59,14 @@ def get_visualization(name, **kwargs): Raises: ValueError: If visualization `name` is not recognized. """ + unsupported_visualizations = { + 'face_verification', + 'person_reid', + } + + if name in unsupported_visualizations: + raise ValueError(f'Visualization is currently not supported for {name}') + visualization_fn_map = { 'classification': visualize_classification_result, 'segmentation': visualize_segmentation_result, diff --git a/hailo_model_zoo/core/postprocessing/segmentation_postprocessing.py b/hailo_model_zoo/core/postprocessing/segmentation_postprocessing.py index 107cca82..13a26760 100644 --- a/hailo_model_zoo/core/postprocessing/segmentation_postprocessing.py +++ b/hailo_model_zoo/core/postprocessing/segmentation_postprocessing.py @@ -75,6 +75,8 @@ def segmentation_postprocessing(endnodes, device_pre_post_layers=None, **kwargs) device_pre_post_layers = device_pre_post_layers if device_pre_post_layers is not None else { 'bilinear': False, 'argmax': False} if device_pre_post_layers['argmax']: + if len(endnodes.shape) == 4: + endnodes = tf.squeeze(endnodes, axis=-1) predictions = endnodes else: if device_pre_post_layers['bilinear']: diff --git a/hailo_model_zoo/core/preprocessing/classification_preprocessing.py b/hailo_model_zoo/core/preprocessing/classification_preprocessing.py index fcdfcea7..478df25e 100644 --- a/hailo_model_zoo/core/preprocessing/classification_preprocessing.py +++ b/hailo_model_zoo/core/preprocessing/classification_preprocessing.py @@ -108,11 +108,11 @@ def _central_crop(image_list, crop_height, crop_width): return outputs -def _resnet_base_preprocessing(image, output_height=None, output_width=None, resize_side=None): +def _resnet_base_preprocessing(image, output_height=None, output_width=None, resize_side=None, method=None): if output_height is not None: assert output_width is not None assert resize_side is not None - image = _aspect_preserving_resize(image, resize_side) + image = _aspect_preserving_resize(image, resize_side, method=method) image = _central_crop([image], output_height, output_width)[0] image.set_shape([output_height, output_width, 3]) image = tf.cast(image, tf.float32) @@ -176,3 +176,10 @@ def vit_tiny(image, image_info=None, output_height=None, output_width=None, **kw if image_info: image_info['img_orig'] = tf.cast(image, tf.uint8) return image, image_info + + +def resnet_pruned(image, image_info=None, output_height=None, output_width=None, **kwargs): + image = _resnet_base_preprocessing(image, output_height, output_width, RESIZE_SIDE, method='bilinear') + if image_info: + image_info['img_orig'] = tf.cast(image, tf.uint8) + return image, image_info diff --git a/hailo_model_zoo/core/preprocessing/preprocessing_factory.py b/hailo_model_zoo/core/preprocessing/preprocessing_factory.py index b7d19c89..98d33090 100644 --- a/hailo_model_zoo/core/preprocessing/preprocessing_factory.py +++ b/hailo_model_zoo/core/preprocessing/preprocessing_factory.py @@ -1,7 +1,6 @@ """Contains a factory for image preprocessing.""" import numpy as np import tensorflow as tf - from hailo_model_zoo.core.preprocessing import classification_preprocessing from hailo_model_zoo.core.preprocessing import segmentation_preprocessing from hailo_model_zoo.core.preprocessing import detection_preprocessing @@ -25,6 +24,15 @@ def convert_rgb_to_yuv(image): return image +def convert_yuv_to_yuy2(image): + y_img = image[..., 0] + uv_img = image[..., 1:] + uv_subsampled = uv_img[:, ::2, :] + uv_unrolled = tf.reshape(uv_subsampled, (image.shape[-3], image.shape[-2])) + yuy2_img = tf.stack([y_img, uv_unrolled], axis=-1) + return yuy2_img + + def image_resize(image, shape): image = tf.expand_dims(image, 0) image = tf.image.resize(image, tuple(shape), method='bilinear') @@ -89,12 +97,14 @@ def get_preprocessing(name, height, width, normalization_params, **kwargs): 'vit': classification_preprocessing.vit_tiny, 'espcn': super_resolution_preprocessing.espcn, 'retinanet_resnext50': detection_preprocessing.retinanet_resnext50, - 'sparseinst': segmentation_preprocessing.sparseinst + 'sparseinst': segmentation_preprocessing.sparseinst, + 'resnet_pruned': classification_preprocessing.resnet_pruned, } if name not in preprocessing_fn_map: raise ValueError('Preprocessing name [%s] was not recognized' % name) flip = kwargs.pop('flip', False) yuv2rgb = kwargs.pop('yuv2rgb', False) + yuy2 = kwargs.pop('yuy2', False) input_resize = kwargs.pop('input_resize', {}) if flip: height, width = width, height @@ -103,9 +113,11 @@ def preprocessing_fn(image, image_info=None): image, image_info = preprocessing_fn_map[name](image, image_info, height, width, flip=flip, **kwargs) if normalization_params: image = normalize(image, normalization_params) - if yuv2rgb: - image = convert_rgb_to_yuv(image) if input_resize.get('enabled', False): image = image_resize(image, input_resize.input_shape) + if yuv2rgb: + image = convert_rgb_to_yuv(image) + if yuy2: + image = convert_yuv_to_yuy2(image) return image, image_info return preprocessing_fn diff --git a/hailo_model_zoo/core/preprocessing/segmentation_preprocessing.py b/hailo_model_zoo/core/preprocessing/segmentation_preprocessing.py index 55aff709..f7cb1e43 100644 --- a/hailo_model_zoo/core/preprocessing/segmentation_preprocessing.py +++ b/hailo_model_zoo/core/preprocessing/segmentation_preprocessing.py @@ -46,19 +46,34 @@ def _pad_tensor(x, max_tensor_padding=MAX_PADDING_LENGTH): return tf.squeeze(tf.pad(tf.expand_dims(x, axis=0), paddings, "CONSTANT", constant_values=-1)) +def _get_resized_shape(size, height, width): + size = tf.cast(size, tf.float32) + h, w = tf.cast(height, tf.float32), tf.cast(width, tf.float32) + scale = size * 1.0 / tf.maximum(h, w) + newh = h * scale + neww = w * scale + neww = int(neww + 0.5) + newh = int(newh + 0.5) + return newh, neww + + def sparseinst(image, image_info=None, height=None, width=None, max_pad=MAX_PADDING_LENGTH, **kwargs): image_resized = image if height and width: - paddings = [[0, height - tf.shape(image)[0]], - [0, width - tf.shape(image)[1]], + assert height == width, 'sparseinst expects a square input but got {height}x{width}' + orig_height, orig_width = tf.shape(image)[0], tf.shape(image)[1] + newh, neww = _get_resized_shape(height, orig_height, orig_width) + image_resized_ar = tf.squeeze(tf.image.resize(image, size=(newh, neww), method='bilinear')) + paddings = [[0, tf.maximum(height - tf.shape(image_resized_ar)[0], 0)], + [0, tf.maximum(width - tf.shape(image_resized_ar)[1], 0)], [0, 0]] - image_padded = tf.squeeze(tf.pad(image, paddings, mode="CONSTANT", constant_values=0)) + image_padded = tf.squeeze(tf.pad(image_resized_ar, paddings, mode="CONSTANT", constant_values=0)) image_resized = tf.cast(image_padded, tf.float32) if image_info: - image_info['height'] = tf.cast(height, tf.int32) - image_info['width'] = tf.cast(width, tf.int32) - image_info['orig_height'] = tf.cast(tf.shape(image)[0], tf.int32) - image_info['orig_width'] = tf.cast(tf.shape(image)[1], tf.int32) + image_info['resized_height'] = tf.cast(newh, tf.int32) + image_info['resized_width'] = tf.cast(neww, tf.int32) + image_info['height'] = tf.cast(tf.shape(image)[0], tf.int32) + image_info['width'] = tf.cast(tf.shape(image)[1], tf.int32) image_info['img_orig'] = tf.cast(image_resized, tf.uint8) keys2pad = ['xmin', 'xmax', 'ymin', 'ymax', 'area', 'category_id', 'is_crowd'] for key in keys2pad: diff --git a/hailo_model_zoo/main.py b/hailo_model_zoo/main.py index fe76fe4f..2889c035 100755 --- a/hailo_model_zoo/main.py +++ b/hailo_model_zoo/main.py @@ -1,13 +1,23 @@ #!/usr/bin/env python import argparse +import importlib from pathlib import Path +import hailo_model_zoo.plugin # we try to minize imports to make 'main.py --help' responsive. So we only import definitions. -from hailo_sdk_common.profiler.profiler_common import ProfilerModes -from hailo_model_zoo.utils import path_resolver -from hailo_model_zoo.utils.hw_utils import TARGETS, DEVICE_NAMES +from hailo_model_zoo.utils.cli_utils import add_model_name_arg, HMZ_COMMANDS +from hailo_model_zoo.utils.constants import TARGETS, DEVICE_NAMES, PROFILER_MODE_NAMES + +from hailo_model_zoo.utils.plugin_utils import iter_namespace + + +discovered_plugins = { + name: importlib.import_module(name) + for finder, name, ispkg + in iter_namespace(hailo_model_zoo.plugin) +} def _make_parsing_base(): @@ -56,17 +66,16 @@ def _make_hef_base(): def _make_profiling_base(): profile_base_parser = argparse.ArgumentParser(add_help=False) - profiler_mode_names = {profiler_mode.value for profiler_mode in ProfilerModes} profile_base_parser.add_argument( '--mode', help='Profiling mode', dest='profile_mode', - type=str, default=ProfilerModes.PRE_PLACEMENT.value, - choices=profiler_mode_names) + type=str, default='pre_placement', + choices=PROFILER_MODE_NAMES) return profile_base_parser def _make_evaluation_base(): evaluation_base_parser = argparse.ArgumentParser(add_help=False) - targets = list(TARGETS.keys()) + targets = TARGETS devices = ', '.join(DEVICE_NAMES) evaluation_base_parser.add_argument( '--target', type=str, choices=targets, metavar='', default='full_precision', @@ -102,20 +111,6 @@ def _make_evaluation_base(): return evaluation_base_parser -def _make_info_base(): - info_base_parser = argparse.ArgumentParser(add_help=False) - add_model_name_arg(info_base_parser) - return info_base_parser - - -def add_model_name_arg(parser, optional=False): - network_names = list(path_resolver.get_network_names()) - # Setting empty metavar in order to prevent listing the models twice - nargs = '?' if optional else None - parser.add_argument('model_name', type=str, nargs=nargs, choices=network_names, metavar='model_name', - help='Which network to run. Choices: ' + ', '.join(network_names)) - - def _create_args_parser(): # --- create shared arguments parsers parsing_base_parser = _make_parsing_base() @@ -123,7 +118,6 @@ def _create_args_parser(): hef_base_parser = _make_hef_base() profile_base_parser = _make_profiling_base() evaluation_base_parser = _make_evaluation_base() - information_base_parser = _make_info_base() # --- create per action subparser parser = argparse.ArgumentParser(epilog='Example: hailomz parse resnet_v1_50') @@ -152,17 +146,20 @@ def _create_args_parser(): parsing_base_parser, optimization_base_parser, hef_base_parser, evaluation_base_parser], help="infer the model using the Hailo Emulator or the Hailo hardware and produce the model accuracy.") - subparsers.add_parser('info', parents=[information_base_parser], - help="Print model information.") - + # add parsers for plugins + for command in HMZ_COMMANDS: + command_parser = command.parser_fn() + subparsers.add_parser(command.name, parents=[command_parser], help=command_parser.description) return parser def run(args): - if args.command == 'info': - from hailo_model_zoo.info_main import info - return info(args) + # search for commands from plugins + command_to_handler = {command.name: command.fn for command in HMZ_COMMANDS} + if args.command in command_to_handler: + return command_to_handler[args.command](args) + # we make sure to only import these now to keep loading & plugins fast from hailo_model_zoo.main_driver import parse, optimize, compile, profile, evaluate handlers = { 'parse': parse, diff --git a/hailo_model_zoo/main_driver.py b/hailo_model_zoo/main_driver.py index 047169a1..bc646a24 100644 --- a/hailo_model_zoo/main_driver.py +++ b/hailo_model_zoo/main_driver.py @@ -1,4 +1,5 @@ import io +from pathlib import Path try: from hailo_platform import HEF, PcieDevice @@ -6,16 +7,19 @@ except ModuleNotFoundError: HEF_EXISTS = False +from hailo_sdk_client import InferenceContext from hailo_sdk_common.targets.inference_targets import SdkFPOptimized, SdkPartialNumeric from hailo_sdk_common.profiler.profiler_common import ProfilerModes from hailo_sdk_client import ClientRunner from hailo_sdk_client.exposed_definitions import States from hailo_sdk_client.tools.profiler.react_report_generator import ReactReportGenerator -from hailo_model_zoo.core.main_utils import (get_network_info, parse_model, - optimize_model, infer_model, compile_model, get_hef_path, - resolve_alls_path, _get_integrated_postprocessing) +from hailo_model_zoo.core.main_utils import ( + get_network_info, parse_model, optimize_model, + compile_model, get_hef_path, + infer_model_tf1, infer_model_tf2, + resolve_alls_path, get_integrated_postprocessing) from hailo_model_zoo.utils.path_resolver import get_network_peformance -from hailo_model_zoo.utils.hw_utils import DEVICE_NAMES, TARGETS +from hailo_model_zoo.utils.hw_utils import DEVICE_NAMES, TARGETS, INFERENCE_TARGETS, DEVICES from hailo_model_zoo.utils.logger import get_logger @@ -25,16 +29,20 @@ def _ensure_performance(model_name, model_script, performance, logger): logger.info(f'Running {model_name} with default model script.\n\ To obtain maximum performance use --performance:\n\ hailomz {model_name} --performance') - if performance and model_script and model_script.parent.name == "base": - logger.info(f'Using base alls script found in {model_script} because there is no performance alls') + if performance and model_script: + if model_script.parent.name == "base": + logger.info(f'Using base alls script found in {model_script} because there is no performance alls') + elif model_script.parent.name != "performance" and model_name in get_network_peformance(): + logger.info('Using alls script {model_script}') def _extract_model_script_path(networks_alls_script, model_script_path, performance): - return model_script_path if model_script_path else resolve_alls_path(networks_alls_script, performance=performance) + return Path(model_script_path) if model_script_path \ + else resolve_alls_path(networks_alls_script, performance=performance) def _ensure_compiled(runner, logger, args, network_info): - if runner.state == States.COMPILED_MODEL: + if runner.state == States.COMPILED_MODEL or runner.hef: return logger.info("Compiling the model (without inference) ...") compile_model(runner, network_info, args.results_dir, allocator_script_filename=args.model_script_path) @@ -43,7 +51,7 @@ def _ensure_compiled(runner, logger, args, network_info): def _ensure_optimized(runner, logger, args, network_info): _ensure_parsed(runner, logger, network_info, args) - integrated_postprocessing = _get_integrated_postprocessing(network_info) + integrated_postprocessing = get_integrated_postprocessing(network_info) if integrated_postprocessing and integrated_postprocessing.enabled and args.model_script_path is not None: raise ValueError(f"Network {network_info.network.network_name} joins several networks together\n" "and cannot get a user model script") @@ -67,14 +75,26 @@ def _ensure_parsed(runner, logger, network_info, args): parse_model(runner, network_info, ckpt_path=args.ckpt_path, results_dir=args.results_dir, logger=logger) -def _ensure_runnable_state(args, logger, network_info, runner, target): +def configure_hef_tf1(hef_path, target): + hef = HEF(hef_path) + network_groups = target.configure(hef) + return network_groups + + +def configure_hef_tf2(runner, hef_path): + if hef_path: + runner.hef = hef_path + return + + +def _ensure_runnable_state_tf1(args, logger, network_info, runner, target): _ensure_parsed(runner, logger, network_info, args) - if isinstance(target, SdkFPOptimized): + if isinstance(target, SdkFPOptimized) or (isinstance(target, PcieDevice) and args.hef_path is not None): if runner.state == States.HAILO_MODEL: - integrated_postprocessing = _get_integrated_postprocessing(network_info) + integrated_postprocessing = get_integrated_postprocessing(network_info) if integrated_postprocessing and integrated_postprocessing.enabled: - runner.apply_model_modification_commands() - return None + runner.optimize_full_precision() + return None if not args.hef_path else configure_hef_tf1(args.hef_path, target) # We intenionally use base model script and assume its modifications # compatible to the performance model script model_script = _extract_model_script_path(network_info.paths.alls_script, @@ -82,13 +102,12 @@ def _ensure_runnable_state(args, logger, network_info, runner, target): False) runner.load_model_script(model_script) - runner.apply_model_modification_commands() - return None + runner.optimize_full_precision() + + return None if not args.hef_path else configure_hef_tf1(args.hef_path, target) if args.hef_path: - hef = HEF(args.hef_path) - network_groups = target.configure(hef) - return network_groups + return configure_hef_tf1(args.hef_path, target) _ensure_optimized(runner, logger, args, network_info) @@ -100,6 +119,41 @@ def _ensure_runnable_state(args, logger, network_info, runner, target): return None +def _ensure_runnable_state_tf2(args, logger, network_info, runner, target): + _ensure_parsed(runner, logger, network_info, args) + if target == InferenceContext.SDK_FP_OPTIMIZED or \ + (target == InferenceContext.SDK_HAILO_HW and args.hef_path is not None): + if runner.state != States.HAILO_MODEL: + configure_hef_tf2(runner, args.hef_path) + return + + integrated_postprocessing = get_integrated_postprocessing(network_info) + if integrated_postprocessing and integrated_postprocessing.enabled: + runner.optimize_full_precision() + configure_hef_tf2(runner, args.hef_path) + return + # We intenionally use base model script and assume its modifications + # compatible to the performance model script + model_script = _extract_model_script_path(network_info.paths.alls_script, + args.model_script_path, + False) + + runner.load_model_script(model_script) + runner.optimize_full_precision() + configure_hef_tf2(runner, args.hef_path) + return + + configure_hef_tf2(runner, args.hef_path) + + _ensure_optimized(runner, logger, args, network_info) + + if target == InferenceContext.SDK_QUANTIZED: + return + + _ensure_compiled(runner, logger, args, network_info) + return + + def _str_to_profiling_mode(name): return ProfilerModes[name.upper()] @@ -185,7 +239,7 @@ def profile(args): args.performance) _ensure_performance(model_name, model_script, args.performance, logger) runner.load_model_script(model_script) - runner.apply_model_modification_commands() + runner.optimize_full_precision() else: # Optimize the model so profile_hn_model could compile & profile it _ensure_optimized(runner, logger, args, network_info) @@ -194,10 +248,9 @@ def profile(args): if profile_mode is not ProfilerModes.PRE_PLACEMENT else None _ensure_performance(model_name, alls_script_path, args.performance, logger) - stats, csv_data, latency_data, accuracy_data = runner.profile_hn_model(profiling_mode=profile_mode, - should_use_logical_layers=True, - allocator_script=alls_script_path, - hef_filename=args.hef_path) + stats, csv_data, latency_data, accuracy_data = runner.profile(profiling_mode=profile_mode, + should_use_logical_layers=True, + hef_filename=args.hef_path) mem_file = io.StringIO() outpath = args.results_dir / f'{model_name}.html' @@ -246,20 +299,32 @@ def evaluate(args): network_groups = None logger.info(f'Chosen target is {args.target}') - hailo_target = TARGETS[args.target] - with hailo_target() as target: - network_groups = _ensure_runnable_state(args, logger, network_info, runner, target) + batch_size = args.batch_size or __get_batch_size(network_info, args.target) + infer_type = network_info.evaluation.infer_type + # legacy tf1 inference flow + if infer_type not in ['runner_infer', 'model_infer', 'np_infer', 'facenet_infer', + 'np_infer_lite', 'model_infer_lite']: + hailo_target = TARGETS[args.target] + with hailo_target() as target: + network_groups = _ensure_runnable_state_tf1(args, logger, network_info, runner, target) + return infer_model_tf1(runner, network_info, target, logger, + args.eval_num_examples, args.data_path, batch_size, + args.print_num_examples, args.visualize_results, args.video_outpath, + dump_results=False, network_groups=network_groups) - batch_size = args.batch_size or __get_batch_size(network_info, target) - result = infer_model(runner, network_info, target, logger, - args.eval_num_examples, args.data_path, batch_size, - args.print_num_examples, args.visualize_results, args.video_outpath, - dump_results=False, network_groups=network_groups) + else: + # new tf2 inference flow + target = INFERENCE_TARGETS[args.target] + _ensure_runnable_state_tf2(args, logger, network_info, runner, target) - return result + device_info = DEVICES.get(args.target) + context = runner.infer_context(target, device_info) + return infer_model_tf2(runner, network_info, context, logger, args.eval_num_examples, args.data_path, + batch_size, args.print_num_examples, args.visualize_results, args.video_outpath, + dump_results=False) def __get_batch_size(network_info, target): - if target.name == 'sdk_fp_optimized': + if target == 'full_precision': return network_info.inference.full_precision_batch_size return network_info.inference.emulator_batch_size diff --git a/hailo_model_zoo/plugin/__init__.py b/hailo_model_zoo/plugin/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/hailo_model_zoo/info_main.py b/hailo_model_zoo/plugin/info_main.py similarity index 77% rename from hailo_model_zoo/info_main.py rename to hailo_model_zoo/plugin/info_main.py index 7dc81d54..2ed067d5 100644 --- a/hailo_model_zoo/info_main.py +++ b/hailo_model_zoo/plugin/info_main.py @@ -1,4 +1,7 @@ +import argparse + from hailo_model_zoo.core.info_utils import get_network_info +from hailo_model_zoo.utils.cli_utils import register_command, add_model_name_arg from hailo_model_zoo.utils.logger import get_logger @@ -21,6 +24,13 @@ def build_dict(info): logger.info(msg_w_line) +def make_info_base(): + info_base_parser = argparse.ArgumentParser(add_help=False, description="Print model information.") + add_model_name_arg(info_base_parser) + return info_base_parser + + +@register_command(make_info_base) def info(args): logger = get_logger() network_info = get_network_info(args.model_name) diff --git a/hailo_model_zoo/utils/cli_utils.py b/hailo_model_zoo/utils/cli_utils.py new file mode 100644 index 00000000..85603868 --- /dev/null +++ b/hailo_model_zoo/utils/cli_utils.py @@ -0,0 +1,22 @@ +from collections import namedtuple +from hailo_model_zoo.utils import path_resolver + + +def add_model_name_arg(parser, optional=False): + network_names = list(path_resolver.get_network_names()) + # Setting empty metavar in order to prevent listing the models twice + nargs = '?' if optional else None + parser.add_argument('model_name', type=str, nargs=nargs, choices=network_names, metavar='model_name', + help='Which network to run. Choices: ' + ', '.join(network_names)) + + +Command = namedtuple('Command', ['name', 'fn', 'parser_fn']) +HMZ_COMMANDS = [] + + +def register_command(parser_factory, *, name=None): + def _register_inner(func): + command_name = name or func.__name__ + HMZ_COMMANDS.append(Command(command_name, func, parser_factory)) + return func + return _register_inner diff --git a/hailo_model_zoo/utils/constants.py b/hailo_model_zoo/utils/constants.py new file mode 100644 index 00000000..5c2f8f9d --- /dev/null +++ b/hailo_model_zoo/utils/constants.py @@ -0,0 +1,19 @@ +try: + from hailo_platform import PcieDevice, HailoRTException + PLATFORM_AVAILABLE = True +except ModuleNotFoundError: + PLATFORM_AVAILABLE = False +PROFILER_MODE_NAMES = {'pre_placement', 'post_placement'} +TARGETS = [ + 'hailo8', + 'full_precision', + 'emulator', +] +DEVICE_NAMES = set() +if PLATFORM_AVAILABLE: + try: + devices = PcieDevice.scan_devices() + DEVICE_NAMES.update([str(name) for name in devices]) + except HailoRTException: + # Ignore HailoRT exception when the driver is not installed + pass diff --git a/hailo_model_zoo/utils/hw_utils.py b/hailo_model_zoo/utils/hw_utils.py index a012e5f8..681a53ce 100644 --- a/hailo_model_zoo/utils/hw_utils.py +++ b/hailo_model_zoo/utils/hw_utils.py @@ -1,25 +1,32 @@ from hailo_sdk_common.targets.inference_targets import SdkPartialNumeric, SdkFPOptimized +from hailo_sdk_client import InferenceContext try: - from hailo_platform import PcieDevice + from hailo_platform import PcieDevice, HailoRTException PLATFORM_AVAILABLE = True except ModuleNotFoundError: PLATFORM_AVAILABLE = False -TARGETS = {'hailo8': PcieDevice if PLATFORM_AVAILABLE else None, - 'full_precision': SdkFPOptimized, - 'emulator': SdkPartialNumeric, - } +TARGETS = { + 'hailo8': PcieDevice if PLATFORM_AVAILABLE else None, + 'full_precision': SdkFPOptimized, + 'emulator': SdkPartialNumeric, +} + +INFERENCE_TARGETS = { + 'hailo8': InferenceContext.SDK_HAILO_HW, + 'full_precision': InferenceContext.SDK_FP_OPTIMIZED, + 'emulator': InferenceContext.SDK_QUANTIZED, +} + +DEVICES = {} DEVICE_NAMES = set() if PLATFORM_AVAILABLE: - try: - from hailo_platform.pyhailort.pyhailort import HailoRTException - except ModuleNotFoundError: - from hailo_platform.drivers.hailort.pyhailort import HailoRTException - try: devices = PcieDevice.scan_devices() TARGETS.update({str(name): lambda: PcieDevice(name) for name in devices}) + INFERENCE_TARGETS.update({str(name): InferenceContext.SDK_HAILO_HW for name in devices}) + DEVICES.update({str(name): name for name in devices}) DEVICE_NAMES.update([str(name) for name in devices]) except HailoRTException: # Ignore HailoRT exception when the driver is not installed diff --git a/hailo_model_zoo/utils/logger.py b/hailo_model_zoo/utils/logger.py index 3cfc81eb..98bc50b3 100644 --- a/hailo_model_zoo/utils/logger.py +++ b/hailo_model_zoo/utils/logger.py @@ -2,10 +2,8 @@ import logging from logging.handlers import RotatingFileHandler from pathlib import Path -from hailo_sdk_common.logger.logger import create_custom_logger, DFC_FOLDER_PATH _g_logger = None -MZ_FOLDER_PATH = str(Path(DFC_FOLDER_PATH, '..', 'modelzoo')) class HailoExamplesFormatter(logging.Formatter): @@ -35,6 +33,11 @@ def namer(name): def get_logger(): + + # for faster loading + from hailo_sdk_common.logger.logger import create_custom_logger, DFC_FOLDER_PATH + MZ_FOLDER_PATH = str(Path(DFC_FOLDER_PATH, '..', 'modelzoo')) + global _g_logger if _g_logger is None: # setting console to False to set a custom console logger diff --git a/hailo_model_zoo/utils/numpy_utils.py b/hailo_model_zoo/utils/numpy_utils.py new file mode 100644 index 00000000..60a924be --- /dev/null +++ b/hailo_model_zoo/utils/numpy_utils.py @@ -0,0 +1,16 @@ +import numpy as np +import tensorflow as tf + + +def to_numpy(tensor, *, decode=False): + if isinstance(tensor, np.ndarray): + return tensor + if hasattr(tensor, 'numpy'): + result = tensor.numpy() + if decode: + result = result.decode('utf8') + return result + if isinstance(tensor, str): + return tensor + + return tf.nest.map_structure(to_numpy, tensor) diff --git a/hailo_model_zoo/utils/parse_utils.py b/hailo_model_zoo/utils/parse_utils.py index f246f983..3e77f7b0 100644 --- a/hailo_model_zoo/utils/parse_utils.py +++ b/hailo_model_zoo/utils/parse_utils.py @@ -1,6 +1,3 @@ -from hailo_sdk_common.hailo_nn.hn_definitions import EWAddPolicy, DEFAULT_EW_ADD_POLICY - - def get_normalize_in_net(network): normalization_params = network['parser'].get('normalization_params', {}) return normalization_params.get('normalize_in_net', False) @@ -20,8 +17,6 @@ def get_normalization_params(network_info): def translate_model(runner, network_info, ckpt_path, *, tensor_shapes=None): model_name = network_info.network.network_name start_node, end_node = network_info.parser.nodes[0:2] - ew_add_policy = network_info.parser.get("ew_add_policy", DEFAULT_EW_ADD_POLICY.value) - ew_add_policy = EWAddPolicy[ew_add_policy] if type(end_node) == str: end_node = [end_node] @@ -34,11 +29,11 @@ def translate_model(runner, network_info, ckpt_path, *, tensor_shapes=None): start_node_names=start_node, end_node_names=end_node, net_input_shapes=tensor_shapes, - ew_add_policy=ew_add_policy) + ) else: runner.translate_tf_model(ckpt_path, model_name, start_node_names=start_node, end_node_names=end_node, tensor_shapes=tensor_shapes, - ew_add_policy=ew_add_policy) + ) return model_name diff --git a/hailo_model_zoo/utils/plugin_utils.py b/hailo_model_zoo/utils/plugin_utils.py new file mode 100644 index 00000000..9adb3f3d --- /dev/null +++ b/hailo_model_zoo/utils/plugin_utils.py @@ -0,0 +1,9 @@ +import pkgutil + + +def iter_namespace(ns_pkg): + # Specifying the second argument (prefix) to iter_modules makes the + # returned name an absolute name instead of a relative one. This allows + # import_module to work without having to do additional modification to + # the name. + return pkgutil.iter_modules(ns_pkg.__path__, ns_pkg.__name__ + ".") diff --git a/hailo_models/license_plate_recognition/Dockerfile b/hailo_models/license_plate_recognition/Dockerfile index ec75e998..c1a2291f 100644 --- a/hailo_models/license_plate_recognition/Dockerfile +++ b/hailo_models/license_plate_recognition/Dockerfile @@ -10,7 +10,7 @@ RUN apt-get update && \ RUN git clone https://github.com/hailo-ai/LPRNet_Pytorch.git && \ cd LPRNet_Pytorch && \ pip install --upgrade pip && \ - pip install -U 'imutils==0.5.4' 'opencv-python>=4.5.5' 'imgaug==0.4.0' 'tensorboard==2.7.0' 'torchsummary' 'pandas==1.3.5' 'strsimpy==0.2.1' 'numpy==1.19.2' 'jupyter' 'protobuf==3.20.1' + pip install -U 'imutils==0.5.4' 'opencv-python==4.5.5.64' 'imgaug==0.4.0' 'tensorboard==2.7.0' 'torchsummary==1.5' 'pandas==1.3.5' 'strsimpy==0.2.1' 'numpy==1.19.2' 'jupyter==1.0.0' 'protobuf==3.20.1' WORKDIR /workspace/LPRNet_Pytorch/ RUN cd /workspace/LPRNet_Pytorch/ && \ diff --git a/hailo_models/personface_detection/Dockerfile b/hailo_models/personface_detection/Dockerfile index 4526c356..162fb576 100644 --- a/hailo_models/personface_detection/Dockerfile +++ b/hailo_models/personface_detection/Dockerfile @@ -7,11 +7,11 @@ ENV DEBIAN_FRONTEND=noninteractive, TZ=$timezone RUN apt-get update && \ apt-get -y --no-install-recommends install git build-essential python3-opencv wget vim -RUN git clone https://github.com/hailo-ai/yolov5.git --branch v2.0 && \ +RUN git clone https://github.com/hailo-ai/yolov5.git --branch v2.0.1 && \ cd yolov5 && \ pip install --upgrade pip && \ pip install -r requirements.txt && \ - pip install -U 'coremltools>=4.1' 'onnx>=1.9.0' 'scikit-learn==0.19.2' 'scipy==1.9.1' 'numpy==1.19.2' + pip install -U 'coremltools==6.1' 'onnx==1.13.0' 'scikit-learn==0.19.2' ENV PYTHONPATH=/workspace/yolov5 WORKDIR /workspace/yolov5 diff --git a/hailo_models/reid/Dockerfile b/hailo_models/reid/Dockerfile index 0b9b6d6e..ac9b0ae3 100644 --- a/hailo_models/reid/Dockerfile +++ b/hailo_models/reid/Dockerfile @@ -9,11 +9,8 @@ RUN apt-get update && \ RUN git clone --depth 1 https://github.com/hailo-ai/deep-person-reid.git && \ cd deep-person-reid && \ - python3 -m pip install --upgrade Pillow && \ - pip install -U 'numpy==1.19.2' 'scipy==1.9.1' && \ pip install -r requirements.txt && \ - python setup.py develop && \ - pip install -U 'numpy==1.19.2' 'scipy==1.9.1' + python setup.py develop ENV PYTHONPATH=/workspace/deep-person-reid WORKDIR /workspace/deep-person-reid diff --git a/hailo_models/vehicle_detection/Dockerfile b/hailo_models/vehicle_detection/Dockerfile index c9ae6d8b..3a17bfe9 100644 --- a/hailo_models/vehicle_detection/Dockerfile +++ b/hailo_models/vehicle_detection/Dockerfile @@ -7,11 +7,11 @@ ENV DEBIAN_FRONTEND=noninteractive, TZ=$timezone RUN apt-get update && \ apt-get -y --no-install-recommends install git build-essential python3-opencv wget -RUN git clone https://github.com/ultralytics/yolov5.git --branch v2.0 && \ +RUN git clone https://github.com/hailo-ai/yolov5.git --branch v2.0.1 && \ cd yolov5 && \ pip install --upgrade pip && \ pip install -r requirements.txt && \ - pip install -U 'coremltools>=4.1' 'onnx>=1.9.0' 'scikit-learn==0.19.2' 'scipy==1.9.1' 'numpy==1.19.2' + pip install -U 'coremltools==6.1' 'onnx==1.13.0' 'scikit-learn==0.19.2' ENV PYTHONPATH=/workspace/yolov5 WORKDIR /workspace/yolov5 diff --git a/setup.py b/setup.py index 012a2869..19135a08 100755 --- a/setup.py +++ b/setup.py @@ -45,11 +45,11 @@ def main(): 'lap==0.4.0', 'motmetrics==1.2.5', 'omegaconf==2.1.0', - 'pillow==8.1.2', + 'pillow<=9.2.0', 'detection-tools==0.3', 'scikit-image==0.19.3'] - model_zoo_version = "2.6.1" + model_zoo_version = "2.7.0" package_data = { "hailo_model_zoo": [ diff --git a/training/arcface/Dockerfile b/training/arcface/Dockerfile index e5ddfab5..2432fb37 100644 --- a/training/arcface/Dockerfile +++ b/training/arcface/Dockerfile @@ -13,16 +13,13 @@ RUN apt-get update && \ # solve mpi conflicts { which mpirun && apt-get remove -y libopenmpi3 || true ; } -# workaround broken opencv-python 4.6 -RUN pip install opencv-python==4.5.5.64 ARG repo=https://github.com/hailo-ai/insightface RUN mkdir /workspace/data && \ git clone $repo && \ cd insightface/recognition/arcface_torch && \ pip install -r requirement.txt && \ - pip install onnxruntime onnxsim && \ - pip install numpy==1.22.0 + pip install onnxruntime==1.13.1 onnxsim==0.4.13 WORKDIR /workspace/insightface/recognition/arcface_torch ARG user=hailo diff --git a/training/centerpose/Dockerfile b/training/centerpose/Dockerfile index 1559e820..a5756827 100644 --- a/training/centerpose/Dockerfile +++ b/training/centerpose/Dockerfile @@ -13,8 +13,7 @@ RUN apt-get update && \ # solve mpi conflicts { which mpirun && apt-get remove -y libopenmpi3 || true ; } -# workaround broken opencv-python 4.6 -RUN pip install opencv-python==4.5.5.64 + ARG repo=https://github.com/hailo-ai/centerpose RUN git clone $repo && \ cd centerpose && pip install -r requirements.txt diff --git a/training/damoyolo/Dockerfile b/training/damoyolo/Dockerfile new file mode 100644 index 00000000..8e0bfb11 --- /dev/null +++ b/training/damoyolo/Dockerfile @@ -0,0 +1,23 @@ +ARG base_image=pytorch/pytorch:1.7.1-cuda11.0-cudnn8-runtime +ARG timezone +FROM $base_image + +ENV DEBIAN_FRONTEND=noninteractive, TZ=$timezone + +RUN apt-get update && \ + apt-get -y --no-install-recommends install git build-essential python3-opencv wget vim + +RUN git clone https://github.com/hailo-ai/DAMO-YOLO.git && \ + cd DAMO-YOLO && \ + pip install --upgrade pip && \ + pip install -r requirements.txt + +RUN pip install git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI + +ENV PYTHONPATH=/workspace/DAMO-YOLO +WORKDIR /workspace/DAMO-YOLO + +RUN cd /workspace/DAMO-YOLO && \ + wget https://idstcv.oss-cn-zhangjiakou.aliyuncs.com/DAMO-YOLO/clean_models/before_distill/damoyolo_tinynasL20_T_418.pth -q && \ + wget https://idstcv.oss-cn-zhangjiakou.aliyuncs.com/DAMO-YOLO/clean_models/before_distill/damoyolo_tinynasL25_S_456.pth -q && \ + wget https://idstcv.oss-cn-zhangjiakou.aliyuncs.com/DAMO-YOLO/clean_models/before_distill/damoyolo_tinynasL35_M_487.pth -q diff --git a/training/damoyolo/README.rst b/training/damoyolo/README.rst new file mode 100644 index 00000000..3689ef50 --- /dev/null +++ b/training/damoyolo/README.rst @@ -0,0 +1,120 @@ +===================== +DAMO-YOLO Retraining +===================== + +* To learn more about DAMO-YOLO visit the `official repository `_ + +---------- + +Prerequisites +------------- + +* docker (\ `docker installation instructions `_\ ) +* nvidia-docker2 (\ `nvidia docker installation instructions `_\ ) + + +**NOTE:** In case you are using the Hailo Software Suite docker, make sure to run all of the following instructions outside of that docker. + +Environment Preparations +------------------------ + + +#. | Build the docker image: + + .. raw:: html + :name:validation + + 

+      cd hailo_model_zoo/training/damoyolo
+      docker build --build-arg timezone=`cat /etc/timezone` -t damoyolo:v0 .
+
+ + | the following optional arguments can be passed via --build-arg: + + * ``timezone`` - a string for setting up timezone. E.g. "Asia/Jerusalem" + * ``user`` - username for a local non-root user. Defaults to 'hailo'. + * ``group`` - default group for a local non-root user. Defaults to 'hailo'. + * ``uid`` - user id for a local non-root user. + * ``gid`` - group id for a local non-root user. + + | * This command will build the docker image with the necessary requirements using the Dockerfile exists in damoyolo directory. + + +#. | Start your docker: + + .. raw:: html + :name:validation + + + docker run --name "your_docker_name" -it --gpus all --ipc=host -v /path/to/local/data/dir:/path/to/docker/data/dir damoyolo:v0 + + + * ``docker run`` create a new docker container. + * ``--name `` name for your container. + * ``-it`` runs the command interactively. + * ``--gpus all`` allows access to all GPUs. + * ``--ipc=host`` sets the IPC mode for the container. + * ``-v /path/to/local/data/dir:/path/to/docker/data/dir`` maps ``/path/to/local/data/dir`` from the host to the container. You can use this command multiple times to mount multiple directories. + * ``damoyolo:v0`` the name of the docker image. + +Training and exporting to ONNX +------------------------------ + + +#. | Train your model: + | Once the docker is started, you can start training your model. + + * | Prepare your custom dataset (must be coco format) - Follow the steps described `here `_. + * | Modify ``num_classes`` and ``class_names`` in the configuration file, for example ``damoyolo_tinynasL20_T.py`` + * | Use ``self.train.batch_size`` / ``self.train.total_epochs`` in the configuration file to modify the batch_size and number of epochs + * | Update the symbolic link to your dataset: ln -sfn /your/coco/like/dataset/path datasets/coco + * | Start training - The following command is an example for training a *damoyolo_tinynasL20_T* model. + + .. raw:: html + :name:validation + + + python tools/train.py -f configs/damoyolo_tinynasL20_T.py + 

+                                 configs/damoyolo_tinynasL25_S.py
+                                 configs/damoyolo_tinynasL35_M.py
+
+ + + * ``configs/damoyolo_tinynasL20_T.py`` - configuration file of the DAMO-YOLO variant you would like to train. In order to change the number of classes make sure you update ``num_classes`` and ``class_names`` in this file. + + +#. | Export to ONNX: + + | In order to export your trained DAMO-YOLO model to ONNX run the following script: + + .. raw:: html + :name:validation + + + python tools/converter.py -f configs/damoyolo_tinynasL20_T.py -c /path/to/trained/model.pth --batch_size 1 --img_size 640 # export at 640x640 with batch size 1 + + +---- + +Compile the Model using Hailo Model Zoo +--------------------------------------- + +| You can generate an HEF file for inference on Hailo-8 from your trained ONNX model. +| In order to do so you need a working model-zoo environment. +| Choose the corresponding YAML from our networks configuration directory, i.e. ``hailo_model_zoo/cfg/networks/damoyolo_tinynasL20_T.yaml``\ , and run compilation using the model zoo: + +.. raw:: html + :name:validation + + + hailomz compile --ckpt damoyolo_tinynasL20_T.onnx --calib-path /path/to/calibration/imgs/dir/ --yaml path/to/damoyolo/variant.yaml + + +* | ``--ckpt`` - path to your ONNX file. +* | ``--calib-path`` - path to a directory with your calibration images in JPEG/png format +* | ``--yaml`` - path to your configuration YAML file. +* | The model zoo will take care of adding the input normalization to be part of the model. + + + More details about YAML files are presented `here <../../docs/YAML.rst>`_. diff --git a/training/fcn/Dockerfile b/training/fcn/Dockerfile index 209edf52..c05da4ac 100644 --- a/training/fcn/Dockerfile +++ b/training/fcn/Dockerfile @@ -15,7 +15,7 @@ RUN apt-get update && \ ARG repo=https://github.com/hailo-ai/mmsegmentation.git RUN git clone $repo && \ cd mmsegmentation && \ - pip install mmcv-full==1.4.5 onnxruntime && \ + pip install mmcv-full==1.4.5 onnxruntime==1.13.1 && \ pip install -e . WORKDIR /workspace/mmsegmentation diff --git a/training/mspn/Dockerfile b/training/mspn/Dockerfile index 879fc2aa..00370aa4 100644 --- a/training/mspn/Dockerfile +++ b/training/mspn/Dockerfile @@ -10,7 +10,7 @@ RUN pip install mmcv-full==1.6.0 && \ git clone https://github.com/hailo-ai/mmpose.git && \ cd mmpose && pip install -e . && \ mkdir -p data/coco/annotations && mkdir pretrained && \ - pip install tensorboard onnx onnxruntime onnx-simplifier netron ipdb && \ + pip install tensorboard==2.11.2 onnxruntime==1.13.1 onnx-simplifier==0.4.13 netron==6.4.7 ipdb==0.13.11 && \ wget -P ./data/coco/ https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/SinglePersonPoseEstimation/mspn_regnetx_800mf/pretrained/2022-07-12/coco_gt_as_results.json && \ wget -P ./pretrained https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/SinglePersonPoseEstimation/mspn_regnetx_800mf/pretrained/2022-07-12/best_AP_epoch_310.pth diff --git a/training/nanodet/Dockerfile b/training/nanodet/Dockerfile index 2ba819b9..8a635b7c 100644 --- a/training/nanodet/Dockerfile +++ b/training/nanodet/Dockerfile @@ -12,10 +12,6 @@ RUN apt-get update && \ # solve mpi conflicts { which mpirun && apt-get remove -y libopenmpi3 || true ; } -# workaround broken opencv-python 4.6 -RUN pip install opencv-python==4.5.5.64 -#newer pytorch-lightning do not include ProgressBar -RUN pip install pytorch-lightning==1.6.5 ARG repo=https://github.com/hailo-ai/nanodet.git RUN git clone $repo && \ diff --git a/training/ssd/Dockerfile b/training/ssd/Dockerfile index 88ff0d14..e41749a9 100644 --- a/training/ssd/Dockerfile +++ b/training/ssd/Dockerfile @@ -27,8 +27,7 @@ RUN apt-get update && apt-get -y --no-install-recommends install \ wget RUN git clone https://github.com/hailo-ai/models.git /home/tensorflow/models \ - && cd /home/tensorflow/models \ - && git checkout 10ee28d + && cd /home/tensorflow/models # Compile protobuf configs RUN (cd /home/tensorflow/models/research/ && protoc object_detection/protos/*.proto --python_out=.) diff --git a/training/yolact/Dockerfile b/training/yolact/Dockerfile index 24967a25..7c8f6140 100644 --- a/training/yolact/Dockerfile +++ b/training/yolact/Dockerfile @@ -11,9 +11,9 @@ RUN apt-get update && \ RUN git clone https://github.com/hailo-ai/yolact.git --branch Model-Zoo-1.5 && \ cd yolact && \ pip install --upgrade pip && \ - pip install cython && \ - pip install opencv-python pillow pycocotools matplotlib && \ - pip install pycls + pip install cython==0.29.30 && \ + pip install opencv-python==4.5.5.64 pillow==8.1.0 pycocotools==2.0.6 matplotlib==3.5.2 && \ + pip install pycls==0.1.1 ENV PYTHONPATH=/workspace/yolact WORKDIR /workspace/yolact diff --git a/training/yolov4/Dockerfile b/training/yolov4/Dockerfile index 410b1d28..31726a53 100644 --- a/training/yolov4/Dockerfile +++ b/training/yolov4/Dockerfile @@ -6,7 +6,7 @@ ENV DEBIAN_FRONTEND=noninteractive, TZ=$timezone RUN apt-get update && \ apt-get -y --no-install-recommends install git build-essential python3-opencv wget vim -RUN pip install gdown +RUN pip install gdown==4.6.0 RUN git clone -b hailo_ai https://github.com/hailo-ai/pytorch-YOLOv4.git && \ git clone https://github.com/hailo-ai/darknet.git diff --git a/training/yolov5/Dockerfile b/training/yolov5/Dockerfile index 54007760..bbca70bf 100644 --- a/training/yolov5/Dockerfile +++ b/training/yolov5/Dockerfile @@ -7,11 +7,11 @@ ENV DEBIAN_FRONTEND=noninteractive, TZ=$timezone RUN apt-get update && \ apt-get -y --no-install-recommends install git build-essential python3-opencv wget vim -RUN git clone https://github.com/hailo-ai/yolov5.git --branch v2.0 && \ +RUN git clone https://github.com/hailo-ai/yolov5.git --branch v2.0.1 && \ cd yolov5 && \ pip install --upgrade pip && \ pip install -r requirements.txt && \ - pip install -U 'coremltools>=4.1' 'onnx>=1.9.0' 'scikit-learn==0.19.2' 'scipy==1.9.1' 'numpy==1.19.2' + pip install -U 'coremltools==6.1' 'onnx==1.13.0' 'scikit-learn==0.19.2' ENV PYTHONPATH=/workspace/yolov5 WORKDIR /workspace/yolov5 diff --git a/training/yolox/Dockerfile b/training/yolox/Dockerfile index 4b3435b0..3711c8ff 100644 --- a/training/yolox/Dockerfile +++ b/training/yolox/Dockerfile @@ -13,14 +13,11 @@ RUN apt-get update && \ # solve mpi conflicts { which mpirun && apt-get remove -y libopenmpi3 || true ; } -# workaround broken opencv-python 4.6 -RUN pip install opencv-python==4.5.5.64 - ARG repo=https://github.com/hailo-ai/YOLOX RUN mkdir /workspace/data && \ git clone $repo && \ cd YOLOX && pip install -r requirements.txt && pip install -e . && \ - pip install cython + pip install cython==0.29.24 ENV YOLOX_DATADIR=/workspace/data WORKDIR /workspace/YOLOX