Merge pull request #145 from hailo-ai/update-to-version-2.14.0

update-to-version-2.14.0

README.rst

@@ -22,14 +22,14 @@ Hailo Model Zoo
    :height: 20
 
 
-.. |compiler| image:: https://img.shields.io/badge/Hailo%20Dataflow%20Compiler-3.29.0-brightgreen.svg
+.. |compiler| image:: https://img.shields.io/badge/Hailo%20Dataflow%20Compiler-3.28.0-brightgreen.svg
    :target: https://hailo.ai/company-overview/contact-us/
    :alt: Hailo Dataflow Compiler
    :width: 180
    :height: 20
 
 
-.. |runtime| image:: https://img.shields.io/badge/HailoRT%20(optional)-4.19.0-brightgreen.svg
+.. |runtime| image:: https://img.shields.io/badge/HailoRT%20(optional)-4.18.0-brightgreen.svg
    :target: https://hailo.ai/company-overview/contact-us/
    :alt: HailoRT
    :width: 170
@@ -57,15 +57,15 @@ The models are divided to:
 
 * Public models - which were trained on publicly available datasets.
 
-  * For Hailo-8 - `Classification <docs/public_models/HAILO8/HAILO8_classification.rst>`_, `Object Detection <docs/public_models/HAILO8/HAILO8_object_detection.rst>`_, `Segmentation <docs/public_models/HAILO8/HAILO8_semantic_segmentation.rst>`_, `other tasks <docs/PUBLIC_MODELS.rst>`_
+    * For Hailo-8 - `Classification <docs/public_models/HAILO8/HAILO8_classification.rst>`_, `Object Detection <docs/public_models/HAILO8/HAILO8_object_detection.rst>`_, `Segmentation <docs/public_models/HAILO8/HAILO8_semantic_segmentation.rst>`_, `other tasks <docs/PUBLIC_MODELS.rst>`_
 
-  * For Hailo-8L - `Classification <docs/public_models/HAILO8L/HAILO8L_classification.rst>`_, `Object Detection <docs/public_models/HAILO8L/HAILO8L_object_detection.rst>`_, `Segmentation <docs/public_models/HAILO8L/HAILO8L_semantic_segmentation.rst>`_, `other tasks <docs/PUBLIC_MODELS.rst>`_
+    * For Hailo-8L - `Classification <docs/public_models/HAILO8L/HAILO8L_classification.rst>`_, `Object Detection <docs/public_models/HAILO8L/HAILO8L_object_detection.rst>`_, `Segmentation <docs/public_models/HAILO8L/HAILO8L_semantic_segmentation.rst>`_, `other tasks <docs/PUBLIC_MODELS.rst>`_
 
-  * For Hailo-15H - `Classification <docs/public_models/HAILO15H/HAILO15H_classification.rst>`_, `Object Detection <docs/public_models/HAILO15H/HAILO15H_object_detection.rst>`_, `Segmentation <docs/public_models/HAILO15H/HAILO15H_semantic_segmentation.rst>`_, `other tasks <docs/PUBLIC_MODELS.rst>`_
+    * For Hailo-15H - `Classification <docs/public_models/HAILO15H/HAILO15H_classification.rst>`_, `Object Detection <docs/public_models/HAILO15H/HAILO15H_object_detection.rst>`_, `Segmentation <docs/public_models/HAILO15H/HAILO15H_semantic_segmentation.rst>`_, `other tasks <docs/PUBLIC_MODELS.rst>`_
 
-  * For Hailo-15M - `Classification <docs/public_models/HAILO15M/HAILO15M_classification.rst>`_, `Object Detection <docs/public_models/HAILO15M/HAILO15M_object_detection.rst>`_, `Segmentation <docs/public_models/HAILO15M/HAILO15M_semantic_segmentation.rst>`_, `other tasks <docs/PUBLIC_MODELS.rst>`_
+    * For Hailo-15M - `Classification <docs/public_models/HAILO15M/HAILO15M_classification.rst>`_, `Object Detection <docs/public_models/HAILO15M/HAILO15M_object_detection.rst>`_, `Segmentation <docs/public_models/HAILO15M/HAILO15M_semantic_segmentation.rst>`_, `other tasks <docs/PUBLIC_MODELS.rst>`_
 
-  * For Hailo-10 - `Classification <docs/public_models/HAILO10/HAILO10_classification.rst>`_, `Object Detection <docs/public_models/HAILO10/HAILO10_object_detection.rst>`_, `Segmentation <docs/public_models/HAILO10/HAILO10_semantic_segmentation.rst>`_, `other tasks <docs/PUBLIC_MODELS.rst>`_
+    * For Hailo-10 - `Classification <docs/public_models/HAILO10/HAILO10_classification.rst>`_, `Object Detection <docs/public_models/HAILO10/HAILO10_object_detection.rst>`_, `Segmentation <docs/public_models/HAILO10/HAILO10_semantic_segmentation.rst>`_, `other tasks <docs/PUBLIC_MODELS.rst>`_
 
 
 

docs/CASCADE.rst

@@ -0,0 +1,30 @@
+Cascade
+===============
+
+| This document provides instructions and basic usage examples for inferring cascade networks.
+
+General
+-------
+
+* *cascade* allows to generate a sequence of models by cascading them one after the other to generate complex deep learning pipeline. For example, 3D object detection in birds-eye-view pipeline such as PETRv2.
+
+* Currently cascade eval API supports PETRv2 only, see ``petrv2_repvggB0.yaml`` for further configurations.
+
+* The user needs existing hars/hefs: both ``petrv2_repvggB0_backbone_pp_800x320`` & ``petrv2_repvggB0_transformer_pp_800x320``.
+
+Evaluation
+----------
+
+To evaluate a cascade model on different targets, use the cascade flag:
+
+.. code-block::
+
+    hailomz cascade eval petrv2
+    hailomz cascade eval petrv2 --override target=emulator
+    hailomz cascade eval petrv2 --override target=hardware
+
+To explore other options use:
+
+.. code-block::
+
+   hailomz cascade eval --help

docs/CHANGELOG.rst

@@ -1,5 +1,38 @@
-Changelog
-=========
+**v2.14**
+
+* Update to use Dataflow Compiler v3.30.0 (`developer-zone <https://hailo.ai/developer-zone/>`_)
+* Update to use HailoRT 4.20.0 (`developer-zone <https://hailo.ai/developer-zone/>`_)
+
+* New cascade API (experimental)
+
+  * Currently supports PETRv2, bird-eye-view network for 3D object detection, see ``petrv2_repvggB0.yaml`` for configurations.
+
+  * The user needs existing hars/hefs: both ``petrv2_repvggB0_backbone_pp_800x320`` & ``petrv2_repvggB0_transformer_pp_800x320``
+
+  * full_precision evaluation: ``hailomz cascade eval petrv2``
+
+  * hardware evaluation: ``hailomz cascade eval petrv2 --override target=hardware``
+
+* New task:
+
+  * Human Action Recognition
+
+    * Added support for (partial) Kinetics-400 dataset
+
+    * Added r3d_18 to support this task
+
+* New Models:
+
+  * `YOLOv11 <https://arxiv.org/pdf/2410.17725>`_ - nano, small, medium, large, x-large - Latest YOLO detectors
+  * `CLIP <https://arxiv.org/pdf/2103.00020>`_ ViT-Large-14-Laion2B - Contrastive Language-Image Pre-training model [H15H and H10H only]
+  * `SWIN <https://arxiv.org/pdf/2103.14030>`_ - tiny, small - Shifted-Windows Transformer based classification model
+  * `DaViT <https://arxiv.org/pdf/2204.03645>`_ - tiny - Dual Attention Vision Transformer classification model [H15H and H10H only]
+  * `LeViT <https://arxiv.org/pdf/2104.01136>`_ - levit128, levit192, levit384 - Transformer based classification model
+  * `EfficientFormer <https://arxiv.org/pdf/2212.08059>`_ - l1 - Transformer based classification model
+  * `Real-ESRGAN <https://arxiv.org/pdf/2107.10833>`_ - x2 - Super Resolution model
+  * `R3D_18 <https://pytorch.org/vision/stable/models.html#video-classification>`_ - r3d_18 - Video Classification network for Human Action Recognition [H8 only]
+
+* Bug fixes
 
 **v2.13**
 

docs/DATA.rst

@@ -1046,7 +1046,9 @@ NUSCENES
 
    .. code-block::
 
-      python hailo_model_zoo/datasets/create_nuscenes_tfrecord.py calib --ann_file <train_annotation_file.pkl>
-      python hailo_model_zoo/datasets/create_nuscenes_tfrecord.py val --ann_file <val_annotation_file.pkl>
+      python hailo_model_zoo/datasets/create_nuscenes_petrv2_cascade_tfrecord.py calib --ann_file <train_annotation_file.pkl> --coords-dir <coords3d_directory_path>
+      python hailo_model_zoo/datasets/create_nuscenes_petrv2_cascade_tfrecord.py val --ann_file <val_annotation_file.pkl> --coords-dir <coords3d_directory_path>
 
-   where <\*_annotation_file.pkl> is the train / val .pkl annotation file generated from the PETR training environment.
+   Where <\*_annotation_file.pkl> is the train / val .pkl annotation file generated from the `PETR training environment <https://github.com/hailo-ai/PETR>`_.
+
+   Notice: In order to benchmark our PETRv2 cascade (petrv2), please download the annotation .pkl file from `here <https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ObjectDetection3d/Detection3d-Nuscenes/petrv2/pretrained/2024-09-30/mmdet3d_nuscenes_30f_infos_val_2024_08_21.pkl>`_ and create a symbolic link (softlink) from */fastdata/data/nuscenes/nuesence/* to your nuscenes dataset folder.

docs/GETTING_STARTED.rst

@@ -9,8 +9,8 @@ System Requirements
 
 * Ubuntu 20.04/22.04, 64 bit (supported also on Windows, under WSL2)
 * Python 3.8/3.9/3.10, including ``pip`` and ``virtualenv``
-* Hailo Dataflow Compiler v3.29.0 (Obtain from `hailo.ai <http://hailo.ai>`_\ )
-* HailoRT 4.19.0 (Obtain from `hailo.ai <http://hailo.ai>`_\ ) - required only for inference on Hailo-8.
+* Hailo Dataflow Compiler v3.30.0 (Obtain from `hailo.ai <http://hailo.ai>`_\ )
+* HailoRT 4.20.0 (Obtain from `hailo.ai <http://hailo.ai>`_\ ) - required only for inference on Hailo-8.
 * The Hailo Model Zoo supports Hailo-8 / Hailo-10H connected via PCIe only.
 * Nvidia’s Pascal/Turing/Ampere GPU architecture (such as Titan X Pascal, GTX 1080 Ti, RTX 2080 Ti, or RTX A4000)
 * GPU driver version 525
@@ -317,13 +317,15 @@ To eval model with input resize, use the resize flag:
 
 Do not use the flag if resize already exist in the alls or in the YAML.
 
+To evaluate a cascade model please refer to `CASCADE <CASCADE.rst>`_.
+
 To explore other options (for example: changing the default batch-size) use:
 
 .. code-block::
 
    hailomz eval --help
 
-* Currently MZ evaluation can be done only on hailo8 and hailo10h.
+* Currently MZ evaluation can be done only on Hailo-8 and Hailo-10H.
 
 Visualization
 -------------

docs/OPTIMIZATION.rst

@@ -68,21 +68,6 @@ Once optimization is finished and met our accuracy requirements, we can compile
 Citations
 ---------
 
-
-.. _Finkelstein2022:
-
-Finkelstein2022
-^^^^^^^^^^^^^^^
-
-.. code-block::
-
-   @InProceedings{Finkelstein2022,
-     title = {QFT: Post-training quantization via fast joint finetuning of all degrees of freedom},
-     author = {Alex Finkelstein and Ella Fuchs and Idan Tal and Mark Grobman and Niv Vosco and Eldad Meller},
-     booktitle = {ECCV},
-     year = {2022}
-   }
-
 .. _Vosco2021:
 
 Vosco2021

docs/YAML.rst

@@ -53,6 +53,7 @@ Properties
   * | **meta_arch** *(['string', 'null'])*\ : Postprocessing meta architecture name.
     | For example: yolo_v3, yolo_v4, etc. Default: ``None``.
   * | **postprocess_config_file** *(['string', 'null'])*\ : Path to a file with the postprocessing configuration (for example, for offloading NMS to the Hailo-8). Default: ``None``.
+  * | **hpp** *(boolean)*\ : Activate the Hailo postprocessing on host. Default: ``False``.
   * | **device_pre_post_layers**\ : Whether to use postprocessing on chip or do it on the host.
 
     * | **bilinear** *(boolean)*\ : Activate the bilinear PPU layer. Default: ``False``.