Skip to content

Commit

Permalink
Merge branch '52-ajout_interne_fo_pcc' into 'release'
Browse files Browse the repository at this point in the history
Resolve "[interne] Ajout flots optiques+ corrélation de phase"

See merge request 3d/PandoraBox/pandora2d!59
  • Loading branch information
adebardo committed Feb 19, 2024
2 parents aff8399 + cac35bd commit 3084519
Show file tree
Hide file tree
Showing 48 changed files with 2,055 additions and 279 deletions.
2 changes: 1 addition & 1 deletion data_samples/json_conf_files/a_basic_pipeline.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
"invalid_disparity": "NaN"
},
"refinement": {
"refinement_method": "interpolation"
"refinement_method": "optical_flow"
}
}
}
31 changes: 31 additions & 0 deletions data_samples/json_conf_files/an_estimation_pipeline.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
{
"input": {
"left": {
"img": "./maricopa/left.tif",
"nodata": -9999
},
"right": {
"img": "./maricopa/right.tif",
"nodata": -9999
}
},
"pipeline": {
"estimation": {
"estimation_method": "phase_cross_correlation",
"range_row": 5,
"range_col": 5,
"sample_factor": 100
},
"matching_cost": {
"matching_cost_method": "zncc",
"window_size": 5
},
"disparity": {
"disparity_method": "wta",
"invalid_disparity": "NaN"
},
"refinement": {
"refinement_method": "interpolation"
}
}
}
Binary file modified docs/source/Images/Pandora2D_pipeline.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/source/Images/estimation_schema.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/source/Images/img_pipeline/dp_step.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/source/Images/img_pipeline/mc_step.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/source/Images/img_pipeline/refi_step.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/source/Images/optical_flow_schema.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/source/Images/range_schema.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
9 changes: 9 additions & 0 deletions docs/source/exploring_the_field.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
Exploring the field
===================

.. toctree::
:maxdepth: 2

exploring_the_field/initial_disparity.rst
exploring_the_field/refining_disparity.rst

38 changes: 38 additions & 0 deletions docs/source/exploring_the_field/initial_disparity.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
.. _initial_disparity:

Disparity range exploration
===========================

The user is required to set up pandora2d by specifying a range of disparity to be explored.
There are two available methods to do this.

Setting an interval
-------------------

In the configuration file, the user is required to enter disparity range, as a list with two elements each, indicating
the minimum and maximum values for both row and columns disparity.

.. code:: json
:name: Setting disparity ranges example
{
"input":
{
"col_disparity": [-2, 2],
"row_disparity": [-2, 2]
}
}
.. figure:: ../Images/range_schema.png


Setting a range
---------------

In situations where the user does not know the required interval range, an alternative method is provided.
The user should leave the 'disparity_col' and 'disparity_row' parameters empty. Instead, they need to enable an estimation stage in the pipeline. This stage calculates a global shift throughout the image. By using 'range_col' and 'range_row' parameters, the user can then approximate an interval around this determined shift.

The following diagram illustrates how the disparity intervals are initialized using the estimation step:

.. figure:: ../Images/estimation_schema.png
15 changes: 15 additions & 0 deletions docs/source/exploring_the_field/refining_disparity.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
.. _refining disparity:

Refinement step
===============
The purpose of this step is to refine the disparity identified in the previous step.
So, the refinement step involves transforming a pixel disparity map into a sub-pixel disparity map.


Two methods are available in pandora2d:

- Interpolation
- Optical flow.

.. warning::
The optical flow method is still in an experimental phase.
2 changes: 2 additions & 0 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,12 @@ Welcome to Pandora2D's documentation!
:caption: Contents:

getting_started
exploring_the_field
userguide
api_reference/index.rst
developer_guide


.. api_reference is automatically generated by sphinx-autoapi
Expand Down
16 changes: 8 additions & 8 deletions docs/source/userguide/as_an_api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -39,12 +39,6 @@ Pandora2D provides a full python API which can be used to compute disparity maps
}
}

# read images
image_datasets = create_datasets_from_inputs(input_config=image_cfg["input"])

# instantiate Pandora2D Machine
pandora2d_machine = Pandora2DMachine()

# define pipeline configuration
user_pipeline_cfg = {
'pipeline':{
Expand All @@ -57,11 +51,17 @@ Pandora2D provides a full python API which can be used to compute disparity maps
"invalid_disparity": -9999
},
"refinement" : {
"refinement_method" : "interpolation"
"refinement_method" : "optical_flow"
}
}
}

# read images
image_datasets = create_datasets_from_inputs(input_config=image_cfg["input"], estimation_cfg=user_pipeline_cfg.get("estimation"))

# instantiate Pandora2D Machine
pandora2d_machine = Pandora2DMachine()

# check the configurations and sequences steps
pipeline_cfg = check_configuration.check_pipeline_section(user_pipeline_cfg, pandora2d_machine)

Expand Down Expand Up @@ -90,7 +90,7 @@ Images
######

Pandora2D reads the input images before the stereo computation and creates two datasets, one for the left and one for the right
image which contain the data's image, data's mask and additionnal information.
image which contain the data's image and additional information.

Example of an image dataset

Expand Down
4 changes: 2 additions & 2 deletions docs/source/userguide/input.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,12 +33,12 @@ Input section is composed of the following keys:
- Minimal and Maximal disparities for columns
- [int, int]
-
- Yes
- If the estimation step is not present
* - *row_disparity*
- Minimal and Maximal disparities for rows
- [int, int]
-
- Yes
- If the estimation step is not present

Left and Right properties are composed of the following keys:

Expand Down
2 changes: 1 addition & 1 deletion docs/source/userguide/output.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,4 @@ Saved images
Saved configuration
*******************

- ./res/cfg/config.json : the config file used to run Pandora2D
- ./res/cfg/config.json : the config file used to run Pandora2D and estimation information if computed
14 changes: 8 additions & 6 deletions docs/source/userguide/overview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,13 @@ The following interactive diagram highlights all steps available in Pandora2D.
.. image:: ../Images/img_pipeline/arrow.png
:align: center

.. image:: ../Images/img_pipeline/estimation_step.png
:align: center
:target: step_by_step/refinement.html

.. image:: ../Images/img_pipeline/arrow.png
:align: center

.. image:: ../Images/img_pipeline/mc_step.png
:align: center
:target: step_by_step/matching_cost.html
Expand Down Expand Up @@ -44,11 +51,6 @@ The following interactive diagram highlights all steps available in Pandora2D.

<font color="white">forced line break,</font>

.. note::
- Dark red blocks represent mandatory steps.
- Pink blocks represent optional steps.



Configuration file
******************
Expand Down Expand Up @@ -114,7 +116,7 @@ Example
"invalid_disparity": -999
},
"refinement": {
"refinement_method": "interpolation"
"refinement_method": "optical_flow"
}
}
}
Expand Down
8 changes: 5 additions & 3 deletions docs/source/userguide/sequencing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,20 @@ transition defined by the Pandora2D Machine (`transitions <https://github.com/py

Pandora2D Machine defines 3 possible states:
- begin
- assumption
- cost_volumes
- disparity_maps

and 3 transitions, each one corresponding to a step described in :ref:`step_by_step` chapter:
and 4 transitions, each one corresponding to a step described in :ref:`step_by_step` chapter:
- estimation (:ref:`estimation`)
- matching_cost (:ref:`matching_cost`)
- disparity (:ref:`disparity`)
- refinement (:ref:`refinement`)

Pandora2D machine starts at the begin state. To go from one state to another one, transitions are called and triggered
by specific name. It corresponds to the name of Pandora2D steps you can write in configuration file.

The following diagram highligts all states and possible transitions.
The following diagram highlights all states and possible transitions.

.. figure:: ../Images/Pandora2D_pipeline.png

Expand Down Expand Up @@ -63,7 +65,7 @@ interpolation method.
},
"refinement":
{
"refinement_method": "interpolation"
"refinement_method": "optical_flow"
}
}
}
2 changes: 1 addition & 1 deletion docs/source/userguide/step_by_step.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ The following sections describe Pandora2D's stereo matching steps.

.. toctree::


step_by_step/estimation.rst
step_by_step/matching_cost.rst
step_by_step/disparity.rst
step_by_step/refinement.rst
97 changes: 97 additions & 0 deletions docs/source/userguide/step_by_step/estimation.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
.. _estimation:

Estimation computation
=========================

This step aims to calculate a steady shift in both row and column to establish the starting central
point of the disparity intervals that are applied in the matching_cost process.

**Phase cross correlation method**

The phase cross correlation method depends on frequency domain.
It isolates the phase information of cross-correlation from two similar images.

The phase cross correlation algorithm is divided into 4 steps:

- Firstly, we compute the Fourier transform of both the right and left images.
- Secondly, we calculate the cross-correlation between the two Fourier transforms.
- Then, we identify the maximum peak and retrieve a pixel-level shift.
- Finally, for sub-pixel level shifting, we perform an interpolation around this peak.

.. note:: Currently, only the **phase_cross_correlation** method is implemented in Pandora2d.
We use the phase_cross_correlation function from scipy, for further information please see
`Scipy documentation <https://scikit-image.org/docs/stable/api/skimage.registration.html#skimage.registration.phase_cross_correlation>`__


Configuration and parameters
----------------------------
.. warning::

You don't need to set disparities in input section if you set the estimation step

.. list-table:: Parameters
:header-rows: 1


* - Name
- Description
- Type
- Default value
- Available value
- Required
* - *estimation_method*
- estimation measure
- string
-
- "phase_cross_correlation"
- Yes
* - *range_col*
- Exploration around the initial disparity for columns
- int
- 5
- >0, odd number
- No
* - *range_row*
- Exploration around the initial disparity for rows
- int
- 5
- >0, odd number
- No
* - *sample_factor*
- | Upsampling factor.
| Images will be registered to within 1 / upsample_factor of a pixel
- int
- 1
- >= 1
- No


**Example**

.. sourcecode:: text

{
"input" :
{
...
},
"pipeline" :
{
"estimation":
{
"estimation_method": "phase_cross_correlation",
"range_col": 5,
"range_row": 5,
"sample_factor": 20
}
...
}
}


Outputs:
--------

- Showed in log in verbose mode
- Written in the output configuration file
- Stored in the inputs_dataset
Loading

0 comments on commit 3084519

Please sign in to comment.