Update md files

benchmark/multi_reference/deepremaster/index.md

@@ -3,19 +3,24 @@ title: 'Deepremaster: Multi reference'
 layout: default
 tag: deepremaster
 category: multi_reference
+last_modified_at: '2024-03-19 16:00:27.259599'
 ---
 # Multi reference
 ## Deepremaster
 
 This model is originally designed for film colorization.
-
 To run this benchmark the input image is duplicated 5 times.
-
 The reference images are supposed to be colored frames chosen from the movies.
 
 This means that significant differences in the reference images cannot be used, as illustrated below.
 
-Another interesting finding is that the temporal convolution, responsible for homigenizing the color between conscutive frames, learned to color the sky and trees without reference.
+
+An interesting finding is that certain objects are colored even when they don't appear on the refernce images, as long as those colors are present in the reference images.
+This suggests that instead of semantic to semantic matching between gray and reference image, semantic to color correspondence is learned (at least partially).
+For example, the sky is colored blue and the leaves green.
+The semantic matching takes place in feature space where the spatial information is degraded.
+See noise test vs gray test.
+
 
 |  Task | Image #1 | Image #2 | Image #3 | Reference |
 |  ----- | ----- | ----- | ----- | ----- |
@@ -26,10 +31,16 @@ Another interesting finding is that the temporal convolution, responsible for ho
 | Semantic correspondence weak | <img src="{{'/benchmark/multi_reference/deepremaster/semantic_correspondence_weak/0/fortepan_148611_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/multi_reference/deepremaster/semantic_correspondence_weak/0/fortepan_148611_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/multi_reference/deepremaster/semantic_correspondence_weak/1/fortepan_84203_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/multi_reference/deepremaster/semantic_correspondence_weak/1/fortepan_84203_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/multi_reference/deepremaster/semantic_correspondence_weak/2/fortepan_84203_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/multi_reference/deepremaster/semantic_correspondence_weak/2/fortepan_84203_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/input/fortepan_183722.jpg' | relative_url }}" width="200"/> |
 | Distractors | <img src="{{'/benchmark/multi_reference/deepremaster/distractors/0/fortepan_18098_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/multi_reference/deepremaster/distractors/0/fortepan_18098_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/multi_reference/deepremaster/distractors/1/fortepan_276876_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/multi_reference/deepremaster/distractors/1/fortepan_276876_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/multi_reference/deepremaster/distractors/2/fortepan_40115_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/multi_reference/deepremaster/distractors/2/fortepan_40115_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/input/fortepan_183722.jpg' | relative_url }}" width="200"/> |
 
+### Additional Information
+
+- Last updated: {{ "2024-03-19 16:01:17" | date: site.minima.date_format }}
+- Paper: [https://github.com/satoshiiizuka/siggraphasia2019_remastering]()
+
+
 ### Other categories:
 
 {% for p in site.pages %}
 {% if p.tag == "deepremaster" and p.url != page.url %}
 - [{{ p.title }}]({{ p.url | relative_url }})
 {% endif %}
-{% endfor %}
+{% endfor %}

benchmark/single_reference/deepremaster/index.md

@@ -3,19 +3,18 @@ title: 'Deepremaster: Single reference'
 layout: default
 tag: deepremaster
 category: single_reference
+last_modified_at: '2024-03-19 15:58:21.484082'
 ---
 # Single reference
 ## Deepremaster
 
 This model is originally designed for film colorization.
-
 To run this benchmark the input image is duplicated 5 times.
-
 The reference images are supposed to be colored frames chosen from the movies.
 
 This means that significant differences in the reference images cannot be used, as illustrated below.
+('Recolor source' vs other rows)
 
-Another interesting finding is that the temporal convolution, responsible for homigenizing the color between conscutive frames, learned to color the sky and trees without reference.
 
 |  Task | Image #1 | Image #2 | Image #3 | Reference |
 |  ----- | ----- | ----- | ----- | ----- |
@@ -33,11 +32,19 @@ Another interesting finding is that the temporal convolution, responsible for ho
 | Distractors | <img src="{{'/benchmark/single_reference/deepremaster/distractors/0/fortepan_18098_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/deepremaster/distractors/0/fortepan_18098_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/single_reference/deepremaster/distractors/1/fortepan_276876_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/deepremaster/distractors/1/fortepan_276876_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/single_reference/deepremaster/distractors/2/fortepan_40115_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/deepremaster/distractors/2/fortepan_40115_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/input/fortepan_251236.jpg' | relative_url }}" width="200"/> |
 | Random noise | <img src="{{'/benchmark/single_reference/deepremaster/random_noise/0/fortepan_18098_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/deepremaster/random_noise/0/fortepan_18098_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/single_reference/deepremaster/random_noise/1/fortepan_276876_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/deepremaster/random_noise/1/fortepan_276876_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/single_reference/deepremaster/random_noise/2/fortepan_40115_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/deepremaster/random_noise/2/fortepan_40115_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/input/noise.jpg' | relative_url }}" width="200"/> |
 | Random noise | <img src="{{'/benchmark/single_reference/deepremaster/random_noise/3/fortepan_201867_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/deepremaster/random_noise/3/fortepan_201867_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/single_reference/deepremaster/random_noise/4/fortepan_229825_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/deepremaster/random_noise/4/fortepan_229825_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/single_reference/deepremaster/random_noise/5/fortepan_102400_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/deepremaster/random_noise/5/fortepan_102400_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/input/noise.jpg' | relative_url }}" width="200"/> |
+| Gray | <img src="{{'/benchmark/single_reference/deepremaster/gray/0/fortepan_18098_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/deepremaster/gray/0/fortepan_18098_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/single_reference/deepremaster/gray/1/fortepan_276876_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/deepremaster/gray/1/fortepan_276876_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/single_reference/deepremaster/gray/2/fortepan_40115_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/deepremaster/gray/2/fortepan_40115_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/input/fortepan_40115.jpg' | relative_url }}" width="200"/> |
+| Gray | <img src="{{'/benchmark/single_reference/deepremaster/gray/3/fortepan_201867_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/deepremaster/gray/3/fortepan_201867_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/single_reference/deepremaster/gray/4/fortepan_229825_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/deepremaster/gray/4/fortepan_229825_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/single_reference/deepremaster/gray/5/fortepan_102400_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/deepremaster/gray/5/fortepan_102400_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/input/fortepan_102400.jpg' | relative_url }}" width="200"/> |
+
+### Additional Information
+
+- Last updated: {{ "2024-03-19 16:00:27" | date: site.minima.date_format }}
+- Paper: [https://github.com/satoshiiizuka/siggraphasia2019_remastering]()
+
 
 ### Other categories:
 
 {% for p in site.pages %}
 {% if p.tag == "deepremaster" and p.url != page.url %}
 - [{{ p.title }}]({{ p.url | relative_url }})
 {% endif %}
-{% endfor %}
+{% endfor %}

benchmark/single_reference/pdnla_net/index.md

@@ -3,6 +3,7 @@ title: 'Pdnla net: Single reference'
 layout: default
 tag: pdnla_net
 category: single_reference
+last_modified_at: '2024-03-19 16:01:18.739274'
 ---
 # Single reference
 ## Pdnla net
@@ -25,11 +26,19 @@ category: single_reference
 | Distractors | <img src="{{'/benchmark/single_reference/pdnla_net/distractors/0/fortepan_18098_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/pdnla_net/distractors/0/fortepan_18098_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/single_reference/pdnla_net/distractors/1/fortepan_276876_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/pdnla_net/distractors/1/fortepan_276876_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/single_reference/pdnla_net/distractors/2/fortepan_40115_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/pdnla_net/distractors/2/fortepan_40115_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/input/fortepan_251236.jpg' | relative_url }}" width="200"/> |
 | Random noise | <img src="{{'/benchmark/single_reference/pdnla_net/random_noise/0/fortepan_18098_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/pdnla_net/random_noise/0/fortepan_18098_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/single_reference/pdnla_net/random_noise/1/fortepan_276876_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/pdnla_net/random_noise/1/fortepan_276876_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/single_reference/pdnla_net/random_noise/2/fortepan_40115_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/pdnla_net/random_noise/2/fortepan_40115_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/input/noise.jpg' | relative_url }}" width="200"/> |
 | Random noise | <img src="{{'/benchmark/single_reference/pdnla_net/random_noise/3/fortepan_201867_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/pdnla_net/random_noise/3/fortepan_201867_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/single_reference/pdnla_net/random_noise/4/fortepan_229825_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/pdnla_net/random_noise/4/fortepan_229825_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/single_reference/pdnla_net/random_noise/5/fortepan_102400_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/pdnla_net/random_noise/5/fortepan_102400_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/input/noise.jpg' | relative_url }}" width="200"/> |
+| Gray | <img src="{{'/benchmark/single_reference/pdnla_net/gray/0/fortepan_18098_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/pdnla_net/gray/0/fortepan_18098_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/single_reference/pdnla_net/gray/1/fortepan_276876_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/pdnla_net/gray/1/fortepan_276876_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/single_reference/pdnla_net/gray/2/fortepan_40115_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/pdnla_net/gray/2/fortepan_40115_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/input/fortepan_40115.jpg' | relative_url }}" width="200"/> |
+| Gray | <img src="{{'/benchmark/single_reference/pdnla_net/gray/3/fortepan_201867_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/pdnla_net/gray/3/fortepan_201867_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/single_reference/pdnla_net/gray/4/fortepan_229825_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/pdnla_net/gray/4/fortepan_229825_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/single_reference/pdnla_net/gray/5/fortepan_102400_color.jpg' | relative_url }}" width="200"/><img src="{{'/benchmark/single_reference/pdnla_net/gray/5/fortepan_102400_chromaticity.jpg' | relative_url }}" width="200"/> |<img src="{{'/benchmark/input/fortepan_102400.jpg' | relative_url }}" width="200"/> |
+
+### Additional Information
+
+- Last updated: {{ "2024-03-19 16:04:12" | date: site.minima.date_format }}
+- Paper: [https://ieeexplore.ieee.org/abstract/document/10183846]()
+
 
 ### Other categories:
 
 {% for p in site.pages %}
 {% if p.tag == "pdnla_net" and p.url != page.url %}
 - [{{ p.title }}]({{ p.url | relative_url }})
 {% endif %}
-{% endfor %}
+{% endfor %}