QGIS not recognizing HDF4 on macOS

[afitzgerrell]

What is the bug or the crash?

When I add a file (MOD10A1.A2021346.h03v10.061.2021348044717.hdf) to a new project (from a fresh QGIS download/install) of QGIS at 3.16.15, I receive a warning that this "Invalid Data Source: is not a valid or recognized data source.". I'm working on a MacBook Pro. macOS Big Sur at v11.6.2.

Background:
My prior version of QGIS at 3.10 A Coruña is compiled against GDAL/OGR | 2.4.1 and handles MODIS files well, but not HDF5 files. That prompted me to try upgrading to the latest LTR. While I think it's the GDAL version that's leading to A Coruña not accepting HDF5 files, I'm confused why Hannover, compiled against GDAL/OGR | 3.2.1, is not accepting HDF4 since my independent GDAL install (3.2.2) has drivers for both HDF4 and HDF5 files. Checking the GDAL website, and GDAL 3.2.1 should also contain drivers for HDF4.

I'll note that I don't think I'm doing anything incorrectly, but anything's possible! I apologize in advance if this is user-error as opposed to not an actual bug, but I'd appreciate learning what I should be doing differently if that's all it takes.

Many thanks!
amy

Steps to reproduce the issue

Visit Earthdata Search and download the MOD10A1 v61 file MOD10A1.A2021346.h03v10.061.2021348044717.hdf. The following link should bring you to the granule to then download - you'll need to have an Earthdata Login to proceed with downloading the file, however (I'd have attached it here for you but that doesn't look to be allowed):
https://search.earthdata.nasa.gov/search/granules?p=C1646610417-NSIDC_ECS&pg[0][v]=f&pg[0][id]=MOD10A1.A2021346.h03v10.061.2021348044717*&pg[0][gsk]=-start_date&g=G2183651300-NSIDC_ECS&q=mod10a1&tl=1640199283.717!3!!&m=-14.994140625000002!-150.626953125!4!1!0!0%2C2

Add this (either via dragging and dropping or via the Data Source Manager - Raster option) to a freshly installed, empty project, and the red message appears at the top of the application window.

Versions

<style type="text/css"> p, li { white-space: pre-wrap; } </style>

QGIS version	3.16.15-Hannover	QGIS code revision	e7fdad6431
Compiled against Qt	5.14.2	Running against Qt	5.14.2
Compiled against GDAL/OGR	3.2.1	Running against GDAL/OGR	3.2.1
Compiled against GEOS	3.9.1-CAPI-1.14.2	Running against GEOS	3.9.1-CAPI-1.14.2
Compiled against SQLite	3.31.1	Running against SQLite	3.31.1
PostgreSQL Client Version	12.3	SpatiaLite Version	4.3.0a
QWT Version	6.1.4	QScintilla2 Version	2.11.4
Compiled against PROJ	6.3.2	Running against PROJ	Rel. 6.3.2, May 1st, 2020
OS Version	macOS 11.6
Active python plugins	pointsamplingtool; processing; db_manager; MetaSearch

Supported QGIS version

• I'm running a supported QGIS version according to the roadmap.

New profile

• I tried with a new QGIS profile

Additional context

No response

[gioman]

you'll need to have an Earthdata Login to proceed with downloading the file, however (I'd have attached it here for you but that doesn't look to be allowed)

@afitzgerrell it would be better if you could attach/link directly a file. Most of us won't have a Earthdata Login (I don't), thanks!

Are you on a Intel mac or a M1?

[afitzgerrell]

@gioman I'm on an intel mac.

When I try to attach a file, I'm informed "We don't support that type of file type". Is there another avenue available for me to attach a data (.hdf) file?

[gioman]

When I try to attach a file, I'm informed "We don't support that type of file type". Is there another avenue available for me to attach a data (.hdf) file?

@afitzgerrell you can attach ZIP files up to 10mb. If the ZIP is bigger use a Dropbox/Google Drive/etc share link.

[afitzgerrell]

Ah ha! Zip now attached.
Many thanks!

MOD10A1.A2021346.h03v10.061.2021348044717.hdf.zip

[gioman]

Works on Linux and Windows, will try on macOS tomorrow.

[afitzgerrell]

Interesting, thanks @gioman. I have Windows running on a VM, so I downloaded and installed Hannover there. I see it does indeed work well on any type of HDF file (I don't normally work on that machine). I noticed the Windows-running Hannover is compiled with GDAL 3.4.0, but perhaps that's a red herring I'm focusing in on.

Looking forward to see how your testing with macOS goes tomorrow. Thanks!

Copied from the Windows VM Hannover "About QGIS" window:
QGIS version
3.16.15-Hannover
QGIS code revision
e7fdad64
Compiled against Qt
5.15.2
Running against Qt
5.15.2
Compiled against GDAL/OGR
3.4.0
Running against GDAL/OGR
3.4.0
Compiled against GEOS
3.10.0-CAPI-1.16.0
Running against GEOS
3.10.0-CAPI-1.16.0
Compiled against SQLite
3.35.2
Running against SQLite
3.35.2
PostgreSQL Client Version
13.0
SpatiaLite Version
5.0.1
QWT Version
6.1.3
QScintilla2 Version
2.11.5
Compiled against PROJ
8.2.0
Running against PROJ
Rel. 8.2.0, November 1st, 2021
OS Version
Windows Server 2012 R2 Version 6.3 (Build 9600)
Active python plugins
db_manager;
MetaSearch;
processing

[gioman]

Looking forward to see how your testing with macOS goes tomorrow. Thanks!

This is gdalinfo --formats on Windows/Linux

  netCDF -raster,multidimensional raster,vector- (rw+s): Network Common Data Format
  HDF4 -raster,multidimensional raster- (ros): Hierarchical Data Format Release 4
  HDF4Image -raster- (rw+): HDF4 Dataset

On macOS it just returns

netCDF -raster,multidimensional raster,vector- (rw+s): Network Common Data Format

[github-actions]

The QGIS project highly values your report and would love to see it addressed. However, this issue has been left in feedback mode for the last 14 days and is being automatically marked as "stale".
If you would like to continue with this issue, please provide any missing information or answer any open questions. If you could resolve the issue yourself meanwhile, please leave a note for future readers with the same problem and close the issue.
In case you should have any uncertainty, please leave a comment and we will be happy to help you proceed with this issue.
If there is no further activity on this issue, it will be closed in a week.

[afitzgerrell]

Hi,
Per the bot notice, I may have misunderstood or missed something, but I thought I was awaiting feedback on developer testing this issue in MacOS. I am quite interested in pursuing a fix for macOS if warranted, if possible.

[gioman]

I am quite interested in pursuing a fix for macOS if warranted, if possible.

@afitzgerrell see #143 (comment)