Collecting ideas to improve the PCD file format

[mvieth]

Currently, the PCD file format is at version 0.7 (and has been for many years.
This issue represents a collection of ideas for improving the file format. Everyone is welcome to suggest ideas or comment on ideas. It would also be helpful if people comment which ideas are most important in their opinion.
These ideas (or some of them) could some day be implemented in a version 0.8 of the PCD file format.

• binary compressed data: currently, this uses two 32-bit unsigned binary numbers to specify how large the compressed and uncompressed data is. This should instead be two 64-bit unsigned binary numbers, to allow very large datasets. It might make sense to add a new binary_compressed64 mode for this. Additionally, it could be advantageous if the data can be stored in several sections, where the data in each section is spatially close and can be decompressed independently. Then the reader does not have to decompress all the data at once, which might not fit into memory (in case of very large clouds) (related: PCDWriter::writeBinaryCompressed cannot write large pcd file due to 32-bit limitation #2152)
• additional header entries: currently, there is no possibility to store sequence number, timestamp, and frame id (related: [Module IO] savePCDFile() function does not save the PCLheader information of the point cloud #6152)
• it could make sense to make some header entries optional, meaning that it is allowed that the entry does not appear, in which case a certain default value is assumed
• officially declare lines starting with # as comments?
• maybe make 0.8 a superset of 0.7, so that every valid PCD 0.7 file is also a valid PCD 0.8 file?

It would be good if we can create a list of software that can read or write PCD files, so that we can notify them in case of a PCD version 0.8:

• https://github.com/PDAL/PDAL