lecture: Towards open, interoperable, and transdisciplinary point clouds for high performance computing
Large point clouds have emerged across a wide range of disciplines, however users and managers face a bewildering range of storage formats, large datasets and convoluted workflows for analysing point clouds alongside other data. Services like OpenTopography and the PDAL toolkit enable point cloud discovery and use, but integration with other earth systems data is not transparently supported.
The Australian National Computational Infrastructure (NCI) hosts 10+PB of research data, predominantly in the realm of Earth Systems. These include extensive point cloud data which need to be discoverable alongside, and interoperable with, substantial collections of geospatial observations and model data using common tools in a High Performance Computing (HPC) and High Performance Data (HPD) environment. NCI has created a National Environmental Research Data Interoperability Platform (NERDIP) to help manage and analyse the data, both locally and remotely using web services which makes use of advanced features in HDF/NetCDF.
We have demonstrated that deploying other geospatial data using a HDF5 model has the potential to directly improve large-scale usage and increase data interoperability between diverse geospatial collections. Models such as the Sensor Independent Point Cloud (SIPC) and SPDLib are based on HDF5. NCI are currently evaluating the use of these formats to aid discovery, extraction and processing using readily available tools, as well as interrogation via web services. The end goal for NCI is making point data discoverable and accessible to end-users in ways which allow seamless interoperability with other datasets and processing techniques.