Cloud-optimize your scientific data
without copying it
Virtual Zarr gives you cloud-optimized access to archival formats like netCDF and HDF5 without duplicating any data.
The Ecosystem
Tools that bring cloud-native workflows to the archives you already have.
Zarr
The array format underneath all of this. Chunked, compressed, N-dimensional arrays that read directly from object storage.
VirtualiZarr
Create virtual Zarr stores from archival formats with the xarray API. Supports netCDF4, HDF5, FITS, and more.
Icechunk
A transactional storage engine for Zarr. Commit virtual references with version control, time travel, and distributed writes.
One protocol, any format
VirtualiZarr parses HDF5 / netCDF4, HDF4, netCDF3, FITS, Zarr, Kerchunk, DMR++, Icechunk out of the box. Community parsers cover the rest:
Working with NASA data? earthaccess covers search, Earthdata Login, and per-DAAC S3 credentials. earthaccess-auth packages just the login and credential half for services that need nothing else.
Why Virtual Zarr?
Cloud-native access to legacy scientific data, without migrating it.
Faster Processing
Open a year of TEMPO data in 10 minutes instead of about a day. Virtual references allow parallel reads.
No Data Duplication
Virtual datacubes reference the files you already have. Nothing is copied or converted.
Works with Archives
Access netCDF, HDF5, and other legacy formats as if they were cloud-optimized Zarr stores.
Familiar Workflow
Use the xarray and Python tools you already know. Your existing code does not change.
What Virtual Zarr builds on
Decades of work in scientific data formats, remote data access, and computer science fundamentals.
OPeNDAP
Pioneered remote data access and the DMR++ metadata format
HDF Group
Chunk-level access and the foundations of scientific data storage
Kerchunk
Originated the concept of virtual Zarr references
fsspec
Python filesystem abstraction enabling cloud-native access
Contributors
Virtual Zarr is made possible by ASDC, ASF, CarbonPlan, Development Seed, Earthmover, GES DISC, LP DAAC, NASA Earthdata, NSIDC, OB.DAAC, Openscapes, ORNL DAAC, PO.DAAC, and the Data Systems Evolution team at NASA Marshall Space Flight Center.