Status of LOFAR Data in HDF5 Format

Abstract : The Hierarchical Data Format, version 5 (HDF5) is a data model, library, and file format for storing and managing data. It is designed for flexible and efficient I/O and for high volume, complex data. The Low Frequency Array (LOFAR) project is solving the challenge of data size and complexity using HDF5. Most of LOFAR's standard data products will be stored using HDF5; the beam-formed time-series data and transient buffer board data have already transitioned from project-specific binary format to HDF5. We report on our effort to pave the way towards new astronomical data encapsulation using HDF5, which can be used by future ground and space projects. The LOFAR project has formed a collaboration with NRAO, the Virtual Astronomical Observatory (VAO) and the HDF Group to obtain funding for a full-time staff member to work on documenting and developing standards for astronomical data written in HDF5. We hope our effort will enhance HDF5 visibility and usage within the community, specifically for LSST, the SKA pathfinders (ASKAP, MeerKAT, MWA, LWA), and other major new radio telescopes such as EVLA, ALMA, and eMERLIN.
Complete list of metadatas

https://hal-insu.archives-ouvertes.fr/insu-01335837
Contributor : Nathalie Pothier <>
Submitted on : Wednesday, June 22, 2016 - 1:59:01 PM
Last modification on : Thursday, July 25, 2019 - 4:16:02 PM

Identifiers

  • HAL Id : insu-01335837, version 1

Collections

Citation

A. Alexov, P. Schellart, S. ter Veen, M. van den Akker, L. Bahren, et al.. Status of LOFAR Data in HDF5 Format. P. Ballester & D. Egret & N.P.F. Lorente. Astronomical data analysis software and systems XXI: proceedings of a conference held at Marriott Rive Gauche Conference Center, 461, San Francisco: Astronomical Society of the Pacific, pp.283-286, 2012, 9781583818046. ⟨insu-01335837⟩

Share

Metrics

Record views

186