Skip to main content Skip to main navigation

Project | SAGE

Duration:
Percipient Storage for Exascale Data-Centric Computing

Percipient Storage for Exascale Data-Centric Computing

Images

Worldwide data volumes are exploding and islands of storage remote from compute will not scale. In scientific research domains such as physics, space sciences, meteorology, genetics and biology, experiments and simulations generate increasingly large data sets. As these scale into the range of exabytes (billions of gigabytes), novel storage, processing, and analytics solutions must be devised to continue deriving insights and innovation in research.

The SAGE project proposes an advanced object based storage solution, termed Percipient Storage, with a very flexible new API enabling applications to achieve Exascale I/O loads exploiting deep I/O hierarchies. The solution will have the capability to run computations on data from any tier – with a homogenous view of data throughout the stack. The SAGE architecture reflects the need for reducing data movement in order to improve energy efficiency, as well as the technology trend towards new non-volatile memory technologies. DFKI will work in cooperation with European partners at Allinea, Bull, CCFE, CEA, Diamond Light Source, FSZ Jülich, KTH, and STFC in this Horizon 2020-funded project lead by Seagate, to meet the requirements of exascale scientific computing.

In the course of the project, DFKI will integrate the advanced data analytics platform Apache Flink with the native object interface together with data-local computations. Apache Flink will benefit from the full performance and features offered by the storage platform, lifting its analytics processing capabilities to the exascale level.

The project has received funding from the European Union’s Horizon2020 Research & Innovation Programme under grant agreement 671500.

Partners

Seagate, Allinea, Bull, CCFE, CEA, Diamond Light Source, FSZ Jülich, KTH, STFC

Publications about the project

  1. Efficient SIMD Vectorization for Hashing in OpenCL

    Tobias Behrens; Viktor Rosenfeld; Jonas Traub; Sebastian Breß; Volker Markl

    In: Michael Böhlen; Reinhard Pichler; Norman May; Erhard Rahm; Shan-Hung Wu; Katja Hose (Hrsg.). Advances in Database Technology — EDBT 2018. International Conference on Extending Database Technology (EDBT-2018), 21th International Conference on Extending Database Technology, March 26-29, Vienna, Austria, Pages 489-492, ISBN 978-3-89318-078-3, OpenProceedings, Konstanz, Germany, 2018.
  2. Efficient k-Means on GPUs

    Clemens Lutz; Sebastian Breß; Tilmann Rabl; Steffen Zeuch; Volker Markl

    In: Proceedings of the 14th International Workshop on Data Management on New Hardware. International Workshop on Data Management on New Hardware (DaMoN-2018), 14th, located at ACM SIGMOD International Conference on Management of Data, June 10-15, Houston, TX, USA, ISBN 978-1-4503-5853-8/18/06, ACM, New York, NY, USA, 2018.

Sponsors

EU - European Union

671500

EU - European Union