Publikation
Apache Flink in current research
Tilmann Rabl; Jonas Traub; Asterios Katsifodimos; Volker Markl
In: IT - information technology (IT), Vol. 58, No. 4, Pages 157-165, De Gruyter, 2016.
Zusammenfassung
Recent trends in data collection and the decreasing prices of storage result in constantly growing amounts of analyzable data. These masses of data cannot easily be processed by traditional database systems as these do not allow for a sufficient degree of scalability. Programs especially designed for parallel data analysis on large-scale distributed systems are required. Developing such programs on clusters of commodity hardware is a complex challenge for even the most experienced system developers. Frameworks such as Apache Hadoop are scalable, but – when compared to SQL – extremely hard to program. The open-source platform Apache Flink is a link between conventional database systems and big data analysis frameworks. Flink is based on a fault tolerant runtime for data stream processing, which manages the distribution of data as well as communications within the cluster. A high diversity of use cases can be supported through various interfaces that allow for the implementation of data analysis processes. In this paper, we present an overview of Apache Flink as well as some current research activities on top of the Apache Flink ecosystem.