Marta Paes and Aljoscha Krettek announce a new release of Apache Flink:
– The community has added support for efficient batch execution in the DataStream API. This is the next major milestone towards achieving a truly unified runtime for both batch and stream processing.
– Kubernetes-based High Availability (HA) was implemented as an alternative to ZooKeeper for highly available production setups.
– The Kafka SQL connector has been extended to work in upsert mode, supported by the ability to handle connector metadata in SQL DDL. Temporal table joins can now also be fully expressed in SQL, no longer depending on the Table API.
– Support for the DataStream API in PyFlink expands its usage to more complex scenarios that require fine-grained control over state and time, and it’s now possible to deploy PyFlink jobs natively on Kubernetes.
Read on for more details on these as well as other changes.