How Apache Flink Delivers for Deliveroo
Deliveroo is a leading food delivery company that builds tools and services using Apache Flink.
The Flink framework offers a distributed processing engine for “stateful computations over unbounded and bounded data streams.”
Deliveroo has a three-sided marketplace. They have delivery drivers, restaurants, and customers who place orders on the application to get food or groceries delivered to their door. Joining us to discuss using Apache Flink and the Amazon Managed Service for Apache Flink were two engineers from Deliveroo: Felix Angell and Duc Anh Khu.
Deliveroo sought to do more real-time streaming. They explored behaviors to understand the customer journey and how those customers use the Deliveroo application.
That meant modernizing to a more stable platform. The old platform could scale up but not down due to earlier decisions made about the technology. They looked at other services such as Apache Spark and Kafka Streams. But Flink had feature parity with the legacy platform, and large companies used Flink for production.
Deliveroo started experimenting with Flink as a proof-of-concept on Kubernetes. They used third-party operators but found many needed more support and were not maintained. They turned to the Amazon Managed Service for Flink (MSF), which allowed the Deliveroo team to focus on its core responsibilities, such as CI/CD and taking updates to production.
Angell said he would like more Deliveroo teams using Apache Flink. Duc said they move very fast to roll out the latest product. And that means their modeling could be more flexible to adapt to that demand.
Flexibility comes with a cost, he said. Sometimes, you need to remodel things, though other times, you need to normalize the data model. It would help make that process easier for teams to do.
“And for me, one of the features that we would like to see is a self-serve configuration from MSF, so that we can just tweak some of the low-level configuration as well as auto-scaling requirements based on application metrics,” Duc said.
Learn more about Apache Flink and AWS:
Kinesis, Kafka and Amazon Managed Service for Apache Flink
Apache Flink for Real Time Data Analysis
