Azure-Data-Factory

I was working at a project that involved munching data generated by multiple data sources, transforming it and making it available for a downstream service that would index this data into a data store that could make it easy to run analytical workloads at run time. stack used: azure-data-factory flink azure-function azure-entra azure-storage-account elasticsearch What were the challenges? The original team that built this workflow used Flink which is an awesome tool for transforming data streams. However this decision involved the development team to manage its Flink installation (it was 2019, the company that I worked for was based out of Azure and at that point of time Azure did not have a managed Flink service). ...