Distributed Processing for Machine Learning Production Pipelines

Production ML workloads often require very large compute and system resources, which leads to the application of distributed processing on clusters. On premises or cloud-based infrastructure cost requires maximum efficient use of resources. This makes distributed processing pipeline frameworks such as Apache Flink ideal for ML workloads.
In addition, production ML must address issues of modern software methodology, as well as issues unique to ML. Different types of ML have different requirements, often driven by the different data lifecycles and sources of ground truth. Implementations often suffer from limitations in modularity, scalability, and extensibility.
In this talk, we discuss production ML applications and review TensorFlow Extended (TFX), Flink, Apache Beam, and Google experience with ML in production.

[AICamp]

Q&A:
Q: When should we use TFX for input processes and when should we use tf.data? They seem like they serve a similar purpose.
Q: Can you paste quickstart link?