What is Batch and Real Time Streaming Data Process in DataEngineering

Hello Geeks,

This is the introduction video of what is batch and real time streaming data processing.

When to use Batch Processing?
1. When you want to process large amounts of data.
2. Usually in a scheduled or ad-hoc manner
3. Latency could be minutes, hours or days
4. Batch processing is a bit lengthy and is not suitable for data that is time-sensitive
5. Because batch jobs run occasionally, it is a cost saving option

When to use streaming?
1. When you want to process small amount of data in real-time.
2. Data is streaming continuously.
3. Latency must be roughly 1 minute or less.

#data #dataengineer #coding #spark #hastag #llm #datascience #dataengineer #dataanalysis #sql