Patterns in Stream Processing: Deduplication
I generally recommend Flink SQL to developers starting a new stream processing project because it includes a powerful collection of built-in operators. These built-in operators can be composed together to satisfy the requirements of most use cases — provided you are able to see how to decompose your use case into a combination of these building blocks.
These built-in operators can do things like transforming, enriching, or correlating events, aggregating them in windows, and looking for patterns. It’s common to perform several of these operations together in one application.
