Flink timestamps in events & stream records

0
votes

looking at documentation and books of Flink I have doubt about the timestamps: if a stream is set in event-time mode and this means the timestamps have the time of the source before getting into Flink (even before going through a messaging queue, which could be Kafka), why Flink attaches timestamps to records as metadata? slide 3 having the different types of timestamps according to what they account for: https://www.slideshare.net/dataArtisans/apache-flink-training-time-and-watermarks

If the timestamp is coming inside of the event why to pass that value to a metadata of the record? also what would exactly be the difference between the event and the recod?

1
apache-flinkflink-streaming
Those training slides are quite out of date. You'll find more up-to-date content at training.ververica.com. – David Anderson

1 Answers

1
votes

The timestamps don't always come from inside of the events. For example, the Flink Kafka consumer copies the timestamps in the Kafka metadata to the Flink metadata. (You can supply a timestamp assigner if you wish to overwrite these timestamps.)

These timestamps carried in the stream record metadata are used internally in various ways:

  • the built-in event-time window assigners use these timestamps to assign events to windows
  • CEP uses these timestamps to sort the stream into event time order
  • Flink SQL can also use these timestamps for sorting, windowing, etc.