Search


	Podcast:		Software Engineering Radio - the podcast for professional software developers
	Episode:		SE-Radio Episode 272: Frances Perry on Apache Beam
	Category:		Technology
	Duration:		00:57:42
	Publish Date:		2016-10-25 18:41:15
	Description:		Jeff Meyerson talks with Frances Perry about Apache Beam, a unified batch and stream processing model. Topics include a history of batch and stream processing, from MapReduce to the Lambda Architecture to the more recent Dataflow model, originally defined in a Google paper. Dataflow overcomes the problem of event time skew by using watermarks and other methods discussed between Jeff and Frances. Apache Beam defines a way for users to define their pipelines in a way that is agnostic of the underlying execution engine, similar to how SQL provides a unified language for databases. This seeks to solve the churn and repeated work that has occurred in the rapidly evolving stream processing ecosystem.
	Total Play:		0