May 8th, 2015 by Chance

Chance Coble will present “Building an Early Warning System with Spark Streaming” at Data Day Seattle this summer. The data day event has grown to several hundred people that join together for a day of expert talks, panels and workshops on the latest trends, tools and techniques in data. Chance is one of the few experts in data that has deployed a solution with Spark, a cutting edge in-memory distributed processing engine.   “We believe Spark has a big role to play in the future of business analytics and applications.  In this talk we will get to tell a story on that” said Mr. Coble.  His focus on streaming in the talk will provide the audience with insight into building real-time analytics systems, and especially systems that need to recognize patterns and notify the people that would be most interested.   Lynn Bender, who organizes the event, has said that “Spark has been the most requested topic at this event”.  A summary of the talk is below.

Chance Coble

        “We believe Spark has a big role to play in the future of business analytics and applications.  In this talk we will get to tell a story on that”.   -Chance Coble

 

Building an Early Warning System with Spark Streaming

While the lambda architecture has received a great deal of attention as a high scale data design, Spark’s approach to streaming has created a way to unify components for batch and speed into a single code base. In this use case, Chance Coble will describe an early warning system built with Spark Streaming and the value of a single code base to handle business rules in both streaming and batch. The talk will introduce Spark, it’s streaming architecture and will then dive into the actual design, analytic calculations and deployment of the system on AWS and Hadoop. The speaker will walk through the concepts, code, implementation of an early warning system based on statistical learning methods and the results achieved using Spark.