by Tomasz Lelek | Released June 2017 | ISBN: 9781788391528

Analyze data in real-time using the Apache Spark Streaming APIAbout This VideoUnderstand the concepts and problems behind streaming processing.Get to know the Apache Spark Streaming API and create jobs that analyze data in near real time.Analyze traffic in real time using Spark Streaming on a web page while users are browsing it.In DetailSpark is the technology that allows us to perform big data processing in the MapReduce paradigm very rapidly, due to performing the processing in memory without the need for extensive I/O operations.Recently, the streaming approach to processing events in near real time became more widely adopted and more necessary. In this course, you will learn how to handle big amount of unbounded infinite streams of data. You will analyze data and draw conclusions from it. Furthermore, we will look at common problems when processing event streams: sorting, watermarks, deduplication, and keeping state (for example, user sessions). You will also implement streaming processing using Spark Streaming and analyze traffic on a web page in real time. Show and hide more

1. Chapter 1 : Understanding a Spark Streaming
   • The Course Overview 00:02:20
   • Introduction to Spark Streaming API 00:09:14
   • Creating a Project in Spark Streaming 00:05:26
   • Defining Data Source and Data Sink 00:07:58
   • Creating Base for Testing Spark Streaming 00:03:30
2. Chapter 2 : Implementing Stream Processing
   • Handling Unbounded Data 00:02:57
   • Using Event Time and Processing Time 00:02:49
   • Sorting Stream Data 00:05:16
   • Deduplicating Data 00:05:52
3. Chapter 3 : Implementing Transformations and Processing Logic
   • Implementing Job Processing Logic 00:02:49
   • Writing Test for Steaming Job 00:03:06
   • Creating Processing Logic That Needs to Keep State of the User Session 00:04:43
   • Summary of Stream Processing 00:03:39
Show and hide more