by Nicole Tache | Released July 2016 | ISBN: 9781491969694

This year, Twitter open sourced two powerful real-time analytics tools -- DistributedLog, a high-performance log service, and Heron, a distributed stream computation system.A few weeks after Heron was open sourced, Karthik Ramasamy, engineering manager and technical lead for real-time analytics at Twitter, delivered a talk at Strata + Hadoop World in London to unveil the system and discuss:An overview of Heron as a micro stream engine and its architectural componentsHow Twitter has been running Heron in productionThe operational experience and challenges of running Heron at scale, including a discussion of stragglersHeron's minimal resource usage and performance numbersLeading up to Twitter's open sourcing of DistributedLog, software engineer and tech lead of the DistributedLog project Sijie Guo spoke at Strata + Hadoop World in San Jose to introduce the service. Key components of his talk include:Why Twitter built DistributedLogTechnical decisions and challenges behind building DistributedLogHow Twitter uses DistributedLog to support different workloadsHow Twitter runs the same software stack in multiple data centers to achieve global consistency Show and hide more Publisher resources View/Submit Errata

1. Building DistributedLog, a high-performance replicated log service - Sijie Guo (Twitter) 00:40:13
2. Processing billions of events in real time with Heron - Karthik Ramasamy (Twitter) 00:48:05
Show and hide more