Loading…
Data By the Bay has ended
Data By the Bay is the first Data Grid conference matrix with 6 vertical application areas  spanned by multiple horizontal data pipelines, platforms, and algorithms.  We are unifying data science and data engineering, showing what really works to run businesses at scale.
Back To Schedule
Monday, May 16 • 1:10pm - 1:30pm
Akka Streams for Large Scale Data Processing

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

With over 50 million members, Credit Karma is the most utilized and trusted personal finance platform in the U.S. To handle tens of millions of Americans’ credit information, we use Akka Streams for high throughput data transfer. We will discuss how we quickly built a solution using Akka Actors to help us parallelize, parse and send data to our ingestion service. We then used Akka Streams to pull data through our actors based on demand, allowing us to easily control our memory buffers and prevent Out of Memory issues. Akka Streams also allowed us to apply parsing and simple business logic. In this panel, Credit Karma shares best practices on how to implement Akka Streams at scale.

Speakers
avatar for Zack Loebel-Begelman

Zack Loebel-Begelman

Senior Software Engineer, Credit Karma
As a senior software engineer on the data and analytics pipeline, Zack’s work allows Credit Karma to provide tailored recommendations for each individual member’s specific financial situation. Zack joined Credit Karma after two years designing and launching data engines to support... Read More →
avatar for Dustin Lyons

Dustin Lyons

Senior Software Engineer, Credit Karma
Dustin is the technical lead of Credit Karma’s data services, which allows over 60 million members access to a personalized and seamless user experience. Before joining Credit Karma, Dustin worked in product development and infrastructure operations for over nine years. He holds... Read More →



Monday May 16, 2016 1:10pm - 1:30pm PDT
Ada