The session starts with the fundamentals of reliable data by providing both a retrospective of the database and data warehousing age and how those concepts differ and resonate in the age of data. In the rush to store everything and parallelize all of our data processing, the rigor behind building reliable data systems was lost. The impact of storing any and all data with disregard for why, what, and how you are storing it created a whole set of new problems.
Denny Lee is a Developer Advocate at Databricks. He is a hands-on distributed systems and data sciences engineer with extensive experience developing internet-scale infrastructure, data platforms, and predictive analytics systems for both on-premise and cloud environments. He also has a Masters of Biomedical Informatics from Oregon Health and Sciences University and has architected and implemented powerful data solutions for enterprise Healthcare customers. His current technical focuses include Distributed Systems, Apache Spark, Deep Learning, Machine Learning, and Genomics.