About

Dustin Vannoy is a data engineering and analytics consultant experienced in solving business problems with analytics and big data solutions. He is passionate about all aspects of data work, including modeling, building scalable data pipelines, and creating intuitive dashboards. He is experienced in using cloud technologies to transition legacy ETL jobs into streaming pipelines and building out modern data lakes and warehouses.

Dustin is a technical leader in San Diego and the co-founder of the San Diego Data Engineering Group. He now encourages others to grow their data skills by creating tutorials and speaking at user groups and conferences.

Primary technologies: Azure, AWS, Databricks, Spark, Hadoop, Kafka, SQL, Python, and Scala

Contact: dustin@dustinvannoy.com

Professional Services

Streaming & Big Data

Are you ready to make the leap to streaming data or a more modern big data system? I have experience introducing new streaming and big data technologies to organizations and leading successful implementations.

Data Lakes & Warehouses

Do you have analytics needs that are solved by a data warehouse but are ready to evolve the system? I have built multiple data warehouses and data lakes and am experienced transitioning the workloads to cloud services or big data systems.

Leadership

Do you need someone who can mentor others on data engineering? Do you want input on organizational strategy or technical architecture? I am experienced in both and happy to help.

Past Events

Speaking

Sep 2, 2021Free the Data AcademyStreaming Data Using Spark
Jul 27, 2021Data Collab LabMonitoring Spark Streaming with Custom Streaming Query Listener
Feb 17, 2021Microsoft Data & AI South FloridaAzure Databricks for Stream Processing
Dec 3, 2020Data Engineering San DiegoSpark Data Pipelines in Azure: Batch and Streaming
Nov 12, 2020PASS SummitBuilding Data Lakes in Azure
Jun 13, 2020SQL Saturday Los AngelesData Lakes with Azure Databricks
May 8, 2020Data Engineering San DiegoData Engineer Skills and Traits
Nov 20, 2019Microsoft Azure + AI ConferenceSpark Streaming with Azure Databricks
Nov 19, 2019Microsoft Azure + AI ConferenceAzure Storage Options for Analytics
Nov 19, 2019Microsoft Azure + AI ConferenceAzure Databricks with Delta Lake
Nov 6, 2019PASS SummitAzure Storage Options for Analytics
Sep 21, 2019SQL Saturday – San DiegoAzure Data Lakes and Data Warehouses