Dustin Vannoy is a data engineering and analytics consultant experienced in solving business problems with analytics and big data solutions. He is passionate about all aspects of data work, including modeling, building scalable data pipelines, and creating intuitive dashboards. He is experienced in using cloud technologies to transition legacy ETL jobs into streaming pipelines and building out modern data lakes and warehouses.
Dustin is a technical leader in San Diego and the co-founder of the San Diego Data Engineering Group. He now encourages others to grow their data skills by creating tutorials and speaking at user groups and conferences.
Primary technologies: Azure, AWS, Databricks, Spark, Hadoop, Kafka, SQL, Python, and Scala
Contact: dustin@dustinvannoy.com
Professional Services
Streaming & Big Data
Are you ready to make the leap to streaming data or a more modern big data system? I have experience introducing new streaming and big data technologies to organizations and leading successful implementations.
Data Lakes & Warehouses
Do you have analytics needs that are solved by a data warehouse but are ready to evolve the system? I have built multiple data warehouses and data lakes and am experienced transitioning the workloads to cloud services or big data systems.
Leadership
Do you need someone who can mentor others on data engineering? Do you want input on organizational strategy or technical architecture? I am experienced in both and happy to help.
Past Events
Speaking
| Sep 2, 2021 | Free the Data Academy | Streaming Data Using Spark |
| Jul 27, 2021 | Data Collab Lab | Monitoring Spark Streaming with Custom Streaming Query Listener |
| Feb 17, 2021 | Microsoft Data & AI South Florida | Azure Databricks for Stream Processing |
| Dec 3, 2020 | Data Engineering San Diego | Spark Data Pipelines in Azure: Batch and Streaming |
| Nov 12, 2020 | PASS Summit | Building Data Lakes in Azure |
| Jun 13, 2020 | SQL Saturday Los Angeles | Data Lakes with Azure Databricks |
| May 8, 2020 | Data Engineering San Diego | Data Engineer Skills and Traits |
| Nov 20, 2019 | Microsoft Azure + AI Conference | Spark Streaming with Azure Databricks |
| Nov 19, 2019 | Microsoft Azure + AI Conference | Azure Storage Options for Analytics |
| Nov 19, 2019 | Microsoft Azure + AI Conference | Azure Databricks with Delta Lake |
| Nov 6, 2019 | PASS Summit | Azure Storage Options for Analytics |
| Sep 21, 2019 | SQL Saturday – San Diego | Azure Data Lakes and Data Warehouses |