Instagram LinkedIn

DUSTIN VANNOY

  • Home
  • Tutorials
    • Data Kickstart
    • Spark
  • Blog
  • About
  • Home
  • Tutorials
    • Data Kickstart
    • Spark
  • Blog
  • About
Why Apache Kafka?

Why Apache Kafka?

As a data engineer, you should not be trying to convince your colleagues that everything can be a scheduled batch job. It's time to learn how to building streaming data pipelines. For many data engineers, Apache Kafka is the go to platform for enabling real-time data pipelines. Let's quickly cover why and how to get started.

Continue Reading
Top Traits of a Data Engineer

Top Traits of a Data Engineer

Data engineer roles vary but some core traits stand out for any data engineer. If you missed it, check out my first posts in this series on What is a Data Engineer? and Data Engineer Skills for Success. Let's finish off this series with the traits I see as most critical for success as a data engineer.

Continue Reading
Data Engineer Skills for Success

Data Engineer Skills for Success

Data engineers job descriptions vary significantly as they are asked to work on many different projects. Yet, there are categories of skills that are consistently desired in a data engineer and serve as a foundation for learning new technologies. Here are the skills I see as most critical for success as a data engineer.

Continue Reading
Journey of a Data Engineer: From BI Developer to Data Engineer

Journey of a Data Engineer: From BI Developer to Data Engineer

This is part 2 of my Journey of a Data Engineer series which all started from the question “What’s the best path to be a great data engineer?” Check out Part…

Continue Reading
Data Lake Introduction

Data Lake Introduction

Hearing a lot of mention of Data Lakes but still not sure what that means or why anyone cares? This video will cover a brief introduction to what a Data Lake is and why so many organizations are adding them to their analytics ecosystem. To show what interacting with a data lake may look like for a typical data analyst, I included a demo of how you would use Spark SQL to query the data lake from Azure Databricks.

Continue Reading
Apache Spark Introduction

Apache Spark Introduction

This video we will quickly cover Apache Spark.  The goal is to cover why use Spark and where it fits in the data ecosystem.  If you want to just get…

Continue Reading
Big Data Kickstart

Big Data Kickstart

Managing big data is critical for many organizations. Analytics can improve products and inform critical business decisions. Using data can provide distinct advantages, and it’s likely that an organization’s competitors are already leveraging their data.

Continue Reading
Azure, Azure Databricks, Data Engineer

Azure Data Lake FAQ

By dustinvannoy Nov 13, 2020 / Leave a comment
Data Engineer

Why Apache Kafka?

By dustinvannoy Nov 10, 2020 / Leave a comment
Data Engineer

Top Traits of a Data Engineer

By dustinvannoy Jul 21, 2020 / Leave a comment
Spark

Spark Summit Takeaways

By dustinvannoy Jun 26, 2020 / Leave a comment
Data Engineer

Data Engineer Skills for Success

By dustinvannoy May 20, 2020 / 2 Comments
Azure

Azure Synapse Analytics: What the WHAT?

By dustinvannoy May 19, 2020 / 2 Comments
Data Engineer

What is a Data Engineer?

By dustinvannoy May 15, 2020 / Leave a comment
Data Engineer

Journey of a Data Engineer: From BI Developer to Data Engineer

By dustinvannoy Apr 26, 2020 / 3 Comments
Data Engineer

Journey of a Data Engineer: From College to BI Developer

By dustinvannoy Apr 7, 2020 / Leave a comment
Data Engineer

Effective Remote Work

By dustinvannoy Mar 10, 2020 / Leave a comment
01 03
About

Dustin Vannoy is a consultant in data analytics and engineering. His specialties are modern data pipelines, data lakes, and data warehouses. He loves to share knowledge with the data science community.

This site is a resource for you to learn about modern data technologies and practices, from kickstart tutorials to blog posts about the latest tips, tricks, and trends.  If you are new to data engineering or data science check out the Data Kickstart tutorials.

Learn more…

Stay informed

Subscribe to get occasional email updates

Thank you for subscribing.

Something went wrong.

Your data will not be sold or shared with others

Follow me on Twitter
My Tweets

Powered by WordPress.com.