No terms found for this post.
With the rapid growth of data, its tooling and its applications, numerous roles have emerged to manage data and derive value from it. One of the more recent roles within data teams is the analytics engineer, a.k.a the bridge between data engineering and data analysis.
In this blog, Analytics Engineer Yirong Lo answers the most frequently asked questions about what it means to be an analytics engineer:
One of the most frequently searched questions is how an analytics engineer differs from a data engineer or a data analyst.
Analytics engineers are data professionals tasked with ensuring that data is structured and transformed in a way that is easily decipherable by analysts, data scientists and/or business users.
Analytics engineers fill the gap between data engineering and data analysis by bringing modern engineering practices to the analytics workflow, which reflects the ever-changing landscape of this industry.
Data is a relatively new field, and therefore does not have a set way of working standards (yet). The complexity required to deliver impactful results has drastically increased. The addition of analytics engineering to this field is a natural evolution that comes with data maturity.
We saw a similar evolution in the role of a software engineer, where one role, i.e. an HTML developer, in the early internet days used to involve building static websites, but now involves backend engineers, frontend engineers, UX designers, QA testers, and others roles required to develop web and mobile applications.
To further illustrate the difference between a data engineer, data analyst and analytics engineer, let’s picture your organization’s data infrastructure as a library:
Data engineers lay the foundations of the library by constructing the physical building, and ensuring that it’s structurally sound and can house an ever-growing, vast collection of books. They are the architects that design and implement the layout, make sure books are arriving, and ensure that the library is equipped with all necessary amenities for visitors, from the entrance to the reading areas.
Analysts and data scientists are the library’s regular visitors, coming almost everyday. As avid readers, they delve deep into the content, gain insights and craft stories/solutions to the world outside. Then there are occasional visitors - business users - who may not always have the time to read in depth, but who value and rely on the library's resources.
Okay, so there’s a library and its various visitors, who could be missing?
The librarians, of course!
In our analogy, this role is filled by analytics engineers. Similar to a librarian organizing, ordering, and governing books, analytics engineers are tech-savvy librarians who use the latest tools to recommend books, summarize stories quickly, and ensure that readers have the right access to the right information.
Analytics engineers transform raw data into usable formats used for analytical purposes. Analytics engineers are curators of the so-called "semantic layer", “gold zone” and/or “data marts” — a unified view on data that provides a consistent and business-relevant perspective on data assets.
The analytics engineer plays an essential role by ensuring that data is reliable and (re-)usable, improving data quality, maintaining trust in data, encouraging collaboration and streamlining data workflows.
This often requires mastery in tooling, such as a data manipulation language (i.e. SQL, Python, R), ELT-related tooling (i.e. Airflow, dbt), data discovery tools (i.e. Power BI, Tableau, DataHub), and a solid understanding of data warehousing (i.e. AWS, Azure, Snowflake). The goal of an analytics engineer is to enable self-service analytics from which analysts, data scientists and other business users can extract valuable insights.
It has become increasingly important to not only make data accessible, but also reliable, (re-)usable and trusted by businesses. This is where the analytics engineer shines.
Some tell-tale signs to recognize that perhaps you need an analytics engineer include:
Many big players have recognized the value of Analytics Engineers and have incorporated the role into their teams. Examples include Meta, Google, Airbnb, Netflix, Shopify, Amazon, eBay, Stripe, Adyen, to name a few.
Xomnia's analytics engineering team help organizations like The City of Amsterdam, HEMA, APG, and others achieve their data-driven potential. Interested in learning more about what an analytics engineer can do for your organization? Contact Xomnia
Let’s be honest, choosing a career is hard, and it is even harder if there are various different titles associated with similar tasks. Some tasks of an analytics engineer are also hidden in other roles, such as that of a data analyst, data engineer or any other type of business insight professionals.
All this aside, here are some reasons to consider becoming an analytics engineer:
As a former data analyst, I often found myself cleaning data, figuring out what table and column names mean, and navigating through data requirements across the business. Oftentimes, the required data was either non-existent or still in the collection phase, leading me to extensive liaising between data engineers and business stakeholders. The constant back-and-forth, along with often unclear governance procedures to secure access to the data meant that the actual analysis - the heart of an analyst role - took a backseat. This impacts the business’ ability to make informed decisions, even when you seemingly have the technology and people to do so. I’ve seen peers and other roles with various titles express similar frustrations. I realized that this gap can be filled by an analytics engineer, which is why I made the transition.
As an analytics engineer I’ve pivoted my focus towards making the entire data analytics workflow more efficient and scaling analytics more effectively. I often reflect and think that my ‘data analyst-self’ would have greatly benefited from collaborating with an analytics engineer. And now, I get to be that person!
No terms found for this post.
We are looking more closely recently at EU cloud providers due to geo-political tensions. See a previous blog for more info. In our research on the capabilities of EU clouds we noticed that the support for so-called analytical databases is mostly la...
We are looking more closely recently at EU cloud providers due to geo-political tensions. See a previous blog for more info. In our research on the capabilities of EU clouds we noticed that the support for so-called analytical databases is mostly la...
A common frustration among expats working at Xomnia is the occasional struggle with the communication style of the Dutch government. When a colleague accidentally missed a deadline on a fine which turned out to be a fine for missing a fine, this “fi...
