From Analyst to Analytics Engineer: How 2.5 Years in a Hypergrowth Startup Helped Shape My Career

And why I believe analysts make great analytics engineers

Setting the stage

There is a sweet spot in terms of technical know-how, business acumen and communication skills where most analytics engineers operate. This unique blend of skills is what makes the analytics engineer one of the most impactful roles in today’s data world.

Most analysts already possess two of these three skills: business knowledge and communication. That’s one of the key reasons I believe that analysts are well-positioned to pivot their careers — With a few technical upgrades, they have an incredible opportunity to elevate their roles into analytics engineering.

---

My journey from Analyst to Analytics Engineer

In the world of hypergrowth startups, people are usually required to wear multiple hats, specially when teams are small. This was the case for me when I joined such a startup as a product data analyst. The data team was still small (relative to company size) and there were only a few distinct roles filled, me being the first product analyst in the company.

Imagine me, at my first fast-scaling startup, a bunch of product people bombarding my Slack asking for insights, dashboards, performance analysis… At first, it felt overwhelming! Even more so since most data was coming from completely new sources — most of the data I needed was practically unavailable or unusable in its current form. I found myself often facing a pattern similar to the following:

1. Product dreams up a new feature.

2. Engineering builds it.

3. New data is generated but is stored without analytics in mind.

4. Product wants to know how the new feature is performing, asks analyst (me!).

The frequency of this type of situations led me to realize that I would need to learn how to create my own analytics-driven datasets to fulfill the ever changing requirements of stakeholders.

Before this, I was mostly used to working with “ready-data” — the data sources I would analyse would be shiny tables built by god-knows-who, which were previously tested, polished and fixed up. In the case the data was not in good shape or its current form was unable to fulfill our needs, I would usually spool up my query running tool of choice and end up writing long queries (usually with a bunch of CTEs and joining many tables) where I would transform the data ephemerally into my desired form.

This worked well for ad-hoc analyses and promptly fetching data points, but it quickly falls apart when you ask more of this approach (not scalable, hard to maintain, difficult to share with others, no VCS, etc.).

To say I was intrigued when I learned about DBT and started using it is an understatement. I discovered that my SQL knowledge could now be applied to modeling data with an SQL-esque syntax, and on top of that, we could now easily apply software engineering best practices to data. I was interested in learning more about this tool and the people who use it, which eventually led me to seek a transition into analytics engineering.

---

Key technical skills and tools to make the transition

One of the first gaps I had to fill was in how I approached a task that involved writing SQL. Instead of thinking of the bigger picture and aiming to build an analytical solution that both addressed the business question and was a scalable, reusable asset, I found myself focused on getting to a quick answer. Although this approach is still useful in specific cases, it is not sustainable. I had to shift my mindset from ‘answering questions’ to ‘building solutions’. This shift in mindset was just the beginning. Other skills and tools I had to develop along the way:

• SQL and Data Modeling → Writing SQL with modularity, scalability and maintainability in mind, moving away from one-off queries.

  • Building reusable CTEs for complex querying.

  • Learning database schema designs to structure data for analysis.

  • Window functions and advanced aggregations to build performant datasets.

  • Using date spines for activity models.

• Python/Scripting → Automate repetitive tasks, streamline workflows, write CI/CD scripts.

  • Implementing error handling and logging to improve reliability and troubleshoot issues faster.

  • Ensuring seamless management of dependencies and package consistency across projects.

  • Integrating tools such as Github Actions to handle testing and deployment with version control.

• Analytics Engineering Tools (dbt) → Build, test and manage data transformations to produce tested, reliable and analysis-ready datasets.

  • Learn about materializations (tables, views, incremental) to optimize performance and storage. How, when and why to use each type of materialization.

  • Using best practices when structuring a dbt project (staging, intermediate, etc.).

  • Using the dbt CLI to run and test models, manage versions and debug.

  • Working with yml and markdown files to properly document data models.

• VCS (Version Control Systems) → Working in a collaborative coding environment, managing code changes systematically and ensuring traceability.

  • Understanding branches, commits and pull requests.

  • Learning how to code review (very important).

  • Handling merge conflicts.

• Data Warehousing → Having a centralized repository of data.

  • Understand the concept of data warehousing.

  • Differentiating between available DWHs in the market and what are each’s strengths and weaknesses.

  • Managing costs and performance.

Now, this may seem like a lot to unpack and tackle, but don’t worry — I will now share some practical advice that targets most of these tools and skills through a few key actions, each aiming to help you build your analytics engineering toolbox.

---

Practical Advice for Analysts aspiring to become Analytics Engineers

There is no need to tackle everything at once. By focusing on a few impactful projects/actions, you can start building the confidence and skills to grab the bull by the horns. Here are some practical strategies that address multiple areas in which AEs excel:

• Get hands on experience using dbt:

  • On your own → Search for online datasets from sources such as Kaggle, set up your DBT environment (either locally or dbt cloud, but I would encourage local), load the dataset onto a data warehouse (most offer some form of free tier or trial) and start playing with your data. Go through dbt’s documentation, try out different commands, get acquainted with the tool.

  • At your current job → Pitching a small pilot project for setting up dbt is an incredible way to gain the skills an AE requires. What I absolutely love about this approach is that you will have to learn the WHY of a number of things. You will (hopefully) not be able to say “we need it because everybody else is using it” when pitching it, but rather you will have to convince a set of stakeholders, which will require for you to learn the exact reasons why a dbt-based solution is better, or not, than your current solution. It will also require of you to learn about the “modern data stack”. If dbt is already implemented, then try to involve yourself as much as possible with AEs, offer to take on smaller AE tasks and in general do things that are outside of an analyst’s scope.

  • On the side → If you know of a family or friendly business that is already generating data, but isn’t using it effectively or is not using it at all, offer to help them set up a simple analytics system. Connect their data sources to a data warehouse, then use dbt to model the data into analytics ready tables, then use a simple BI tool that will enable data-driven decision making through dashboarding and reports. It’s important to focus on the business case of the analytics solution and not to simply build for the sake of building. I like to start with the following line of questioning and taking it from there: “What was the last important decision you took? Was your decision backed by data? What could have been better?”.

• Automate a task with Python:

  • Find a small, repetitive task that you encounter regularly and try to automate it. This could be the generation of a recurring report, cleaning a table or data pulls from APIs, among others.

  • Practice connecting to a data warehouse using Python. Use a Jupyter notebook or as a part of a script for automated workflows.

  • Learn about managing Python packaging and dependency management, this is key to having maintainable and consistent Python projects. There are tools, such as Poetry, dedicated to this.

• Participate in an online dbt data-modeling challenge:

  • There are several online data modeling challenges, but one that comes to mind is run by the people at Paradime.io (example). I find they are a great way to learn for a few reasons — First, it gives you a structured and semi-guided approach to building an analytics solution. Second, you can join their slack community and meet other like-minded professionals who are either in the same spot as you or are seasoned AEs from whom you could learn. The people at Paradime also offer to jump on a call with you to answer any questions you might have. And finally, once the competition is done, you can look at other competitor’s work by looking at their submissions, which is great to learn from more experienced AEs.

• Read Analytics Engineering material (like this substack!!):

  • This might seem very straightforward, but I would really recommend reading online material or a related book, such as “Fundamentals of Analytics Engineering” or “Analytics Engineering with SQL and dbt”. Both of these are good reads that will cover key concepts, best practices and real-world use cases.

---

Why Data Analysts make great Analytics Engineers

In my eyes, Data Analysts have a serious head start when it comes to becoming AEs. Technical skills and know-how can be learned with much more ease than communication skills, stakeholder management and business knowledge.

You already know how to take a business problem, understand it, dig up insights and make recommendations. Now, it’s just about adding a layer of technical expertise to build scalable analytics solutions and become the bridge between raw data and impactful business insights.

If you are feeling like you don’t know where to start, drop a comment here or reach out to me, I love chatting about this stuff! Just remember to take it step by step, there is no need to rush — Analytics Engineering won’t be disappearing any time soon, there is plenty of time to make the leap.