Skip to content

3 April 2023

How I transitioned from academia to Data Engineering

logo
Sarah Mapplebeck

Data Engineers are a diverse bunch, utilising their experiences to provide unique perspectives and insights into common challenges.

Whether you enter the industry through university, a related job role, or through a Bootcamp, you’ll experience different challenges and benefits that your background provides.

Below, Sarah Mapplebeck explores her own journey and dives into how she came to swap the chemistry lab for data engineering.

Identifying my interest

My first experience of using computers to interact with data came from doing a masters in theoretical chemistry.

Having enjoyed this, and finding myself not knowing 100% what I wanted to do as a career, the natural next step seemed to be a PhD.

Following this course, my research involved developing new ways to simulate biological processes and included the use of multiple software packages and programming languages.

Although I loved coming up with scientific theories, the part I enjoyed most was translating those theories into code so that they could be tested.

With this in mind, and a desire for a more stable career than that offered by multiple short term academic positions, I decided to try make the transition into commercial space.

An introduction to Engineering

Speaking to other people that had left academia, as well as reading blogs about what the industry is like to work in, data seemed like the right path to take.

Everything seemed to indicate that it would offer many of the qualities I desired from a career – greater stability, better work-life balance and the opportunity to develop technically and professionally.

To ensure I was certain I wanted to make this transition, I spent my evenings completing relevant online courses to get taste of the type of work I’d eventually be doing. Having enjoyed the courses, I was sure this was the right avenue for me.

Preparing for coding success

My background however meant that the transition would not be perfectly smooth.

Although coding formed part of my PhD, the packages I was using were often very specific and tailored towards use in niche areas of scientific computing, rather than the ones more widely used in the data industry.

I decided to apply for the 3-month data specialism academy with Digital Futures to gain the baseline knowledge I needed to enter the job market. Fortunately, I was accepted onto the course and so, with this newly opened door, my transition was underway.

Through the course we covered the basics of working with data, whether that be in a data analysis, science or engineering role. Then, after sampling each of the disciplines, we were offered more specialised training in whichever area we preferred the most.

For me, that was a toss-up between data science and engineering. In the end engineering came out on top, and looking back now realise I made the right decision.

What I wish I’d known

There are many things a PhD teaches you, both in relation to the topic you are researching and your overall approach to working. In my opinion, the latter of these is of the greatest value if you are perusing any career other than academic research.

Without doubt, the most valuable things I took from my PhD were how to approach learning and the importance of resilience when developing new skills; both of which cannot be underestimated when trying to pursue a career in data engineering from a beginner’s standpoint.

But there were some things I wasn’t prepared for.

The working environments in academic and commercial settings are very different. The main aim of a university research group is to publish papers. Therefore, providing the academic in charge of the group is happy with the rate at which you publish them, the working patterns and styles tend to be extremely flexible.

By contrast, working as part of a project team in a consultancy requires more coordination between team members to meet client expectations, meaning working hours tend to be more conventional. This, and the notion of filling in timesheets to track your percentage billable utilisation is something I have found takes a bit of getting used to.

This working style is not necessarily a downside. I have grown to enjoy having more structure in my life, but it is definitely something that I did not consider before undertaking this career change.

Advice for those looking to transition to Data

Whilst online courses and bootcamps are a fantastic way to start building a foundational knowledge base and prepare yourself for working in the world of data, it is easy for certain things to fall through the cracks.

Aside from the very basics, I didn’t have much experience with Git or writing tests for my code before starting at Hippo. From hearing other people’s experiences, this appears to be a common trap people fall into when entering data via unconventional routes.

Every day involves interacting with shared repos on Git, be that pushing my own code or reviewing that of others. Whichever situations you find yourself in, the code needs to be tested so that we can be sure the work we are doing for clients is right.

This is something that online courses can overlook, focussing mainly on how to manipulate data with little attention given to versioning control or testing. Although they were a great way to start my learning, I wish I knew the importance Git and testing earlier and had spent more time on them before entering the world of work.

It can be quite daunting when first trying to enter the world of tech. There is a huge amount to learn and it’s an industry that keeps evolving.

However, having now done it I can truly say that the initial effort needed to gain the base-level skills is worth it.

It’s ok not to know everything from the get-go, there’s plenty of opportunity to learn along the way and from my experience most people are incredibly supportive if you ask for help.

I never thought it possible to have a career that was so enjoyable and with so much potential to develop, all whilst still being able to maintain a healthy work-life balance!