Rewiring My Career: How I Transitioned from Electrical Engineering to Data Engineering | by Loizos Loizou | Nov, 2024

Information is booming. It is available in huge volumes and selection and this explosion comes with a plethora of job alternatives too. Is it value switching to an information profession now? My sincere opinion: completely!

It’s value mentioning that this text comes from an Electrical and Digital Engineer graduate who went all the way in which and spent virtually 8 years in academia studying concerning the Power sector (and after I say all the way in which, I imply from a bachelor diploma to a PhD and postdoc). Although that’s additionally a excessive demand profession within the job market, I made a decision to change to an information engineer path as an alternative.

I consistently come throughout posts in boards and blogs the place individuals from totally different disciplines ask about find out how to change to a profession in information. So this text will take you thru my journey, and the way an engineering graduate has nothing to fret concerning the transition to this new subject. I’ll undergo the marketplace for information jobs, my story, and the talents that engineers have (whether or not it’s electrical, mechanical, digital and many others.) that equip them nicely for this fast paced subject.

As expertise continues to advance exponentially (IoT units, AI, net providers and many others.) so does the quantity of knowledge generated on daily basis. The end result from this? The necessity for AI and Information professionals is presently at an all time excessive and I believe it’s only going to get increased. It’s presently at a stage that the demand for these professionals severely outgrows provide and new job listings are coming out on daily basis.

In line with Dice Tech Job Report, positions like Information Engineers and Information Scientists are amongst the quickest rising tech occupations. The reason is that corporations have lastly come to the conclusion that, with information, you’ll be able to unlock limitless enterprise insights which might reveal their product’s strengths and weaknesses. That’s, if analyzed the right means.

So what does this imply for the longer term information professionals in search of a job? The next ought to be true, not less than for the subsequent few years:

1. Limitless job listings: In line with a latest report by LinkedIn, job postings for AI roles have surged by 119% over the previous two years. Equally, information engineering positions have seen a 98% improve. This highlights the urgency of corporations to rent these sort of professionals.
2. Excessive wage potential: When demand exceeds provide, it instantly results in increased salaries. These are basic legal guidelines of economics. Information professionals at the moment are in an period the place they’ve a number of choices for a job since corporations acknowledge the worth they create to their firm.
3. A number of business alternatives: Take my case for instance. I labored in information for vitality, retail and finance sectors. I think about myself information agnostic, since I’m now in a position to select from alternatives throughout a comparatively wide selection of industries.
4. Future job development: As talked about earlier than, the necessity for these professionals is simply going to get increased since information is available in all sizes and styles and there’s a want for those who know find out how to deal with it.

So right here comes the million greenback query: How can an Engineer, whether or not that’s mechanical, digital, electrical, civil and many others. change to a profession in information? Nice query.

Is it simple? No. Is it value it? Undoubtedly. There’s no appropriate reply for this query. Nevertheless, I can let you know my experiences and you may choose by yourself. I also can let you know the similarities I discovered between my engineering diploma and what I’m doing now. So let’s begin.

A quick story of how I switched to information engineering:

Years 2020–2022

The yr is 2020 and I’m about to complete my PhD. Confused about my choices and what I can do after an extended 4-year PhD (and with a extreme imposter syndrome too), I selected the secure path of academia and a postdoc place at a Analysis and Improvement middle.

While working there, I spotted that I must get out of academia. I now not had the power to learn extra papers, proposals or, much more so, write journal and convention papers to showcase my work. I did all these — I had sufficient. I had like 7–8 journal/convention papers that obtained printed from my PhD and I didn’t actually like the truth that that is the one technique to showcase my work. So, I began in search of a job within the business.

In 2021, I managed to get a job in vitality consulting. And guess what? Extra stories, extra papers and even higher, PowerPoint slides! I felt like my engineering days have been behind me and that I might actually do nothing helpful. After a brief stint at that place, I began in search of jobs once more. One thing with technical challenges and which means that obtained my mind working. That is after I began in search of information professions the place I might use the talents that I acquired all through my profession. Additionally, this was the time that I obtained essentially the most rejections in my life!

Coming from very profitable bachelor and PhD levels, I couldn’t perceive why my expertise weren’t suited to an information place. I used to be making use of to information engineer, analyst and scientist positions however all I acquired was an automatic reply like “Sadly we will’t transfer ahead along with your software”

That’s after I began making use of to actually in all places. So if you’re studying this as a result of you’ll be able to’t make the change, imagine me. I get you.

Years 2022–2023

So, I began making use of in all places to something that even pertains to information. Even to positions that I didn’t have any of the job description expertise. That’s the place the magic occurred.

I obtained an interview from an organization within the retail sector for the place of “Industrial Intelligence Government”. Are you aware what this place is about? No? That’s proper, I didn’t both. All I noticed within the job description was that it required 3–5 years of expertise in Information Science. So I believed, this has one thing to do with information, so why not. I obtained the job and began working there. Seems that “Industrial Intelligence” was a job description that was mainly enterprise intelligence for the industrial division. Fortunate me, it was spot on. It gave me the chance to start out experimenting with enterprise intelligence.

In that place, I used Energy BI at first, for the reason that function was about constructing stories and dashboards. Then, I used to be hungry for extra. I used to be lucky that my supervisor was wonderful so he/she trusted me to do no matter I wished with information. And so I did.

Earlier than I knew it, my engineering expertise have been again. All the issue fixing expertise that I obtained all through the years, the bug for fixing challenges and the publicity to totally different programming languages began connecting with one another. I began constructing automations in Energy BI, then prolonged this to writing SQL to automate extra issues after which constructing information pipelines utilizing Python. In 1 yr’s time, I had all my processes just about automated and I knew that I had the technical functionality to tackle more difficult and technically intensive issues. I constructed unbelievable dashboards that introduced helpful insights to the enterprise house owners and that felt unbelievable.

This was the lightbulb second. That this profession, it doesn’t matter what the information is about, was what I used to be in search of.

Years 2023-present

After one and a half years on the firm, I knew it was time to go for one thing extra technically difficult than simply enterprise intelligence. That’s when a possibility turned out for me for an information engineer place and I took it.

For the previous one and a half years I’ve been working within the finance sector as an information engineer. I expanded my information to extra issues comparable to AI, real-time streaming information pipelines, APIs, automations and a lot extra. Job alternatives are developing on a regular basis and I really feel lucky that I’ve made this change, and I couldn’t advocate it sufficient. Was it difficult? I’ll say that the one difficult half in each BI and information engineering positions was the primary 3 months till I obtained to know the instruments we use and the environments. My engineering experience geared up me nicely to cope with totally different issues with pleasure and do wonderful issues. I wouldn’t change my diploma for the rest. Not even for a Laptop Science diploma. How did my engineering diploma assist all through this transition? That is mentioned within the subsequent part.

How Engineering equips you with expertise that assist in an information profession

So in case you’ve learn this far, you have to be questioning: How is my engineering diploma getting ready me for a profession in information? This man has instructed me nothing about this. You’re proper, let’s get into it.

Engineering levels are vital, not due to the self-discipline however the way in which that they construction the brains of these they research it. That is my private opinion, however going by my engineering levels they’ve uncovered me to so many issues and have ready me to resolve issues in each single bit that I really feel way more assured now. However let’s get to the specifics. These are some key engineering expertise that I see similarities and I get to make use of at my information function each single day:

1. Programming: As {an electrical} and electronics engineer, I obtained publicity to a number of programming languages all through my levels. I used meeting language, Java, VHDL, C and Matlab. Likewise, I believe different engineering disciplines do the identical factor since programming is a technique to carry out simulations in engineering. Although I haven’t used Python or SQL throughout my levels, it was a seamless transition to those two, after getting uncovered to so many issues. I might even say pleasurable, since I used to hate coding throughout my bachelor diploma, however now I like it. It most likely was a matter of tight deadlines and stress from so many issues on the identical time.
2. Downside Fixing: I get to resolve issues on daily basis however as my first college lecturer mentioned to us on the very first day on the college, “Google is your good friend”. In case you have a knack for fixing issues, and you’ve got been uncovered to the way in which engineering tasks are handed out at universities (the place they mainly offer you a one paragraph description for the undertaking and anticipate a product by the top of the week), imagine me you’ll be able to resolve information issues. You’ve gotten been by sufficient preparation.
3. Math and Statistics: Engineering college students get by intense arithmetic comparable to linear algebra, calculus, statistics and others that may make you perceive machine studying in a clean transition. It’s a bit troublesome to understand at first as a result of it’s a brand new territory however you’ll get the cling of it.
4. Black Field Issues: I don’t even know if this can be a formal definition however I think about “Black Field” issues to be those which might be extraordinarily troublesome to resolve, we’ve been utilizing them, they work, however not lots of people really know what’s occurring within the background. In information, the “Black Field Downside” is AI. It’s sizzling, it really works and it’s wonderful however nobody actually is aware of what’s occurring within the background. Equally, engineering disciplines have their very own “Black Field” issues. Positive, AI is troublesome however have you ever tried understanding the facility community downside? That’s no stroll within the park.
5. Modelling and Simulations: Each engineer scholar has been doing modelling and simulations and that’s nothing totally different from ML fashions and information fashions.
6. Information Processing and Analytics: As an engineer scholar in my bachelor and PhD levels I did quite a lot of information processing, transformation and analytics from oscilloscope recordsdata, sensor recordsdata and good units that had tens of millions of rows of knowledge. These are examples of knowledge pipelines as we name them within the information business. I didn’t actually know on the time although that this was the identify for it. Once I obtained to do it in a company atmosphere, these expertise have been transferrable and helped a lot.
7. Automations: Engineers hate repeated procedures. If there’s a technique to automate one thing, they may do it. That is the mindset {that a} information engineer wants. I carried this mindset to my information engineer place and it helps quite a bit since I spend quite a lot of time automating stuff in my each day.
8. Presenting and explaining to non-technical individuals: One quite common factor I used to be doing in my PhD was explaining my undertaking to non-technical individuals in order that they’ll perceive what I’m doing. This occurs quite a bit in information. You put together quite a lot of evaluation for enterprise individuals so you have got to have the ability to clarify it too.

All of the above assist me each single day in my information engineer place. Are you able to see the transferrable expertise now?

While I don’t wish to encourage all of the engineering disciplines to leap into an information place, I nonetheless assume that every one engineers are helpful, I wished to write down this text to encourage the those who wish to do the change. There’s a lot rejection these days however on the identical time alternatives. All you want is the proper alternative after which magic will observe since it is possible for you to to take advantage of your expertise. The vital factor is to maintain making an attempt.

For those who favored this text, please give me a couple of claps and observe me on https://medium.com/@loizosloizou08

There’s extra content material to observe 🙂