TL;DR

To pose myself as a good software engineer candidate, I need to:

• Demonstrate passion to learn
• Provide more utility to the job position as a senior engineer
• Keep my resume short and relevant
• Tailor my resume for each position that I apply to
• Be mentally prepared for a project with relevancy to the position I’m applying to
• Not forget about the basics of computing (data structure and algorithms)
• Demonstrate Passion for coding

Background

In the past few weeks, I participated in 3 interview sessions to hire more software engineers for developing the Spark ETL pipeline (using Scala) of a product. The new hire would join the Singapore development center that I am part of, which is akin to a boutique: it is not large, but everyone in it is proficient in writing production-ready code (we are developing a software product after all). Thus, the new hire is implicitly expected to have a certain level of coding skills.

However, finding a candidate with skills matching our requirements (knows how to use and troubleshoot Spark, knows how to code in Scala) exactly could be difficult, so we wouldn’t mind tutoring a fresh grad, as long as he or she is driven to learn and can pick up new knowledge quickly. As it turns out:

Passion to learn is extremely important in maintaining competitiveness

Because if a candidate don’t have highly relevant skills that the job requires and none of the other candidates do too, the interviewers fall back to use passion to learn as the sole yardstick besides coding skills.

So here’s the important question that I’m trying to reason in this article: if we can only choose a single candidate, who should we hire?

First disclaimer: I know we have multiple vacancies and I can only provide inputs to my manager, who will ultimately make the hiring decision. However, the question in the title serves as a good thought experiment and ultimately inspired me to write it here to serve as a reminder to myself, and hopefully not being a waste of time to anyone who read it.

Second disclaimer: this article is completely based on my personal experience and does not represent my company at all.

A resume gives the first impressions

Judging by their resumes, all candidates have relevant experience in the Data Engineering field, and all of them claim to know Spark (at least PySpark). In terms of industry experience, candidate A has more than 6 years, candidate B has less than 4 years, and candidate C has a few month (a final year student). The instinctive answer for me at this point is to say: hire B or C, simply because their salary requirements are definitely lower. This instinct seems inhumane at first, but when I gave it another thought, the fully learning point came to me:

If a senior candidate cannot provide more utility to the company, the hiring manager will probably choose a junior candidate instead

In this context, I feel that the word utility would refer to one of these:

• coding and troubleshooting experience in Spark
• proficiency in Scala

Notice that I used the word hiring manager instead of interviewer here, because an interviewer like me should only focus on the technical competencies of a candidate.

The gist of this point is that to ensure that my senior self can still be competitive in the job market, I cannot be complacent at all. I have to keep re-learning in order to stay on top of the trend, e.g. what are the new features in Spark 2.4 and Spark 3.0, and at the same time, I have to keep learning other promising technologies, too, e.g. React and Redux for asynchronous UI development (I have little experience in these advanced JavaScript-based languages, but who knows what will replace them in a few years, like how React sort of replaced AngularJS in the mainstream)

As for the resume itself, I realized this when I tried to read a 10-page word document about a candidate, possibly provided by a headhunter:

A long resume is harder to scroll till the end, so an interviewer may not read beyond 1~2 pages To quote a career coach from my alma mater, “a fresh grad should keep his resume within 1 page. As he gain more experience, the resume can be extended to 2 pages”

The headhunter “helpfully” put a long table of summary highlighting the candidate’s skill set, but it doesn’t feel very useful to me, mainly because it listed a very diverse and sometimes inconsistent (e.g. Spark, Livy, Java co-currency, multi-threading, Option) range of technologies. Frankly speaking though, it could help to pass the HR test.

I continued to scroll down until the candidate’s resume, and quickly read it through. I find that I paid more attention to the first few pages than the rest. This particular candidate spent more than a page at the end to describe the projects he has done, and I merely read the project titles and then stopped reading. The project titles seem to duplicate his working experience and the description is in plain text, not bullet points.

But how to keep the resume short when I have so many things to talk about? To quote a career coach:

Tailor my resume to each position that I’m applying to

Since I know I’m helping to hire a data engineer focusing on Spark, I naturally skipped any projects using Python and TensorFlow, or experience developing iOS apps. So if I were to apply for a job again, I’ll just shorten the description to the experiences that doesn’t fully match the job description. If I become more senior, I’ll probably shorten or remove the extracurricular activities section of my resume.

In all, I find that resume is only a conversation starter rather than a decision influencer, so keeping the resume short and easily readable is more important than its layout design, unless of course, I’m interviewing for a UI/UX position.

Any interview can be technical

We have a coding component in our interview process to assess a candidate’s ability to solve a practical challenge with code, but don’t let your guard down during non-coding portion of the interview! I only attended the non-coding portion of the interview process and it feels equally technical. This portion mainly assesses what this candidate has done in his past and usually starts with one of the recent projects that the candidate has worked on, so:

be mentally prepared for a project with relevancy to the position I’m applying to

Both candidate A and B talked about a project in their current company, while candidate C talked about a pet project of his. In essence, it’s a conversation starter, so as long as the project has a certain degree of complexity (in our case, it should involve a data pipeline instead of merely loading the data into Pandas and running XGBoost on top of it) and I’m very familiar with it, the nature of the project (work or personal) doesn’t matter.

This means at first, interviewers will ask me to illustrate the design of the said project and shoot out design consideration questions based on that. But if I’m familiar with the project, this should be relatively easy.

Then, when it’s my turn to ask questions, I came up with the following on the fly:

To candidate A, I asked:

1. Which operation will cause a full shuffle, df.coalesce or df.repartition
2. Performance benefit of DataFrame over RDD

These questions are specifically targeted at Spark because he indicated that he has experience with Spark on his resume as well as while describing his project. But inability to provide a satisfactory answer is not the end, it simply means that the interviewer will have to fall back to the lowest common yardstick to compare the candidates

To candidate B and C, I asked

• Element lookup time complexity of List and Set
• Describe Merge Sort and Quick Sort and their time complexity

Because they don’t know much about Spark (one of them haven’t even used Java before), I opted to ask these simple data structure and algorithm questions that test one’s basic computer science knowledge. As I’ve previously mentioned, we fall back to use the basics as the yardstick, hence:

A software engineer shall not forget about the basics of computing (data structure and algorithms)

Remembering how to write a lowest common ancestor function on a binary tree won’t be necessary, but the basics of computing shall not be forgotten.

The final decision that surprises no one

Ultimately, based on the interviews, I concluded that all 3 candidates are on similar levels. So how do I make the choice if I were to choose only 1 of them? One character of candidate C made him stand out:

Passion for coding

Throughout his interview, he exudes passion for his pet project and actively engages with us interviewers. That ultimately made the difference.

This decision may not come as a surprise. After all, candidate C is probably the cheapest to hire, too. But beyond salary, the rationale behind this decision still provides enough food for thought to myself, and hopefully to some of the fellow software engineers, too.