Remote Hiring – A Paradigm Shift

Remote Hiring

Remote Hiring


Covid-19 pandemic has pushed many companies beyond the tipping point to adopt remote working as the new normal. Remote working is quite easy and natural for the knowledge workers – especially those working in the field of software development. This article focuses on how remote working has impacted hiring software engineers and the demands that it puts on the hiring tool being used by the companies.

Accountability and transparency

Talent acquisition managers would no longer have their teams of recruiters next to them. They would feel disconnected and helpless due to the lack of visibility. This feeling would be even more acutely felt while working under time pressure. They will, on a daily basis, need to know what their teams of recruiters have been doing. There has to be a proper measure of performance. E.g. if a recruiter gathers a lot of missing information which results in rejection of the candidate, then the recruiter should be get recognized for saving the time of all the stake holders and for preventing a potential bad hire. Companies should use tools that have proper metrics to measure the performance of their remote teams. In addition to the conventional quantitative metrics such as number of candidates sourced, screened and tested, companies should focus on qualitative measures such as average suitability score. The tool must judiciously assign importance to various selection criteria while determining suitability.

Removing bias and getting feedback

Personal biases introduced by likes and dislikes get discovered in face to face interactions. Decisionmakers subconsciously apply corrections to arrive at an unbiased decision. We need to have a mechanism to remove biases and look at one version of truth in a remote team. A tool that has a metric associated with each selection criterion lends itself well for such a mechanism.

Such a tool would also allow companies to assign relative importance to these selection criteria by assigning weightages. The rationale behind assigning numerical weightages to these selection criteria while calculating the suitability score can be shared with the hiring managers. In fact, hiring managers should be encouraged to fine tune the requirements by refining these weightages. Changing weightages for a reason could be a way to capture and convert verbose feedback into meaningful numbers.

Technical Assessment

 Most companies administer technical tests to assess programming skills. Some of the companies are using on-line technical tests. The pandemic situation has made it virtually mandatory to conduct the technical assessments on-line. Technical assessment can be a quick test with a few MCQs or a grueling code sharing and pairing session. Given the time and attention needed by an expert interviewer to do the later, it might make sense to use the former as a filter. Mobile based gamified tech quiz can make it easy and light for the candidates. There is a bit of a resistance from the senior candidates to undergo online tests. One way to get them to agree is to explain the shortage of technical panels to do a deep dive coding session with each and every candidate. Most of them see the merit of a quick and light quiz as compared to a telephonic round by a non-technical or a semi-technical interviewer.

It is highly desirable for the tool to ensure full video proctoring to avoid cheating. Also the questions that get asked should be picked from a large question bank to avoid repetition. Also the tool should automatically delete questions that everyone can answer or those that no one can answer.

Overcommunication and Collaboration

There’s no watercooler or cafeteria in the virtual world. Information that used to flow unhindered through these social interactions needs to flow even more rapidly to compensate for the lack of attention and context.

There are many tools that capture the first level of information. E.g. A candidate was rejected in an interview or another candidate submitted his quiz. However it lacks the next level of detail or the color. Why was the candidate rejected? Was he not as good at SQL server as we had assessed? Did the other candidate score well? How long did he take to complete the quiz?

There are many questions which don’t get asked because of the social and physical distance. E.g. a TA Manager won’t bump into a hiring manager and casually ask how he is finding the quality of candidates? Or there is no opportunity for a recruiter to ask why a candidate was rejected? Companies should acquire a tool that proactively answers such questions. Every piece of information needs to be supported by next level of data- ready to provide the detail should someone ask. The tool must provide a reason why a candidate should be considered for a job. The tool must tell why the candidate who was top ranked suddenly went down. The tool must encourage the hiring managers proper feedback and advice to the rejected candidates.

Companies should acquire a hiring tool that captures important events and alerts all the interested stakeholders using popular channels like WhatsApp and Skype. It’s also important to make these events actionable by providing links from the message to the appropriate page in the application.

There are multiple people collaborating to get work done. The agency sponsoring the candidate, the hiring manager, competing candidates and the recruiter should be alerted when the candidate successfully completes and submits the technical quiz. The interviewee candidate, the recruiter and the agency would be interested when the interviewer submits her feedback. The prolific alerts for all events would keep all the stakeholders abreast of the situation.

Employer Branding

The hiring tool should more than make up for the absence of handshakes and smiles, by providing a world class candidate experience. E.g. even rejected candidates should get proper feedback and valuable career advice. There’s nothing more frustrating than lack of communication that follows the interview. Providing proper feedback with current standing of the candidature in terms of the rank would make your company look very professional and enhance the employer brand value in the larger candidate population.

Conclusion: What has changed and how hiring tools should adapt?

Here are some challenges that hiring tools will need to address

  • Remote interviews have already become the norm. Now companies need to get more information about the candidates from all possible channels to make up for the lack of face to face interaction.
  • Companies need to identify gaps in the information shared by the candidate in her resume and get all such missing information from the candidate
  • Remote hiring is not only about remote interviewing. Companies need to remotely collaborate with all the stake holders including the recruiters, talent acquisition managers, hiring managers, interviewers, vendors and candidates.
  • Our tool must present a consistent view of not only the jobs and applicants – but also the status of each applicant in terms of her suitability, rank and interest to all the stakeholders.
  • Stakeholders should find easy answers to questions like why candidate X is more suitable than candidate Y and what Y should do to improve her ranking. This will help them drive the process to accurately identify the suitable candidates to minimize waste and disappointment.
  • To make up for the lack of warmth afforded by in person interviews, Companies need to go an extra mile to provide not only proper feedback but also some advice to the rejected candidates.

You could request a demonstration of Rezoomex- a hiring tool that promises to address all these challenges



Transparency is the foundation of good candidate experience- Part 3

Shroud of Secrecy

There is a general shroud of secrecy around the interview process , hiring decisions and compensation offered. When you don’t communicate, people tend to lose trust. No wonder a recent survey by Team Blind shows that 70% tech employees don’t trust HR. In Part 2 of this 3 part article we saw the lack of transparency in the early parts of the tech hiring funnel. Viz. Resume completeness, job description and technical screening. In this part we will take a peek at what the candidates think about the end of the funnel- technical interviews , hiring decisions and compensation offered. From the results of the survey, it will become clear that lack of communication leaves the candidates guessing and left to themselves they aren’t kind to the IT companies. Remember talent is the primary driver of revenue and profits in the IT industry.

Candidates impression 1: Buzzwords score well in the technical interviews.

Buzzwords & Presentation score well

Buzzwords & Presentation score well

In the same survey more than 70% candidates said that interviewers do go into details of past experience giving more importance to analytical skills and problem solving abilities. About 44% also say that the frequently face deep dive technical interviews involving hands on coding exercises. But an overwhelming 70% surprisingly felt that companies give more importance to presentation skills and ability to use buzzwords. Companies can easily correct this impression by communicating their rationale behind choosing candidate A over candidate B. This information is never explicitly captured in the hiring process.

Candidates impression 2: Technically weaker developers can do better.

Technically weaker candidates can do better

Technically weaker candidates can do better

There is a feeling among candidates that technical abilities don’t translate into higher pay. It could be because there is no formal measure of technical ability. Every candidate thinks that she is technically better than the others. This impression can be corrected if companies adopt and widely share a metric that allows them to justify why someone deserves better pay as compared to others with the same level of experience. Interviewers’ individual opinions lack objectivity. Often competing candidates aren’t interviewed by the same panels leading to further deterioration of objectivity. A metric of technical ability can help the companies to reach more rational and objective decisions. 

Candidates impression 3: Job hoppers get better deals.

Job hoppers do better than loyal incumbents

Job hoppers do better than loyal incumbents

Some candidates become proficient in their interview skills. There are several resources on the internet like Glassdoor which give fair insight into the types of questions that are likely to be asked for specific jobs in specific companies. As a result those who switch jobs more frequently tend to do better in their interviews. Loyal incumbents lag behind. Companies maintain confidentiality and secrecy about the negotiated and offered compensation. The loyal incumbents get upset when the secret becomes widely known . Companies should realize that employees tend to freely share their compensation numbers among themselves. You can’t expect well performing cohesive teams without this level of informal communication. Companies should reward loyalty by correcting their pay structure to match the levels ruling in the market. Companies can use services like paywatch for effective pay corrections.


There is no doubt that there are many problems with technical hiring that need to be fixed. Lack of transparency reflects general apathy of the IT companies towards what candidates think. There will be a few companies who will see this as an opportunity and fix the problem by becoming more progressive and communicative. Software such as the Rezoomex Assessment and Ranking System will help these companies to measure technical ability and set compensation levels based on hard data to bring objectivity to the decisions. May be these progressive companies will get a headstart over others.

Blockchain and Smart Contracts Will Change the Game From Big Upfront to Lean Experiments.

We often make big upfront commitments , decisions, investments and assume the risk of not achieving the desired outcome.

Screen Shot 2017-08-10 at 4.18.31 AM

We hire a software developer or a team of developers at the beginning of the project. Often we are required to define the technology stack and expected level of complexity up front. We wouldn’t be able to hire the team without doing so. Our choice of technology may turn out to be a bad one. Or we might end up committing to a higher budget by hiring a team with higher than the needed level of seniority. But we can’t go back on our commitment and switch to a better team of developers with appropriate level of seniority and expertise in a new technology.

Welcome to the world of smart contracts and blockchains. In not too distant a future it might be possible to enter into a short 2 week contract with the team. (Please read my earlier post) Each developer gets automatically paid in cryptocurrency as their code passes all the automated tests and is deployed in production. At the end of the sprint we have the freedom to suitably modify the team composition by entering into a new set of smart contracts. This gives us the freedom to experiment and evaluation a few options.

What changed is our ability to enter into smaller , more frequent transactions without the need of a central aggregator such as a Bank or the Corporate HR department. Aggregators can’t economically work with shorter contracts and smaller payments because there is a fixed overhead associated with each contract and payment.

The table below shows a few more examples where such a change can occur.

Big Upfront Lean Experimental
Buying a car Sharing a ride- No aggregator such as Uber or credit card needed.
Buying Servers Virtual servers on the cloud can be rented on pay as you use basis- going forward there will be no need to use credit cards. Cloud provider will automatically receive their payment in cryptocurrency.
Buying heavy machinery Renting made possible by more details about usage, wear and tear and damage if any will get communicated by data streamed by IOT sensors
Launching a company to enter a new business with all the associated overheads Assembling a team loosely bound by smart contracts. Blockchain ensures compliance without the protection of corporate law
Group health insurance- healthy members pay more and those susceptible to health issues pay less. Claiming insurance is fraught with legalese embedded in fine print. Wearable devices communicate the current health status frequently enough that makes it possible for a healthcare provider to directly enter into a smart contract with the individual .

This new way of working has become even more possible because of recent advances in IOT which will enable digitalization of lot of information about the physical world. Broader acceptance of “shared economy” is pushing individuals to “rent & try” instead of “buy & regret”. All this works very well for those who think and work in the lean and agile way. It gives them the freedom to conduct experiments to validate their hypotheses before making a large commitment. It lowers the barrier to entry into businesses which were a territory exclusively reserved for the big corporations.

Resume Ranking using Machine Learning- Implementation.

In an earlier posting we saw how ranking resumes can save a lot of time spent by recruiters and hiring managers in the recruitment process. We also saw that it lends itself well to lean hiring by enabling selection of small batch sizes.

Experiment – Manually Ranking Resumes

We had developed a game for ranking resumes by comparing pairs with some reward for the winner. The game didn’t find the level of acceptance we were expecting it to find. So we thought of getting the ranking done by a human expert. It took half a day for an experienced recruiter to rank 35 resumes. Very often the recruiter asked which attribute was to be given higher weightage? Was it experience or location or communication or compensation?

These questions indicate that every time we judge a candidate by his resume; we assign some weightage to various profile attributes like experience, expected compensation, possible start date etc. Every job opening has its own set of weightages which are implicitly assigned as we try to compare the attributes of a resume with the requirements of the job opening.

So the resume ranking problem essentially reduces to find the weightages for each one the attributes.

Challenge – Training Set for standard ranking algorithms.

There are many algorithms to solve the ranking problem. Most of the ranking algorithms fall under the class of “Supervised Learning” which would need a training set consisting of resumes graded by an expert. As we saw earlier this task is quite difficult as the grade will not only depend on the candidate profile but also on the job requirements. Moreover we can’t afford the luxury of a human expert training the algorithm for every job opening. We have to use data that is easily available without additional efforts. We do have some data of every job opening as hiring managers screen resumes and select some for interview. Its easy to extract this data from any ATS (Applicant Tracking System) . Hence we decided to use “Logistic Regression” that predicts the probability of a candidate being shortlisted based on the available data.

We have seen that “Logistic Regression” forecasts the probability based on weightages for various attributes learned from which resumes were shortlisted or rejected in the past. This probability in our case would indicate if the candidate is suitable or not. We would use this number to rank candidates in descending order of suitability.

Available Data

In our company we had access to data of the following 13 attributes for about 3000 candidates that were screened for about 100 openings over the last 6 months.

1)Current Compensation, 2)Expected Compensation, 3)Education, 4)Specialization, 5)Location , 6)Earliest Start Date, 7)Total Experience, 8)Relevant Experience, 9)Communication, 10)Current Employer, 11)Stability , 12)Education Gap and 13)Work Gap.

We needed to quantify some of these attributes like education, stability , communication etc. We applied our own judgment and converted the textual data to numbers.

Data Cleaning

We were unsure whether we will get consistent results as we were falling short of historical data of resumes. We ignored openings that were barely having 10 or less resumes screened. On the other hand we also discovered a problem with large training sets – particularly in the case of job openings that drag and remain open for long. These job openings are likely to have had change of requirements. As we learned later; consistent accuracy was obtained for job openings having training sets whose population was in the range of 40 to 80 resumes.

Running Logistic Regression

We had listed 22 openings for which several hundred resumes were presented to the hiring managers in the last 6 months. We have record of interviews scheduled based on suitability of the resumes. We decided to use 75% of the available data to train (Training Set) and 25% to test (Test Set) our model. The program was written to produce the following output-

  • Vector of weightages for each one of the 13 attributes
  • Prediction whether the set of test cases would be “Suitable” or “Unsuitable”

The result was based on how accurate was the prediction Accuracy is defined as

Accuracy = (True Positives + True Negatives)/ (Total # of resumes in the Test Set)

Where “True Positives” is the number of suitable resumes correctly predicted to be suitable. Similarly “True Negatives” are the number of unsuitable resumes predicted as such. We achieved average accuracy of 80% ranging from 67% to 95%.

Efforts to improve accuracy

Pay Vs Experience -Plot of Suitable Candidates

Plot of suitable and unsuitable resumes on Experience vs Pay didn’t show any consistent pattern. The suitable resumes tended to be more of highly paid individuals who had lower experience. Which is kind of counterintuitive. Other than this the suitable resumes tended to cluster closer to the center of the graph as compared to the unsuitable ones.

Given the nature of the plot the decision boundary would be non linear- probably a quadratic or higher degree polynomial. We decided to test using a 6th degree polynomial – thus creating 28 attributes from 2 main attributes – viz. experience and pay. We ran the program again this time with these 28 sixth degree polynomial and remaining 11 attributes thus a total of 39 attributes. This improved the accuracy from 80% to 88%. We achieved 100% accuracy for 4 job openings.

Regularization had no impact on accuracy. Hence we didn’t use any cross validation set for testing various values of the regularization parameter.

Values of weightages or parameters varied slightly every time we ran to find the minimum of the cost function. Which indicates that the model found a new minimum in the same vicinity every time we ran the program with no changes to the training set data.

Some Observations

Varying Weightages for Candidate Profile Attributes

If you take a close look at the chart above ; we observe the following-

  • One job opening gives extremely negative weightage to “Current Compensation” – this means that candidates earning well are not suitable. While its just the opposite case for most other job openings.
  • C++ Developer position assigns positive weightage to “Total Experience” but negative weightage to “Relevant Experience”. The requirement was for a broader skillset beyond just C++.

We can go on verifying the reasons for what turns out to be a fairly distributed set of values for weightages for various attributes. Each job opening has pretty much independent assessment of the resumes and candidates.

As expected we observed that accuracy generally increases with sample size or size of the training set.As mentioned earlier accuracy was observed to be low in the case of job openings that remained open for long and the selection criteria underwent change.

Ranking Resumes using Machine Learning

In a recent article we saw how ranking resumes can help us keep the WIP within limit to improve efficiency. We also saw an interesting way of achieving this is by playing a mobile game. In this article we will see how machine learning can be applied to rank resumes.


This article covers a “Quick and Dirty” way to get started. This is no way the ultimate machine learning solution to the resume ranking problem. What I did here took me less than a day of programming. This could serve as an example for students of machine learning.

Problem Formulation

We train the machine learning program by using a “training set” of resumes which are pre-screened by a human expert. The resume ranking problem can be seen as a simple classification problem. We are classifying resumes into suitable (y=1) or unsuitable (y=0).  We know from machine learning theory that classification problems are solved by using the logistic regression algorithm.

Sigmoid for Resume

Sigmoid Function Showing Probability of a Resume being Suitable

We know the predictor function represents a value that lies between 0 and 1 as shown in the diagram above. The predictor or hypothesis function hθ(X) is expressed as-

hθ(X)=1/1+e-Z where z= θTX

where X is a vector of various features like experience(x1), education(x2), skills(x3), expected compensation(x4) etc. which decide if a resume is suitable or not suitable. The first feature x0 is always equal to 1.

Features & Parameters

Features & Parameters

hθ(X) can also be interpreted as the probability of the resume being suitable for  given X and θ. So the resume ranking problem is essentially solved by evaluating the function hθ(X) with the resume yielding highest value of hθ(X) getting the top rank.

With this prior knowledge of machine learning and logistic regression we have to find θ by studying a training set of resumes some of which were selected to be suitable -remaining ones being unsuitable.

Simplification of the problem

To further simplify the problem let us not bother about all the attributes like experience, education , skills, expected compensation, notice period etc. while ranking the resumes. As we saw in this earlier post ; we need to worry only about the top constraints. We selected the top constraints as those constraints which address “must have” features that are “hard to find”. Another benefit of limiting  these top constraints is that the same can be quickly and easily evaluated by the recruiters in short telephonic conversations with the candidates. This makes the process more efficient as it precedes and serves as a filter before the preliminary interview by the technical panel.

Decision Boundary

Training set is a set of resumes that are already known to be suitable or not suitable based on past decisions taken by the recruiters or hiring managers. Let us plot the training set for a particular opening based on past records. For the purpose of this article let us say that resumes are ranked only on the basis of 2 top constraints viz. relevant experience (x1) expressed in number of years and expected gross compensation per month(x2). The plot would look somewhat like what we see below.

Decision Boundary

Decision Boundary


If you draw a 450 line cutting the X1 axis at X1=3, the same can be seen dividing the training set so that every point below the line represents a suitable resume and every point above it represents an unsuitable one. This line is machine learning terms is called the decision boundary. We can say that all the points on this line represent resumes where probability of them being suitable is 0.5.  This is also the point where z=0 as we have seen in the diagram above  showing the sigmoid function-


This equation represents a point on the sigmoid function where

Z=0   – replacing Z with θT X

θT X= 0

-3+X1+X2=0  – represents the Decision Boundary

Gradient Descent 

Though we have visually plotted the decision boundary ; it may not be the best fit for the training set data. To get the best fit we can use gradient descent to minimize the error represented by the following equation-

 J(θ)=-1/m[i=1m y(i)*log(hθ(x(i)) )– (1-y(i))*log(1- hθ(x(i)))]

– where m is the number of instances in the training set and X(i) is a vector representing x0,x1,x2 for the ith instance in the training set of resumes. y(i) takes value 1 if the ith instance was suitable and 0 otherwise. Here we are trying to minimize the function J(θ) by finding out a value of θ that minimizes the error function. Here θ is a vector of θ0, θ1 and θ2.

We can minimize J(θ) by iteratively replacing θ with new values as follows. Each iteration is  step of length α is for descending down the slope till we reach the minimum where the slope is zero.

θj:= θj-α(i=1m(hθ(x(i))- y(i))* x(i))


We wrote the code to execute this in octave – as it’s a known bug-free implementation of machine learning algorithms and vector algebra. There are libraries available in Python and Java to build a more robust “production grade” implementation.

Limitations and roadmap for further work

The logistic regression algorithm is useful only if you have a reasonably large training set – at least 25 to 30 resumes. We also need to have the same selection criteria for the algorithms to work – hence you can’t reuse training sets across different job positions.  There are some “niche” positions where its impossible to find enough resumes- its both difficult and unnecessary to implement machine learning in such cases.

There are many “to-dos” before this program can be made useful. We need to use more features – particularly those which are “Must Have” types. We also need to have more iterations of the gradient descent with different values of  α . Lastly we need to have more resumes in the learning set to be able to further break it down into training set, validation set and test set.


Its particularly challenging to rank 20 or more resumes even though the ranking is based only on 2 or 3 attributes. Recruiters often skip this step as it tends to be tedious and end up wasting a lot of hiring managers’ time. Its an error prone process if a junior recruiter is assigned the task.By automating resume ranking, we hope to avoid human error. We also hope to get early feedback and improved understanding of important attributes or top constraints by limiting the short list to top 3 resumes. Lastly it takes a few seconds for this crude Machine Learning program to rank 20 resumes- something that would take 10 minutes for an experienced recruiter.