Catch-22 of Hiring

There is an inherent conflict at the point of basic information acquisition in the process of hiring. The question is how much information should the candidate be required to fill up while uploading his resume? Too much information increases his/her work. On the other hand if minimal information is acquired ; hiring managers are left with a whole lot of resumes and very little information. Its frustrating for hiring managers to read a number of unsuitable resumes before getting one that is suitable.

Catch-22 Situation in Hiring

Catch-22 Situation in Hiring

To elaborate this situation let me take the example of my company. We were getting hardly any interesting resumes from our website. We decided to do away with the lengthy process and made it very simple. Now we have an apply button in front of each opening on the careers page; all that a candidate needs to do is to upload his latest resume. But simplifying this process resulted in a whole lot of resumes being uploaded. Now our HR executives are spending significant amount of their time managing resumes. Our hiring funnel in the chart below shows more than 99% of resumes being filtered out to make less than 1% offers.

Talent Acquisition Funnel

Talent Acquisition Funnel

Should we switch back to our old “elaborate” process? Will the “elaborate” process and form filling ensure that hiring managers get what they want? The reality as we learnt from our experience is quite the opposite. Really interesting candidates don’t bother to go through the ordeal of “registering” and uploading their resumes . And those who do are not really interesting and those who appear to be interesting are just that. They “hype up” their resumes to make themselves appear interesting.

I had this problem on my mind when I attended a day long event  focused on applications of machine learning


Of all the talks I was most inspired by this talk by Nilesh Phadke of BMC Software. He demonstrated applications developed for the IT support – far away from the world of hiring. However I felt that the problem I had on my mind could be solved by applying the machine learning approach.

Information Extraction for Filling up Forms

To automate any workflow ; one needs to enter long forms about entities – be it a support ticket or new candidate for a company. Long forms demotivate users and introduce an element of delay. There also is a tendency to skip non-mandatory fields even though the information is available.

Nilesh demonstrated “Formless Incident Creation” where the user was allowed to type a complaint in one Text Field. As he filled in details, based on the words that he was typing the fuzzy matching algorithm in real time matched the correct entities to those words. Not only did it complete the form needed but also searched and found similar past incidents from a myriad of templates of typical incidents.

Machine Learning Approach to Catch-22 of Hiring

I was immediately reminded of the catch-22 situation in hiring. Most of the information is already present in the resume that is being submitted. Can we not use information extraction algorithm to automate this tedious task? Can we also have different templates for Developers, QA Engineers, IT Support Engineers and Project Managers? Can we use the information extracted to synthesize micro-resumes or short summaries of less than 500 characters to help hiring managers quickly read resumes on their mobile devices?

This gave a new direction of thinking to resolve the catch-22 situation. Here are the set of tools that we plan to use.

Fuzzy Matching – Solr

Search- Lucene

Natural Language Processing – OpenNLP

Analytics for unstructured text-UIMA

Please stay tuned for updates on how we use machine learning for formless resume acquisition and more efficient search by hiring managers.



SMAC Social-Mobile-Analytics-Cloud is more powerful than sum of its parts


Social Media  , Mobile Apps, Big Data Analytics and Cloud computing are four distinct technology areas that are seen rapidly evolving. I had an earlier posting about this Four Way Confluence. There are many products that leverage one of these- but the real power is derived by products that use two or more of these in combination. These areas are very interdependent and in fact consumer grade enterprise applications and apps can only be delivered by using all four in tandem as shown in the video. Industry leaders have coined a short single acronym SMAC to indicate the synergy and cohesiveness of these four areas. Let us examine how leveraging one technology area without the other would be limiting.

Synergy between Big Data back end And Social Media interface

Facebook users population exceeds 1.1 billion. Other social media are behind but all of them are generating huge data in text, photo and video form which can be mined by marketers, researchers , recruiters  and providers of numerous services – only if we have tools to handle such unprecedented volumes and variety of data that is coming in faster than we can process them. Big Data or NoSQL databases like Hadoop, MondoDB and Cassandra use massively parallel processing to deal with Big Data. The explosion of data can be harnessed for proper use by these tools.

Also social media like Facebook, Twitter and LinkedIn use big data analytics when they promote the usage by suggesting friends, trending topics and building social graphs. When I follow someone on twitter ; I instantly get more suggestions which are a result of quickly mining all the tweets for lexical similarities – which is done at lightning speeds by massively parallel processing afforded by big data tools.

Synergy Between Cloud Storage And Big Data back end

Its unimaginable to have large volumes of data without proper management of the same. The need to have backup and disaster recovery process can’t be overemphasized. Cloud offers virtual storage as in the case of Amazon S3, which can grow on demand. No need to worry about installing large hardware and configuring the same as the existing infrastructure deals with data overflow. Cloud computing paradigm allows you to expand or contract storage by merely running a few commands to manage clusters of servers that process big data in massively parallel processing. Cloud based services are now evolving further to offer not only storage but ingestion and processing of big data . Big Data can’t be moved due to its volumes and it needs to be consumed at its source. Ubiquitous nature of cloud storage allows one to capture data near the source no matter where on earth it is.

Synergy between Mobile Apps  And Social Media interface 

More than 40% of social media updates are originating on mobile devices today. In fact there are many people who experienced internet for the first time on their smartphones. Be it pictures or videos or location updates; mobile devices will soon be used as the default way to interact with social media. One can see the reason why all major social media players like Facebook, Google+, Twiter and Linkedin have developed native apps for android and iOS.

Mobile Apps are even more dependent on social media interfaces because users have started moving from email , text messaging and voice calls to Facebook, Twitter, WhatsApp and Hangouts. Telecom providers are now feeling the heat as they see their traditional business under threat and they are monetizing Apps by selling valuable marketing insights by selling data captured by free apps enriched by demographic and geo-location context. Marketers can now sharply focus on their target customers using this information.

Synergy between anytime anywhere Cloud  Service And Mobile Apps 

Besides sharing photos and videos on social media ; mobile users also want to store them for sharing them across multiple devices. Storing them on the cloud provides a mechanism to have the private and shared photos, videos and documents accessible from anywhere at anytime using any device- be it smartphone, laptop or tablet. Cloud storage also serves as a backup in case someone loses his smartphone. Cloud application providers are also providing ways of sharing content selectively on social media. Solutions like GoogleDrive, iCloud, DropBox and SugarSync are already allowing people to upload files from their desktops to view them on their mobile phones and tablets.