AI spells better times ahead for the subject matter experts

It is commonly believed that AI will empower the employees who are willing to learn – others will find it hard to keep their jobs. But there’s an even bigger opportunity for the subject matter experts. They will not only learn the changes being brought about by AI , but permanently enrich their jobs by becoming the trainers and administrators of the AI apps.

Firstly, let us understand why a human being – no matter whether she is a subject matter expert or not,  is needed to train the modern AI systems.

Human-in-the-loop systems

Humans will be instrumental in building what are known as human-in-the-loop systems.

As per this article by Mothi Venkatesh human-in-the-loop system combines “Supervised Machine Learning” and “Active Learning”. I am reproducing the definition from the article.


Supervised ML, curated (labelled) data sets used by ML experts to train algorithms by adjusting parameters, in order to make accurate predictions for incoming data.

Active Learning, the data is taken, trained, tuned, tested and more data is fed back into the algorithm to make it smarter, more confident, and more accurate. This approach–especially feeding data back into a classifier is called active learning.

It is obvious that we need a human being where the confidence level in the algorithm is low due to insufficient training data or the risk associated with an inaccurate inference is very high. But why would we need a subject matter expert. They are not just another “human” augmenting the algorithm’s artificial intelligence

Subject matter experts as trainers and administrators of AI

This article by Daniel Faggella explains the need for subject matter experts. I am reproducing three ways as written in the article.

  • Determining the Business Problem an AI Solution Can Solve. For example, let’s say an insurancecompany wants to become more competitive. A high-level employee at the company, who has been in the industry for more than 20 years, may know from experience that clients often complain about the time it takes to process claims.
  • Determining How AI Can Solve That Business Problem. For example, a marketermight tell a data scientist to make sure the model they build can predict the best price to bid for ad space on Google ad Network without realizing that such an application is nascent in practice and technically challenging. A data scientist would be able to explain this to the marketer, preventing them from promising their team and stakeholders too much.
  • Maintaining and Updating the System Once It’s Built. Take for example a manufacturingcompany with a large number of heavy machinery to maintain. The subject-matter experts collaborate with data scientists to develop an AI system that gathers data from IoT sensors on the machines to predict the optimal time to do preventive maintenance on them.


To understand this better we need to take a closer look at how the AI apps work.

  1. Every AI app needs training data that needs to be prepared by a human expert. E.g. we need to have a lot of data about the films and the people who watch those films to intelligently recommend a bunch of films to a new user.
  2. After preparing the training data a data scientist could use this data to train and test an algorithm. He will repeat this step till he finds a suitable algorithm to generate a model that can predict or infer as precisely as the human expert who has prepared the training data in step 1. This step could take several iterations.
  3. Once the algorithm which can most precisely predict , the same can be used to create a model. This model is handed over to the subject matter expert to be used for predictions or inferences.
  4. The SME uses this model for day to day predictions and inferences. He applies his own judgement to decide if the predictions and inferences are precise enough by comparing what the algorithm has predicted or inferred with what the SME would have done.
  5. If the model is not accurately inferring or predicting the SME would override the algorithm and manually assign predicted or inferred values.
  6. This edited data is added as new input data and the same is used to retrain the algorithm. This retraining will ensure better results going forward. Step 4 to 6 are repeated by the SME with no intervention from the data scientist. The SME would really make the application more and more precise. As it becomes more precise it will more closely mimic the expert’s inferences and predictions.

Who can be a subject matter expert.

You don’t have to be highly placed or academically well qualified. There’s nothing better than years of expertise. E.g. a security guard knows how to spot an intruder in a crowd. He can help a data scientist build an automated threat detection system



Transparency is the foundation of good candidate experience – Part 2

In part 1 of this article we saw that candidates don’t get any constructive feedback in the assessment and interview process. The real problem is not the willingness of the companies. It’s the ability to quickly and efficiently get the information needed to give the feedback. There are 2 problems.

1) General lack of information due to non availability of data or because raw data is left unprocessed.

2) Communication gap between HR, Hiring managers, Clients, BU Heads , Technical Leads and other stakeholders

Non availability of data

It starts with the applicants providing their information in varied levels of detail. Some are sketchy 10000 feet view of decades of experience with no details of time period or job switches. Others are way too detailed giving more information about their employers and projects than the exact roles and responsibilities. This variety makes apples to apples comparison very hard. Do companies bother to get the missing information from the applicants? Some of them do- more of them don’t. That’s exactly what a recent survey discovered


Raw data is left unprocessed

A job description provides broad guidelines. It’s also subject to interpretation. Lastly as the name indicates , it’s descriptive and lacks objectivity. This raw data needs to be processed to arrive at the exact selection criteria and the importance to be associated with each of them- preferably using metrics to measure each criterion and a weight indicating its importance. Do companies take the effort? Do they communicate the outcome to all the stakeholders? Most of them don’t as indicated by our survey administered to a representative sample of candidates. There seems to be a communication gap – which is quite apparent to the candidates.


Tendency to share just enough information

After answering all the technical questions, it’s candidate’s right to know whether her answers were right or wrong. Assessment and interview process is a two way communication. Interviewers who can confidently show the candidates their mistakes and share the right answers draw a lot of respect from the candidates. These technical discussions become more meaty when information isn’t withheld. In fact these are low hanging brand building fruits for companies to pick. Unfortunately most of them don’t as indicated by our survey.


Communication gap and lack of co-ordination

As it might be obvious- there are many participants in the hiring process. They aren’t from the same department or team. In fact every job opening has its own set of stakeholders and each one approaches the problem from a different angle. It’s a team with an ambiguous mandate and no clear owner. Each one has a different approach to hiring. The client wants results – he doesn’t care how. The hiring manager can’t tell the selection criteria , but she has an ostensive job definition. The HR manager is basing the job description on what the hiring manager seems to want. The BU head has his own agenda of building high performing teams and the dev lead who is involved in the day to day grunt work wants the candidate to jumpstart with minimal hand holding.



There seems to be a need for a quantifiable metric that will measure a candidate’s suitability for a job. To arrive at it we will require a set of objective selection criteria and weights depicting the importance of respective selection criteria. This will provide a universal understanding of what is required. It will also provide a mechanism to do apples to apples comparison between candidates. Coordination between various stakeholders will improve as they would speak the same language and recruiters would be able to justify their recommendations with confidence. Readers may want to check the rezoomex assessment and ranking system which provides a framework to measure suitability.

Toolset used by the IT Recruiters

A panelist in a recent conference said that IT recruiters are overwhelmed by the number of software tools that they are required to use. They have to perform many tasks even after sourcing the required resumes . An earlier post covered various sources from where they source the talent.

This article is based on a survey that explores variety of tools used by technical recruiters for various tasks after sourcing a bunch of resumes. Total of 43 respondents who worked as recruiters in medium and large IT companies in Pune participated in the survey.

Technical Assessment

Technical assessment has always been a challenge for the IT recruiters because of their non-technical background. Tools like Hacker Rank, Hacker Earth, Mettl, Reliscope and Glider are being used by one third of the recruiters to overcome this challenge. Among all these Hacker Rank seems to lead the pack. Wonder what is keeping any of these tools from widespread adoption. May be it’s the time and effort to create, administer and analyse results of these tests. May be it’s sheer inertia. Total cost per use is non-trivial. These tests need continuous updates as new technologies get added every day.

Technical Assessment Tools

Technical Assessment Tools

There is no established software solution to conduct technical assessment. Two thirds of the organizations depend on in house on line or off line tools. Hacker Rank undoubtedly leads the branded technical assessment tools lot.

Applicant Tracking Systems (ATS)

 Applicant tracking solutions are by far most used special purpose tools. Here the adoption is close to 50%. There are many ATSs providing varied degree of automation at number of price points. Many enterprises already use Oracle and SAP resulting in their natural affinity towards Taleo and Success Factors respectively. I will be surprised to find a company using the ATS without the ERP. It’s surprising that none of the respondents mentioned the Naukri RMS in spite of it being integrated with the most used source of resumes.


Applicant Tracking System

Applicant Tracking System

The respondents are evenly divided between branded ATSs and other means to track. Taleo and Success Factors tie for the first place in branded ATSs.

Personalised Log of transactions or Tracker

There is no personalised database solution available for the recruiters to maintain their own pipeline of current and past candidates sourced by them for various openings. Excel is the default choice – easy to use and ubiquitous. Looking at 80% adoption there is no reason for any competing solution to try to dislodge Excel. Recruiters are so additced to Excel that they are maintaining their trackers even though the entire log of all the transactions is automatically tracked by the ATS!

Personal Database (Tracker)

Personal Database (Tracker)

It’s a surprise that 100% don’t use excel. Wonder whether the others maintain their personal database.

Interview Scheduling Tools 

Microsoft Outlook and Google calendar are the leading scheduling tools used by the recruiters accounting for more than 91% of adoption. It’s logical that most ATSs provide integration with these 2 scheduling tools. The vCal and iCal format of calendar invites are becoming the de-facto standards.

Interview Scheduling Tools

Interview Scheduling Tools

90% of the respondents use either Outlook or Google Calendar to schedule interviews. Some years back Outlook had even bigger share.

Remote Interviewing Tools 

What about “Specialised” video interviewing tools like InterviewMocha or Talview? There seems to be no significant adoption. Recruiters are still using Skype and phones.

Remote Interviewing Tools

Remote Interviewing Tools

Skype is the winner with 55% respondents using it as the preferred way to conduct remote interviews – landline calling comes distant second.

We love automation but we hate the loss of control 

Wonder why recruiters are maintaining trackers in Excel and schedule interviews using Outlook or Google Calendar when the entire history of their transacations is maintained and automated alerts are given by the ATS? Is it because old habits die hard? The answer is partly that and partly the fact that automation is a black box. People want to be doubly sure that the interviews happen as scheduled and they don’t lose the history including their comments. It’s the same reason why we carry cash even though we hardly ever use it. I am sure for the same reason we will have our hands on the steering wheels and our feet on the pedals when we get to “drive” our autonomous cars.

Build ontologies, let your chatbots socialize!

It’s 2018 and companies need to improve their understanding of customers, vendors, employees and investors. Companies need many more ears and mouths to absorb and disseminate knowledge. Machines engaging with humans in productive conversations using Natural Laguage Processing is no more a dream as well demonstrated by Siri, Alexa and Google-Home! Why is this so important? Obviously because it would empower companies to do what they need to do this year. But before we get to it , let us pull back and take a look at what makes knowledge transfer hard.

Tacit and Explicit knowledge

Tacit knowledge is what we have internalized and that is what we really use when we make a number of decisions in our personal and professional life. A pilot uses his tacit knowledge and has no use for explicit knowledge such as an instruction manual while flying a large passenger aircraft. Doctors, lawyers, architects and soldiers owe their tacit knowledge to years of training alongside their seniors. This tacit knowledge transfer happens while socializing as per Alex Pentland in his famous book Social Physics

Explicit knowledge is what we externalize and express in English or some other language in articles, books, videos, podcasts, databases and spreadsheets etc.

Nonaka and Takeuchi have developed the SECI model for organizations to grow their knowledge-base by cyclically going through the following 4 steps that looks like an ever expanding spiral

  1. Socialization ( Tacit to Tacit)
  2. Externalization (Tacit to Explicit)
  3. Combination (Explicit to Explicit)
  4. Internalization (Explicit to Tacit)


  • SECI

    SECI Model of knowledge transfer

    Tacit knowledge is hard to externalize. No one can communicate all he/she knows by writing it down. On the contrary socialization is easy. It happens easily because of the social setting which provides the context. Human brain snacks on information that is in context. On the contrary most of the communication directed at you on the internet is sheer noise due to lack of context. Wouldn’t it be wonderful if chatbots storing a lot of explicit knowledge could use it in a virtually social setting to engage in conversations with us?

    Let chatbots do the hard work of externalization

    Imagine having a friend by your side who has the patience to answer all your questions that you need to be answered to understand and participate in a conversation. All of us were very inquisitive as children. We stopped asking questions when we saw adults around us losing patience.

    Chatbots can simultaneously engage in thousands of conversations. Socializing with chatbots makes it easy for us to weave bits and pieces of knowledge in the right context. This knowledge is provided accurately in the right context by a chatbot that is lurking the background while we are going about our daily routine of using our enterprise software or our favorite app. Explicit knowledge is internalized by a chatbot in a few seconds. But for chatbots to learn new pieces of knowledge , they need to ask questions and plug incoming pieces of information in the right place. Ontologies provide the underlying infrastructure for chatbots to organize knowledge – somewhat parallel to the way humans learn by organizing related facts in the same part of the brain.

    Building an ontology made easy!

    An ontology is a little more than a taxonomy which is a tree like structure that organizes entities in classes, subclasses and instances. In addition it also establishes connections between leaves of the tree by specifying one way, two way and transient relationships. Its best represented in a graph database such as neo4j. Organizing the knowledge specific to your company’s business domain in a graph database is a daunting task. Many companies have spent years building ontologies. No doubt that it’s an effort that more than pays for itself- but the sheer volume of work deters companies from investing in this project.

    Synerzip has developed a semi-automated way of building ontologies. The system takes documents relevant to your company’s domain – which may be catalogues, brochures, articles, emails, spreadsheets, resumes or handbooks and puts them in a corpus. Top few hundred the frequently occurring terms in the corpus are manually organized by an subject matter expert to form the seed ontology. This might take a few days or weeks at the most.

    Automation kicks in once you have a reasonably well populated seed ontology. The program computes similarity scores based on the context in which terms appear. Terms having high similarity are put in the same class. There are still some places where a human expert is consulted to resolve contention. But it makes the project more feasible and viable for companies planning to build their ontologies from scratch. For companies who already have built their ontologies , this system could be a good way to update and improve utility.

    Building chatbots has been made easy by providers like Facebook messenger, Slack and Skype. By the end of this year , we will see chatbots being preferred over mobile interfaces by many applications.