This newsletter contains useful information about the automation engineering industry. Our staff is mainly comprised of engineers so there is a lot of relevant content for both management level and engineering focused readers. We hope you enjoy the content!

• Core Jobs and Resume Analysis: What is the gap between skills in demand versus skills available.

• Recommended Training: We take and deliver a lot of training. Check out our recommendations below.

• Recommended Tools: With automation engineering, there are a lot of tools. In this section we recommend tools that have had the most impact on recent projects.

• Recent Blogs: CMG is always giving back to the automation engineering community in the forms of videos, blogs, and webinars. This section highlights some great technical blogs you may find interesting.

• Customer Case Study: While we do not share our customers' names (without their permission), we love to share solutions we deliver that increase our customers' productivity.

Core Job Data

Machine Learning/Data Science Jobs

Each quarter in this section, we share trends and insights on IT Automation jobs from both the employer and job seeker perspective. We collect data on job postings and candidate resumes in areas including DevOps, DevSecOps, Data Science, and Full Stack Development. The data is collected from public sources on the Internet, aggregated, and evaluated to provide you pertinent information about the industry. 

We use data engineering, preprocessing, Exploratory Data Analysis (EDA), Machine Learning (ML), and Natural Language Processing (NLP) techniques to provide the information listed below.  There are many questions we can ask of the data and observations we can make. This month we wanted to look at the Machine Learning and Data Science job postings and the candidates available in the United States.

From the job posting data, we answer the following questions:

• Where are the jobs posted?

• What are the most popular skills in demand?

• Where are the jobs located? Which states have the most job postings?

• How do the skills available align with skills in demand? 

From the candidate data, we answer the following questions:

• What skills are advertised on the resumes?

• How do the skills advertised align with the skills in demand in the job postings?

The goal is to provide some insight into the number of skilled candidates versus the number of jobs and skills requested. This helps us to understand the gaps between the skills requested by the job postings versus the skills possessed by the candidates.  We specifically focus on recent graduates from top 130 computer science programs with less than two years of experience. All the analysis was performed using Python, CI/CD Pipelines, Machine Learning (ML), and Natural Language Processing (NLP). As this data changes over time, we will provide periodic updates with new or updated analysis.

Explaining the Data

At the beginning of each quarter, we gather information on the currently posted resumes and currently posted jobs from multiple job sites. For this newsletter we examined the data available for entry-level “Machine Learning and Data Science” positions. To determine if a job was an ML or DS job, we searched the title of the job posting for “Machine Learning, Data Science, ML, Data Engineering”). We determined if a job was entry-level based on the job site tagging the job as entry level. 

Where are the jobs posted? 

This led us to narrow our search data down to a few job sites. In the chart below, you will see the breakdown of the total entry-level Machine Learning and Data Science jobs that we found on each of the sites.

What are the most popular skills in demand? 

The chart below shows the top skills requested for entry-level candidates. No surprise, Python is the top skill, the chart is self-explanatory.

Where are the jobs located? Which states have the most job postings? 

We can also determine where the jobs are physically located in the United States.

What skills are advertised on the resumes? 

The image below represents the skills advertised on candidate resumes. The counts are based on aggregated frequency distribution of all resumes (deduplicated).  We basically consider these the skills available to help compare the supply vs. demand.. 

How do the skills available align with skills in demand? 

When we compare the top skills in demand (those in the job postings) to the skills available (those in the candidate resumes), we see a significant mismatch. What this indicates is that more training is needed for entry-level candidates across a broader spectrum of skills. Colleges and universities could adjust to account for this. Employers need to know when hiring new candidates that there will be significant training required to develop the skills needed. CMG trains our team on current and legacy technologies to ensure we do not suffer these gaps. The typical turnaround time to close the skills gap is approximately twelve to eighteen months (12-18 Months). At CMG we use this data to develop a targeted training and mentoring curriculum to align with the top skills in demand. Hence the CMG “Day One Ready Resources” claim. The image below shows a direct comparison between the skills in demand vs the skills advertised on resumes.

Core Job Data: Summary

CMG uses this data to make our resources “Day One Ready” to support customer projects, staff augmentation, or managed services requirements. You can use this data to determine how you, your team, or company aligns with the state of the industry.

Recommended Training

We assigned "Introduction to Maps in Folium and Python" to the team to help them prepare for an innovative project that incorporated the mapping of publicly available information. This is a short, and effective training to help you learn how to visualize your data using Choropleth and Heat Maps.

Recommended Tools

In mathematics, graph theory is the study of graphs, which are mathematical structures used to model pairwise relations between objects. A graph in this context is made up of vertices (also called nodes or points) which are connected by edges (also called links or lines). A distinction is made between undirected graphs, where edges link two vertices symmetrically, and directed graphs, where edges link two vertices asymmetrically. Graphs are one of the principal objects of study in discrete mathematics. (Source: Wikipedia). On a recent project we used NetworkX and highly recommend this tool. NetworkX works well for mapping the relationships between data. We use it to map social networks relative to specific requests for information about global conflicts.  Aside from creating incredible visualizations, the tool allows for mathematical analysis of the data relationships.

Recent Blogs

Quick Customer Case Study

We recently engaged an enterprise customer that was manually creating tickets and service requests in their ITSM system based on email requests from customers and internal lines of business. The manual process also required the support team to use a shared mailbox which inevitably became a “catch-all” for not just customer care and line of business support, but also report delivery and other messages. The real problem for the customer arose from the need to duplicate the efforts of managing the shared mailbox and the manually created tickets. Another challenge was that tickets were created and often assigned to the wrong groups for remediation. This further exacerbated the duplication of efforts.

As a solution we proposed the following: “Automated Ticket Creation and Assignment using Machine Learning(ML) and Natural Language Processing(NLP)." The overall architecture proposed and created is represented in the image below.

The numbered list below describes each component:

1. The overall infrastructure required to support the solution was comprised of custom-built APIs, NLP models, container images, and a CI/CD pipeline.
2. The first API was developed to be an email listener. This API connected to the shared mailbox and processed all unread messages, collecting all the data about the message including date/time, sender, recipients, subject, and message body. This data was pre-processed for delivery to the NLP API.
3. The NLP API was a custom build REST API coupled with a convolutional neural network model designed and trained to classify over 160 classes of text. Two separate models were developed. The first was developed to read the message subject, and the other was used to process the message body. Whichever had the higher confidence for group assignment in the ticketing system was selected for routing the auto-created ticket.
4. To support the deployment, management, and monitoring of the API, we built custom CI/CD pipelines.
5. To deploy the APIs (models included), we leveraged custom docker container images. The container images were hosted on-premises and called as needed by the CI/CD pipeline.
6. To integrate the automation with the existing systems (M365 hosted shared mailbox and SaaS based ITSM system), we created API Access Tokens for each service and connected to the APIs from the internally running container based custom APIs.
7. Once an email was received in the shared mailbox it was processed by the “Email Listener” and sent to the “ML/NLP API” for predictions on the correct group in the ITSM platform. The ITSM platform was also configured to support ITSM business rules to support existing conditional logic.

If you have similar challenges or have tried and failed with “Chatbots”, give us a call to discuss more about how we can build a solution that is designed to handle your data and workflows.

If you want to discuss how you might be able to leverage DevOps, DevSecOps, Automation, Machine Learning or Natural Language Processing in your environment, reach out to us here at CMG. You can even set up a meeting with us if you’d like. We’d love to discuss how we can help you with your automation and integration needs!