Access to data is just the first step; the second step is discovery, to really build a full picture of what the data is actually saying
In the hiring landscape today, data has become the backbone of recruitment strategies. By leveraging data-driven techniques, such as automating pre-screening and shortlisting processes, companies can save up to 23 hours of manual labor every week.
But the benefits don’t stop there. Using in-depth data allows organizations to make smarter hiring decisions, identify ideal candidates more accurately, and reduce both the time and costs associated with recruitment. This approach not only improves the quality of hires but also helps reduce bias and streamline the entire hiring process.
I spoke to Shannon Ogborn, Community Lead and Recruiting Advisory at Ashby to understand what role data plays in improving the recruitment process. Shannon also hosts Ashby’s “Offer Accepted” podcast and manages the Recruiting Operations community, using these platforms to foster discussions and share valuable insights across the industry.
In this article, we will:
- Dive into how data can be effectively used throughout the recruitment journey, and what metrics to track
- Can we use data to build a more diverse / representative candidate pipeline?
- We’ll also explore how AI has changed the world of recruitment – for the better.
Q: Can you tell me a little bit about the recruitment operations function? (P.S. this is not the same as HR operations)
Shannon: Recruiting ops is definitely getting bigger. When you think about other functions like sales, marketing, or product, a lot of them have had operations for a long time, but recruiting operations is still in its infancy, relatively speaking.
There’s been recruiting operations for a couple of decades, but it hasn’t become a popular path until the last five to seven years.
Now, we have all these amazing strategic people in place who can really accelerate the work of talent teams. We even have a podcast episode with Max Butler about advocating for a recruiting ops person early on. If you have a bad process and your team isn’t enabled with the tools they use, there’s not going to be much success.
Q: Can you help me understand how a data-driven approach is beneficial to streamline the recruitment process?
Shannon:
- Service Provider vs Strategic Partner: Using data in recruiting is the key to becoming a strategic advisor rather than just a service provider. If you’re only given headcount targets or a list of job openings to fill, you’re functioning as a service provider without adding strategic value. However, when leaders come to you for insights—asking about the feasibility of hiring plans, analyzing data trends, and involving you in their decision-making process—you transform into a valuable strategic partner for the business.
- From Pipeline Insights to Candidate Experience: Data can be utilized in so many different areas of recruiting—from the top of the funnel, getting candidates into the pipeline, to building a representative pipeline overall. You can also use data to track pipeline metrics, like seeing where candidates drop out of the process. Are they withdrawing because of compensation or because the process is taking too long? Data helps in so many ways.
One thing I always tell people about data is that it only tells part of the story—you have to do the discovery on the rest. For example, pass-through rates will tell you where drop-offs are happening, but they won’t tell you why. Even with good dropout reason collection, you still need to investigate.
Access to data is just the first step; the second step is discovery, to really build a full picture of what the data is actually saying.
Q: Can you tell me some relevant metrics that are being used today in recruiting?
Shannon: At Ashby, we talk about modern recruiting metrics versus the ones historically used. We focus on metrics that are actionable, real-time, and goal-oriented, rather than traditional ones like time-to-fill or time-to-hire.
For example, we encourage looking at progress-to-hiring-plan, which is real-time and actionable. Other metrics include:
- Pass-through rates
- Interview hours per hire
- TA activity by stage
- Pipeline diversity
- Candidate quality
The goal is to use these real-time metrics to make immediate adjustments rather than retroactively trying to fix things later!
Q: With such a competitive job market, how can companies use data to effectively pre-screen and shortlist candidates, saving time and resources?
Shannon: We actually have some interesting data around this from our Talent Trends reports.
- Between January 2021 and January 2024, there was a threefold increase in job applications for business roles—a 207% rise—and a 161% increase in technical roles.
Recruiting teams are dealing with more applications than ever, often with fewer resources than they had in 2021. That year was the peak for recruiting, especially in tech, and things have declined since then, with interest rates and less VC funding playing a role.
In terms of data, knowledge is power.
- The first step is understanding your team’s capacity—can they feasibly review all the applications manually? You need to consider average application rates and staffing levels. Are you missing out on great candidates because your team can’t operate efficiently?
- The second point is about efficiency. Ashby recently released an AI-assisted application review feature that helps teams optimize human resources where it matters most and minimize tasks where human input is less valuable — for example, reviewing 20 resumes from candidates in a location you’re not hiring for You can set criteria for each role and the AI scans resumes, looking for evidence to determine whether candidates meet those criteria.
Q: So there’s no human intervention during the pre-screen shortlisting process?
Shannon: There is human intervention. The AI Assisted Application Review feature doesn’t reject anyone—it determines criteria met, allowing recruiters to focus on applications that might otherwise get missed. The AI evaluates and gives you the score, but it’s not rejecting anyone.
It’s saying, “This person matches four or five of your criteria, this person matches one.” But it’s really important that humans are still involved in the process because you might miss people.
This type of data-driven approach helps sift through applicants more efficiently, especially with application volumes being higher than ever.
Data gives you chapter one, but it doesn’t give you chapters two, three, or four
Q: What other areas do you use a data-driven approach throughout the recruiting pipeline?
Shannon: You can use data in so many parts of the hiring process.
- Pass Through Rates: One thing I like to look at the most is pass-through rates. There’s so much information in pass-through rates that can help you determine if you have a potentially inequitable process.
For example, if you’re looking at pass-through rates by demographics and notice a significant drop-off—let’s say, for women in an engineering role at a take-home test stage—you’d want to ask, “What’s happening there? Is there something we could change?”
- Progress to Plan: Another important metric is progress to plan. It doesn’t necessarily matter how many people you’ve hired if you didn’t hire the right person for the right role at the right time.
Let’s say you wanted to hire someone in December but ended up hiring them in October. Great, that one’s done, but if you’re supposed to hire four people in October and it’s now November, that’s a problem. You really want to prioritize and align by progress to plan.
- Recruiting Planner: We also have something in Ashby called the recruiting planner. It’s a great model for capacity, showing you how many people you need to have in each stage per week and overall within a timeframe to hit your hiring goals, based on historical data. All of this data gives a real picture into the feasibility of your hiring plan. JT Haskell, who was on our podcast, said it best: the smartest, most effective recruiting teams think into the future. You have to look at that data and ask, “Are we going to be successful in the future? Do we have the pipeline to hire for this role by December?” Thinking into the future is another way to use data to ensure you’re hitting your goals and operating effectively.
Q: With AI now in the picture – what has changed in the world of recruitment and rec ops?
Shannon: We’re still seeing it evolve. A lot of recruiting tools are building AI features now.
AI Adoption: One of the biggest challenges with AI in recruiting ops is comfortability. You have to get your team on board with the tools you’re using. I can’t tell you how many teams I know that have tools they barely use! You really have to encourage your team to utilize them and use them well. With AI, we’re in this middle stage where some people are early adopters, but there are still AI detractors—people uncomfortable with using AI for things like note-taking or recording interviews.
That’s important for recruiting ops because they need to get people on board with AI, both philosophically and practically.
AI Enablement: Another key aspect is enablement. Recruiting ops folks need to be good at change management—showing their team the tools, explaining how they work, why they’ll make jobs easier, and how they provide efficiencies. Ideally, these tools give people time back so they can enjoy life outside of work, whatever that means for them.
Q: How can analytics support DEI initiatives within the hiring process?
Shannon: As I mentioned earlier, data gives you chapter one, but it doesn’t give you chapters two, three, or four. Chapter one is collecting the data, and chapter two is asking why things are happening.
For example, if you don’t have a representative pipeline in your application pool, you need to look at your job descriptions. Is something in them detracting a representative pipeline? You can supplement with sourcing, but you must look deeper at the underlying process.
Building pipeline representation is one of the most important things you can do with data. To build a representative workforce, you need to know where your company is at and understand the market data.
What not to do:
- In terms of quotas, don’t do that—it doesn’t serve anyone. Quotas, especially in the U.S., are illegal. In other places, they may be legal, but even then, it’s not the best way to create lasting change. For example, women in engineering roles don’t make up 50% of the workforce, so setting a goal of 50% isn’t realistic. But you also don’t want to benchmark against other companies either because many have the same representation problems. If you’re just swapping candidates between companies, you’re not fixing the issue!
- People have good intentions when they’re trying to increase representation in their employee base. They might look at their gaps and say, “We only have men in leadership roles, so we need to hire a woman for this role.” But that’s not the best way to go about it. If you approach it in a certain way, the feeling can trickle down to the candidate or employee, making them feel like a token hire, like they didn’t get there on merit. You have to be very conscientious about how you go about building representation.
What you can do:
- It all comes down to building a representative pipeline from the beginning, investigating if there are drop-offs in certain demographics during the process, and figuring out if the process sets everyone up for success.
- Look at your historical data and set realistic goals. Build your pipeline in a representative way, so the numbers work out naturally. It’s not about lowering the bar, a common misconception amongst hiring managers; it’s about being more conscientious in building your pipeline and increasing representation at every stage. Data helps you achieve that without guessing. But you need a clear goal and process, rather than arbitrarily deciding to hire a specific number of people from a particular group.
Long-term change comes from pipeline representation, not just checking a box by hiring someone to fill a specific role. Just saying, “We hired a woman in this role, so we’re good,” will solve nothing in the long term.
Related Reads:
- CHRO Insights: While DEIB is Evolving, it’s not Disappearing | In Conversation with Sonia Mathai
- Q&A: DEIB in 2024 with Patricia Souza | Can AI Really Push the Needle?
Q: With the legal landscape for AI in hiring shifting, particularly following the recent settlement of a discrimination lawsuit, what should HR leaders and talent professionals know about these changing laws? How can they ensure compliance while leveraging AI in their recruitment processes?
Shannon:
- Do not put proprietary data into tools like ChatGPT, because many AI systems are taking data as well as providing output. You have to be careful about the inputs.
- Another thing I’ll say about tools: you should always ask the companies behind these tools how they’re staying compliant with laws and regulations. It’s important to be up to date on this, but since things move quickly, your tools need to keep up as well.
For example, in EMEA, they tend to have stricter privacy laws than in the U.S. Before AI, there were already privacy concerns when hiring in EMEA, and companies needed to ensure their tools complied with those laws.
If a tool doesn’t fulfill compliance needs, reconsider using it. The more manual processes you add, the higher the chances of mistakes and liability. So not only should you stay informed, but your tools must be compliant too.
- I always err on the side of whichever geography has the most stringent compliance laws. For instance, New York has new regulations on AI use in hiring, and in Colorado, you now have to put a closing date in job descriptions. If you hire across states with varying laws, it’s safest to follow the most stringent ones.
Q: Can you name some AI use cases or features for recruiters that they might not be aware of?
Shannon: I’m a bit biased, but Ashby has been very thoughtful in implementing AI, rather than rushing features out there. In recruiting, there’s been a big push to just put out AI features, but AI needs to be used thoughtfully, and that requires developers to build it that way.
- One feature we have is an AI personalization token for email outreach, and the results have been amazing. Without the personalized AI tokens, the reply rate was 24%. With them, it jumped to 35%. A 46% increase is significant.This feature allows talent teams to increase their volume of outreach without compromising personalization, which candidates are really looking for!
- Another feature we have at Ashby is AI candidate search. You can ask it to show you all candidates who interviewed for a specific role in the last year from certain companies, and it provides a list. It helps people who may not be great at building reports by letting them use natural language prompts.
- I also recommend MetaView, which records interviews and takes notes. The AI doesn’t provide opinions, just the facts—what was asked and what was answered. It helps create a more unbiased process because humans can react emotionally to things they hear in an interview.
- Another thing I love about Ashby is the AI debrief feature. It can summarize interview notes, highlighting what went well and what concerns might have been raised from interview feedback. It helps hiring teams be more efficient and reduces bias in the process, which is especially important for historically marginalized communities. So, it’s a win-win.
Q: How can smaller companies adopt more data-driven recruitment strategies to build a pipeline?
Shannon: When you don’t have a lot of historic data, certain things might be a bit more difficult, but that’s why actionable, real-time metrics are so important.
- You can also find benchmarks from other sources. We publish a lot of benchmarks at Ashby, and other recruiting tech companies do too, often segmented by company size. So even if you don’t have historic data for your own company, you can still use those benchmarks.
- The most important thing is not to over-index on certain outcomes where there is low volume. For example, if your offer acceptance rate is 100%, but you’ve only hired three people, and then one person declines, it drops to 75%. As talent leaders, you need to be conscientious of volume and how it impacts outcomes. It’s easier for larger companies with a bigger pool to maintain consistent metrics, but for smaller companies, a single event can skew results.
I encourage smaller companies to keep an eye on modern recruiting metrics, but don’t consider one instance as a trend. Setting up your data collection process early will benefit you in the future, even if you can’t act on anomalies right away. For example, if compensation is an issue causing people to drop out, you may not notice it immediately without proper data collection.
Smaller companies should focus on
- Progress to plan—it’s probably the most important one.
- Another one is interview hours per hire, especially for small teams. I’ve worked at companies where the engineering team spends 90% of their time hiring, which is not sustainable. So, you need to backtrack and assess if your team has the capacity to hire the number of people planned for a quarter or year.
- Pass-through rates are also helpful. Even in small companies, you can track if a candidate passes from recruiter screen to hiring manager interview. If there’s a 75% drop-off, for example, there’s misalignment or a problem that needs addressing to reduce unnecessary time spend.
While these metrics work better with more data, as all metrics do, they’re still useful even in smaller setups. Capacity modeling is another one that can be done without tons of historic data—check out our podcast episode with JT Haskell for more on that.
Q: What trends do you see in the field of recruitment / rec ops?
Shannon:
- Be friends with AI! I genuinely believe that recruiters who don’t lean into AI will be left behind. There’s fear in the recruiting community that AI will replace us, but I say no way. AI can’t do the human elements.
People join companies because they love their recruiter, had a positive candidate experience, or connected with the people they met. AI can help facilitate that experience, but this is still a human-to-human job. The most successful recruiters will be those who evolve with the times, and that’s always been true—not just with AI.
- Compliance is likely to become increasingly complex and challenging – but probably for the right reasons, so keep an eye on that.
- The trend I love, though, is seeing several rec ops folks getting promoted by showing their value and strategic nature through data and analytics. It’s heartwarming because rec ops have done this work for a long time without recognition, but now they can showcase their value.
Not only does it provide job security, but it also leads to promotions, and I think that’s the best.
