Regulatory Consulting at MDRC

Overview

MDRC is a nonpartisan research organization renowned for rigorous, often randomized, evaluations of social programs — welfare, education, workforce development — and it has begun applying that same evaluative rigor directly to AI tools deployed in social programs, rather than treating AI as a subject outside its evaluation mandate. Its 2025 study ‘Using AI to Help Job Seekers: Lessons from SkillUp AI,’ conducted with the nonprofit SkillUp Coalition, is a genuinely distinctive contribution to the AI-in-government-services conversation because it tested a specific, real AI tool against a specific alternative and reported what it actually found — including results that complicate the standard AI-optimism narrative. That willingness to publish a nuanced or negative finding, rather than a promotional case study, is exactly the kind of independent evaluation the AI-in-social-services space badly needs and rarely gets from vendors themselves.

AI-Focused Regulatory Work

MDRC’s SkillUp AI study is a small but important data point for anyone weighing whether generative AI can responsibly substitute for human services in safety-net and workforce programs. Career coaching provides real value to job seekers, MDRC’s researchers (Bennett Otten, Kelsey Schaberg, and Jason Cheng) note, but it is labor-intensive, requires knowledge of many career options and localities, and is often hard for resource-constrained job seekers to access — exactly the kind of scaling problem generative AI is often pitched as solving. MDRC and SkillUp tested that pitch directly: they piloted AI-powered career coaching against traditional site browsing, and the honest finding was that AI coaching led to lower engagement among job seekers, not higher. That is a genuinely useful negative result — it suggests that simply inserting an AI chatbot into a workforce-services pipeline doesn’t automatically replicate the value of human coaching, and that program designers need real evidence, not vendor promises, before scaling AI tools into safety-net services. MDRC’s broader value to this directory is less about publishing AI-policy frameworks and more about supplying exactly the kind of rigorous, program-level evidence that policymakers need to decide whether specific AI interventions in specific social programs actually work — the same evaluative discipline MDRC has applied to welfare-to-work and education programs for decades, now extended to the AI tools entering those same systems.

Notable AI Policy Work

• Published ‘Using AI to Help Job Seekers: Lessons from SkillUp AI’ (2025, with the SkillUp Coalition), testing AI-powered career coaching against traditional job-search methods.
• Found that AI-powered career coaching led to lower engagement among job seekers than traditional site browsing — a candid, non-promotional finding that complicates simple narratives about AI improving access to workforce services.
• Applies MDRC’s decades-long rigorous, often randomized, evaluation methodology directly to AI tools entering social-service delivery, rather than treating AI evaluation as outside its core mandate.
• Continues its core work evaluating welfare-to-work, education, and workforce-development programs — the same evidentiary discipline increasingly needed as AI tools enter those systems.

Thought Leadership

1. Using AI to Help Job Seekers: Lessons from SkillUp AI
   MDRC and SkillUp Coalition’s evaluation of AI-powered career coaching, finding it led to lower job-seeker engagement than traditional site browsing — a candid result on the limits of AI substitution in workforce services.

Clients

Nonpartisan nonprofit; funded by government agencies and foundations for program evaluation. Collaborated with the SkillUp Coalition on its AI-in-workforce-services study. No commercial clients; independence is central to its evaluation credibility.