How Can AI Revolutionize the Match Day Process?

Many articles have been published that support the prospect of better patient care and work-life balance with AI at the helm of a new world. AI harnesses the power of computer systems to simulate human intelligence processes.^1,2

It is important to understand the nuances and key differences regarding the terminology associated with this technology. “Big Data,” for example, is a term used when a dataset has reached a size that requires advanced tools to manage it, including acquiring and analyzing the data.³

When considering all the data produced and collected by residency and fellowship applicants, program directors, and third-party organizations such as the National Resident Matching Program (NRMP) and the Electronic Residency Application Service, among others, one can only imagine the significant amount of data points acquired during the more than 70 years since the match system has been in place.⁴

While processes continue to evolve regarding the electronic application systems, the changes that have occurred, specifically within the last several years, have been transformative, including the US Medical Licensing Examination converting to a pass/fail system rather than a numbered score (Step 1), and the rise of ChatGPT, which is a generative AI chatbot that can generate human-like conversational responses.

How can applicants, program directors, and the profession strategically and successfully employ these tools to build a better future for surgical care?

Benefits for Programs

The organizational process required every year for the match process is not negligible. A significant amount of time is spent by program directors and program coordinators sending invitation emails and deadline reminders, and scheduling interviews.⁵ AI tools could help create and send this messaging, thereby reducing program leadership and administrative workloads.

AI tools also can help gather and summarize information about residency programs (i.e., match statistics, faculty interests, resident experiences, research opportunities, American Board of Surgery In-Training Examination statistics) that can be presented to applicants during their interview day.

AI is capable of analyzing large volumes of data and application packets, so one of the great promises of how this technology can revolutionize the match process is in screening and ranking applicants more efficiently.⁶ According to the NRMP, more than 50,000 applicants registered in the 2024 Main Residency Match, an all-time high and 4.7% higher than in 2023.

As the number of residency applications continues to grow,^7,8 it has become increasingly challenging for program leadership to review all the applications and find individuals who would best fit their program. For example, the Internal Medicine Residency Program at Indiana University Health Ball Memorial Hospital in Muncie receives approximately 2,000 applications per year for 10 positions. The program leaders are able to personally interview only 4% of the applicants. Johns Hopkins University in Baltimore, Maryland, receives hundreds of applications each year for its highly competitive surgical residency programs, and the Orthopaedic Surgery Residency Program has received more than 730 applications in recent years.

AI technology can analyze vast amounts of applicant data such as grades, test scores, letters of recommendation, personal statements, and clinical experiences to identify patterns that may not be readily apparent to human reviewers, and AI can complete this process much faster than program leadership.

According to ChatGPT, a detailed evaluation of an application—with a thorough analysis that includes “parsing unstructured data” (e.g., personal statements), would take 1–2 minutes, whereas a human reviewer might be expected to spend 70+ minutes reviewing an application.

In addition, using AI to review applications could help identify strong candidates who might otherwise be overlooked. In fact, by analyzing applicant data against program-specific criteria, AI is able generate a ranked list of applicants based on their potential fit with the program’s needs and goals.

AI also can have an important role in reducing the unconscious bias of application reviewers by focusing on objective criteria, reducing the influence of personal biases held by human reviewers and leading to a more equitable matching process.^9-11 AI algorithms can analyze large volumes of applicant data like academic performance, standardized test scores, clinical rotations evaluations, and research experience, focusing solely on relevant metrics rather than subjective interpretations influenced by factors like gender, ethnicity, or alma mater. AI systems also can anonymize applications by removing identifiable information like names and demographics; this helps ensure that reviewers assess qualifications without being influenced by personal characteristics.

Benefits for Applicants

From an applicant’s perspective, AI can be an invaluable resource for identifying ideal programs and optimizing a candidate’s application. AI tools can help analyze the applicant-program fit by comparing applicants’ qualifications and preferences with program requirements and culture.

AI can provide personalized recommendations to both applicants and programs about the likelihood of a good match using historical data about current and previous residents at a particular residency program.¹¹ Aside from editing personal statements and suggesting compelling “hobbies,” AI can bolster residency applications in meaningful ways. Letters of recommendation play a significant role in resident selection but may be stymied by a program director’s implicit bias, specifically for women and underrepresented groups.^12,13 AI models could be used to identify cases of implicit bias in language^10,14,15 and to create more standardized gender-neutral descriptions of candidates.^16,17

Benefits for the Surgical Profession

AI is helping to synthesize large quantities of data for both the individual applicant and the program. How can this technology be taken a step further to assist the surgical profession overall? What if the trends regarding surgical specialties were ascertained from the data already available and AI used predictive models to understand the specific deficits in the surgeon workforce a decade from now?

This information also can be used by applicants to determine if it is possible to find employment within their chosen specialty and desired geographic location. Program directors also could use AI to help select applicants with interests that align within their geographic location. These capabilities could result in surgeons who are content with their specialty of choice and satisfied that the job market aligns with those interests in their preferred region.

AI-based decision support tools like Career Insights can help applicants make informed decisions by integrating all relevant data, including specialty and location. These software systems leverage AI algorithms to analyze large amounts of data, identify patterns, provide insights to assist humans in making decisions across various fields, like healthcare, and offer recommendations and predictions based on analyzed data. The tools include SWOT analysis, decision matrices, career development assessments, skills gap analysis, job market trend reports, interest inventories, personality assessments, decision trees, and various online career planning platforms that provide data-driven insights to help individuals make informed career choices based on their strengths, preferences, and market demands.

Challenges Associated with AI and the Match Process

There are potential pitfalls of using AI in the residency application process, both from an applicant and reviewer standpoint. While AI can help candidates organize their applications and present themselves in the best light, candidates also run the risk of becoming too reliant on technology to create previously self-generated ideas and narratives.

Research has shown that when presented with ChatGPT-generated personal statements and those written by human applicants, residency application reviewers could not verifiably discern between the two.^1,9,18 Notably, an editorial published in the Journal of Graduate Medical Education recommended that applicants clearly disclose if they’ve used AI and use the technology as “a supplement, not a replacement.”¹⁹

Many of the nuances associated with the traits that make an individual a “good” resident cannot be easily duplicated by a generative AI model. While AI could help standardize the application screening process, it is imperative that these systems are designed to account for equity and impartiality. AI models have already been shown to create statements that contain unintentional cultural biases.^20,21 Therefore, AI tools must be carefully structured to avoid perpetuating biases that are present in historical data.

There is an overwhelming lack of transparency among most AI models, and residency applicants, programs, and stakeholders have a right to understand how AI tools manage their data and make decisions that significantly impact their lives and businesses. Programs must ensure that human judgment remains a critical part of the process and resist an over-reliance on machine learning. To be credible, AI models used in the match process should be designed or at least greatly influenced by the dedicated professionals using these tools.

AI is a tool just like the ones we use daily in the OR. Using a tool appropriately allows for ease of operation and a safe surgical setting. Using the tool inappropriately can cause a catastrophic event. Every new tool that is used in the OR must be tested and its background fully investigated prior to use. Additionally, the surgeon must be trained in how to use the tool. AI models—if they are to be used by programs or the surgical profession—need to be created by the surgical profession as all AI is designed to simulate human learning, comprehension, problem-solving, and decision-making.

New tools are being created all the time, and it is important to recognize which are going to be more permanent applications of this technology. AI and big data both seem to be, for the time being, here to stay. In that case, we must harness these tools and use them to help the entire surgical community, including trainees, programs, and patients, and we must stay vigilant of the limitations while appropriately applying the tool.

Dr. Rachael Essig is a first-year cardiothoracic surgery fellow in the Department of Surgery in the Division of Cardiothoracic Surgery at The University of Utah in Salt Lake City.

References

1. Johnstone RE, Neely G, Sizemore DC. Artificial intelligence software can generate residency application personal statements that program directors find acceptable and difficult to distinguish from applicant compositions. J Clin Anesth. 2023;89:111185.
2. Mir MM, Mir GM, Raina NT, Mir SM, et al. Application of artificial intelligence in medical education: current scenario and future perspectives. J Adv Med Educ Prof. 2023;11(3):133-140.
3. Pastorino R, De Vito C, Migliara G, Glocker K, et al. Benefits and challenges of Big Data in healthcare: An overview of the European initiatives. Eur J Public Healt 2019; Oct 1;29(Supplement_3):23-27.
4. Mao R, Williams T, Price A, Colvill K, et al. Predicting general surgery match outcomes using standardized ranking metrics. J Surg Res. March 2023; 283:817-823.
5. Strand EA, Sonn TS. The residency interview season: Time for commonsense reform. Obstet Gynecol. 2018;132(6):1437-1442.
6. Hassan M, Ayad M, Nembhard C, Hayes-Dixon A, et al. Artificial intelligence compared to manual selection of prospective surgical residents. J Surg Educ. 2025;82(1):103308.
7. Meyer AM, Hart AA, Keith JN. COVID-19 increased residency applications and how virtual interviews impacted applicants. Cureus. 2022;14(6):e26096.
8. Singh NP, Boyd CJ. Rapidly increasing number and cost of residency applications in surgery. Am Surg. 2023;89(12):5729-5736.
9. Lum ZC, Guntupalli L, Saiz AM, Leshikar H, et al. Can artificial intelligence fool residency selection committees? Analysis of personal statements by real applicants and generative AI, a randomized, single-blind multicenter study. JB JS Open Access. 2024 Oct 24;9(4):e24.00028.
10. Sarraf D, Vasiliu V, Imberman B, Lindeman B. Use of artificial intelligence for gender bias analysis in letters of recommendation for general surgery residency candidates. Am J Surg. 2021;222(6):1051-1059.
11. John AS, Kavic SM. Leveraging artificial intelligence for resident recruitment: Can the dream of holistic review be realized?. Art Int Surg. 2022;2:195-206.
12. Trix F, Psenk AC. Exploring the color of glass: Letters of recommendation for female and male medical faculty. Discourse Soc. 2003;14(2):191-220.
13. Madera JM, Hebl MR, Martin RC. Gender and letters of recommendation for academia: Agentic and communal differences. J Appl Psychol. 2009;94(6):1591-1599.
14. #BiasCorrect Install. Catalyst. Available at: https://www.catalyst.org/biascorrect-install/. Accessed January 8, 2025.
15. Gender Bias Calculator. Tom Forth. Available at: https://www.tomforth.co.uk/genderbias/. Accessed January 8, 2025.
16. Friedman R, Fang CH, Hasbun J, et al. Use of standardized letters of recommendation for otolaryngology head and neck surgery residency and the impact of gender. The Laryngoscope. 2017;127(12):2738-2745.
17. Leung TI, Sagar A, Shroff S, Henry TL. Can AI mitigate bias in writing letters of recommendation? JMIR Med Educ. 2023;9(1):e51494.
18. Patel V, Deleonibus A, Wells MW, Bernard SL, et al. Distinguishing authentic voices in the age of ChatGPT: Comparing AI-generated and applicant-written personal statements for plastic surgery residency application. Ann Plast Surg. 2023;91(3):324.
19. Mangold S, Ream M. Artificial intelligence in graduate medical education applications. J Grad Med Educ. 2024;16(2):115-118.
20. Cao Y, Zhou L, Lee S, Cabello L, et al. Assessing cross-cultural alignment between ChatGPT and human societies: An empirical study. ARXIV Labs. March 31, 2023. Available at: https://arxiv.org/abs/2303.17466#. Accessed January 8, 2025.
21. Ferrara E. Should ChatGPT be biased? Challenges and risks of bias in large language models. First Monday. November 7, 2023. Available at: https://firstmonday.org/ojs/index.php/fm/article/view/13346/11369. Accessed January 8, 2025.