• Participants Share Lessons Learned From Virtual Match

    August 24, 2021, 11:52 a.m. David Mitchell — The AAFP and four other family medicine organizations recently urged residency programs to strongly consider conducting virtual interviews and virtual visits for all applicants during the 2021-22 Match season. The 2020-21 Match season was forced online due to the pandemic, and the family medicine organizations called for another year of data to study the virtual format.

    q&a wooden blocks

    With another interview season looming, AAFP News gathered a program director, a student advisor and an intern to get their insights from the 2021 Match and their thoughts regarding 2022.

    AAFP News: What was challenging about last year, and what actually seemed to go better than expected with a virtual format?

    Carmen Simmons, M.D., first-year resident at the Morehouse School of Medicine Family Medicine Residency: I actually was very surprised. I thought I was going to hate it, but I loved virtual interviewing. I had a great time. I thought I wasn’t going to be able to get a good sense of programs and people, but I think people really shined through. No matter what platform or format you use, people’s personalities are going to shine through — good or bad.

    headshot of Carmen Simmons, M.D.

    [Photo: Carmen Simmons, M.D., first-year resident at the Morehouse School of Medicine Family Medicine Residency.]

    The other plus was financial. Of course, people have made in-person interviews work in the past, but I cannot imagine trying to do that financially and move and be an intern making that transition.

    Another positive was that sometimes I just didn’t like a program, and it was nice that I didn’t spend the money to go there and use all this energy to find out that I don’t vibe with a program. And that’s not saying it was a bad program, just that it clearly wasn’t the program for me.

    I honestly don’t have a lot of cons for virtual interviews. It was an overall positive experience. I would say it’s exhausting, but it’s probably way more exhausting in person.

    headshot of Steve Brown, M.D.

    Steve Brown, M.D., program director of the University of Arizona College of Medicine - Phoenix Family Medicine Residency: Carmen, do you think you interviewed at more programs than you would have if it had been in-person?

    Simmons: I didn’t pass Step 1 on the first try, so I had a little bit of “application fever.” I applied to more programs that I probably would have otherwise. I do not feel like I applied to more because it was virtual. I think that also had to do with COVID exhaustion. I just couldn’t bring myself to apply to more programs. I applied to 30 programs. I was actually advised to apply to a lot more than that — around 60 — specifically because of my Step 1 failure.

    AAFP News: How many programs did you interview with?

    Simmons: I was invited to 17, and I did 13 or 14. I ranked 12. One program I just did not rank. It was a very successful season for me. I don’t know if I would have done anything differently as far as the numbers had it been in-person. There was one time I decided to do five interviews in one week. I do not suggest that. It sounds OK at first, but when you’re actually doing it, it’s awful. But you do have the advantage of being able to do that if you have to. I had a rotation and at the end of it I crammed those interviews together, which was helpful.

    [Photo: Steve Brown, M.D., program director of the University of Arizona College of Medicine - Phoenix Family Medicine Residency.]

    Brown: I really appreciate your transparency, Carmen. I think that will be super helpful for people.

    I was also surprised at how well it went. I expected it to be logistically challenging. I expected not be able to get to know the applicants. I was worried that they wouldn’t be able to understand what our program was like and whether it was a fit for them, whether they would get along with the residents and feel like these are people they would want to work with for three years. I had great conversations with people. People got to know our program. It was great.

    headshot of Annie Rutter, M.D., M.S.

    There was definitely Zoom fatigue. We did 25-minute interviews, and we made it so that it literally cut off at 25 minutes. That would definitely be something that I would change. It was super awkward. It’s hard to say goodbye on Zoom.

    I think the one downside for programs is that maybe the travel (which I totally agree with Carmen is not necessary) is a hurdle for students that is determining how interested they are in programs. So, when it’s on Zoom, it’s kind of like, “Well, I don’t really know if I want to go to that program, but it’s on Zoom, so I’ll go anyway.” That’s wasting everybody’s time. For programs, there was definitely a feeling that you didn’t know as much about whether students were really interested in your program.

    [Photo: Annie Rutter, M.D., M.S., director of medical student education, Department of Family & Community Medicine, Albany Medical College.]

    AAFP News: Did you interview more students than usual?

    Brown: We increased about 20%. Part of that was fear because we had no idea what it would be like. There were myths going around in program director circles that students were applying to every program, and the best students were taking all the interview slots. There was definitely an application fever on the program side, too, with a virtual format we weren’t used to. I think most programs interviewed more than in previous years, but it turned out that we didn’t actually need to. Our interns are a great fit for our program.

    AAFP News: Based on what you’re hearing at your own school, and what you’re hearing from colleagues around the country, what is different this year? What are the stressors now? What are your expectations?

    Annie Rutter, M.D., M.S., director of medical student education, Department of Family & Community Medicine, Albany Medical College: Last year there was no vaccine, and people were fearful of travel for their own personal safety, for their family’s safety. I think there was this layer of fear of COVID that was real. These were students who were doing clinical work for the first time, and they were doing it in a pandemic with protective equipment and struggling with their own clinical burden of being a student-physician for the first time in uncharted territory. That kind of overlaid everything.

    I’ve been through the match 10 times, and I feel like students have the same questions and fears whether a pandemic is happening or not: “Am I going to find the right program that’s the right fit?” “Am I going to live in the right city for the next three years to either bring my family to or start a family in?” “How many programs should I apply to?” “I failed Step 1; what do I do now?”

    Those anxieties were already there, but then we added, “I can’t get letters of recommendation because I didn’t do enough clinical experiences because my clinicals were truncated.” “My board exam was delayed three times due to COVID protocols.”

    Those are stressors we’ve never seen students go through during an application season. Some of those are alleviated this year. Most of our students are vaccinated, at least at my institution. I think that overall that’s less of a fear. I think the current surges and delta variant could become really scary things, but I think the personal safety and the stressors related to COVID are a little less this year. But their clinical education was impacted by COVID, so are they going to have robust enough experiences to get letters of recommendation? So there is still anxiety students are feeling, kind of a COVID hangover from last year when students were pulled from clinical experiences.

    How has that impacted this year? I think we’re better prepared. I think students saw what their colleagues went through last year. There’s an understanding there are benefits to virtual interviews, but the concerns are the same. There’s 700-plus family medicine recencies across the country. How do I find the one that’s the right fit for me? How is the community? It’s not just about the program, but where am I going to live? How do I know that that’s a good fit for me? Does that community have the supports I need? Are there people who look like me there? Those are real issues.

    The away rotations thing is a big wrench. Students traditionally have loved being able to have the opportunity to do away, or “audition,” rotations. They get to interview a program for a month, and a program gets to interview them for a month. It’s great for all types of students, high-caliber students but also students who have red flags on their applications. How do they show that they’re awesome? By showing up and doing amazing work for a month. The limitation on these audition rotations is a reality for students.

    AAFP News: Steve, you mentioned your 25-minute limit that maybe you’re not going to repeat in interviews this year. What else did you learn from last year? How will you tweak your process this year?

    Brown: In the past, I would meet applicants when they showed up in person and talk to them and describe the program and go over a PowerPoint. I made that into a video, which I loved because by the end of three months, I would get sick of saying the same thing. And I would be using the same jokes and trying to pretend like I had made up a new joke just for that day. So I made a video, we made a welcome video for our hospital, which people said they loved. It fit with our program’s character, which is a little bit irreverent. And then we did actually pretty similar interviews. We had three faculty interviews on Zoom. We put a gap in there so that the students didn’t have to do those three interviews back to back to back. We were cautious about Zoom fatigue, and I think that is something that we’ll still pay attention to this year. We really wanted that five-minute break between interviews so that people could let their dogs out or just decompress for a little while.

    I would like to do something different with the resident hangouts. We’re going talk to our interns who went through this process, but it’s really hard to have authentic conversations with multiple people in a Zoom format. I think my suggestion to students, and what we would tell our residents, is really bring your best self to every one of those little chats. Think about who you are and what you’re passionate about. That’s how you’re going to figure out what a fit is. I know it can be awkward, and we were worried that people weren’t going to be themselves in a virtual format, but they were.

    AAFP News: Carmen, you saw a lot of programs. How much variation did you see in how programs approach this? What worked, and what didn’t?

    Simmons: I think one of the ways we can gauge student interest is to have virtual interviews and then do in-person second looks. That’s reasonable. I loved my program in the virtual format. I was like, “ I would do anything to see this place in person. I would really like to meet these people in person.” In interviews in general, you’re bringing your best self and programs are bringing their best selves. I don’t think people can truly hide who they are regardless of the medium.

    I would say that the virtual interviews that are all day — maybe not. I get having an all-day interview in person because you’re doing the tour of the hospital, etc., but an all-day Zoom interview? No, just no.

    Some programs did videos, which I thought was helpful. However, some programs made us watch the videos on Zoom, and that was unnecessary. Just give us the link. Anybody who’s truly interested in the program is going to watch those videos before their interview.

    Brown: Dr. Simmons brings up a good point, which is, what do we do for second looks? Is that going to be an option? The family of family medicine and the program directors are very divided on that. Even though there’s been a recommendation from the family medicine organizations that interviews should be virtual, there are many programs that are going to attempt in-person interviews. And there’s lots of discussion about that among the program directors. There’s lots of discussion about the second look, which is also inequitable because if there’s a program that you really want to go to but you can’t pay for it there will be a perception that you’re not expressing interest in the program. Or maybe you can’t go to all of them in person for second looks. Hopefully, programs will not penalize students who decide to do a virtual second look instead of an in-person second look — if there even is a second look. I think there’s going to be some uncertainty because there are going to be programs encouraging students to do in-person interviews.

    Hear from Panelists Who Experienced 2021 Virtual Match

    Watch “Applying to Family Medicine Residencies” to plan your own journey through another unique season.

    AAFP News: Annie, what are you telling students regarding number of applications?

    Rutter: This is a question I have been asked since I first started working with medical students almost 10 years ago. I think the question is, did virtual interviewing and COVID change the game? The answer is, we don’t know yet. Anecdotally, Steve mentioned that he did more interviews. I think programs did more interviews because of that fear. I know our program did.

    There’s no right number for everybody. Different students should apply to different numbers of programs. To determine what the number is, we have to look at the application as a whole. We have to think about what are their work experiences? What are their life experiences? What are their Step scores? What does their transcript look like? We have to think about geography — where do they want to be? Certain areas of the country are more competitive than others due to location or number of family medicine residencies in that area.

    The AAMC talks about the point of diminishing returns, which is variable based on Step 1 scores and specialty. They don’t have data for IMGs, but for U.S. M.D. graduates, the number varies based on Step score. For students who do well on Step 1, we’re talking about 16 programs. If they’re doing not as well on Step 1, it’s about 23. The point of diminishing returns basically means that above that number students are actually not getting an increased likelihood of matching, and they actually may be going down.

    I think when students start to hear their number, then they start wrapping their mind around, “OK, I’m not looking at 65 applications,” because family medicine is different. It’s different than your friends who are going into one of whatever number of neurosurgery residencies there are in the country, right? It’s different, and so I think we have to think about family medicine-specific advisors. It doesn’t mean they have to be family docs. It means they have to understand the family medicine Match. The National Residency Matching Program gives us data every year, and for 2020, we can look at how many programs did a student need to put on their rank list in order to successfully match in a family medicine residency. The median number of what they call contiguous ranks — number of ranks on their rank list before they get to a different specialty or their list ends — was 12 for U.S. M.D. seniors, and it was 11 for U. S. D.O. seniors.

    The other thing to look at that is interesting is the probability of matching based on number of programs on your list. The probability of a U.S. M.D. senior matching if they have five contiguous family medicine programs ranked is 0.9. The same is true for U.S. D.O. seniors if they have six contiguous ranks. That’s really good.

    Now, let’s say a medical student has a blip on their radar from a Step 1 failure. You have this many volunteer opportunities, you have this score on Step 2. Let’s work together as advisor and student to put all that information together and come up with a list that we feel comfortable with. We do this with patient care all the time. We take public health data, and we decide how to apply it to the patient in front of us, right? And so there’s always going to be the person sitting in front of you, and how do you tweak it to make it work for that particular student? You have the conversation, you have the risks and benefits and you figure out what makes the most sense. I think it’s easy as an advisor and as a student to fall in the trap of, “Well, if I just do five more, I’m more likely to match.” And that’s where we see this application fever taking off.

    There are limitations to this data. I will point out specifically the limitation to the AAMC data is heavily relying on Step 1 scores. Family medicine programs are not as contingent on Step 1. I think they use it because it’s easy to measure.

    AAFP News: What’s the challenge for programs when students apply to too many programs? What needs to happen to change that? And do you think that’s going to be a problem this year?

    Brown: I think family medicine program directors always reviewed applications in a holistic way, but I think there’s been a dramatic change to increase that in the last several years. That’s been a huge focus of family medicine education, reviewing people not just from their numbers but what barriers have they overcome? What’s their story? Not so much where are you now, but where did you start and how did you get there? That is hard for program directors because there’s no checkbox to sort people for what their life journey has been.

    We would love to holistically review all the applications, but that’s really hard when you get 1,000 applications. You still have to sort in some way, and program directors fear that we’re not spending the time on the students that are really interested in our programs, so you still have to sometimes default to more traditional measures of student excellence. This is a huge problem. How do you match student interest and passions with what a program can give that student?

    I still don’t think we have a great resource. It’s hard for students to sort programs based on characteristics. Obviously location is not hard. Suburban, rural, urban is not that hard, but after that the students basically have to search our websites to find the information they need to match their passions, and that’s really challenging when there’s 711 programs. Our specialty has not solved that problem yet. All the things that we can do to improve application fever are not going to work super well until the students feel confident that they can find the programs that are the right fit for them without trying to interview at 100 programs.

    AAFP News: How many applications did you get last year?

    Brown: Compared to typical years, ours didn’t go up much. I don’t know the exact number, but I think for multiple years we have received between 1,000 and 1,500 applications.

    Rutter: I think transparency among programs — what criteria are they using to look at students — is really helpful. That will help curb students from applying if they know, “We don’t have any students with X characteristic,” whether that’s a Step score or some other filterable thing. Then a student is able to assess their competitiveness for a program. When you think of certain undergrad institutions like Stanford or Harvard, you’re like, “They’re really hard to get into,” but there’s nothing like that for family medicine. There’s no communal understanding of competitiveness of programs, other than institutional memory: “Where have we matched students who are highly competitive before?” Or, “Where have we matched students who have had blips on their radar in the past?” That’s not fair to students, because if they don’t have an advisor who has been around for a long time, or they don’t have students ahead of them who has gone through that, then it is nearly impossible.

    The other thing, though, falls on the advisors of undergraduate medical education, and I think that’s paring back the applications of students who are highly competitive. Can we encourage students who are going to need fewer interviews at fewer programs to have a successful match to accept fewer interviews to allow their counterparts who actually need more interviews and applications to be successful in the match to get those spots? I don’t have a formula for how that works. I think it’s just something that we should strive toward, guiding students as best we can so that the precious interview slots that a program has available are distributed to students who are appropriately fit for them and students that would be happy there. Nobody wants to see a match be unsuccessful and after a few months an intern decides this is not the right program for them. Nobody wants that. So, how do we strive toward getting people as happy as they can be and positioned to be the family doc they want to be in the program that is the right fit for them?

    AAFP News: There are some pretty negative forums online. Are there websites that you advise people to avoid, and what resources do you think students can trust in this process?

    Simmons: Reddit and Student Doctor Network are horrible places. It’s miserable, and it really freaks you out. I know that’s much easier said than done because I was on Reddit, too. I’m not here to promote anything, but for my year, especially applying, our GroupMe for Black applicants applying to family medicine was extremely helpful in creating community, and we’re doing it again this year.

    Something that was a really good safe haven, and not an ethnicity-based GroupMe, was Twitter. I found Twitter and med Twitter to be extremely helpful throughout the match process creating a community of people who are applying.

    Brown: I think Doximity is flawed. There’s some buzz among the program directors that you don’t give your data to Doximity. It’s not judging the right things.

    We looked a few years ago at hits to our program website, and the pages that had our residents’ pictures and bios had 10 times more traffic than any other page. So, look at the residents. Where are they from? What are their passions? What are their backgrounds? Would do you feel comfortable there? Obviously, you can’t totally judge a book by its cover, but I think that’s a good way to find out about what kind of students match in that program.

    Rutter: Look at graduates, too. Are there people who graduated from this program that are doing what you want to do in five years? You know, I want to be a hospitalist. I want to do OB. I want to do the addiction medicine, whatever your interests are. It’s following those passions.

    The AAFP residency directory is really helpful. We send students to that all the time. For prepping for the Match in general, Strolling Through the Match. And I’m not just saying that because I’m in an AAFP group. Those two resources in particular are extremely helpful.

    And again, building a community of people going into family medicine is helpful because part of the issue with places like Reddit is it’s not specialty specific. Students get pulled in by their colleagues who are going into dermatology and applying to every derm program. You’re not applying to every family med program; it’s apples and oranges. You just can’t even make that comparison. There are different numbers of programs. Competitiveness is variable. Again, I encourage students to work with family med-specific advisors. It doesn’t have to be a family doc, but it needs to be someone who knows about the family medicine Match. And use resources that are specific to family medicine because that’s where you’re going to get information that is relevant to what you’re trying to accomplish.

    Simmons: In the Association (of Black Family Medicine Applicants), a lot of applicants look into whether a program has had a Black person or a person of color in the last five years. And honestly, a lot of times the answer is no. It’s not that you think you can’t go to that program, but it is discouraging. Nobody wants to be the first or only minority. When I was looking at programs’ websites, the first thing I did was look at the residents.

    Rutter: Using residency fairs is a great way for students to get a sense of a program, and for one plane ticket or one login you’re able to meet a lot of different people. Students get really excited about the process and start to build a community of family medicine across the country, which may not be present at their medical school. How supported are they in applying to family medicine? Students choose to go into family medicine, but is there someone at their institution that can help them? I’m talking about that as an advisor, but there are schools that that don’t have the ability to have an in-person advisor who can really help them. I think building a network of people, like Dr. Simmons has said, can be really helpful.

    AAFP News: Carmen, what was the most fun, or most memorable thing that happened on any of your interviews? How can programs make themselves stand out in a virtual process?

    Simmons: I was fortunate to be able to do an away rotation. I was really freaking out about that as one of those students who had some red flags on my application. I was thinking, “How am I going to make myself stand out?” I interviewed there, as well, but I told them, “I 100% saw the same type of excitement and creativity y’all had in the virtual format as when I was at your away rotation. I don’t think anything would’ve changed about how I would have ranked you or how I engaged with the program even if I didn’t see you in person.” They sent boxes to the applicants, and we all opened them together. It was super fun. And that just kind of speaks to the type of program that they are.

    For my own personal program, we had a Zoom game night that was fun. Obviously, I’m biased because that’s the program that I really wanted to go to. That was my number one. We had a really good time with that. I feel like a lot of places made things special. And you didn’t have to have a game night, a Zoom party, a gift bag or anything like that. You can tell what programs valued during their virtual format. Some programs want you to come do your work and go home, and that was kind of their culture, which is good for a lot of people. I’m very social. I talk a lot. That’s not kind of the program that I would thrive at. For a lot of people, that is exactly where they need to be. You can see how a game night, for some people, would be like, “OK, this program is doing way too much with a game night. It’s too involved. It’s too much.”

    Brown: Cones of Dunshire?  

    Simmons: Right. (Laughing.) Hey, we get it. Do we have to spend this much time together? No. I want programs to know that your personality is shining through.

    Brown: For virtual interviews, students need to know that, yes, they need to be professional. Yes, students should have professional backgrounds, but also students are going to get a break from programs. If something unexpected happens in your virtual interview, which happens all the time, just roll with it, be positive and laugh about it. I had one applicant who couldn’t be at home for some reason. He went to his school, and the room had a motion sensing light. Every five minutes of his interview he had to wave his arms around. The fact that he was able to laugh about that was a plus. No program is going to be like, “Oh, my gosh, that’s the crazy kid with the motion sensing light.” Give yourself a break and enjoy it, and let your personality shine through.

    Rutter: We had a similar story. I also do residency interviews, and we had a student who had a new puppy. We were chatting, and he was totally professional from the waist up. He said, “I’m happy to show you my dog, but you need to understand that if I stand up, I am wearing sweatpants.” It broke the ice. Everybody was laughing.

    We thought virtual interviews wouldn’t be impacted by weather, but we had people losing WiFi because there were snowstorms or windstorms when they were trying to call in. We interviewed people from very different time zones, so we had to be thoughtful about it being late at night or early in the morning. There’s a lot that can influence the experience during an in-person interview, and that’s not lost in a virtual format. There are plenty of opportunities for interviews to go off the rails and to be guided back on. It shows you a lot about an applicant’s ability to roll with the punches.

    Advisor’s Guide to Strolling Through the Match

    Answer applicants’ top Match questions with these key sections from Strolling Though the Match. This resource is updated annually and has been a trusted source for decades.

    Question: How can I stand out when I apply to programs?

    Advisor key: Use the information in section 2 on building a CV and gaining experience to help students in any year of training set themselves up to stand out as leaders and show a commitment to family medicine.

    Question: How do I choose which programs to apply to?

    Advisor key: Guide students exploring programs with tips from Section 4, specifically, advise them on the do’s and don’ts of residency selection and narrowing down their list of programs.

    Question: How many programs should I apply to?

    Advisor’s key: Give students customized advice using graphs on the point of diminishing returns in Section 5, and data on chances of matching in Section 7 to help them avoid over- or underapplying.

    Question: How do I prepare for interviews?

    Advisor key: Read Section 6 on interviewing for the information students need to stand out in virtual interviews and make decisions about what they learn about programs.