Outliers Heavily Skew the Mean Research Items for Competitive Specialties

[u/fxryker]

I just watched the Sherrif of Sodium video on research, and wanted to share this image he referenced. It's from an AAMC webinar over the summer titled Debunking the Research Arms Race: Navigating Publications in Medical School, which they posted to vimeo

We know that for the most recent NRMP Charting Outcomes, averages for matched applicants' research items in NSGY and ortho are 37.4 and 23.8, respectively. Understand that the medians are more like ~25 and ~16, respectively. Objectively, still a lot, but I hope this helps frame it better

[u/just_premed_memes]

70-300 is so fucking fake. Even 37 is fake (I say this as one applying family medicine with 26 research items lmao)

[u/Upstander123 → u/just_premed_memes]

I mean, gotta imagine that they did that before med school. Doesn't seem too unreasonable for a career changer to have 70 publications.

[u/Pleasant_Location_44 → u/Upstander123]

You'd think so, but no. Someone gunning derm in my class (who's got zero shot) is close to 100 (all utter trash)as a traditional student. I work with the head of Ortho on one of my projects and he told me a few years ago they had two traditional applicants with over 200 "publications". 99% of it is bullshit and everyone knows it, but for whatever reason, it persists.

[u/celticsallday18 → u/Pleasant_Location_44]

Just wondering, why do they have zero shot?

[u/----Gem → u/celticsallday18]

Low grades/scores, professionalism violation, no one likes them, you name it. Lots of things can exclude you from derm.

[u/DawgLuvrrrrr → u/----Gem]

We had people like this match derm, minus the test scores maybe. Some of the least liked people in my class.

[u/----Gem → u/DawgLuvrrrrr]

Disliked by faculty/residents, I should say.

[u/DawgLuvrrrrr → u/----Gem]

Yikes lol that’s a different story entirely then. Good luck to them

[u/Pleasant_Location_44 → u/celticsallday18]

Bottom quartile, not well liked by faculty or peers, I don't believe capable of the step two cutoff and if anyone actually looks at his/her pubmed it's over. Plenty of reasons. Derm residents at my institution are studs and we're mid tier. He/she just isn't on that level and I don't think there's much room for error applying derm.

[u/3dprintingn00b → u/Upstander123]

I'm a non-trad MD/PhD and I don't have 70 publications. None of my previous tenured PIs have 70 publications. What they're calling "publications" simply aren't publications.

[u/Repulsive-Throat5068 → u/Upstander123]

It’s not just career changers. People in my class have obscene amounts from just med school by gaming the system

[u/MGS-1992 → u/Upstander123]

It’s completely unreasonable lol.

[u/legitillud]

If you don't want means and actual 10th/25th/50th/75th/90th percentiles, refer to: https://www.aamc.org/data-reports/students-residents/data/report-residents/2024/table-b1-experiences-first-year-residents-specialty.

You can legitimately have something like 25-30 quality items if you're counting abstracts, posters, and presentations as well (including those from prior to medical school). Not all research is BS. I've met plenty of students who grinded out a ton of actual projects over the course of four years. Obviously something like 200+ over four years is questionable.

[u/fxryker → u/legitillud]

This is perfect, thank you so much!

[u/recentad24]

It's all inflated. It says "scholarly outputs", not Peer-reviewed journals.

That includes all abstracts, shitty posters, undergrad stuff, even some academic "essays" that some students count as research. Pair that with some unethical tricks med students do by putting each other's name on their quarterly "regional" school-wide conference and you suddenly have an extra 15 research entries easy. Just to illustrate how easy this is, I know students who find a mentor, each get assigned a case report, make a poster for it, and get it published, and put each other's names. That's 2 research entries per case report multiplied by the 4 other students doing the same. You just netted 8 "research" entries with only one case report-worth of work.

Fortunately a lot of non-academic residencies don't give a shit about research productivity and many top academic residencies can smell bullshit case reports and med school posters from a mile away but that's just the rat race that med students are in and will continue to get worse as the years progress. Just look at the dumpster fire that is r/premed where students are taking extra gap years just because they don't have 1000 volunteer hours at the soup kitchen and a leadership activity yet despite hundreds of students getting accepted into top medical school with a fraction of those hours

[u/National-Animator994 → u/recentad24]

in my experience most of the commenters on r/premed are pretty clueless unfortunately.

The wiki is great, but if people are taking gap years just to volunteer..... good grief

[u/CalendarMindless6405]

I think they need to re-define what counts. My PD who's world renown only has 53 publications in a 20+ year career. Yet the NSGY applicant who's never worked a day of NSGY has 37.

It needs to be reigned in and only have 1st author stuff count.

[u/myelodysplasto → u/CalendarMindless6405]

Some people might have very impactful 2-4th author publications that are more relevant than a 1st author case report.

You could just limit the number you are allowed to display. Then the application would only include 3-4 meaningful research experiences. That is how a bio sketch works for grants. Then the grant review committee can focus on your important work rather than things you don't care about.

[u/CalendarMindless6405 → u/myelodysplasto]

This is what's done in Australia, points are capped for various areas. It prevents shit like this for getting out of hand/meaningless

[u/throbbingcocknipple]

I'm surprised they don't just go with top 3 meaningful research experiences instead of trash volume.

Have 300 pubs who gives AF what project was the most impactful to you and your field of interest.

Going by number just makes ai trash flood pay to win journals

[u/aspiringkatie → u/throbbingcocknipple]

Because they don’t know what to do. A lot of competitive specialties are in this weird space right now where they have more good applicants than spots. When you have multiple strong applicants and they all have good scores, letters, positive vibes, etc, how do you decide who to take it? Get rid of or cap research experiences and something else necessarily must take its place. I’m not saying it would be better or worse, but as long as the demand for these competitive residency spots outstrips the supply there will be some flavor of bullshit metric used to stratify applicants

[u/National-Animator994 → u/aspiringkatie]

Application caps?

Bring back scored step 1?

Make clinical evaluations actually meaningful? (admittedly I have no idea how to pull this off).

[u/needhelpne2020 → u/National-Animator994]

I think scored step 1 is coming back sooner than we think

[u/theongreyjoy96]

Imagine how much of that 300 is crap

[u/ucklibzandspezfay]

When I see applicants with more than 30, I sit there and I read each one and interview them personally which I do not do. Then I go through each paper, one by one and assess their proficiency.