Loading…
5
Pro tip: I taught myself to spot biased data sets in machine learning projects
I was helping with a hiring AI tool and saw it favored certain backgrounds. After some reading, I learned to check data for unfair patterns. What other checks should I run to make sure the model is fair?
3 comments
Log in to join the discussion
Log In3 Comments
simonp477mo ago
Wait, you saw it favoring certain backgrounds? Isn't that a huge problem for hiring? What else did you check besides the data?
5
felixfisher7mo ago
Yeah you asked what else to check. We ran the same resume but swapped company names. Like "Smith and Sons" versus "Global Data Corp". Same exact role. The AI picked the corporate name every single time. Then we changed just the years worked, making one candidate have a two month gap. Instant rejection. It's not looking at skills at all. You have to test these tiny details to see the bias.
8
the_spencer7mo ago
Oh man, it's wild. The AI would see someone went to an Ivy League school and basically throw a parade, while a state school grad with the same skills got a "thanks but no thanks." I even tested it with made-up resumes where the only difference was the job title at their last company - "Senior Analyst" got picked, "Lead Data Specialist" doing the exact same work got passed over. It's not checking skills, it's just playing a weird game of status bingo.
1