Had to choose between explaining bias in AI or just fixing the data fast

I was working on a school project where we used an AI tool to sort student essays by reading level. But it kept flagging kids from certain neighborhoods as lower level even when they weren't. I had two options: either dig into the training data and try to explain the bias to the team, or just patch the algorithm quickly to get better results. I chose to explain the bias because I figured teaching everyone about the problem would help long term. It took three extra meetings and a lot of eye rolls, but a few teachers actually started checking their own assumptions after that. Has anyone else had to pick between educating people and just taking the shortcut?

2 comments

2 Comments

anna_fox721d ago

Oh great, nothing says "fun team bonding" like explaining systemic bias to people who just want the numbers to look pretty. Three extra meetings and a lot of eye rolls sounds like my entire Tuesday... and Wednesday... and Thursday. But hey, at least now you know who actually cares about doing things right versus who just wants to slap a bandaid on a bullet wound. Probably saved yourself a bigger headache later when someone would have noticed the "fixed" data was still rotten underneath.

paigep2021d ago

Youre totally right @anna_fox7. I used to roll my eyes at people bringing up bias in data, thought they were just making things complicated. But now I get it, a pretty number that's wrong is worse than an ugly one that's honest.