Honestly I've had the opposite happen with local stuff. Got some clay from a nearby spot and it was full of gritty bits, kind of tore up my skin. Maybe it's just me but quality can be all over the place.
Okay wait, are we talking about just digging up mud from the shore? Because that's kind of genius. I spent money on a fancy bentonite clay mask that left my face red and tight. The stuff from my creek might be less dried out and harsh. It makes sense that local, fresh clay would be gentler. Has it ever caused any irritation for you?