Switched my whole crew from GPT-4 to an open source model last week after our token bill hit $3,400

Robert Anderson · 2026-09-29T14:50:06.421668+00:00

I was dead set on keeping the paid API because I figured the open model would fall apart on our messy internal docs, but after running 500 test queries side by side the open one actually nailed 87% of them versus 89%. Saving that cash matters more than chasing two points, and honestly half our team can't tell the difference anyway. Curious if anyone here has made the flip for a real production workload, did the edge cases end up biting you later?

Opinions Matter

Loading…

Switched my whole crew from GPT-4 to an open source model last week after our token bill hit $3,400 | AI Innovations - Opinions Matter