My Bangalore Kannada vs. rural dialect: am I learning the 'wrong' one?

I’ve lived in Bangalore for a year for work and picked up a bit of colloquial Kannada, but whenever I visit family in North Karnataka, they seem confused by my pronunciation and some of the slang I use. I feel like my Kannada is a weird hybrid that doesn't fit in anywhere. Is it better to stick to one regional variant, or should I aim for a 'neutral' Kannada? I’m worried that if I don’t pick a lane, I’ll never sound fluent.