Why does my Oma look confused when I use 'het' words? Am I overthinking gender?

I'm a heritage learner trying to reconnect with my Dutch family, but I constantly stumble over de/het. I grew up hearing phrases in the kitchen, but I never learned the 'rules' behind the articles. My Oma understands me, but she always winces when I misgender common objects. Is there a trick to memorizing these, or should I just focus on getting the sentence out and accept I'll sound like a 'foreigner' to my own relatives?