Struggling with vowel harmony in compound words—is there a pattern here?

I'm an absolute beginner and I keep bouncing between three different apps, but none of them properly explained why some compound words break the vowel harmony rules I've been memorizing. I get that front/back vowels don't mix, but when I see two words joined together, I get tripped up. Is this a common frustration for people starting out, or am I missing a core concept about how Finnish words are built?