Transitioning from subtitles: The 'case' nightmare

I'm trying to move from watching Slovenian shows with English subtitles to watching with no subs, but I'm completely lost. I find that when I listen, I can’t process the case endings (like the instrumental or genitive) fast enough to keep up with the sentence structure, and then I lose the plot entirely. Does anyone else struggle with this, or should I go back to focusing on grammar drills before I try to tackle native media again?