Hi Oscar, welcome to the forum!
I use ARC with an Samsung TV and I have never experienced the behaviour you described. I hear now and then some pops and audio hiccups through ARC, however I’m always able to understand the audio dialogue. Please refer to the existing topic ARC: audio interruptions are noticeable, ref#92 for audio recordings.
Could you please detail on the TV model used? And if possible could you please record some audio and attach it to this topic?