Very annoying. I've been putting up with it, but it has been slowly driving me crazy. As the problem didn't occur on lossless FLAC files I've been slowly moving toward using them instead.
Version 1.22 of the firmware arrived recently and I upgraded this morning.
So far, everything seems fine. I've only tried MP3s (no FLAC yet).
Overall I'm happy with the player, especially since I bought some real headphones (some in-ear SkullCandys, brilliant) but I think the battery life could be better.