Ok, I figured it out. It wasn't any sort of data corruption.
I had the songs organised in a directory structure like this:
*-De Grigny- Premier Livre D'Orgue:
|--cover.jpg
|--cover-small.jpg
|--booklet.pdf
|--*-Tome I:
| |--Bunch of flacs from CD 1
|--*-Tome II:
|--Bunch of flacs from CD 2
When I delete cover.jpg (which is like 600dpi or something large) it works. So I guess it was trying to load a large cover from the parent directory.