The ones I'm aware of:
- The old Archos devices (used AA / AAA batteries)
- Iriver H10 (custom battery)
- Sandisk c200 series (custom battery), e200 series (custom battery, requires unscrewing the back cover)
Though I don't expect a realistic chance to get one of those these days, let alone the custom batteries.
When it comes to swapping fixed batteries a few can be swapped with a bit of work.