SDXC MBR ID issues

[RBBrittain]

At present there's no microSDXC cards, but with microSDHC now at its limit (32GB) and full-sized SDXC cards (64GB and up) now on the market that can be used with either SD-to-microSD adapters or as SSDs (with appropriate internal adapters that support IDE access), SDXC is an emerging issue for Rockbox.  This may have been better placed in the feature-suggestions forum, but I came here because it treads uncomfortably close to the NoDo list (though I do *NOT* advocate changing it) and it'll be awhile before it's urgent.

It's known that the first generation of SDXC (SD 3.0 or UHS-I) is electrically identical to SDHC, but is formatted using Microsoft's proprietary exFAT format; thus conventional wisdom (here and here) has been that they can be used with Rockbox simply by reformatting them as FAT32 with a third-party utility.  (Not only are file systems other than FAT32 NoDo, but AFAIK there is *no* open-source implementation of exFAT, though reportedly one is in the works for Linux.)  Second-generation SDXC cards (SD 4.0 or UHS-II) will have a different pin-out to support higher speeds, which (as assumed so far) will make them incompatible with current slots; they're too speculative to discuss for now.

However, this post says it is *NOT* enough to simply reformat the SDXC card as FAT32; you must *also* change its Master Boot Record (MBR) partition type ID to 0Ch (FAT32 LBA), which is *NOT* always automatic.  (ExFAT uses 07h, same as NTFS.)  If the MBR ID is wrong, Rockbox won't read the card--whether as a SSD (per the poster's experience) or thru a card slot (as I tested with a microSDHC card in my Sansa e280v1 running Rockbox 3.9, with FAT32 formatting but an MBR ID of 07h).

From both the poster's experience and my own testing with a 160GB external USB HDD, it appears third-party utilities are inconsistent about changing the MBR ID.  Though he reported the GUI version of FAT32Format didn't change the MBR ID, in my testing both the GUI and command-line versions of FAT32Format *did* change it; the difference may be that I was formatting an HDD instead of a SDXC card.  However, in my testing Partition Wizard Home Edition 6.0 does *not* automatically change the MBR ID, though a separate setting will change the MBR ID manually.  He also tested the Windows version of mkdosfs, but I didn't; I suspect it doesn't change the MBR ID either.  He used the command-line version of MBRWizard (click "CLI Freeware"--its GUI version is commercial; also check their old site for tips) to change the MBR ID; I used both that and Partition Wizard.  (Note:  All command-line tools require an elevated command prompt in both Vista & 7.)

From this testing, it's clear that once microSDXC hits the market, many users will find that even reformatting their cards to FAT32 may *NOT* make them readable in Rockbox.  It'll be hard enough to point them towards third-party FAT32 formatters; making them also change the MBR ID manually might be a step too far for ordinary users.  Given my experience with FAT32Format was different than the earlier poster's, it may not even be enough to point them to the "right" FAT32 formatter that fixes the MBR ID automatically.

A possible solution for a future Rockbox build is for it to do what Windows apparently does--ignore the MBR ID and detect FAT32 formatting directly, at least for microSD slots; I suspect that could be done in firmware without modifying the bootloader.  (That's especially true of older Sansa models like mine where the OF does *not* support microSDHC, but Rockbox does.)  It would be nice if that could also be done with SDXC cards (and other >32GB cards) used as SSDs; but that implicates the bootloader, which may make it just as much a NoDo as exFAT.  (Besides, those modders *should* have enough tech experience to do the MBR ID fix themselves--not just for Rockbox, but for the OF as well.)  Any comments or suggestions?

[Llorean]

Just to summarize, your overall statement appears to be "some programs don't properly set the partition type when reformatting a partition, so Rockbox should ignore the set partition and attempt to detect the filesystem independently?"

I mean no offense, but about 95% of the stuff in there isn't really important to the idea you're trying to bring up and a lot of people will get bogged down in the non-essential details. The question seems to be "should Rockbox attempt to compensate for reformatting tools that may not set things like MBR partition type, especially if it's likely to be a common problem with new physical media?"

[bluebrother]

A formatting tool should not change the partition type in the MBR, that's what partitioning tools do.

[Llorean]

It does seem a formatting tool should at least object if you're trying to format to something that contradicts with the set partition type, though... Much like some file managers these days will warn you "renaming the extension of this file may cause it not to be recognized by programs."

But regardless of whether they're supposed to, I think the question is "should Rockbox ignore declared partition type?"

[torne]

Virtually all current operating systems ignore the partition type and just detect the filesystem by probing for superblocks in known formats. We should probably just do that too.

[RBBrittain]

Virtually all current operating systems ignore the partition type and just detect the filesystem by probing for superblocks in known formats. We should probably just do that too.

I intentionally stepped away from the thread for awhile to let more knowledgeable people discuss it, but I agree that's the best option--as long as it can be done without using a lot more memory (the reason for the NoDo on non-FAT32 filesystems).  That way we can just point users with first-gen SDXC cards (same pinout as SD/SDHC) to any FAT32 formatter that ignores MS' arbitrary 32MB limit, whether it changes the MBR ID or not.  IMO that should be enough to keep the NoDo in place for all existing targets, especially since they will likely never support second-gen SDXC cards (different pins).

[torne]

Now that you've reminded me that this thread existed, I have made this change in r30566. There is no RAM issue, it's nothing to do with filesystem support; it's actually slightly less code. Any partition type other than 0 or 5 (unallocated and extended, respectively) is checked for a FAT superblock and mounted if it has one.

[toehser]

I'm surprised no-one mentioned that you don't have to have a partition table at all - you can just format the _whole device_ to FAT32.

On my Clip-Zip, Clip+, and Fuze-V2 devices, formatting it without a partition table works fine.

It gives a few more K of space, it is easier to loop-mount images, and there certainly can't be issues with the partition ID if you just format the whole device as FAT32 without partitions...

After all, if you aren't ever going to boot from it, and are never going to have more than the one single file system, why not?

[bluebrother]

After all, if you aren't ever going to boot from it, and are never going to have more than the one single file system, why not?

Not all programs / devices can cope with that. While what you said is true for Rockbox you might run into problems with some devices. Just to give an example, I have a simple DSO that can save screenshots to a uSD card, but it requires the chard to have a partition table.

So in the end it depends on your use case. If you're aware of the differences and know it won't cause problems for your use case there is no reason not to format without a partition table (sometimes called "superfloppy" format).

[torne]

Just in case anyone finds this thread later: the fix I referred to in a previous post got reverted, so you do still have to change the partition ID (or format as superfloppy) to use it in Rockbox.

The fix broke mounting superfloppy cards in certain cases, which we need to fix by making our MBR and FAT validation code more robust before the fix can be reapplied.