Change wrong partition alignment after restoring iPod in iTunes

[ptrk.mj]

Hello.

I have an 5.5G iPod Video. Recently, I replaced it's broken HDD. The disk I am using now (Samsung HS12YHA, 120 GB) is a new generation product and has 4K physical sector size (instead of conventional 512 bytes). The industry term for that is Advanced Format. This is not a compatibility problem since drive firmware provides translation for host (whether it's a PC, media player or anything else). It presents a disk to the host as if it had 'classic' sector size. Well at least it does when connected directly to PC (see below).

The problem is that 4K physical sector size requires that partitions are aligned properly. Put briefly their starting LBA block number (considering each logical block consist of 512 bytes) needs to be multiple of 8 (512*8 = 4096) so that partition starts at the beginning of the physical sector and not in the middle of it so that clusters are aligned with physical sectors. If that doesn't happen severe performance issues arise (check here for more info).

Unfortunately, iTunes with it's 'Restore' function is completely unaware of that and starts the partition according to old CHS convention. And that's a real bummer. Write speed speed becomes extremely low. E.g. it took me 1 minute 15 seconds to write 160 MB of mp3 files.

The funny thing is how the disk appears to my PC software when it's connected through iPod. Instead of appearing as an array of 234,441,648 LBA blocks (512 bytes per block) as it does when connected directly and as a sticker on it's housing states it appears as an array of 58,610,412 blocks (2048 bytes per block). Now, when iTunes restores and repartitions the iPod it starts it's partition at logical sector 48,195 (not a multiple of 8, instead according to CHS drive geometry convention). Everything to the left (sectors 0 - 48,194 ≈ 94,13 MB) remains 'unallocated' and reserved for firmware.

I tried using GParted Live CD to realign partition by expanding preceding unallocated space to 95 MB but something prevents me from doing that. I end up with an error and partitioning remains intact.

Another funny thing is that when I take the disk out of iPod (after restoring) and plug it to my PC partitioning becomes "corrupted". The addresses of partition start (48,195) and end (58,610,410) remain the same. However, logical block size is four times smaller (512 bytes vs 2048 bytes) so that both the partition and unallocated space preceding it are 'shrinked' and there is additional unused space at the end of the disk.
Even so, it's impossible to resize the partition because it appears to GParted as corrupted/raw (instead of FAT32). After re-inserting disk into iPod everything returns to previous state (as I described in fourth paragraph).

HELP ME PLEASE!

[torne]

Yes, the iPod Video exports its disk with a *logical* sector size of 2048 bytes. This makes repartitioning it complicated. You shouldn't try and do it by removing the disk unless you understand exactly what this entails - leave it in the ipod so that it gets exported with 2048 byte logical sectors.

One thing you should be aware of is that the "unallocated space" is not in fact unallocated: that's a real partition, containing the firmware. The data partition is partition 2. Partition 1 has partition type "none" which makes it appear to be nonexistant in most partitioning tools, but it does indeed exist and has a start/end location like any other partition. In theory you can move partition 1 around as well but you shouldn't really need to as it's never written to during normal operation and thus the performance issue is somewhat irrelevant - better to leave it entirely alone.

If you're going to modify the partition table you need to use a very low level tool such as sfdisk which will not mess up the entry for partition 1. Just move the start of the data partition to an even numbered logical sector and it will be at the start of a physical sector.

Edit: Also, don't try and do it while leaving the filesystem on the data partition intact; the odds of any partition resizing tool understanding the 2048 byte sector filesystem correctly and doing the right thing are pretty low Just accept that the filesystem will be destroyed when you change the partition table, and format it again with 2048 byte sectors using the command from the manual restore instructions on the wiki.

[ptrk.mj]

torne, thank you for your informing answer.

In theory you can move partition 1 around as well but you shouldn't really need to as it's never written to during normal operation and thus the performance issue is somewhat irrelevant

So the wrong partition alignment has no effect on read speed at all? Doesn't it slower the boot time (loading firmware from partition 1) for example? I suppose partition 1 is misaligned just the same way as data partition yet none of my tools is capable of showing it's starting/end sector.

If you're going to modify the partition table you need to use a very low level tool such as sfdisk

It's a part of Linux system, isn't it? Do you know any such tools for Windows environment? Or at least any bootable ones?

Just move the start of the data partition to an even numbered logical sector

How about the 'end of partition' sector? To make partition end at the end of the physical sector I would need to move it to an odd numbered logical sector. How important is that? (Probably not as much as 'start of partition' proper placement)

the odds of any partition resizing tool understanding the 2048 byte sector filesystem correctly and doing the right thing are pretty low

Probably, that's what GParted was trying to achieve to no success.

Just accept that the filesystem will be destroyed when you change the partition table

How about "system" files and directories on data partition? (iTunesControl, iTunesDB, music FXX catalogs) Are they going to be recreated by firmware/iTunes automatically? Or do I have to backup them?

...and format it again with 2048 byte sectors...

You were thinking 2048 byte clusters?

using the command from the manual restore instructions on the wiki.

Can you point me to that? I had trouble finding anything relevant.

[torne]

So the wrong partition alignment has no effect on read speed at all? Doesn't it slower the boot time (loading firmware from partition 1) for example? I suppose partition 1 is misaligned just the same way as data partition yet none of my tools is capable of showing it's starting/end sector.

The firmware is a single contiguous blob in that partition, there's no "filesystem" as such in there and the "files" cannot be fragmented. It is fractionally slower to read unaligned data, but the penalty only applies to the start and end of a contiguous read - you are effectively just reading an extra 2KB at one or both ends and throwing it away, which is a minuscule performance difference. The firmware is read as a single contiguous read, so this penalty only applies once. It's massively less of a problem than writing in any case.

If you're going to modify the partition table you need to use a very low level tool such as sfdisk

It's a part of Linux system, isn't it? Do you know any such tools for Windows environment? Or at least any bootable ones?

A hex editor?
Runtime Software's DiskExplorer can let you edit raw fields of a partition table but it's even less friendly than sfdisk (that's not really what it's for), and it's not free.
Any bootable linux system at all will have sfdisk in it. Note that sfdisk is *extremely hard to use* since it effectively has no user interface - you kinda have to dump the partition layout to a textfile, edit the file, then feed it back into sfdisk to apply the changes.

How about the 'end of partition' sector? To make partition end at the end of the physical sector I would need to move it to an odd numbered logical sector. How important is that? (Probably not as much as 'start of partition' proper placement)

Not at all. Whether the end of the partition is aligned is totally uninteresting.

How about "system" files and directories on data partition? (iTunesControl, iTunesDB, music FXX catalogs) Are they going to be recreated by firmware/iTunes automatically? Or do I have to backup them?

Everything gets recreated when you boot the original firmware. There's no need to involve iTunes unless you have music you want to resync on there. The firmware is all in the hidden partition.

...and format it again with 2048 byte sectors...

You were thinking 2048 byte clusters?

No, 2048 byte sectors. FAT filesystems contain a logical sector size indication as well, and this needs to be 2048 for everything on the ipod to work correctly, to match the logical sector size used when exporting over USB. The cluster size can be anything you like, as far as I know. The default is probably fine.

using the command from the manual restore instructions on the wiki.

Can you point me to that? I had trouble finding anything relevant.

http://www.rockbox.org/wiki/IpodManualRestore

[ptrk.mj]

I've made it!

I used PowerQuest Partition Table Editor to modify MBR (had to plug the disk directly since it doesn't support USB). After that iPod went crazy (entered a booting loop) so I had to put it in Disk Mode. Then formatted it with fat32format (Windows CLI utility). It had no problem detecting 2048 byte sector size. It defaulted to 32 KB clusters.

I also backed up modified MBR.

http://www.rockbox.org/wiki/IpodManualRestore

Great news! Finally, I can get rid of iTunes once and for all (assuming I figure out how to put firmware .ipsw file onto first partition).

Thank you for your guidance!

Best regards

[torne]

I was assuming you had already restored it with iTunes, which would've put the firmware in place for you. You shouldn't need to do this again after modifying the partition table if you left the firmware partition alone.

[ptrk.mj]

That is exactly what has happened. Future restore without iTunes is what I had on my mind.

[thegeek6]

kinda stupid question, but did it speed it up? i  got a 5.5gen with a 160GB Toshiba drive with 4K Sectors, and i was wondering about trying it.

[ptrk.mj]

Yes, it did. It's now working with pretty decent speed. If your partition indeed happens to be misaligned it's definitely worth it.

[thegeek6]

my next question is how can you re-align through usb?

[fennectech]

If you use Linux fdisk to write a brand new MBR to it then you can set the alignment manually. Then you can create the partition to start at a multiple of 8

[torne]

If you write a brand new MBR then you will likely end up screwing up the sector size and/or the firmware partition. It doesn't matter very much if the firmware partition is aligned (it's not very big and the data read from it is read sequentially), so it's safer to just edit the existing MBR and only change partition 2.

[xtr3me316]

I got the same 120gb hard disk , Can I request the step by step process you did. Thanks in advance.

[Reima]

I got the same 120gb hard disk , Can I request the step by step process you did. Thanks in advance.

I am interested in the step by step process as well.