Will 160GB HDD (MK1634GAL) work in H10?

[p.h.]

I am considering replacing Toshiba MK6008GAH (60GB) with MK1634GAL (160GB).

Do you know if the MK1634GAL does work with H10?

[[Saint]]

If I remember rightly, the 60GB drive you have is a single platter, 5mm model. The 160GB is a dual platter, 8mm model. 3mm may not sound like a lot of difference, but it might just be the deciding factor as to whether or not you'll be able to fit it into the enclosure or not (for example, the iPod 5G Video needs a "fat" backplate to accommodate the larger capacity drives).

I would definitely do some research to see if anyone else has achieved this modification, and how, before committing to purchasing the drive. But there are other options available, however, they can be much more expensive. For example, you could use an adapter and a Compact Flash card, or even an adapter and an mSATA SSD, but both of those options require the same forethought regarding space constraints also.

Good luck, let us know how you get on, and please do post regarding your decisions and process if you are successful.


[Saint]

[p.h.]

The 60GB MK6008GAH is 2 platter, 8mm model. Original Hitachi was 7mm thick. The 1mm difference resulted in very tight fit and high tension.

The 160GB MK1634GAL is 1 platter 5mm model (a few mm shorter too). The problem may be it has 4kB sector size http://www.semicon.toshiba.co.jp/product/storage/pdf/hdd18_34.pdf.

As far as I know I would have to use custom bootloader. Futhermore it could turn out not to be enough. According to Rockbox Hard Drive Replacement: "Drives with Large Physical Sectors that do not support unaligned reads and unaligned writes transparently in the drive firmware may not work in certain devices." http://www.rockbox.org/wiki/HardDriveReplacement

So at the moment I know that I do not know:
- if there is appropriate bootloader for H10 and if so where to look for it,
- whether or not MK1634GAL supports unaligned R/W.
Any help on the matter would be appreciated.

This post is my attempt to do some research before buying the HDD. I thought someone could have done this before me and may share some valuable findings.
As to the CF and mSATA mods - they are way too expensive. Both would be tempting if only priced favorably. Not yet.
--
p.h.

[p.h.]

After a long time of wondering and lingering to buy any bigger drive I have forgotten, that buying the MK1634GAL could be risky, and so I have bought it.
The drive works just fine , although it seems it is a bit slower than the MK6008GAH was, despite the fact MK1634GAL itself is much faster than MK6008GAH.

MK1634GAL is faster than USB2.0 allows, i.e. >30MB/s (read speed tested using USB case). Knowing this it's a real pity, that DMA patch was abandoned. (It was working, hugely improved transfer rates, but it had a bug, which render it unstable)

Having so big drive I have tried to create 2 partitions - the 1st for the music and the 2nd for large file storage, like movies etc. Unluckily NTFS, exFAT and EXT3 2nd partitions were not normally accessible in Windows 7. They were seen as raw space. Only disc image creating tools, like Acronis True Image could read files on the 2nd partition.

[dreamlayers]

The drive works just fine , although it seems it is a bit slower than the MK6008GAH was, despite the fact MK1634GAL itself is much faster than MK6008GAH.

MK1634GAL is faster than USB2.0 allows, i.e. >30MB/s (read speed tested using USB case). Knowing this it's a real pity, that DMA patch was abandoned. (It was working, hugely improved transfer rates, but it had a bug, which render it unstable)

The PP502x DMA patch is now being used on most PP502x targets. In firmware/export/config/iriverh10.h I see "#define HAVE_ATA_DMA", which should enable it. UDMA 4 is the maximum supported but I believe UDMA 2 is used because higher speeds require boosting. You could try editing firmware/target/arm/pp/ata-target.h to enable UDMA 4. It should automatically cause boosting when needed and work.

The problem that was preventing DMA in the past was an error in the PP502x cache, which was causing data corruption. There is a workaround for that in place now.

Did you align your partitions on a 4k boundary (multiples of 8 in terms of 512 byte sectors)? I'm using HS12YHA in my 5G iPod. I aligned partitions manually in fdisk, and performance doesn't seem significantly worse than MK3008GAL.

[p.h.]

Thanks for your reply. I haven't knew this.

I remember that MK6008GAH with DMA had transfers about 18-20MB/s, and without about 8-11MB/s. The new MK1634GAL achieves roughly 6MB/s.
Maybe I have unaligned partition? I will have to check it when only I will find out how.

I will turn on UDMA4 and see how it will improve data transfers.

[p.h.]

I've had unaligned partition. After aligning the partition and recompiling RB with UDMA4 enabled for read, the transfer rate get back to usual level of 8-9MB/s (write) and about 13MB/s (read).
Still much slower than I remember it was possible. I think FS#9621 is the missing link. I might have been using it when I have achieved the highest speeds.
I guess not only UDMA is needed for really fast transfers, but some wise buffering, like FAT prefetch.

How to configure higher UDMA mode for writing?

One more thing - in my case after abrupt power down of a drive with unaligned partition huge amount of crosslinked files appeared. Eventually I have had to clear the partition and start from scratch.

[p.h.]

I am wondering why USB transfers are slower than they could be. I've found my old (10-15.03.2009) UDMA tests performed using test_disk plugin.
For example for UDMA1 mode boosting significantly increases write rates.
1. Unboosted:
Create (1048576,A): 9568 KB/s
Write  (1048576,A): 10839 KB/s
Read   (1048576,A): 14431 KB/s
2. Boosted:
Create (1048576,A): 14014 KB/s (+46%)
Write  (1048576,A): 17713 KB/s (+63%)
Read   (1048576,A): 12730 KB/s (-11%)

Now I have RB built with UDMA4 mode for read. I get 8-9MB/s read speed.

I have tried to force boost from Debug menu -> CPU clock. It has set 80MHz, but either it was unboosted for the time of a USB connection or simply this was ineffective.
At the time when the UDMA patch was developed I've had applied a few other patches that might have contributed to the overall higher USB speeds.
They were: "buffer_cacheline_alignment_v2" #9708, "chained-tds-3" #10015 (accepted), "dma_cacheline_alignment_v3" #9708 and "fatpreload-v0.3" #9621 (undecided).

What I would like to check:
- transfer speeds with UDMA enabled for write (how to enable it?)
- transfer speeds with boost enabled (how to enable it?) or make sure somehow manual boost was been effective.

What may be a bottleneck for USB transfers?

[dreamlayers]

I'm pretty sure that USB is the bottleneck. If you want to test disk I/O speed, use test_disk. (When running configure, select an (A)dvanced build and then (T)est plugins.)

In firmware/usbstack/usb_storage.c you can #define USB_USE_RAMDISK to see what USB by itself is capable of. Reading an entire 4 MB ramdisk, I couldn't get more than 10.1 MB/s.

With test_disk, you're not necessarily testing DMA! The "aligned" tests use 32-bit word alignment, but DMA requires cache line alignment to prevent cache line interference. I just uploaded a change addressing this: http://gerrit.rockbox.org/r/#/c/1187/

Also note that test_disk only shows you the CPU frequency at that instant. You might see 80 MHz due to temporary wheel boost. Ensuring boosting by increasing the boost counter via the debug menu works. I used values like 6 or 7 just to be extra sure. My results with a HS12YHA are below. The drive is almost full so this is using space near the end, but there shouldn't be much fragmentation. Some observations:

• 512 byte access performance is abysmal, but in actual use I don't notice significant problems
• PIO reads are slow, even at 80 MHz. DMA makes reads much faster
• PIO writes are fast. They were faster than DMA so DMA was disabled for writes. DMA for writes should be enabled if you #define ATA_DMA_WRITES, and it should work. It might be beneficial because the CPU is free to do other stuff during DMA transfers.

Test:	30 MHz	80 MHz	30 MHz, align	80 MHz, align
--------------------
Create	4	5	5	6	files/s
Open	2170	2193	844	2030	files/s
Dirscan	18628	45111	17102	47515	files/s
Dirscan w info	14841	39472	14855	41309	files/s
Delete	5	5	5	6	files/s
--------------------
Create (512,A)	15	15	15	15	KB/s
Write  (512,A)	15	15	15	15	KB/s
Read   (512,A)	1006	1262	838	1258	KB/s
--------------------
Create (512,U)	15	15	15	15	KB/s
Write  (512,U)	15	15	15	15	KB/s
Read   (512,U)	891	1377	1010	1377	KB/s
--------------------
Create (4096,A)	4420	7776	4414	7777	KB/s
Write  (4096,A)	4505	8066	4536	8084	KB/s
Read   (4096,A)	2215	4991	5343	9542	KB/s
--------------------
Create (4096,U)	4429	7821	4382	7785	KB/s
Write  (4096,U)	4552	8063	4509	8084	KB/s
Read   (4096,U)	2136	4922	2141	4911	KB/s
--------------------
Create (1048576,A)	6819	17697	6819	17782	KB/s
Write  (1048576,A)	6916	18225	6930	18237	KB/s
Read   (1048576,A)	2544	6777	18796	24045	KB/s
--------------------
Create (1048576,U)	6840	17764	6833	17764	KB/s
Write  (1048576,U)	6923	18218	6923	18255	KB/s
Read   (1048576,U)	2437	6490	2437	6493	KB/s

[p.h.]

My tests results: https://infogr.am/rockbox_h10_mk1634gal_160gb_hdd_test_results

USB transfer speed is, indeed, a bottleneck here. In my case it is about 7MB/s. As I've mentioned above one of the patches I have applied when UDMA was being developed very significantly improved USB transfers.

Let me assume partition in aligned. Correct me if I am wrong, but:
1. For HDDs of >=20GB capacity FAT32 won't use 512B blocks as clusters, which means it is justified to ignore results for them.
2. In a typical usage scenario, most of the time songs are read and written, so results for 1MB data blocks are the most important here. Having a large HDD means a lot of space to use for files, which encourages to use it as a portable drive. It means read speed should be treated on par with write speed - the less time is needed to store and read files, the better.
When you consider the above it becomes clear a weighted average should be used to judge, which mode is better in general.

Enabling DMA for write.
Read speed. Excluding a few exceptions enabling DMA for write has no or minor influence on read speed (as expected). Exceptions are:
- +14% increase in read speed for UDAM4 @30MHz and 1MB data chunks,
- -14% decrease in read speed for UDAM4 @30MHz and 4kB data chunks,
- -23% decrease in read speed for UDAM4 @80MHz and 4kB data chunks.
Write speed.
- for current default - UDMA1-W @30MHz results are clear - significant improvement for 4kB and 1MB data blocks and neglectable -1% decrease for 512B blocks (test procedure's error is probably higher than this),
- UDMA4 @30MHz results in a 118% write speed improvement (34% when compared to UDMA1+W), but at a cost of -39% decrease for 4kB blocks and -81% for 512B blocks,
Further increase in the write speed can only be made with the help of boosting.
- UDMA1 and UDMA4 @80MHz brings almost the same write speed improvement over UDMA1-W @30MHz - 145 vs 147%, but for 4kB blocks UDMA4 outperforms UDMA1 - 222% vs 139%.
With DMA disabled for write results are just as one would expect them to be - changing UDMAx-W mode at the same speed makes no practical difference (DMA is not used for write).

Read speed. As the write speed results have shown enabling DMA for write is beneficial I will focus on the results for tests with DMA enabled.
- the best result was achieved for UDMA4+W @30MHz; note that with the help of boosting UDMA4 gives the best write results too, and without it the best result for large data blocks,
- UDMA4-+W @30MHz yields -50% decrease in 4kB blocks read speed and -65% in 512B blocks,
- what's surprising UDMA4+W @30MHz was 39% faster than @80MHz.

Assuming no USB bottleneck - in my opinion the best would be to use UDMA4+W @30MHz and boost it for write. But it will make sense only, when USB won't be the bottleneck any longer.
Otherwise UDMA1+W @30MHz would do just fine.

                    UDMA1-W@30M    UDMA1-W@80M    UDMA1+W@30M    UDMA1+W@80M    UDMA4-W@30M    UDMA4-W@80M    UDMA4+W@30M    UDMA4+W@80M
Test no.:           11             12             13             14             15             16             17             18
CPU clock:          30000000       80000000       30000000       80000000       30000000       80000000       30000000       80000000
Create:             4              4              4              4              4              4              4              4
Open:               1304           3473           1335           3519           1342           3548           1343           3523
Dirscan:            19053          51469          19148          51641          19212          51896          19242          51934
Dirscan w info:     16327          44087          16407          44277          16482          44563          16464          44537
Delete:             4              4              4              4              4              4              4              4
Create (512,A):     174            2995           1536           2946           1615           2975           310            3244
Create (4096,A):    3471           7802           4642           9115           3551           7781           2213           11200
Create (1048576,A): 2943           9511           6974           11502          1016           9513           2538           11444
Create (512,U):     1544           2974           1518           2977           1575           2998           1624           2959
Create (4096,U):    3412           7721           2265           7511           3403           7763           3539           7565
Create (1048576,U): 5044           9708           5039           8677           5094           9581           5059           6540
Write  (512,A):     1644           3357           1627           3352           1804           3358           316            3682
Write  (4096,A):    4050           11576          5759           9683           4120           11579          2481           13030
Write  (1048576,A): 5369           9872           10093          13269          5565           9767           11680          13160
Write  (512,U):     1634           3326           1638           3352           1807           3324           1800           3322
Write  (4096,U):    3995           11576          3447           11128          4022           11535          4125           11236
Write  (1048576,U): 6522           9997           6530           15515          6564           9949           6564           10832
Read   (512,A):     1056           2286           1065           2293           367            2626           365            2574
Read   (4096,A):    5663           8631           5661           7402           2814           15257          2787           11824
Read   (1048576,A): 13139          11038          13306          10999          26947          22146          30720          22067
Read   (512,U):     1415           3319           1414           3333           1448           3422           1453           3425
Read   (4096,U):    2160           5649           2158           5664           2162           5679           2166           5681
Read   (1048576,U): 2313           5505           2312           6151           2318           5519           2318           6147