Creative Zen Vision:M

[GuybrushThreepwood]

You only need to svn username and password to commit.  To check out source you don't need anything.

http://www.rockbox.org/twiki/bin/view/Main/UsingSVN

Thanks for your reply. I've downloaded the sources: it was my mistake; to download no username or password is required. Now that I have all the right files, I would try to use cmake again then let you know. Excuse me for the dumb questions but I'm quite new to these things... 

[mcuelenaere]

Thanks for your reply. I've downloaded the sources: it was my mistake; to download no username or password is required. Now that I have all the right files, I would try to use cmake again then let you know. Excuse me for the dumb questions but I'm quite new to these things... 

Instead of directly downloading through SVN, you could've also downloaded the most recent source tarball..

Also it is possible you'll have problems with update_extract recognizing the correct offset; I've got a fix for it ready but as I'm not at home I can't commit it now (though this doesn't affect ZVM v.1.62.02 AFAIK).

For help compiling, don't forget to read utils/zenutils/notes.txt.

p.s. Is there a manual way alternative to using update extract to extract the firmware from the package executable?

Yes there is. While upgrading your ZVM, copy C:\CtJbFW\cttemp\nk.bin to a safe place.

[GuybrushThreepwood]

Instead of directly downloading through SVN, you could've also downloaded the most recent source tarball..

Yes, you're right: I've realized this reading the guide on the wiki but this isn't the problem because now I have no problems using the SVN.

For help compiling, don't forget to read utils/zenutils/notes.txt.

Thanks. I've read it and followed the instructions.

p.s. Is there a manual way alternative to using update extract to extract the firmware from the package executable?

Yes there is. While upgrading your ZVM, copy C:\CtJbFW\cttemp\nk.bin to a safe place.

I already know this trick: it's described on many forums and I've already used it to extract the 1.62.02 nk.bin. The problem is that I've a faulty Zen and on some forums I've read that flashing back to 1.30.02 could help. Though I don't believe this, I'd like to make a try but the 1.30.02 updater doesn't even recognize my player (this isn't the problem with the player which is instead recognized by the other updaters, the problem is another) and because of this doesn't extract the firmware and I can't use that trick.
Updater 1.30.02 is among those that don't work with update_extract and I can't succeed in compiling the revised version. Here's what I did:
- downloaded the sources from the SVN
- set up the build environment for Watcom (I use open-watcom-c-win32-1.7a) by cmake
but, when I try to launch wmake, I get a lot of errors and can't go on any further...
I certainly miss something (as said, I'm quite new to these things) though it seems to me that everything is fine (obviously the situation is different: my environment has certainly some kind of issue)...

[mcuelenaere]

...

I already know this trick: it's described on many forums and I've already used it to extract the 1.62.02 nk.bin. The problem is that I've a faulty Zen and on some forums I've read that flashing back to 1.30.02 could help. Though I don't believe this, I'd like to make a try but the 1.30.02 updater doesn't even recognize my player and because of this doesn't extract the firmware and I can't use that trick.
fw 1.30.02 is among those firmwares that don't work with update_extract and I can't succeed in compiling the revised version. Here's what I did:
- downloaded the sources from the SVN
- set up the build environment for Watcom (I use open-watcom-c-win32-1.7a) by cmake
but, when I try to launch wmake, I get a lot of errors and can't go on any further...
I certainly miss something (as said, I'm quite new to these things) though it seems to me that everything is fine (obviously the situation is different: my environment has certainly some kind of issue)...

I'm sorry, but this forum (and this thread) is dedicated to porting Rockbox to the Zen Vision (:M), not helping people fix their player.

And I've never tested Watcom, but as I heard MingW on Windows requires some patches to the source I can imagine open-watcom won't compile zenutils magically without changes...

[GuybrushThreepwood]

I'm sorry, but this forum (and this thread) is dedicated to porting Rockbox to the Zen Vision (:M), not helping people fix their player.

And I've never tested Watcom, but as I heard MingW on Windows requires some patches to the source I can imagine open-watcom won't compile zenutils magically without changes...

I understand, in fact I've written about update_extract (I see that many other people did this in this thread) and not about the issue with my player: I don't think I have done something wrong or at least I hope and I think that many other people would like to learn how to compile Rockbox and the tools related to it. At least, this is what I'd like to do. Thank you anyway.

p.s. If the code has to be modified to be compiled under Windows, how was the version on pimpmyzen compiled? Was it cross-compiled?

p.p.s What I meant by "manual way to extract the firmware" was asking what's the updater file structure and how to manually extract the firmware from it.

[quetzalcoatl]

This is as far as I get now, but I'm stuck at the cluster chain's stuff.

I can't figure out what the structure of the cluster chain is, where it's located and how I need to find the data that belongs to the file attached to the cluster..

i've just put all the current notes into the root/notes on the SF

what you need most now is notes/minifs/fs.txt:
- remember that 'sector' = 512bytes for the whole drive
- remember that 'cluster' = 8 sectors for the whole drive

line 13: the layout of the volume. the cluster IDs in second column are for yours drive. those are clusters of the VOLUME, not drive. minifs volume tends to start at drive's sector number one, as its seen in the volume table in MBLK

line 24, 26: structure of the "cluster chain". a single cluster chain is a structure consisting of some data and the 'chainlist' (a list of _logical_ clusterids that describe the file's contents' location). the cluster chain looks like:
- uint32 unknown
- "chainlist", the list of uint16, constant-length, 0xFFFF-terminated, padded with 0x0000 to the end
- uint32 unknown2
- uint32 number of entries on the 'chainlist'

warning: the whole 'cluster chains' part of the volume is a sequential list of consecutive clusterchains. placed one by one, from the first to the last, so finding the right clusterchain is relatively easy. BUT the size of the clusterCHAIN is not chosen well by the designers, it doesnt multiplicate to the size of cluster. the easiest way to find the right chain is to calculate its address in terms of BYTES on the drive (not secotrs, not clusters) and then translate the address to clusters/sectors.

a "logical clusterid" is a cluster ID offsetted by the beginning of the volume's dataspace. if the file's clusterchain says that a logicalclusterid is 0x0123 and the minifs's dataspace starts at cluster 0x4321, then the REAL in-volume clusterid is 0x4321+0x0123 = 0x4444, and there you should look for the file's contents

[ and the in-volume clusterid = 0x4444 of course points to the drive's physical sector = 0x4444 * clustersize + minifsstart  = 0x4444 * 8 + 1]

line 39: the structure of the directory/direntry
blahblah, please se the notes..
the most important is that, there are 2 identical numbers - and those are the file's clusterchain's numbers. i don;t know why two of them, they're always the same, so whatever.

mind that that all names are in char on yours volumes, not wchar_t as the notes may state!

[GuybrushThreepwood]

Hi! I've succeeded in building the zenutils sorces under Linux (it was very easy and I haven't found any problem): it's clear that something is wrong with my Windows build environment. I would do my best to find a solution to this and, if I would succeed, I would  report the results here. I hope it could be useful to someone else who could want to do the work under Windows.
By now, I've to report a problem: even the newer version of update_extract fails to extract the firmware from the updater package; perhaps the problem causing this issue isn't still resolved, the strange thing is that, as said, it works on some updater packages and doesn't work on others.
If there is something more I could do, I'm free to help!

p.s. As said in my last post, I hope that I haven't done anything wrong writing again here on this thread, instead I hope that sharing the results of my tests could be useful.

[mcuelenaere]

@quetzalcoatl:
Thanks for the notes! I'll look into it sometime this week..

...

p.s. If the code has to be modified to be compiled under Windows, how was the version on pimpmyzen compiled? Was it cross-compiled?

No, it was compiled using VS2005.

p.p.s What I meant by "manual way to extract the firmware" was asking what's the updater file structure and how to manually extract the firmware from it.

The structure of it isn't really described somewhere as it's more a binary Win32 program containing two binary scrambled & compressed binary blobs placed at a changeable spot.

But if you would like to really know it:
  * the nk.bin data starts at the first DWORD in the .data section; the DWORD is the size of the compressed/scrambled blob and is preceded with some other data (you can recognize it because after the DWORD there won't be any 0x00 anymore)
  * the key will need to be found in the other sections and is recognizable by the prefix 34d
  * the descrambling algorithm is fixed and documented in zenutils
  * the descrambled blob can be decompressed using standard Zlib

...
If there is something more I could do, I'm free to help!

You could always help porting Rockbox

p.s. As said in my last post, I hope that I haven't done anything wrong writing again here on this thread, instead I hope that sharing the results of my tests could be useful.

No, it was just a little warning in order to not convert this thread into some kind of 'Fix my ZVM!' help thread

edit:
I just committed some fixes for ZenUtils, try recompiling and see what it gives..

[GuybrushThreepwood]

@mcuelenaere

I've updated my copy of the SVN: tomorrow I would build again the sources and see what happens then let you know. Tomorrow I would try to set up a better Windows environment too...
I've opened the updater executable with a PE Editor and the .rdata section is only 60Kb worth while the .data section is a lot bigger (a little less than the entire package dimension): isn't the firmware (scrambled and compressed as you've said and I've expected) supposed to be in the .data section? BTW I have to go deeper into this...

[mcuelenaere]

@mcuelenaere

I've updated my copy of the SVN: tomorrow I would build again the sources and see what happens then let you know. Tomorrow I would try to set up a better Windows environment too...
I've opened the updater executable with a PE Editor and the .rdata section is only 60Kb worth while the .data section is a lot bigger (a little less than the entire package dimension): isn't the firmware (scrambled and compressed as you've said and I've expected) supposed to be in the .data section? BTW I have to go deeper into this...

Sorry, I was wrong. You're right It is the .data section; the info I gave you wasn't double checked.

For example: ZENVPlus_PCFW_L22_1_32_01.exe :
   * the data is in .data; starting exactly at 0x5F0C4; while it's length is at 0x5F0C0 (or relatively to start of .data: 0x0C4 & 0x0C0)
   * the key (34d12E9276d9bB6796efe6457749886) is also in .data

[GuybrushThreepwood]

@mcuelenaere

Hi! I've built the revised version of zenutils: everything is fine now and it extracts the updater packages v1.62.02 and v1.30.02 too. Great work! Where was the problem?
Unfortunately, I haven't still succeeded in setting up a working environment under Windows: I've tried with MinGW but, though the build process with cmake succeeds, when I try to make the executables I still get many errors... As soon as I would have some spare time, I would try with Microsoft Visual Studio 2008 Express... Have a good day!

---EDIT---

I've tried to flash the v1.62.02 and v1.30.02 firmwares extracted by the newer update_extract version to my ZVM: both CreativeWizard and the hacked firmware updater succeed in flashing the 1.62.02 version but not the 1.30.02 (exactly the one I was interested in ).
This means that update_extract correctly extracts the firmware file (otherwise the firmware v1.60.02 shouldn't be flashed too) but I don't understand what's wrong with the firmware v1.30.02. I've noticed that the file size is the greatest (21,4Mb) among all the firmwares I've seen: perhaps the problem is there and CW nor the hacked updater can handle such a big firmware file (maybe they truncate the firmware file...).
I've tried also the older versions of CW but they don't even recognize my player (though CW v0.8.4.0, Windows and Creative Media Explorer do).

[mcuelenaere]

@mcuelenaere

Hi! I've built the revised version of zenutils: everything is fine now and it extracts the updater packages v1.62.02 and v1.30.02 too. Great work! Where was the problem?
Unfortunately, I haven't still succeeded in setting up a working environment under Windows: I've tried with MinGW but, though the build process with cmake succeeds, when I try to make the executables I still get many errors... As soon as I would have some spare time, I would try with Microsoft Visual Studio 2008 Express... Have a good day!

---EDIT---

I've tried to flash the v1.62.02 and v1.30.02 firmwares extracted by the newer update_extract version to my ZVM: both CreativeWizard and the hacked firmware updater succeed in flashing the 1.62.02 version but not the 1.30.02 (exactly the one I was interested in ).
This means that update_extract correctly extracts the firmware file (otherwise the firmware v1.60.02 shouldn't be flashed too) but I don't understand what's wrong with the firmware v1.30.02. I've noticed that the file size is the greatest (21,4Mb) among all the firmwares I've seen: perhaps the problem is there and CW nor the hacked updater can handle such a big firmware file (maybe they truncate the firmware file...).
I've tried also the older versions of CW but they don't even recognize my player (though CW v0.8.4.0, Windows and Creative Media Explorer do).

Try opening it with CreativeWizard and see if it can recognize the format...

Then try calculating the checksum and see if it's correct with the one in the NULL block

BTW why can't you just use the original fw updater which contains v1.30.02 ?

[GuybrushThreepwood]

Try opening it with CreativeWizard and see if it can recognize the format...

It recognizes and opens the firmware.

Then try calculating the checksum and see if it's correct with the one in the NULL block

I don't know if I've calculated the checksum well (I've used CW with the default key 'CTL:N0MAD|PDE0.DPMP.') but the checksum doesn't correspond to that in the null block while it corresponds for the other firmwares I've tested. What's wrong??? I've thoroughly tested update_extract and I'm almost sure that now it works as expected so where is the problem?

BTW why can't you just use the original fw updater which contains v1.30.02 ?

Because, as said, my ZVM is faulty and, among the other issues I'm working on, it doesn't correctly reports the battery charge level and the v1.30.02 updater package hangs up on the low battery warning (it's impossible to ignore it like with the newer versions also when the player is connected to the AC adapter).

p.s. You haven't told me what was wrong with the older update_extract version. I'd like to know it if you want. Have a good day!

[mcuelenaere]

It recognizes and opens the firmware.

I get the same result..

I don't know if I've calculated the checksum well (I've used CW with the default key 'CTL:N0MAD|PDE0.DPMP.') but the checksum doesn't correspond to that in the null block while it corresponds for the other firmwares I've tested. What's wrong??? I've thoroughly tested update_extract and I'm almost sure that now it works as expected so where is the problem?

I also get the same result with CreativeWizard, but if you try zen_crypt it reports the checksum is correct.. (keep in mind CreativeWizard is my code & ZenUtils is zook's; so there could be some mistakes in mine)

I also decrypted FRESC (v.1.30.02) and checked for the checksum key and it's exactly the same; so that can't be the problem either.

Because, as said, my ZVM is faulty and, among the other issues I'm working on, it doesn't correctly reports the battery charge level and the v1.30.02 updater package hangs up on the low battery warning (it's impossible to ignore it like with the newer versions also when the player is connected to the AC adapter).

Try using other/older versions? I found this site.

p.s. You haven't told me what was wrong with the older update_extract version. I'd like to know it if you want. Have a good day!

The problem was the find_firmware_offset() routine, like I explained previously.

[GuybrushThreepwood]

I also decrypted FRESC (v.1.30.02) and checked for the checksum key and it's exactly the same; so that can't be the problem either.

What do you mean by "FRESC"? Excuse me for the dumb question! I'm wondering if CW checks the correspondence between the checksum and the checksum stored in the NULL block and, if the check fails, doesn't go on with uploading the firmware... I think it could be a useful check but I don't know if you've included it: that's why I'm just guessing.
Maybe even the hacked updater makes that check... I don't know.

Try using other/older versions? I found this site.

It's exactly the place I get older firmwares from! All the older firmwares have that check on the battery level which isn't possible to ignore...

The problem was the find_firmware_offset() routine, like I explained previously.

Thanks for letting me know!