I think the daily builds should be less dishonest about changes

[Jason Arthur Taylor]

Right now a compression package is apparently given flags to create daily [device][today'sdate].zip builds which are downloadable here: http://build.rockbox.org/ .  This date is today even if the last binary change was a long time ago.  This obfuscates if there was any change, and is bad because this information is useful if you are debugging and because it wastes bandwidth. For example, I wrote up a few bugs this week, and others do as well, but it is harder to know if anyone might have made changes to fix any of these many bugs. 

But, it is worse.  A flag given to the compression program apparently gives inside files in the daily build the compression date, even if the files inside are really old, and the same as before.  This obfuscates if there was a change.  I hereby propose not obfuscating if there was a binary change.  The date of the files should be the date of each was last changed (i.e., modified), and the overall file name should not be set to today's date.  Rather, it should be set to the newest modification date of the inside files. 

The "-f" "freshen: only changed files" zip option is what does this automatically, although the purpose of that option is largely to reduce cpu usage during the packaging of the compiled files.

I assume this proposal will not be allowed, as development is rather dead here, but I wanted to put this out there.

[pamaury]

Hi,
The daily builds are rebuilt on each commit and the commit log is publicily available (https://www.rockbox.org/recent.shtml#code and https://git.rockbox.org/?p=rockbox.git;a=summary) so I am not sure what you mean by binary change. Some commits may not affect a particular device and thus produce exactly the same binary but given than we have over 50 builds, there is no way we can track this by hand. I don't see what is dishonest about this.

[Jason Arthur Taylor]

First, the default is usually to NOT alter the file modification dates but to retain them.  So I think there was some effort to obfuscate that.  But whatever.

"there is no way we can track this [changes] by hand"

I think that is wrong in two ways.  First you are imply that I'm saying it should be done by hand.  I'm not.  Where did I say you should do anything by hand?

Second, there are an infinite number of ways to have accurate last-modified modification times, and many are very easy.

For example, if you don't want to do this via git, you can probably just use something like the -f option on zip or whatever the compression routine is and then none of the other zip files would have changed.  You could also just write a line to make a file renamer script.

So for example, the last commit changed only the files for the nwz-e470.  Actually it was one line of code.  So probably one file.  Then one downloadable would have a new date and only one file in that downloadable would have the commit date, since the -f option would only modify one file in that one zip.  Then, the date checker that puts the modification date in the file name would only rename one file.  Done.

I sometimes think that you mods spend more time here arguing that one cannot or should not improve rockbox than it would take to just do it.

[saratoga]

^^ we do actually have daily builds, although no one actually uses them :

https://www.rockbox.org/dl.cgi?bin=sansac200

I think he is suggesting changing the script to only generate a daily build on days in which there is a commit.

[saratoga]

So for example, the last commit changed only the files for the nwz-e470.  Actually it was one line of code.  So probably one file.  Then one downloadable would have a new date and only one file in that downloadable would have the commit date, since the -f option would only modify one file in that one zip.  Then, the date checker that puts the modification date in the file name would only rename one file.  Done.

Can you suggest an algorithm for determining what builds a given line of c code will alter?  This is actually a pretty hard thing to do correctly.

[pamaury]

Hi,
I am sorry but I think you really misunderstand how the whole building process is working. A one-line modification could change all the binaries in the zip. Many of the files in the zip do not actually exists in our repistory, they are the product of a build process (usually compilation). Thus the modification date makes no sense, because the file is never modified by itself. The only useful information is how this file is produced and the modification logs of the source files, which is exactly what git produces.

Also, just saying but you are asking for more work to provide very very little benefit (change dates in the ZIP) and at the same time implying that we should be doing something useful...

[Jason Arthur Taylor]

Just to clarify again I'm suggesting two changes.  One is the file name to avoid giving the false impression people are working very hard here.  The other is internal to the zip files to allow people to more easily know what is actually changed.  I know you can pontificate if a commit affects you, but that is not as easy, as was pointed out.

"Can you suggest an algorithm for determining what builds a given line of c code will alter?  This is actually a pretty hard thing to do correctly."

I already did.  As I said there are two main paths/ways to do the two changes.  Via git and via a script after git is done.  You seem to prefer git, but I can write only the latter.  (I have forgotten a lot of what I knew about git so I'm not the one to help you.  I'd guess it is an option to avoid compiling when all of the files used are the same as before.  You can just add an if condition before each compilation.  But I would be shocked if this isn't already a part of git and well known how to fix the dates and avoid wasted makes/compilation.)  You can for example have output of the compilations go to a directory called "future" and then use a zip -f freshen (or equivalent switch) to freshen the current zips recursively.  Then you can run a rename script to have only the changed zips get renamed.  I don't have, e.g., a password to your server, so that is about as much help as I can do here.  This isn't rocket science.  I know the bad news is people might not donate since they will see old files on the latest builds but it is for the best IMO.

[pamaury]

One is the file name to avoid giving the false impression people are working very hard here.

Personally, I stopped reading there. I usually don't start discussing with people by implying that they are lazy, especially when they do it for free on their spare time.

[Jason Arthur Taylor]

I could be wrong but this quote below (despite the question) implies that the default is for it do what I'm asking for:

https://git.wiki.kernel.org/index.php/Git_FAQ

Why isn't Git preserving modification time on files?

Modification time on files is a feature that affects build tools. Most build tools compare the timestamp of the source(s) with the timestamp of the derived file(s). If the source is newer, then a rebuild takes place, otherwise nothing happens. This speeds up the build process a lot.
Now consider what would happen if you check out another branch, and modification times were preserved. We assume you already have a fully-built project. If a source file on that other branch has a timestamp that is older than that of the corresponding derived file, the derived file will not be built even if it is different, because the build system only compares modification times. At best, you'll get some kind of weird secondary error; but most likely everything will look fine at first, but you will not get the same result as you would have with a clean build. That situation is unhealthy since you really do not know what code you are executing and the source of the problem is hard to find. You will end up always having to make a clean build when switching branches to make sure you are using the correct source. (Git bisect is another Git procedure that checks out old and new revisions where you need a reliable rebuild.)
Git sets the current time as the timestamp on every file it modifies, but only those. The other files are left untouched, which means build tools will be able to depend on modification time and rebuild properly. If build rules change, that can cause a failure anyway, but that is a far less common problem than accidentally not rebuilding.

[pamaury]

All builds are clean, we restart every single build from scratch on each commit to avoid any problem. There are good reasons to do so and thus the all the files have almost the same timestamp since they are all created during the build.

[__builtin]

One is the file name to avoid giving the false impression people are working very hard here.

Insulting the contributors is not a good way to get things done.

[saratoga]

I could be wrong but this quote below (despite the question) implies that the default is for it do what I'm asking for:

That just tells you if source code files changed (which is trivial to figure out with git anyway), but not if binaries (which of course are not in git) have changed.  That is the basic problem here.  It is not trivial to figure out if a build has changed based on source code alone. 

Anyway, rather than speculate about what could be done, why don't you look at the build system and see what will work yourself?  The whole thing is in git, so you can check it out and experiment.  I think you will find that it is not so easy to do what you want, but I could be wrong.  If you figure out a way, post the source changes to gerrit. 

[bluebrother]

This date is today even if the last binary change was a long time ago.  This obfuscates if there was any change, and is bad because this information is useful if you are debugging and because it wastes bandwidth.

Who claimed that this date has anything to do with code changes? Who claimed that this date is anything else but the date the binary has been compiled?