Is this being done?
Not for each individual file, no. Though the whole release package has md5sums generated for it and uploaded, as do all plugins.
(This was added in 6512)
Oh. I wanted one for every file. Like this:
has an extension of this - its a combo file for all files on all packages:
This lets the TWiki:Plugins.TWikiReleaseTracker
work out whether anyone has fiddled with any file in the install.
It keeps a central database of correct MD5s, of all releases, betas and all plugins, so you know exactly where your files come from.
- It knows which files have been tampered with.
- It knows what files should be there, so you know whether files have been added or deleted.
- It also matches by MD5, so you know which files have been renamed.
That all said, I don't particularly need it to be shipped with every install. I just need to keep a central database updated, so having a trigger notifying me is sufficient.
It took about a month's solid effort to build - it would be good to put it to use. I think it could really help with security.
I know what you wanted, but unless someone else puts in the effort to
- write a proper spec for this, and
- code up building such a file from the build scripts,
it won't happen, because I'm not going to do it.
The spec: very roughly, it looks like what I wrote above. A line for each file. Filename MD5. Optional packagename= before the file name. Not much more to say.
I built the code about a year ago. TWikiReleaseTrackerPlugin
/lib/TWiki/Plugins/TWikiReleaseTrackerPlugin/IndexDistributions.pm, called from ./build.pl
After I've squashed the user Item452
I'll see if I can make it work again.
But where in the distro should I put this package? TWiki::Contrib? It creates an MD5 list for any tree.
It seems a shame to turn it back into a script just so to squish it into the tools dir.
We are not
shipping the TWikiReleaseTracker
, or any part thereof, in the release. The maximum that should be added to the release package is one
file, that contains the md5sums for the entire release tree. I don't see a need for additional code to generate that file, since it can easily be generated using md5sum e.g.
md5sum `find . -type f -print` > <project>md5sums.md5
this file needs to be generated after the release is compiled into the staging area, and before the zips are built. The file name must be chosen not to conflict with the .md5 file, that contains the md5s for the zip and tgz.
Note that there are two places the sums could be generated; in tools/build.pl, in which case a single file is generated for the entire release; or in Build.pm in the BuildContrib, where they will automatically be generated for any plugin built using the BuildContrib (which these days is all of them). In either case, the generated filename must be chosen so that the md5sums from different packages don't overwrite eachother. My preference is for the latter (I prefer generic solutions).
I don't care at all how the md5sums are made. I was simply pointing out a pure perl way to generate them.
Manic projects permitting, I'll take a look over the next couple of days.
Ok. I've generated an index against every package in twikiplugins, these are saved in /md5sums.
If you think these are useful we can work out where to best put them and how to keep them up to date.
If useful, I suggest we move the md5sum generation code into its own contrib package. It provides a facility to filter which files get in the file, it does this by walking a tree. Arguably we'd want to have it read the initial list from the MANIFEST file.
I've not generated for TWiki install packages though I have done so in the past.
results are in r6728
My code to build the indices is an uncommitted change to the TWikiReleaseTrackerPlugin. (the Config for which for DakarRelease can be markedly simplified).
I've made those sys_action changes to tools/build.pl, plus done a lot to tidy the messaging.
More importantly I've built the MD5 stuff I wanted into BuildContrib.pm and tidied the resulting /tmp directory, but am hestitant to check it in as it is a critical part of all builds.
I currently generate a package name.md5 that contains, e.g.
And a DEPS.md5 (temporary name) that contains MD5s from dependencies, e.g.
As you know, the current SVN HEAD has simply a TWiki.md5 containing, e.g.:
I think I need two names to for the MD5s, one for your package level security, and another for my file level security (containing both the package files and its deps). Do you agree?
If so, what should these two be called for any given package, p?
Alternatively I could check my changes in. I'd put it in scratch but I am too tired right now.
PS. It seems that it builds up the hierarchy of deps and then unlocks the files listed in the main manifest. I suspect this means that the plugins topics are not unlocked.
After checking things manually, things seem to work. No test case I could see. So...
twiki$ svn commit -m "Item437: Revised BuildContrib to build MD5s. Cr to ditch anything you don't like. (I sent you email about this a couple of days ago). MD5 files are generated for each package and then aggregated during a hands-off install to collect all dependent MD5s into package/DEPS.md5" twikiplugins/BuildContrib/lib/TWiki/Contrib/Build.pm
Transmitting file data ..
Committed revision 7207.
Reverted in 7214 and 7215.
Undeferred, post Dakar CC
There is no infrastructure to do this, or use the sums if they exist, so that would have to be developed.
Cleared checkins - they are all against the TWiki SVN repo. Otherwise shows up as a work in process.