Priority: Normal
Current State: Closed
Released In: n/a
Target Release: n/a
Storable::freeze()
and
Storable::store()
are platform dependent. If a Foswiki install is migrated from
ia32
to
x64
, the filehash files cannot be read on the
x64
system.
Byte order is not compatible at /usr/lib64/perl5/vendor_perl/5.16.3/x86_64-linux-thread-multi/Storable.pm line 380,
Byte order is not compatible at /usr/lib64/perl5/vendor_perl/5.16.3/x86_64-linux-thread-multi/Storable.pm line 380,
at /var/www/foswiki/lib/Foswiki/Plugins/DirectedGraphPlugin.pm line 1065
The solution is to use the
nstore()
and
nfreeze()
functions which use network order and are portable.
For an existing DGP install, either remove all of the
working/work_areas/DirectedGraphPlugin/*-filehash
files, or on the source system, use a small perl utility to read in each file using
thaw()
or
retrieve()
and then write them back out using
nstore()
or
nfreeze()
.
--
GeorgeClark - 15 Dec 2013
There are a few more plugins that make use of
Storable
, i.e.
DBCacheContrib. For maximum performance it generates caches in native
store/freeze
format. I have no real numbers comparing native vs cross-platform store formats, but compatibility for sure comes at a certain price. The trade off is between cleaning up
working/*
once as part of the migration process or having to pay the extra performance penalty (how large it ever might be) on a daily use. I actually prefer the former for
DBCacheContrib as
Storable
is being used quite heavily. For
DirectedGraphPlugin it might be okay as its use of
Storable
isn't as critical, i guess.
--
MichaelDaum - 16 Dec 2013
I'll check in a small perl script that will rewrite the
-filehash
files using nstore. This could be used in a migration. Run utility on the ia32 system, then copy to the x64 system. The issue with the
DirectedGraphs is that some of them can run an extremely long time to generate, so loosing the cached versions could be quite expensive.
Well, when in doubt, bring data. ... It's counter intuitive, but nfreeze is a bit faster than freeze, and generates a smaller results.
Length: FROZEN 15958 NFROZEN 15949
COMPARING
Rate a b
a 23635/s -- -3%
b 24355/s 3% --
Code to test was:
my $frozen = Storable::freeze( \%hash );
my $nfrozen = Storable::nfreeze( \%hash );
print STDERR "Length FROZEN " . length($frozen) . " NFROZEN " . length($nfrozen) . "\n";
print "COMPARING\n";
use Benchmark qw( cmpthese ) ;
use Storable qw(freeze nfreeze);
cmpthese( -20, {
a => sub{
my $X = Storable::freeze( \%hash );
},
b => sub{
my $X = Storable::nfreeze( \%hash );
}
} );
--
GeorgeClark - 17 Dec 2013
Hm, seems you are not the first to find a performance & caching advantage in nfreeze:
http://blogs.perl.org/users/ovid/2010/10/storable-freeze-versus-nfreeze.html
Maybe I should experiment with it using big day-to-day data structures...
--
MichaelDaum - 18 Dec 2013