MediaWikiToFoswikiContrib broken: lots of use of uninitialized value warnings

Hello there,

I recenlty installed the MediaWikiToFoswikiContrib with a view to migrating data from MediaWiki to Foswiki.

I took a full XML dump from MediaWiki and I faced the following problem

[root@mediawiki mediawiki-1.21.1]# php ./maintenance/dumpBackup.php --full > ./dump_full.xml (I even used the --conf ./LocalSettings.php option but made no difference).

and when i tried to use the convertor tool I got this error:

-- root@linux-client:/export/foswiki/tools# ./mediawiki2foswiki --file /root/dump_full.xml --debug

0: version 0.8 unsupported at /export/foswiki/lib/CPAN/lib/Parse/MediaWikiDump.pm line 270.
at /export/foswiki/lib/CPAN/lib/Parse/MediaWikiDump.pm line 270

As such I editted the version of the dump_full_conf.xml and replaced it with 0.3 (that's the check inMediaWikiDump.pm line 270), so that this check will be overriden.

root@linux-client:/export/foswiki/tools# vi /root/dump_full.xml

...

Indeed that seemed to work and then the next error I got was the following:

-- root@linux-client:/export/foswiki/tools# ./mediawiki2foswiki --file /root/dump_full.xml --debug

unable to properly parse comprehensive dump files at /export/foswiki/lib/CPAN/lib/Parse/MediaWikiDump.pm line 490.
at /export/foswiki/lib/CPAN/lib/Parse/MediaWikiDump.pm line 490.

I searched online and I saw that that was the case quite often with full dumps. Therefore I retook the MediaWiki xml dump with the --current flag

[root@mediawiki mediawiki-1.21.1]# php ./maintenance/dumpBackup.php --current > ./dump_current.xml

Indeed when i tried to rerun the conversion I ran into another problem :

-- root@linux-client:/export/foswiki/tools# ./mediawiki2foswiki --file /root/dump_current.xml --debug

Use of uninitialized value $charset in pattern match (m//) at /export/foswiki/lib/Foswiki/Contrib/MediaWikiToFoswikiContrib/Converter.pm line 134 2.

<i><b>Use of uninitialized value $name in substitution (s///) at /usr/lib/perl/5.18/Encode.pm line 104.</b></i><br /><i><b>Use of uninitialized value $name in exists at /usr/lib/perl/5.18/Encode.pm line 107.</b></i><br /><i><b>Use of uninitialized value $name in lc at /usr/lib/perl/5.18/Encode.pm line 108.</b></i><br /><i><b>Use of uninitialized value $find in exists at /usr/lib/perl/5.18/Encode/Alias.pm line 25.</b></i><br /><i><b>Use of uninitialized value $find in hash element at /usr/lib/perl/5.18/Encode/Alias.pm line 26.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 40.</b></i><br /><i><b>Use of uninitialized value $find in pattern match (m//) at /usr/lib/perl/5.18/Encode/Alias.pm line 31.</b></i><br /><i><b>Use of uninitialized value $find in string eq at /usr/lib/perl/5.18/Encode/Alias.pm line 44.</b></i><br /><i><b>Use of uninitialized value $find in hash element at /usr/lib/perl/5.18/Encode/Alias.pm line 57.</b></i><br /><i><b>Use of uninitialized value $find in lc at /usr/lib/perl/5.18/Encode/Alias.pm line 58.</b></i><br /><i><b>Use of uninitialized value $find in hash element at /usr/lib/perl/5.18/Encode/Alias.pm line 77.</b></i><br /><i><b>Use of uninitialized value $name in string ne at /usr/lib/perl/5.18/Encode.pm line 113.</b></i><br /><i><b>Use of uninitialized value $name in hash element at /usr/lib/perl/5.18/Encode.pm line 117.</b></i><br /><i><b>Use of uninitialized value $name in concatenation (.) or string at /usr/lib/perl/5.18/Encode.pm line 174</b></i><br /><i><b>Unknown encoding '' at /usr/lib/perl/5.18/Encode.pm line 174.</b></i><br /><i><b> Encode::decode(undef, '__NOATTACHMENTS__\x{a}\x{a}== [[FieldWiki]] =\x{a}\x{a}= [[Analytics]] <span>\x{a}&lt;...') called at /export/foswiki/lib/Foswiki/Contrib/MediaWikiToFoswikiContrib/Converter.pm line 1345</span></b></i><br /><i><b> Foswiki::Contrib::MediaWikiToFoswikiContrib::Converter::toUtf8('Foswiki::Contrib::MediaWikiToFoswikiContrib::Converter=HASH(0x9262c10)', '__NOATTACHMENTS__\x{a}\x{a}</b> [[FieldWiki]] ==\x{a}\x{a}== [[Analytics]] <b><span>\x{a}&lt;...') called at /export/foswiki/lib/Foswiki/Contrib/MediaWikiToFoswikiContrib/Converter.pm line 1237</span></b></i><br /><i><b> Foswiki::Contrib::MediaWikiToFoswikiContrib::Converter::getPageText('Foswiki::Contrib::MediaWikiToFoswikiContrib::Converter=HASH(0x9262c10), 'Parse::MediaWikiDump::page=HASH(0x93ce944)') called at /export/foswiki/lib/Foswiki/Contrib/MediaWikiToFoswikiContrib/Converter.pm line 310</b></i><br /><i><b> Foswiki::Contrib::MediaWikiToFoswikiContrib::Converter::convert('Foswiki::Contrib::MediaWikiToFoswikiContrib::Converter=HASH(0x9262c10)) called at /export/foswiki/lib/Foswiki/Contrib/MediaWikiToFoswikiContrib.pm line 87</b></i><br /><i><b> Foswiki::Contrib::MediaWikiToFoswikiContrib::main() called at ./mediawiki2foswiki line 82</b></i><br /><i><b> at /usr/lib/perl/5.18/Encode.pm line 174.</b></i><br /><i><b> Encode::decode(undef, '__NOATTACHMENTS__\x{a}\x{a}</b> [[FieldWiki]] ==\x{a}\x{a}== [[Analytics]] <b><span>\x{a}&lt;...') called at /export/foswiki/lib/Foswiki/Contrib/MediaWikiToFoswikiContrib/Converter.pm line 1345</span></b></i><br /><i><b> Foswiki::Contrib::MediaWikiToFoswikiContrib::Converter::toUtf8('Foswiki::Contrib::MediaWikiToFoswikiContrib::Converter=HASH(0x9262c10)', '__NOATTACHMENTS__\x{a}\x{a}</b> [[FieldWiki]] ==\x{a}\x{a}== [[Analytics]] ==\x{a}&lt;...') called at /export/foswiki/lib/Foswiki/Contrib/MediaWikiToFoswikiContrib/Converter.pm line 1237</i><br /><i><b> Foswiki::Contrib::MediaWikiToFoswikiContrib::Converter::getPageText('Foswiki::Contrib::MediaWikiToFoswikiContrib::Converter=HASH(0x9262c10)', 'Parse::MediaWikiDump::page=HASH(0x93ce944)') called at /export/foswiki/lib/Foswiki/Contrib/MediaWikiToFoswikiContrib/Converter.pm line 310</b></i><br /><i><b> Foswiki::Contrib::MediaWikiToFoswikiContrib::Converter::convert('Foswiki::Contrib::MediaWikiToFoswikiContrib::Converter=HASH(0x9262c10) called at /export/foswiki/lib/Foswiki/Contrib/MediaWikiToFoswikiContrib.pm line 87</b></i><br /><i><b> Foswiki::Contrib::MediaWikiToFoswikiContrib::main() called at ./mediawiki2foswiki line 82</b></i><br /><i><b>root@linux-client:/export/foswiki/tools# vi /usr/lib/perl/5.18/Encode.pm</b></i>

I'm not much of a Perl expert myself, but I understand the the 'use of uninitialized value' are warnings and the main problem seems to lie within the ...Unknown encoding '' at /usr/lib/perl/5.18/Encode.pm line 174.
Encode::decode(undef, '__NOATTACHMENTS__\x{a}\x{a}== FieldWiki \x{a}\x{a} Analytics ==\x{ ... part of the output.

There seems to be an encoding problem. I would appreciate any help on this. I even tried the same tool with another MediaWiki xml dump I downloaded off the web and I get the same errors.

I'm not sure if the workarounds for the first 2 issues I faced are correct, but I believe they are not related to the 3rd issue.

At your disposal for any further clarificaton

Thanks in advance

Pavlos

I'm using perl 5.18.2 and the foswiki host system is:

root@linux-client:/export/foswiki/tools# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.2 LTS
Release: 14.04
Codename: trusty

-- PavlosKaimakis - 26 May 2015