Item5659: scrambled output on edit

Existing Topics are scrambled by the Plugin. This is what I get when editing a Topic:

%25TREEVIEW%25%20%0A%0A--%20Main.CedricWeber%20-%2006%20May%202008%0A

Tested with revision 16836 & 16720. As soon as I go back to http://svn.twiki.org/svn/twiki/tags/TWikiRelease04x02x00/twikiplugins/WysiwygPlugin/ everything works.

-- TWiki:Main/CedricWeber - 27 May 2008

It would be good if you attach the a .txt file example of a "before" document because I cannot see this error with the current SVN versions.

Which Charsets are you running? Before and after.

-- TWiki:Main.KennethLavrsen - 28 May 2008

Email reminder sent to Cedric

-- TWiki:Main.KennethLavrsen - 03 Jun 2008

Before Wysiwyg editing:

---+ Willkommen im Sandbox Web

Das Sandbox Web ist ein sogenannter Sandkasten, den Sie für Testzwecke nutzen können. Jeder ist eingeladen Inhalte zu erstellen und zu löschen. Es wird empfohlen TWiki .TWikiTutorial durchzuschauen, um einen Überblick über das Eberspächer Wiki Werkzeug zu bekommen. Es gehört zum guten Ton am Ende eines Kommentars seine Signatur mit Datum und dem TWiki .WikiName zu hinterlassen.

-- Main.CedricWeber - 27 May 2008

After an svn switch to trunk (:# svn sw http://svn.twiki.org/svn/twiki/trunk/WysiwygPlugin/) revision 16877:

%3C%21--WYSIWYG%20content%20-%20do%20not%20remove%20this%20comment%2C%20and%20never%20use%20this%20identical%20text%20in%20your%20topics--%3E%20%3Ch1%20class%3D%22TML%22%3E%20%2520Willkommen%2520im%2520Sandbox%2520Web%3Cspan%20class%3D%22WYSIWYG_PROTECTED%22%3E%250A%25%3C/span%3E0ADas%2520Sandbox%2520Web%2520ist%2520ein%2520sogenannter%2520Sandkasten%3Cspan%20class%3D%22WYSIWYG_PROTECTED%22%3E%252C%25%3C/span%3E20den%2520Sie%2520f%25FCr%2520Testzwecke%2520nutzen%2520k%25F6nnen.%2520Jeder%2520ist%2520eingeladen%2520Inhalte%2520zu%2520erstellen%2520und%2520zu%2520l%25F6schen.%2520Es%2520wird%2520empfohlen%2520TWiki%2520.TWikiTutorial%2520durchzuschauen%3Cspan%20class%3D%22WYSIWYG_PROTECTED%22%3E%252C%25%3C/span%3E20um%2520einen%3Cspan%20class%3D%22WYSIWYG_PROTECTED%22%3E%2520%25%3C/span%3EDCberblick%3Cspan%20class%3D%22WYSIWYG_PROTECTED%22%3E%2520%25%3C/span%3EFCber%2520das%2520Ebersp%25E4cher%2520Wiki%2520Werkzeug%2520zu%2520bekommen.%2520Es%2520geh%25F6rt%2520zum%2520guten%2520Ton%2520am%2520Ende%2520eines%2520Kommentars%2520seine%2520Signatur%2520mit%2520Datum%2520und%2520dem%2520TWiki%2520.WikiName%2520zu%2520hinterlassen.%3Cspan%20class%3D%22WYSIWYG_PROTECTED%22%3E%250A%25%3C/span%3E0A--%2520Main.CedricWeber%2520-%3Cspan%20class%3D%22WYSIWYG_PROTECTED%22%3E%252027%25%3C/span%3E20May%3Cspan%20class%3D%22WYSIWYG_PROTECTED%22%3E%25202008%25%3C/span%3E0A%20%3C/h1%3E

I switch back to # svn sw http://svn.twiki.org/svn/twiki/tags/TWikiRelease04x02x00/twikiplugins/WysiwygPlugin everything is fine.

-- CedricWeber - 04 Jun 2008

The trunk and TWikiRelease04x02 versions of WysiwygPlugin are supposed to be in sync.

You did not answer what charset you run? Is it utf-8?

And you did not attach the .txt file either.

And further. You do some SVN checkouts. What version of TWiki do you run? 4.2.0? The TWikiRelease04x02 branch?

Please give a detailed description of your setup.

-- TWiki:Main.KennethLavrsen - 04 Jun 2008

I was happly running twiki 4.2.0 and mod_perl

I got the exact same error today. I used twiki configure installer to update wysiwyg and tinymce. then every page i edited i got the scrambled junk.

I now cant work out how to revert back to the shipped version so I have disabled mod_perl and the wysiwyg editor and would like to know if there is any solution for this problem

-- TWiki:Main.TerryRankine - 09 Jun 2008

Terry AND Cedric

Please give us

• The charset you run with in configure. Ie the values in {Site}{Locale}, and {Site}{Charset}. We need this information to just have a slim chance to understand the problem.
• Attach a topic files to this bug report which is of before and after you edited. We need to see the RAW .txt files to see what is actually stored.

This bug item does not stand a chance to be fixed unless we are given the exact info we ask for.

-- TWiki:Main.KennethLavrsen - 09 Jun 2008

I have seen this error actually.

Or something very similar. I saw it when I was testing which version of Wysiwyg that introduced a bug which is now resolved.

And the reason was that the browser was caching the old javascript files even though I was running with a newer Wysiwyg.

The cure is control-F5 while in edit mode at any topic. After this I could cancel out and edit again and the problem was 100% cured.

Terry and Cedric can you confirm that you have cleared your cache in the browser?

-- TWiki:Main.KennethLavrsen - 11 Jun 2008

email reminder for feedback sent

-- TWiki:Main.KennethLavrsen - 18 Jun 2008

what email?..... I will add the content you required on monday when i get a chance to revisit the errors.

$TWiki::cfg{Site}{Locale} = 'en_US.ISO-8859-1'; $TWiki::cfg{Site}{LocaleRegexes} = 1; $TWiki::cfg{Site}{CharSet} = 'iso-8859-1'; $TWiki::cfg{Site}{Lang} = 'en'; $TWiki::cfg{Site}{FullLang} = 'en-us';

-- TWiki:Main.TerryRankine - 19 Jun 2008

Thanks Terry. The email to you bounced so the twiki.org email address you used when you registered is no longer valid. This also means that you do not get email reminders.

Your setup looks identical to the one I run. As I said I have seen the exact same problem right after an upgrade of Wysiwyg/TinyMCE and then a control-F5 in the browser while editing cured the problem. Both in IE and FF a control-F5 forces the browser to reload and recache all the background files including CSS and Javascript files. This is the first thing you should try when you attempt the upgrade again. Your users will see the same unless it is a while ago that they edited so they do not have these files cached or the cache is expired so an email to everyone with the control-F5 trick is a good idea. I used the BROADCASTMESSAGE for one day to help my users also but many do not read this message so an email is usually a good idea. If the company is large you may just email the few users that edit in the TWiki all the time.

-- TWiki:Main.KennethLavrsen - 19 Jun 2008

Hi Kenneth,

thanx for your reminder. There is no change after deleting my browser cache. The actual revision I updated is 16914. Thisis the content of my LocalSite.cfg:

$TWiki::cfg{Site}{Locale} = 'de_DE.ISO-8859-1';$TWiki::cfg{Site}{LocaleRegexes} = 1; $TWiki::cfg{Site}{CharSet} = 'iso-8859-1';  

I have also attached the Topic before and after edit.

-- CedricWeber - 20 Jun 2008

Hi Guys, I have the same problem with TWiki and the WysiwygPlugin. I'am running the current stable version of TWiki and charset = UTF-8. With the shipped version of the WysiwygPlugin I just get scrambled chars, when I use the WysiwygPlugin and file sayes "ISO-8859 text" to a new topic created with the WysiwygPlugin. After upgrading to the current version of the plugin from 17. june, writing text works and file sayes "UTF-8 Unicode text", but the first line of a new topic is always scrambeld as shown above in this topic. Also editing an existing topic is not good idea, because linebreaks, whitespaces and so on are scrambeld too. With charset = iso-8859-15 everything works, but I do want to use UTF-8. Also when I disable the WysiwygPlugin everything works too. My settings are:

$TWiki::cfg{Site}{Locale} = 'de_DE.utf8'; $TWiki::cfg{Site}{LocaleRegexes} = 0; $TWiki::cfg{UpperNational} = 'ÖÄÜ'; $TWiki::cfg{LowerNational} = 'öäüß'; $TWiki::cfg{UserInterfaceInternationalisation} = 1; $TWiki::cfg{UseLocale} = 1; $TWiki::cfg{Site}{CharSet} = 'UTF-8'; $TWiki::cfg{Site}{Lang} = 'en'; $TWiki::cfg{Site}{FullLang} = 'en-us';

Thanks for your help

-- TWiki:Main.ChristophHeinsch - 21 Jun 2008

Thanks to Cedric. Crawford is away the next week or so - so I doubt he will have a chance to follow up on this.

It is a bit of mystery what Cedric sees.

Christoph's case is less of a mystery. utf-8 is still quite unfinished and in order to support German I would for sure use ISO-8859-1 or -15.

The utf-8 is more a must have for people writing Chinese or other non-latin characters.

Christoph you also seem to mix up quite a few things. You run utf-8 defined as locale but have turned off locale regexes and uses instead specific Upper and Lower National.

{Lang} is set to 'en' and {FullLang} to 'en-us' which does not match the 'de_DE.utf8'.

In 4.2.1 (not yet released) I removed Lang and FullLang because they were too confusing and could be derived from {Site}{Locale}

Cedrik maybe you should also list all your relevant i18n/locale settings.

-- KennethLavrsen - 21 Jun 2008

Thanks for your response Kenneth. In the meantime I got it to work, without changing anything, I believe disabling and then enabling the plugin again was the trick. I also use, $TWiki::cfg{Site}{Lang} = 'de'; $TWiki::cfg{Site}{FullLang} = 'de-de'; now. Without $TWiki::cfg{Site}{LocaleRegexes} = 0 Topicnames are not allowed to have german Umlaute in it. Also without setting $TWiki::cfg{Site}{CharSet} = 'UTF-8' the webpages are deliverd in utf8, but the in the html header still stands "charset=iso-8859-1" which causes some problems, that was the reason for me to set the charset manually. We are running a utf8 system and editing the rcs files with an editor could cause problems, this a main reason why we would like to use utf8, but it is not recommed at all?

-- TWiki:Main.ChristophHeinsch - 14 Jul 2008

Since problem sorted itself No Action

-- TWiki:Main.KennethLavrsen - 22 Jul 2008 per TWiki:Codev.GeorgetownReleaseMeeting2008x07x21

received via email

Sorry for emailing directly, but I cannot for the life of me manage to register at twiki.org, or I'd update on this page:

http://develop.twiki.org/~twiki4/cgi-bin/view/Bugs/Item5659

I'm having the same problem - garbled pages in the WYSIWYG editor since installing the 25 May release of WysiwygPlugin, but for me no amount of disabling and reenabling has magically fixed it. To my knowledge all the charset stuff is set to default; the server presents the pages with Content-Type: text/html; charset=ISO-8859-1 which matches what TWiki is set to.

I have a bunch of really frustrated users screaming at me, have had the WYSIWYG turned off for a month, and really didn't expect to see that bug closed with no action. Can I get it reopened somehow?

Reopening but as a normal level report. 4.2.1 does not have this problem so it is a Plugin problem for upgraders it seems.

Karen, please provide us with more details about your installation. (I'll see of we can get Karen registered, otherwise I will relay the messages.

-- KennethLavrsen - 25 Jul 2008

Thanks for helping with the registration, Kenneth - not sure if I just caught twiki.org at a bad time or what, but it wasn't happy with me.

The web server is Debian 4.0, Apache/2.2.3, with mod_perl/2.0.2 installed but not in use. TWiki is 4.2.0 (Tue, 22 Jan 2008/16278) under Perl 5.8.8, i486-linux-gnu-thread-multi.

We've got the PerlDocPlugin and SpacedWikiWordPlugin, and I'm pretty sure that's it for nonstandard.

Unfortunately, I don't know what release of WysiwygPlugin I had before I upgraded it, which might have been enlightening. I'm reasonably sure it's whatever shipped with that build, though... reasonably sure there were no interim updates. If it's significant, I can dig up the records.

The wiki install itself is at http://wiki.phoenyx.net/, if there's anything useful I've left out. Currently, the wysiwyg editor is disabled, but I can set up a login and turn the plugin back on if someone would like to try some test edits.

The relevant parts of the config (though if any of this is non-standard, it's probably because I poked at it after the problem started, before I ran across this page):

$TWiki::cfg{Site}{Locale} = 'en_US.ISO-8859-1';
$TWiki::cfg{Site}{LocaleRegexes} = 1;
$TWiki::cfg{Site}{CharSet} = 'iso-8859-1';
$TWiki::cfg{Site}{Lang} = 'en';
$TWiki::cfg{Site}{FullLang} = 'en-us';
$TWiki::cfg{UseLocale} = 0;

I'll dump the full TWiki and/or Apache config somewhere if that's insufficient. Oh, and my Encode library is 2.25; we've upgraded a lot of Unicode-related Perl libraries recently, so it wouldn't surprise me to find there's an interaction there. (Though that one was ten days after the plugin upgrade, so would have been after the problem popped up.)

-- TWiki:Main.KarenCravens - 26 Jul 2008

Karen, the fact that your TWiki is public is a great help.

I can see that your TWiki is not working as normal at all.

You have a special apache config which makes short URLs. This is a very important information and known to give problems.

You have a non working TWiki in general. Your TWiki web is not visible. Your history links at the bottom shows twiki variables. The pages are suffixed by html so we see some strange mirror instead of the real pages.

Headings also look odd often displayed as ---+

I would say that the problems you have is from the fact that you have setup TWiki wrong or tried to be a little too inventive.

-- TWiki:Main.KennethLavrsen - 26 Jul 2008

Actually, no: that's the output from a publish plugin, and I'm an idiot for not remembering that. We've been having some performance issues, and I've been experimenting with static pages for search-engine use, with (as you can see) very limited success.

The actual TWiki is at http://wiki.phoenyx.net/wiki/bin/view . Sorry about that, I forgot it was still directing there.

-- TWiki:Main.KarenCravens - 26 Jul 2008

Karen. Could we talk you into upgrading your 4.2.0 to 4.2.2?

You should be able to just throw the upgrade package on top of your installation (make a backup).

You will need to save a copy of your tailored pages like the registration page.

This may fix your problem.

Same to Cedrik, Christoph, and Terry. Has upgrading TWiki (making the core and the Wysiwyg/TinyMCE plugins match what we have tested) fixed the problem?

-- TWiki:Main.KennethLavrsen - 14 Aug 2008

I've actually been putting off the 4.2.2 upgrade in case you wanted a (non-)working example to figure out what was going on with it. We'll bump it up in the next few days and see what happens.

-- TWiki:Main.KarenCravens - 16 Aug 2008

Tentatively, 4.2.2 WYSIWYG looks good. I'll know more once my international users play with more interesting character sets, but at least it's not coming up with double-encodings anymore.

-- TWiki:Main.KarenCravens - 20 Aug 2008

4.2.0 upgraded to 4.2.3, problem is that now when i am editing any topic with tinymce its some sort double encoding my scandinavian (Finnish) charasters and acting very strangly. I am currently using latest wysiwyg and tinymcepluging (just upgraded them). It does not matter if i activate utf-8 or use iso-8859-1, its always scrambing topic content if using wysiwyg editor.

Under 4.2.0 all was ok, wysiwyg editor worked perfectly but 4.2.3 brokes everything. What additional information is needed so anyone can fix this?

-- TWiki:Main.PekkaPanula - 16 Sep 2008

We probably do not hear more on this one.

My guess is that Pekka needed to control F5 and that the problem went away

• KennethLavrsen - 24 Dec 2008