Item761: WYSIWYG break Chinese chars

• Chinese Editing is well supported in Raw Edit but not WYSIWYG Edit.
• You can input Chinese chars in the Edit Window, but fail to save. The error you get is

Foswiki detected an internal error - please check your Foswiki logs and webserver logs for more information. Cannot decode string with wide characters 

• You'll find the following errors in your /foswiki/data/warnXXXXXX.txt

| 10 Jan 2009 - 08:45 | UTF-8 not yet supported as site charset -Foswiki is likely to have problems 
| 10 Jan 2009 - 08:47 | Cannot decode string with wide characters at /usr/lib/perl5/5.8.8/i386-linux-thread-multi/Encode.pm line 166. 

• You can Edit in WYSIWYG editor, but turn to TWiki Raw Edit and save it.

Solution

• See Support.Question41

Edit foswiki/lib/Foswiki/Plugins/WysiwygPlugin/HTML2TML.pm:

113c113,116
<         $text = Encode::decode_utf8($text);
---
> #       $text = Encode::decode_utf8($text);
>         if (utf8::valid($text)) {
>             utf8::decode($text);
>         }

---

IIRC I tried this when I was working on utf8 before, and while it fixes Chinese, it breaks other non-utf8 character sets such as Cyrillic. Fixes for character sets unfortunately have to work for all character sets, at least until we have switched to utf8 everywhere.

This needs to be considered by the WysiwygTaskTeam and the (as yet unnamed) task team that does utf8.

-- CrawfordCurrie - 24 Jan 2009

Related:

• Tasks.Item5990
• Support.Question41

-- PaulHarvey - 24 Jan 2010

Fixed on trunk. For example, see http://trunk.foswiki.org/Sandbox/TestTopic44464.

-- MichaelTempest - 20 Jun 2010