Item14420: New TinyMCE on Foswiki 2.2. alpha is not correctly saving as TML

Entering bold text saves as:

<p>Some *&#65279;bold*&#65279; text</p>

And another topic ended up with:

<p>This is a test, of the <b>new</b> <font color="#008080">editor.</font></p>
<p>&#160;</p>
<p>Insert</p>
<p style="padding-left: 30px;">Insert</p>

-- %WIKIUSERNAME% - %DATE%

Tried to create a table using only the editor's controls. It used html bold tags for the headings because there were no spaces between the | and the text. I re-opened the topic and attempted to insert the spaces around the heading. and save crashed.

Use of uninitialized value $t in pattern match (m//) at /var/www/trunk.foswiki.org/core/lib/Foswiki/Plugins/WysiwygPlugin/HTML2TML/Node.pm line 1557.
 at /var/www/.plenv/versions/5.22.2/lib/perl5/site_perl/5.22.2/CGI/Carp.pm line 362.
   CGI::Carp::realdie("Use of uninitialized value \$t in pattern match (m//) at /var/"...) called at /var/www/.plenv/versions/5.22.2/lib/perl5/site_perl/5.22.2/CGI/Carp.pm line 454
   CGI::Carp::die("Use of uninitialized value \$t in pattern match (m//) at /var/"...) called at /var/www/trunk.foswiki.org/core/lib/AssertOn.pm line 15
   Assert::__ANON__("Use of uninitialized value \$t in pattern match (m//) at /var/"...) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/Plugins/WysiwygPlugin/HTML2TML/Node.pm line 1557
   Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node::_handleB(Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node=HASH(0x50d2530), 532) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/Plugins/WysiwygPlugin/HTML2TML/Node.pm line 610
   Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node::generate(Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node=HASH(0x50d2530), 532) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/Plugins/WysiwygPlugin/HTML2TML/Node.pm line 650
   Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node::_flatten(Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node=HASH(0x50d2458), 532) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/Plugins/WysiwygPlugin/HTML2TML/Node.pm line 1021
   Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node::_isConvertableTableRow(Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node=HASH(0x50d22d8), 20, ARRAY(0x50db968)) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/Plugins/WysiwygPlugin/HTML2TML/Node.pm line 962
   Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node::_isConvertableTable(Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node=HASH(0x50d2158), 20, ARRAY(0x509c790)) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/Plugins/WysiwygPlugin/HTML2TML/Node.pm line 954
   Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node::_isConvertableTable(Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node=HASH(0x50cfbf0), 20, ARRAY(0x509c790)) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/Plugins/WysiwygPlugin/HTML2TML/Node.pm line 2027
   Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node::_handleTABLE(Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node=HASH(0x50cfbf0), 16) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/Plugins/WysiwygPlugin/HTML2TML/Node.pm line 610
   Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node::generate(Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node=HASH(0x50cfbf0), 16) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/Plugins/WysiwygPlugin/HTML2TML/Node.pm line 650
   Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node::_flatten(Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node=HASH(0x50cf218), 16) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/Plugins/WysiwygPlugin/HTML2TML/Node.pm line 620
   Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node::generate(Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node=HASH(0x50cf218), 16) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/Plugins/WysiwygPlugin/HTML2TML/Node.pm line 269
   Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node::rootGenerate(Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node=HASH(0x50cf218), 16) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/Plugins/WysiwygPlugin/HTML2TML.pm line 222
   Foswiki::Plugins::WysiwygPlugin::HTML2TML::convert(Foswiki::Plugins::WysiwygPlugin::HTML2TML=HASH(0x50aa620), "\x{a}<h1 class=\"TML\">Topic with a table</h1>\x{a}<p class=\"WYSIWYG_NB"..., HASH(0x4eee9c0)) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/Plugins/WysiwygPlugin/Handlers.pm line 200
   Foswiki::Plugins::WysiwygPlugin::Handlers::TranslateHTML2TML("\x{a}<h1 class=\"TML\">Topic with a table</h1>\x{a}<p class=\"WYSIWYG_NB"..., "TestANewTable", "Sandbox") called at /var/www/trunk.foswiki.org/core/lib/Foswiki/Plugins/WysiwygPlugin/Handlers.pm line 161
   Foswiki::Plugins::WysiwygPlugin::Handlers::afterEditHandler("<!--WYSIWYG content - do not remove this comment, and never u"..., "TestANewTable", "Sandbox", Foswiki::Meta=HASH(0x4eb1220)) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/Plugins/WysiwygPlugin.pm line 166
   Foswiki::Plugins::WysiwygPlugin::_execute("afterEditHandler", "<!--WYSIWYG content - do not remove this comment, and never u"..., "TestANewTable", "Sandbox", Foswiki::Meta=HASH(0x4eb1220)) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/Plugins/WysiwygPlugin.pm line 346
   Foswiki::Plugins::WysiwygPlugin::afterEditHandler("<!--WYSIWYG content - do not remove this comment, and never u"..., "TestANewTable", "Sandbox", Foswiki::Meta=HASH(0x4eb1220)) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/Plugin.pm line 311
   Foswiki::Plugin::invoke(Foswiki::Plugin=HASH(0x3fc3ad8), "afterEditHandler", "<!--WYSIWYG content - do not remove this comment, and never u"..., "TestANewTable", "Sandbox", Foswiki::Meta=HASH(0x4eb1220)) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/Plugins.pm line 380
   Foswiki::Plugins::dispatch(Foswiki::Plugins=HASH(0x39cf910), "afterEditHandler", "<!--WYSIWYG content - do not remove this comment, and never u"..., "TestANewTable", "Sandbox", Foswiki::Meta=HASH(0x4eb1220)) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/UI/Save.pm line 681
   Foswiki::UI::Save::save(Foswiki=HASH(0x39cf730)) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/UI.pm line 378
   Foswiki::UI::__ANON__() called at /var/www/.plenv/versions/5.22.2/lib/perl5/site_perl/5.22.2/Error.pm line 421
   eval {...} called at /var/www/.plenv/versions/5.22.2/lib/perl5/site_perl/5.22.2/Error.pm line 413
   Error::subs::try(CODE(0x232f6a0), HASH(0x39cf178)) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/UI.pm line 504
   Foswiki::UI::_execute(Foswiki::Request=HASH(0x239aca0), CODE(0x39c2600), "save", 1) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/UI.pm line 330
   Foswiki::UI::handleRequest(Foswiki::Request=HASH(0x239aca0)) called at /var/www/trunk.foswiki.org/core/lib/Foswiki/Engine/CGI.pm line 100
   Foswiki::Engine::CGI::run(Foswiki::Engine::CGI=HASH(0x31ae9d0)) called at /var/www/trunk.foswiki.org/core/bin/save line 31.

-- GeorgeClark - 13 Jun 2017

To recreate the trash. Edit https://trunk.foswiki.org/Sandbox/TestANewTable - position cursor into the h1 cell, insert a space before and after the h1. Then do the same for h2, and save. It consistently crashes.

-- GeorgeClark - 13 Jun 2017

The issue with bold has to do with use of the keyboard shortcuts vs. the bold button. If Ctrl-B is used to start/stop bold, then the &#65279 marker gets inserted and preserved in the topic has an invisible character.

-- GeorgeClark - 13 Jun 2017

I can't reproduce the described problem with bold, though I can see the marker in the pre-saved HTML. The marker is spanned by an element carrying a data-mce-bogus attribute. From my reading of the TinyMCE code, any element carrying this marker is supposed to be pruned from the output. If this is not happening it points to an upstream bug - possibly related to our use of the legacyoutput plugin. But we have to be able to reproduce it reliably to report it.

On the crash; a simple test cures that, already checked in under a different task. However there is a conflict. TinyMCE doesn't offer any way to generate a TH - it just lets you style text in a standard table row. That bolded text doesn't get converted back to TML stars because of Item9651.

-- Main.CrawfordCurrie - 10 Aug 2017 - 08:08

Here is a partial recreation. See https://trunk.foswiki.org/Sandbox/SimpleTest2

• Wysiwyg edited a new topic
• Typed at end:

  This is a (click bold button)Bold(click bold button) text
  
  This is a (ctrl-b)Bold(ctrl-b) text
  

Repeated it twice. The Ctrl-b preserves a <b> tag, while the bold button will save as asterisk-bold. Note that these fully "stick" across subsequent wysiwyg edits. Then tried a third approach. Typed the sentence, then double-clicked the word bold and pressed the bold button, or ctrl-b ... In this case, TML markup always results, so it's only the ctrl-b inline that is causing HTML markup to stick.

If I Edit with Wysiwyg, and then click the <> view source button, I see:

<p>Test some <b>Bold</b> text with bold button</p>
<p>Test some <b>&#65279;Bold</b>&#65279; text with ctrl-b</p>
<p>Another <b>Bold</b> with button</p>
<p>Another <b>&#65279;Bold</b>&#65279; with ctrl-b</p>
<p>This sentence will have <b>bold</b> using double-click select and button</p>
<p>And this will have <b>bold</b> using double-click select and ctrl-b</p>

The reason it's sticking is that, while not visible in raw view, the text does contain markers. Using "vi" from the shell, I see:

Test some *Bold* text with bold button

Test some <b><feff>Bold</b><feff> text with ctrl-b

Another *Bold* with button

Another <b><feff>Bold</b><feff> with ctrl-b

This sentence will have *bold* using double-click select and button

And this will have *bold* using double-click select and ctrl-b

The <feff> is not encoded when viewing the raw topic. So it's not easy to detect this using the raw editor.

Ran one more test. Copied the topic to https://trunk.foswiki.org/Sandbox/SimpleTest2Copy and edited/saved using the NatEdit text editor. The invisible <feff> markup was fully preserved. So to the user, they cannot get TML and there is no obvious reason why. -- GeorgeClark - 10 Aug 2017

Not sure if there is another semi-related tasks buried in here... It seems as though raw edit or raw view should have some way to see/remove this hidden marker.

-- GeorgeClark - 10 Aug 2017

Sure, I see that it's sticking, and we could (at a push) strip these markers in HTML2TML - but, it's not our job to do that, it's an upstream error IMHO.

-- CrawfordCurrie - 14 Aug 2017

I'll leave this as a blocker for 2.2. If we don't get an upstream fix for this before releasing 2.2, we should work around the issue.

-- GeorgeClark - 16 Aug 2017

OK. I haven't reported it upstream, &#FEFF is a zero-width non-breaking space, and is a valid unicode character, so edit and NatEdit are treating it correctly. However see http://unicode.org/faq/utf_bom.html#bom6 which recommends it's interpretation as an "unsupported character" - as such, stripping it from the text would appear valid.

I believe this is closed now, unless you can see any other problems.

-- Main.CrawfordCurrie - 26 Aug 2017 - 07:56

-- WikiGuest - 26 Apr 2024

Commenting is disabled while not logged in