You are here: Foswiki>Tasks Web>Item13834 (16 Nov 2015, GeorgeClark)Edit Attach

Item13834: EditRowPlugin cannot edit cells that contain Umlauts and other non-ASCII characters.

pencil
Priority: Urgent
Current State: Closed
Released In: 2.0.3
Target Release: patch
Applies To: Extension
Component: EditRowPlugin
Branches: master
Reported By: PascalSchuppli
Waiting For:
Last Change By: GeorgeClark
(Using a Foswiki 2.0.2 with UTF-8 store)

To reproduce, create any simple table, for example
First Second
One
Zwöi
Three
Four

If you then change cell 'Two' to 'Zwöi', different things happen depending on whether you're running on CGI or mod_perl:

CGI

Everything looks fine after saving the cell. But when you open it again for editing, the 'ö' is turned into a '�' character.

mod_perl

Everything looks fine after saving the cell. But when you open it again for editing, the cell content changes to "Loading..." and stays that way.

-- PascalSchuppli - 21 Oct 2015

Two separate issues here.

  • The UTF8 encoding issue. I'm unable to recreate it, confirmed on foswiki.org, locally. Even in this task topic. Click the corner paint on the utf-8 field and it's incorrectly encoded.

  • The "Loading..." confirmed ... When you click the corner paint to open a cell, the rest request is responded to, but the close quote is missing. So on a CGI system, the response is "Zwöi", but on a mod_perl system,. it responds with "Zwöi, no closing quote. ... Digging a bit further, fields containing utf-8 are double-quote enclosed. Fields with ascii are single-quote enclosed. That might be related. Not really sure though what's going on.

(Behaves differently on foswiki.org. Always double-quoted, but no quotes eaten. Now I'm wondering if it's an issue in the JSON module that builds the response.

Crawford... any ideas? I'm dumping the response in EditRowPlugin/Get.pm just prior to the JSON encoding. Both quotes are present there, but the trailing quote gets eaten when received from the web GET request.

-- GeorgeClark - 22 Oct 2015

I'm wrong on the closing quotes. The issue on the mod_perl system, we are calculating the length of the string using characters instead of bytes. So each 2-byte character shortens the response by 1.

I have a fix. EditRowPlugin/Get.pm generates results using the body() call, which does not utf-8 encode the results. This also throws off the length calculation. Changing Get.pm to call print() causes the results to be utf-8 encoded, and the length is then calculated correctly, fixing both issues.

-- GeorgeClark - 22 Oct 2015
 

ItemTemplate edit

Summary EditRowPlugin cannot edit cells that contain Umlauts and other non-ASCII characters.
ReportedBy PascalSchuppli
Codebase 2.0.2
SVN Range
AppliesTo Extension
Component EditRowPlugin
Priority Urgent
CurrentState Closed
WaitingFor
Checkins distro:5c91079656f2
TargetRelease patch
ReleasedIn 2.0.3
CheckinsOnBranches master
trunkCheckins
masterCheckins distro:5c91079656f2
ItemBranchCheckins
Release01x01Checkins
Topic revision: r4 - 16 Nov 2015, GeorgeClark - This page was cached on 24 May 2016 - 16:37.

The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License