You are here: Foswiki>Tasks Web>Item12569 (03 Feb 2016, GeorgeClark)Edit Attach

Item12569: Sorting in tableplugin is wrong in german

Priority: Normal
Current State: Closed
Released In: 2.1.0
Target Release: minor
Applies To: Extension
Component: I18N, TablePlugin
Branches: trunk master Item13405
Reported By: FlorianPribahsnik, PhilippGortan
Waiting For:
Last Change By: GeorgeClark
If i sort a table with TablePlugin in german umlaute are at the end of the table. This looks like:

a b c

x y z ä ö ü

but it should look like: a ä b c



x y z

-- FlorianPribahsnik - 23 Aug 2013

Same issue in our installation (Foswiki 1.1.8, UseLocale=1, Site::Locale=de_AT.ISO-8859-1): When using the column sort feature on a table like this the correct sort order would be A - Ä - B - ..., but instead the Umlauts show up at the very end of the table.

-- PhilippGortan - 24 Feb 2014

The patch seems to be rather trivial: When UseLocale is enabled in the foswiki config, then the specified locale should be used for sorting. The implementation is identical to Foswiki::Search, Foswiki::Render, etc.

Item12569.patch: Proposed patch

-- PhilippGortan - 24 Feb 2014

Unfortunately UseLocale creates all sorts of other problems together with perl's tainted check.

-- MichaelDaum - 25 Feb 2014

The attached script tries to verify my assumption that:

  • using "use locale" is scoped, so calling it in a block does not have any effect on the outer code
  • "use locale" can thus be switched on and off at runtime without using the begin-if-require-import-hack
  • outside of the block, \w does not taint the submatch

Can someone please verify that? Calling the script without params sorts without locale, adding any parameter when calling sorts with locale. In both cases, $res is not tainted.

-- PhilippGortan - 25 Feb 2014

Jan, Could you validate the addition of the UseLocale patch to TablePlugin/ The proposed fix here is the same as what you used in just about every other module. Mark this waiting for release if it looks good. Thanks.

-- GeorgeClark - 12 Jan 2015

Definitely not fixed in 1.2.0. Note that client side sorting in javascript is also broken. In the core perl side, the fundamental issue is that we do not "normalize" the unicode characters. The following patch fixes the perl table sort implementation, but is NOT applied, as this needs much more discussion. Also the javascript side would need equivalent work.

diff --git a/TablePlugin/lib/Foswiki/Plugins/TablePlugin/ b/TablePlugin/lib/Foswiki/Plugins/TablePlugin/
index 7764d25..9c9065a 100644
--- a/TablePlugin/lib/Foswiki/Plugins/TablePlugin/
+++ b/TablePlugin/lib/Foswiki/Plugins/TablePlugin/
@@ -10,6 +10,8 @@ use Foswiki::Plugins::TablePlugin ();
 use Foswiki::Time;
 use Error qw(:try);
+use Unicode::Normalize;
     if ( $Foswiki::cfg{UseLocale} ) {
         require locale;
@@ -1502,7 +1504,7 @@ sub emitTable {
             && $currentSortDirection == $SORT_DIRECTION->{'ASCENDING'} )
             @curTable = sort {
-                     $a->[$sortCol]->{sortText} cmp $b->[$sortCol]->{sortText}
+                     NFKD($a->[$sortCol]->{sortText}) cmp NFKD($b->[$sortCol]->{sortText})
                   || $a->[$sortCol]->{number} <=> $b->[$sortCol]->{number}
                   || $a->[$sortCol]->{dateString}
                   cmp $b->[$sortCol]->{dateString}
@@ -1512,7 +1514,7 @@ sub emitTable {
             && $currentSortDirection == $SORT_DIRECTION->{'DESCENDING'} )
             @curTable = sort {
-                     $b->[$sortCol]->{sortText} cmp $a->[$sortCol]->{sortText}
+                     NFKD($b->[$sortCol]->{sortText}) cmp NFKD($a->[$sortCol]->{sortText})
                   || $b->[$sortCol]->{number} <=> $a->[$sortCol]->{number}
                   || $b->[$sortCol]->{dateString}
                   cmp $a->[$sortCol]->{dateString}

-- GeorgeClark - 25 Jun 2015

See discussion In the IRC Logs

-- GeorgeClark - 25 Jun 2015

Fixed in EditRowPlugin, so removed from the "Applies To"

-- Main.CrawfordCurrie - 28 Sep 2015 - 08:49

Topic attachments
I Attachment Action Size Date Who Comment
Item12569.patchpatch Item12569.patch manage 551 bytes 24 Feb 2014 - 22:51 PhilippGortan Proposed patch manage 388 bytes 25 Feb 2014 - 19:22 PhilippGortan locale test script
Topic revision: r19 - 03 Feb 2016, GeorgeClark - This page was cached on 16 Jan 2020 - 12:54.

The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy