You are here: Foswiki>Tasks Web>Item13388 (05 Jul 2015, GeorgeClark)Edit Attach

Item13388: The %QUERY{...}% regex searches - fails on utf8 texts.

pencil
Priority: Normal
Current State: Closed
Released In: 2.0.0
Target Release: major
Applies To: Engine
Component:
Branches: master
Reported By: JozefMojzis
Waiting For:
Last Change By: GeorgeClark
The %QUERY% fails when for any QuerySearch where the search is utf8 and
  • the query contains a rexex with character class, such :upper: or the simple \w
  • contains the ignore case modifier, e.g. (?i}utf8text
  • or contains the lc uc for utf8

How to reproduce

Download the attachment. It is an raw topic file and contains a demo.

Tested on the trunk (commit 9fde37a000da7d8e1e3dfe4ab4cbe5871d09fec2).

Also, other errors

Because don't want spam the Tasks web with many utf8 related errors, the demo-topic contains other errors too, such
  • QUERY type json fails for utf8,
  • any CALC command from the spreadheet plugin manipulating utf8-texts fails
  • search/regex fails
  • EDITTABLE plugin causes some strange errors and so on...

-- JozefMojzis - 30 Apr 2015

I updated the topic to be ( i hope ) self-contained. (my test web has many testing topics and many testing variables). The previous version
  • used some %VARIABLES% what i has defined in the WebPreferences - now changed
  • and in the QUERY tried to query another topic's preferences value - now corrected to query itself.

So, GeorgeClark please download the topic and test it, not it should show: 15 errors. For ensure, i will attach the pdf version of the print... The page is intended to test clean distro testing (e.g. patternskin, etc..) so the Javascript could fail in other environments (i'm not an Javascript developer). It is tested with Mac's Safari and with Firefox/Firebug - and with fresh github clone. (Would be nice if someone could give some advices, how to make the JS part more universal.)

-- JozefMojzis - 04 May 2015

Ad comment from IRC
We'll probably document your Item13388 as a known issue.

Agree. IMHO we need release 1.2 :). Foswiki is now usable for editing UTF8 texts. Need document the issues:
  • any %QUERY% or %SEARCH% will fail
    1. for regex searches containing character classes such \w or [[:upper:]] and such e.g. the case insensitive operator too (?i)
    2. for lc and uc case conversions
  • the string functions of the SpreadSheetPlugin doesn't works with utf8
  • the %EDITABLE% macro somewhat broke the utf8 topic text... (only some characters). Maybe this issue got fixed with the Item13391 - not tested yet.

-- JozefMojzis - 04 May 2015

The example given is working as expected, on unicode core. Closing.

-- CrawfordCurrie - 19 May 2015
 

ItemTemplate edit

Summary The %QUERY{...}% regex searches - fails on utf8 texts.
ReportedBy JozefMojzis
Codebase trunk
SVN Range
AppliesTo Engine
Component
Priority Normal
CurrentState Closed
WaitingFor
Checkins distro:980aebc2a4c9 distro:6e027c7276dc
TargetRelease major
ReleasedIn 2.0.0
CheckinsOnBranches master
trunkCheckins
masterCheckins distro:980aebc2a4c9 distro:6e027c7276dc
ItemBranchCheckins
Release01x01Checkins
Topic attachments
I Attachment Action Size Date Who Comment
UtfErrors.pdfpdf UtfErrors.pdf manage 224 K 04 May 2015 - 18:03 JozefMojzis the pdf print of the topic
UtfErrors.txttxt UtfErrors.txt manage 14 K 04 May 2015 - 17:52 JozefMojzis updated source (i hope) now really self-contained...
Topic revision: r7 - 05 Jul 2015, GeorgeClark - This page was cached on 25 Aug 2016 - 03:01.

The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License