You are here: Foswiki>Tasks Web>Item5351 (08 Jan 2009, KwangErnLiew)Edit Attach

Item5351: WebCreateNewTopic filters out international characters

pencil
Priority: Normal
Current State: Closed
Released In: 1.0.0
Target Release: patch
Applies To: Engine
Component:
Branches:
Reported By: TWiki:Main.MartinMayer
Waiting For:
Last Change By: KwangErnLiew
WebCreateNewTopic filters out international Characters. While typing a new topic name international characters are automatically removed and the next character after the international one is capitalised. As capital letters have a special meaning in topic names this is quite confusing.

This sounds a bit like Item4419 but the solution suggested there does not solve the problem.

-- TWiki:Main/MartinMayer - 11 Feb 2008

The problem is two-fold:
  • webtopiccreator.js in PatternSkin uses removePunctuation() to sanitize topic names
  • removePunctuations (from TWikiJavascripts/twikiString.js) basically strips out all but MIXED_ALPHANUM_CHARS, which are [A-Za-z\d]

-- TWiki:Main.AndrewPantyukhin - 19 Apr 2008

The javascrripts use the same regexes as TWiki core. Updates to TWiki regexes should be reflected in the javascripts, but not the way around.

-- TWiki:Main.ArthurClemens - 19 Apr 2008

I don't know much JavaScript and I can't grasp the rationale behind twikiString.js. We've got character class definitions as global vars in twiki.js and unicode_chars.js. For some reason, though, twikiString.js does not use them, but introduces a curious OO-based definitions. It looks like someone was trying to do some refactoring, but never finished it. Anyway, twikiStringUnicodeChars.js is an attempt to convert unicode_chars.js to the same OO interface, but it only contains one class (UPPER_ALPHA_CHARS), and is not included from PatternSkin (while unicode_chars.js is).

I may be wrong, but I'd say we need to forget about the OO-thing and just run s/twiki.StringConstants.getInstance().// in twikiString.js. That way it'll use the complete definitions.

-- TWiki:Main.AndrewPantyukhin - 24 Apr 2008

This little patch fixes the problem: http://heka.cenkes.org/sat/diffs/non-ascii.wikiword.diff

-- TWiki:Main.AndrewPantyukhin - 29 Apr 2008

I applied Andrew's patch after an hour of testing

It works great. We non-English can again create I18N topics (non UTF8)

(from Japan:) This will break twiki.String because it now won't function without unicode_chars.js. Running TestCaseTWikiJavascripts will reveal the errors in the new code.

I see I did not finish foswikiStringUnicodeChars.js which is the proper replacement for unicode_chars.js (I did not include the lowercase chars).

Proper fix:

-- ArthurClemens - 26 Dec 2008

ItemTemplate edit

Summary WebCreateNewTopic filters out international characters
ReportedBy TWiki:Main.MartinMayer
Codebase trunk
SVN Range TWiki-5.0.0, Wed, 23 Jan 2008, build 16283
AppliesTo Engine
Component
Priority Normal
CurrentState Closed
WaitingFor
Checkins Rev 1574 not found distro:0318e6c0669f
TargetRelease patch
ReleasedIn 1.0.0
Topic revision: r13 - 08 Jan 2009, KwangErnLiew
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy