Item13442: Convert all content to UTF-8

Priority: Urgent
Current State: Closed
Released In: n/a
Target Release: n/a
Applies To: Engine
Branches: master
Reported By: CrawfordCurrie
Waiting For:
Last Change By: CrawfordCurrie
As part of the forthcoming upgrade to Foswiki 1.2.0 on, we need to convert existing content to UTF-8 and set {Site}{CharSet} appropriately.

It's clear from simple analysis that there are a mixture of encodings in use on the site, mainly where people have pasted encoded content into the text editor and the bytes have been trivially saved without re-encoding. The encodings detected are:

  • 1464 strings are recognised as cp-1252
  • 72 as UTF-8
  • 30 are recognised as a different encoding
    • Big5
    • EUC-JP
    • EUC-KR
    • IBM866
    • ISO-8859-5
    • ISO-8859-7
    • ISO-8859-8
    • KOI8-R
    • Shift_JIS
    • gb18030
    • windows-1251
    • windows-1255
    • x-mac-cyrillic

To support this change I have added a "repair" option to the CharSetConverterContrib.

-- CrawfordCurrie - 01 Jun 2015

Looking good (George did most of the work)

-- CrawfordCurrie - 05 Jun 2015

Topic revision: r6 - 05 Jun 2015, CrawfordCurrie - This page was cached on 15 Feb 2018 - 13:36.

The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License