This question about Configuration, Topic Markup Language and applications: Task filed

INCLUDE Macro render different from what the charset of the original url sends.

Foswiki-2.1.6,
Plugin API version 2.4
Apache 2.4.29

Cgi made by me, need a <meta charset="utf-8" /> tag in order to get correct rendering of spahish accents (diacritics) on the web browser.They work fine.


But when I % INCLUDE{ url ..} % them on a foswiki topic, accents do now show correctly.

For example 1) Logístico      renders as     2) Logístico

The correct one is number 1 with an í

Thank you in advance.

-- AntonioVega - 21 Oct 2020

That could very well be the case. By default content in a Foswiki already is encoded in utf-8. Adding external content has to be converted to this encoding if it differs. This encoding is sent as part of the HTTP response as being fetched from the different website.

Have you got an example url to test with that is causing the problems?

-- MichaelDaum - 22 Oct 2020

Sorry for not pointing to a public url, the cgi I made only runs on the intranet. But let me elaborate:

For example on following file "logis1.html", the browser pointing to http://logis1.html, will render incorrectly (Logà ­stico)
<html>
<head><title>logis1.html</title></head>
<body><p>Logístico</p></body>
</html>

On the following file "logis2.html", the browser pointing to http://logis2.html, will render correctrly (Logístico) due to the use of the < meta charset=utf-8 / > tag.

<html>
<head><title>logis2.html</title><meta charset=utf-8 /></head>
<body><p>Logístico</p></body>
</html>

Now, both of them will render incorrectly (Logà ­stico) using % INCLUDE {"http://logis?.html" } % either logis1.html or logis2.html

-- AntonioVega - 21 Oct 2020

Oh I see. So the %INCLUDE does not fall back to utf-8 when no character set encoding is specified in a meta tag. Maybe it should? Have you got control over logis1.html adding the correct charset there?

-- MichaelDaum - 22 Oct 2020


Not sure if I have control over the correct character set, only know that i use the regular vi editor and type using spanish keyboard. bout maybe the following helps to let you know de octal dump of the file. (303 255) is í
od -c logis2.html
0000000   <   h   t   m   l   >   <   h   e   a   d   >   <   t   i   t
0000020   l   e   >   R   o   l   e   s   <   /   t   i   t   l   e   >
0000040   <   m   e   t   a       c   h   a   r   s   e   t   =   u   t
0000060   f   -   8       /   >   <   /   h   e   a   d   >   <   b   o
0000100   d   y   >  \n   <   p   >   L   o   g 303 255   s   t   i   c
0000120   o   <   /   p   >  \n   <   /   b   o   d   y   >  \n   <   /
0000140   h   t   m   l   >  \n
0000146

Please try this patch:
diff --git a/core/lib/Foswiki/IncludeHandlers/http.pm b/core/lib/Foswiki/IncludeHandlers/http.pm
index d0a842404..c04e2a893 100644
--- a/core/lib/Foswiki/IncludeHandlers/http.pm
+++ b/core/lib/Foswiki/IncludeHandlers/http.pm
@@ -106,7 +106,7 @@ sub INCLUDE {
     my $response = $session->net->getExternalResource($url);
     if ( !$response->is_error() ) {
         my $contentType = $response->header('content-type');
-        $text = $response->content();
+        $text = $response->decoded_content();
         if ( $contentType =~ m/^text\/html/ ) {
             if ( !$control->{raw} ) {
                 $url =~ m!^([a-z]+:/*[^/]*)(/[^#?]*)!;

Instead of reading the raw content it decodes it according to its content type.

-- MichaelDaum - 22 Oct 2020

Oops, the patch... I have to go over how to do that...jajaja ..let me try and I will report

-- AntonioVega - 22 Oct 2020

Success, now it works. thank you!

-- AntonioVega - 22 Oct 2020
 

QuestionForm edit

Subject Configuration, Topic Markup Language and applications
Extension
Version Foswiki 2.1.6
Status Task filed
Related Topics Item14970
Topic revision: r8 - 22 Oct 2020, MichaelDaum
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy