Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.javascript > #24595 > unrolled thread

copyright symbol

Back to article view | Back to comp.lang.javascript

Contents

  copyright symbol Andrew Poulos <ap_prog@hotmail.com> - 2014-06-04 14:36 +1000
    Re: copyright symbol "Jukka K. Korpela" <jkorpela@cs.tut.fi> - 2014-06-04 08:51 +0300
      Re: copyright symbol "Evertjan." <exxjxw.hannivoort@inter.nl.net> - 2014-06-04 10:37 +0200
    Re: copyright symbol Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2014-06-04 11:06 +0200
      Re: copyright symbol Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2014-06-04 14:34 +0200
        Re: copyright symbol "Jukka K. Korpela" <jkorpela@cs.tut.fi> - 2014-06-04 15:41 +0300
      Re: copyright symbol Andrew Poulos <ap_prog@hotmail.com> - 2014-06-05 10:28 +1000
        Re: copyright symbol "Evertjan." <exxjxw.hannivoort@inter.nl.net> - 2014-06-05 09:12 +0200
    Re: copyright symbol Denis McMahon <denismfmcmahon@gmail.com> - 2014-06-04 11:21 +0000

#24595 — copyright symbol

I have an MS SQL Server database with a copyright symbol in a field of 
data type varchar(80). When this is read by JScript in a classic ASP and 
then displayed in the page the symbol displays as a black diamond with a 
question mark in it.

If I change the page's charset from
   <meta charset="UTF-8">
to
   <meta charset="iso-8859-1">
the symbol correctly displays as a copyright symbol.

Why does changing the charset "fix" the display of the symbol?

I'd rather keep the charset as UTF but I'm at a loss as to what's going on.

Do I need to change the data type?
Do I need to encode the symbol before it goes into the database?
Do I need to unescape/decode the value from the database?

Andrew Poulos

[toc] | [next] | [standalone]

#24596

2014-06-04 7:36, Andrew Poulos wrote:

> Why does changing the charset "fix" the display of the symbol?

Because the <meta charset=...> element, in the absence of overriding 
information (HTTP headers or BOM) tells the browser how to interpret the 
bytes in an HTML document as characters. Apparently, your code generates 
an HTML document where "@" appears as iso-8859-1 encoded, so the browser 
needs to be instructed about the iso-8859-1 encoding.

JavaScript (or JScript) hasn't got much to do with this. The real issues 
are with the character encodings used in the database and in HTML 
documents generated. They should be the same encoding, or a controlled 
and correct code conversion should be performed.

-- 
Yucca, http://www.cs.tut.fi/~jkorpela/

[toc] | [prev] | [next] | [standalone]

#24599

"Jukka K. Korpela" <jkorpela@cs.tut.fi> wrote on 04 jun 2014 in 
comp.lang.javascript:

> 2014-06-04 7:36, Andrew Poulos wrote:
> 
>> Why does changing the charset "fix" the display of the symbol?
> 
> Because the <meta charset=...> element, in the absence of overriding 
> information (HTTP headers or BOM) tells the browser how to interpret the 
> bytes in an HTML document as characters. Apparently, your code generates 
> an HTML document where "@" appears as iso-8859-1 encoded, so the browser 
> needs to be instructed about the iso-8859-1 encoding.
> 
> JavaScript (or JScript) hasn't got much to do with this. The real issues 
> are with the character encodings used in the database and in HTML 
> documents generated. They should be the same encoding, or a controlled 
> and correct code conversion should be performed.

Indeed, the question is off topic, should be in a html-NG.

Use this stringlet, works in html: 

&copy;

-- 
Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)

[toc] | [prev] | [next] | [standalone]

#24600

Andrew Poulos wrote:

> I have an MS SQL Server database with a copyright symbol in a field of
> data type varchar(80). When this is read by JScript in a classic ASP and
> then displayed in the page the symbol displays as a black diamond with a
> question mark in it.

That is a possible rendering of U+FFFD REPLACEMENT CHARACTER.  It is used 
here because the code sequence leading to the character in the database (A9) 
is not a proper UTF-8 code sequence (A9 is reserved as continuation byte).

<https://en.wikipedia.org/wiki/ISO-8859-1#Codepage_layout>
 
> If I change the page's charset from
>    <meta charset="UTF-8">
> to
>    <meta charset="iso-8859-1">
> the symbol correctly displays as a copyright symbol.
> 
> Why does changing the charset "fix" the display of the symbol?
> 
> I'd rather keep the charset as UTF but I'm at a loss as to what's going
> on.

The term “charset” is historical and misleading.  This “meta” element 
specifies the character *encoding* of the resource unless there is an HTTP 
“Content-Type” header field value that takes precedence.  This client-side 
change has an effect, so you are not serving a “Content-Type” header field 
value with a ”charset” parameter; your server-side code is incomplete or in 
error.

<http://www.w3.org/TR/2014/CR-html5-20140204/infrastructure.html#fetching-resources>

UTF-8 is the Unicode encoding that uses 8 bits per code unit.  A Unicode 
character, precisely its code point, can be encoded with up to four such 
code units.

<http://unicode.org/faq/>

> Do I need to change the data type?

No.

> Do I need to encode the symbol before it goes into the database?

No, but it would be a good idea.

> Do I need to unescape/decode the value from the database?

Yes, you need to *re*code string values if the database uses a different 
character encoding than the document resource you are generating.

This problem has very little to do with ECMAScript implementations.

Next time, do research.

<http://www.joelonsoftware.com/articles/unicode.html>

-- 
PointedEars
FAQ: <http://PointedEars.de/faq> | SVN: <http://PointedEars.de/wsvn/>
Twitter: @PointedEars2 | ES Matrix: <http://PointedEars.de/es-matrix>
Please do not Cc: me. / Bitte keine Kopien per E-Mail.

[toc] | [prev] | [next] | [standalone]

#24603

Thomas 'PointedEars' Lahn wrote:

> UTF-8 is the Unicode encoding that uses 8 bits per code unit.  A Unicode
> character, precisely its code point, can be encoded with up to four such
> code units.
> 
> <http://unicode.org/faq/>

Up to _six_ code units are possible.  I stand corrected by
<https://en.wikipedia.org/wiki/UTF-8#Description>.

-- 
PointedEars
FAQ: <http://PointedEars.de/faq> | SVN: <http://PointedEars.de/wsvn/>
Twitter: @PointedEars2 | ES Matrix: <http://PointedEars.de/es-matrix>
Please do not Cc: me. / Bitte keine Kopien per E-Mail.

[toc] | [prev] | [next] | [standalone]

#24604

2014-06-04 15:34, Lahn wrote:

> I stand corrected by
> <https://en.wikipedia.org/wiki/UTF-8#Description>.

Do trolls really ever stand corrected? No, but sometimes they pretend 
to. Lahn is telling that he was corrected, in an off-topic issue, about 
which he still lectured pointedlessly - and corrected by a Wikipedia 
page, which could be written by anyone and changed anonymously at any 
moment. And the Unicode Standard is even freely available on the web.

-- 
Yucca, http://www.cs.tut.fi/~jkorpela/

[toc] | [prev] | [next] | [standalone]

#24611

On 4/06/2014 7:06 PM, Thomas 'PointedEars' Lahn wrote:
> Andrew Poulos wrote:
>
>> I have an MS SQL Server database with a copyright symbol in a field of
>> data type varchar(80). When this is read by JScript in a classic ASP and
>> then displayed in the page the symbol displays as a black diamond with a
>> question mark in it.
>
> That is a possible rendering of U+FFFD REPLACEMENT CHARACTER.  It is used
> here because the code sequence leading to the character in the database (A9)
> is not a proper UTF-8 code sequence (A9 is reserved as continuation byte).
>
> <https://en.wikipedia.org/wiki/ISO-8859-1#Codepage_layout>
>
>> If I change the page's charset from
>>     <meta charset="UTF-8">
>> to
>>     <meta charset="iso-8859-1">
>> the symbol correctly displays as a copyright symbol.
>>
>> Why does changing the charset "fix" the display of the symbol?
>>
>> I'd rather keep the charset as UTF but I'm at a loss as to what's going
>> on.
>
> The term “charset” is historical and misleading.  This “meta” element
> specifies the character *encoding* of the resource unless there is an HTTP
> “Content-Type” header field value that takes precedence.  This client-side
> change has an effect, so you are not serving a “Content-Type” header field
> value with a ”charset” parameter; your server-side code is incomplete or in
> error.
>
> <http://www.w3.org/TR/2014/CR-html5-20140204/infrastructure.html#fetching-resources>
>
> UTF-8 is the Unicode encoding that uses 8 bits per code unit.  A Unicode
> character, precisely its code point, can be encoded with up to four such
> code units.
>
> <http://unicode.org/faq/>
>
>> Do I need to change the data type?
>
> No.
>
>> Do I need to encode the symbol before it goes into the database?
>
> No, but it would be a good idea.
>
>> Do I need to unescape/decode the value from the database?
>
> Yes, you need to *re*code string values if the database uses a different
> character encoding than the document resource you are generating.
>
> This problem has very little to do with ECMAScript implementations.
>
> Next time, do research.
>
> <http://www.joelonsoftware.com/articles/unicode.html>

I added this js to the top of the page
   Response.CodePage = 65001;
   Response.CharSet = "utf-8";
and the symbol is now displaying correctly.

For me reading about unicode is like being an island in an ocean of 
information where the island is my understanding and the shore is my 
awareness of my ignorance. The more I read about unicode (ie the more my 
island grows) the larger the shoreline gets.

Andrew Poulos

[toc] | [prev] | [next] | [standalone]

#24612

Andrew Poulos <ap_prog@hotmail.com> wrote on 05 jun 2014 in 
comp.lang.javascript:

> I added this js to the top of the page
>    Response.CodePage = 65001;
>    Response.CharSet = "utf-8";
> and the symbol is now displaying correctly.

This is serverside ASP, either js or vbs,
only available on servers that support this.

It will alter the "HEAD" part of the stream, 
not the html <head></head>, methinks.


-- 
Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)

[toc] | [prev] | [next] | [standalone]

#24602

On Wed, 04 Jun 2014 14:36:26 +1000, Andrew Poulos wrote:

> I have an MS SQL Server database with a copyright symbol in a field of
> data type varchar(80). When this is read by JScript in a classic ASP and
> then displayed in the page the symbol displays as a black diamond with a
> question mark in it.
> 
> If I change the page's charset from
>    <meta charset="UTF-8">
> to
>    <meta charset="iso-8859-1">
> the symbol correctly displays as a copyright symbol.

I'm guessing that the data stored in the sql database isn't a utf-8 
copyright symbol, but rather an iso 8859-1 copyright symbol. I'm also 
guessing that an iso-8859-1 copyright symbol is an illegal character in 
utf-8.

> Why does changing the charset "fix" the display of the symbol?

Because it makes the browser display the symbol in the correct character 
set.

> I'd rather keep the charset as UTF but I'm at a loss as to what's going
> on.

My guess would be that someone is filling in a form using windows and 
you're receiving iso-8859-1 character data. This is then being put into a 
database. Later it's being read from the database and placed in a 
document as utf-8.

> Do I need to encode the symbol before it goes into the database?

This would need you to detect the character encoding of the input and 
convert it to utf-8 (or html entities). This is probably the safest 
method, as hopefully the data source provides some information as to the 
encoding the data is generated with. Failing that, you can try and sniff 
the incoming data to guess at the encoding. User agent strings may give 
possibly useful hints.

If all the character data is between values 32 and 127 inclusive (' ' .. 
'~') you can probably treat it as utf-8. Otherwise you need to determine 
the encoding. This is probably something you need to do in the server 
when processing the input (presumably form) data. This question probably 
be,longs in a perl / php / python / cgi of some description newsgroup.

> Do I need to unescape/decode the value from the database?

This is also a possible solution. However, you may have lost track of 
what the original character encoding was at this point.

-- 
Denis McMahon, denismfmcmahon@gmail.com

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.javascript

csiph-web