Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.php > #17865

Re: Twice UTF-8 byte-order mark (BOM)

Show all headers | View raw

In article <5c30f891$0$22360$e4fe514c@news.xs4all.nl>, Luuk
<luuk@invalid.lan> wrote:

>On 5-1-2019 17:33, Tim Streater wrote:
>> In article <2V4YD.191363$lx7.100121@fx44.iad>, Richard Damon
>> <Richard@Damon-Family.org> wrote:
>> 
>>> On 1/5/19 5:04 AM, J.O. Aho wrote:
>>>> On 1/4/19 5:29 PM, vfhhutt45hghjjdewrghb@gmail.com wrote:
>>>>> Am Mittwoch, 26. Dezember 2018 15:41:06 UTC+1 schrieb J.O. Aho:
>>>>>> As long as it begins with <?php it will not output anything, but 
>>>>>> the BOM
>>>>>> is put before the <?php and so you will have output which is 
>>>>>> outside the
>>>>>> php engines control.
>>>>>>
>>>>>
>>>>> I understand the modus operandi but I think it's not a good way. UTF-8
>>>>> should be used as a norm. And the BOM is part of a UTF-8 file. In
>>>>> other cases (without BOM) it is a ISO 8859-1 file or anything else.
>>>>
>>>> The BOM was just an addition for issues in some lesser operating
>>>> systems, in newer versions like UTF-16 BOM has been removed from the
>>>> standard.
>>>
>>> To my knowledge, the Unicode Standard still defines the BOM, and it is
>>> actually very important for formats like UTF-16 (but not needed for
>>> UTF-16LE or UTF-16BE, where the endianness is provided by the encoding
>>> definition).

>Wikipedia is pretty clear about this:
>
>Wikipedia:
>In UTF-16, a BOM (U+FEFF) may be placed as the first character of a file 
>or character stream to indicate the endianness (byte order) of all the 
>16-bit code units of the file or stream.

A good enough reason not to use UTF-16.

>> Or UTF-8.

>Wikipedia:
>The Unicode Standard permits the BOM in UTF-8,[3] but does not require 
>or recommend its use.[4] Byte order has no meaning in UTF-8,[5] so its 
>only use in UTF-8 is to signal at the start that the text stream is 
>encoded in UTF-8, or that it was converted to UTF-8 from a stream that 
>contained an optional BOM. The standard also does not recommend removing 
>a BOM when it is there, so that round-tripping between encodings does 
>not lose information, and so that code that relies on it continues to 
>work.[6][7] The IETF recommends that if a protocol either (a) always 
>uses UTF-8, or (b) has some other way to indicate what encoding is being 
>used, then it "SHOULD forbid use of U+FEFF as a signature."[8]

The best way to indicate the encoding is out-of-band, such as the
charset attribute of the Content-Type header.

Even if the BOM is there, the sender could be lying about the actual
content. The BOM does not rescue you from this problem, it just adds
another as this thread shows.

-- 
When I saw how the European Union was developing, it was very obvious what they
had in mind was not democratic. In Britain you vote for a government so the
government has to listen to you, and if you don't like it you can change it.

Tony Benn

Back to comp.lang.php | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Twice UTF-8 byte-order mark (BOM) vfhhutt45hghjjdewrghb@gmail.com - 2018-12-24 08:39 -0800
  Re: Twice UTF-8 byte-order mark (BOM) Tim Streater <timstreater@greenbee.net> - 2018-12-24 16:49 +0000
    Re: Twice UTF-8 byte-order mark (BOM) Luuk <luuk@invalid.lan> - 2018-12-24 19:07 +0100
      Re: Twice UTF-8 byte-order mark (BOM) "J.O. Aho" <user@example.net> - 2018-12-24 20:54 +0100
        Re: Twice UTF-8 byte-order mark (BOM) Luuk <luuk@invalid.lan> - 2018-12-24 21:15 +0100
      Re: Twice UTF-8 byte-order mark (BOM) Arno Welzel <usenet@arnowelzel.de> - 2018-12-26 09:13 +0100
        Re: Twice UTF-8 byte-order mark (BOM) Luuk <luuk@invalid.lan> - 2018-12-26 12:41 +0100
          Re: Twice UTF-8 byte-order mark (BOM) Arno Welzel <usenet@arnowelzel.de> - 2018-12-26 15:32 +0100
        Re: Twice UTF-8 byte-order mark (BOM) "J.O. Aho" <user@example.net> - 2018-12-26 15:41 +0100
          Re: Twice UTF-8 byte-order mark (BOM) vfhhutt45hghjjdewrghb@gmail.com - 2019-01-04 08:29 -0800
            Re: Twice UTF-8 byte-order mark (BOM) Richard Damon <Richard@Damon-Family.org> - 2019-01-04 21:08 -0500
            Re: Twice UTF-8 byte-order mark (BOM) "J.O. Aho" <user@example.net> - 2019-01-05 11:04 +0100
              Re: Twice UTF-8 byte-order mark (BOM) Richard Damon <Richard@Damon-Family.org> - 2019-01-05 11:26 -0500
                Re: Twice UTF-8 byte-order mark (BOM) Tim Streater <timstreater@greenbee.net> - 2019-01-05 16:33 +0000
                Re: Twice UTF-8 byte-order mark (BOM) Luuk <luuk@invalid.lan> - 2019-01-05 19:34 +0100
                Re: Twice UTF-8 byte-order mark (BOM) Tim Streater <timstreater@greenbee.net> - 2019-01-05 19:08 +0000
                Re: Twice UTF-8 byte-order mark (BOM) Luuk <luuk@invalid.lan> - 2019-01-06 20:25 +0100

csiph-web