Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.clipper.visual-objects > #8805 > unrolled thread

UTF-16LE to UTF-8 or ANSI

Started by"Philippe Bayard (Belgium)" <philbayard@gmail.com>
First post2015-12-15 03:47 -0800
Last post2015-12-15 13:38 -0800
Articles 4 — 3 participants

Back to article view | Back to comp.lang.clipper.visual-objects


Contents

  UTF-16LE to UTF-8 or ANSI "Philippe Bayard (Belgium)" <philbayard@gmail.com> - 2015-12-15 03:47 -0800
    Re: UTF-16LE to UTF-8 or ANSI norbert@kolb.cx - 2015-12-15 08:54 -0800
    Re: UTF-16LE to UTF-8 or ANSI "Arne Ortlinghaus" <Arne.Ortlinghaus@acs.it> - 2015-12-15 18:25 +0100
      Re: UTF-16LE to UTF-8 or ANSI "Philippe Bayard (Belgium)" <philbayard@gmail.com> - 2015-12-15 13:38 -0800

#8805 — UTF-16LE to UTF-8 or ANSI

From"Philippe Bayard (Belgium)" <philbayard@gmail.com>
Date2015-12-15 03:47 -0800
SubjectUTF-16LE to UTF-8 or ANSI
Message-ID<4345e777-424b-4b7c-a95f-937e0ab3e85e@googlegroups.com>
Dear everyone,

in some routine, i must read a file containing a datetime string.
This file is encoded in UTF-16LE ( or at least it's what i think ).

I've opened in with notepad++ and i have a FFFE BOM which seems to indicate this.

The string in Hex is : 

ff fe 32 20 30 20 31 20 35 20 31 20 32 20 31 20 31 20 31 20 32 20 30 20 30 20 31 20 36 20 0d 20 0a 20  

And notepad++ gave this ( and notepad++ also says is UTF16LE) : 

ÿþ2 0 1 5 1 2 1 1 1 2 0 0 1 6 

Did someone already convert such a string from UTF16-LE (Little Endian) to another encoding ( such as UTF-8 or ANSI )?

I've tried using the convertCodePage with 1200	

https://msdn.microsoft.com/en-us/library/dd317756%28v=vs.85%29.aspx
-> 1200 utf-16	Unicode UTF-16, little endian byte order (BMP of ISO 10646); available only to managed applications

But i can't get my string back...

Any help would be appreciate!

Best regards,
Jean-Philippe


 

[toc] | [next] | [standalone]


#8808

Fromnorbert@kolb.cx
Date2015-12-15 08:54 -0800
Message-ID<b222f359-dd82-4c0f-b605-9fdfbc137e32@googlegroups.com>
In reply to#8805
Hi Jean-Philippe,

write your own function like this:

FUNCTION UTF16Date2Vo(cDate)
  LOCAL dRet 	AS DATE
  LOCAL cTime	AS STRING

  IF SubStr(cDate,1,2) == CHR(255) + CHR(254)
    cDate := StrTran(cDate, " ", "")
    dRet := SToD(SubStr(cDate,3,8))
    cTime := substr(cDate,11,2) +":" + substr(cDate,13,2) +":" + substr(cDate,15,2)
  ELSE
    dRet:= NULL_DATE
  ENDIF
	
  RETURN dRet
	

Best regards
Norbert

[toc] | [prev] | [next] | [standalone]


#8809

From"Arne Ortlinghaus" <Arne.Ortlinghaus@acs.it>
Date2015-12-15 18:25 +0100
Message-ID<n4pi9k$bd2$1@dont-email.me>
In reply to#8805
Hello Philippe,

the FFFE at the beginning is a character sequence that is used to indicate 
UTF16 LE, the standard Windows Unicode format.

These characters should be eliminated when reading the file.

Converting from UTF 16 Little Endian to ANSI is simple.
You read the file into a string without looking at the many "NULLS".
Then you use the function below to transform the string to ANSI, useable 
normally in VO.

Converting to UTF-8 is also possible but does not make any sense within the 
program. It should be used only when saving a file in UTF-8 format or 
reading it from UTF-8.


function UnicodeToAnsi (c)
//p Konvertierung Unicodestrings nach Ansi (Western)
//a c - String: Eingabestring
//r String: Geänderter String
//g Strings
local cRet as string
local n as dword
local ncp as dword
local pUnicode, pAnsi as ptr
local iSLen as int
local iNum as dword

cRet := ""
n := SLen(c)
if n > 0
   ncp := 0
   iSLen := int(n) / 2
 // Must call WideCharToMultiByte() function once to determine length
 // of resultant Ansi string
 pUnicode := String2Psz(c)
  iNum := dword(WideCharToMultiByte(;
   ncp,;      // CodePage AS DWORD
   0,;         // dwFlags AS DWORD
   pUnicode,;    // lpWideCharStr AS PSZ
   isLen,; // cchWideChar AS INT
   null_psz,;    // lpMultiByteStr AS PSZ
   0,;         // cchMultiByte AS INT
   null_ptr,;     // lpDefaultChar AS PSZ
   null_ptr ))    // lpUsedDefaultChar AS LOGIC PTR

 // Physically allocate the memory for the buffer
  pAnsi := MemAlloc( iNum + 1 ) // NOTE: +1 for terminating NULL char
  if pAnsi != null_ptr
   MemClear(pAnsi, iNum + 1 )
    iNum := dword(WideCharToMultiByte(;
   ncp, 0, pUnicode, iSLen, pAnsi, int(iNum), null_ptr, null_ptr ))
  if iNum > 0
         cRet := Mem2String(pAnsi, iNum)
      endif
    MemFree( pAnsi )

 endif
endif
return cRet


Arne Ortlinghaus
ACS Data Systems




"Philippe Bayard (Belgium)" <philbayard@gmail.com> schrieb im Newsbeitrag 
news:4345e777-424b-4b7c-a95f-937e0ab3e85e@googlegroups.com...
Dear everyone,

in some routine, i must read a file containing a datetime string.
This file is encoded in UTF-16LE ( or at least it's what i think ).

I've opened in with notepad++ and i have a FFFE BOM which seems to indicate 
this.

The string in Hex is :

ff fe 32 20 30 20 31 20 35 20 31 20 32 20 31 20 31 20 31 20 32 20 30 20 30 
20 31 20 36 20 0d 20 0a 20

And notepad++ gave this ( and notepad++ also says is UTF16LE) :

ÿþ2 0 1 5 1 2 1 1 1 2 0 0 1 6

Did someone already convert such a string from UTF16-LE (Little Endian) to 
another encoding ( such as UTF-8 or ANSI )?

I've tried using the convertCodePage with 1200

https://msdn.microsoft.com/en-us/library/dd317756%28v=vs.85%29.aspx
-> 1200 utf-16 Unicode UTF-16, little endian byte order (BMP of ISO 10646); 
available only to managed applications

But i can't get my string back...

Any help would be appreciate!

Best regards,
Jean-Philippe


[toc] | [prev] | [next] | [standalone]


#8813

From"Philippe Bayard (Belgium)" <philbayard@gmail.com>
Date2015-12-15 13:38 -0800
Message-ID<e452f185-f430-48cb-8077-d74e72e3640e@googlegroups.com>
In reply to#8809
Thank you very much to both of you.
It works like a charm.

Best regards,
Philippe.


Le mardi 15 décembre 2015 18:25:32 UTC+1, Arne Ortlinghaus a écrit :
> Hello Philippe,
> 
> the FFFE at the beginning is a character sequence that is used to indicate 
> UTF16 LE, the standard Windows Unicode format.
> 
> These characters should be eliminated when reading the file.
> 
> Converting from UTF 16 Little Endian to ANSI is simple.
> You read the file into a string without looking at the many "NULLS".
> Then you use the function below to transform the string to ANSI, useable 
> normally in VO.
> 
> Converting to UTF-8 is also possible but does not make any sense within the 
> program. It should be used only when saving a file in UTF-8 format or 
> reading it from UTF-8.
> 
> 
> function UnicodeToAnsi (c)
> //p Konvertierung Unicodestrings nach Ansi (Western)
> //a c - String: Eingabestring
> //r String: Geänderter String
> //g Strings
> local cRet as string
> local n as dword
> local ncp as dword
> local pUnicode, pAnsi as ptr
> local iSLen as int
> local iNum as dword
> 
> cRet := ""
> n := SLen(c)
> if n > 0
>    ncp := 0
>    iSLen := int(n) / 2
>  // Must call WideCharToMultiByte() function once to determine length
>  // of resultant Ansi string
>  pUnicode := String2Psz(c)
>   iNum := dword(WideCharToMultiByte(;
>    ncp,;      // CodePage AS DWORD
>    0,;         // dwFlags AS DWORD
>    pUnicode,;    // lpWideCharStr AS PSZ
>    isLen,; // cchWideChar AS INT
>    null_psz,;    // lpMultiByteStr AS PSZ
>    0,;         // cchMultiByte AS INT
>    null_ptr,;     // lpDefaultChar AS PSZ
>    null_ptr ))    // lpUsedDefaultChar AS LOGIC PTR
> 
>  // Physically allocate the memory for the buffer
>   pAnsi := MemAlloc( iNum + 1 ) // NOTE: +1 for terminating NULL char
>   if pAnsi != null_ptr
>    MemClear(pAnsi, iNum + 1 )
>     iNum := dword(WideCharToMultiByte(;
>    ncp, 0, pUnicode, iSLen, pAnsi, int(iNum), null_ptr, null_ptr ))
>   if iNum > 0
>          cRet := Mem2String(pAnsi, iNum)
>       endif
>     MemFree( pAnsi )
> 
>  endif
> endif
> return cRet
> 
> 
> Arne Ortlinghaus
> ACS Data Systems
> 
> 
> 
> 
> "Philippe Bayard (Belgium)" <philbayard@gmail.com> schrieb im Newsbeitrag 
> news:4345e777-424b-4b7c-a95f-937e0ab3e85e@googlegroups.com...
> Dear everyone,
> 
> in some routine, i must read a file containing a datetime string.
> This file is encoded in UTF-16LE ( or at least it's what i think ).
> 
> I've opened in with notepad++ and i have a FFFE BOM which seems to indicate 
> this.
> 
> The string in Hex is :
> 
> ff fe 32 20 30 20 31 20 35 20 31 20 32 20 31 20 31 20 31 20 32 20 30 20 30 
> 20 31 20 36 20 0d 20 0a 20
> 
> And notepad++ gave this ( and notepad++ also says is UTF16LE) :
> 
> ÿþ2 0 1 5 1 2 1 1 1 2 0 0 1 6
> 
> Did someone already convert such a string from UTF16-LE (Little Endian) to 
> another encoding ( such as UTF-8 or ANSI )?
> 
> I've tried using the convertCodePage with 1200
> 
> https://msdn.microsoft.com/en-us/library/dd317756%28v=vs.85%29.aspx
> -> 1200 utf-16 Unicode UTF-16, little endian byte order (BMP of ISO 10646); 
> available only to managed applications
> 
> But i can't get my string back...
> 
> Any help would be appreciate!
> 
> Best regards,
> Jean-Philippe

[toc] | [prev] | [standalone]


Back to top | Article view | comp.lang.clipper.visual-objects


csiph-web