Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
| From | Keith Thompson <kst-u@mib.org> |
|---|---|
| Newsgroups | comp.lang.c |
| Subject | Re: Line read from csv file with fgets has two characters before the terminating one |
| Date | 2016-12-16 12:02 -0800 |
| Organization | None to speak of |
| Message-ID | <lninqjk5gu.fsf@kst-u.example.com> (permalink) |
| References | (6 earlier) <p6865cd4ikg893u9hqr6a9olpdlm6otjio@4ax.com> <lnd1gslqj3.fsf@kst-u.example.com> <5nr65c5eljh0h12807v0qgtihk4a9cp6ah@4ax.com> <ln4m24l5qs.fsf@kst-u.example.com> <ng085c53ikcet2dvjlakor9ipmtu05mkca@4ax.com> |
Geoff <geoff@invalid.invalid> writes:
> On Thu, 15 Dec 2016 22:58:51 -0800, Keith Thompson <kst-u@mib.org> wrote:
[...]
>>Which means that a CSV file exported in "CSV-UTF-8" format is not a
>>text file. If you try to open it with a text editor, it will likely
>>appear as one very long line with no line endings at all (unless some
>>MacOS editors still recognize pre-OSx text files). It means that
>>"wc -l filename.csv" won't tell you how many lines are in the file.
>>
>>That's astonishing.
>
> It has magic at the beginning of the file but that's all:
>
> hexdump -C Workbook1.csv
> 00000000 ef bb bf 44 61 74 65 2c 4e 75 6d 62 65 72 0d 33 |...Date,Number.3|
> 00000010 2f 36 2f 30 30 2c 31 38 39 0d 39 2f 35 2f 30 31 |/6/00,189.9/5/01|
> 00000020 2c 31 39 31 0d 31 30 2f 39 2f 30 32 2c 31 39 32 |,191.10/9/02,192|
> 00000030 0d 31 31 2f 37 2f 30 32 2c 31 38 39 0d 32 2f 32 |.11/7/02,189.2/2|
> 00000040 30 2f 30 34 2c 31 39 38 0d 33 2f 38 2f 30 34 2c |0/04,198.3/8/04,|
> 00000050 31 39 38 0d 31 2f 32 37 2f 30 35 2c 32 30 34 0d |198.1/27/05,204.|
> 00000060 31 30 2f 31 39 2f 30 35 2c 32 30 30 0d 34 2f 35 |10/19/05,200.4/5|
> 00000070 2f 30 36 2c 31 39 39 0d 36 2f 39 2f 30 36 2c 31 |/06,199.6/9/06,1|
> 00000080 39 38 0d 31 2f 37 2f 30 38 2c 32 30 34 0d 31 2f |98.1/7/08,204.1/|
> 00000090 39 2f 30 38 2c 32 30 34 0d 31 2f 32 37 2f 30 39 |9/08,204.1/27/09|
> 000000a0 2c 32 30 30 0d 34 2f 38 2f 30 39 2c 32 30 33 0d |,200.4/8/09,203.|
> 000000b0 35 2f 34 2f 30 39 2c 32 30 31 0d 38 2f 31 32 2f |5/4/09,201.8/12/|
> 000000c0 30 39 2c 32 30 35 0d 31 30 2f 31 35 2f 30 39 2c |09,205.10/15/09,|
> 000000d0 32 30 32 0d 37 2f 31 39 2f 31 30 2c 31 39 37 |202.7/19/10,197|
> 000000df
[...]
The "magic" at the beginning is a UTF-8 BOM (Byte Order Mark).
A BOM are normally necessary only for UTF-16. It's a U+FEFF character
whose byte order tells you whether you're dealing with big-endian or
little-endian UTF-16. A BOM in a UTF-8 file tells you nothing about
byte ordering (since the issue doesn't arise for UTF-8). It can tell
you that the file is (probably) UTF-8, but outside of Microsoft software
UTF-8 BOMs are not necessary and not recommended.
When I unconvert the hex dump, the "file" command (on Ubuntu) says:
foo.csv: UTF-8 Unicode (with BOM) text, with CR line terminators
Ick.
No doubt Excel itself can read such a file, and LibreOffice Calc
and Gnumeric can open it as well, but other tools (not designed
specifically to deal with spreadsheets or CSV files) are likely to
have problems with it.
--
Keith Thompson (The_Other_Keith) kst-u@mib.org <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Line read from csv file with fgets has two characters before the terminating one Alla _ <modelling.data@gmail.com> - 2016-12-14 00:04 -0800
Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-14 11:10 +0000
Re: Line read from csv file with fgets has two characters before the terminating one Alla _ <modelling.data@gmail.com> - 2016-12-14 23:50 -0800
Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 08:52 +0000
Re: Line read from csv file with fgets has two characters before the terminating one Alla _ <modelling.data@gmail.com> - 2016-12-15 00:58 -0800
Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 09:09 +0000
Re: Line read from csv file with fgets has two characters before the terminating one Alla _ <modelling.data@gmail.com> - 2016-12-15 04:23 -0800
Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 12:50 +0000
Re: Line read from csv file with fgets has two characters before the terminating one BartC <bc@freeuk.com> - 2016-12-15 14:14 +0000
Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 14:42 +0000
Re: Line read from csv file with fgets has two characters before the terminating one Keith Thompson <kst-u@mib.org> - 2016-12-15 09:49 -0800
Re: Line read from csv file with fgets has two characters before the terminating one gazelle@shell.xmission.com (Kenny McCormack) - 2016-12-15 17:58 +0000
Re: Line read from csv file with fgets has two characters before the terminating one BartC <bc@freeuk.com> - 2016-12-15 18:22 +0000
Re: Line read from csv file with fgets has two characters before the terminating one Keith Thompson <kst-u@mib.org> - 2016-12-15 11:51 -0800
Re: Line read from csv file with fgets has two characters before the terminating one BartC <bc@freeuk.com> - 2016-12-15 20:35 +0000
Re: Line read from csv file with fgets has two characters before the terminating one Keith Thompson <kst-u@mib.org> - 2016-12-15 12:58 -0800
Re: Line read from csv file with fgets has two characters before the terminating one Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-12-15 13:11 -0800
Re: Line read from csv file with fgets has two characters before the terminating one Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-12-16 02:14 +0000
Re: Line read from csv file with fgets has two characters before the terminating one BartC <bc@freeuk.com> - 2016-12-16 11:25 +0000
Re: Line read from csv file with fgets has two characters before the terminating one Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-12-16 16:41 +0000
Re: Line read from csv file with fgets has two characters before the terminating one BartC <bc@freeuk.com> - 2016-12-16 22:08 +0000
Re: Line read from csv file with fgets has two characters before the terminating one Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-12-17 02:48 +0000
Re: Line read from csv file with fgets has two characters before the terminating one BartC <bc@freeuk.com> - 2016-12-17 11:54 +0000
Re: Line read from csv file with fgets has two characters before the terminating one Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-12-17 15:33 +0000
Re: Line read from csv file with fgets has two characters before the terminating one supercat@casperkitty.com - 2016-12-17 08:27 -0800
Re: Line read from csv file with fgets has two characters before the terminating one BartC <bc@freeuk.com> - 2016-12-17 17:25 +0000
Re: Line read from csv file with fgets has two characters before the terminating one Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-12-17 09:53 -0800
Re: Line read from csv file with fgets has two characters before the terminating one BartC <bc@freeuk.com> - 2016-12-17 18:21 +0000
Re: Line read from csv file with fgets has two characters before the terminating one supercat@casperkitty.com - 2016-12-17 10:30 -0800
Re: Line read from csv file with fgets has two characters before the terminating one BartC <bc@freeuk.com> - 2016-12-17 22:39 +0000
Re: Line read from csv file with fgets has two characters before the terminating one Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-12-17 14:51 -0800
Re: Line read from csv file with fgets has two characters before the terminating one BartC <bc@freeuk.com> - 2016-12-17 23:12 +0000
Re: Line read from csv file with fgets has two characters before the terminating one supercat@casperkitty.com - 2016-12-18 08:37 -0800
Re: Line read from csv file with fgets has two characters before the terminating one Keith Thompson <kst-u@mib.org> - 2016-12-18 13:14 -0800
Re: Line read from csv file with fgets has two characters before the terminating one Geoff <geoff@invalid.invalid> - 2016-12-15 15:03 -0800
Re: Line read from csv file with fgets has two characters before the terminating one Keith Thompson <kst-u@mib.org> - 2016-12-15 15:29 -0800
Re: Line read from csv file with fgets has two characters before the terminating one Geoff <geoff@invalid.invalid> - 2016-12-15 20:48 -0800
Re: Line read from csv file with fgets has two characters before the terminating one Keith Thompson <kst-u@mib.org> - 2016-12-15 22:58 -0800
OSX line endings (Was: Line read from csv file with fgets has two characters before the terminating one) gazelle@shell.xmission.com (Kenny McCormack) - 2016-12-16 10:22 +0000
Re: OSX line endings (Was: Line read from csv file with fgets has two characters before the terminating one) Geoff <geoff@invalid.invalid> - 2016-12-16 07:29 -0800
Making OSX's boot drive case-sensitve (Was: OSX line endings (Was: Line read from csv file with fgets has two characters before the terminating one)) gazelle@shell.xmission.com (Kenny McCormack) - 2016-12-17 10:06 +0000
Re: Making OSX's boot drive case-sensitve (Was: OSX line endings (Was: Line read from csv file with fgets has two characters before the terminating one)) Geoff <geoff@invalid.invalid> - 2016-12-17 12:39 -0800
Re: Making OSX's boot drive case-sensitve (Was: OSX line endings (Was: Line read from csv file with fgets has two characters before the terminating one)) gazelle@shell.xmission.com (Kenny McCormack) - 2016-12-18 13:54 +0000
Re: Making OSX's boot drive case-sensitve (Was: OSX line endings (Was: Line read from csv file with fgets has two characters before the terminating one)) Geoff <geoff@invalid.invalid> - 2016-12-18 17:57 -0800
Re: Line read from csv file with fgets has two characters before the terminating one Geoff <geoff@invalid.invalid> - 2016-12-16 07:12 -0800
Re: Line read from csv file with fgets has two characters before the terminating one Keith Thompson <kst-u@mib.org> - 2016-12-16 12:02 -0800
Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 23:29 +0000
Re: Line read from csv file with fgets has two characters before the terminating one Alla _ <modelling.data@gmail.com> - 2016-12-16 02:37 -0800
Re: Line read from csv file with fgets has two characters before the terminating one Geoff <geoff@invalid.invalid> - 2016-12-16 07:51 -0800
Re: Line read from csv file with fgets has two characters before the terminating one Manfred <noname@invalid.add> - 2016-12-15 12:22 +0100
Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 11:29 +0000
Re: Line read from csv file with fgets has two characters before the terminating one Melzzzzz <Melzzzzz@zzzzz.com> - 2016-12-15 12:40 +0100
Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 12:00 +0000
Re: Line read from csv file with fgets has two characters before the terminating one Melzzzzz <mel@zzzzz.com> - 2016-12-15 13:09 +0100
Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 12:43 +0000
Re: Line read from csv file with fgets has two characters before the terminating one "James R. Kuyper" <jameskuyper@verizon.net> - 2016-12-15 09:18 -0500
Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 14:49 +0000
Re: Line read from csv file with fgets has two characters before the terminating one "James R. Kuyper" <jameskuyper@verizon.net> - 2016-12-15 10:34 -0500
Re: Line read from csv file with fgets has two characters before the terminating one Manfred <noname@invalid.add> - 2016-12-15 16:34 +0100
Re: Line read from csv file with fgets has two characters before the terminating one Alla _ <modelling.data@gmail.com> - 2016-12-15 00:22 -0800
Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 09:04 +0000
Re: Line read from csv file with fgets has two characters before the terminating one mark.bluemel@gmail.com - 2016-12-14 03:38 -0800
csiph-web