Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #99315 > unrolled thread

Line read from csv file with fgets has two characters before the terminating one

Back to article view | Back to comp.lang.c

Contents

  Line read from csv file with fgets has two characters before the terminating one Alla _ <modelling.data@gmail.com> - 2016-12-14 00:04 -0800
    Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-14 11:10 +0000
      Re: Line read from csv file with fgets has two characters before the terminating one Alla _ <modelling.data@gmail.com> - 2016-12-14 23:50 -0800
        Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 08:52 +0000
          Re: Line read from csv file with fgets has two characters before the terminating one Alla _ <modelling.data@gmail.com> - 2016-12-15 00:58 -0800
            Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 09:09 +0000
              Re: Line read from csv file with fgets has two characters before the terminating one Alla _ <modelling.data@gmail.com> - 2016-12-15 04:23 -0800
                Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 12:50 +0000
                  Re: Line read from csv file with fgets has two characters before the terminating one BartC <bc@freeuk.com> - 2016-12-15 14:14 +0000
                    Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 14:42 +0000
                    Re: Line read from csv file with fgets has two characters before the terminating one Keith Thompson <kst-u@mib.org> - 2016-12-15 09:49 -0800
                      Re: Line read from csv file with fgets has two characters before the terminating one gazelle@shell.xmission.com (Kenny McCormack) - 2016-12-15 17:58 +0000
                      Re: Line read from csv file with fgets has two characters before the terminating one BartC <bc@freeuk.com> - 2016-12-15 18:22 +0000
                        Re: Line read from csv file with fgets has two characters before the terminating one Keith Thompson <kst-u@mib.org> - 2016-12-15 11:51 -0800
                          Re: Line read from csv file with fgets has two characters before the terminating one BartC <bc@freeuk.com> - 2016-12-15 20:35 +0000
                            Re: Line read from csv file with fgets has two characters before the terminating one Keith Thompson <kst-u@mib.org> - 2016-12-15 12:58 -0800
                            Re: Line read from csv file with fgets has two characters before the terminating one Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-12-15 13:11 -0800
                    Re: Line read from csv file with fgets has two characters before the terminating one Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-12-16 02:14 +0000
                      Re: Line read from csv file with fgets has two characters before the terminating one BartC <bc@freeuk.com> - 2016-12-16 11:25 +0000
                        Re: Line read from csv file with fgets has two characters before the terminating one Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-12-16 16:41 +0000
                          Re: Line read from csv file with fgets has two characters before the terminating one BartC <bc@freeuk.com> - 2016-12-16 22:08 +0000
                            Re: Line read from csv file with fgets has two characters before the terminating one Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-12-17 02:48 +0000
                              Re: Line read from csv file with fgets has two characters before the terminating one BartC <bc@freeuk.com> - 2016-12-17 11:54 +0000
                                Re: Line read from csv file with fgets has two characters before the terminating one Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-12-17 15:33 +0000
                              Re: Line read from csv file with fgets has two characters before the terminating one supercat@casperkitty.com - 2016-12-17 08:27 -0800
                                Re: Line read from csv file with fgets has two characters before the terminating one BartC <bc@freeuk.com> - 2016-12-17 17:25 +0000
                                  Re: Line read from csv file with fgets has two characters before the terminating one Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-12-17 09:53 -0800
                                    Re: Line read from csv file with fgets has two characters before the terminating one BartC <bc@freeuk.com> - 2016-12-17 18:21 +0000
                                  Re: Line read from csv file with fgets has two characters before the terminating one supercat@casperkitty.com - 2016-12-17 10:30 -0800
                                    Re: Line read from csv file with fgets has two characters before the terminating one BartC <bc@freeuk.com> - 2016-12-17 22:39 +0000
                                      Re: Line read from csv file with fgets has two characters before the terminating one Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-12-17 14:51 -0800
                                        Re: Line read from csv file with fgets has two characters before the terminating one BartC <bc@freeuk.com> - 2016-12-17 23:12 +0000
                                        Re: Line read from csv file with fgets has two characters before the terminating one supercat@casperkitty.com - 2016-12-18 08:37 -0800
                                          Re: Line read from csv file with fgets has two characters before the terminating one Keith Thompson <kst-u@mib.org> - 2016-12-18 13:14 -0800
              Re: Line read from csv file with fgets has two characters before the terminating one Geoff <geoff@invalid.invalid> - 2016-12-15 15:03 -0800
                Re: Line read from csv file with fgets has two characters before the terminating one Keith Thompson <kst-u@mib.org> - 2016-12-15 15:29 -0800
                  Re: Line read from csv file with fgets has two characters before the terminating one Geoff <geoff@invalid.invalid> - 2016-12-15 20:48 -0800
                    Re: Line read from csv file with fgets has two characters before the terminating one Keith Thompson <kst-u@mib.org> - 2016-12-15 22:58 -0800
                      OSX line endings (Was: Line read from csv file with fgets has two characters before the terminating one) gazelle@shell.xmission.com (Kenny McCormack) - 2016-12-16 10:22 +0000
                        Re: OSX line endings (Was: Line read from csv file with fgets has two characters before the terminating one) Geoff <geoff@invalid.invalid> - 2016-12-16 07:29 -0800
                          Making OSX's boot drive case-sensitve (Was: OSX line endings (Was: Line read from csv file with fgets has two characters before the terminating one)) gazelle@shell.xmission.com (Kenny McCormack) - 2016-12-17 10:06 +0000
                            Re: Making OSX's boot drive case-sensitve (Was: OSX line endings (Was: Line read from csv file with fgets has two characters before the terminating one)) Geoff <geoff@invalid.invalid> - 2016-12-17 12:39 -0800
                              Re: Making OSX's boot drive case-sensitve (Was: OSX line endings (Was: Line read from csv file with fgets has two characters before the terminating one)) gazelle@shell.xmission.com (Kenny McCormack) - 2016-12-18 13:54 +0000
                                Re: Making OSX's boot drive case-sensitve (Was: OSX line endings (Was: Line read from csv file with fgets has two characters before the terminating one)) Geoff <geoff@invalid.invalid> - 2016-12-18 17:57 -0800
                      Re: Line read from csv file with fgets has two characters before the terminating one Geoff <geoff@invalid.invalid> - 2016-12-16 07:12 -0800
                        Re: Line read from csv file with fgets has two characters before the terminating one Keith Thompson <kst-u@mib.org> - 2016-12-16 12:02 -0800
                Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 23:29 +0000
                Re: Line read from csv file with fgets has two characters before the terminating one Alla _ <modelling.data@gmail.com> - 2016-12-16 02:37 -0800
                  Re: Line read from csv file with fgets has two characters before the terminating one Geoff <geoff@invalid.invalid> - 2016-12-16 07:51 -0800
            Re: Line read from csv file with fgets has two characters before the terminating one Manfred <noname@invalid.add> - 2016-12-15 12:22 +0100
              Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 11:29 +0000
                Re: Line read from csv file with fgets has two characters before the terminating one Melzzzzz <Melzzzzz@zzzzz.com> - 2016-12-15 12:40 +0100
                  Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 12:00 +0000
                    Re: Line read from csv file with fgets has two characters before the terminating one Melzzzzz <mel@zzzzz.com> - 2016-12-15 13:09 +0100
                      Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 12:43 +0000
                    Re: Line read from csv file with fgets has two characters before the terminating one "James R. Kuyper" <jameskuyper@verizon.net> - 2016-12-15 09:18 -0500
                      Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 14:49 +0000
                        Re: Line read from csv file with fgets has two characters before the terminating one "James R. Kuyper" <jameskuyper@verizon.net> - 2016-12-15 10:34 -0500
                Re: Line read from csv file with fgets has two characters before the terminating one Manfred <noname@invalid.add> - 2016-12-15 16:34 +0100
      Re: Line read from csv file with fgets has two characters before the terminating one Alla _ <modelling.data@gmail.com> - 2016-12-15 00:22 -0800
        Re: Line read from csv file with fgets has two characters before the terminating one Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 09:04 +0000
    Re: Line read from csv file with fgets has two characters before the terminating one mark.bluemel@gmail.com - 2016-12-14 03:38 -0800

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

#99729 — Making OSX's boot drive case-sensitve (Was: OSX line endings (Was: Line read from csv file with fgets has two characters before the terminating one))

In article <44185cpktkrvh14cc5p2egudqtbilbmf81@4ax.com>,
Geoff  <geoff@invalid.invalid> wrote:
...
>>OSX is still littered with bits of "Old Mac OS" compatibility things, such
>>as the non-case-sensitive filesystem.  This, in particular, makes OSX "not
>>quite Unix".
>
>It's case-insensitive by default but you can enable it.

I'd be worried about doing that on the boot (system) drive, for fear of
breaking something in the system that depends on it.

Besides, wouldn't you have to reformat the volume in order to enable it?

-- 
The randomly chosen signature file that would have appeared here is more than 4
lines long.  As such, it violates one or more Usenet RFCs.  In order to remain
in compliance with said RFCs, the actual sig can be found at the following URL:
	http://user.xmission.com/~gazelle/Sigs/ForFoxViewers

[toc] | [prev] | [next] | [standalone]

#99770 — Re: Making OSX's boot drive case-sensitve (Was: OSX line endings (Was: Line read from csv file with fgets has two characters before the terminating one))

On Sat, 17 Dec 2016 10:06:50 +0000 (UTC), gazelle@shell.xmission.com
(Kenny McCormack) wrote:

>In article <44185cpktkrvh14cc5p2egudqtbilbmf81@4ax.com>,
>Geoff  <geoff@invalid.invalid> wrote:
>...
>>>OSX is still littered with bits of "Old Mac OS" compatibility things, such
>>>as the non-case-sensitive filesystem.  This, in particular, makes OSX "not
>>>quite Unix".
>>
>>It's case-insensitive by default but you can enable it.
>
>I'd be worried about doing that on the boot (system) drive, for fear of
>breaking something in the system that depends on it.
>
>Besides, wouldn't you have to reformat the volume in order to enable it?

I believe the recommendation from Apple is not to do it to the boot
drive. AFAIK, you don't have to reformat to change it. I've never
personally attempted it.

[toc] | [prev] | [next] | [standalone]

#99798 — Re: Making OSX's boot drive case-sensitve (Was: OSX line endings (Was: Line read from csv file with fgets has two characters before the terminating one))

In article <lf8b5cpppuds63l7djvtbqg05l1654lqvs@4ax.com>,
Geoff  <geoff@invalid.invalid> wrote:
...
>>>It's case-insensitive by default but you can enable it.
>>
>>I'd be worried about doing that on the boot (system) drive, for fear of
>>breaking something in the system that depends on it.
>>
>>Besides, wouldn't you have to reformat the volume in order to enable it?
>
>I believe the recommendation from Apple is not to do it to the boot
>drive. AFAIK, you don't have to reformat to change it. I've never
>personally attempted it.

Yeah, that's about what I expected - that doing it on the boot volume would
be risky.  And the point is, that's exactly where you'd want to do it - if
the goal is to make OSX act like real Unix.

Not a big deal - I admit I was a little thrown when I first discovered this
little tidbit, but I've gotten used to it.  Just my point is, as I stated
in a previous response, that OSX does still have these little bits of
pre-10 compatibility in it.
9
-- 
"The party of Lincoln has become the party of John Wilkes Booth."

	- Carlos Alazraqui -

[toc] | [prev] | [next] | [standalone]

#99852 — Re: Making OSX's boot drive case-sensitve (Was: OSX line endings (Was: Line read from csv file with fgets has two characters before the terminating one))

On Sun, 18 Dec 2016 13:54:24 +0000 (UTC), gazelle@shell.xmission.com
(Kenny McCormack) wrote:

>In article <lf8b5cpppuds63l7djvtbqg05l1654lqvs@4ax.com>,
>Geoff  <geoff@invalid.invalid> wrote:
>...
>>>>It's case-insensitive by default but you can enable it.
>>>
>>>I'd be worried about doing that on the boot (system) drive, for fear of
>>>breaking something in the system that depends on it.
>>>
>>>Besides, wouldn't you have to reformat the volume in order to enable it?
>>
>>I believe the recommendation from Apple is not to do it to the boot
>>drive. AFAIK, you don't have to reformat to change it. I've never
>>personally attempted it.
>
>Yeah, that's about what I expected - that doing it on the boot volume would
>be risky.  And the point is, that's exactly where you'd want to do it - if
>the goal is to make OSX act like real Unix.
>

The goal was to make OSX act like OS9 and Windows. The vast majority
of installed systems are case-insensitive. 

I run into problems where tools are used that generate file names in
case-insensitive systems and then expect those files to migrate
seamlessly to *nix servers.

>Not a big deal - I admit I was a little thrown when I first discovered this
>little tidbit, but I've gotten used to it.  Just my point is, as I stated
>in a previous response, that OSX does still have these little bits of
>pre-10 compatibility in it.
>9

[toc] | [prev] | [next] | [standalone]

#99656

On Thu, 15 Dec 2016 22:58:51 -0800, Keith Thompson <kst-u@mib.org> wrote:

>Geoff <geoff@invalid.invalid> writes:
>> On Thu, 15 Dec 2016 15:29:52 -0800, Keith Thompson <kst-u@mib.org>
>> wrote:
>>>Geoff <geoff@invalid.invalid> writes:
>>>> On Thu, 15 Dec 2016 09:09:19 +0000, Richard Heathfield
>>>> <rjh@cpax.org.uk> wrote:
>>>>>Excel is by Microsoft. Microsoft wrote Windows, which uses screwy line 
>>>>>endings for text files. You don't think Excel might write a screwy text 
>>>>>file even on MacOS, just out of sheer habit?
>>>>
>>>> Excel 2016 for Mac outputs 0D-only end of lines in csv files.
>>>
>>>Are you sure it's not 0A?
>>
>> Absolutely sure. But there are FOUR different csv format selections in
>> the Save As... dialog, only one of them outputs a single-byte EOL.
>>
>> The four selections are:
>> CSV UTF-8 (Comma delimited) (.csv)
>> Comma Separated Values (.csv)
>> Windows Comma Separated (.csv)
>> MS-DOS Comma Separated (.csv)
>>
>> Only the first one (CSV UTF-8) uses the 0D ending. All the others use
>> 0D, 0A. This was proven by using hexdump on a sample file.
>>
>> The CSV UTF-8 format is at the top of the list, under "Common
>> Formats". The other are listed as "Specialty Formats".
>
>Which means that a CSV file exported in "CSV-UTF-8" format is not a
>text file.  If you try to open it with a text editor, it will likely
>appear as one very long line with no line endings at all (unless some
>MacOS editors still recognize pre-OSx text files).  It means that
>"wc -l filename.csv" won't tell you how many lines are in the file.
>
>That's astonishing.

It has magic at the beginning of the file but that's all:

hexdump -C Workbook1.csv
00000000  ef bb bf 44 61 74 65 2c  4e 75 6d 62 65 72 0d 33  |...Date,Number.3|
00000010  2f 36 2f 30 30 2c 31 38  39 0d 39 2f 35 2f 30 31  |/6/00,189.9/5/01|
00000020  2c 31 39 31 0d 31 30 2f  39 2f 30 32 2c 31 39 32  |,191.10/9/02,192|
00000030  0d 31 31 2f 37 2f 30 32  2c 31 38 39 0d 32 2f 32  |.11/7/02,189.2/2|
00000040  30 2f 30 34 2c 31 39 38  0d 33 2f 38 2f 30 34 2c  |0/04,198.3/8/04,|
00000050  31 39 38 0d 31 2f 32 37  2f 30 35 2c 32 30 34 0d  |198.1/27/05,204.|
00000060  31 30 2f 31 39 2f 30 35  2c 32 30 30 0d 34 2f 35  |10/19/05,200.4/5|
00000070  2f 30 36 2c 31 39 39 0d  36 2f 39 2f 30 36 2c 31  |/06,199.6/9/06,1|
00000080  39 38 0d 31 2f 37 2f 30  38 2c 32 30 34 0d 31 2f  |98.1/7/08,204.1/|
00000090  39 2f 30 38 2c 32 30 34  0d 31 2f 32 37 2f 30 39  |9/08,204.1/27/09|
000000a0  2c 32 30 30 0d 34 2f 38  2f 30 39 2c 32 30 33 0d  |,200.4/8/09,203.|
000000b0  35 2f 34 2f 30 39 2c 32  30 31 0d 38 2f 31 32 2f  |5/4/09,201.8/12/|
000000c0  30 39 2c 32 30 35 0d 31  30 2f 31 35 2f 30 39 2c  |09,205.10/15/09,|
000000d0  32 30 32 0d 37 2f 31 39  2f 31 30 2c 31 39 37     |202.7/19/10,197|
000000df

If you select "Comma Separated Values (.csv) you get plain ASCII with crlf line
endings and no magic:

hexdump -C Workbook1.csv
00000000  44 61 74 65 2c 4e 75 6d  62 65 72 0d 0a 33 2f 36  |Date,Number..3/6|
00000010  2f 30 30 2c 31 38 39 0d  0a 39 2f 35 2f 30 31 2c  |/00,189..9/5/01,|
00000020  31 39 31 0d 0a 31 30 2f  39 2f 30 32 2c 31 39 32  |191..10/9/02,192|
00000030  0d 0a 31 31 2f 37 2f 30  32 2c 31 38 39 0d 0a 32  |..11/7/02,189..2|
00000040  2f 32 30 2f 30 34 2c 31  39 38 0d 0a 33 2f 38 2f  |/20/04,198..3/8/|
00000050  30 34 2c 31 39 38 0d 0a  31 2f 32 37 2f 30 35 2c  |04,198..1/27/05,|
00000060  32 30 34 0d 0a 31 30 2f  31 39 2f 30 35 2c 32 30  |204..10/19/05,20|
00000070  30 0d 0a 34 2f 35 2f 30  36 2c 31 39 39 0d 0a 36  |0..4/5/06,199..6|
00000080  2f 39 2f 30 36 2c 31 39  38 0d 0a 31 2f 37 2f 30  |/9/06,198..1/7/0|
00000090  38 2c 32 30 34 0d 0a 31  2f 39 2f 30 38 2c 32 30  |8,204..1/9/08,20|
000000a0  34 0d 0a 31 2f 32 37 2f  30 39 2c 32 30 30 0d 0a  |4..1/27/09,200..|
000000b0  34 2f 38 2f 30 39 2c 32  30 33 0d 0a 35 2f 34 2f  |4/8/09,203..5/4/|
000000c0  30 39 2c 32 30 31 0d 0a  38 2f 31 32 2f 30 39 2c  |09,201..8/12/09,|
000000d0  32 30 35 0d 0a 31 30 2f  31 35 2f 30 39 2c 32 30  |205..10/15/09,20|
000000e0  32 0d 0a 37 2f 31 39 2f  31 30 2c 31 39 37        |2..7/19/10,197|
000000ee

A naive C program written on MacOS would have problems parsing either one.

[toc] | [prev] | [next] | [standalone]

#99682

Geoff <geoff@invalid.invalid> writes:
> On Thu, 15 Dec 2016 22:58:51 -0800, Keith Thompson <kst-u@mib.org> wrote:
[...]
>>Which means that a CSV file exported in "CSV-UTF-8" format is not a
>>text file.  If you try to open it with a text editor, it will likely
>>appear as one very long line with no line endings at all (unless some
>>MacOS editors still recognize pre-OSx text files).  It means that
>>"wc -l filename.csv" won't tell you how many lines are in the file.
>>
>>That's astonishing.
>
> It has magic at the beginning of the file but that's all:
>
> hexdump -C Workbook1.csv
> 00000000  ef bb bf 44 61 74 65 2c  4e 75 6d 62 65 72 0d 33  |...Date,Number.3|
> 00000010  2f 36 2f 30 30 2c 31 38  39 0d 39 2f 35 2f 30 31  |/6/00,189.9/5/01|
> 00000020  2c 31 39 31 0d 31 30 2f  39 2f 30 32 2c 31 39 32  |,191.10/9/02,192|
> 00000030  0d 31 31 2f 37 2f 30 32  2c 31 38 39 0d 32 2f 32  |.11/7/02,189.2/2|
> 00000040  30 2f 30 34 2c 31 39 38  0d 33 2f 38 2f 30 34 2c  |0/04,198.3/8/04,|
> 00000050  31 39 38 0d 31 2f 32 37  2f 30 35 2c 32 30 34 0d  |198.1/27/05,204.|
> 00000060  31 30 2f 31 39 2f 30 35  2c 32 30 30 0d 34 2f 35  |10/19/05,200.4/5|
> 00000070  2f 30 36 2c 31 39 39 0d  36 2f 39 2f 30 36 2c 31  |/06,199.6/9/06,1|
> 00000080  39 38 0d 31 2f 37 2f 30  38 2c 32 30 34 0d 31 2f  |98.1/7/08,204.1/|
> 00000090  39 2f 30 38 2c 32 30 34  0d 31 2f 32 37 2f 30 39  |9/08,204.1/27/09|
> 000000a0  2c 32 30 30 0d 34 2f 38  2f 30 39 2c 32 30 33 0d  |,200.4/8/09,203.|
> 000000b0  35 2f 34 2f 30 39 2c 32  30 31 0d 38 2f 31 32 2f  |5/4/09,201.8/12/|
> 000000c0  30 39 2c 32 30 35 0d 31  30 2f 31 35 2f 30 39 2c  |09,205.10/15/09,|
> 000000d0  32 30 32 0d 37 2f 31 39  2f 31 30 2c 31 39 37     |202.7/19/10,197|
> 000000df
[...]

The "magic" at the beginning is a UTF-8 BOM (Byte Order Mark).

A BOM are normally necessary only for UTF-16.  It's a U+FEFF character
whose byte order tells you whether you're dealing with big-endian or
little-endian UTF-16.  A BOM in a UTF-8 file tells you nothing about
byte ordering (since the issue doesn't arise for UTF-8).  It can tell
you that the file is (probably) UTF-8, but outside of Microsoft software
UTF-8 BOMs are not necessary and not recommended.

When I unconvert the hex dump, the "file" command (on Ubuntu) says:

    foo.csv: UTF-8 Unicode (with BOM) text, with CR line terminators

Ick.

No doubt Excel itself can read such a file, and LibreOffice Calc
and Gnumeric can open it as well, but other tools (not designed
specifically to deal with spreadsheets or CSV files) are likely to
have problems with it.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#99584

On 15/12/16 23:03, Geoff wrote:
> On Thu, 15 Dec 2016 09:09:19 +0000, Richard Heathfield
> <rjh@cpax.org.uk> wrote:
>
>> Excel is by Microsoft. Microsoft wrote Windows, which uses screwy line
>> endings for text files. You don't think Excel might write a screwy text
>> file even on MacOS, just out of sheer habit?
>
> Excel 2016 for Mac outputs 0D-only end of lines in csv files.

0D? Not 0A? I thought the Mac had grown out of that.

-- 
Richard Heathfield
Email: rjh at cpax dot org dot uk
"Usenet is a strange place" - dmr 29 July 1999
Sig line 4 vacant - apply within

[toc] | [prev] | [next] | [standalone]

#99622

On Friday, December 16, 2016 at 2:03:28 AM UTC+3, Geoff wrote:
> On Thu, 15 Dec 2016 09:09:19 +0000, Richard Heathfield
> <rjh@cpax.org.uk> wrote:
> 
> >Excel is by Microsoft. Microsoft wrote Windows, which uses screwy line 
> >endings for text files. You don't think Excel might write a screwy text 
> >file even on MacOS, just out of sheer habit?
> 
> Excel 2016 for Mac outputs 0D-only end of lines in csv files.

I have 2011 version 

[toc] | [prev] | [next] | [standalone]

#99660

On Fri, 16 Dec 2016 02:37:51 -0800 (PST), Alla _
<modelling.data@gmail.com> wrote:

>On Friday, December 16, 2016 at 2:03:28 AM UTC+3, Geoff wrote:
>> On Thu, 15 Dec 2016 09:09:19 +0000, Richard Heathfield
>> <rjh@cpax.org.uk> wrote:
>> 
>> >Excel is by Microsoft. Microsoft wrote Windows, which uses screwy line 
>> >endings for text files. You don't think Excel might write a screwy text 
>> >file even on MacOS, just out of sheer habit?
>> 
>> Excel 2016 for Mac outputs 0D-only end of lines in csv files.
>
>I have 2011 version 

It should have similar output modes as Excel 2016 so I expect you'll
have a list of (.csv) formats to choose from but even though I am
seeing four different "modes" three of them are essentially the same.

If you are not using Richard's sample csv file then use any one of the
three choices that give you the 0d 0a line endings. You'll have to
modify your code to account for the line endings. Your program will
interpret '\n' to be 0x0a and you'll need to test for and chomp off
the '0x0d' that might precede it if you want to be able to handle both
formats. (0x0d vs. 0x0d, 0x0a)

I say "test for and chomp" because it's also possible to generate a
.csv file that has the 0x0a line endings per current OSX and Linux
standard. This would happen if you write your own program to output
csv using '\n' line endings on your MacBook.

[toc] | [prev] | [next] | [standalone]

#99477

On 12/15/2016 9:58 AM, Alla _ wrote:
> On Thursday, December 15, 2016 at 11:53:06 AM UTC+3, Richard Heathfield wrote:
>> The only thing that occurred to me was this:
>>
>> You *might* get the symptoms you suggest *if* you created the CSV file
>> on Windows (giving you \r\n line endings), and then either tried to open
>> it as a binary stream rather than a text stream, or perhaps FTPd it to a
>> Unix-like system (Linux or MacOS, say) in BINary mode (no line-end
>> translation), and then opened it on that system.
>>
> No, I work on Mac as usual. I think there are some issues with my Excel
> parameteres, or something of this sort because, as we have discussed
> earlier, including in my conversation with Keith, I get strange results
> upon saving xlsx files to csv.

It might still be worth verifying that you opened the file with the "t" 
flag.

[toc] | [prev] | [next] | [standalone]

#99479

On 15/12/16 11:22, Manfred wrote:
> On 12/15/2016 9:58 AM, Alla _ wrote:
>> On Thursday, December 15, 2016 at 11:53:06 AM UTC+3, Richard
>> Heathfield wrote:
>>> The only thing that occurred to me was this:
>>>
>>> You *might* get the symptoms you suggest *if* you created the CSV file
>>> on Windows (giving you \r\n line endings), and then either tried to open
>>> it as a binary stream rather than a text stream, or perhaps FTPd it to a
>>> Unix-like system (Linux or MacOS, say) in BINary mode (no line-end
>>> translation), and then opened it on that system.
>>>
>> No, I work on Mac as usual. I think there are some issues with my Excel
>> parameteres, or something of this sort because, as we have discussed
>> earlier, including in my conversation with Keith, I get strange results
>> upon saving xlsx files to csv.
>
> It might still be worth verifying that you opened the file with the "t"
> flag.

No, it's worth verifying that you didn't. The fopen function is required 
to recognise the following modes:

r
w
wx (new in C11)
a
rb
wb
wbx (new in C11)
ab
r+
w+
w+x (new in C11)
a+
r+b
rb+
w+b
wb+
w+bx (new in C11)
wb+x (new in C11)
a+b
ab+

If any other mode is used, the behaviour is undefined.

-- 
Richard Heathfield
Email: rjh at cpax dot org dot uk
"Usenet is a strange place" - dmr 29 July 1999
Sig line 4 vacant - apply within

[toc] | [prev] | [next] | [standalone]

#99481

On 15/12/16 12:29 PM, Richard Heathfield wrote:
> On 15/12/16 11:22, Manfred wrote:
>> On 12/15/2016 9:58 AM, Alla _ wrote:
>>> On Thursday, December 15, 2016 at 11:53:06 AM UTC+3, Richard
>>> Heathfield wrote:
>>>> The only thing that occurred to me was this:
>>>>
>>>> You *might* get the symptoms you suggest *if* you created the CSV file
>>>> on Windows (giving you \r\n line endings), and then either tried to
>>>> open
>>>> it as a binary stream rather than a text stream, or perhaps FTPd it
>>>> to a
>>>> Unix-like system (Linux or MacOS, say) in BINary mode (no line-end
>>>> translation), and then opened it on that system.
>>>>
>>> No, I work on Mac as usual. I think there are some issues with my Excel
>>> parameteres, or something of this sort because, as we have discussed
>>> earlier, including in my conversation with Keith, I get strange results
>>> upon saving xlsx files to csv.
>>
>> It might still be worth verifying that you opened the file with the "t"
>> flag.
>
> No, it's worth verifying that you didn't. The fopen function is required
> to recognise the following modes:
>
> r
> w
> wx (new in C11)
> a
> rb
> wb
> wbx (new in C11)
> ab
> r+
> w+
> w+x (new in C11)
> a+
> r+b
> rb+
> w+b
> wb+
> w+bx (new in C11)
> wb+x (new in C11)
> a+b
> ab+
>
> If any other mode is used, the behaviour is undefined.
>
Hm, why undefined? It seems to me that it is completely unnecessary in 
this case...


-- 
press any key to continue or any other to quit...

[toc] | [prev] | [next] | [standalone]

#99483

On 15/12/16 11:40, Melzzzzz wrote:
> On 15/12/16 12:29 PM, Richard Heathfield wrote:
>> On 15/12/16 11:22, Manfred wrote:
<snip>
>>>
>>> It might still be worth verifying that you opened the file with the "t"
>>> flag.
>>
>> No, it's worth verifying that you didn't. The fopen function is required
>> to recognise the following modes:

<snip>

>>
>> If any other mode is used, the behaviour is undefined.
>>
> Hm, why undefined? It seems to me that it is completely unnecessary in
> this case...

Implementations are free to support other modes if they choose, and some 
do. Microsoft, for example, supports a "t" suffix on the mode string. 
IBM mainframe compilers support all kinds of entries in the mode string, 
such as "lrecl=xx", "recfm=X", "noseek", "blksize=xx", even 
"password=xxxxxxx".

What should the Standard say about these?

Should it say that they are implementation-defined? If so, every 
implementation would have to define what it does in all these cases and 
all kinds of others, which would constitute an unreasonable burden on 
implementations.

Unspecified? Every implementation would have to make some choice or 
other, but wouldn't have to document it, so you'd be no better off.

It therefore makes perfect sense to have the behaviour undefined. That 
way, if you want to take advantage of a particular implementation's 
support for a particular mode, you can, but at the cost of recognising 
that your code isn't going to work as you might wish on implementations 
that don't support that mode.

-- 
Richard Heathfield
Email: rjh at cpax dot org dot uk
"Usenet is a strange place" - dmr 29 July 1999
Sig line 4 vacant - apply within

[toc] | [prev] | [next] | [standalone]

#99486

On Thu, 15 Dec 2016 12:00:35 +0000
Richard Heathfield <rjh@cpax.org.uk> wrote:

> On 15/12/16 11:40, Melzzzzz wrote:
> > On 15/12/16 12:29 PM, Richard Heathfield wrote:  
> >> On 15/12/16 11:22, Manfred wrote:  
> <snip>
> >>>
> >>> It might still be worth verifying that you opened the file with
> >>> the "t" flag.  
> >>
> >> No, it's worth verifying that you didn't. The fopen function is
> >> required to recognise the following modes:  
> 
> <snip>
> 
> >>
> >> If any other mode is used, the behaviour is undefined.
> >>  
> > Hm, why undefined? It seems to me that it is completely unnecessary
> > in this case...  
> 
> Implementations are free to support other modes if they choose, and
> some do. Microsoft, for example, supports a "t" suffix on the mode
> string. IBM mainframe compilers support all kinds of entries in the
> mode string, such as "lrecl=xx", "recfm=X", "noseek", "blksize=xx",
> even "password=xxxxxxx".
> 
> What should the Standard say about these?
> 
> Should it say that they are implementation-defined? If so, every 
> implementation would have to define what it does in all these cases
> and all kinds of others, which would constitute an unreasonable
> burden on implementations.
> 
> Unspecified? Every implementation would have to make some choice or 
> other, but wouldn't have to document it, so you'd be no better off.
> 
> It therefore makes perfect sense to have the behaviour undefined.
> That way, if you want to take advantage of a particular
> implementation's support for a particular mode, you can, but at the
> cost of recognising that your code isn't going to work as you might
> wish on implementations that don't support that mode.
> 

Well, I have learned that undefined behavior means bad thing ;) If I
use fopen in particular mode and it is undefined, in this case it
actually does not means undefined as bad things can happen?
Or we can take advantage of undefined behavior elsewhere ;)?

-- 
press any key to continue or any other to quit...

[toc] | [prev] | [next] | [standalone]

#99489

On 15/12/16 12:09, Melzzzzz wrote:
> On Thu, 15 Dec 2016 12:00:35 +0000
> Richard Heathfield <rjh@cpax.org.uk> wrote:
>

[non-standard fopen modes]

<snip>

>> It therefore makes perfect sense to have the behaviour undefined.
>> That way, if you want to take advantage of a particular
>> implementation's support for a particular mode, you can, but at the
>> cost of recognising that your code isn't going to work as you might
>> wish on implementations that don't support that mode.
>>
>
> Well, I have learned that undefined behavior means bad thing ;)  If I
> use fopen in particular mode and it is undefined, in this case it
> actually does not means undefined as bad things can happen?

It means that the behaviour is not defined by the C Standard, and that's 
all it means. So, if you want your program to work on many different 
implementations, you should stick to the standard modes. But if you are 
writing for a particular implementation, you are free to take up the 
guarantees that /it/ provides, to the extent that you are prepared to 
trust (or test) those guarantees.

Sometimes you have little choice, as with IBM's mainframe modes for 
VSAMs and so on. But when you go down that road, you accept that you are 
limiting the portability of your program.

> Or we can take advantage of undefined behavior elsewhere ;)?

Of course you can, /if/ you know the consequences and /if/ you are 
prepared to accept those consequences in return for a benefit that 
outweighs the cost.

Since the C Standard doesn't say what will happen in the case of 
undefined behaviour (in fact, that's what "undefined behaviour" actually 
means!), you had better have some other way of knowing what will happen, 
and you had better be very, very sure, and there had better be a pay-off 
that outweighs the fact that you are breaking the rules and thus 
sacrificing portability.

Let me give you two examples, one good and one bad. The bad one first.

i = i++; /* BAD! */

The C Standard doesn't say what will happen here. Now, you might do some 
experiments and learn that your implementation happens to interpret the 
above in exactly the way you want --- but there's still no point in 
writing the code that way. You know what you want to happen to i, and 
it's easy to express in conforming code, so why not express it in 
conforming code? That way, you know for sure that it'll work on /any/ 
implementation, not just the one you happen to be using today.

Now for the good example (well, not that good, since it doesn't do much 
checking, but good enough for our purposes):

unsigned char *scrbase = (unsigned char *)0xb0000000L;
if(_getgraphmode() == _GMODE_COLOUR)
{
   scrbase += 0x08000000L;
}
scrbase += row * 160;
scrbase += col * 2;
while(*s)
{
   *scrbase++ = (unsigned char)*s++;
   *scrbase++ = attr;
}

The C Standard doesn't say what will happen here, either. But if you 
want to write a string to a particular place on the screen and you 
happen to be using MS-DOS, this is a /really/ fast way to do it. You 
give up portability, but you gain performance.

So, as with so many things in life, it's really important to know 
exactly what the rules are, so that you know exactly what you're letting 
yourself in for when you break them.

The i = i++ code breaks the rules for *no good reason*. There is a cost 
(portability) without a commensurate gain (such as performance, for 
example). The graphics code breaks the rules too, but it's for an 
excellent reason, and we accept the cost in order to gain the benefit.

-- 
Richard Heathfield
Email: rjh at cpax dot org dot uk
"Usenet is a strange place" - dmr 29 July 1999
Sig line 4 vacant - apply within

[toc] | [prev] | [next] | [standalone]

#99498

On 12/15/2016 07:00 AM, Richard Heathfield wrote:
> On 15/12/16 11:40, Melzzzzz wrote:
>> On 15/12/16 12:29 PM, Richard Heathfield wrote:
>>> On 15/12/16 11:22, Manfred wrote:
> <snip>
>>>>
>>>> It might still be worth verifying that you opened the file with the "t"
>>>> flag.
>>>
>>> No, it's worth verifying that you didn't. The fopen function is required
>>> to recognise the following modes:
>
> <snip>
>
>>>
>>> If any other mode is used, the behaviour is undefined.
>>>
>> Hm, why undefined? It seems to me that it is completely unnecessary in
>> this case...
>
> Implementations are free to support other modes if they choose, and some
> do. Microsoft, for example, supports a "t" suffix on the mode string.
> IBM mainframe compilers support all kinds of entries in the mode string,
> such as "lrecl=xx", "recfm=X", "noseek", "blksize=xx", even
> "password=xxxxxxx".
>
> What should the Standard say about these?
>
> Should it say that they are implementation-defined? If so, every
> implementation would have to define what it does in all these cases and
> all kinds of others, which would constitute an unreasonable burden on
> implementations.

I don't see how it would constitute an unreasonable burden - if you've 
gone to the trouble of implementing a feature, and you have your own 
internal documentation of that feature so your testers know what it's 
supposed to do, it shouldn't a significant amount of extra work to 
create corresponding user documentation.

[toc] | [prev] | [next] | [standalone]

#99501

On 15/12/16 14:18, James R. Kuyper wrote:
> On 12/15/2016 07:00 AM, Richard Heathfield wrote:
>> On 15/12/16 11:40, Melzzzzz wrote:
>>> On 15/12/16 12:29 PM, Richard Heathfield wrote:
>>>> On 15/12/16 11:22, Manfred wrote:
>> <snip>
>>>>>
>>>>> It might still be worth verifying that you opened the file with the
>>>>> "t"
>>>>> flag.
>>>>
>>>> No, it's worth verifying that you didn't. The fopen function is
>>>> required
>>>> to recognise the following modes:
>>
>> <snip>
>>
>>>>
>>>> If any other mode is used, the behaviour is undefined.
>>>>
>>> Hm, why undefined? It seems to me that it is completely unnecessary in
>>> this case...
>>
>> Implementations are free to support other modes if they choose, and some
>> do. Microsoft, for example, supports a "t" suffix on the mode string.
>> IBM mainframe compilers support all kinds of entries in the mode string,
>> such as "lrecl=xx", "recfm=X", "noseek", "blksize=xx", even
>> "password=xxxxxxx".
>>
>> What should the Standard say about these?
>>
>> Should it say that they are implementation-defined? If so, every
>> implementation would have to define what it does in all these cases and
>> all kinds of others, which would constitute an unreasonable burden on
>> implementations.
>
> I don't see how it would constitute an unreasonable burden - if you've
> gone to the trouble of implementing a feature, and you have your own
> internal documentation of that feature so your testers know what it's
> supposed to do, it shouldn't a significant amount of extra work to
> create corresponding user documentation.

I'm not talking about the implementation that supports it. I'm talking 
about all the implementations that /don't/.

This is what the Standard has to say about implementation-defined behaviour:

"unspecified behavior where each implementation documents how the choice 
is made"

So either every implementation has to document what happens when a mode 
string of, say, "lrecl=xx" is supplied, or they don't.

If they do, there's the unreasonable burden I was talking about. (It's 
reasonable for implementations that support it to say so, but it's 
unreasonable to expect every implementation that does /not/ support it 
to say "the following modes are unsupported".

And if they don't, then effectively you have undefined behaviour, so you 
might as well call it that.

-- 
Richard Heathfield
Email: rjh at cpax dot org dot uk
"Usenet is a strange place" - dmr 29 July 1999
Sig line 4 vacant - apply within

[toc] | [prev] | [next] | [standalone]

#99506

On 12/15/2016 09:49 AM, Richard Heathfield wrote:
> On 15/12/16 14:18, James R. Kuyper wrote:
>> On 12/15/2016 07:00 AM, Richard Heathfield wrote:
...
>>> Implementations are free to support other modes if they choose, and some
>>> do. Microsoft, for example, supports a "t" suffix on the mode string.
>>> IBM mainframe compilers support all kinds of entries in the mode string,
>>> such as "lrecl=xx", "recfm=X", "noseek", "blksize=xx", even
>>> "password=xxxxxxx".
>>>
>>> What should the Standard say about these?
>>>
>>> Should it say that they are implementation-defined? If so, every
>>> implementation would have to define what it does in all these cases and
>>> all kinds of others, which would constitute an unreasonable burden on
>>> implementations.
>>
>> I don't see how it would constitute an unreasonable burden - if you've
>> gone to the trouble of implementing a feature, and you have your own
>> internal documentation of that feature so your testers know what it's
>> supposed to do, it shouldn't a significant amount of extra work to
>> create corresponding user documentation.
>
> I'm not talking about the implementation that supports it. I'm talking
> about all the implementations that /don't/.
>
> This is what the Standard has to say about implementation-defined behaviour:
>
> "unspecified behavior where each implementation documents how the choice
> is made"
>
> So either every implementation has to document what happens when a mode
> string of, say, "lrecl=xx" is supplied, or they don't.

The documentation can explain the syntax and semantics the supported 
modes, and document the behavior when someone passes a mode not on the 
list. Assuming that the handling of unsupported modes is the same for 
all such modes (which seems both reasonable and not difficult to 
implement), then that documentation would cover ALL of the unsupported 
modes, without having to name any of them. I don't see it as a 
significant burden.

> If they do, there's the unreasonable burden I was talking about. (It's
> reasonable for implementations that support it to say so, but it's
> unreasonable to expect every implementation that does /not/ support it
> to say "the following modes are unsupported".

Right, and that's not what I'd want. They should say something like "any 
modes not described above [are ignored | cause fopen() to fail]", 
whichever fits the way fopen() actually handles them. Listing all of the 
modes not supported is (thankfully!) not necessary.

> And if they don't, then effectively you have undefined behaviour, so you
> might as well call it that.

"ignores unsupported modes" or "fails if any unsupported modes are used" 
are both much more useful things to know than "undefined behavior", and 
I see no reason why it would be a burden to require that it be 
documented. I don't see any reason why an implementation might choose to 
handle unsupported modes in a more complicated fashion than one of those 
two, so I can't imagine how much more of a burden documenting such 
behavior would be.

[toc] | [prev] | [next] | [standalone]

#99507

On 12/15/2016 12:29 PM, Richard Heathfield wrote:
> On 15/12/16 11:22, Manfred wrote:
>> On 12/15/2016 9:58 AM, Alla _ wrote:
>>> On Thursday, December 15, 2016 at 11:53:06 AM UTC+3, Richard
>>> Heathfield wrote:
>>>> The only thing that occurred to me was this:
>>>>
>>>> You *might* get the symptoms you suggest *if* you created the CSV file
>>>> on Windows (giving you \r\n line endings), and then either tried to
>>>> open
>>>> it as a binary stream rather than a text stream, or perhaps FTPd it
>>>> to a
>>>> Unix-like system (Linux or MacOS, say) in BINary mode (no line-end
>>>> translation), and then opened it on that system.
>>>>
>>> No, I work on Mac as usual. I think there are some issues with my Excel
>>> parameteres, or something of this sort because, as we have discussed
>>> earlier, including in my conversation with Keith, I get strange results
>>> upon saving xlsx files to csv.
>>
>> It might still be worth verifying that you opened the file with the "t"
>> flag.
>
> No, it's worth verifying that you didn't. The fopen function is required
> to recognise the following modes:
>
> r
> w
> wx (new in C11)
> a
> rb
> wb
> wbx (new in C11)
> ab
> r+
> w+
> w+x (new in C11)
> a+
> r+b
> rb+
> w+b
> wb+
> w+bx (new in C11)
> wb+x (new in C11)
> a+b
> ab+
>
> If any other mode is used, the behaviour is undefined.
>

You are right, my fault.
I should have specified that this is only true for a Microsoft 
implementation.
Still the suspect stands for an unexpected '\r' in the line (even more 
suspect if you don't use a Microsoft implementation with a 
Microsoft-generated text file).
It really looks like the dilemma cannot be solved without more input, as 
of now.

[toc] | [prev] | [next] | [standalone]

#99455

On Wednesday, December 14, 2016 at 2:10:52 PM UTC+3, Richard Heathfield wrote:
> On 14/12/16 08:04, Alla _ wrote:
> > Hello!
> >
> > This is not truly a C question, but I am working on a C program, and I will be
> > grateful for your help on this matter.
> > Please, do me a favor and read my question and explanation to the end, before
> > answering.
> >
> > When reading a line from a csv file with fgets function, and then computing
> > the length of that line, I get two unknown additional characters at the end of
> > the line, which are not represented by "vivid" values, and are located right
> > before the terminating '\0'.
> >
> > Here is the line from csv file:
> > Interest Expense,4,3,3,1,4,
> 
> If this is the CSV file I sent you, here is a hex dump of that part of 
> the file (offsets removed because of line length considerations):
> 
> 70 65 72 61 74 69 6e 67  20 50 72 6f 66 69 74 2c  |perating Profit,|
> 31 32 39 2c 31 33 36 2c  31 37 36 2c 32 33 38 2c  |129,136,176,238,|
> 33 37 34 0a 49 6e 74 65  72 65 73 74 20 45 78 70  |374.Interest Exp|
> 65 6e 73 65 2c 34 2c 33  2c 33 2c 31 2c 34 0a 4f  |ense,4,3,3,1,4.O|
> 74 68 65 72 20 47 61 69  6e 73 20 61 6e 64 20 4c  |ther Gains and L|
> 
> As you can see, there's nothing weird going on in the file. So there 
> must be something weird going on in the code.
> 
> > And the length strlen function has returned is 29, while the line has only 28
> > characters, including '\n', which, as I thought, would be at index 27 (= 28th
> > position right after the last comma). And then '\0', which is not computed by strlen.
> > Therefore I expected the length = 28. My assumptions seem to be
> > incorrect.
> 
> Let's see the code.
> 
> > Same is for each line from my file.
> > When I have converted all required characters into integers, I got two additional
> > zeros at the end of each line.
> 
> Let's see the code.
> 
> Please make sure that the code you post is the code of the program that 
> is exhibiting the problem. Please make sure you don't ask any other 
> questions about that code. Just post the exact code that is showing this 
> exact problem. Only.
> 
It seems it's not the code but the problem in the file. I will be back on this. 

[toc] | [prev] | [next] | [standalone]

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

Back to top | Article view | comp.lang.c

csiph-web