Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #15272 > unrolled thread

strtok

Back to article view | Back to comp.lang.c

Contents

  strtok "Bill Cunningham" <nospam@nspam.invalid> - 2012-01-01 15:30 -0500
    Re: strtok Nick Keighley <nick_keighley_nospam@hotmail.com> - 2012-01-01 12:41 -0800
      Re: strtok "Bill Cunningham" <nospam@nspam.invalid> - 2012-01-01 16:26 -0500
        Re: strtok John Tsiombikas <nuclear@member.fsf.org> - 2012-01-02 12:50 +0000
    Re: strtok Johann Klammer <klammerr@NOSPAM.aon.at> - 2012-01-01 21:53 +0100
    Re: strtok gordonb.o4t45@burditt.org (Gordon Burditt) - 2012-01-01 15:04 -0600
    Re: strtok "osmium" <r124c4u102@comcast.net> - 2012-01-01 15:10 -0600
      Re: strtok jacob navia <jacob@spamsink.net> - 2012-01-01 23:25 +0100
    Re: strtok Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-01-01 21:37 +0000
      Re: strtok "Bill Cunningham" <nospam@nspam.invalid> - 2012-01-01 16:58 -0500
    Re: strtok blp@cs.stanford.edu (Ben Pfaff) - 2012-01-02 10:04 -0800
      Re: strtok David Thompson <dave.thompson2@verizon.net> - 2012-01-19 02:48 -0500

#15272 — strtok

    Why does my man page for strtok and an extension for a thread safe 
version of this function say not to use these functions? Are they buggy or 
deprecated? What should one use instead?

Bill

[toc] | [next] | [standalone]

#15273

On Jan 1, 8:30 pm, "Bill Cunningham" <nos...@nspam.invalid> wrote:
>     Why does my man page for strtok and an extension for a thread safe
> version of this function say not to use these functions? Are they buggy or
> deprecated?

neither,what does the man page say? Some people don't like what
strtok() does. It modifies the string you pass to it and doesn't
handle empty fields in the way you might like.

> What should one use instead?

good question. There's no portable answer. So write your own I
suppose. I've used strtok() its ok it does what it says on the can

[toc] | [prev] | [next] | [standalone]

#15282

Nick Keighley wrote:

> neither,what does the man page say? Some people don't like what
> strtok() does. It modifies the string you pass to it and doesn't
> handle empty fields in the way you might like.
>
>> What should one use instead?
>
> good question. There's no portable answer. So write your own I
> suppose. I've used strtok() its ok it does what it says on the can

    Under bugs I have :
"Never use these functions. If you do note
These functions modify first arg
These functions can't be used with const strings.
The identity of the delimiting character is lost.
strtok uses a static buffer while parsing so is not thread safe. If using 
threads user strtok_r."

The man pages online don't say this.

Bill

[toc] | [prev] | [next] | [standalone]

#15319

On 2012-01-01, Bill Cunningham <nospam@nspam.invalid> wrote:
> Nick Keighley wrote:
>
>> neither,what does the man page say? Some people don't like what
>> strtok() does. It modifies the string you pass to it and doesn't
>> handle empty fields in the way you might like.
>>
>>> What should one use instead?
>>
>> good question. There's no portable answer. So write your own I
>> suppose. I've used strtok() its ok it does what it says on the can
>
>     Under bugs I have :
> "Never use these functions. If you do note

Retard manpage author. Nevermind that, use it if it does what you need,
just be aware of its limitations as others pointed out.

-- 
John Tsiombikas
http://nuclear.mutantstargoat.com/

[toc] | [prev] | [next] | [standalone]

#15277

Bill Cunningham wrote:
>      Why does my man page for strtok and an extension for a thread safe
> version of this function say not to use these functions? Are they buggy or
> deprecated? What should one use instead?
>
> Bill
>
>

To my knowledge, it uses internal state to maintain the current position
which can generate race conditions if used from multiple threads 
concurrently.

eglibc has strtok_r() which is a reentrant function.
It allows the user to specify the address of a variable to use for 
storing position.

[OT]
This is similar to the errno races observed with threaded code.
Some c libraries thus #define errno to be a function call returning a 
pointer, which then is dereferenced to effectively create a thread-local 
errno.

[toc] | [prev] | [next] | [standalone]

#15279

>    Why does my man page for strtok and an extension for a thread safe 
> version of this function say not to use these functions? 

What function(s) does it say not to use?  The not-thread-safe complaint
does not apply to the thread-safe variant strtok_r().

> Are they buggy or 
> deprecated? What should one use instead?

strtok() saves state internally.  When you call strtok(NULL,
delimiter_set), it has to have saved where it left off in a previous
call.

strtok() also alters the input string, which may or may not be a problem,
depending on how you are using it.

This saved state can be messed up if multiple threads call strtok(),
or even if in a single thread a function using strtok() calls another
function using strtok() on an independent string.

strsep() has some advantages, but it does not parse exactly like
strtok().

[toc] | [prev] | [next] | [standalone]

#15280

"Bill Cunningham" wrote:

>    Why does my man page for strtok and an extension for a thread safe 
> version of this function say not to use these functions? Are they buggy or 
> deprecated? What should one use instead?

Bill, I suggest you put any concerns about thread safe operations on the 
back burner, next to "How will my program accommodate  quantum computers?"

If you would quit trying to learn everything you might possibly learn 
*something*. 

[toc] | [prev] | [next] | [standalone]

#15288

Le 01/01/12 22:10, osmium a écrit :
> "Bill Cunningham" wrote:
>
>>     Why does my man page for strtok and an extension for a thread safe
>> version of this function say not to use these functions? Are they buggy or
>> deprecated? What should one use instead?
>
> Bill, I suggest you put any concerns about thread safe operations on the
> back burner, next to "How will my program accommodate  quantum computers?"
>
> If you would quit trying to learn everything you might possibly learn
> *something*.
>
>
Good words!

[toc] | [prev] | [next] | [standalone]

#15283

"Bill Cunningham" <nospam@nspam.invalid> writes:

>     Why does my man page for strtok and an extension for a thread safe 
> version of this function say not to use these functions? Are they buggy or 
> deprecated? What should one use instead?

If your question is still about reading your market data (as you posted
on comp.programming) you don't need strtok.

If you are sure that the data will be in the format you previously
described, fscanf can do the job perfectly well.  If you want a little
more control, read each line using fgets and use sscanf to "parse" the
data you want.

Example:

  #include <stdio.h>
   
  int main(void)
  {
       double price;
       int line_no, day, month, year;
       char line[100];
       while (fgets(line, sizeof line, stdin) &&
              sscanf(line, "%d %lf %2d%2d%2d",
                     &line_no, &price, &day, &month, &year) == 5)
            printf("line number: %d, price=%f on %d/%02d/%02d\n",
                   line_no, price, day, month, year);
       return 0;
  }

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#15287

Ben Bacarisse wrote:
> "Bill Cunningham" <nospam@nspam.invalid> writes:
>
>>     Why does my man page for strtok and an extension for a thread
>> safe version of this function say not to use these functions? Are
>> they buggy or deprecated? What should one use instead?
>
> If your question is still about reading your market data (as you
> posted on comp.programming) you don't need strtok.

    Ok

> If you are sure that the data will be in the format you previously
> described, fscanf can do the job perfectly well.

Ok

  If you want a little
> more control, read each line using fgets and use sscanf to "parse" the
> data you want.
>
> Example:
>
>  #include <stdio.h>
>
>  int main(void)
>  {
>       double price;
>       int line_no, day, month, year;
>       char line[100];
>       while (fgets(line, sizeof line, stdin) &&
>              sscanf(line, "%d %lf %2d%2d%2d",
>                     &line_no, &price, &day, &month, &year) == 5)
>            printf("line number: %d, price=%f on %d/%02d/%02d\n",
>                   line_no, price, day, month, year);
>       return 0;
>  }

    Thanks. That sscanf is alittle hard to read but then again I'm not 
familiar with that function.

Bill

[toc] | [prev] | [next] | [standalone]

#15329

"Bill Cunningham" <nospam@nspam.invalid> writes:

>     Why does my man page for strtok and an extension for a thread safe 
> version of this function say not to use these functions? Are they buggy or 
> deprecated? What should one use instead?

strtok() has at least these problems:

	* It merges adjacent delimiters.  If you use a comma as your
          delimiter, then "a,,b,c" will be divided into three tokens,
          not four.  This is often the wrong thing to do.  In fact, it
          is only the right thing to do, in my experience, when the
          delimiter set contains white space (for dividing a string
          into "words") or it is known in advance that there will be
          no adjacent delimiters.

	* The identity of the delimiter is lost, because it is
          changed to a null terminator.

	* It modifies the string that it tokenizes.  This is bad
          because it forces you to make a copy of the string if
          you want to use it later.  It also means that you can't
          tokenize a string literal with it; this is not
          necessarily something you'd want to do all the time but
          it is surprising.

	* It can only be used once at a time.  If a sequence of
          strtok() calls is ongoing and another one is started,
          the state of the first one is lost.  This isn't a
          problem for small programs but it is easy to lose track
          of such things in hierarchies of nested functions in
          large programs.  In other words, strtok() breaks
          encapsulation.

-- 
char a[]="\n .CJacehknorstu";int putchar(int);int main(void){unsigned long b[]
={0x67dffdff,0x9aa9aa6a,0xa77ffda9,0x7da6aa6a,0xa67f6aaa,0xaa9aa9f6,0x11f6},*p
=b,i=24;for(;p+=!*p;*p/=4)switch(0[p]&3)case 0:{return 0;for(p--;i--;i--)case+
2:{i++;if(i)break;else default:continue;if(0)case 1:putchar(a[i&15]);break;}}}

[toc] | [prev] | [next] | [standalone]

#15943

On Mon, 02 Jan 2012 10:04:24 -0800, blp@cs.stanford.edu (Ben Pfaff)
wrote:

> "Bill Cunningham" <nospam@nspam.invalid> writes:
> 
> >     Why does my man page for strtok and an extension for a thread safe 
> > version of this function say not to use these functions? Are they buggy or 
> > deprecated? What should one use instead?
> 
> strtok() has at least these problems:
> 
> 	* It merges adjacent delimiters.  If you use a comma as your
>           delimiter, then "a,,b,c" will be divided into three tokens,
>           not four.  This is often the wrong thing to do.  In fact, it
>           is only the right thing to do, in my experience, when the
>           delimiter set contains white space (for dividing a string
>           into "words") or it is known in advance that there will be
>           no adjacent delimiters.
> 
Yes.

> 	* The identity of the delimiter is lost, because it is
>           changed to a null terminator.
> 
Yes but. IME at least half the time the delimiter is unique (so
identity doesn't matter). There are a few cases where even nonunique
delimiters don't matter; one I personally like -- for vanishingly
small values of like -- is the punctuation in US telephone numbers.

> 	* It modifies the string that it tokenizes.  This is bad
>           because it forces you to make a copy of the string if
>           you want to use it later.  It also means that you can't
>           tokenize a string literal with it; this is not
>           necessarily something you'd want to do all the time but
>           it is surprising.
> 
Yes but. If you want to parse nondestructively and use the tokens as C
strings, you have to copy anyway -- and often allocate each token
separately which is almost certainly costlier. Although that second
condition isn't a given: I worked on one largish project that
religiously used {ptr,len} into existing/shared buffers. It required a
rigid policy on lifetime of the shared buffers (which for this app was
fairly easy) and a substantial custom library (basically duplicating
string.h plus more) but after that was paid once it worked nicely.

And that literals are not (safely) modifiable should be surprising at
most once, and needs to be learned anyway even without strtok();
consider mktemp and mkstemp, strupr and strlwr, or doing  parity or
rot13 or other simple ciphering in place.

> 	* It can only be used once at a time.  If a sequence of
>           strtok() calls is ongoing and another one is started,
>           the state of the first one is lost.  This isn't a
>           problem for small programs but it is easy to lose track
>           of such things in hierarchies of nested functions in
>           large programs.  In other words, strtok() breaks
>           encapsulation.

Yes. Although it's often a good idea (sometimes even a requirement) to
separate the parsing from the 'real' processing, and if the parsing by
itself is so complicated you can't grok all the code in one fwoop,
your input syntax is in danger of being unusable anyway.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.c

csiph-web