Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c++ > #10023

Re: Best way to tokenize in String

Show all headers | View raw

Paul wrote:
> On Sep 10, 4:14 am, Ian Collins <ian-n...@hotmail.com> wrote:
> 
>>On 09/10/11 02:25 PM, sravanreddy001 wrote:
>>
>>
>>>Hi,
>>>what is the efficient way to tokenize the string and splitting into
>>>words based on the delimiter?
>>
>>The simplest way is to create an istringstream from the string and
>>stream out the words.
>>
> 
> But is that the most efficient way?
> 
> However simplicty is sometime preferred over efficiency and the stream
> method is quite handy, it is described here, bottom of page:
> http://www.oopweb.com/CPP/Documents/CPPHOWTO/Volume/C++Programming-HOWTO-7.html
> 
> Is there a way to use this stream method when the delimiter is not
> whitespace?
> 
> 
Yes, there's a technique called the "whitespace redefinition approach" 
which will let you do that:

https://groups.google.com/group/comp.lang.c++/msg/4de0be2e4eb8e0ba?output=gplain&hl=de&pli=1


Personally I like the ability to use 'cin >> foo >> bar' style to read 
delimited data from a stream, although most C++ programmers I know don't 
really know/care much about locales, ctypes and facets, and 
unfortunately I would expect they'd probably consider it to be "a bit 
too weird" to use in their code.




Another one which I like, using TR1/Boost RegEx:

#include <string>
#include <regex>
#include <iostream>

int main()
{
     std::string str = "the\t    quick  brown\n-\n- fox"
                       " jumped..over,the,lazy,.dog";
     std::tr1::regex re("[\\s-,.]+");

     std::tr1::sregex_token_iterator
         iter(str.begin(), str.end(), re, -1),
         end;

     while(iter != end)
     {
         std::cout << *iter++ << std::endl;
     }
}

Back to comp.lang.c++ | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Best way to tokenize in String sravanreddy001 <sravanreddy001@gmail.com> - 2011-09-09 19:25 -0700
  Re: Best way to tokenize in String Ian Collins <ian-news@hotmail.com> - 2011-09-10 15:14 +1200
    Re: Best way to tokenize in String Paul <pchristor@yahoo.co.uk> - 2011-09-09 22:30 -0700
      Re: Best way to tokenize in String red floyd <no.spam.here@its.invalid> - 2011-09-09 23:11 -0700
        Re: Best way to tokenize in String "Paul" <pchristor@yahoo.co.uk> - 2011-09-10 11:32 +0100
      Re: Best way to tokenize in String Ben Cottrell <email@address.invalid> - 2011-09-10 13:05 +0100
        Re: Best way to tokenize in String Paul <pchristor@yahoo.co.uk> - 2011-09-10 08:25 -0700
  Re: Best way to tokenize in String Leigh Johnston <leigh@i42.co.uk> - 2011-09-10 13:15 +0100
  Re: Best way to tokenize in String Asger-P <junk@asger-p.dk> - 2011-09-10 14:54 +0200
  Re: Best way to tokenize in String Moshbear dot Net <moshbear.net@gmail.com> - 2011-09-10 22:46 -0700

csiph-web