Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #73127

Re: Working with files on different operating systems

Show all headers | View raw

Morten W. Petersen wrote:
> On 16.10.2015 03:09, Malcolm McLean wrote:
>> On Friday, October 9, 2015 at 5:50:47 PM UTC+1, Morten W. Petersen wrote:
>
>>> However, that the program can read from any stream and has as little
>>> platform-specific code, are two significant good things..  I think
>>> it is better to go back later and add optimizations.
>>>
>> Generally it's more important to to portable and flexible than to
>> be as fast as possible. Sadly, C's FILE *s can't be used in all
>> situations. Really streams are one of those things which need
>> object-based design, a stream that is attached to an unreliable
>> connection needs to be able to query the user for timeout,
>> for example, sometimes the user will quit, other times he'll
>> tell it to keep trying and go for a cup of coffee. An XML parser
>> shouldn't be concerned with that.
>
> Well, the parser can't do any parsing if it runs out of characters
> to work on.  I think threading here can be one of the things that
> makes working with streams more practical.

Not really, you can't finish parsing the document until all of the 
document has been received.  Assuming you can process the file faster 
than you can read it, any threads would be idle.

> As for design without threads, maybe it is a good idea to allocate
> some xml_item structs, compress data and so on while idling for input.

Where threads can be of use is for data processing unrelated to the 
parsing activity.  One example is an application I had a while back 
where fields in incoming SOAP messages triggered database queries.

>>> Same goes for the storage type; incremental reads will make the program
>>> use less memory at peaks which counts towards stability and the ability
>>> to deal with large files.
>>>
>>> The "window" will most of the time not be that large, but that doesn't
>>> help if there are huge windows in a given XML stream.  And that's
>>> something that has to be taken into account.
>>>
>> You can't design for everything.
>> Is the goal to have really good performance on normal XML, even at
>> the cost of poor performance or even failure on very abnormal
>> documents? Or do you want to be able to handle absolutely anything
>> that is within the capacity of the machine, reasonably quickly,
>> even if that means the parser is quite a bit slower than competitors
>> on normal XML?
>
> Take for example an XML element attribute name.  It will probably not
> be that long in most cases.  But the attribute value could be long,
> very long or extremely long.

You can't prejudge anything when it comes to XML document structure 
unless you are targeting a specific document type.  For example an Open 
Document XML with usually have attribute names way longer then the 
value!  Attributes such as 'text:display-outline-level="0"' abound.

I work with may different types of XML and I haven't found any 
micro-optimisations worth adding to my parser.  Relative to providing a 
comprehensive DOM model, parsing is a doddle in both code complexity and 
size!
-- 
Ian Collins

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Working with files on different operating systems "Morten W. Petersen" <morphex@gmail.com> - 2015-10-09 05:11 +0200
  Re: Working with files on different operating systems Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2015-10-08 22:12 -0600
  Re: Working with files on different operating systems Les Cargill <lcargill99@comcast.com> - 2015-10-09 00:18 -0500
    Re: Working with files on different operating systems Ben Bacarisse <ben.usenet@bsb.me.uk> - 2015-10-09 11:13 +0100
      Re: Working with files on different operating systems "Morten W. Petersen" <morphex@gmail.com> - 2015-10-09 18:05 +0200
        Re: Working with files on different operating systems Les Cargill <lcargill99@comcast.com> - 2015-10-09 12:49 -0500
        Re: Working with files on different operating systems Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-10-09 17:32 -0400
        Re: Working with files on different operating systems Ian Collins <ian-news@hotmail.com> - 2015-10-10 12:37 +1300
      Re: Working with files on different operating systems Les Cargill <lcargill99@comcast.com> - 2015-10-09 12:48 -0500
        Re: Working with files on different operating systems Ben Bacarisse <ben.usenet@bsb.me.uk> - 2015-10-09 19:53 +0100
          Re: Working with files on different operating systems "Morten W. Petersen" <morphex@gmail.com> - 2015-10-09 21:09 +0200
            Re: Working with files on different operating systems Ben Bacarisse <ben.usenet@bsb.me.uk> - 2015-10-09 20:28 +0100
              Re: Working with files on different operating systems "Morten W. Petersen" <morphex@gmail.com> - 2015-10-09 21:40 +0200
                Re: Working with files on different operating systems James Kuyper <jameskuyper@verizon.net> - 2015-10-09 15:59 -0400
                Re: Working with files on different operating systems "Morten W. Petersen" <morphex@gmail.com> - 2015-10-09 22:07 +0200
                Re: Working with files on different operating systems James Kuyper <jameskuyper@verizon.net> - 2015-10-09 16:24 -0400
                Re: Working with files on different operating systems "Morten W. Petersen" <morphex@gmail.com> - 2015-10-10 03:08 +0200
                Re: Working with files on different operating systems Ben Bacarisse <ben.usenet@bsb.me.uk> - 2015-10-09 21:50 +0100
        Re: Working with files on different operating systems James Kuyper <jameskuyper@verizon.net> - 2015-10-09 14:54 -0400
          Re: Working with files on different operating systems "Morten W. Petersen" <morphex@gmail.com> - 2015-10-09 21:05 +0200
            Re: Working with files on different operating systems "Morten W. Petersen" <morphex@gmail.com> - 2015-10-09 21:47 +0200
            Re: Working with files on different operating systems James Kuyper <jameskuyper@verizon.net> - 2015-10-09 15:53 -0400
              Re: Working with files on different operating systems "Morten W. Petersen" <morphex@gmail.com> - 2015-10-09 22:01 +0200
                Re: Working with files on different operating systems James Kuyper <jameskuyper@verizon.net> - 2015-10-09 16:05 -0400
          Re: Working with files on different operating systems glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2015-10-10 01:16 +0000
            Re: Working with files on different operating systems "Morten W. Petersen" <morphex@gmail.com> - 2015-10-10 03:46 +0200
            Re: Working with files on different operating systems Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-10-10 08:08 -0400
          Re: Working with files on different operating systems raltbos@xs4all.nl (Richard Bos) - 2015-10-10 17:02 +0000
        Re: Working with files on different operating systems Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-10-09 15:53 -0400
    Re: Working with files on different operating systems "Morten W. Petersen" <morphex@gmail.com> - 2015-10-09 18:00 +0200
      Re: Working with files on different operating systems Les Cargill <lcargill99@comcast.com> - 2015-10-09 12:59 -0500
        Re: Working with files on different operating systems Keith Thompson <kst-u@mib.org> - 2015-10-09 12:37 -0700
          Re: Working with files on different operating systems Les Cargill <lcargill99@comcast.com> - 2015-10-09 17:36 -0500
            Re: Working with files on different operating systems Keith Thompson <kst-u@mib.org> - 2015-10-09 16:52 -0700
              Re: Working with files on different operating systems raltbos@xs4all.nl (Richard Bos) - 2015-10-14 11:27 +0000
                Re: Working with files on different operating systems James Kuyper <jameskuyper@verizon.net> - 2015-10-14 11:27 -0400
                Re: Working with files on different operating systems Jorgen Grahn <grahn+nntp@snipabacken.se> - 2015-10-14 16:46 +0000
                Re: Working with files on different operating systems Keith Thompson <kst-u@mib.org> - 2015-10-14 10:18 -0700
              Re: Working with files on different operating systems Les Cargill <lcargill99@comcast.com> - 2015-10-14 23:06 -0500
    Re: Working with files on different operating systems Keith Thompson <kst-u@mib.org> - 2015-10-09 09:16 -0700
      Re: Working with files on different operating systems Les Cargill <lcargill99@comcast.com> - 2015-10-09 17:43 -0500
        Re: Working with files on different operating systems Keith Thompson <kst-u@mib.org> - 2015-10-09 16:57 -0700
          Re: Working with files on different operating systems Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-10-09 22:41 -0700
    Re: Working with files on different operating systems David Thompson <dave.thompson2@verizon.net> - 2015-10-31 06:50 -0400
  Re: Working with files on different operating systems Siri Cruz <chine.bleu@yahoo.com> - 2015-10-08 23:25 -0700
    Re: Working with files on different operating systems Geoff <geoff@invalid.invalid> - 2015-10-10 10:35 -0700
  Re: Working with files on different operating systems Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-10-09 08:42 -0400
    Re: Working with files on different operating systems "Morten W. Petersen" <morphex@gmail.com> - 2015-10-09 18:51 +0200
      Re: Working with files on different operating systems Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-10-15 18:09 -0700
        Re: Working with files on different operating systems "Morten W. Petersen" <morphex@gmail.com> - 2015-10-17 04:01 +0200
          Re: Working with files on different operating systems Ian Collins <ian-news@hotmail.com> - 2015-10-17 15:55 +1300
            Re: Working with files on different operating systems Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-10-16 20:09 -0700
              Re: Working with files on different operating systems Ian Collins <ian-news@hotmail.com> - 2015-10-17 16:36 +1300
            Re: Working with files on different operating systems "Morten W. Petersen" <morphex@gmail.com> - 2015-10-17 05:10 +0200
        Re: Working with files on different operating systems supercat@casperkitty.com - 2015-10-31 12:05 -0700
          Re: Working with files on different operating systems Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-10-31 12:42 -0700
            Re: Working with files on different operating systems supercat@casperkitty.com - 2015-10-31 13:40 -0700
          Re: Working with files on different operating systems Ben Bacarisse <ben.usenet@bsb.me.uk> - 2015-10-31 19:55 +0000
            Re: Working with files on different operating systems supercat@casperkitty.com - 2015-10-31 13:43 -0700
              Re: Working with files on different operating systems Ben Bacarisse <ben.usenet@bsb.me.uk> - 2015-10-31 21:38 +0000

csiph-web