Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c++ > #14821 > unrolled thread

copy/pass a va_list to another thread

Back to article view | Back to comp.lang.c++

Contents

  copy/pass a va_list to another thread Christopher <cpisz@austin.rr.com> - 2012-03-07 14:36 -0800
    Re: copy/pass a va_list to another thread "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2012-03-08 02:04 +0100
      Re: copy/pass a va_list to another thread Christopher <cpisz@austin.rr.com> - 2012-03-08 11:54 -0800
        Re: copy/pass a va_list to another thread "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2012-03-08 21:41 +0100
          Re: copy/pass a va_list to another thread Christopher <cpisz@austin.rr.com> - 2012-03-08 13:56 -0800
            OT: Free Usenet Access red floyd <no.spam.here@its.invalid> - 2012-03-08 16:09 -0800
              Re: OT: Free Usenet Access Christopher Pisz <someone@somewhere.net> - 2012-03-09 10:26 -0600
            New Google Groups problems (was Re: copy/pass a va_list to another thread) Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-09 12:27 +0000
            Re: copy/pass a va_list to another thread Robert Miles <milesrf@Usenet-News.net> - 2012-04-21 23:43 -0500
              XSUsenet.COM  @Jeff-Relf.Me - 2012-04-21 22:12 -0700
          Re: copy/pass a va_list to another thread Miles Bader <miles@gnu.org> - 2012-03-10 05:12 +0900
            Re: copy/pass a va_list to another thread "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2012-03-10 00:00 +0100
              Re: copy/pass a va_list to another thread Miles Bader <miles@gnu.org> - 2012-03-10 08:40 +0900
                Re: copy/pass a va_list to another thread Miles Bader <miles@gnu.org> - 2012-03-10 08:42 +0900
                MIME (Re: copy/pass a va_list to another thread) Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-10 07:04 +0000
                  Re: MIME (Re: copy/pass a va_list to another thread) Miles Bader <miles@gnu.org> - 2012-03-10 16:56 +0900
                    Re: MIME (Re: copy/pass a va_list to another thread) Gareth Owen <gwowen@gmail.com> - 2012-03-11 07:37 +0000
                Re: copy/pass a va_list to another thread "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2012-03-10 10:44 +0100
                  Re: copy/pass a va_list to another thread Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-10 14:43 +0000
                    Re: copy/pass a va_list to another thread "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2012-03-10 21:17 +0100
                  Re: copy/pass a va_list to another thread Miles Bader <miles@gnu.org> - 2012-03-11 10:49 +0900
                    Re: copy/pass a va_list to another thread Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-11 07:11 +0000
                      Re: copy/pass a va_list to another thread Miles Bader <miles@gnu.org> - 2012-03-11 19:03 +0900
                        Re: copy/pass a va_list to another thread Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-11 11:49 +0000
                          Re: copy/pass a va_list to another thread Miles Bader <miles@gnu.org> - 2012-03-11 23:01 +0900
                            Re: copy/pass a va_list to another thread "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2012-03-11 18:25 +0100
                            Re: copy/pass a va_list to another thread Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-11 23:53 +0000
              Re: copy/pass a va_list to another thread Mike McCarty <mcmccarty@gmail.com> - 2012-03-10 05:47 -0800
                Re: copy/pass a va_list to another thread Dombo <dombo@disposable.invalid> - 2012-03-10 19:51 +0100
                  Re: copy/pass a va_list to another thread "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2012-03-10 21:23 +0100
                    Re: copy/pass a va_list to another thread Dombo <dombo@disposable.invalid> - 2012-03-11 14:36 +0100
                      Re: copy/pass a va_list to another thread "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2012-03-11 18:33 +0100
                        Re: copy/pass a va_list to another thread Dombo <dombo@disposable.invalid> - 2012-03-11 20:41 +0100
                          Re: copy/pass a va_list to another thread "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2012-03-11 21:49 +0100
                    Re: copy/pass a va_list to another thread Rui Maciel <rui.maciel@gmail.com> - 2012-03-11 17:18 +0000
                      Re: copy/pass a va_list to another thread "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2012-03-11 18:39 +0100
                        Re: copy/pass a va_list to another thread Rui Maciel <rui.maciel@gmail.com> - 2012-03-11 18:54 +0000
                        Re: copy/pass a va_list to another thread Christian Gollwitzer <auriocus@gmx.de> - 2012-03-12 19:03 +0100
                Re: copy/pass a va_list to another thread Rui Maciel <rui.maciel@gmail.com> - 2012-03-11 17:09 +0000
            Re: copy/pass a va_list to another thread scott@slp53.sl.home (Scott Lurndal) - 2012-03-11 01:26 +0000
              Re: copy/pass a va_list to another thread Miles Bader <miles@gnu.org> - 2012-03-11 11:18 +0900
                Re: copy/pass a va_list to another thread scott@slp53.sl.home (Scott Lurndal) - 2012-03-11 16:41 +0000
        Re: copy/pass a va_list to another thread scott@slp53.sl.home (Scott Lurndal) - 2012-03-08 23:07 +0000
        Re: copy/pass a va_list to another thread Paavo Helde <myfirstname@osa.pri.ee> - 2012-03-08 23:17 -0600
        Re: copy/pass a va_list to another thread Stuart Redmann <DerTopper@web.de> - 2012-03-10 05:10 -0800
        Re: copy/pass a va_list to another thread 88888 Dihedral <dihedral88888@googlemail.com> - 2012-03-10 05:42 -0800
        Re: copy/pass a va_list to another thread Robert Miles <milesrf@Usenet-News.net> - 2012-04-21 23:20 -0500
    Re: copy/pass a va_list to another thread BGB <cr88192@hotmail.com> - 2012-03-08 09:25 -0700

Page 3 of 3 — ← Prev page 1 2 [3]

#14877

scott@slp53.sl.home (Scott Lurndal) writes:
> Miles Bader <miles@gnu.org> writes:
>>Are there people out there still using newsreaders from 1987...?
>
> Yes.  See headers.

Heh; I remember trying that for a few days ... in like 1991!

[To be fair, the newsreader I use now, Gnus, originally dates in some
sense from about 1988 (tho along the way, it's been rewritten
completely at least once).]

-miles

-- 
The car has become... an article of dress without which we feel uncertain,
unclad, and incomplete.  [Marshall McLuhan, Understanding Media, 1964]

[toc] | [prev] | [next] | [standalone]

#14886

Miles Bader <miles@gnu.org> writes:
>scott@slp53.sl.home (Scott Lurndal) writes:
>> Miles Bader <miles@gnu.org> writes:
>>>Are there people out there still using newsreaders from 1987...?
>>
>> Yes.  See headers.
>
>Heh; I remember trying that for a few days ... in like 1991!

I started using it in 1989.

I've had to modify it a couple of times (once for compatibility with
some odd typhoon/tornado NNTP protocol differences from cnews, once to compile correctly
on 64-bit).

scott

[toc] | [prev] | [next] | [standalone]

#14840

Christopher <cpisz@austin.rr.com> writes:
>On Wednesday, March 7, 2012 7:04:26 PM UTC-6, Alf P. Steinbach wrote:
>> You have already stated that copying takes too long.
>>=20
>> And you can't pass a reference or pointer to it without letting the=20
>> original thread wait.
>>=20
>> So at first glance the problem is insoluble.
>
>Maybe. I am hoping for some boost or C++ standard mechanism, which I haven'=
>t heard of yet, that solves this problem that I presume many developers hav=
>e run across, when dealing with a function that takes a variable argument l=
>ist parameter.

if all the variadic arguments are passed by value (e.g. sizeof(long) or less),
then you, depending on your compiler, may be able to get away with using
va_copy to make a copy of the variadic argument structure in a non-stack
location accessible to the original thread, hand it off to the worker bee
and when the worker bee is finished, it can va_end() to free any resources.

both va_copy and va_end are standard C/C++.

If you throw portability aside and study how the compiler generates the
variadic arguments associated with va_list, you can write some very non-portable
code to deep-copy the va-list and hand it to the worker thread, with the caveat
that any reference parameters or pointers may have changed by the time the
worker thread gets around to dereferenceing them (since the original thread
has gone on to bigger and better things.   You may need to craft this with
inline assembler (if the compiler supports it) or hand-assembled code to access
the portions of the stack you'll need to deep copy from).  Dangerous, this
code could easily corrupt the stack, and it will certainly be likely to
fail with newer compiler versions or alternate compilers.

scott

[toc] | [prev] | [next] | [standalone]

#14842

Christopher <cpisz@austin.rr.com> wrote in
news:7509868.1455.1331236446852.JavaMail.geo-discussion-forums@ynjd19: 

> On Wednesday, March 7, 2012 7:04:26 PM UTC-6, Alf P. Steinbach wrote:
>> You have already stated that copying takes too long.
>> 
>> And you can't pass a reference or pointer to it without letting the 
>> original thread wait.
>> 
>> So at first glance the problem is insoluble.
> 
> Maybe. I am hoping for some boost or C++ standard mechanism, which I
> haven't heard of yet, that solves this problem that I presume many
> developers have run across, when dealing with a function that takes a
> variable argument list parameter. 

va_args are a necessary evil in the C language, which otherwise lacks the 
means to pass varying parameter lists conveniently. In C++ we can do 
better, and I am fairly certain there are not so many C++ developers 
worrying about va_arg functions, let alone about marshalling such calls 
into another thread.

[snip a big rant about inadequancy of va_arg calling]

The types of the variables are known at the call point. Of course the 
performance suffers if one loses that information, encodes it in some 
format strings instead and tediously recovers later. Moving this into 
another thread is like to brush the litter away under the carpet.

> performance analysis revealed that the processing of the list, just to
> extract the types and use them to build the output to make the
> database call with, took longer than the database call itself! 

It's still hard to believe. Maybe the list processing code does something 
very stupid, like applying an O(N^2) algorithm instead of O(N)? You 
mentioned there are thousands of arguments, so a big-O analysis is in 
place.

hth
Paavo

[toc] | [prev] | [next] | [standalone]

#14866

On 8 Mrz., 20:54, Christopher wrote:

[snip]

> Such is often life when dealing with other people's real life code
> and manager's rules and time allotments vs academic examples. Given
> those constants, it doesn't matter what the context is, but here:
>
> // Creates a database log record
> // \param sprintf like message string including placeholders
> // \param ... - takes tuples
> //              1- string; name of argument
> //              2- string; formatting code string
> //              3- any type; identified by previous formatting code
> // <list of 500 formatting codes and argument types>
> void SomeFunction(std::wstring message, ...)   // Cannot change
> {
>    va_list va;  // May change anything here down
>    va_start(va, x);
>
[snip]
>    // 4000 lines of parsing custom format codes and arguments,
>    //      converting to types using those types to build strings
>    //      ala sprintf
[snip]
>    va_end(va);
>
> }
>
> The actual conversion from unknown types into usuable typed
> objects using format codes in strings is what is taking a
> long time.
[snip]
> performance analysis revealed that the processing of the list,
> just to extract the types and use them to build the output to
> make the database call with, took longer than the database call
> itself!

As far as I understand, you don't want to process the command
in the thread that called SomeFunction but in a worker thread,
but you don't know how many parameters must be copied to the
worker thread without parsing the parameter "message".

This could be solved in an unsafe way by copying "suffiently much"
stack parameters to some heap location where the worker thread can
access it.

However, most likely it will be the case that some parameters
of SomeFunction are only valid as long as SomeFunction runs. So you
cannot simply copy the parameters and return from SomeFunctions
because your worker thread will then operate on invalid data.

I guess you will have to bite the bullet and try to tell your
management that your function needs to be redesigned. Hard luck!

Regards,
Stuart

[toc] | [prev] | [next] | [standalone]

#14867

Christopher於 2012年3月9日星期五UTC+8上午3時54分06秒寫道：
> On Wednesday, March 7, 2012 7:04:26 PM UTC-6, Alf P. Steinbach wrote:
> > You have already stated that copying takes too long.
> > 
> > And you can't pass a reference or pointer to it without letting the 
> > original thread wait.
> > 
> > So at first glance the problem is insoluble.
> 
> Maybe. I am hoping for some boost or C++ standard mechanism, which I haven't heard of yet, that solves this problem that I presume many developers have run across, when dealing with a function that takes a variable argument list parameter.
> 
> > Just post real code instead of a vague and most likely incorrect 
> > description.
> 
> I think you are reading to much into it and want to look at the design for a fix. I don't blame you. I already accept that the design is flawed, the code sucks, and I'd never use a variable argument list in the first place. Such is often life when dealing with other people's real life code and manager's rules and time allotments vs academic examples. Given those constants, it doesn't matter what the context is, but here:
> 
> // Creates a database log record
> // \param sprintf like message string including placeholders
> // \param ... - takes tuples
> //              1- string; name of argument
> //              2- string; formatting code string
> //              3- any type; identified by previous formatting code
> // <list of 500 formatting codes and argument types>
> void SomeFunction(std::wstring message, ...)   // Cannot change
> {
>    va_list va;                                 // May change anything here down
>    va_start(va, x);
> 
>    SomeDataStructure myOutput;    // Contains arguments for a database call
>                                   //     to create a log record
>                                   //     which is also crappy design
>                                   //     with tons of params
>                                   //     and encoded, formatted, strings
>    
>    // 4000 lines of parsing custom format codes and arguments, 
>    //      converting to types
>    //      using those types to build strings
>    //      ala sprintf
>    //      With a dump truck of business rules applied to string encoding
>    //      AKA long and tedious stuff that I want to do on another thread
>    //      Including actually making a call to write to database which is
>    //      also time consuming
>    //
>    //      i.e. This is a logging type method and should return immediatly
> 
>    va_end(va);
> }
> 
> The actual conversion from unknown types into usuable typed objects using format codes in strings is what is taking a long time. I really don't think I mentioned copying as you say. There is no copying yet. I envision copying as a means to pass them to another thread. As obviously I can't give something to another thread from the stack without blocking the first thread or they will no longer be valid. 
> 
> The problem itself is how to copy something of unknown type and length from the "stack pointer?" given by va_list, without actually processing the list to find the types first.
> 
> performance analysis revealed that the processing of the list, just to extract the types and use them to build the output to make the database call with, took longer than the database call itself!



Christopher於 2012年3月9日星期五UTC+8上午3時54分06秒寫道：
> On Wednesday, March 7, 2012 7:04:26 PM UTC-6, Alf P. Steinbach wrote:
> > You have already stated that copying takes too long.
> > 
> > And you can't pass a reference or pointer to it without letting the 
> > original thread wait.
> > 
> > So at first glance the problem is insoluble.
> 
> Maybe. I am hoping for some boost or C++ standard mechanism, which I haven't heard of yet, that solves this problem that I presume many developers have run across, when dealing with a function that takes a variable argument list parameter.
> 
> > Just post real code instead of a vague and most likely incorrect 
> > description.
> 
> I think you are reading to much into it and want to look at the design for a fix. I don't blame you. I already accept that the design is flawed, the code sucks, and I'd never use a variable argument list in the first place. Such is often life when dealing with other people's real life code and manager's rules and time allotments vs academic examples. Given those constants, it doesn't matter what the context is, but here:
> 
> // Creates a database log record
> // \param sprintf like message string including placeholders
> // \param ... - takes tuples
> //              1- string; name of argument
> //              2- string; formatting code string
> //              3- any type; identified by previous formatting code
> // <list of 500 formatting codes and argument types>
> void SomeFunction(std::wstring message, ...)   // Cannot change
> {
>    va_list va;                                 // May change anything here down
>    va_start(va, x);
> 
>    SomeDataStructure myOutput;    // Contains arguments for a database call
>                                   //     to create a log record
>                                   //     which is also crappy design
>                                   //     with tons of params
>                                   //     and encoded, formatted, strings
>    
>    // 4000 lines of parsing custom format codes and arguments, 
>    //      converting to types
>    //      using those types to build strings
>    //      ala sprintf
>    //      With a dump truck of business rules applied to string encoding
>    //      AKA long and tedious stuff that I want to do on another thread
>    //      Including actually making a call to write to database which is
>    //      also time consuming
>    //
>    //      i.e. This is a logging type method and should return immediatly
> 
>    va_end(va);
> }
> 
> The actual conversion from unknown types into usuable typed objects using format codes in strings is what is taking a long time. I really don't think I mentioned copying as you say. There is no copying yet. I envision copying as a means to pass them to another thread. As obviously I can't give something to another thread from the stack without blocking the first thread or they will no longer be valid. 
> 
> The problem itself is how to copy something of unknown type and length from the "stack pointer?" given by va_list, without actually processing the list to find the types first.
> 
> performance analysis revealed that the processing of the list, just to extract the types and usfuncte them to build the output to make the database call with, took longer than the database call itself!


I don't like this tyle of coding, and I'll suggest to use an input  vector 
to be passed to a function thus that all parameters are received in 
the callee function to start the action.

Well, to pass by register does matter a lot in calling work-horse level
actions. 

[toc] | [prev] | [next] | [standalone]

#15508

On 3/8/2012 1:54 PM, Christopher wrote:
> On Wednesday, March 7, 2012 7:04:26 PM UTC-6, Alf P. Steinbach wrote:
>> You have already stated that copying takes too long.
>>
>> And you can't pass a reference or pointer to it without letting the
>> original thread wait.
>>
>> So at first glance the problem is insoluble.
>
> Maybe. I am hoping for some boost or C++ standard mechanism, which I haven't heard of yet, that solves this problem that I presume many developers have run across, when dealing with a function that takes a variable argument list parameter.
>
>> Just post real code instead of a vague and most likely incorrect
>> description.
>
> I think you are reading to much into it and want to look at the design for a fix. I don't blame you. I already accept that the design is flawed, the code sucks, and I'd never use a variable argument list in the first place. Such is often life when dealing with other people's real life code and manager's rules and time allotments vs academic examples. Given those constants, it doesn't matter what the context is, but here:
>
> // Creates a database log record
> // \param sprintf like message string including placeholders
> // \param ... - takes tuples
> //              1- string; name of argument
> //              2- string; formatting code string
> //              3- any type; identified by previous formatting code
> //<list of 500 formatting codes and argument types>
> void SomeFunction(std::wstring message, ...)   // Cannot change
> {
>     va_list va;                                 // May change anything here down
>     va_start(va, x);
>
>     SomeDataStructure myOutput;    // Contains arguments for a database call
>                                    //     to create a log record
>                                    //     which is also crappy design
>                                    //     with tons of params
>                                    //     and encoded, formatted, strings
>
>     // 4000 lines of parsing custom format codes and arguments,
>     //      converting to types
>     //      using those types to build strings
>     //      ala sprintf
>     //      With a dump truck of business rules applied to string encoding
>     //      AKA long and tedious stuff that I want to do on another thread
>     //      Including actually making a call to write to database which is
>     //      also time consuming
>     //
>     //      i.e. This is a logging type method and should return immediatly
>
>     va_end(va);
> }
>
> The actual conversion from unknown types into usuable typed objects using format codes in strings is what is taking a long time. I really don't think I mentioned copying as you say. There is no copying yet. I envision copying as a means to pass them to another thread. As obviously I can't give something to another thread from the stack without blocking the first thread or they will no longer be valid.
>
> The problem itself is how to copy something of unknown type and length from the "stack pointer?" given by va_list, without actually processing the list to find the types first.
>
> performance analysis revealed that the processing of the list, just to extract the types and use them to build the output to make the database call with, took longer than the database call itself!

Is there a way to split the list of format codes in half, then quickly
test which half is used?  If so, consider splitting it in quarters or
eighths with more quick tests of which part of the list.  That's likely
to be much faster than testing for every possibility, one at a time.

[toc] | [prev] | [next] | [standalone]

#14830

On 3/7/2012 3:36 PM, Christopher wrote:
> Working on a pre-existing interface that I am not allowed to change, which uses a va_list argument. The method takes way too long to execute and is a canidate for splitting off into another thread. However, I am not sure how I can safely copy the va_list without actually processing its real components. The processing into its real components is a good chunk of what is taking too long.
>
> Any ideas on how to safely pass a varibale argument list to another thread to be processed?

no, va_list can't be safely passed (given they depend on the callers' 
stack frame, which is thread-local).

best option I know of:
read arguments from va_list into some other format (such as an array);
pass a copy of this data instead.

granted, this requires some means to know the layout of the list in 
advance (number and types of arguments), which is presumably available 
somehow?

[toc] | [prev] | [standalone]

Page 3 of 3 — ← Prev page 1 2 [3]

Back to top | Article view | comp.lang.c++

csiph-web