Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.programming > #16811 > unrolled thread

Somebody Got It The Wrong Way Round ...

Back to article view | Back to comp.programming

Contents

  Somebody Got It The Wrong Way Round ... Lawrence D'Oliveiro <ldo@nz.invalid> - 2025-07-16 23:21 +0000
    Re: Somebody Got It The Wrong Way Round ... c186282 <c186282@nnada.net> - 2025-07-18 02:14 -0400

#16811 — Somebody Got It The Wrong Way Round ...

From
<https://www.infoworld.com/article/4018856/4-tips-for-getting-started-with-free-threaded-python.html>:

    As an example, if you have a job that writes a lot of files,
    having each job in its own thread is less effective if each job
    also writes the file. This is because writing files is an
    inherently serial operation. A better approach would be to divide
    jobs across threads and use one thread for writing to disk. As
    each job finishes, it sends work to the disk-writing job. This
    way, jobs don’t block each other and aren’t themselves blocked by
    file writing.

Actually, blocking system calls (whether for I/O or something else)
only block the current thread. So having each thread do its own I/O
should be faster than funnelling it all through one bottleneck thread.

If you don’t want your worker threads blocked waiting for I/O to
complete, then each worker context can be a pair of threads: one does
the CPU-intensive stuff, while the other handles the blocking I/O.

[toc] | [next] | [standalone]

#16812

On 7/16/25 7:21 PM, Lawrence D'Oliveiro wrote:
> From
> <https://www.infoworld.com/article/4018856/4-tips-for-getting-started-with-free-threaded-python.html>:
> 
>      As an example, if you have a job that writes a lot of files,
>      having each job in its own thread is less effective if each job
>      also writes the file. This is because writing files is an
>      inherently serial operation. A better approach would be to divide
>      jobs across threads and use one thread for writing to disk. As
>      each job finishes, it sends work to the disk-writing job. This
>      way, jobs don’t block each other and aren’t themselves blocked by
>      file writing.
> 
> Actually, blocking system calls (whether for I/O or something else)
> only block the current thread. So having each thread do its own I/O
> should be faster than funnelling it all through one bottleneck thread.

   SEEMS true ... but not ALWAYS.

   Software design has to be customized for the
   exact job in mind.

> If you don’t want your worker threads blocked waiting for I/O to
> complete, then each worker context can be a pair of threads: one does
> the CPU-intensive stuff, while the other handles the blocking I/O.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.programming

csiph-web