Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #38335 > unrolled thread

select.epoll question

Back to article view | Back to comp.lang.python

Contents

  select.epoll question Paul Rubin <no.email@nospam.invalid> - 2013-02-06 23:08 -0800
    Re: select.epoll question Chris Angelico <rosuav@gmail.com> - 2013-02-07 23:49 +1100
      Re: select.epoll question Paul Rubin <no.email@nospam.invalid> - 2013-02-07 08:15 -0800
        Re: select.epoll question Chris Angelico <rosuav@gmail.com> - 2013-02-08 03:24 +1100
          Re: select.epoll question Paul Rubin <no.email@nospam.invalid> - 2013-02-07 12:19 -0800
            Re: select.epoll question Kushal Kumaran <kushal.kumaran+python@gmail.com> - 2013-02-08 07:36 +0530

#38335 — select.epoll question

I'm trying to listen to a bunch of sockets using epoll under Linux, e.g.

   import select, socket
   socket1 = socket.socket() ...

   p = select.epoll()
   p.register(socket1); p.register(socket2); ...
   result = p.poll()

This returns `result' as a list of 2-tuples (fd, status) where fd
is a Linux file descriptor, i.e. a small integer like comes back 
from socket.fileno().  That's different from select.select, which
returns a list of actual socket objects that I can read from with
socket.recv().  

Any idea of a good way to map the file descriptors back to socket
objects?  Is there some kind of hidden interface that I don't know
about, that gives back sockets directly?

The docs for epoll in select.html don't seem very good.  I haven't
yet examined the source code.

Thanks for any advice.

--Paul

[toc] | [next] | [standalone]

#38347

On Thu, Feb 7, 2013 at 6:08 PM, Paul Rubin <no.email@nospam.invalid> wrote:
> Any idea of a good way to map the file descriptors back to socket
> objects?  Is there some kind of hidden interface that I don't know
> about, that gives back sockets directly?

I don't know of any, but you can get the file descriptor from a socket
via fileno(), and build your own dictionary:

fd_to_sock={sock.fileno():sock for sock in list_of_sockets}

You'd need to manually maintain that as sockets get created/destroyed, though.

ChrisA

[toc] | [prev] | [next] | [standalone]

#38354

Chris Angelico <rosuav@gmail.com> writes:
> fd_to_sock={sock.fileno():sock for sock in list_of_sockets}
> You'd need to manually maintain that as sockets get created/destroyed,
> though

Thanks, I was hoping to avoid that.  I'll have to check how
select.select manages to return sockets.  Maybe it builds such a dict
from the object list before it calls the system's select function, then
maps the result  back afterwards.  Ugh.

[toc] | [prev] | [next] | [standalone]

#38355

On Fri, Feb 8, 2013 at 3:15 AM, Paul Rubin <no.email@nospam.invalid> wrote:
> Chris Angelico <rosuav@gmail.com> writes:
>> fd_to_sock={sock.fileno():sock for sock in list_of_sockets}
>> You'd need to manually maintain that as sockets get created/destroyed,
>> though
>
> Thanks, I was hoping to avoid that.  I'll have to check how
> select.select manages to return sockets.  Maybe it builds such a dict
> from the object list before it calls the system's select function, then
> maps the result  back afterwards.  Ugh.

Yeah, I figured fileno() probably wouldn't be news to you. I don't
suppose there's anything convenient in the rest of your application
that makes such a list/dict plausible? For instance, if you need to
maintain a list of all current socket connections to support broadcast
operations, then it's not much harder to also maintain the fd->socket
mapping.

ChrisA

[toc] | [prev] | [next] | [standalone]

#38372

Chris Angelico <rosuav@gmail.com> writes:
> Yeah, I figured fileno() probably wouldn't be news to you. I don't
> suppose there's anything convenient in the rest of your application
> that makes such a list/dict plausible? 

In fact it's rather annoying, sockets are created and destroyed in
multiple places in the program.  I have to be careful about leaking them
in case a thread crashes, etc.  I dealt with the poll issue by manually
tracking what was happening, but it wasn't pretty, so I wondered if
there was a better solution I was overlooking.  I don't want to mess
with it too much more since I'm planning a completely different approach
for the next version of the program.

[toc] | [prev] | [next] | [standalone]

#38392

Paul Rubin <no.email@nospam.invalid> writes:

> Chris Angelico <rosuav@gmail.com> writes:
>> Yeah, I figured fileno() probably wouldn't be news to you. I don't
>> suppose there's anything convenient in the rest of your application
>> that makes such a list/dict plausible? 
>
> In fact it's rather annoying, sockets are created and destroyed in
> multiple places in the program.  I have to be careful about leaking them
> in case a thread crashes, etc.  I dealt with the poll issue by manually
> tracking what was happening, but it wasn't pretty, so I wondered if
> there was a better solution I was overlooking.  I don't want to mess
> with it too much more since I'm planning a completely different approach
> for the next version of the program.

You don't have to maintain the mapping globally.  You could wrap
socket.epoll in a class of your own that manages the mapping, and passes
through calls to socket.epoll.  That would make the wrapper a drop-in
replacement for socket.epoll.

-- 
regards,
kushal

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web