Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.networking > #1071 > unrolled thread

Data sent through sockets are *sometimes* inverted ?!!!

Back to article view | Back to comp.os.linux.networking

Contents

  Data sent through sockets are *sometimes* inverted ?!!! mast4as <mast4as@yahoo.com> - 2012-02-17 00:38 -0800
    Re: Data sent through sockets are *sometimes* inverted ?!!! Chris Davies <chris-usenet@roaima.co.uk> - 2012-02-17 09:50 +0000
    Re: Data sent through sockets are *sometimes* inverted ?!!! Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-02-17 10:25 +0000
      Re: Data sent through sockets are *sometimes* inverted ?!!! Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2012-02-17 13:42 -0700
        Re: Data sent through sockets are *sometimes* inverted ?!!! Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-02-19 19:50 +0000
          Re: Data sent through sockets are *sometimes* inverted ?!!! Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2012-02-19 21:19 -0700
            Re: Data sent through sockets are *sometimes* inverted ?!!! Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-02-20 09:01 +0000
              Re: Data sent through sockets are *sometimes* inverted ?!!! Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-02-22 08:56 +0000
                Re: Data sent through sockets are *sometimes* inverted ?!!! Jerry Peters <jerry@example.invalid> - 2012-02-22 22:59 +0000
                  Re: Data sent through sockets are *sometimes* inverted ?!!! Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-02-23 00:13 +0000
                    Re: Data sent through sockets are *sometimes* inverted ?!!! Ralph Spitzner <rasp@spitzner.org> - 2012-02-23 06:00 +0100
                      Re: Data sent through sockets are *sometimes* inverted ?!!! Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-02-23 07:34 +0000
                        Re: Data sent through sockets are *sometimes* inverted ?!!! Chris Davies <chris-usenet@roaima.co.uk> - 2012-02-23 11:12 +0000
                          Re: Data sent through sockets are *sometimes* inverted ?!!! Rick Jones <rick.jones2@hp.com> - 2012-02-24 19:25 +0000
                            Re: Data sent through sockets are *sometimes* inverted ?!!! Chris Davies <chris-usenet@roaima.co.uk> - 2012-02-24 21:03 +0000
                            Re: Data sent through sockets are *sometimes* inverted ?!!! Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-02-25 04:49 +0000
                              Re: Data sent through sockets are *sometimes* inverted ?!!! Chris Davies <chris-usenet@roaima.co.uk> - 2012-02-26 12:18 +0000
                              Re: Data sent through sockets are *sometimes* inverted ?!!! Pascal Hambourg <boite-a-spam@plouf.fr.eu.org> - 2012-02-26 14:10 +0100
                                Re: Data sent through sockets are *sometimes* inverted ?!!! Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-02-26 14:02 +0000
                        Re: Data sent through sockets are *sometimes* inverted ?!!! Ralph Spitzner <rasp@spitzner.org> - 2012-02-23 17:03 +0100
                          Re: Data sent through sockets are *sometimes* inverted ?!!! Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2012-02-23 10:17 -0700
                        Re: Data sent through sockets are *sometimes* inverted ?!!! Grant Edwards <invalid@invalid.invalid> - 2012-02-23 19:13 +0000
                          Failing TCP checksums (was Re: Data sent through sockets are *sometimes* inverted ?!!!) Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-02-23 20:40 +0000
                    Re: Data sent through sockets are *sometimes* inverted ?!!! Jerry Peters <jerry@example.invalid> - 2012-02-23 21:34 +0000
                      Re: Data sent through sockets are *sometimes* inverted ?!!! Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-02-23 22:12 +0000
                      Re: Data sent through sockets are *sometimes* inverted ?!!! Grant Edwards <invalid@invalid.invalid> - 2012-02-24 15:23 +0000
                    Re: Data sent through sockets are *sometimes* inverted ?!!! Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-02-25 10:54 +0000
    Re: Data sent through sockets are *sometimes* inverted ?!!! Richard Kettlewell <rjk@greenend.org.uk> - 2012-02-17 10:07 +0000

Page 1 of 2  [1] 2  Next page →

#1071 — Data sent through sockets are *sometimes* inverted ?!!!

Hi everyone

I have a simple client/server apps where the client send data (TCP/IP)
in the form of square of pixels colors (3 floats) to the server. I can
change the size of the square but by default, I started with something
like 32x32 pixels. So overall that's packets of 32*32*3*8 (8 bytes per
float) floats sent to the server. These packets are send to the server
for an entire image which can be say 1024x1024 in resolution. For my
testing I set the pixel color to always be 1 0 0.1.

On the client side:
* 1 write -> 4 integers to specify pos and size of the square in the
frame
* 1 write -> square dim^2 * 3 (RGB) * 8 bytes pixel color data

On the server side I do 2 reads:
* do a first read to find the dimension and position of the square in
the frame with 1 read
* read all the pixels for the current square (square size^2 * 3 (RGB)
* 8 bytes) with 1 read

It runs okay but occasionally the squares have wrong values. For a
block of 32x32 pixels say the first 20 pixels have RGB values that are
1 0 0.1 then the next 20 pixels have their RGB inverted say: 0 0.1 1
then the rest of the pixels will be 0.1 0 1 say. This problem only
happens when the square reach a certain size. When the square are 8x8
or 16x16.

So I was wondering if it is possible that when the packed of data sent
is too big, that the order of the bytes changes when it's read by the
server. Or is this not supposed to happen at all which would suggest a
bug in the code (however I really checked that I was writing and
reading the correct number of bytes so I find that strange).

If the write/read process doesn't guarantee that the bytes are not re-
ordered, is that just happening after the data written and read go
over a certain limit. Can I find this limit ? In other words is there
a limit (number of bytes) under which it is certain that bytes won't
be re-ordered. It seems that I can send the data in much smaller
packets (by sending say 8 pixels at a time from a square of 64x64) but
isn't that not very efficient ?

Thanks a lot for your help

-coralie

[toc] | [next] | [standalone]

#1074

mast4as <mast4as@yahoo.com> wrote:
> I have a simple client/server apps where the client send data (TCP/IP)
> in the form of square of pixels colors (3 floats) to the server.

I can't help but feel you're reinventing an image format  and/or part
of the VNC protocol.


> If the write/read process doesn't guarantee that the bytes are not re-
> ordered [...]

TCP guarantees ordered delivery. (Or no delivery at all.) So there will be
nothing at the transport layer that changes the order of your bytes.

Is your code going to be portable across multiple CPU
architectures? (Might it need to be so? Consider mobile devices, for a
start.) If so, you really need to be aware of things such as "network
byte order".

Chris

[toc] | [prev] | [next] | [standalone]

#1075

On Fri, 2012-02-17, mast4as wrote:
> Hi everyone
>
> I have a simple client/server apps where the client send data (TCP/IP)
> in the form of square of pixels colors (3 floats) to the server.

TCP only deals with series of octets/bytes. The rest is up to your
application to handle.  If you're thinking of it as "sending three
floats", your approach is wrong. If you cannot describe the protocol
in terms of octets, your approach is wrong.

[snip]

I didn't read the rest carefully, but I think this may be the core of
your problem.

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#1076

Jorgen Grahn <grahn+nntp@snipabacken.se> writes:

> On Fri, 2012-02-17, mast4as wrote:
>> Hi everyone
>>
>> I have a simple client/server apps where the client send data (TCP/IP)
>> in the form of square of pixels colors (3 floats) to the server.
>
> TCP only deals with series of octets/bytes. The rest is up to your
> application to handle.  If you're thinking of it as "sending three
> floats", your approach is wrong. If you cannot describe the protocol
> in terms of octets, your approach is wrong.
>
> [snip]
>
> I didn't read the rest carefully, but I think this may be the core of
> your problem.

And, a particular way this can cause a bug in your code:  when you
write() some number n bytes and then attempt to read() n bytes, there is
no guarantee you will actually get all n bytes in a single read() -- you
may well need to make several read() calls to get all the bytes, using
something like

for (count = 0; count < n; count += num)
   num = read(fd, buf+count, n-count);
   if (num < 0) {
       // some sort of error-handling code
   }
}

(untested off-the-top-of-my-head code but you get the idea)

If your code is assuming you get all the bytes in one read() (and I do
read your description as saying that's what you're doing), then part
of your buffer may contain old values and your next read() may return
some data from the prior write().  This could cause the sort of symptoms
you're reporting.

write() is also not guaranteed to send all the bytes out in a single
try, so careful programming would require a similar loop for your
write()s.  I've never actually seen a write() do anything but either
send all the bytes or else fail, however.

[toc] | [prev] | [next] | [standalone]

#1080

On Fri, 2012-02-17, Joe Pfeiffer wrote:
> Jorgen Grahn <grahn+nntp@snipabacken.se> writes:
>
>> On Fri, 2012-02-17, mast4as wrote:
>>> Hi everyone
>>>
>>> I have a simple client/server apps where the client send data (TCP/IP)
>>> in the form of square of pixels colors (3 floats) to the server.
>>
>> TCP only deals with series of octets/bytes. The rest is up to your
>> application to handle.  If you're thinking of it as "sending three
>> floats", your approach is wrong. If you cannot describe the protocol
>> in terms of octets, your approach is wrong.
>>
>> [snip]
>>
>> I didn't read the rest carefully, but I think this may be the core of
>> your problem.
>
> And, a particular way this can cause a bug in your code:  when you
> write() some number n bytes and then attempt to read() n bytes, there is
> no guarantee you will actually get all n bytes in a single read()

That wasn't what I was writing about, but you're right, of course.
With TCP you're responsible for cutting the byte stream into pieces if
asked to, and reassembling the pieces you read, when needed.

...
> write() is also not guaranteed to send all the bytes out in a single
> try, so careful programming would require a similar loop for your
> write()s.  I've never actually seen a write() do anything but either
> send all the bytes or else fail, however.

Have you looked? If you have a TCP socket with just N bytes of free Tx
buffer, and you write N+1 bytes to it, I'd expect a partial write to
happen.  I'd consider /not/ handling that case a serious bug.

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#1081

Jorgen Grahn <grahn+nntp@snipabacken.se> writes:

> On Fri, 2012-02-17, Joe Pfeiffer wrote:
>> Jorgen Grahn <grahn+nntp@snipabacken.se> writes:
>>
>>> On Fri, 2012-02-17, mast4as wrote:
>>>> Hi everyone
>>>>
>>>> I have a simple client/server apps where the client send data (TCP/IP)
>>>> in the form of square of pixels colors (3 floats) to the server.
>>>
>>> TCP only deals with series of octets/bytes. The rest is up to your
>>> application to handle.  If you're thinking of it as "sending three
>>> floats", your approach is wrong. If you cannot describe the protocol
>>> in terms of octets, your approach is wrong.
>>>
>>> [snip]
>>>
>>> I didn't read the rest carefully, but I think this may be the core of
>>> your problem.
>>
>> And, a particular way this can cause a bug in your code:  when you
>> write() some number n bytes and then attempt to read() n bytes, there is
>> no guarantee you will actually get all n bytes in a single read()
>
> That wasn't what I was writing about, but you're right, of course.
> With TCP you're responsible for cutting the byte stream into pieces if
> asked to, and reassembling the pieces you read, when needed.

I'd actually argue that that was *exactly* what you were writing about.
The programmer needs to make sure that his code correctly translates the
objects he wants to think about into the bytes that TCP will be
"thinking" about, and that those bytes get sent and received
correctly :)

> ...
>> write() is also not guaranteed to send all the bytes out in a single
>> try, so careful programming would require a similar loop for your
>> write()s.  I've never actually seen a write() do anything but either
>> send all the bytes or else fail, however.
>
> Have you looked? If you have a TCP socket with just N bytes of free Tx
> buffer, and you write N+1 bytes to it, I'd expect a partial write to
> happen.  I'd consider /not/ handling that case a serious bug.

Yes, I have -- as part of debugging more instances than I want to admit
to of discovering I wasn't doing a good enough job of what we discuss
above.

I've wound up dumping lots and lots of "tried to write %d, write
returned %d" and I can't remember ever seeing an instance of something
other than -1 or what I was trying to send.  It's an easy enough
loop to write that I always do it (you're right, not handling the case
would be a bug that might turn up now and might turn up in a decade),
but so far I can't remember ever going around that loop twice.

[toc] | [prev] | [next] | [standalone]

#1082

On Mon, 2012-02-20, Joe Pfeiffer wrote:
> Jorgen Grahn <grahn+nntp@snipabacken.se> writes:
>
>> On Fri, 2012-02-17, Joe Pfeiffer wrote:
>>> Jorgen Grahn <grahn+nntp@snipabacken.se> writes:
>>>
>>>> On Fri, 2012-02-17, mast4as wrote:
>>>>> Hi everyone
>>>>>
>>>>> I have a simple client/server apps where the client send data (TCP/IP)
>>>>> in the form of square of pixels colors (3 floats) to the server.
>>>>
>>>> TCP only deals with series of octets/bytes. The rest is up to your
>>>> application to handle.  If you're thinking of it as "sending three
>>>> floats", your approach is wrong. If you cannot describe the protocol
>>>> in terms of octets, your approach is wrong.
>>>>
>>>> [snip]
>>>>
>>>> I didn't read the rest carefully, but I think this may be the core of
>>>> your problem.
>>>
>>> And, a particular way this can cause a bug in your code:  when you
>>> write() some number n bytes and then attempt to read() n bytes, there is
>>> no guarantee you will actually get all n bytes in a single read()
>>
>> That wasn't what I was writing about, but you're right, of course.
>> With TCP you're responsible for cutting the byte stream into pieces if
>> asked to, and reassembling the pieces you read, when needed.
>
> I'd actually argue that that was *exactly* what you were writing about.
> The programmer needs to make sure that his code correctly translates the
> objects he wants to think about into the bytes that TCP will be
> "thinking" about, and that those bytes get sent and received
> correctly :)

Ok. My mental image of it is two separate parts: (a) expressing your
data in terms of streams of octets (or lines of text, as in the more
popular RFCs), and (b) mapping those streams to socket read/write
semantics.

>> ...
>>> write() is also not guaranteed to send all the bytes out in a single
>>> try, so careful programming would require a similar loop for your
>>> write()s.  I've never actually seen a write() do anything but either
>>> send all the bytes or else fail, however.
>>
>> Have you looked? If you have a TCP socket with just N bytes of free Tx
>> buffer, and you write N+1 bytes to it, I'd expect a partial write to
>> happen.  I'd consider /not/ handling that case a serious bug.
>
> Yes, I have -- as part of debugging more instances than I want to admit
> to of discovering I wasn't doing a good enough job of what we discuss
> above.
>
> I've wound up dumping lots and lots of "tried to write %d, write
> returned %d" and I can't remember ever seeing an instance of something
> other than -1 or what I was trying to send.  It's an easy enough
> loop to write that I always do it (you're right, not handling the case
> would be a bug that might turn up now and might turn up in a decade),
> but so far I can't remember ever going around that loop twice.

Interesting -- I must try it tonight. (On Linux, since I don't have
easy access to any other Unix outside work.)

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#1083

On Mon, 2012-02-20, Jorgen Grahn wrote:
> On Mon, 2012-02-20, Joe Pfeiffer wrote:
>> Jorgen Grahn <grahn+nntp@snipabacken.se> writes:
>>
>>> On Fri, 2012-02-17, Joe Pfeiffer wrote:
...
>>>> write() is also not guaranteed to send all the bytes out in a single
>>>> try, so careful programming would require a similar loop for your
>>>> write()s.  I've never actually seen a write() do anything but either
>>>> send all the bytes or else fail, however.
>>>
>>> Have you looked? If you have a TCP socket with just N bytes of free Tx
>>> buffer, and you write N+1 bytes to it, I'd expect a partial write to
>>> happen.  I'd consider /not/ handling that case a serious bug.
>>
>> Yes, I have -- as part of debugging more instances than I want to admit
>> to of discovering I wasn't doing a good enough job of what we discuss
>> above.
>>
>> I've wound up dumping lots and lots of "tried to write %d, write
>> returned %d" and I can't remember ever seeing an instance of something
>> other than -1 or what I was trying to send.  It's an easy enough
>> loop to write that I always do it (you're right, not handling the case
>> would be a bug that might turn up now and might turn up in a decade),
>> but so far I can't remember ever going around that loop twice.
>
> Interesting -- I must try it tonight. (On Linux, since I don't have
> easy access to any other Unix outside work.)

So far I have failed to disprove what you wrote. Netcat to a
SIGSTOPped server, until first the server's RX buffer got filled and
then the client's TX buffer got filled. The client blocked instead of
performing a partial write.

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#1084

Jorgen Grahn <grahn+nntp@snipabacken.se> wrote:
> On Mon, 2012-02-20, Jorgen Grahn wrote:
>> On Mon, 2012-02-20, Joe Pfeiffer wrote:
>>> Jorgen Grahn <grahn+nntp@snipabacken.se> writes:
>>>
>>>> On Fri, 2012-02-17, Joe Pfeiffer wrote:
> ...
>>>>> write() is also not guaranteed to send all the bytes out in a single
>>>>> try, so careful programming would require a similar loop for your
>>>>> write()s.  I've never actually seen a write() do anything but either
>>>>> send all the bytes or else fail, however.
>>>>
>>>> Have you looked? If you have a TCP socket with just N bytes of free Tx
>>>> buffer, and you write N+1 bytes to it, I'd expect a partial write to
>>>> happen.  I'd consider /not/ handling that case a serious bug.
>>>
>>> Yes, I have -- as part of debugging more instances than I want to admit
>>> to of discovering I wasn't doing a good enough job of what we discuss
>>> above.
>>>
>>> I've wound up dumping lots and lots of "tried to write %d, write
>>> returned %d" and I can't remember ever seeing an instance of something
>>> other than -1 or what I was trying to send.  It's an easy enough
>>> loop to write that I always do it (you're right, not handling the case
>>> would be a bug that might turn up now and might turn up in a decade),
>>> but so far I can't remember ever going around that loop twice.
>>
>> Interesting -- I must try it tonight. (On Linux, since I don't have
>> easy access to any other Unix outside work.)
> 
> So far I have failed to disprove what you wrote. Netcat to a
> SIGSTOPped server, until first the server's RX buffer got filled and
> then the client's TX buffer got filled. The client blocked instead of
> performing a partial write.
> 
> /Jorgen
> 
What happens if you set the socket to non-blocking?

	Jerry

[toc] | [prev] | [next] | [standalone]

#1085

On Wed, 2012-02-22, Jerry Peters wrote:
> Jorgen Grahn <grahn+nntp@snipabacken.se> wrote:
>> On Mon, 2012-02-20, Jorgen Grahn wrote:
>>> On Mon, 2012-02-20, Joe Pfeiffer wrote:
>>>> Jorgen Grahn <grahn+nntp@snipabacken.se> writes:
>>>>
>>>>> On Fri, 2012-02-17, Joe Pfeiffer wrote:
>> ...
>>>>>> write() is also not guaranteed to send all the bytes out in a single
>>>>>> try, so careful programming would require a similar loop for your
>>>>>> write()s.  I've never actually seen a write() do anything but either
>>>>>> send all the bytes or else fail, however.
>>>>>
>>>>> Have you looked? If you have a TCP socket with just N bytes of free Tx
>>>>> buffer, and you write N+1 bytes to it, I'd expect a partial write to
>>>>> happen.  I'd consider /not/ handling that case a serious bug.
>>>>
>>>> Yes, I have -- as part of debugging more instances than I want to admit
>>>> to of discovering I wasn't doing a good enough job of what we discuss
>>>> above.
>>>>
>>>> I've wound up dumping lots and lots of "tried to write %d, write
>>>> returned %d" and I can't remember ever seeing an instance of something
>>>> other than -1 or what I was trying to send.  It's an easy enough
>>>> loop to write that I always do it (you're right, not handling the case
>>>> would be a bug that might turn up now and might turn up in a decade),
>>>> but so far I can't remember ever going around that loop twice.
>>>
>>> Interesting -- I must try it tonight. (On Linux, since I don't have
>>> easy access to any other Unix outside work.)
>> 
>> So far I have failed to disprove what you wrote. Netcat to a
>> SIGSTOPped server, until first the server's RX buffer got filled and
>> then the client's TX buffer got filled. The client blocked instead of
>> performing a partial write.

> What happens if you set the socket to non-blocking?

Something else, probably. But at this point I'd rather look at the
linux/net/ sources -- when I have the time and energy.

Noone here has claimed that you don't need to check for partial writes,
but it could be very useful to know (e.g. for debugging purposes) how
Linux behaves in this case.

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#1086

Jorgen Grahn wrote:
[...]
> Noone here has claimed that you don't need to check for partial writes,
> but it could be very useful to know (e.g. for debugging purposes) how
> Linux behaves in this case.
>
> /Jorgen
>

Well write(2) and send(2) both return the number
of octets written, or -1.

What's the question here ?
Of course both might wait forever if the socket is
blocked for some reason.

Set O_NONBLOCK upon creating the socket and see what happens :-)
Another possibility is using a 'timed write', which could get you out
of waiting for(;;) or while(1) :-P

	-rasp
PS: The OP should apply some sort of checksumming if he
     really thinks he's losing some bytes....


-- 
RTMPDump & ffmpeg are your friends..
     -icke

[toc] | [prev] | [next] | [standalone]

#1087

On Thu, 2012-02-23, Ralph Spitzner wrote:
> Jorgen Grahn wrote:
> [...]
>> Noone here has claimed that you don't need to check for partial writes,
>> but it could be very useful to know (e.g. for debugging purposes) how
>> Linux behaves in this case.
>>
>> /Jorgen
>>
>
> Well write(2) and send(2) both return the number
> of octets written, or -1.
>
> What's the question here ?

The question is "under which situations does write() on a TCP socket
return a partial write?". ("Partial write" meaning that some, but not
all of the data you fed it got sent.)

> PS: The OP should apply some sort of checksumming if he
>      really thinks he's losing some bytes....

Why? TCP doesn't lose bytes[1]; only programming errors cause that.

/Jorgen

[1] Except the rare cases where packets get corrupted in transit
    in such a way that the checksum is still valid.

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#1088

Jorgen Grahn <grahn+nntp@snipabacken.se> wrote:
> The question is "under which situations does write() on a TCP socket
> return a partial write?". ("Partial write" meaning that some, but not
> all of the data you fed it got sent.)

Empirically, I can confirm that partial writes can happen when the
network connection is broken.

I started with a socket connection to a remote host, and then had a loop
that wrote approximately 500 bytes per iteration. I used usleep() to slow
the loop a little. I had nc listening on the other end. I then manipulated
the firewall to simulate a network glitch - variously experimenting with
REJECT and DROP.

With the data session established but then blocked, I would stop the
netcat (nc) and then unblock the data path. The result was an immediate
failure of write. 

For those who want to repeat my experiment, you can get my client from
http://www.roaima.co.uk/stuff/2012/02/23/client.c, and the commands I
ran were these:

  nc -l -vvv \* 50194					# Listener

  make client && ./client				# Client

  iptables -I OUTPUT -p tcp --dport 50194 -j REJECT	# Client block
  iptables -D OUTPUT -p tcp --dport 50194 -j REJECT	# Client unblock

Chris

[toc] | [prev] | [next] | [standalone]

#1097

Chris Davies <chris-usenet@roaima.co.uk> wrote:
> Empirically, I can confirm that partial writes can happen when the
> network connection is broken.

> I started with a socket connection to a remote host, and then had a loop
> that wrote approximately 500 bytes per iteration. I used usleep() to slow
> the loop a little. I had nc listening on the other end. I then manipulated
> the firewall to simulate a network glitch - variously experimenting with
> REJECT and DROP.

> With the data session established but then blocked, I would stop the
> netcat (nc) and then unblock the data path. The result was an immediate
> failure of write. 

Failure exactly how - return status of less than the send size, or
return status of -1 and errno set?  I wouldn't call the latter a
partial write.

> For those who want to repeat my experiment, you can get my client from
> http://www.roaima.co.uk/stuff/2012/02/23/client.c, and the commands I
> ran were these:

>   nc -l -vvv \* 50194                                   # Listener

>   make client && ./client                               # Client

>   iptables -I OUTPUT -p tcp --dport 50194 -j REJECT     # Client block
>   iptables -D OUTPUT -p tcp --dport 50194 -j REJECT     # Client unblock

In iptables what is the difference between REJECT and DROP?  Doesn't
the former actively send a TCP Reset segment?

rick jones
-- 
denial, anger, bargaining, depression, acceptance, rebirth...
                                     where do you want to be today?
these opinions are mine, all mine; HP might not want them anyway... :)
feel free to post, OR email to rick.jones2 in hp.com  but NOT BOTH...

[toc] | [prev] | [next] | [standalone]

#1098

Rick Jones <rick.jones2@hp.com> wrote:
> Failure exactly how - return status of less than the send size, or
> return status of -1 and errno set?

Good question. I had intended to provide some results to demonstrate
that this really is a partial write we're talking about, but forgot.

  ./client
       0: written 508 bytes of 508 available (OK)
  [...]
     100: written 508 bytes of 508 available (OK)
  [...somewhere around here the network connection is blocked...]
     146: written 508 bytes of 508 available (OK)
     147: written 508 bytes of 508 available (OK)
     148: written 508 bytes of 508 available (OK)
  [...at this point the local buffer fills and the client blocks...]
     149: written 284 bytes of 508 available (TRUNCATED)
  [...program ends...]

Chris

[toc] | [prev] | [next] | [standalone]

#1099

On Fri, 2012-02-24, Rick Jones wrote:
> Chris Davies <chris-usenet@roaima.co.uk> wrote:
...
>> For those who want to repeat my experiment, you can get my client from
>> http://www.roaima.co.uk/stuff/2012/02/23/client.c, and the commands I
>> ran were these:
>
>>   nc -l -vvv \* 50194                                   # Listener
>
>>   make client && ./client                               # Client
>
>>   iptables -I OUTPUT -p tcp --dport 50194 -j REJECT     # Client block
>>   iptables -D OUTPUT -p tcp --dport 50194 -j REJECT     # Client unblock
>
> In iptables what is the difference between REJECT and DROP?  Doesn't
> the former actively send a TCP Reset segment?

No, REJECT sends an ICMP port unreachable. (And I think you know
better than me what that does to TCP; "host unreachable" means another
path may work later, but "port unreachable" in the middle of a session
seems more final.)

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#1111

Jorgen Grahn <grahn+nntp@snipabacken.se> wrote:
> REJECT sends an ICMP port unreachable

And DROP simply discards the packet. I couldn't be bothered to wait for
a TCP timeout, so I used REJECT.

Chris

[toc] | [prev] | [next] | [standalone]

#1112

Hello,

Jorgen Grahn a écrit :
> On Fri, 2012-02-24, Rick Jones wrote:
>> Chris Davies <chris-usenet@roaima.co.uk> wrote:
> ...
>> In iptables what is the difference between REJECT and DROP?  Doesn't
>> the former actively send a TCP Reset segment?
> 
> No, REJECT sends an ICMP port unreachable.

By default. But it as an option to send various other ICMP destination
unreachable error codes, or TCP Reset.

[toc] | [prev] | [next] | [standalone]

#1113

On Sun, 2012-02-26, Pascal Hambourg wrote:
> Jorgen Grahn a écrit :
>> No, REJECT sends an ICMP port unreachable.
>
> By default. But it as an option to send various other ICMP destination
> unreachable error codes,

True; I didn't bother to mention it -- there is a manpage, after all.

> or TCP Reset.

I missed that in the manpage, though. Thanks!

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#1089

Jorgen Grahn wrote:
[...]
> The question is "under which situations does write() on a TCP socket
> return a partial write?". ("Partial write" meaning that some, but not
> all of the data you fed it got sent.)

Well if the connection somehow gets interrupted write will return
a number less that the number of bytes you fed to it.

[....]
> Why? TCP doesn't lose bytes[1]; only programming errors cause that.
>
> /Jorgen
>
> [1] Except the rare cases where packets get corrupted in transit
>      in such a way that the checksum is still valid.
>

Well yes, but the OP _believes_that TCP is doing some evil to his data,
apart from this not being true and not knowing his programming
techniques and/or skills, a checksum might help him :-P

	-rasp


-- 
RTMPDump & ffmpeg are your friends..
     -icke

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.os.linux.networking

csiph-web