Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #8489 > unrolled thread

Buffered reading seems to corrupt data stream

Back to article view | Back to comp.lang.java.programmer

Contents

  Buffered reading seems to corrupt data stream "Qu0ll" <Qu0llSixFour@gmail.com> - 2011-10-04 01:58 +1100
    Re: Buffered reading seems to corrupt data stream "Qu0ll" <Qu0llSixFour@gmail.com> - 2011-10-04 02:26 +1100
      Re: Buffered reading seems to corrupt data stream Ronny Schütz <usenet.r96@gishpuppy.com> - 2011-10-03 17:54 +0200
        Re: Buffered reading seems to corrupt data stream "Qu0ll" <Qu0llSixFour@gmail.com> - 2011-10-04 07:07 +1100
          Re: Buffered reading seems to corrupt data stream Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2011-10-03 13:18 -0700
        Re: Buffered reading seems to corrupt data stream "Qu0ll" <Qu0llSixFour@gmail.com> - 2011-10-04 10:54 +1100
      Re: Buffered reading seems to corrupt data stream Roedy Green <see_website@mindprod.com.invalid> - 2011-10-03 11:46 -0700
      Re: Buffered reading seems to corrupt data stream Silvio Bierman <silvio@moc.com> - 2011-11-07 11:00 +0100
    Re: Buffered reading seems to corrupt data stream Peter Duniho <NpOeStPeAdM@NnOwSlPiAnMk.com> - 2011-10-03 08:44 -0700
    Re: Buffered reading seems to corrupt data stream markspace <-@.> - 2011-10-03 11:33 -0700
      Re: Buffered reading seems to corrupt data stream Lew <lewbloch@gmail.com> - 2011-10-04 08:48 -0700
        Re: Buffered reading seems to corrupt data stream Peter Duniho <NpOeStPeAdM@NnOwSlPiAnMk.com> - 2011-10-04 09:07 -0700
          Re: Buffered reading seems to corrupt data stream Lew <lewbloch@gmail.com> - 2011-10-04 12:06 -0700
          Re: Buffered reading seems to corrupt data stream Arne Vajhøj <arne@vajhoej.dk> - 2011-11-06 18:23 -0500
            Re: Buffered reading seems to corrupt data stream Peter Duniho <NpOeStPeAdM@NnOwSlPiAnMk.com> - 2011-11-06 16:03 -0800
              Re: Buffered reading seems to corrupt data stream Arne Vajhøj <arne@vajhoej.dk> - 2011-11-06 19:46 -0500
                Re: Buffered reading seems to corrupt data stream Peter Duniho <NpOeStPeAdM@NnOwSlPiAnMk.com> - 2011-11-06 17:35 -0800
                  Re: Buffered reading seems to corrupt data stream Arne Vajhøj <arne@vajhoej.dk> - 2011-11-06 21:16 -0500
                    Re: Buffered reading seems to corrupt data stream Peter Duniho <NpOeStPeAdM@NnOwSlPiAnMk.com> - 2011-11-06 18:30 -0800
                      Re: Buffered reading seems to corrupt data stream Arne Vajhøj <arne@vajhoej.dk> - 2011-11-06 21:50 -0500

#8489 — Buffered reading seems to corrupt data stream

I am trying to repeatedly send a byte array from a server to a client where 
both the writing and the reading is done in a buffered manner with the 
following code on the server:

DataOutputStream dos = new DataOutputStream(os);
dos.writeInt(bytes.length);
dos.write(bytes);
dos.flush();

and this on the client:

System.out.println("Reading size...");
final DataInputStream dis = new DataInputStream(new 
BufferedInputStream(is));
final int size = dis.readInt();
final byte[] bytes = new byte[size];
System.out.println("Reading " + size + " bytes...");
dis.readFully(bytes);

The problem is that for the second or third read, the size variable is 
coming back as garbage (extremely high or even negative value) and the 
reading of the byte array blocks as it waits for a large amount of data to 
be received or crashes with the invalid negative size even though the 
previous size value and the previous bytes themselves are correct.  However, 
if I remove the BufferedInputStream wrapping of stream, it works perfectly.

Why would that be?

--
And loving it,

-Qu0ll (Rare, not extinct)
_________________________________________________
Qu0llSixFour@gmail.com
[Replace the "SixFour" with numbers to email me] 

[toc] | [next] | [standalone]

#8490

"Qu0ll"  wrote in message 
news:y4Cdne-uvN4kUBTTnZ2dnUVZ_hWdnZ2d@westnet.com.au...

> I am trying to repeatedly send a byte array from a server to a client 
> where both the writing and the reading is done in a buffered manner with 
> the following code on the server:
>
> DataOutputStream dos = new DataOutputStream(os);
> dos.writeInt(bytes.length);
> dos.write(bytes);
> dos.flush();
>
> and this on the client:
>
> System.out.println("Reading size...");
> final DataInputStream dis = new DataInputStream(new 
> BufferedInputStream(is));
> final int size = dis.readInt();
> final byte[] bytes = new byte[size];
> System.out.println("Reading " + size + " bytes...");
> dis.readFully(bytes);
>
> The problem is that for the second or third read, the size variable is 
> coming back as garbage (extremely high or even negative value) and the 
> reading of the byte array blocks as it waits for a large amount of data to 
> be received or crashes with the invalid negative size even though the 
> previous size value and the previous bytes themselves are correct. 
> However, if I remove the BufferedInputStream wrapping of stream, it works 
> perfectly.
>
> Why would that be?

I forgot to mention that there is only *one* input stream (is) and output 
stream (os) being used in to the above code. Once the connection between 
client and server is established I repeatedly send the fixed byte array data 
(fixed for testing purposes) on the connected stream.

--
And loving it,

-Qu0ll (Rare, not extinct)
_________________________________________________
Qu0llSixFour@gmail.com
[Replace the "SixFour" with numbers to email me] 

[toc] | [prev] | [next] | [standalone]

#8492

>> The problem is that for the second or third read, the size variable is
>> coming back as garbage (extremely high or even negative value) and the
>> reading of the byte array blocks as it waits for a large amount of
>> data to be received or crashes with the invalid negative size even
>> though the previous size value and the previous bytes themselves are
>> correct. However, if I remove the BufferedInputStream wrapping of
>> stream, it works perfectly.
>>
>> Why would that be?
> 
> I forgot to mention that there is only *one* input stream (is) and
> output stream (os) being used in to the above code. Once the connection
> between client and server is established I repeatedly send the fixed
> byte array data (fixed for testing purposes) on the connected stream.

It might be, that the BufferedInputStream used for the first read
already read data belonging to the second message, that will get lost
when you dispose the BufferedInputStream and will be missing from the
next read call. The garbage size read is most likely actually a part of
the payload bytes. Try to synchronize server and client somehow to
verify. Better reuse the complete stream chain.

Ronny

[toc] | [prev] | [next] | [standalone]

#8503

"Ronny Schütz"  wrote in message news:pe2pl8-h72.ln1@groombridge34.de...

> It might be, that the BufferedInputStream used for the first read
> already read data belonging to the second message, that will get lost
> when you dispose the BufferedInputStream and will be missing from the
> next read call. The garbage size read is most likely actually a part of
> the payload bytes. Try to synchronize server and client somehow to
> verify. Better reuse the complete stream chain.

Thanks Ronny.  Could you please explain what you mean by "reuse the complete 
stream chain"?

--
And loving it,

-Qu0ll (Rare, not extinct)
_________________________________________________
Qu0llSixFour@gmail.com
[Replace the "SixFour" with numbers to email me] 

[toc] | [prev] | [next] | [standalone]

#8504

On 10/3/11 1:07 PM, Qu0ll wrote:
> "Ronny Schütz" wrote in message news:pe2pl8-h72.ln1@groombridge34.de...
>
>> It might be, that the BufferedInputStream used for the first read
>> already read data belonging to the second message, that will get lost
>> when you dispose the BufferedInputStream and will be missing from the
>> next read call. The garbage size read is most likely actually a part of
>> the payload bytes. Try to synchronize server and client somehow to
>> verify. Better reuse the complete stream chain.
>
> Thanks Ronny. Could you please explain what you mean by "reuse the
> complete stream chain"?
>
> --
> And loving it,
>
> -Qu0ll (Rare, not extinct)
> _________________________________________________
> Qu0llSixFour@gmail.com
> [Replace the "SixFour" with numbers to email me]
He means specifically not creating a new BufferedInputStream (or any 
stream) each time you are reading it.  You should only have one instance 
of BufferedInputStream, which should be stored along-side the original 
input stream (or socket, or whatever).

[toc] | [prev] | [next] | [standalone]

#8506

"Ronny Schütz"  wrote in message news:pe2pl8-h72.ln1@groombridge34.de...

> It might be, that the BufferedInputStream used for the first read
> already read data belonging to the second message, that will get lost
> when you dispose the BufferedInputStream and will be missing from the
> next read call. The garbage size read is most likely actually a part of
> the payload bytes. Try to synchronize server and client somehow to
> verify. Better reuse the complete stream chain.

Yes, you are right.  By creating only one DataInputStream and reusing it 
each time I have been able to resolve this issue.  Thanks.

--
And loving it,

-Qu0ll (Rare, not extinct)
_________________________________________________
Qu0llSixFour@gmail.com
[Replace the "SixFour" with numbers to email me] 

[toc] | [prev] | [next] | [standalone]

#8502

On Tue, 4 Oct 2011 02:26:45 +1100, "Qu0ll" <Qu0llSixFour@gmail.com>
wrote, quoted or indirectly quoted someone who said :

>I forgot to mention that there is only *one* input stream (is) and output 
>stream (os) being used in to the above code. Once the connection between 
>client and server is established I repeatedly send the fixed byte array data 
>(fixed for testing purposes) on the connected stream.
 
You do know the dataoutput stream is a pure binary stream with counted
Strings.  There should not be any sort of encoding involved.

Have a look at code generated by
http://mindprod.com/applet/fileio.html
to see how it compares with yours. I have never had trouble with
buffered or unbufferd DataOutput.

-- 
Roedy Green Canadian Mind Products
http://mindprod.com
It should not be considered an error when the user starts something
already started or stops something already stopped. This applies
to browsers, services, editors... It is inexcusable to 
punish the user by requiring some elaborate sequence to atone,
e.g. open the task editor, find and kill some processes.

[toc] | [prev] | [next] | [standalone]

#9728

On 10/03/2011 05:26 PM, Qu0ll wrote:
> "Qu0ll" wrote in message
> news:y4Cdne-uvN4kUBTTnZ2dnUVZ_hWdnZ2d@westnet.com.au...
>
>> I am trying to repeatedly send a byte array from a server to a client
>> where both the writing and the reading is done in a buffered manner
>> with the following code on the server:
>>
>> DataOutputStream dos = new DataOutputStream(os);
>> dos.writeInt(bytes.length);
>> dos.write(bytes);
>> dos.flush();
>>
>> and this on the client:
>>
>> System.out.println("Reading size...");
>> final DataInputStream dis = new DataInputStream(new
>> BufferedInputStream(is));
>> final int size = dis.readInt();
>> final byte[] bytes = new byte[size];
>> System.out.println("Reading " + size + " bytes...");
>> dis.readFully(bytes);
>>
>> The problem is that for the second or third read, the size variable is
>> coming back as garbage (extremely high or even negative value) and the
>> reading of the byte array blocks as it waits for a large amount of
>> data to be received or crashes with the invalid negative size even
>> though the previous size value and the previous bytes themselves are
>> correct. However, if I remove the BufferedInputStream wrapping of
>> stream, it works perfectly.
>>
>> Why would that be?
>
> I forgot to mention that there is only *one* input stream (is) and
> output stream (os) being used in to the above code. Once the connection
> between client and server is established I repeatedly send the fixed
> byte array data (fixed for testing purposes) on the connected stream.
>
> --
> And loving it,
>
> -Qu0ll (Rare, not extinct)
> _________________________________________________
> Qu0llSixFour@gmail.com
> [Replace the "SixFour" with numbers to email me]

Then make sure that there is also only one BufferedInputStream on is and 
one BufferedOutputStream on os. If you hold on to is/os than also hold 
on to the buffered wrappers instead of rewrapping them each time.

[toc] | [prev] | [next] | [standalone]

#8491

On 10/3/11 7:58 AM, Qu0ll wrote:
> [...]
> The problem is that for the second or third read, the size variable is
> coming back as garbage (extremely high or even negative value) and the
> reading of the byte array blocks as it waits for a large amount of data
> to be received or crashes with the invalid negative size even though the
> previous size value and the previous bytes themselves are correct.
> However, if I remove the BufferedInputStream wrapping of stream, it
> works perfectly.
>
> Why would that be?

If you don't post a concise-but-complete code example that reliably 
reproduces the problem (i.e. an SSCCE), there's no way to answer that 
question.

The problem you are describing is typical of failure to treat the 
streams as streams, but of course that's the kind of thing that 
DataInputStream is supposed to address for you (and which I believe 
will, as long as you use it correctly…unless perhaps you're using some 
odd-ball Java implementation that might be more buggy than the ones 
you'd usually find on mainstream OSs).

Another way I suppose you could observe the behavior you're describing 
is to be using a datagram socket instead of a stream-oriented socket.

But without a proper code example, there's no way to know what you've 
done wrong.

Pete

[toc] | [prev] | [next] | [standalone]

#8499

On 10/3/2011 7:58 AM, Qu0ll wrote:
> I am trying to repeatedly send a byte array from a server to a client
> where both the writing and the reading is done in a buffered manner with
> the following code on the server:
>
> DataOutputStream dos = new DataOutputStream(os);
> dos.writeInt(bytes.length);
> dos.write(bytes);
> dos.flush();


I got to repeat the request for an SSCCE.  The code above is certainly 
not where the problem is.  The problems is certainly in some other area 
which you haven't shown us.

As a suggestion, scan the input above before you write it.  You'll 
certainly find the same values going in as you are reading back out. 
Then trace the path up the input side to determine what is actually 
writing that data in the the array "bytes".

[toc] | [prev] | [next] | [standalone]

#8528

markspace wrote:
>  Qu0ll wrote:
>> I am trying to repeatedly send a byte array from a server to a client
>> where both the writing and the reading is done in a buffered manner with
>> the following code on the server:
>>
>> DataOutputStream dos = new DataOutputStream(os);
>> dos.writeInt(bytes.length);
>> dos.write(bytes);
>> dos.flush();
> 
> I got to repeat the request for an SSCCE.  The code above is certainly 
> not where the problem is.  The problems is certainly in some other area 
> which you haven't shown us.
> 
> As a suggestion, scan the input above before you write it.  You'll 
> certainly find the same values going in as you are reading back out. 
> Then trace the path up the input side to determine what is actually 
> writing that data in the the array "bytes".

Just this once the question was answerable (and answered, elsethread) without the SSCCE, but truthfully the respondents had to infer the context that an SSCCE would've made explicit.  They just happened to guess right.  (Well, it was fairly obvious.)

But to the OP and the rest of us, it really is a good idea to package your example as an SSCCE.  Just because we think something is obvious doesn't mean everyone does.

Coincidentally I was just facing the same problem in a project.  I want to independently dip into an input stream without trashing the stream location by buffering.  In my case I did away with the buffering, but for the OP's case here it is much easier to preserve the stream at wider scope than they originally had it.

-- 
Lew

[toc] | [prev] | [next] | [standalone]

#8530

On 10/4/11 8:48 AM, Lew wrote:
> Just this once the question was answerable (and answered, elsethread) without the SSCCE [...]

By that standard, _all_ questions are answerable without an SSCCE.  A 
person attempting an answer can always make a guess, and that guess 
always has a non-zero probability of being correct.

It was not possible from the code posted to _know_ that the OP was 
actually recreating the BufferedInputStream anew each time.  One could 
certainly guess, and of course Ronny did (and did so correctly, as it 
turns out).

But without an SSCCE, we know that the code posted deviates from the 
_actual_ code in some way, and it could just as easily have been that 
part of the deviation included an oversimplification of how 
BufferedInputStream was used.

Other guesses were made as well (i.e. my own), based on the same 
incomplete information, and they were wrong.  The guess that 
BufferedInputStream was being recreated each time could have been wrong 
too, since the code posted was obviously not really the code the OP was 
using.

That guess turned out to be right, and I give all credit to Ronny for 
taking the code at face value and making the guess he did.  He succeeded 
where at least some of the rest of us failed.  But there was no 
assurance at all that would have happened (*)

An SSCCE isn't just a good idea; it's a practical requirement.

Pete


(*)  One of the problems is that we all approach such questions with our 
own biases and experiences.  In my own case, I just assumed that the OP 
had left out the loop structures showing where BufferedInputStream was 
being created outside the loop, because who in their right mind would 
discard previously buffered data before they'd had a chance to read it?

Of course, that turned out to be exactly the problem, and it was my own 
blind spot that prevented me from considering that as a possibility. 
But we all have our blind spots.

[toc] | [prev] | [next] | [standalone]

#8542

Peter Duniho wrote:
> Lew wrote:
>> Just this once the question was answerable (and answered, elsethread) without the SSCCE [...]
> 
> By that standard, _all_ questions are answerable without an SSCCE.  A 
> person attempting an answer can always make a guess, and that guess 
> always has a non-zero probability of being correct.

You're preaching to the choir!

I did point out that the omission only worked "just this once", not recommend it.

And as you point out, the score is less than one for one.

> It was not possible from the code posted to _know_ that the OP was 
> actually recreating the BufferedInputStream anew each time.  One could 
> certainly guess, and of course Ronny did (and did so correctly, as it 
> turns out).
> 
> But without an SSCCE, we know that the code posted deviates from the 
> _actual_ code in some way, and it could just as easily have been that 
> part of the deviation included an oversimplification of how 
> BufferedInputStream was used.
> 
> Other guesses were made as well (i.e. my own), based on the same 
> incomplete information, and they were wrong.  The guess that 
> BufferedInputStream was being recreated each time could have been wrong 
> too, since the code posted was obviously not really the code the OP was 
> using.
> 
> That guess turned out to be right, and I give all credit to Ronny for 
> taking the code at face value and making the guess he did.  He succeeded 
> where at least some of the rest of us failed.  But there was no 
> assurance at all that would have happened (*)
> 
> An SSCCE isn't just a good idea; it's a practical requirement.

Amen, brother!

-- 
Lew

[toc] | [prev] | [next] | [standalone]

#9699

On 10/4/2011 12:07 PM, Peter Duniho wrote:
> It was not possible from the code posted to _know_ that the OP was
> actually recreating the BufferedInputStream anew each time. One could
> certainly guess, and of course Ronny did (and did so correctly, as it
> turns out).
>
> But without an SSCCE, we know that the code posted deviates from the
> _actual_ code in some way, and it could just as easily have been that
> part of the deviation included an oversimplification of how
> BufferedInputStream was used.
>
> Other guesses were made as well (i.e. my own), based on the same
> incomplete information, and they were wrong. The guess that
> BufferedInputStream was being recreated each time could have been wrong
> too, since the code posted was obviously not really the code the OP was
> using.

Really?

The code posted (excl. debug print was):

final DataInputStream dis = new DataInputStream(new 
BufferedInputStream(is));
final int size = dis.readInt();
final byte[] bytes = new byte[size];
dis.readFully(bytes);

It seems rather obvious to me that a new BufferedInputStream
would be created every time this code is called.

Arne

[toc] | [prev] | [next] | [standalone]

#9703

On 11/6/11 3:23 PM, Arne Vajhøj wrote:
> [...]
> It seems rather obvious to me that a new BufferedInputStream
> would be created every time this code is called.

Really?  You resurrected a thread that's over a month old, just to tell 
us what's obvious to you?  Really?

[toc] | [prev] | [next] | [standalone]

#9707

On 11/6/2011 7:03 PM, Peter Duniho wrote:
> On 11/6/11 3:23 PM, Arne Vajhøj wrote:
>> [...]
>> It seems rather obvious to me that a new BufferedInputStream
>> would be created every time this code is called.
>
> Really? You resurrected a thread that's over a month old, just to tell
> us what's obvious to you?

It seemed relevant to point out that the guy that provided
that solution was not guessing, but was a logical conclusion
based on the code posted and that SSCCE requestors just
BS'ed.

Arne

[toc] | [prev] | [next] | [standalone]

#9719

On 11/6/11 4:46 PM, Arne Vajhøj wrote:
> On 11/6/2011 7:03 PM, Peter Duniho wrote:
>> On 11/6/11 3:23 PM, Arne Vajhøj wrote:
>>> [...]
>>> It seems rather obvious to me that a new BufferedInputStream
>>> would be created every time this code is called.
>>
>> Really? You resurrected a thread that's over a month old, just to tell
>> us what's obvious to you?
>
> It seemed relevant to point out that the guy that provided
> that solution was not guessing  [...]

Sure he was.  It was a good guess.  But still a guess.

Since the code that was posted was useless on its own, it's obvious that 
code was not the code the OP actually was using.

Once we know the code posted isn't the code actually being used, it's 
impossible to say for sure what the code actually being used really 
looks like.

Maybe it was, as the posted code did, creating a new object each time. 
Maybe it wasn't.

You can ASSume it was if you like, but some of us prefer to stick with 
what we actually _know_.

Pete

[toc] | [prev] | [next] | [standalone]

#9721

On 11/6/2011 8:35 PM, Peter Duniho wrote:
> On 11/6/11 4:46 PM, Arne Vajhøj wrote:
>> On 11/6/2011 7:03 PM, Peter Duniho wrote:
>>> On 11/6/11 3:23 PM, Arne Vajhøj wrote:
>>>> [...]
>>>> It seems rather obvious to me that a new BufferedInputStream
>>>> would be created every time this code is called.
>>>
>>> Really? You resurrected a thread that's over a month old, just to tell
>>> us what's obvious to you?
>>
>> It seemed relevant to point out that the guy that provided
>> that solution was not guessing [...]
>
> Sure he was. It was a good guess. But still a guess.
>
> Since the code that was posted was useless on its own, it's obvious that
> code was not the code the OP actually was using.
>
> Once we know the code posted isn't the code actually being used, it's
> impossible to say for sure what the code actually being used really
> looks like.
>
> Maybe it was, as the posted code did, creating a new object each time.
> Maybe it wasn't.
>
> You can ASSume it was if you like, but some of us prefer to stick with
> what we actually _know_.

The posted code was incorrect and could give the described symptoms.

Assuming under those circumstances that the incorrectness indeed cause
the symptoms is more than guessing/assuming.

Not pointing that out but wanting a complete program just in case
there were a second incorrectness that coincidentally would cause
the same symptoms is absurd.

Arne

[toc] | [prev] | [next] | [standalone]

#9722

On 11/6/11 6:16 PM, Arne Vajhøj wrote:
> [...]
> Not pointing that out but wanting a complete program just in case
> there were a second incorrectness that coincidentally would cause
> the same symptoms is absurd.

Actually, what's absurd is reviving a thread over a month later just say 
what you _think_ is obvious.

But whatever…seems par for the course for you.  Whatever floats your 
boat…you seem to get a thrill out of pointless critiques of others.  Far 
be it from me to waste any more time trying to convince you otherwise.

[toc] | [prev] | [next] | [standalone]

#9723

On 11/6/2011 9:30 PM, Peter Duniho wrote:
> On 11/6/11 6:16 PM, Arne Vajhøj wrote:
>> [...]
>> Not pointing that out but wanting a complete program just in case
>> there were a second incorrectness that coincidentally would cause
>> the same symptoms is absurd.
>
> Actually, what's absurd is reviving a thread over a month later just say
> what you _think_ is obvious.

Are you telling me that you don't think it is obvious that

final DataInputStream dis = new DataInputStream(new 
BufferedInputStream(is));
final int size = dis.readInt();
final byte[] bytes = new byte[size];
dis.readFully(bytes);

will create a new BufferedInputStream every time the code
is executed??

Arne

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.java.programmer

csiph-web