Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #9460 > unrolled thread

Piggypack Encoding/Decoding on RandomAccessFile

Back to article view | Back to comp.lang.java.programmer

Contents

  Piggypack Encoding/Decoding on RandomAccessFile Jan Burse <janburse@fastmail.fm> - 2011-11-03 19:18 +0100
    Re: Piggypack Encoding/Decoding on RandomAccessFile Joshua Cranmer <Pidgeot18@verizon.invalid> - 2011-11-03 14:32 -0500
      Re: Piggypack Encoding/Decoding on RandomAccessFile Jan Burse <janburse@fastmail.fm> - 2011-11-03 20:50 +0100
        Re: Piggypack Encoding/Decoding on RandomAccessFile markspace <-@.> - 2011-11-03 13:52 -0700
          Re: Piggypack Encoding/Decoding on RandomAccessFile Jan Burse <janburse@fastmail.fm> - 2011-11-03 23:13 +0100
          Re: Piggypack Encoding/Decoding on RandomAccessFile Knute Johnson <nospam@knutejohnson.com> - 2011-11-03 16:17 -0700
        Re: Piggypack Encoding/Decoding on RandomAccessFile Lew <lewbloch@gmail.com> - 2011-11-03 13:58 -0700
        Re: Piggypack Encoding/Decoding on RandomAccessFile Eric Sosman <esosman@ieee-dot-org.invalid> - 2011-11-03 20:40 -0400
          Re: Piggypack Encoding/Decoding on RandomAccessFile Jan Burse <janburse@fastmail.fm> - 2011-11-04 02:28 +0100
            Re: Piggypack Encoding/Decoding on RandomAccessFile Jan Burse <janburse@fastmail.fm> - 2011-11-04 03:06 +0100
              Re: Piggypack Encoding/Decoding on RandomAccessFile Eric Sosman <esosman@ieee-dot-org.invalid> - 2011-11-04 08:05 -0400
                Re: Piggypack Encoding/Decoding on RandomAccessFile Jan Burse <janburse@fastmail.fm> - 2011-11-04 16:12 +0100
          Re: Piggypack Encoding/Decoding on RandomAccessFile rossum <rossum48@coldmail.com> - 2011-11-04 16:54 +0000
    Re: Piggypack Encoding/Decoding on RandomAccessFile Jan Burse <janburse@fastmail.fm> - 2011-11-03 23:24 +0100
    Re: Piggypack Encoding/Decoding on RandomAccessFile Arne Vajhøj <arne@vajhoej.dk> - 2011-11-03 20:14 -0400
    Re: Piggypack Encoding/Decoding on RandomAccessFile Roedy Green <see_website@mindprod.com.invalid> - 2011-11-03 21:56 -0700
      [OT] Conspiracy theories are BS (Was: Piggypack Encoding/Decoding on RandomAccessFile) Lew <lewbloch@gmail.com> - 2011-11-04 10:50 -0700
        Re: [OT] Conspiracy theories are BS (Was: Piggypack Encoding/Decoding on RandomAccessFile) Arne Vajhøj <arne@vajhoej.dk> - 2011-11-04 21:07 -0400
        Re: [OT] Conspiracy theories are BS (Was: Piggypack Encoding/Decoding on RandomAccessFile) Roedy Green <see_website@mindprod.com.invalid> - 2011-11-05 20:21 -0700
        Re: [OT] Conspiracy theories are BS (Was: Piggypack Encoding/Decoding on RandomAccessFile) Roedy Green <see_website@mindprod.com.invalid> - 2011-11-05 20:24 -0700
          Re: [OT] Conspiracy theories are BS (Was: Piggypack Encoding/Decoding on RandomAccessFile) Jan Burse <janburse@fastmail.fm> - 2011-11-06 10:36 +0100
        Re: [OT] Conspiracy theories are BS (Was: Piggypack Encoding/Decoding on RandomAccessFile) Joshua Cranmer <Pidgeot18@verizon.invalid> - 2011-11-06 14:16 -0600
    Re: Piggypack Encoding/Decoding on RandomAccessFile Stanimir Stamenkov <s7an10@netscape.net> - 2011-11-05 16:51 +0200
      Re: Piggypack Encoding/Decoding on RandomAccessFile Jan Burse <janburse@fastmail.fm> - 2011-11-05 16:27 +0100
        Re: Piggypack Encoding/Decoding on RandomAccessFile Lew <lewbloch@gmail.com> - 2011-11-05 10:03 -0700
          Re: Piggypack Encoding/Decoding on RandomAccessFile Jan Burse <janburse@fastmail.fm> - 2011-11-05 19:37 +0100
            Re: Piggypack Encoding/Decoding on RandomAccessFile Lew <lewbloch@gmail.com> - 2011-11-05 13:25 -0700
          Re: Piggypack Encoding/Decoding on RandomAccessFile Jan Burse <janburse@fastmail.fm> - 2011-11-05 19:47 +0100
          Re: Piggypack Encoding/Decoding on RandomAccessFile Jan Burse <janburse@fastmail.fm> - 2011-11-05 19:56 +0100
            Re: Piggypack Encoding/Decoding on RandomAccessFile Lew <lewbloch@gmail.com> - 2011-11-05 13:29 -0700
              Re: Piggypack Encoding/Decoding on RandomAccessFile Jan Burse <janburse@fastmail.fm> - 2011-11-06 10:42 +0100

Page 2 of 2 — ← Prev page 1 [2]

#9643 — Re: [OT] Conspiracy theories are BS (Was: Piggypack Encoding/Decoding on RandomAccessFile)

Roedy Green schrieb:
> On Fri, 4 Nov 2011 10:50:17 -0700 (PDT), Lew<lewbloch@gmail.com>
> wrote, quoted or indirectly quoted someone who said :
>
>> You have not established that bugs are left in deliberately, nor that even if they are it's the "focus on money" that makes them do that.
>>
>> Pure B.S., like most pseudo-socialist rhetoric.
>
>
> Another source of evidence is the release notes of when the vendor
> expects you to pay for an upgrade when the only important difference
> is bug fixes.  The sloppier he is initially, the more money he makes.
>

Yes, windows 3.1 was quite simple. And it took years
and billion$ to get windows 7. I think B. Gates is
the one who openly stated his concept of early
release and let the customer pay. But I would need
to cite reference, otherwise people would call me
paranoid.

But on the other hand it is the only reasonable
way to develop software. To put it quickly into
the wild, to gather bug reports, user feedback
and new requirements. How could you do this when
you keep it in the lab. It would only generate cost,
but when putting in the wild, you can have the
cake and eat it too.

LoL

[toc] | [prev] | [next] | [standalone]

#9665 — Re: [OT] Conspiracy theories are BS (Was: Piggypack Encoding/Decoding on RandomAccessFile)

On 11/4/2011 12:50 PM, Lew wrote:
> Roedy Green wrote in his tag line:
>> Capitalism has spurred the competition that makes CPUs faster and
>> faster each year, but the focus on money makes software
>> manufacturers do some peculiar things like deliberately leaving
>> bugs and deficiencies in the software so they can soak the
>> customers for upgrades later.
>
> Facts not in evidence.
>
> You have not established that bugs are left in deliberately, nor that
> even if they are it's the "focus on money" that makes them do that.

Given some of the limited experience I have with corporate development,
I would reason that it is very likely that most companies will release 
products with known software bugs. However, this is not because they 
want to make money on the upgrades but rather because they need to 
actually release software at some point in time.

I will admit that all of my experience with software is related to 
consumer-targeted software and not enterprise-targeted; however, in 
consumer-targeted software, I am not aware of a single instance where I 
had to pay for a bug-only fix.

-- 
Beware of bugs in the above code; I have only proved it correct, not
tried it. -- Donald E. Knuth

[toc] | [prev] | [next] | [standalone]

#9573

Thu, 03 Nov 2011 19:18:50 +0100, /Jan Burse/:

> How can I go about and encode/decode bytes read
> from a random access file.
>
> I am used to FileInputStream and FileOutputStream,
> but I don't see right now how I could piggypack
> encoding/deconding on a RandomAccessFile:
>
> http://download.oracle.com/javase/7/docs/api/java/io/RandomAccessFile.html
>
> Should I mingle with file descriptor, and get
> associated input and output streams, and then
> move forward?

I think it is o.k. to use the FileDescriptor obtained from a 
RandomAccessFile to create a FileInputStream if you need to read the 
contents of the file through InputStream interface further.

One could also obtain InputStream to the RandomAccessFile using its 
FileChannel:

import java.io.InputStream;
import java.io.RandomAccessFile;
import java.nio.channels.Channels;

   RandomAccessFile raf;
   ...
   InputStream in = Channels.newInputStream(raf.getChannel());

java.nio.channels.Channels also provide:

import java.io.Reader;

   Reader reader = Channels.newReader(raf.getChannel(), "UTF-8");

-- 
Stanimir

[toc] | [prev] | [next] | [standalone]

#9574

Hi,

Thank you for pointing to this API. Wasn't aware.
I just had a look at the API, and also found:

     public static Reader newReader(ReadableByteChannel ch,
                CharsetDecoder dec,
                int minBufferCap)

So via the minBufferCap parameter also problems of
overreadinging the underlying raf can be solved to
some extend, if this is a problem.

Thanks

Bye

Stanimir Stamenkov schrieb:
> Thu, 03 Nov 2011 19:18:50 +0100, /Jan Burse/:
>
>> How can I go about and encode/decode bytes read
>> from a random access file.
>>
>> I am used to FileInputStream and FileOutputStream,
>> but I don't see right now how I could piggypack
>> encoding/deconding on a RandomAccessFile:
>>
>> http://download.oracle.com/javase/7/docs/api/java/io/RandomAccessFile.html
>>
>>
>> Should I mingle with file descriptor, and get
>> associated input and output streams, and then
>> move forward?
>
> I think it is o.k. to use the FileDescriptor obtained from a
> RandomAccessFile to create a FileInputStream if you need to read the
> contents of the file through InputStream interface further.
>
> One could also obtain InputStream to the RandomAccessFile using its
> FileChannel:
>
> import java.io.InputStream;
> import java.io.RandomAccessFile;
> import java.nio.channels.Channels;
>
> RandomAccessFile raf;
> ...
> InputStream in = Channels.newInputStream(raf.getChannel());
>
> java.nio.channels.Channels also provide:
>
> import java.io.Reader;
>
> Reader reader = Channels.newReader(raf.getChannel(), "UTF-8");
>

[toc] | [prev] | [next] | [standalone]

#9578

Jan Burse wrote:
> Thank you for pointing to this API. Wasn't aware.
> I just had a look at the API, and also found:
> 
>      public static Reader newReader(ReadableByteChannel ch,
>                 CharsetDecoder dec,
>                 int minBufferCap)
> 
> So via the minBufferCap parameter also problems of
> overreadinging the underlying raf can be solved to
> some extend, if this is a problem.

This has been a very interesting question and ensuing thread.  I've seen this sort of multiple interaction (conflict) of resource clients a couple of times but most of those cases were unintentional and considered bugs.

One lesson is that such situations are always fraught with peril.

Which is why they pay us the big bucks.  Fraught with peril is the programmer's bread and butter.  But you can't be careless, and the questions addressed in this conversation are key.

Inevitably I wonder if the functional need can be served at a different level of the architecture.  Here's the syllogism:

The synchronization between resource clients, in this case a random access mechanism and a stream mechanism attached to the same file descriptor, will require a unified view that they share, otherwise completely independent views with no interaction between them whatsoever.  

The questions here address how the problem would be solved in breadth at the resource-access level.  The baseline model is two concurrent clients with shared state.

What about a solution in depth with a serial model?

'DataInput' and 'DataOutput' make good first responders to mixed-format binary files. That's why 'RandomAccessFile' implements them.  They're intended for the sort of low-level operations with manual bookkeeping for data location as discussed.  They are the clear choice for direct access to the resource.

If you need sequential stream access to all or part of that data, have sequential clients work with scraps piped through the lower-lying direct access instances rather than fight over the same prey.

-- 
Lew

[toc] | [prev] | [next] | [standalone]

#9584

Lew schrieb:
> This has been a very interesting question and
 > ensuing thread.  I've seen this sort of multiple
 > interaction (conflict) of resource clients a couple
 > of times but most of those cases were unintentional
 > and considered bugs.

If you open the raf with mode "r" no such issues
pertain. If you open the raf with mode "rw" then
of course you have an update problem if multiple
threads access the raf, not to speak what happens
accross processes.

But the channel also offers locks. You can even
parameterize them by ranges. And they are even
synchronized accross processes. It happens that I
have used the locks for byte based access, but I
guess they are also useful when a character reader/
write is placed on the byte stream.

Bye

[toc] | [prev] | [next] | [standalone]

#9599

Jan Burse wrote:
> Lew schrieb:
> > This has been a very interesting question and
>  > ensuing thread.  I've seen this sort of multiple
>  > interaction (conflict) of resource clients a couple
>  > of times but most of those cases were unintentional
>  > and considered bugs.
> 
> If you open the raf with mode "r" no such issues
> pertain. If you open the raf with mode "rw" then
> of course you have an update problem if multiple
> threads access the raf, not to speak what happens
> accross processes.

Not true.  Two clients reading from the same file descriptor interact with each other as you yourself have discussed extensively upthread.

This isn't about update of the file, it's about update of the descriptor.  It's also not about threads at all.

> But the channel also offers locks. You can even
> parameterize them by ranges. And they are even
> synchronized accross processes. It happens that I
> have used the locks for byte based access, but I
> guess they are also useful when a character reader/
> write is placed on the byte stream.

Interesting but not relevant to my point.

-- 
Lew

[toc] | [prev] | [next] | [standalone]

#9586

Lew schrieb:
> The synchronization between resource clients, in
 > this case a random access mechanism and a stream
 > mechanism attached to the same file descriptor, will
 > require a unified view that they share, otherwise
 > completely independent views with no interaction
 > between them whatsoever.

 From what I have learned from Stanimir my estime
of java.nio has raisen. Kind of think now that it
is a *bad ass* nice API.

@Lew: But I agree, being sceptical can be good.

Bye

[toc] | [prev] | [next] | [standalone]

#9588

Lew schrieb:
> If you need sequential stream access to all or part of
 > that data, have sequential clients work with scraps piped
 > through the lower-lying direct access instances rather
 > than fight over the same prey.

What is "scraps piped"? How do you eliminate the common prey
through "scraps piped"? Will you not only create a buffering
layer, but the problem will not be evaded?

Bye

[toc] | [prev] | [next] | [standalone]

#9601

On Saturday, November 5, 2011 11:56:06 AM UTC-7, Jan Burse wrote:
> Lew schrieb:
> > If you need sequential stream access to all or part of
>  > that data, have sequential clients work with scraps piped
>  > through the lower-lying direct access instances rather
>  > than fight over the same prey.
> 
> What is "scraps piped"? How do you eliminate the common prey
> through "scraps piped"? Will you not only create a buffering
> layer, but the problem will not be evaded?

Let's say you want to stream in a UTF-8 entry from the 'RandomAccessFile' (or any other 'DataInput').  If you attach the stream directly to the file descriptor, you have to figure out exactly where to begin somehow or you will not get a validly decoded string.  OTOH, the 'DataInput' already knows where the characters begin, as it reads the source using 'readUTF()'.  It can then make that string available via a 'StringReader' wrapper.

And no, the problem of where to start is not "evaded".  It's just isolated to one place.  The problem of conflicts between the streamed reader and the 'DataInput' is completely evaded.

-- 
Lew

[toc] | [prev] | [next] | [standalone]

#9644

Lew schrieb:
> 'DataInput' already knows where the
 > characters begin, as it reads the source using 'readUTF()'.

But the problem is posed that I want to piggy pack arbitrary
encoders/decoders, not only UTF-8. For example UTF-16 etc..
Or some cyricllic code page which then needs to be mapped to
unicode.

How do think this is solved by readUTF()? And why do you
think that the file positions do not communicate between
the input stream and the raf, although they derive from
the same file descriptor. In fact they do.

Bye

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.java.programmer

csiph-web