Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #21512 > unrolled thread

creating byte[] with subfields

Back to article view | Back to comp.lang.java.programmer

Contents

  creating byte[] with subfields Roedy Green <see_website@mindprod.com.invalid> - 2013-01-18 00:35 -0800
    Re: creating byte[] with subfields Jim Janney <jjanney@shell.xmission.com> - 2013-01-18 08:31 -0700
      Re: creating byte[] with subfields Jim Janney <jjanney@shell.xmission.com> - 2013-01-18 10:29 -0700
    Re: creating byte[] with subfields Nigel Wade <nmw@ion.le.ac.uk> - 2013-01-18 16:23 +0000
    Re: creating byte[] with subfields Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2013-01-18 10:53 -0800
      Re: creating byte[] with subfields Roedy Green <see_website@mindprod.com.invalid> - 2013-01-18 14:23 -0800
        Re: creating byte[] with subfields Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2013-01-18 15:16 -0800
          Re: creating byte[] with subfields Roedy Green <see_website@mindprod.com.invalid> - 2013-01-18 23:24 -0800
            Re: creating byte[] with subfields Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2013-01-19 21:46 -0800
              Re: creating byte[] with subfields Joshua Cranmer <Pidgeot18@verizon.invalid> - 2013-01-20 11:01 -0600
                Re: creating byte[] with subfields Arne Vajhøj <arne@vajhoej.dk> - 2013-01-20 12:54 -0500
              Re: creating byte[] with subfields Arne Vajhøj <arne@vajhoej.dk> - 2013-01-20 12:51 -0500
          Re: creating byte[] with subfields Arne Vajhøj <arne@vajhoej.dk> - 2013-01-19 13:34 -0500
    Re: creating byte[] with subfields Arne Vajhøj <arne@vajhoej.dk> - 2013-01-18 20:33 -0500
      Re: creating byte[] with subfields Arne Vajhøj <arne@vajhoej.dk> - 2013-01-18 20:48 -0500

#21512 — creating byte[] with subfields

I often have to construct byte arrays with binary fields, or read
them.

I use a ByteArrayOutputStream and a DataOutputStream to compose them.
I wondered if there is a simpler way, one that does not require
computing each byte individually with shifts.
-- 
Roedy Green Canadian Mind Products http://mindprod.com
The first 90% of the code accounts for the first 90% of the development time.
The remaining 10% of the code accounts for the other 90% of the development 
time. 
~ Tom Cargill  Ninety-ninety Law 

[toc] | [next] | [standalone]

#21518

Roedy Green <see_website@mindprod.com.invalid> writes:

> I often have to construct byte arrays with binary fields, or read
> them.
>
> I use a ByteArrayOutputStream and a DataOutputStream to compose them.
> I wondered if there is a simpler way, one that does not require
> computing each byte individually with shifts.

In C, sure.  Java by design doesn't allow those kinds of tricks.

-- 
Jim Janney

[toc] | [prev] | [next] | [standalone]

#21529

Jim Janney <jjanney@shell.xmission.com> writes:

> Roedy Green <see_website@mindprod.com.invalid> writes:
>
>> I often have to construct byte arrays with binary fields, or read
>> them.
>>
>> I use a ByteArrayOutputStream and a DataOutputStream to compose them.
>> I wondered if there is a simpler way, one that does not require
>> computing each byte individually with shifts.
>
> In C, sure.  Java by design doesn't allow those kinds of tricks.

The hotspot compiler might notice that you're shifting and masking by 8
bits and replace it with direct byte addressing.  That seems like a
straightforward optimization.  OTOH, the speed difference is probably
too tiny to notice anyway.

-- 
Jim Janney

[toc] | [prev] | [next] | [standalone]

#21521

On 18/01/13 08:35, Roedy Green wrote:
> I often have to construct byte arrays with binary fields, or read
> them.
>
> I use a ByteArrayOutputStream and a DataOutputStream to compose them.
> I wondered if there is a simpler way, one that does not require
> computing each byte individually with shifts.
>

BitSet, and its toByteArray() method may do what you require.

I've not tried it myself. I still do it by shift/mask.

-- 
Nigel Wade

[toc] | [prev] | [next] | [standalone]

#21535

On 1/18/13 12:35 AM, Roedy Green wrote:
> I often have to construct byte arrays with binary fields, or read
> them.
>
> I use a ByteArrayOutputStream and a DataOutputStream to compose them.
> I wondered if there is a simpler way, one that does not require
> computing each byte individually with shifts.
>
How about using a BitSet instead?

Or, avoid this level of low-level work?

Or, create a helper wrapper class. BitStream, where you can write a 
number of bits, rather than whole bytes.

[toc] | [prev] | [next] | [standalone]

#21547

On Fri, 18 Jan 2013 10:53:34 -0800, Daniel Pitts
<newsgroup.nospam@virtualinfinity.net> wrote, quoted or indirectly
quoted someone who said :

>How about using a BitSet instead?

I don't often have to do anything finer than the byte.
On project I did recently involved XOR encryption, where you need
everything in bytes to be able to xor.

Last night I was composing/decomposing a DataPacket for
SingleInstance.

I did one to handle Network Time Protocol.
-- 
Roedy Green Canadian Mind Products http://mindprod.com
The first 90% of the code accounts for the first 90% of the development time.
The remaining 10% of the code accounts for the other 90% of the development 
time. 
~ Tom Cargill  Ninety-ninety Law 

[toc] | [prev] | [next] | [standalone]

#21553

On 1/18/13 2:23 PM, Roedy Green wrote:
> On Fri, 18 Jan 2013 10:53:34 -0800, Daniel Pitts
> <newsgroup.nospam@virtualinfinity.net> wrote, quoted or indirectly
> quoted someone who said :
>
>> How about using a BitSet instead?
>
> I don't often have to do anything finer than the byte.
> On project I did recently involved XOR encryption, where you need
> everything in bytes to be able to xor.
>
> Last night I was composing/decomposing a DataPacket for
> SingleInstance.
>
> I did one to handle Network Time Protocol.
>
FWIW, XOR encryption is more like XOR obfuscation. It doesn't actually 
encrypt, at least not the naive approach to it.

[toc] | [prev] | [next] | [standalone]

#21570

On Fri, 18 Jan 2013 15:16:45 -0800, Daniel Pitts
<newsgroup.nospam@virtualinfinity.net> wrote, quoted or indirectly
quoted someone who said :

>FWIW, XOR encryption is more like XOR obfuscation. It doesn't actually 
>encrypt, at least not the naive approach to it.

It depends. I just finished a project which I will be permitted to
release into the public domain in a year which uses one time pads,
which XOR messages with strings of truly random bits.

The only way you can crack it is to crack the machine at either end or
intercept the key transfer.  You can't do it just by studying
messages.
-- 
Roedy Green Canadian Mind Products http://mindprod.com
The first 90% of the code accounts for the first 90% of the development time.
The remaining 10% of the code accounts for the other 90% of the development 
time. 
~ Tom Cargill  Ninety-ninety Law 

[toc] | [prev] | [next] | [standalone]

#21579

On 1/18/13 11:24 PM, Roedy Green wrote:
> On Fri, 18 Jan 2013 15:16:45 -0800, Daniel Pitts
> <newsgroup.nospam@virtualinfinity.net> wrote, quoted or indirectly
> quoted someone who said :
>
>> FWIW, XOR encryption is more like XOR obfuscation. It doesn't actually
>> encrypt, at least not the naive approach to it.
>
> It depends. I just finished a project which I will be permitted to
> release into the public domain in a year which uses one time pads,
> which XOR messages with strings of truly random bits.
>
> The only way you can crack it is to crack the machine at either end or
> intercept the key transfer.  You can't do it just by studying
> messages.
>
It really does depend.

If the plain-text are formed poorly (eg. plain ascii text, or contains 
any well known substrings), and the key is sufficiently small (needs to 
be repeated several times) one could easily figure out the key, or at 
least part of it. That could be enough to turn it into a simple word 
game, which will eventually reveal the entire plain-text.

[toc] | [prev] | [next] | [standalone]

#21585

On 1/19/2013 11:46 PM, Daniel Pitts wrote:
> On 1/18/13 11:24 PM, Roedy Green wrote:
>> On Fri, 18 Jan 2013 15:16:45 -0800, Daniel Pitts
>> <newsgroup.nospam@virtualinfinity.net> wrote, quoted or indirectly
>> quoted someone who said :
>>
>>> FWIW, XOR encryption is more like XOR obfuscation. It doesn't actually
>>> encrypt, at least not the naive approach to it.
>>
>> It depends. I just finished a project which I will be permitted to
>> release into the public domain in a year which uses one time pads,
>> which XOR messages with strings of truly random bits.
>>
>> The only way you can crack it is to crack the machine at either end or
>> intercept the key transfer.  You can't do it just by studying
>> messages.
>>
> It really does depend.

No, it doesn't. A one-time pad is provably 100% perfectly secure, and is 
the only cryptographic method to be so proven. The requirements is that 
you have a string of random bits as long as the message you send and you 
never reuse those random bits ever again.

-- 
Beware of bugs in the above code; I have only proved it correct, not 
tried it. -- Donald E. Knuth

[toc] | [prev] | [next] | [standalone]

#21587

On 1/20/2013 12:01 PM, Joshua Cranmer wrote:
> On 1/19/2013 11:46 PM, Daniel Pitts wrote:
>> On 1/18/13 11:24 PM, Roedy Green wrote:
>>> On Fri, 18 Jan 2013 15:16:45 -0800, Daniel Pitts
>>> <newsgroup.nospam@virtualinfinity.net> wrote, quoted or indirectly
>>> quoted someone who said :
>>>
>>>> FWIW, XOR encryption is more like XOR obfuscation. It doesn't actually
>>>> encrypt, at least not the naive approach to it.
>>>
>>> It depends. I just finished a project which I will be permitted to
>>> release into the public domain in a year which uses one time pads,
>>> which XOR messages with strings of truly random bits.
>>>
>>> The only way you can crack it is to crack the machine at either end or
>>> intercept the key transfer.  You can't do it just by studying
>>> messages.
>>>
>> It really does depend.
>
> No, it doesn't. A one-time pad is provably 100% perfectly secure, and is
> the only cryptographic method to be so proven. The requirements is that
> you have a string of random bits as long as the message you send and you
> never reuse those random bits ever again.

And there is a good reason why it is not called two times pad.

In the late forties the Russians had a shortage of one time pads
and started using twice - once for military stuff and one for
civilian stuff. In the sixties the British and Americans discovered
that and started cracking it.

Arne

[toc] | [prev] | [next] | [standalone]

#21586

On 1/20/2013 12:46 AM, Daniel Pitts wrote:
> On 1/18/13 11:24 PM, Roedy Green wrote:
>> On Fri, 18 Jan 2013 15:16:45 -0800, Daniel Pitts
>> <newsgroup.nospam@virtualinfinity.net> wrote, quoted or indirectly
>> quoted someone who said :
>>
>>> FWIW, XOR encryption is more like XOR obfuscation. It doesn't actually
>>> encrypt, at least not the naive approach to it.
>>
>> It depends. I just finished a project which I will be permitted to
>> release into the public domain in a year which uses one time pads,
>> which XOR messages with strings of truly random bits.
>>
>> The only way you can crack it is to crack the machine at either end or
>> intercept the key transfer.  You can't do it just by studying
>> messages.
>>
> It really does depend.
>
> If the plain-text are formed poorly (eg. plain ascii text, or contains
> any well known substrings), and the key is sufficiently small (needs to
> be repeated several times)

A one time pad by definition has a key size equal to message
size so the key does not repeat.

Arne

[toc] | [prev] | [next] | [standalone]

#21576

On 1/18/2013 6:16 PM, Daniel Pitts wrote:
> FWIW, XOR encryption is more like XOR obfuscation. It doesn't actually
> encrypt, at least not the naive approach to it.

Not today. 200 years ago it would (1 byte XOR is really
a special case of 1 replacement alphabet and N byte XOR
of N replacements alphabets).

Arne

[toc] | [prev] | [next] | [standalone]

#21564

On 1/18/2013 3:35 AM, Roedy Green wrote:
> I often have to construct byte arrays with binary fields, or read
> them.
>
> I use a ByteArrayOutputStream and a DataOutputStream to compose them.
> I wondered if there is a simpler way, one that does not require
> computing each byte individually with shifts.

A DataOutputStream on top of a ByteArrayOutputStream is the
old Java way of doing it. And if you do not need to manipulate bits
and you are happy with big endian then you should not to use shifts.

The new Java way is ByteBuffer which support both big and little
endian, so you only need shift for bit manipulation.

Those are both pretty low level. If you need something more
high level, then you will need to look outside of standard
Java API.

Arne

PS: I actually have something on the shelf for that.

[toc] | [prev] | [next] | [standalone]

#21565

On 1/18/2013 8:33 PM, Arne Vajhøj wrote:
> On 1/18/2013 3:35 AM, Roedy Green wrote:
>> I often have to construct byte arrays with binary fields, or read
>> them.
>>
>> I use a ByteArrayOutputStream and a DataOutputStream to compose them.
>> I wondered if there is a simpler way, one that does not require
>> computing each byte individually with shifts.
>
> A DataOutputStream on top of a ByteArrayOutputStream is the
> old Java way of doing it. And if you do not need to manipulate bits
> and you are happy with big endian then you should not to use shifts.
>
> The new Java way is ByteBuffer which support both big and little
> endian, so you only need shift for bit manipulation.
>
> Those are both pretty low level. If you need something more
> high level, then you will need to look outside of standard
> Java API.

> PS: I actually have something on the shelf for that.

Data class:

import dk.vajhoej.record.Endian;
import dk.vajhoej.record.FieldType;
import dk.vajhoej.record.Struct;
import dk.vajhoej.record.StructField;

@Struct(endianess=Endian.LITTLE)
public class Data {
	@StructField(n=0,type=FieldType.BIT,length=4)
	private int bf1;
     @StructField(n=1,type=FieldType.BIT,length=4)
	private int bf2;
     @StructField(n=2,type=FieldType.INT4)
     private int iv;
 
@StructField(n=3,type=FieldType.VARSTR,prefixlength=1,encoding="ISO-8859-1")
     private String sv;
     public Data() {
     	this(0, 0, 0, "");
     }
     public Data(int bf1, int bf2, int iv, String sv) {
		this.bf1 = bf1;
		this.bf2 = bf2;
		this.iv = iv;
		this.sv = sv;
	}
	public int getBf1() {
		return bf1;
	}
	public void setBf1(int bf1) {
		this.bf1 = bf1;
	}
	public int getBf2() {
		return bf2;
	}
	public void setBf2(int bf2) {
		this.bf2 = bf2;
	}
	public int getIv() {
		return iv;
	}
	public void setIv(int iv) {
		this.iv = iv;
	}
	public String getSv() {
		return sv;
	}
	public void setSv(String sv) {
		this.sv = sv;
	}
}

Test program:

import dk.vajhoej.record.RecordException;
import dk.vajhoej.record.StructReader;
import dk.vajhoej.record.StructWriter;

public class RW {
	public static void main(String[] args) throws RecordException {
		Data o1 = new Data(1, 2, 3, "ABC");
		StructWriter sw = new StructWriter();
		sw.write(o1);
		byte[] ba = sw.getBytes();
		for(byte b1 : ba) {
			System.out.printf(" %02X" , b1);
		}
		System.out.println();
		StructReader sr = new StructReader(ba);
		Data o2 = sr.read(Data.class);
		System.out.println(o2.getBf1() + " " + o2.getBf2() + " " + o2.getIv() 
+ " " + o2.getSv());
	}
}

Output:

  12 03 00 00 00 03 41 42 43
1 2 3 ABC

Arne

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.java.programmer

csiph-web