Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #11893

Re: Interplatform (interprocess, interlanguage) communication

Show all headers | View raw

On 2/9/2012 10:40 AM, Lew wrote:
> BGB wrote:
>> Lew wrote:
>>> Arne Vajhøj wrote:
>>>> BGB wrote:
>>>>> as noted, many people neither use schemas nor any sort of schema
>>>>> validation. in many use-cases, schemas are overly constraining to the
>>>>> ability of using XML to represent free-form data, or using them
>>>>> otherwise would offer little particular advantage.
>>>>
>>>> xsd:any do provide some flexibility in schemas.
>>>>
>>>>> say, if one is using XML for compiler ASTs or similar (say, the XML is
>>>>> used to represent a just-parsed glob of source-code), do they really
>>>>> need any sort of schema?
>>>>
>>>> I would expect syntax trees to follow certain rules and not be free
>>>> form.
>>>
>>> In one breath we're singing the praises of binary formats, in the next we
>>> complain that XML isn't sufficiently flexible.
>>>
>>
>> it is not like one can't have both:
>
> XML is much easier to modify and maintain when flexibility is a requirement.
>
>> have a format which is at the same time is a compressed binary format,
>> and can also retain the full flexibility of representing free-form XML
>> semantics, ideally without a major drop in compactness (this happens
>> with WBXML, and IIRC should also happen with EXI about as soon as one
>> starts encoding nodes which lie outside the schema).
>>
>> this is partly why I was advocating a sort of pattern-building adaptive
>> format: it can build the functional analogue of a schema as it encodes
>
> That rather defeats the purpose of having a schema.
>
> A schema is a contract that the various processes or other stakeholders use to
> guarantee correctness of the XML and guide processing. If you develop it /ad
> hoc/ you lose that contract.
>

there is no schema in use in this case, however...

hence, why the format would ideally need to be adaptive:
so one doesn't need a schema for it to work correctly, and also so the 
use of free-form data will not hinder compression.


>> the data, and likewise does not depend on a schema to properly decode
>> the document. it is mostly a matter of having the format predict when it
>> doesn't need to specify tag and attribute names (it is otherwise similar
>> to a traditional data-compressor).
>
> I'm sure that's very clever, but it defeats the purpose of XML schema.
>

which is not being used in this case to begin with...


>> this is functionally similar to the sliding-window as used in deflate
>> and LZMA (7zip) and similar (in contrast to codebook-based data
>> compressors). functionally, it would have a little more in common with
>> LZW+MTF than with LZ77 though.
>
> ... and now you're off on some weird tangential topic.
>

data compression...


>> granted, potentially a binary format could incorporate both support for
>> schemas and the use of adaptive compression.
>>
>>
>> is XML really the text, or is it actually the structure?
>
> Huh?
>

as I see it, XML exists at 2 levels:
as a textual syntax;
as a semantic (a tree of tags with attributes and so on) structure, 
which can be expressed via the textual syntax.

conceptually, the semantic structure of XML is more or less equivalent 
to a tree of DOM nodes.


>> I had operated under the premise that it was the data-structure (tags,
>> attributes, namespaces, ...), which allows for pretty much anything
>> which can faithfully encode the structure (without imposing too many
>> arbitrary restrictions).
>
> Huh?
>
> XML is a formal specification for structured documents that is devoid of
> semantics.
>

the existence of the tags and attributes is the semantics...


>>> "Do they really need any sort of schema?" with XML is usually a "yes".
>>>
>>> But only if you're interested in clear, unambiguous, readily-parsable and
>>> maintainable XML document formats.
>>>
>>
>> fair enough, I have mostly been using it "internally", and as noted, for
>> some of my file-formats, I had used a custom binary coded variant
>> (roughly similar to WBXML, but generally more compact and supporting
>> more features, such as namespaces and similar, which I had called SBXE).
>> it didn't make use of schemas, and worked by simply encoding the tag
>> structure into the file, and using basic contextual modeling strategies.
>
> Bully.  Good on ye.
>
>> it also compared favorably with XML+GZ in my tests (which IIRC was also
>> generally smaller than WBXML). remotely possible would also be XML+BZip2
>> or XML+LZMA.
>
> Compared "favorably" according to what criteria?
>

smaller output size.

I was testing each scenario, and comparing the output sizes.


>> I had considered the possibility of a more "advanced" format (with more
>> advanced predictive modeling), but didn't bother (couldn't see much
>> point at the time of trying to shave off more bytes at the time, as it
>> was already working fairly well).
>
> Huh?
>
>>> People often excoriate the supposed verbosity of XML as though it were the only
>>> criterion to measure utility.
>>>
>>
>> well, a lot depends...
>>
>> for disk files, really, who cares?...
>> for a link where a several kB message might only take maybe 250-500ms
>> and is at typical "user-interaction" speeds (say, part of a generic "web
>> app"), likewise, who cares?...
>>
>>
>> it may matter a little more in a 3D interactive world where everything
>> going on in the visible scene has to get through at a 10Hz or 24Hz
>> clock-tick, and if the connection bogs down the user will be rather
>> annoyed (as their game world has essentially stalled).
>
> And that's a use case for XML how, exactly?
>
> Saying "XML is bad because it doesn't keep bananas ripe" would be equally
> relevant.
>

because one can use XML as the client/server messaging protocol, say, in 
place of "well, I am going to send message tags as raw bytes and have 
each followed by some values...".


>> one may have to make due with about 16-24kB/s (or maybe less) to better
>> ensure a good user experience (little is to say that the user has a
>> perfect internet connection either).
>>
>> so, some sort of compression may be needed in this case.
>> (yes, XML+GZ would probably be sufficient).
>
> Back in the universe where we're discussing XML's suitability, please.
>
>> if it were dial-up, probably no one would even consider using XML for
>> the network protocol in a 3D game.
>
> Oh, you're talking about inter-node communication in a distributed game. Thanks
> for finally making that clear. XML would be just fine as a transmission protocol for such a thing. I'm not saying ideal, but just fine.
>
> If you're talking about network protocols you certainly are not talking about
> frame-by-frame transmission of data with reply at 10 Hz, no matter what the
> protocol, so your entire argument against XML for such a thing is moot.
>

both ends transmit concurrently and asynchronously, but one needs to get 
the messages through at roughly 10Hz for things to remain playable 
(otherwise, real-time interactivity starts to fall apart).


so, the server sends a 10Hz stream of updates to the client;
the client sends a 10Hz stream of movement impulses back to the server;
...

granted, ping time is a bit of an issue, as what actions the player is 
trying to do, and what is going on at the servers' end, will invariably 
drift somewhat.

typically, things like linear extrapolation and similar are used to try 
to make up for sub-optimal ping (ideally, one tries to hide the results 
of the ping time, where possible).


but, it does all work, as evidenced by the prevalence of online gaming 
and similar.


I doubt anyone uses ping/pong or request/response based protocols for 
this, as the ping times over the internet would likely render something 
like this unusable (one would probably need to be on a LAN or something...).


>>> There is no inherent advantage of a LISP/list-like format over any other, nor vice versa; it's all accordin'. If the convention is agreeable to all parties,
>>> it will work. If all projects were one-off and isolated from the larger world,
>>> we'd never need to adhere to a standard. If we don't mind inventing our own
>>> tools for anything, we'd never have to adopt a standard with extensive tools
>>> support.
>>>
>>
>> it is possible, it all depends.
>>
>> a swaying factor in my last choice was the effort tradeoff of writing
>> the code (because working with DOM is kind of a pain...). IIRC, I may
>
> Huh? again. There's very little effort in writing XML code, whether DOM, JAXB,
> SAX or StAX, given the wide availability of libraries to do so.
>

well, the issue is that one needs an method call every time they want to 
fetch an attribute's value or look up a node, which is a little more 
painful than it could be.

it isn't like major pain or anything, but it does tend to result in 
slightly longer and more awkward code.

with lists, traditionally there are operations like:
"cadr", "caddr", "cadddr", ..., "caadr", ..., and so on, which make it a 
bit easier (and more compact) to reference particular items within a 
list (since the operations essentially encode where to fetch the item from).

OTOH, with DOM one might end up with a chain a several statements to 
access an item, and yet more statements if one is checking for null, so 
it is just a little more awkward and verbose to work with, but granted 
it is not like the difference is all that huge (IIRC, the performance 
concern may well have been a bigger factor).


>> have also been worrying about performance (mostly passing around lots of
>> numeric data as ASCII strings, ...).
>
> Based on what measurements?
>

this case was based mostly on speculation that if I am creating piles of 
new strings to pass numbers around, and I am passing a scene-graph 
update on a 10Hz basis, most of which will become garbage immediately 
afterwards, than creating all of those strings could get a little 
expensive (mostly causing the garbage collector to start "doing its 
thing" and reduce performance and similar).

I chose another option (namely lists) which had the option of passing 
the the numbers without allocating any memory on the heap.


>> but, I may eventually need to throw together a basic encoding scheme for
>> this case (a binary encoder for list-based data), that or just reuse an
>> existing data serializer of mine (mostly intended for generic data
>> serialization, which supports lists). it lacks any sort of prediction or
>> context modeling though, and is used in my stuff mostly as a container
>> format for bytecode for my VM and similar.
>>
>>
>>> Where are the *real* costs of a software system?
>>>
>>
>> who knows?...
>
> Anyone who thinks about it realistically.
>
>> probably delivering the best reasonable user experience?...
>
> That's not a cost, that's a goal.
>
>> for a game:
>> reasonably good graphics;
>> reasonably good performance (ideally, consistently over 30fps);
>> hopefully good gameplay, plot, story, ...
>>
>> well, that and "getting everything done" (this is the hard one).
>
> Those aren't costs. Those are goals.
>

not effectively achieving a goal is a cost...


> Clear conclusions require clear reasoning on actual facts with relevance.
>

dunno.


a lot of time one works based on "the feel of the code" or "the feel of 
the problem" or similar (if one "feels" that an option will lead to 
suck, it often does lead to suck). one doesn't necessarily know what the 
reasoning is, one can just follow along where it leads (it can be almost 
like that of a physical sensation or similar, like "what does it feel 
like the code wants to do here?").

also, estimating things based on past experiences and known behaviors 
and "rules of thumb" and so on.

if one knows what something does, one can make an educated guess for 
what it will do in a given situation.

everything else becomes mostly likelihoods and probabilities (like, how 
likely is a good outcome, vs a sucky outcome, ...).


or such...

Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Re: Interplatform (interprocess, interlanguage) communication jebblue <n@n.nnn> - 2012-02-07 12:11 -0600
  Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-07 16:38 -0700
    Re: Interplatform (interprocess, interlanguage) communication Arved Sandstrom <asandstrom3minus1@eastlink.ca> - 2012-02-07 20:26 -0400
      Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-08 01:41 -0700
        Re: Interplatform (interprocess, interlanguage) communication Arved Sandstrom <asandstrom3minus1@eastlink.ca> - 2012-02-08 07:19 -0400
          Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-08 12:07 -0700
            Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-08 21:16 -0500
              Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-08 19:50 -0700
                Re: Interplatform (interprocess, interlanguage) communication Arved Sandstrom <asandstrom3minus1@eastlink.ca> - 2012-02-09 06:24 -0400
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-09 09:15 -0700
                Re: Interplatform (interprocess, interlanguage) communication Arved Sandstrom <asandstrom3minus1@eastlink.ca> - 2012-02-09 18:58 -0400
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-09 16:15 -0700
                Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-09 18:50 -0500
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-09 21:40 -0700
                Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-11 14:47 -0500
                Re: Interplatform (interprocess, interlanguage) communication Lew <lewbloch@gmail.com> - 2012-02-11 12:06 -0800
                Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-11 15:18 -0500
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-11 23:03 -0700
                Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-12 09:27 -0500
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-12 13:33 -0700
                Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-12 15:50 -0500
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-12 14:34 -0700
                Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-09 18:48 -0500
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-09 21:46 -0700
                Re: Interplatform (interprocess, interlanguage) communication Lew <lewbloch@gmail.com> - 2012-02-10 08:51 -0800
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-10 10:43 -0700
                Re: Interplatform (interprocess, interlanguage) communication Lew <lewbloch@gmail.com> - 2012-02-10 13:15 -0800
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-10 14:50 -0700
                Re: Interplatform (interprocess, interlanguage) communication Lew <lewbloch@gmail.com> - 2012-02-10 14:32 -0800
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-10 17:10 -0700
                Re: Interplatform (interprocess, interlanguage) communication Arved Sandstrom <asandstrom3minus1@eastlink.ca> - 2012-02-10 22:08 -0400
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-11 00:49 -0700
                Re: Interplatform (interprocess, interlanguage) communication Arved Sandstrom <asandstrom3minus1@eastlink.ca> - 2012-02-11 14:04 -0400
                Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-11 14:55 -0500
                Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-11 14:52 -0500
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-11 20:06 -0700
                Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-11 22:41 -0500
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-12 00:46 -0700
                Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-12 09:29 -0500
                Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-12 09:31 -0500
                Re: Interplatform (interprocess, interlanguage) communication Martin Gregorie <martin@address-in-sig.invalid> - 2012-02-12 16:02 +0000
                Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-12 11:16 -0500
                Re: Interplatform (interprocess, interlanguage) communication Martin Gregorie <martin@address-in-sig.invalid> - 2012-02-12 22:46 +0000
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-12 11:33 -0700
                Re: Interplatform (interprocess, interlanguage) communication Lew <lewbloch@gmail.com> - 2012-02-11 20:18 -0800
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-12 01:36 -0700
                Re: Interplatform (interprocess, interlanguage) communication Joshua Cranmer <Pidgeot18@verizon.invalid> - 2012-02-12 13:52 -0600
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-12 14:43 -0700
                Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-11 14:49 -0500
                Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-09 18:46 -0500
                Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-09 18:45 -0500
        Re: Interplatform (interprocess, interlanguage) communication Lew <lewbloch@gmail.com> - 2012-02-08 14:02 -0800
          Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-08 18:49 -0700
            Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-08 21:14 -0500
              Re: Interplatform (interprocess, interlanguage) communication Lew <lewbloch@gmail.com> - 2012-02-08 20:07 -0800
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-08 23:29 -0700
                Re: Interplatform (interprocess, interlanguage) communication Lew <lewbloch@gmail.com> - 2012-02-09 09:40 -0800
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-09 17:02 -0700
              Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-08 21:10 -0700
                Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-09 18:54 -0500
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-10 10:25 -0700
                Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-11 14:45 -0500
                Re: Interplatform (interprocess, interlanguage) communication Lew <lewbloch@gmail.com> - 2012-02-11 12:14 -0800
                Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-11 15:20 -0500
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-11 22:20 -0700
                Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-12 09:23 -0500
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-12 12:13 -0700
    Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-07 20:24 -0500
    Re: Interplatform (interprocess, interlanguage) communication Martin Gregorie <martin@address-in-sig.invalid> - 2012-02-08 01:31 +0000
      Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-08 00:55 -0700

csiph-web