Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #11821 > unrolled thread

Re: Interplatform (interprocess, interlanguage) communication

Back to article view | Back to comp.lang.java.programmer

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Interplatform (interprocess, interlanguage) communication jebblue <n@n.nnn> - 2012-02-07 12:11 -0600
    Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-07 16:38 -0700
      Re: Interplatform (interprocess, interlanguage) communication Arved Sandstrom <asandstrom3minus1@eastlink.ca> - 2012-02-07 20:26 -0400
        Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-08 01:41 -0700
          Re: Interplatform (interprocess, interlanguage) communication Arved Sandstrom <asandstrom3minus1@eastlink.ca> - 2012-02-08 07:19 -0400
            Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-08 12:07 -0700
              Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-08 21:16 -0500
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-08 19:50 -0700
                  Re: Interplatform (interprocess, interlanguage) communication Arved Sandstrom <asandstrom3minus1@eastlink.ca> - 2012-02-09 06:24 -0400
                    Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-09 09:15 -0700
                      Re: Interplatform (interprocess, interlanguage) communication Arved Sandstrom <asandstrom3minus1@eastlink.ca> - 2012-02-09 18:58 -0400
                        Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-09 16:15 -0700
                        Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-09 18:50 -0500
                          Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-09 21:40 -0700
                            Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-11 14:47 -0500
                              Re: Interplatform (interprocess, interlanguage) communication Lew <lewbloch@gmail.com> - 2012-02-11 12:06 -0800
                                Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-11 15:18 -0500
                                  Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-11 23:03 -0700
                                    Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-12 09:27 -0500
                                      Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-12 13:33 -0700
                                        Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-12 15:50 -0500
                                          Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-12 14:34 -0700
                      Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-09 18:48 -0500
                        Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-09 21:46 -0700
                          Re: Interplatform (interprocess, interlanguage) communication Lew <lewbloch@gmail.com> - 2012-02-10 08:51 -0800
                            Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-10 10:43 -0700
                              Re: Interplatform (interprocess, interlanguage) communication Lew <lewbloch@gmail.com> - 2012-02-10 13:15 -0800
                                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-10 14:50 -0700
                                  Re: Interplatform (interprocess, interlanguage) communication Lew <lewbloch@gmail.com> - 2012-02-10 14:32 -0800
                                    Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-10 17:10 -0700
                                      Re: Interplatform (interprocess, interlanguage) communication Arved Sandstrom <asandstrom3minus1@eastlink.ca> - 2012-02-10 22:08 -0400
                                        Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-11 00:49 -0700
                                          Re: Interplatform (interprocess, interlanguage) communication Arved Sandstrom <asandstrom3minus1@eastlink.ca> - 2012-02-11 14:04 -0400
                                Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-11 14:55 -0500
                              Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-11 14:52 -0500
                                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-11 20:06 -0700
                                  Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-11 22:41 -0500
                                    Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-12 00:46 -0700
                                      Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-12 09:29 -0500
                                      Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-12 09:31 -0500
                                    Re: Interplatform (interprocess, interlanguage) communication Martin Gregorie <martin@address-in-sig.invalid> - 2012-02-12 16:02 +0000
                                      Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-12 11:16 -0500
                                        Re: Interplatform (interprocess, interlanguage) communication Martin Gregorie <martin@address-in-sig.invalid> - 2012-02-12 22:46 +0000
                                      Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-12 11:33 -0700
                                  Re: Interplatform (interprocess, interlanguage) communication Lew <lewbloch@gmail.com> - 2012-02-11 20:18 -0800
                                    Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-12 01:36 -0700
                                      Re: Interplatform (interprocess, interlanguage) communication Joshua Cranmer <Pidgeot18@verizon.invalid> - 2012-02-12 13:52 -0600
                                        Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-12 14:43 -0700
                          Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-11 14:49 -0500
                    Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-09 18:46 -0500
                  Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-09 18:45 -0500
          Re: Interplatform (interprocess, interlanguage) communication Lew <lewbloch@gmail.com> - 2012-02-08 14:02 -0800
            Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-08 18:49 -0700
              Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-08 21:14 -0500
                Re: Interplatform (interprocess, interlanguage) communication Lew <lewbloch@gmail.com> - 2012-02-08 20:07 -0800
                  Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-08 23:29 -0700
                    Re: Interplatform (interprocess, interlanguage) communication Lew <lewbloch@gmail.com> - 2012-02-09 09:40 -0800
                      Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-09 17:02 -0700
                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-08 21:10 -0700
                  Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-09 18:54 -0500
                    Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-10 10:25 -0700
                      Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-11 14:45 -0500
                        Re: Interplatform (interprocess, interlanguage) communication Lew <lewbloch@gmail.com> - 2012-02-11 12:14 -0800
                          Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-11 15:20 -0500
                            Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-11 22:20 -0700
                              Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-12 09:23 -0500
                                Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-12 12:13 -0700
      Re: Interplatform (interprocess, interlanguage) communication Arne Vajhøj <arne@vajhoej.dk> - 2012-02-07 20:24 -0500
      Re: Interplatform (interprocess, interlanguage) communication Martin Gregorie <martin@address-in-sig.invalid> - 2012-02-08 01:31 +0000
        Re: Interplatform (interprocess, interlanguage) communication BGB <cr88192@hotmail.com> - 2012-02-08 00:55 -0700

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

#11984

On Sat, 11 Feb 2012 22:41:42 -0500, Arne Vajhøj wrote:

> 
> C is not standard on Windows either.
> 
> You need to get some things.
>
Yes, so if you're intending to write code that ports easily between 
Windows and *nix/POSIX (and in my case, OS-9), you end up writing a 
compatibility library for each target OS. This is mainly a collection of 
functions that are standard in one of the other target OSen and absent on 
its target system. A good example is the command line parser getopt(), 
which is absent from OS_9 and (IIRC) Windows libraries. 
 

-- 
martin@   | Martin Gregorie
gregorie. | Essex, UK
org       |

[toc] | [prev] | [next] | [standalone]

#11988

On 2/12/2012 11:02 AM, Martin Gregorie wrote:
> On Sat, 11 Feb 2012 22:41:42 -0500, Arne Vajhøj wrote:
>> C is not standard on Windows either.
>>
>> You need to get some things.
>>
> Yes, so if you're intending to write code that ports easily between
> Windows and *nix/POSIX (and in my case, OS-9), you end up writing a
> compatibility library for each target OS. This is mainly a collection of
> functions that are standard in one of the other target OSen and absent on
> its target system. A good example is the command line parser getopt(),
> which is absent from OS_9 and (IIRC) Windows libraries.

Doesn't getopt exist in some GNU lib that you can get for all
platforms?

Arne

[toc] | [prev] | [next] | [standalone]

#12007

On Sun, 12 Feb 2012 11:16:03 -0500, Arne Vajhøj wrote:

> 
> Doesn't getopt exist in some GNU lib that you can get for all platforms?
> 
Pass. I know it was missing from Borland C on DOS and Windows and wasn't 
published for OS-9. I ended up extending a PD version that was originally 
published in the '68 MicroJournal for OS-9 and porting it to the other 
two platforms. 

I had a good (to me) reason for doing that. At the time I was more 
familiar with OS-9 than Windows or Unix (Linux didn't exist at the time) 
and I had got used to the OS-9 command line parser's ability to handle a 
mix of options and arguments in any order rather than the straitjacket of 
Unix's rigid options before arguments rule. My extension basically just 
added the -x=value notation (also used by OS-9) to the standard Unix -
xvalue and -x value notation. I've since rewritten it for Java, adding 
long option names (--xxxx and --xxxx=val) - something I've not gotten 
round to adding to the C version.


-- 
martin@   | Martin Gregorie
gregorie. | Essex, UK
org       |

[toc] | [prev] | [next] | [standalone]

#11995

On 2/12/2012 9:02 AM, Martin Gregorie wrote:
> On Sat, 11 Feb 2012 22:41:42 -0500, Arne Vajhøj wrote:
>
>>
>> C is not standard on Windows either.
>>
>> You need to get some things.
>>
> Yes, so if you're intending to write code that ports easily between
> Windows and *nix/POSIX (and in my case, OS-9), you end up writing a
> compatibility library for each target OS. This is mainly a collection of
> functions that are standard in one of the other target OSen and absent on
> its target system. A good example is the command line parser getopt(),
> which is absent from OS_9 and (IIRC) Windows libraries.
>

yes, this is what is often done.

one ends up with essentially a pile of code intended to wrap various 
APIs for various OS's, such that internally the app can use a more 
consistent API.

there is also SDL, which on one hand wraps a lot of this stuff, but OTOH 
is a 3rd party library which carries the usual issues. a way to make 
this work though is to treat SDL as if it were a pseudo-OS (its wrappers 
are wrapped in much the same way).

lots of other apps I have looked at seem to contain similar wrapper layers.


some people go further, and try to write wrappers to hide the 
differences between Direct3D and OpenGL, but I don't personally go that 
far (I just use OpenGL and regard it as "good enough").

however, due to secondary reasons (mostly making things more consistent, 
like having a more consistent APIs for dealing with things like 
shader/material effects, lighting, ...), a lot of OpenGL has ended up 
being wrapped (in an an admittedly often ad-hoc manner).

I have noted that Doom3 tends to wrap OpenGL far more significantly 
(though, it would probably be going off on a bit of a tangent to 
describe Doom3's renderer here).

[toc] | [prev] | [next] | [standalone]

#11960

BGB wrote:
> Arne Vajhøj wrote:
> > LIBXML2 works fine on Windows, so you can use it on both platforms.
> >
> 
> yeah, it is an option.
> however, it is not a standard library on Windows (in certain cases, one 

First you spend half a newsgroup thread decrying standards and 
proudly boasting how you flout them, now you suddenly denigrate a 
library for not being standard?

And yes, libxml2 is, too, standard, as such things go.

> may need to provide for it, or expect anyone who wants to build from 
> source to provide for it, ...).

"provide for it"?

> >> but, anyways, it is like asking a person never to write their own JPEG
> >> loader/saver, or their own scripting-language compiler. yes, maybe a
> >> person doesn't technically need to, but they may forsake potentially
> >> valuable learning experiences (or the claim to having the skills to do
> >> so).
> >
> > I think you should very clearly distinguish between when you talk about
> > learning and programming production code.
> >
> > The goals are just so different.
> >
> 
> in my case, both often end up being the same code.

So you don't write code for someone else?

> one may end up doing something initially as a learning activity, but if 
> one does so, and the code works fairly well, why write the same code 
> again?...

Asked and answered upthread. Don't cycle.

> granted, being a programmer working for a corporation or something, vs 
> being an independent game developer, could also be a factor.

If no one but you ever looks at your code or has to maintain it, 
you can be as idiosyncratic and antisocial as you like. I know 
people who've lived alone so long they cannot maintain a civil 
discourse in public. Their habits don't bother them when they're 
home alone, but that doesn't make them optimal.

Most of your arguments sound like apologetics for undisciplined, 
egocentric programming with little connection to facts or the real 
world of the workaday programmer, or the costs thereof. Heck, you 
haven't even answered the question as to what the costs of software 
development are.

-- 
Lew

[toc] | [prev] | [next] | [standalone]

#11969

On 2/11/2012 9:18 PM, Lew wrote:
> BGB wrote:
>> Arne Vajhøj wrote:
>>> LIBXML2 works fine on Windows, so you can use it on both platforms.
>>>
>>
>> yeah, it is an option.
>> however, it is not a standard library on Windows (in certain cases, one
>
> First you spend half a newsgroup thread decrying standards and
> proudly boasting how you flout them, now you suddenly denigrate a
> library for not being standard?
>
> And yes, libxml2 is, too, standard, as such things go.
>

standard, as-in, comes bundled with the OS and compiler.

libxml2 comes on Linux, but not on Windows with the "Windows SDK" or 
similar (AFAIK it does come with Cygwin though).


safely, one can use, on Windows:
the ANSI C runtime (more or less C89/C90);
any Win32 API provided stuff (Winsock, GDI, OpenGL, ...);
...

on Linux, one has:
the ANSI C runtime (C99);
POSIX;
X11 and OpenGL;
a big pile of FOSS libraries.

so, what is common:
the C runtime;
OpenGL.

and, what needs to be provided by OS specific shims:
BSD Sockets / Winsock;
GDI / GLX;
getting user-input (keyboard / mouse);
stuff for low-level memory management, threads, managing dynamic libraries;
...


most other things tend to be kept either internal or are optional.
I don't think architecturally it is all that unusual (take, for example, 
Doom 3 and Mozilla, both of which appear to have a similar architecture).


>> may need to provide for it, or expect anyone who wants to build from
>> source to provide for it, ...).
>
> "provide for it"?
>

have to make sure it is built, installed, and on the OS's library and 
include paths (like, mandating a particular build configuration).

otherwise, the library needs to be copied into the application's build 
tree, and built along with the application.


>>>> but, anyways, it is like asking a person never to write their own JPEG
>>>> loader/saver, or their own scripting-language compiler. yes, maybe a
>>>> person doesn't technically need to, but they may forsake potentially
>>>> valuable learning experiences (or the claim to having the skills to do
>>>> so).
>>>
>>> I think you should very clearly distinguish between when you talk about
>>> learning and programming production code.
>>>
>>> The goals are just so different.
>>>
>>
>> in my case, both often end up being the same code.
>
> So you don't write code for someone else?
>

like a contractor or something?... nope.


I write software for myself, and hopefully to try to get money more 
directly (selling directly to end users).


>> one may end up doing something initially as a learning activity, but if
>> one does so, and the code works fairly well, why write the same code
>> again?...
>
> Asked and answered upthread. Don't cycle.
>

with a thread this long, how is one supposed to remember all of what 
they have said already?


>> granted, being a programmer working for a corporation or something, vs
>> being an independent game developer, could also be a factor.
>
> If no one but you ever looks at your code or has to maintain it,
> you can be as idiosyncratic and antisocial as you like. I know
> people who've lived alone so long they cannot maintain a civil
> discourse in public. Their habits don't bother them when they're
> home alone, but that doesn't make them optimal.
>
> Most of your arguments sound like apologetics for undisciplined,
> egocentric programming with little connection to facts or the real
> world of the workaday programmer, or the costs thereof. Heck, you
> haven't even answered the question as to what the costs of software
> development are.
>

costs of software development?...
besides the obvious stuff, like writing the program and making it work?...

I don't personally believe my strategy to be making it all that much 
harder (or expensive) for myself.


but, anyways, discipline itself is a matter of cost/benefit tradeoffs, 
like "what do I get out of it?" and similar. the general idea is that 
people act in their self-interest to try to maximize their own gain. I 
don't think my behaviors are inconsistent with this ideal (except being 
lazy at times, where time spent not doing stuff is time wasted).

for example, there is the larger society, but each person is themselves, 
and the people and things around them can be valued in terms of how they 
may be of benefit to oneself and similar, but, with the side effect that 
often whatever is most benefit to oneself may potentially be of benefit 
to others as well (note: I don't currently personally believe altruism 
actually exists, but rather it is likely a sort of misconception 
regarding what really is in ones' best-interests).

similarly, the relative value of an action could be evaluated primarily 
in terms of its most likely costs and benefits (to oneself, although not 
necessarily immediate, for example a more immediate gain may cost more 
later on, ...).

discipline can itself cost in terms of time or effort, whereas hacking 
together something in the name of expedience can get it into working 
order quicker.


or such...

[toc] | [prev] | [next] | [standalone]

#11999

On 2/12/2012 2:36 AM, BGB wrote:
> safely, one can use, on Windows:
> the ANSI C runtime (more or less C89/C90);
> any Win32 API provided stuff (Winsock, GDI, OpenGL, ...);
> ...

I call BS on this, having worked on a major open-source project that 
works on the major platforms of Windows, Mac OS X, and Linux (and also 
Android, and I think Solaris and *BSD are still reasonably 
well-supported, although the Haiku and OS/2 ports are now thoroughly 
dead). What libraries does this? A small list:
* libpng
* libjpg
* libogg + related
* cairo
* thebes
* sqlite
* freetype
* libbz2
* libjar
* zlib

And all of these are still used on Windows; there are even more that are 
used only on Linux or Mac OS X.

Which application is this? Mozilla.

It's not that hard to use other libraries on Windows.

-- 
Beware of bugs in the above code; I have only proved it correct, not 
tried it. -- Donald E. Knuth

[toc] | [prev] | [next] | [standalone]

#12006

On 2/12/2012 12:52 PM, Joshua Cranmer wrote:
> On 2/12/2012 2:36 AM, BGB wrote:
>> safely, one can use, on Windows:
>> the ANSI C runtime (more or less C89/C90);
>> any Win32 API provided stuff (Winsock, GDI, OpenGL, ...);
>> ...
>
> I call BS on this, having worked on a major open-source project that
> works on the major platforms of Windows, Mac OS X, and Linux (and also
> Android, and I think Solaris and *BSD are still reasonably
> well-supported, although the Haiku and OS/2 ports are now thoroughly
> dead). What libraries does this? A small list:
> * libpng
> * libjpg
> * libogg + related
> * cairo
> * thebes
> * sqlite
> * freetype
> * libbz2
> * libjar
> * zlib
>
> And all of these are still used on Windows; there are even more that are
> used only on Linux or Mac OS X.
>
> Which application is this? Mozilla.
>
> It's not that hard to use other libraries on Windows.
>

yes, but it is also worth noting that Mozilla does the whole "Mozilla 
build" thingy on Windows, and are essentially they are bundling many of 
the needed libraries and tools with the application as a part of the 
build system.


as noted:
it is not saying that one *can't* use 3rd party libraries, but one may 
need to make special provisions for them.

Mozilla does this.

not everyone may want to do this, as it is a reasonably heavy-weight 
solution to the problem (but still better than "hey random person, go 
download and build all of these libraries yourself", which is what some 
applications have gone and done).

[toc] | [prev] | [next] | [standalone]

#11938

On 2/9/2012 11:46 PM, BGB wrote:
> On 2/9/2012 4:48 PM, Arne Vajhøj wrote:
>> On 2/9/2012 11:15 AM, BGB wrote:
>>> On 2/9/2012 3:24 AM, Arved Sandstrom wrote:
>>>> On 12-02-08 10:50 PM, BGB wrote:
>>>>> On 2/8/2012 7:16 PM, Arne Vajhøj wrote:
>>>>>> On 2/8/2012 2:07 PM, BGB wrote:
>>>>>>> On 2/8/2012 4:19 AM, Arved Sandstrom wrote:
>>>>>>>> On 12-02-08 04:41 AM, BGB wrote:
>>>>>>>>> note: my main way of working with XML is typically via DOM-style
>>>>>>>>> interfaces (if I am using it, it is typically because I am
>>>>>>>>> directly
>>>>>>>>> working with the data structure, and not as the result of some
>>>>>>>>> dumb-ass
>>>>>>>>> "data binding" crud...).
>>>>>>>>
>>>>>>>> I haven't been able to completely avoid using the DOM, but I
>>>>>>>> loathe the
>>>>>>>> API. If I'm using XML at all, and JAXB suits, I'll use JAXB. More
>>>>>>>> generally I'll use SAX or StAX.
>>>>>>>>
>>>>>>>
>>>>>>> I have rarely done things for which SAX has made sense...
>>>>>>> usually in cases where SAX would make sense, I end up using
>>>>>>> line-oriented text formats instead (because there is often little
>>>>>>> obvious reason for why XML syntax would make much sense).
>>>>>>
>>>>>> Non flat structure and validation comes to mind.
>>>>>
>>>>> fair enough.
>>>>>
>>>>> often, one can implement non-flat structures with line-oriented
>>>>> formats,
>>>>> for example:
>>>>> ...
>>>>> groupDef {
>>>>> ...
>>>>> groupDef {
>>>>> itemDef {
>>>>> ...
>>>>> }
>>>>> ...
>>>>> }
>>>>> ...
>>>>> }
>>>> [ SNIP ]
>>>>
>>>> No need for the braces, if you're going to use those all you gain over
>>>> the XML is terseness.
>>>>
>>>
>>> well, if the format is still line-oriented, one can still parse the
>>> files using a loop, getting and splitting strings, and checking the
>>> first token of each line.
>>>
>>> parsing XML is a little more invovlved, since:
>>> items may be split across lines, or multiple items may exist on the same
>>> line;
>>> one can no longer use whitespace or commas as the primary deliminator;
>>
>> ????
>>
>> No one in their right mind would parse XML manually.
>>
>> You can pick between lots of nice XML API's (many of them
>> shipping with Java) that will handle all that.
>>
>
> depends on which language one is using at the time...
>
> if one is using Java, then XML parsing is basically free.
> if one is using C, then it is either "write some code to do it", or
> suffer with a 3rd party library dependency (one might validly choose to
> write the code themselves in this case).
>
> I don't expect it is all that uncommon for a person to switch between
> several different languages, and maybe deal with the strengths and
> weaknesses of whichever language they are using at the time.

But given that there is little code reuse between Java and C
in the first place, then C not having builtin XML parser
is no reason not use the builtin one in Java.

And if we talk C then look at LIBXML2 - it works pretty well.

Arne

[toc] | [prev] | [next] | [standalone]

#11889

On 2/9/2012 5:24 AM, Arved Sandstrom wrote:
> As an example take a look at log4j .properties and XML configuration
> files. All you gain with the XML is the ability to validate against a
> log4j DTD.

And problems appending to an existing file ...

Arne

[toc] | [prev] | [next] | [standalone]

#11888

On 2/8/2012 9:50 PM, BGB wrote:
> On 2/8/2012 7:16 PM, Arne Vajhøj wrote:
>> On 2/8/2012 2:07 PM, BGB wrote:
>>> On 2/8/2012 4:19 AM, Arved Sandstrom wrote:
>>>> On 12-02-08 04:41 AM, BGB wrote:
>>>>> note: my main way of working with XML is typically via DOM-style
>>>>> interfaces (if I am using it, it is typically because I am directly
>>>>> working with the data structure, and not as the result of some
>>>>> dumb-ass
>>>>> "data binding" crud...).
>>>>
>>>> I haven't been able to completely avoid using the DOM, but I loathe the
>>>> API. If I'm using XML at all, and JAXB suits, I'll use JAXB. More
>>>> generally I'll use SAX or StAX.
>>>>
>>>
>>> I have rarely done things for which SAX has made sense...
>>> usually in cases where SAX would make sense, I end up using
>>> line-oriented text formats instead (because there is often little
>>> obvious reason for why XML syntax would make much sense).
>>
>> Non flat structure and validation comes to mind.
>>
>
> fair enough.
>
> often, one can implement non-flat structures with line-oriented formats,
> for example:
> ...
> groupDef {
> ...
> groupDef {
> itemDef {
> ...
> }
> ...
> }
> ...
> }

But then the parser becomes more complex than using the
builtin XML parser.

> typically, I have not used validation:
> if there is anything to validate, typically this logic will be placed in
> the logic to parse the text.

But yout get it for free with XML - you just need to enable
validation.

Arne

[toc] | [prev] | [next] | [standalone]

#11860

BGB wrote:
> ...
> an example is this:
> <foo> <bar value="3"/> </foo>
> and:
> (foo (bar 3))
> 
> now, consider one wants to add a new field to 'foo' (say 'ln').
> <foo ln="15"> <bar value="3"/> </foo>
> and:
> (foo 15 (bar 3))
> 
> a difference here is that existing code will probably not even notice 
> the new XML attribute, whereas the positional nature of most 

Ahem. You mean other than failing schema validation?

> S-Expressions makes the latter far more likely to break something (and 

More likely than failing schema validation was for that well-designed XML-based 
application?

> there is no good way to "annotate" an S-Exp, whereas with XML it is 
> fairly solidly defined that one can simply add new attributes).

Attributes in XML are not annotation (with or without quotes). That role is  filled by the actual 'annotation' element 
http://www.w3schools.com/schema/el_annotation.asp

> note: my main way of working with XML is typically via DOM-style 
> interfaces (if I am using it, it is typically because I am directly 
> working with the data structure, and not as the result of some dumb-ass 
> "data binding" crud...).

Sorry, "dumb-ass 'data-binding' crud"?

Why the extreme pejoratives? I would not say that there's anything wrong with 
XML data-binding /per se/, although as with documented-oriented approaches it 
can be done very badly.

> typically, the "internal representation" and "concrete serialization" 
> are different:

I don't understand what you mean here. You cite these terms in quotes as though 
they are a standard terminology for some specific things, but use them in their 
ordinary meaning. The internal representation of what? The serialization 
("concrete" or otherwise) of what? I don't mean to be obtuse here, but I am not 
grokking the referents.

> I may use a textual XML serialization, or just as easily, I could use a 
> binary format;
> likewise for S-Exps (actually, I probably far more often represent 
> S-Exps as a binary format of one form or another than I use them in a 
> form externally serialized as text).
> 
> all hail the mighty DOM-node or CONS-cell...

WTF?

-- 
Lew

[toc] | [prev] | [next] | [standalone]

#11862

On 2/8/2012 3:02 PM, Lew wrote:
> BGB wrote:
>> ...
>> an example is this:
>> <foo>  <bar value="3"/>  </foo>
>> and:
>> (foo (bar 3))
>>
>> now, consider one wants to add a new field to 'foo' (say 'ln').
>> <foo ln="15">  <bar value="3"/>  </foo>
>> and:
>> (foo 15 (bar 3))
>>
>> a difference here is that existing code will probably not even notice
>> the new XML attribute, whereas the positional nature of most
>
> Ahem. You mean other than failing schema validation?
>

many of us don't use schemas with our XML.

I think the issue is that one particular technology, XML, is used in 
significantly different ways by different people and for different reasons.

many people use XML for data-binding, and many other people who use it 
could care less about data-binding.


some people may use XML for similar purposes to how people using Lisp 
would use lists (never-mind if this is kind of awkward, it does work).

like, doing Lisp type stuff in Java using DOM-nodes in place of 
cons-based lists... +1 now that Java also (sort of) has closures.


>> S-Expressions makes the latter far more likely to break something (and
>
> More likely than failing schema validation was for that well-designed XML-based
> application?
>

as noted, many people neither use schemas nor any sort of schema 
validation. in many use-cases, schemas are overly constraining to the 
ability of using XML to represent free-form data, or using them 
otherwise would offer little particular advantage.

say, if one is using XML for compiler ASTs or similar (say, the XML is 
used to represent a just-parsed glob of source-code), do they really 
need any sort of schema?

http://en.wikipedia.org/wiki/Abstract_syntax_tree


>> there is no good way to "annotate" an S-Exp, whereas with XML it is
>> fairly solidly defined that one can simply add new attributes).
>
> Attributes in XML are not annotation (with or without quotes). That role is  filled by the actual 'annotation' element
> http://www.w3schools.com/schema/el_annotation.asp
>

they can be used for annotating the nodes in many sane use cases...

a lot depends on how one is using the XML in a given context.


>> note: my main way of working with XML is typically via DOM-style
>> interfaces (if I am using it, it is typically because I am directly
>> working with the data structure, and not as the result of some dumb-ass
>> "data binding" crud...).
>
> Sorry, "dumb-ass 'data-binding' crud"?
>
> Why the extreme pejoratives? I would not say that there's anything wrong with
> XML data-binding /per se/, although as with documented-oriented approaches it
> can be done very badly.
>

yeah, this may have been stated overly strongly.

personally, IMO, data-binding is probably one of the worse and 
technically more pointless ways of using XML (as, IMO, it leads to such 
similarly ill-designed technologies as SOAP and similar...).

not that data-binding is itself necessarily itself pointless, but doing 
it via overly verbose namespace-ridden XML is probably one of the worse 
ways of doing it (vs either specialized file-formats, or the use of 
binary data-binding formats, which IMO should also not be used for data 
interchange).


admittedly, I also partly dislike traditional ways of using data-binding 
as it often exposes things which are theoretically internal to the app, 
namely structural data representation (via classes/...), with things 
which should theoretically be isolated from the internal data 
representation: file formats.

or, IOW: a file-format (or protocol/...) should express the data in 
itself, and not express how it is physically represented within the 
application.

likewise, data going into or coming out of a piece of code should be 
ideally documented and defined in a form separate from the component in 
question.

otherwise, data-binding is not that much different than a more modern 
variant of writing raw structures and arrays to files.


>> typically, the "internal representation" and "concrete serialization"
>> are different:
>
> I don't understand what you mean here. You cite these terms in quotes as though
> they are a standard terminology for some specific things, but use them in their
> ordinary meaning. The internal representation of what? The serialization
> ("concrete" or otherwise) of what? I don't mean to be obtuse here, but I am not
> grokking the referents.
>

the internal representation of the data within the application code.

if one knows which objects or classes exist, what sorts of members they 
contain, ... then one is essentially exposing data which should not be 
visible, or for that matter relied upon for data interchange (or, for 
that matter, relevant).

ideally, any data represented externally should be defined in terms of 
its semantics: something will be present if it is relevant to the 
meaning of the data. the serialization will then be defined in terms of 
expressing the structure and semantics of the data, which may bear very 
little resemblance to how the data is represented in the actual 
classes/arrays/whatever which make up how the data is represented 
internally to the application.

similarly, file formats should be as much abstracted from the 
application code as is reasonably possible, with a "concrete" 
specification for the file-format or data-representation being written 
instead.


both XML and S-Expressions can be used as structured ways of 
representing semantics, rather than as ways of representing the contents 
of given a data-object.


>> I may use a textual XML serialization, or just as easily, I could use a
>> binary format;
>> likewise for S-Exps (actually, I probably far more often represent
>> S-Exps as a binary format of one form or another than I use them in a
>> form externally serialized as text).
>>
>> all hail the mighty DOM-node or CONS-cell...
>
> WTF?
>

DOM nodes can be very powerful (and are probably a much better way of 
using XML than using it as some sort of data-binding thing).


cons-cells are pairs of dynamically-typed values, typically called "car" 
and "cdr" and used to implement lists and similar (and are the main 
building block of "everything" in languages like Lisp and Scheme, well, 
along with "symbols" and "fixnums" and similar).

http://en.wikipedia.org/wiki/Cons_cell

they can also be implemented in C, C++, and Java without too much 
trouble, and can be a fairly useful way of building various sorts of 
data structures (although, sadly, they aren't nearly as efficient in 
Java as they could be, but OTOH it is also sort of a pain to build a 
dynamic type-system in C, so it probably evens out...).

then one can proceed to build logic based mostly on building and 
processing lists.

or, conceptually, they can be regarded as a type of linked-list based 
containers, however the ways they are traditionally used are 
significantly different from traditional ways of using containers (they 
are typically used as ways of building tree-structures, rather than 
usually as ways of storing a collection of items).


it may be worthwhile to look-up information regarding Lisp and Scheme 
and similar, not that there is necessarily much reason to actually use 
the languages, but there are some ideas and ways of doing things which 
can be mapped fairly nicely onto other, more common, languages.

[toc] | [prev] | [next] | [standalone]

#11863

On 2/8/2012 8:49 PM, BGB wrote:
> as noted, many people neither use schemas nor any sort of schema
> validation. in many use-cases, schemas are overly constraining to the
> ability of using XML to represent free-form data, or using them
> otherwise would offer little particular advantage.

xsd:any do provide some flexibility in schemas.

> say, if one is using XML for compiler ASTs or similar (say, the XML is
> used to represent a just-parsed glob of source-code), do they really
> need any sort of schema?

I would expect syntax trees to follow certain rules and not be free
form.

Arne

[toc] | [prev] | [next] | [standalone]

#11866

On Wednesday, February 8, 2012 6:14:31 PM UTC-8, Arne Vajhøj wrote:
> On 2/8/2012 8:49 PM, BGB wrote:
> > as noted, many people neither use schemas nor any sort of schema
> > validation. in many use-cases, schemas are overly constraining to the
> > ability of using XML to represent free-form data, or using them
> > otherwise would offer little particular advantage.
> 
> xsd:any do provide some flexibility in schemas.
> 
> > say, if one is using XML for compiler ASTs or similar (say, the XML is
> > used to represent a just-parsed glob of source-code), do they really
> > need any sort of schema?
> 
> I would expect syntax trees to follow certain rules and not be free
> form.

In one breath we're singing the praises of binary formats, in the next we 
complain that XML isn't sufficiently flexible.

"Do they really need any sort of schema?" with XML is usually a "yes".

But only if you're interested in clear, unambiguous, readily-parsable and 
maintainable XML document formats.

People often excoriate the supposed verbosity of XML as though it were the only 
criterion to measure utility.

There is no inherent advantage of a LISP/list-like format over any other, nor vice versa; it's all accordin'. If the convention is agreeable to all parties, 
it will work. If all projects were one-off and isolated from the larger world, 
we'd never need to adhere to a standard. If we don't mind inventing our own 
tools for anything, we'd never have to adopt a standard with extensive tools 
support.

Where are the *real* costs of a software system?

-- 
Lew

[toc] | [prev] | [next] | [standalone]

#11870

On 2/8/2012 9:07 PM, Lew wrote:
> On Wednesday, February 8, 2012 6:14:31 PM UTC-8, Arne Vajhøj wrote:
>> On 2/8/2012 8:49 PM, BGB wrote:
>>> as noted, many people neither use schemas nor any sort of schema
>>> validation. in many use-cases, schemas are overly constraining to the
>>> ability of using XML to represent free-form data, or using them
>>> otherwise would offer little particular advantage.
>>
>> xsd:any do provide some flexibility in schemas.
>>
>>> say, if one is using XML for compiler ASTs or similar (say, the XML is
>>> used to represent a just-parsed glob of source-code), do they really
>>> need any sort of schema?
>>
>> I would expect syntax trees to follow certain rules and not be free
>> form.
>
> In one breath we're singing the praises of binary formats, in the next we
> complain that XML isn't sufficiently flexible.
>

it is not like one can't have both:
have a format which is at the same time is a compressed binary format, 
and can also retain the full flexibility of representing free-form XML 
semantics, ideally without a major drop in compactness (this happens 
with WBXML, and IIRC should also happen with EXI about as soon as one 
starts encoding nodes which lie outside the schema).

this is partly why I was advocating a sort of pattern-building adaptive 
format: it can build the functional analogue of a schema as it encodes 
the data, and likewise does not depend on a schema to properly decode 
the document. it is mostly a matter of having the format predict when it 
doesn't need to specify tag and attribute names (it is otherwise similar 
to a traditional data-compressor).

this is functionally similar to the sliding-window as used in deflate 
and LZMA (7zip) and similar (in contrast to codebook-based data 
compressors). functionally, it would have a little more in common with 
LZW+MTF than with LZ77 though.

granted, potentially a binary format could incorporate both support for 
schemas and the use of adaptive compression.


is XML really the text, or is it actually the structure?
I had operated under the premise that it was the data-structure (tags, 
attributes, namespaces, ...), which allows for pretty much anything 
which can faithfully encode the structure (without imposing too many 
arbitrary restrictions).


> "Do they really need any sort of schema?" with XML is usually a "yes".
>
> But only if you're interested in clear, unambiguous, readily-parsable and
> maintainable XML document formats.
>

fair enough, I have mostly been using it "internally", and as noted, for 
some of my file-formats, I had used a custom binary coded variant 
(roughly similar to WBXML, but generally more compact and supporting 
more features, such as namespaces and similar, which I had called SBXE). 
it didn't make use of schemas, and worked by simply encoding the tag 
structure into the file, and using basic contextual modeling strategies.

it also compared favorably with XML+GZ in my tests (which IIRC was also 
generally smaller than WBXML). remotely possible would also be XML+BZip2 
or XML+LZMA.


I had considered the possibility of a more "advanced" format (with more 
advanced predictive modeling), but didn't bother (couldn't see much 
point at the time of trying to shave off more bytes at the time, as it 
was already working fairly well).


> People often excoriate the supposed verbosity of XML as though it were the only
> criterion to measure utility.
>

well, a lot depends...

for disk files, really, who cares?...
for a link where a several kB message might only take maybe 250-500ms 
and is at typical "user-interaction" speeds (say, part of a generic "web 
app"), likewise, who cares?...


it may matter a little more in a 3D interactive world where everything 
going on in the visible scene has to get through at a 10Hz or 24Hz 
clock-tick, and if the connection bogs down the user will be rather 
annoyed (as their game world has essentially stalled).

one may have to make due with about 16-24kB/s (or maybe less) to better 
ensure a good user experience (little is to say that the user has a 
perfect internet connection either).

so, some sort of compression may be needed in this case.
(yes, XML+GZ would probably be sufficient).

if it were dial-up, probably no one would even consider using XML for 
the network protocol in a 3D game.


> There is no inherent advantage of a LISP/list-like format over any other, nor vice versa; it's all accordin'. If the convention is agreeable to all parties,
> it will work. If all projects were one-off and isolated from the larger world,
> we'd never need to adhere to a standard. If we don't mind inventing our own
> tools for anything, we'd never have to adopt a standard with extensive tools
> support.
>

it is possible, it all depends.

a swaying factor in my last choice was the effort tradeoff of writing 
the code (because working with DOM is kind of a pain...). IIRC, I may 
have also been worrying about performance (mostly passing around lots of 
numeric data as ASCII strings, ...).

but, I may eventually need to throw together a basic encoding scheme for 
this case (a binary encoder for list-based data), that or just reuse an 
existing data serializer of mine (mostly intended for generic data 
serialization, which supports lists). it lacks any sort of prediction or 
context modeling though, and is used in my stuff mostly as a container 
format for bytecode for my VM and similar.


> Where are the *real* costs of a software system?
>

who knows?...

probably delivering the best reasonable user experience?...

for a game:
reasonably good graphics;
reasonably good performance (ideally, consistently over 30fps);
hopefully good gameplay, plot, story, ...

well, that and "getting everything done" (this is the hard one).

[toc] | [prev] | [next] | [standalone]

#11882

BGB wrote:
> Lew wrote:
> > Arne Vajhøj wrote:
> >> BGB wrote:
> >>> as noted, many people neither use schemas nor any sort of schema
> >>> validation. in many use-cases, schemas are overly constraining to the
> >>> ability of using XML to represent free-form data, or using them
> >>> otherwise would offer little particular advantage.
> >>
> >> xsd:any do provide some flexibility in schemas.
> >>
> >>> say, if one is using XML for compiler ASTs or similar (say, the XML is
> >>> used to represent a just-parsed glob of source-code), do they really
> >>> need any sort of schema?
> >>
> >> I would expect syntax trees to follow certain rules and not be free
> >> form.
> >
> > In one breath we're singing the praises of binary formats, in the next we
> > complain that XML isn't sufficiently flexible.
> >
> 
> it is not like one can't have both:

XML is much easier to modify and maintain when flexibility is a requirement.

> have a format which is at the same time is a compressed binary format, 
> and can also retain the full flexibility of representing free-form XML 
> semantics, ideally without a major drop in compactness (this happens 
> with WBXML, and IIRC should also happen with EXI about as soon as one 
> starts encoding nodes which lie outside the schema).
> 
> this is partly why I was advocating a sort of pattern-building adaptive 
> format: it can build the functional analogue of a schema as it encodes 

That rather defeats the purpose of having a schema.

A schema is a contract that the various processes or other stakeholders use to 
guarantee correctness of the XML and guide processing. If you develop it /ad 
hoc/ you lose that contract.

> the data, and likewise does not depend on a schema to properly decode 
> the document. it is mostly a matter of having the format predict when it 
> doesn't need to specify tag and attribute names (it is otherwise similar 
> to a traditional data-compressor).

I'm sure that's very clever, but it defeats the purpose of XML schema.

> this is functionally similar to the sliding-window as used in deflate 
> and LZMA (7zip) and similar (in contrast to codebook-based data 
> compressors). functionally, it would have a little more in common with 
> LZW+MTF than with LZ77 though.

... and now you're off on some weird tangential topic.

> granted, potentially a binary format could incorporate both support for 
> schemas and the use of adaptive compression.
> 
> 
> is XML really the text, or is it actually the structure?

Huh?

> I had operated under the premise that it was the data-structure (tags, 
> attributes, namespaces, ...), which allows for pretty much anything 
> which can faithfully encode the structure (without imposing too many 
> arbitrary restrictions).

Huh?

XML is a formal specification for structured documents that is devoid of 
semantics.

> > "Do they really need any sort of schema?" with XML is usually a "yes".
> >
> > But only if you're interested in clear, unambiguous, readily-parsable and
> > maintainable XML document formats.
> >
> 
> fair enough, I have mostly been using it "internally", and as noted, for 
> some of my file-formats, I had used a custom binary coded variant 
> (roughly similar to WBXML, but generally more compact and supporting 
> more features, such as namespaces and similar, which I had called SBXE). 
> it didn't make use of schemas, and worked by simply encoding the tag 
> structure into the file, and using basic contextual modeling strategies.

Bully.  Good on ye.

> it also compared favorably with XML+GZ in my tests (which IIRC was also 
> generally smaller than WBXML). remotely possible would also be XML+BZip2 
> or XML+LZMA.

Compared "favorably" according to what criteria?

> I had considered the possibility of a more "advanced" format (with more 
> advanced predictive modeling), but didn't bother (couldn't see much 
> point at the time of trying to shave off more bytes at the time, as it 
> was already working fairly well).

Huh?

> > People often excoriate the supposed verbosity of XML as though it were the only
> > criterion to measure utility.
> >
> 
> well, a lot depends...
> 
> for disk files, really, who cares?...
> for a link where a several kB message might only take maybe 250-500ms 
> and is at typical "user-interaction" speeds (say, part of a generic "web 
> app"), likewise, who cares?...
> 
> 
> it may matter a little more in a 3D interactive world where everything 
> going on in the visible scene has to get through at a 10Hz or 24Hz 
> clock-tick, and if the connection bogs down the user will be rather 
> annoyed (as their game world has essentially stalled).

And that's a use case for XML how, exactly?

Saying "XML is bad because it doesn't keep bananas ripe" would be equally 
relevant.

> one may have to make due with about 16-24kB/s (or maybe less) to better 
> ensure a good user experience (little is to say that the user has a 
> perfect internet connection either).
> 
> so, some sort of compression may be needed in this case.
> (yes, XML+GZ would probably be sufficient).

Back in the universe where we're discussing XML's suitability, please.

> if it were dial-up, probably no one would even consider using XML for 
> the network protocol in a 3D game.

Oh, you're talking about inter-node communication in a distributed game. Thanks 
for finally making that clear. XML would be just fine as a transmission protocol for such a thing. I'm not saying ideal, but just fine.

If you're talking about network protocols you certainly are not talking about 
frame-by-frame transmission of data with reply at 10 Hz, no matter what the 
protocol, so your entire argument against XML for such a thing is moot.

> > There is no inherent advantage of a LISP/list-like format over any other, nor vice versa; it's all accordin'. If the convention is agreeable to all parties,
> > it will work. If all projects were one-off and isolated from the larger world,
> > we'd never need to adhere to a standard. If we don't mind inventing our own
> > tools for anything, we'd never have to adopt a standard with extensive tools
> > support.
> >
> 
> it is possible, it all depends.
> 
> a swaying factor in my last choice was the effort tradeoff of writing 
> the code (because working with DOM is kind of a pain...). IIRC, I may 

Huh? again. There's very little effort in writing XML code, whether DOM, JAXB, 
SAX or StAX, given the wide availability of libraries to do so.

> have also been worrying about performance (mostly passing around lots of 
> numeric data as ASCII strings, ...).

Based on what measurements?

> but, I may eventually need to throw together a basic encoding scheme for 
> this case (a binary encoder for list-based data), that or just reuse an 
> existing data serializer of mine (mostly intended for generic data 
> serialization, which supports lists). it lacks any sort of prediction or 
> context modeling though, and is used in my stuff mostly as a container 
> format for bytecode for my VM and similar.
> 
> 
> > Where are the *real* costs of a software system?
> >
> 
> who knows?...

Anyone who thinks about it realistically.

> probably delivering the best reasonable user experience?...

That's not a cost, that's a goal.

> for a game:
> reasonably good graphics;
> reasonably good performance (ideally, consistently over 30fps);
> hopefully good gameplay, plot, story, ...
> 
> well, that and "getting everything done" (this is the hard one).

Those aren't costs. Those are goals.

Clear conclusions require clear reasoning on actual facts with relevance.

-- 
Lew

[toc] | [prev] | [next] | [standalone]

#11893

On 2/9/2012 10:40 AM, Lew wrote:
> BGB wrote:
>> Lew wrote:
>>> Arne Vajhøj wrote:
>>>> BGB wrote:
>>>>> as noted, many people neither use schemas nor any sort of schema
>>>>> validation. in many use-cases, schemas are overly constraining to the
>>>>> ability of using XML to represent free-form data, or using them
>>>>> otherwise would offer little particular advantage.
>>>>
>>>> xsd:any do provide some flexibility in schemas.
>>>>
>>>>> say, if one is using XML for compiler ASTs or similar (say, the XML is
>>>>> used to represent a just-parsed glob of source-code), do they really
>>>>> need any sort of schema?
>>>>
>>>> I would expect syntax trees to follow certain rules and not be free
>>>> form.
>>>
>>> In one breath we're singing the praises of binary formats, in the next we
>>> complain that XML isn't sufficiently flexible.
>>>
>>
>> it is not like one can't have both:
>
> XML is much easier to modify and maintain when flexibility is a requirement.
>
>> have a format which is at the same time is a compressed binary format,
>> and can also retain the full flexibility of representing free-form XML
>> semantics, ideally without a major drop in compactness (this happens
>> with WBXML, and IIRC should also happen with EXI about as soon as one
>> starts encoding nodes which lie outside the schema).
>>
>> this is partly why I was advocating a sort of pattern-building adaptive
>> format: it can build the functional analogue of a schema as it encodes
>
> That rather defeats the purpose of having a schema.
>
> A schema is a contract that the various processes or other stakeholders use to
> guarantee correctness of the XML and guide processing. If you develop it /ad
> hoc/ you lose that contract.
>

there is no schema in use in this case, however...

hence, why the format would ideally need to be adaptive:
so one doesn't need a schema for it to work correctly, and also so the 
use of free-form data will not hinder compression.


>> the data, and likewise does not depend on a schema to properly decode
>> the document. it is mostly a matter of having the format predict when it
>> doesn't need to specify tag and attribute names (it is otherwise similar
>> to a traditional data-compressor).
>
> I'm sure that's very clever, but it defeats the purpose of XML schema.
>

which is not being used in this case to begin with...


>> this is functionally similar to the sliding-window as used in deflate
>> and LZMA (7zip) and similar (in contrast to codebook-based data
>> compressors). functionally, it would have a little more in common with
>> LZW+MTF than with LZ77 though.
>
> ... and now you're off on some weird tangential topic.
>

data compression...


>> granted, potentially a binary format could incorporate both support for
>> schemas and the use of adaptive compression.
>>
>>
>> is XML really the text, or is it actually the structure?
>
> Huh?
>

as I see it, XML exists at 2 levels:
as a textual syntax;
as a semantic (a tree of tags with attributes and so on) structure, 
which can be expressed via the textual syntax.

conceptually, the semantic structure of XML is more or less equivalent 
to a tree of DOM nodes.


>> I had operated under the premise that it was the data-structure (tags,
>> attributes, namespaces, ...), which allows for pretty much anything
>> which can faithfully encode the structure (without imposing too many
>> arbitrary restrictions).
>
> Huh?
>
> XML is a formal specification for structured documents that is devoid of
> semantics.
>

the existence of the tags and attributes is the semantics...


>>> "Do they really need any sort of schema?" with XML is usually a "yes".
>>>
>>> But only if you're interested in clear, unambiguous, readily-parsable and
>>> maintainable XML document formats.
>>>
>>
>> fair enough, I have mostly been using it "internally", and as noted, for
>> some of my file-formats, I had used a custom binary coded variant
>> (roughly similar to WBXML, but generally more compact and supporting
>> more features, such as namespaces and similar, which I had called SBXE).
>> it didn't make use of schemas, and worked by simply encoding the tag
>> structure into the file, and using basic contextual modeling strategies.
>
> Bully.  Good on ye.
>
>> it also compared favorably with XML+GZ in my tests (which IIRC was also
>> generally smaller than WBXML). remotely possible would also be XML+BZip2
>> or XML+LZMA.
>
> Compared "favorably" according to what criteria?
>

smaller output size.

I was testing each scenario, and comparing the output sizes.


>> I had considered the possibility of a more "advanced" format (with more
>> advanced predictive modeling), but didn't bother (couldn't see much
>> point at the time of trying to shave off more bytes at the time, as it
>> was already working fairly well).
>
> Huh?
>
>>> People often excoriate the supposed verbosity of XML as though it were the only
>>> criterion to measure utility.
>>>
>>
>> well, a lot depends...
>>
>> for disk files, really, who cares?...
>> for a link where a several kB message might only take maybe 250-500ms
>> and is at typical "user-interaction" speeds (say, part of a generic "web
>> app"), likewise, who cares?...
>>
>>
>> it may matter a little more in a 3D interactive world where everything
>> going on in the visible scene has to get through at a 10Hz or 24Hz
>> clock-tick, and if the connection bogs down the user will be rather
>> annoyed (as their game world has essentially stalled).
>
> And that's a use case for XML how, exactly?
>
> Saying "XML is bad because it doesn't keep bananas ripe" would be equally
> relevant.
>

because one can use XML as the client/server messaging protocol, say, in 
place of "well, I am going to send message tags as raw bytes and have 
each followed by some values...".


>> one may have to make due with about 16-24kB/s (or maybe less) to better
>> ensure a good user experience (little is to say that the user has a
>> perfect internet connection either).
>>
>> so, some sort of compression may be needed in this case.
>> (yes, XML+GZ would probably be sufficient).
>
> Back in the universe where we're discussing XML's suitability, please.
>
>> if it were dial-up, probably no one would even consider using XML for
>> the network protocol in a 3D game.
>
> Oh, you're talking about inter-node communication in a distributed game. Thanks
> for finally making that clear. XML would be just fine as a transmission protocol for such a thing. I'm not saying ideal, but just fine.
>
> If you're talking about network protocols you certainly are not talking about
> frame-by-frame transmission of data with reply at 10 Hz, no matter what the
> protocol, so your entire argument against XML for such a thing is moot.
>

both ends transmit concurrently and asynchronously, but one needs to get 
the messages through at roughly 10Hz for things to remain playable 
(otherwise, real-time interactivity starts to fall apart).


so, the server sends a 10Hz stream of updates to the client;
the client sends a 10Hz stream of movement impulses back to the server;
...

granted, ping time is a bit of an issue, as what actions the player is 
trying to do, and what is going on at the servers' end, will invariably 
drift somewhat.

typically, things like linear extrapolation and similar are used to try 
to make up for sub-optimal ping (ideally, one tries to hide the results 
of the ping time, where possible).


but, it does all work, as evidenced by the prevalence of online gaming 
and similar.


I doubt anyone uses ping/pong or request/response based protocols for 
this, as the ping times over the internet would likely render something 
like this unusable (one would probably need to be on a LAN or something...).


>>> There is no inherent advantage of a LISP/list-like format over any other, nor vice versa; it's all accordin'. If the convention is agreeable to all parties,
>>> it will work. If all projects were one-off and isolated from the larger world,
>>> we'd never need to adhere to a standard. If we don't mind inventing our own
>>> tools for anything, we'd never have to adopt a standard with extensive tools
>>> support.
>>>
>>
>> it is possible, it all depends.
>>
>> a swaying factor in my last choice was the effort tradeoff of writing
>> the code (because working with DOM is kind of a pain...). IIRC, I may
>
> Huh? again. There's very little effort in writing XML code, whether DOM, JAXB,
> SAX or StAX, given the wide availability of libraries to do so.
>

well, the issue is that one needs an method call every time they want to 
fetch an attribute's value or look up a node, which is a little more 
painful than it could be.

it isn't like major pain or anything, but it does tend to result in 
slightly longer and more awkward code.

with lists, traditionally there are operations like:
"cadr", "caddr", "cadddr", ..., "caadr", ..., and so on, which make it a 
bit easier (and more compact) to reference particular items within a 
list (since the operations essentially encode where to fetch the item from).

OTOH, with DOM one might end up with a chain a several statements to 
access an item, and yet more statements if one is checking for null, so 
it is just a little more awkward and verbose to work with, but granted 
it is not like the difference is all that huge (IIRC, the performance 
concern may well have been a bigger factor).


>> have also been worrying about performance (mostly passing around lots of
>> numeric data as ASCII strings, ...).
>
> Based on what measurements?
>

this case was based mostly on speculation that if I am creating piles of 
new strings to pass numbers around, and I am passing a scene-graph 
update on a 10Hz basis, most of which will become garbage immediately 
afterwards, than creating all of those strings could get a little 
expensive (mostly causing the garbage collector to start "doing its 
thing" and reduce performance and similar).

I chose another option (namely lists) which had the option of passing 
the the numbers without allocating any memory on the heap.


>> but, I may eventually need to throw together a basic encoding scheme for
>> this case (a binary encoder for list-based data), that or just reuse an
>> existing data serializer of mine (mostly intended for generic data
>> serialization, which supports lists). it lacks any sort of prediction or
>> context modeling though, and is used in my stuff mostly as a container
>> format for bytecode for my VM and similar.
>>
>>
>>> Where are the *real* costs of a software system?
>>>
>>
>> who knows?...
>
> Anyone who thinks about it realistically.
>
>> probably delivering the best reasonable user experience?...
>
> That's not a cost, that's a goal.
>
>> for a game:
>> reasonably good graphics;
>> reasonably good performance (ideally, consistently over 30fps);
>> hopefully good gameplay, plot, story, ...
>>
>> well, that and "getting everything done" (this is the hard one).
>
> Those aren't costs. Those are goals.
>

not effectively achieving a goal is a cost...


> Clear conclusions require clear reasoning on actual facts with relevance.
>

dunno.


a lot of time one works based on "the feel of the code" or "the feel of 
the problem" or similar (if one "feels" that an option will lead to 
suck, it often does lead to suck). one doesn't necessarily know what the 
reasoning is, one can just follow along where it leads (it can be almost 
like that of a physical sensation or similar, like "what does it feel 
like the code wants to do here?").

also, estimating things based on past experiences and known behaviors 
and "rules of thumb" and so on.

if one knows what something does, one can make an educated guess for 
what it will do in a given situation.

everything else becomes mostly likelihoods and probabilities (like, how 
likely is a good outcome, vs a sucky outcome, ...).


or such...

[toc] | [prev] | [next] | [standalone]

#11867

On 2/8/2012 7:14 PM, Arne Vajhøj wrote:
> On 2/8/2012 8:49 PM, BGB wrote:
>> as noted, many people neither use schemas nor any sort of schema
>> validation. in many use-cases, schemas are overly constraining to the
>> ability of using XML to represent free-form data, or using them
>> otherwise would offer little particular advantage.
>
> xsd:any do provide some flexibility in schemas.
>

yep, but one can wonder what is the gain of using a schema if one is 
just going to use "xsd:any"?...

it is also a mystery how well EXI behaves in this case (admittedly, I 
have not personally looked into EXI in-depth, as I only briefly skimmed 
over the spec a long time ago).


>> say, if one is using XML for compiler ASTs or similar (say, the XML is
>> used to represent a just-parsed glob of source-code), do they really
>> need any sort of schema?
>
> I would expect syntax trees to follow certain rules and not be free
> form.
>

well, there are some rules, but the question is more if a schema or the 
use of validation would offer much advantage to make using it worth the 
bother?...

the other possibility would be to make the next compiler stage, upon 
seeing invalid data, give an error message essentially like "what the 
hell is this?..." and halt compilation (typically this is what happens 
if the compiler logic encounters a node type it doesn't know how to do 
anything with in a situation where a known node-type is expected, or if 
some required node is absent or similar).


so, one can have a schema to validate, say, that ones' "if" node looks like:
<if>
   <cond> expr </cond>
   <then> statement </then>
   <else> statement </else>
</if>

but, OTOH, if upon getting back a null node when looking for "cond" or 
"then", it causes an internal-error message to get displayed, it is the 
same effect. even if it just ungracefully tries to use the null and 
causes the program to crash, it is probably still not a huge loss (apart 
from the annoyance that is a crash-prone compiler...).



I think the original point though was more about XML vs S-Expressions in 
this case though:
XML allows more easily just stuffing-in new tags or contents for 
existing tag-types, if this makes sense (it doesn't necessarily break 
existing code or structures, and actually, protocols like XMPP make use 
of this property fairly directly). for S-Exps, which are often 
essentially, this is much less nice, and will often include needing more 
node-types to deal with the presence or absence of certain features 
(whereas with XML one can use different logic based on whether or not 
certain attributes or tags are present or absent).

granted, it does still leave the possibility that one could structure 
things more loosely (with S-Exps), say, rather than:
( if /cond/ /then/ /else/ )
one has:
( if (cond /cond/ ) (then /then/ ) (else /else/ ) )

so, gaining a little more flexibility at the cost of a little more 
verbosity, which is possibly a reasonable point one could argue (my 
client/server frame-delta protocol works more like this, typically using 
marker tags before everything in place of lots of fixed argument lists, 
although fixed-lists are used in many places as well).


trivia: the frame-delta protocol was originally intended to be 
XML-based, but I switched out to S-Expressions at the last minute (just 
prior to actually implementing it) mostly on the ground that S-Exps 
would have been less effort (and I didn't feel like jerking off with the 
added awkwardness using XML would bring at the moment).

a funny irony would be if someone were to devise some sort of schema 
system and use it to try to validate their S-Expressions.

it is still an open question as to which is ultimately "better", as each 
has strengths, and mostly seems to boil down to a tradeoff between 
flexibility and ease-of-use.

[toc] | [prev] | [next] | [standalone]

#11892

On 2/8/2012 11:10 PM, BGB wrote:
> On 2/8/2012 7:14 PM, Arne Vajhøj wrote:
>> On 2/8/2012 8:49 PM, BGB wrote:
>>> as noted, many people neither use schemas nor any sort of schema
>>> validation. in many use-cases, schemas are overly constraining to the
>>> ability of using XML to represent free-form data, or using them
>>> otherwise would offer little particular advantage.
>>
>> xsd:any do provide some flexibility in schemas.
>>
>
> yep, but one can wonder what is the gain of using a schema if one is
> just going to use "xsd:any"?...

You still have some structure.

> it is also a mystery how well EXI behaves in this case (admittedly, I
> have not personally looked into EXI in-depth, as I only briefly skimmed
> over the spec a long time ago).

No idea. But I would assume EXI supports what is valid XML and XSD.

>>> say, if one is using XML for compiler ASTs or similar (say, the XML is
>>> used to represent a just-parsed glob of source-code), do they really
>>> need any sort of schema?
>>
>> I would expect syntax trees to follow certain rules and not be free
>> form.
>>
>
> well, there are some rules, but the question is more if a schema or the
> use of validation would offer much advantage to make using it worth the
> bother?...

Enforcing correctness of data is usually a good idea.

Arne

[toc] | [prev] | [next] | [standalone]

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

Back to top | Article view | comp.lang.java.programmer

csiph-web