Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > microsoft.public.scripting.vbscript > #12187 > unrolled thread

ADODB.Stream binary array to binary string failed unless x-user-defined is used

Back to article view | Back to microsoft.public.scripting.vbscript

Contents

  ADODB.Stream binary array to binary string failed unless x-user-defined is used JJ <jj4public@vfemail.net> - 2019-09-16 08:23 +0700
    Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-15 23:03 -0400
      Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used GS <gs@v.invalid> - 2019-09-15 23:58 -0400
        Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-16 10:08 -0400
          Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used GS <gs@v.invalid> - 2019-09-16 12:32 -0400
            Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-16 13:13 -0400
              Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used GS <gs@v.invalid> - 2019-09-16 20:26 -0400
                Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-16 22:08 -0400
                  Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used GS <gs@v.invalid> - 2019-09-17 22:08 -0400
      Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used JJ <jj4public@vfemail.net> - 2019-09-16 18:27 +0700
        Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-16 09:55 -0400
          Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-16 10:32 -0400
          Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used JJ <jj4public@vfemail.net> - 2019-09-17 20:15 +0700
            Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-17 10:04 -0400
            Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-17 23:04 -0400
              Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used JJ <jj4public@vfemail.net> - 2019-09-18 16:17 +0700
                Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-18 10:12 -0400
                  Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used JJ <jj4public@vfemail.net> - 2019-09-19 21:23 +0700
                    Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-19 11:47 -0400
              Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "R.Wieser" <address@not.available> - 2019-09-18 11:30 +0200
                Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used JJ <jj4public@vfemail.net> - 2019-09-18 17:43 +0700
                  Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-18 10:16 -0400
                  Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "R.Wieser" <address@not.available> - 2019-09-18 16:49 +0200
                    Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-19 09:29 -0400
                    Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "R.Wieser" <address@not.available> - 2019-09-19 20:10 +0200
                      Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "R.Wieser" <address@not.available> - 2019-09-20 08:04 +0200
                      Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used JJ <jj4public@vfemail.net> - 2019-09-20 16:06 +0700
                        Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "R.Wieser" <address@not.available> - 2019-09-22 12:15 +0200
                          Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-22 09:38 -0400
                            Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "R.Wieser" <address@not.available> - 2019-09-22 16:49 +0200
    Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-16 15:58 -0400
    Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used Schmidt <ng@vbRichClient.com> - 2019-09-20 21:44 +0200
      Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-20 17:11 -0400
        Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used Schmidt <ng@vbRichClient.com> - 2019-09-21 00:18 +0200
          Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-20 21:26 -0400
          Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-20 23:50 -0400
            Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used Schmidt <ng@vbRichClient.com> - 2019-09-22 16:32 +0200
              Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-22 12:45 -0400
                Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used Schmidt <ng@vbRichClient.com> - 2019-09-22 19:41 +0200
                  Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-22 14:10 -0400
                    Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used Schmidt <ng@vbRichClient.com> - 2019-09-22 20:46 +0200
                      Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-22 15:09 -0400
                        Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used Schmidt <ng@vbRichClient.com> - 2019-09-22 22:30 +0200
                      Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-22 15:10 -0400
                        Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used Schmidt <ng@vbRichClient.com> - 2019-09-22 22:32 +0200
              Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used "Mayayana" <mayayana@invalid.nospam> - 2019-09-22 13:45 -0400
      Re: ADODB.Stream binary array to binary string failed unless x-user-defined is used JJ <jj4public@vfemail.net> - 2019-09-22 02:07 +0700

Page 3 of 3 — ← Prev page 1 2 [3]

#12240

Am 22.09.2019 um 20:10 schrieb Mayayana:
> "Schmidt" <ng@vbRichClient.com> wrote
> 
> | Sorry, but every code-snippet I've posted in this thread,
> | already *is* working code - even the longer text in my last post.
> |
> 
>     No. The first one didn't work at all. 
> It was just the 4 helper functions. 

Well, those *exact* same functions are contained (unchanged)
in my larger example, which you just confirmed, did work.
;-)

> | E.g. what you wrote out as Base64-Text in your DemoCode,
> | is unnecessarily blown-up two UTF16-LE TextFormat,
> | which is twice as large as the written file needs to be.
> |
> 
>     Interesting. It does no such thing on my end.  

Please check again, I'm getting (when encoding a 3MB-TestFile), not
the expected 4MB (Base64 "blows up" the bin-content by factor 1.33),
but 8MB instead - when using your code...

I've tested this on Win10 - as well as on a Win8-VM and also
on an old XP-VM).

>     I guess you could add that function to my code, but I'm
> not sure it's necessary. And when I tried it I got "not
> allowed in this context" at the charset assignment.

Then do it in the same sequence (as shown in my functions
StringToBytes and BytesToString).

As for the speed-differences:

You can bring that "up-to-yours", when you change
the BytesToString-Function this way:

Function BytesToString(Bytes, Charset)
   With CreateObject("ADODB.Stream")
     .Open
       .Charset = Charset
       .Type = 1: .Write Bytes: .Position = 0
       .Type = 2
       Dim Arr(), i: i = 0
       Do Until .EOS
         Redim Preserve Arr(i)
         Arr(i) = .ReadText(2^18)
         i = i + 1
       Loop
     .Close
     BytesToString = Join(Arr,"")
   End With
End Function

I've used "normal String-Concats", because it was entirely
sufficient for my usage at the WebServer-side (in Classic-ASP),
because I've never had to deal with Files larger than 2MB or so
on the server-side.

Another (slighter) performance-increase can be accomplished,
when you change the Charset-Encoding I was using in my original
"Main-Script-Code" (at the top) from "utf-8" to "x-ansi".

After these two changes, the two versions should perform
absolutely identical (perhaps mine being a bit faster now,
because it has only to write + later read "half the bytes"
from the intermediate Base64-TextFile.

...

Finally, Code-Constructs like:

With CreateObject("ADODB.Stream")
   '...
End With

Are perfectly fine, because (as said) - the instantiation
via CreateObject(...) takes usually only about 10 Micro-Seconds.

And as for "Destroying the instance" - that's actually
"the beauty" of the With-Construct...
Since it destroys the COM-instance in question exactly at the
point of "End With" (no explicit "Set Nothing" is required).

HTH

Olaf

[toc] | [prev] | [next] | [standalone]

#12241

"Schmidt" <ng@vbRichClient.com> wrote

| > | E.g. what you wrote out as Base64-Text in your DemoCode,
| > | is unnecessarily blown-up two UTF16-LE TextFormat,
| > | which is twice as large as the written file needs to be.
| > |
| >
| >     Interesting. It does no such thing on my end.
|
| Please check again, I'm getting (when encoding a 3MB-TestFile), not
| the expected 4MB (Base64 "blows up" the bin-content by factor 1.33),
| but 8MB instead - when using your code...
|

  This is intriguing. And annoying. My code was fine until
I tried yours with the utf-8. It then started saving as
unicode, like you said. After some searching online it
seems that ADODB 1) sniffs text and decides what it
should be and 2) maintains some kind of memory as to its
default format.

  It seems to solve the problem if I just set Charset.
So with encode, when writing it back to disk, I do like so.:

With ADO
       .Open
       .Type = 2 'text
       .Charset = "windows-1252"
       .WriteText s1
       .SaveToFile Arg & "-64.txt", 2 'OverWrite
       .Close
     End With

 That works fine and eliminates the problem of utf-8
encoding putting a 3-byte marker into the file. And
it shouldn't matter that the encoding is English since
Base64 will all be characters under 128 and therefore
the same regardless of codepage.

  Similarly, when I load the file for decoding, I do the
same:

 With ADO
        .Open
        .Type = 2 'text
        .Charset = "windows-1252"
        .LoadFromFile Arg
        Dim iA, A2()


 My resulting speeds for the 25 MB file are .75 and 1.0.
Pretty close to the original. With your updated code I'm
getting .95 and 1.51. Pretty close again. About 1/3 to 1/2
slower, but insignificant in real usage.

  So the only real difference is that my code is pretty
and easy to read, while yours is confusing and needs
editing before use. :)

   But it is discouraging that ADODB is as undependable
in its mangling of text as FSO is. However, if Charset
can be used to make it behave then that helps. 

[toc] | [prev] | [next] | [standalone]

#12243

Am 22.09.2019 um 21:09 schrieb Mayayana:
> "Schmidt" <ng@vbRichClient.com> wrote
> 
> | > | E.g. what you wrote out as Base64-Text in your DemoCode,
> | > | is unnecessarily blown-up two UTF16-LE TextFormat,
> | > | which is twice as large as the written file needs to be.
> | > |
> | >
> | >     Interesting. It does no such thing on my end.
> |
> | Please check again, I'm getting (when encoding a 3MB-TestFile), not
> | the expected 4MB (Base64 "blows up" the bin-content by factor 1.33),
> | but 8MB instead - when using your code...
> |
> 
>    This is intriguing. And annoying. My code was fine...

No, it wasn't - because in your prior code you did *not* set
the Charset-Property of the Stream-Object - and thus used
the default (which is "Unicode" aka "UTF16-LE" on Windows).

> ...until I tried yours with the utf-8. It then started saving 
> as unicode, like you said. 

Arrgh, could you please stick to the truth in a public NewsGroup.
Such "face-saving excuses" are exactly, how "misleading myths"
will take root.

> After some searching online it seems that ADODB  
> 1) sniffs text and decides what it should be and 
> 2) maintains some kind of memory as to its default format.

Nope, the ADO.Stream Object does no such thing.
It was (as said above), just using it's default-charset,
which you failed to specify explicitely (in both -
the read- and write-directions).

>   My resulting speeds for the 25 MB file are .75 and 1.0.
> Pretty close to the original. With your updated code I'm
> getting .95 and 1.51. 

I've done such a test now (after finding a 26MB-file here),
and the speeds were - as expected - identical (using my -
as well as your updated versions).

>    So the only real difference is that my code is pretty
> and easy to read, while yours is confusing and needs
> editing before use. :)

No - you can ridicule my coding-style as you like -
but that does not change the fact, that your code (still)
is a perfect example for "spaghetti", seriously.

Really - it's not the coding-style which decides about
"spaghetti" or not, it's whether you applied "modularization",
KISS- and DRY-principles.

On my WebServer I use these VBScript-functions I've posted
via an Include-File (which ASP supports, but the WSH sadly not).

And thus my "UserCode" (the one that I've had to type in my Editor)
is really only this one here (leaving out the Timing-Code):

'--------------------
Dim File, bInp, sB64
     File = WScript.Arguments(0)
     bInp = ReadBytesFromFile(File)

If MsgBox("Click yes to encode (no to decode)", vbYesNo) = vbYes Then
     sB64 = Base64Encode(bInp)
     WriteBytesToFile File & "-64.txt", StringToBytes(sB64, "x-ansi")
Else
     sB64 = BytesToString(bInp, "x-ansi")
     WriteBytesToFile File & "-64.dat", Base64Decode(sB64)
End If
'------------------

Furthermore, my (relatively unnecessary) fix for a bit more speed,
was only applied in a single small Function, which is also
documenting the functionality of its  handful of Lines over
its given Function-name (also later in UserCode, by using that Name).

All the other UserCode (spread over dozens of other *.asp-Files),
will now automatically profit from that little speed-up-change,
because it was done on an *include-file*.

Whereas in such non-generic code as yours, you had to actually
make a change in *two* (much harder to find) places, to fix "a bug".

And no other CodeModule of yours will profit from that change
(who knows, where else you used such non-function-encapsulated stuff).

> But it is discouraging that ADODB is as undependable
> in its mangling of text as FSO is. 

As already said, such statements will only create further myths
in the community - nobody really needs or wants that...

Please run the following Single-Line-Script:
MsgBox CreateObject("ADODB.Stream").Charset

It will answer with: "Unicode"
(which is synonymous with "UTF16-LE" - and this will be reported
also on machines with an english locale - I've just checked that here)

Olaf

[toc] | [prev] | [next] | [standalone]

#12242

   I must say I'm glad we had this debate. I had no
idea that ADODB might pull tricks with text formatting
and might never have noticed if you hadn't had trouble
with my script.

[toc] | [prev] | [next] | [standalone]

#12244

Am 22.09.2019 um 21:10 schrieb Mayayana:
>     I must say I'm glad we had this debate. I had no
> idea that ADODB might pull tricks with text formatting
> and might never have noticed if you hadn't had trouble
> with my script.

Just another cleanup-sweep... before myths evolve...

Please run the following Single-Line-Script:
MsgBox CreateObject("ADODB.Stream").Charset

It will answer with: "Unicode"
(which is synonymous with "UTF16-LE" - and this will be reported
also on machines with an english locale - I've just checked that here)

Olaf

[toc] | [prev] | [next] | [standalone]

#12238

"Schmidt" <ng@vbRichClient.com> wrote

| With just the two additional Helpers I've mentioned
| (StringToBytes and BytesToString) you'd have all you need,
| to replicate your Script with this short(er) code:
|

  I didn't realize until I re-read it that you actually had
posted a working script. Thanks. But I don't see why you think
it's better. All those external functions seem clunky to me.
If I want base-64 code that's portable then it's easy enough
to put mine in a class.

  But I think that's really just a matter of personal preference.
I also don't like the method of mashing multiple operations
together in order to have less lines of code. It's hard to read.
Things like With CreateObject(... just encourage bad coding,
while providing no advantage. And it leaves to option to
explicitly dereference.

  But I know some people prefer it. There's no accounting
for taste. :)

 Speed:

  On a 25 MB file my code encodes
in .7 seconds and decodes in .8 seconds. Yours, with
the same file, takes 1.0 and 5.28 seconds respectively.
The lag is mostly in BytesToString. You didn't try this code,
I gather?

  I tried taking the loop out of BytesToString, thinking
that maybe ADODB wouldn't be so slow converting its
own stream to a string. But that was horrendous. On
the 25 MB file I finally just killed the process after
a couple of minutes. 

[toc] | [prev] | [next] | [standalone]

#12231

On Fri, 20 Sep 2019 21:44:17 +0200, Schmidt wrote:
> 
> Not sure, why there's so much "confusion" about this
> (and why one should read binary FileData into a String first).
> 
> The modus-operandi with Base64 is Binary:
> - passed as Input (as ByteArray) into the enconder
> - and returned as Output-(ByteArray) from the decoder
> 
> Here's some (pretty symmetrical) Helpers,
> which do as they should in my *.asp-Scripts (on Win2008/Win2012 and 
> Win2016):

Thanks. I guess I failed to realize that ADODB.Stream is much better suited
for handling binary data.

[toc] | [prev] | [standalone]

Page 3 of 3 — ← Prev page 1 2 [3]

Back to top | Article view | microsoft.public.scripting.vbscript

csiph-web