Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #257642

Re: sha256sum --text generating blank spaces and hyphens?

Show all headers | View raw

On 4/26/23, David Wright <deblis@lionunicorn.co.uk> wrote:
> I guess you need the expense of sha256 rather than md5 as you're
> downloading the entire web?

 I am not downloading the entire web. I have no way of knowing how
they entertained those ideations but I think we could use their
estimate when google said that approximately 1 million and a half
books have been ever published. Think of it! It is not that much data.
It would all fit nicely in one hard drive; include some searching
capability and "bye bye google" will be the name of your movie. At
times you need to gain a sense of things before going into exposed
mode to search for something (which these days means making sure you
are not being baited into something else)

On 4/26/23, Dan Ritter <dsr@randomstring.org> wrote:
> The only characters used in the sha256 hash itself are [a-f] and
> [0-9]

 Yes, I knew that; that is why I could not understand why sha256sum
was being "courteous" to me.

On 4/26/23, Nicolas George <george@nsup.org> wrote:
> shaXsum always writes X/4 hexadecimal nibbles then two spaces then the
> file name. If the input is from stdin, then the convention is the file
> name is ‘-’.
>
> (Well, not always always: if the file name contains very special
> characters, it will use an escaped output format. And there is the -z
> option.)

On 4/26/23, Thomas Schmitt <scdbackup@gmx.net> wrote:
> "FILE" is the minus-sign for standard input. The second blank is there
> to indicate the text mode of sha256sum.
> Only the first blank is somewhat puzzling. But it's always there.
>
>
> https://www.gnu.org/software/coreutils/manual/html_node/sha2-utilities#sha2-utilities
> points to
>
> https://www.gnu.org/software/coreutils/manual/html_node/md5sum-invocation.html
> which says
>   For each file, ‘md5sum’ outputs by default, the MD5 checksum, a space,
>   a flag indicating binary or text input mode, and the file name. Binary
>   mode is indicated with ‘*’, text mode with ‘ ’ (space). Binary mode is
>   the default on systems where it’s significant, otherwise text mode is
>   the default. The cksum command always uses binary mode and a ‘ ’
>   (space) flag.
>
> So the first blank can be relied on and thus the proposal by Andy Smith
> to use "awk '{print $1}'" is valid.

 OK, now I see why cutting off the string on the first space that
appears is safe. I never saw such cases because I always used sha*sums
on files. I would expect if a user enters a string via printf that was
all there was to it. Of course, sha*sums can tell apart a file from a
text string.

On 4/26/23, Jeffrey Walton <noloader@gmail.com> wrote:
> There's no guarantee a URL will map onto a filesystem.

> I seem to
> recall Stunnel tried to do that in a caching mode, but it had weird
> corner cases. (In addition to problems with filesystems that had
> character set and path limitations).

 Well, no; and I am fine with:
 a) trying to best match both; the URL path as best as possible
 b) the extra malabarism base64-ing and hashsing the name of the file ...

 Something I have learned as a corpora research kind of guy is not to
ever try to "educate" people. I would just take their sh!t as they
dump it and cleanse, deal with it!

 You would not hear the end of it if I start telling stories of the
kind of cr@p you find out there when you look at the web from that
point of view.

> I think your best bet is to digest the URL into a representation. I
> suggest using SipHash+Base64 or Base64URL. SipHash provides collision
> resistance, a uniform distribution, and its fast. SipHash has a very
> good pedigree since it was designed by Jean-Philippe Aumasson and
> Daniel J. Bernstein. The final Base64 or Base64URL encoding ensures
> you stay within printable character range without reserved file system
> characters.

 Thank you I will look into what they did when I get a chance,

 lbrtchx


On 4/26/23, Albretch Mueller <lbrtchx@gmail.com> wrote:
> On 4/26/23, David Wright <deblis@lionunicorn.co.uk> wrote:
>> I guess you need the expense of sha256 rather than md5 as you're
>> downloading the entire web?
>
>  I am not downloading the entire web. I have no way of knowing how
> they entertained those ideations but I think we could use their
> estimate when they said that approximately 1 million and a half books
> have been ever published. Think of it! It is not that much data. It
> would all fit nicely in one hard drive include some searching
> capability and "bye bye google" will be the name of your movie.
>
> On 4/26/23, Dan Ritter <dsr@randomstring.org> wrote:
>> The only characters used in the sha256 hash itself are [a-f] and
>> [0-9]
>
>  Yes, I knew that; that is why I could not understand why sha256sum
> was being "courteous" to me.
>
> On 4/26/23, Nicolas George <george@nsup.org> wrote:
>> shaXsum always writes X/4 hexadecimal nibbles then two spaces then the
>> file name. If the input is from stdin, then the convention is the file
>> name is ‘-’.
>>
>> (Well, not always always: if the file name contains very special
>> characters, it will use an escaped output format. And there is the -z
>> option.)
>
> On 4/26/23, Thomas Schmitt <scdbackup@gmx.net> wrote:
>> "FILE" is the minus-sign for standard input. The second blank is there
>> to indicate the text mode of sha256sum.
>> Only the first blank is somewhat puzzling. But it's always there.
>>
>>
>> https://www.gnu.org/software/coreutils/manual/html_node/sha2-utilities#sha2-utilities
>> points to
>>
>> https://www.gnu.org/software/coreutils/manual/html_node/md5sum-invocation.html
>> which says
>>   For each file, ‘md5sum’ outputs by default, the MD5 checksum, a space,
>>   a flag indicating binary or text input mode, and the file name. Binary
>>   mode is indicated with ‘*’, text mode with ‘ ’ (space). Binary mode is
>>   the default on systems where it’s significant, otherwise text mode is
>>   the default. The cksum command always uses binary mode and a ‘ ’
>>   (space) flag.
>>
>> So the first blank can be relied on and thus the proposal by Andy Smith
>> to use "awk '{print $1}'" is valid.
>
>  OK, now I see why cutting off the string on the first space that
> appears is safe. I never saw such cases because I always used sha*sums
> on files. I would expect if a user enters a string via printf that was
> all there was to it. Of course, sha*sums can tell apart a file from a
> string a plain text.
>
> On 4/26/23, Jeffrey Walton <noloader@gmail.com> wrote:
>> There's no guarantee a URL will map onto a filesystem.
>
>> I seem to
>> recall Stunnel tried to do that in a caching mode, but it had weird
>> corner cases. (In addition to problems with filesystems that had
>> character set and path limitations).
>
>  Well, no; and I am fine with:
>  a) trying to best match both; the URL path as best as possible
>  b) the extra malabarism base64-ing and hashsing the name of the file ...
>
>  Something I have learned as a corpora research kind of guy is not to
> ever try to "educate" people. I would just take their sh!t as they
> dump it and cleanse, deal with it!
>
>  You would not hear the end of it if I start telling stories of the
> kind of cr@p you find out there when you look at the web from that
> point of view: from folks at archive.org who would list: "Henry
> Valentine Miller", "Henry V. Miller", "Henry Miller", "henry miller",
> "Miller, Henry", "Miller, Henry 12-1891 06-1980" apparently as
> different authors/"creators", to the gutenberb.org large text bank
> including some protagonistic bs in the actual texts, to developers of
> libreoffice watermarking text with some cr@p which of course is being
> used for "monitoring" purposes by the kinds of folks who put
> "intelligence" in the names of the organizations they work for and to
> make sure they are making sense they put flags around them when they
> fart through their mouths whatever nonsense they think of.
>
>  I had had rehearsing day dreams about becoming a dictator of the
> world ;-) and making people do "the right thing" (tm) ... until I had
> once an epiphany while watching Trump talk to a media prestitude who
> caracteristically wasn't making much sense. After asking a few
> questions trying to make sense of what she was saying, prestitude said
> "let me formulate it better". Trump quietly sat back saying: "OK, take
> your time"!!!
>
>  I was amazed! There you have someone the U.S. media, who as a mouth
> piece of the status quo, were being viscerally offensive towards
> anything relating to him, including posting on the front page of
> mainstream US news papers naked pictures of his wife and mother of his
> child one month before she became "the first lady" and he took it
> easy, respectfully on her! That was the best case I have noticed so
> far of "separating the message from the messenger". I mean people who
> erect all those pay walls and somehow see themselves as authoring,
> guarding content are not even the messengers and we all have to put up
> with their bs.
>
>> I think your best bet is to digest the URL into a representation. I
>> suggest using SipHash+Base64 or Base64URL. SipHash provides collision
>> resistance, a uniform distribution, and its fast. SipHash has a very
>> good pedigree since it was designed by Jean-Philippe Aumasson and
>> Daniel J. Bernstein. The final Base64 or Base64URL encoding ensures
>> you stay within printable character range without reserved file system
>> characters.
>
>  Thank you I will look into what they did when I get a chance,
>
>  lbrtchx
>

Back to linux.debian.user | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

sha256sum --text generating blank spaces and hyphens? Albretch Mueller <lbrtchx@gmail.com> - 2023-04-26 09:50 +0200
  Re: sha256sum --text generating blank spaces and hyphens? <tomas@tuxteam.de> - 2023-04-26 10:20 +0200
    Re: sha256sum --text generating blank spaces and hyphens? Albretch Mueller <lbrtchx@gmail.com> - 2023-04-26 16:40 +0200
      Re: sha256sum --text generating blank spaces and hyphens? Nicolas George <george@nsup.org> - 2023-04-26 16:50 +0200
        Re: sha256sum --text generating blank spaces and hyphens? Max Nikulin <manikulin@gmail.com> - 2023-04-28 04:30 +0200
          Re: sha256sum --text generating blank spaces and hyphens? Nicolas George <george@nsup.org> - 2023-04-28 10:10 +0200
            Re: sha256sum --text generating blank spaces and hyphens? Max Nikulin <manikulin@gmail.com> - 2023-04-28 18:50 +0200
              Re: sha256sum --text generating blank spaces and hyphens? Nicolas George <george@nsup.org> - 2023-04-29 11:00 +0200
                Re: sha256sum --text generating blank spaces and hyphens? Max Nikulin <manikulin@gmail.com> - 2023-04-29 14:20 +0200
                Re: sha256sum --text generating blank spaces and hyphens? Nicolas George <george@nsup.org> - 2023-04-29 14:30 +0200
      Re: sha256sum --text generating blank spaces and hyphens? Dan Ritter <dsr@randomstring.org> - 2023-04-26 17:10 +0200
      Re: sha256sum --text generating blank spaces and hyphens? David Wright <deblis@lionunicorn.co.uk> - 2023-04-26 17:10 +0200
      Re: sha256sum --text generating blank spaces and hyphens? "Thomas Schmitt" <scdbackup@gmx.net> - 2023-04-26 17:10 +0200
      Re: sha256sum --text generating blank spaces and hyphens? <tomas@tuxteam.de> - 2023-04-26 19:00 +0200
        Re: sha256sum --text generating blank spaces and hyphens? Albretch Mueller <lbrtchx@gmail.com> - 2023-04-26 22:40 +0200
          Re: sha256sum --text generating blank spaces and hyphens? Albretch Mueller <lbrtchx@gmail.com> - 2023-04-26 23:10 +0200
          Re: sha256sum --text generating blank spaces and hyphens? Andy Smith <andy@strugglers.net> - 2023-04-26 23:50 +0200
  Re: sha256sum --text generating blank spaces and hyphens? Andy Smith <andy@strugglers.net> - 2023-04-26 12:50 +0200
  Re: sha256sum --text generating blank spaces and hyphens? Jeffrey Walton <noloader@gmail.com> - 2023-04-26 18:00 +0200
  Re: sha256sum --text generating blank spaces and hyphens? David Christensen <dpchrist@holgerdanske.com> - 2023-04-27 00:50 +0200
    Re: sha256sum --text generating blank spaces and hyphens? Nicolas George <george@nsup.org> - 2023-04-27 00:50 +0200
      Re: sha256sum --text generating blank spaces and hyphens? David Christensen <dpchrist@holgerdanske.com> - 2023-04-27 06:10 +0200
        Re: sha256sum --text generating blank spaces and hyphens? Max Nikulin <manikulin@gmail.com> - 2023-04-27 08:00 +0200
          Re: sha256sum --text generating blank spaces and hyphens? Albretch Mueller <lbrtchx@gmail.com> - 2023-04-27 09:30 +0200
        Re: sha256sum --text generating blank spaces and hyphens? Nicolas George <george@nsup.org> - 2023-04-27 10:10 +0200
          Re: sha256sum --text generating blank spaces and hyphens? David Christensen <dpchrist@holgerdanske.com> - 2023-04-27 17:20 +0200
            Re: sha256sum --text generating blank spaces and hyphens? Albretch Mueller <lbrtchx@gmail.com> - 2023-05-02 02:10 +0200
    Re: sha256sum --text generating blank spaces and hyphens? Albretch Mueller <lbrtchx@gmail.com> - 2023-04-27 01:30 +0200
      Re: sha256sum --text generating blank spaces and hyphens? David Christensen <dpchrist@holgerdanske.com> - 2023-04-27 06:30 +0200

csiph-web