Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #257651

Re: sha256sum --text generating blank spaces and hyphens?

From David Christensen <dpchrist@holgerdanske.com>
Newsgroups linux.debian.user
Subject Re: sha256sum --text generating blank spaces and hyphens?
Date 2023-04-27 06:10 +0200
Message-ID <Gp17b-4NGa-1@gated-at.bofh.it> (permalink)
References <GoI4y-4BMS-5@gated-at.bofh.it> <GoI4y-4BMS-3@gated-at.bofh.it> <GoW7v-4KvJ-1@gated-at.bofh.it> <GoW7v-4KvJ-7@gated-at.bofh.it>
Organization linux.* mail to news gateway

Show all headers | View raw

On 4/26/23 15:48, Nicolas George wrote:
> David Christensen (12023-04-26):
>> I suggest hashing the document content rather than the URL.  This would work
>> nicely for static documents.
> 
> That will be very convenient to retrieve the document content from the
> URL.


My suggestion assumes that the URL => hash => content mapping is saved 
somehow.  For example, save the content in a file named after the hash 
and save the URL in a file whose name is the hash plus a suffix. 
Finding a document by URL then becomes a grep(1) invocation.


Things get more interesting when you approach the problem as a database. 
  Save the content wherever and put the metadata into a table -- content 
hash (primary key), URL, download timestamp, author, subject, title, 
keywords, etc..  Create fully inverted indexes.  Create a search engine. 
  Create a spider.  Implementation could range from a CSV/TSV flat-file 
and shell/P* scripts, to a desktop database/UI, to a LAMP stack, and 
beyond (NoSQL, N-tier).  There are distributed file sharing systems 
based on such ideas.


David

Back to linux.debian.user | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

sha256sum --text generating blank spaces and hyphens? Albretch Mueller <lbrtchx@gmail.com> - 2023-04-26 09:50 +0200
  Re: sha256sum --text generating blank spaces and hyphens? <tomas@tuxteam.de> - 2023-04-26 10:20 +0200
    Re: sha256sum --text generating blank spaces and hyphens? Albretch Mueller <lbrtchx@gmail.com> - 2023-04-26 16:40 +0200
      Re: sha256sum --text generating blank spaces and hyphens? Nicolas George <george@nsup.org> - 2023-04-26 16:50 +0200
        Re: sha256sum --text generating blank spaces and hyphens? Max Nikulin <manikulin@gmail.com> - 2023-04-28 04:30 +0200
          Re: sha256sum --text generating blank spaces and hyphens? Nicolas George <george@nsup.org> - 2023-04-28 10:10 +0200
            Re: sha256sum --text generating blank spaces and hyphens? Max Nikulin <manikulin@gmail.com> - 2023-04-28 18:50 +0200
              Re: sha256sum --text generating blank spaces and hyphens? Nicolas George <george@nsup.org> - 2023-04-29 11:00 +0200
                Re: sha256sum --text generating blank spaces and hyphens? Max Nikulin <manikulin@gmail.com> - 2023-04-29 14:20 +0200
                Re: sha256sum --text generating blank spaces and hyphens? Nicolas George <george@nsup.org> - 2023-04-29 14:30 +0200
      Re: sha256sum --text generating blank spaces and hyphens? Dan Ritter <dsr@randomstring.org> - 2023-04-26 17:10 +0200
      Re: sha256sum --text generating blank spaces and hyphens? David Wright <deblis@lionunicorn.co.uk> - 2023-04-26 17:10 +0200
      Re: sha256sum --text generating blank spaces and hyphens? "Thomas Schmitt" <scdbackup@gmx.net> - 2023-04-26 17:10 +0200
      Re: sha256sum --text generating blank spaces and hyphens? <tomas@tuxteam.de> - 2023-04-26 19:00 +0200
        Re: sha256sum --text generating blank spaces and hyphens? Albretch Mueller <lbrtchx@gmail.com> - 2023-04-26 22:40 +0200
          Re: sha256sum --text generating blank spaces and hyphens? Albretch Mueller <lbrtchx@gmail.com> - 2023-04-26 23:10 +0200
          Re: sha256sum --text generating blank spaces and hyphens? Andy Smith <andy@strugglers.net> - 2023-04-26 23:50 +0200
  Re: sha256sum --text generating blank spaces and hyphens? Andy Smith <andy@strugglers.net> - 2023-04-26 12:50 +0200
  Re: sha256sum --text generating blank spaces and hyphens? Jeffrey Walton <noloader@gmail.com> - 2023-04-26 18:00 +0200
  Re: sha256sum --text generating blank spaces and hyphens? David Christensen <dpchrist@holgerdanske.com> - 2023-04-27 00:50 +0200
    Re: sha256sum --text generating blank spaces and hyphens? Nicolas George <george@nsup.org> - 2023-04-27 00:50 +0200
      Re: sha256sum --text generating blank spaces and hyphens? David Christensen <dpchrist@holgerdanske.com> - 2023-04-27 06:10 +0200
        Re: sha256sum --text generating blank spaces and hyphens? Max Nikulin <manikulin@gmail.com> - 2023-04-27 08:00 +0200
          Re: sha256sum --text generating blank spaces and hyphens? Albretch Mueller <lbrtchx@gmail.com> - 2023-04-27 09:30 +0200
        Re: sha256sum --text generating blank spaces and hyphens? Nicolas George <george@nsup.org> - 2023-04-27 10:10 +0200
          Re: sha256sum --text generating blank spaces and hyphens? David Christensen <dpchrist@holgerdanske.com> - 2023-04-27 17:20 +0200
            Re: sha256sum --text generating blank spaces and hyphens? Albretch Mueller <lbrtchx@gmail.com> - 2023-05-02 02:10 +0200
    Re: sha256sum --text generating blank spaces and hyphens? Albretch Mueller <lbrtchx@gmail.com> - 2023-04-27 01:30 +0200
      Re: sha256sum --text generating blank spaces and hyphens? David Christensen <dpchrist@holgerdanske.com> - 2023-04-27 06:30 +0200

csiph-web