Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > linux.debian.user > #194469

Re: Invalid UTF-8 byte? (was: Re: utf)

From rhkramer@gmail.com
Newsgroups linux.debian.user
Subject Re: Invalid UTF-8 byte? (was: Re: utf)
Date 2018-04-04 03:20 +0200
Message-ID <vAFfH-7nW-1@gated-at.bofh.it> (permalink)
References <vzLQd-4cg-1@gated-at.bofh.it> <vAt4S-7NM-7@gated-at.bofh.it> <vAtod-7V4-15@gated-at.bofh.it>
Organization linux.* mail to news gateway

Show all headers | View raw


On Tuesday, April 03, 2018 08:30:04 AM Greg Wooledge wrote:
> WHAT ARE YOU TRYING TO DO?

I am building (have built several iterations) of a free format database to 
work something like askSam.  It is a mashup of several applications, things 
like recol, kmail, nail, kate and the data is stored in mbox formatted files.

Each record is treated as an email.

(Earlier iterations of the mashup used nedit, some nedit macros, and a custome 
file format using a 4 byte sequence (0x80 0x81 0x82 0x83) as the record 
separator.)

I occasionally want to sort the data in order by what I call the record title, 
which is stored in quotation marks ("") after the "From" in the mbox header.

I have seen some utilities that might sort the mbox files, but may require that 
I somehow move them into IMAP or some other manipulations that might be 
inconvenient.

msort was mentioned on this list within the last few months and might do the 
job for me if I can insert a temporary one byte record separator into the file 
(in addition to the mbox From line).  

Most likely this would be only a temporary addition, and I would need to do 
things like make sure that one byte will be unique in the file.  It sounds like 
there are at least a few candidates.


> 
> There was a glimpse a few messages back that looked like you were trying
> to parse information out of an mbox-format mail folder.  (I.e. a flat
> file that has a concatenated series of mbox-format mail messages in it,
> with all the silliness and problems inherent in this format, like having
> to prefix body lines with ">" if they begin with the word "From".)
> 
> "I want to write a shell script to parse an mbox folder..." is enough
> to send most people running away screaming.  What other horrors are we
> in store for next?
> 
> Of course, that might be a red herring, since you didn't actually tell
> us what your goal is, or what your inputs are, and we're having to
> guess at the moment based on tiny hints and information leaks.

Back to linux.debian.user | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread


Thread

utf mess-mate <mess-mate@gmx.com> - 2018-04-01 16:10 +0200
  Re: utf Curt <curty@free.fr> - 2018-04-01 17:50 +0200
  Re: utf Ionel Mugurel Ciobîcă <I.M.Ciobica@upcmail.nl> - 2018-04-01 18:30 +0200
  Re: utf Andre Majorel <aym-naibed@teaser.fr> - 2018-04-01 18:40 +0200
  Re: utf Ben Caradoc-Davies <ben@transient.nz> - 2018-04-01 22:10 +0200
    Re: utf Cindy-Sue Causey <butterflybytes@gmail.com> - 2018-04-02 00:50 +0200
      Re: utf Curt <curty@free.fr> - 2018-04-02 09:50 +0200
        Re: utf Richard Hector <richard@walnut.gen.nz> - 2018-04-02 11:40 +0200
        Re: utf Greg Wooledge <wooledg@eeg.ccf.org> - 2018-04-02 15:30 +0200
    Re: utf Andre Majorel <aym-naibed@teaser.fr> - 2018-04-02 09:50 +0200
      Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-02 14:40 +0200
        Re: Invalid UTF-8 byte? (was: Re: utf) <tomas@tuxteam.de> - 2018-04-02 15:10 +0200
        Re: Invalid UTF-8 byte? (was: Re: utf) Henrique de Moraes Holschuh <hmh@debian.org> - 2018-04-02 15:10 +0200
          Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-02 19:50 +0200
            Re: Invalid UTF-8 byte? (was: Re: utf) Henrique de Moraes Holschuh <hmh@debian.org> - 2018-04-02 20:20 +0200
              Re: Invalid UTF-8 byte? (was: Re: utf) <tomas@tuxteam.de> - 2018-04-02 20:50 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) tomas@tuxteam.de - 2018-04-02 21:00 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-02 21:10 +0200
        Re: Invalid UTF-8 byte? (was: Re: utf) Michael Lange <klappnase@freenet.de> - 2018-04-03 00:50 +0200
          Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-03 13:50 +0200
            Re: Invalid UTF-8 byte? (was: Re: utf) Nicolas George <george@nsup.org> - 2018-04-03 14:00 +0200
              Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-03 14:30 +0200
            Re: Invalid UTF-8 byte? (was: Re: utf) Michael Lange <klappnase@freenet.de> - 2018-04-03 14:10 +0200
              Re: Invalid UTF-8 byte? (was: Re: utf) Michael Lange <klappnase@freenet.de> - 2018-04-03 14:20 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) <tomas@tuxteam.de> - 2018-04-03 14:40 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) Michael Lange <klappnase@freenet.de> - 2018-04-03 21:50 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) Greg Wooledge <wooledg@eeg.ccf.org> - 2018-04-03 21:50 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) Michael Lange <klappnase@freenet.de> - 2018-04-03 22:10 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) Greg Wooledge <wooledg@eeg.ccf.org> - 2018-04-03 14:40 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-04 03:10 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-04 03:20 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) Nicolas George <george@nsup.org> - 2018-04-04 13:30 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-04 14:10 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) Nicolas George <george@nsup.org> - 2018-04-04 14:10 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-04 14:50 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) Nicolas George <george@nsup.org> - 2018-04-04 15:00 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) Andre Majorel <aym-naibed@teaser.fr> - 2018-04-04 16:20 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) Greg Wooledge <wooledg@eeg.ccf.org> - 2018-04-04 16:30 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-04 18:00 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-04 18:00 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) Jonathan de Boyne Pollard <J.deBoynePollard-newsgroups@NTLWorld.COM> - 2018-04-04 20:50 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) Greg Wooledge <wooledg@eeg.ccf.org> - 2018-04-04 14:30 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-04 15:10 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) Don Armstrong <don@debian.org> - 2018-04-04 19:40 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) Nicolas George <george@nsup.org> - 2018-04-04 19:50 +0200
                mbox vs maildir vs better formats [Re: Invalid UTF-8 byte? (was: Re:  utf)] Don Armstrong <don@debian.org> - 2018-04-04 20:00 +0200
                Re: mbox vs maildir vs better formats [Re: Invalid UTF-8 byte? (was:  Re: utf)] Nicolas George <george@nsup.org> - 2018-04-04 20:10 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-04 20:40 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) Don Armstrong <don@debian.org> - 2018-04-04 20:50 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-05 14:40 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) deloptes <deloptes@gmail.com> - 2018-04-05 00:20 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-04 21:30 +0200
              Re: Invalid UTF-8 byte? (was: Re: utf) Henrique de Moraes Holschuh <hmh@debian.org> - 2018-04-04 13:20 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) <tomas@tuxteam.de> - 2018-04-04 16:10 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) Henrique de Moraes Holschuh <hmh@debian.org> - 2018-04-04 20:50 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) <tomas@tuxteam.de> - 2018-04-04 21:30 +0200
                Re: Invalid UTF-8 byte? Ben Caradoc-Davies <ben@transient.nz> - 2018-04-04 23:50 +0200
                Re: Invalid UTF-8 byte? Michael Stone <mstone@debian.org> - 2018-04-05 01:00 +0200
                Re: Invalid UTF-8 byte? (was: Re: utf) Jonathan de Boyne Pollard <J.deBoynePollard-newsgroups@NTLWorld.COM> - 2018-04-04 21:00 +0200
        Re: Invalid UTF-8 byte? (was: Re: utf) Jonathan de Boyne Pollard <J.deBoynePollard-newsgroups@NTLWorld.COM> - 2018-04-04 20:30 +0200
          Re: Invalid UTF-8 byte? (was: Re: utf) rhkramer@gmail.com - 2018-04-04 21:30 +0200
      Re: utf Ben Caradoc-Davies <ben@transient.nz> - 2018-04-02 23:40 +0200
      Re: utf Darac Marjal <mailinglist@darac.org.uk> - 2018-04-03 11:00 +0200
        Re: utf Richard Hector <richard@walnut.gen.nz> - 2018-04-03 11:20 +0200
          Re: utf <tomas@tuxteam.de> - 2018-04-03 11:30 +0200
        Re: utf Nicolas George <george@nsup.org> - 2018-04-03 12:20 +0200
        Re: utf Ben Caradoc-Davies <ben@transient.nz> - 2018-04-03 22:50 +0200
          Re: utf Nicolas George <george@nsup.org> - 2018-04-03 23:00 +0200
            Re: utf Greg Wooledge <wooledg@eeg.ccf.org> - 2018-04-03 23:00 +0200
              Re: utf Nicolas George <george@nsup.org> - 2018-04-03 23:30 +0200
                Re: utf deloptes <deloptes@gmail.com> - 2018-04-04 19:00 +0200
                Re: utf Nicolas George <george@nsup.org> - 2018-04-04 19:10 +0200
                Re: utf Greg Wooledge <wooledg@eeg.ccf.org> - 2018-04-04 19:30 +0200
                Re: utf Nicolas George <george@nsup.org> - 2018-04-04 19:40 +0200
                Re: utf deloptes <deloptes@gmail.com> - 2018-04-04 19:40 +0200
                Re: utf Nicolas George <george@nsup.org> - 2018-04-04 19:40 +0200
                Re: utf Greg Wooledge <wooledg@eeg.ccf.org> - 2018-04-04 19:50 +0200
                Re: utf Nicolas George <george@nsup.org> - 2018-04-04 20:00 +0200
                Re: utf deloptes <deloptes@gmail.com> - 2018-04-04 23:20 +0200
                Re: utf Richard Hector <richard@walnut.gen.nz> - 2018-04-05 02:30 +0200
                Re: utf Nicolas George <george@nsup.org> - 2018-04-05 14:10 +0200
                Re: utf Stefan Monnier <monnier@iro.umontreal.ca> - 2018-04-05 00:00 +0200
                Re: utf rhkramer@gmail.com - 2018-04-04 20:30 +0200
                Re: utf Joel Roth <joelz@pobox.com> - 2018-04-04 21:30 +0200
                Re: utf deloptes <deloptes@gmail.com> - 2018-04-04 23:40 +0200
                Re: utf <tomas@tuxteam.de> - 2018-04-05 08:30 +0200
                Re: utf rhkramer@gmail.com - 2018-04-05 14:50 +0200
                Re: utf <tomas@tuxteam.de> - 2018-04-05 15:00 +0200
                Re: utf Nicolas George <george@nsup.org> - 2018-04-05 15:00 +0200
                Re: utf tomas@tuxteam.de - 2018-04-05 15:10 +0200
                Re: utf deloptes <deloptes@gmail.com> - 2018-04-05 21:40 +0200
                Re: utf Stefan Monnier <monnier@iro.umontreal.ca> - 2018-04-05 23:30 +0200
                Re: utf deloptes <deloptes@gmail.com> - 2018-04-05 23:40 +0200
                Re: utf Stefan Monnier <monnier@iro.umontreal.ca> - 2018-04-06 03:20 +0200
                Re: utf <tomas@tuxteam.de> - 2018-04-06 09:00 +0200
                Re: utf Ben Caradoc-Davies <ben@transient.nz> - 2018-04-06 04:20 +0200
                Re: utf <tomas@tuxteam.de> - 2018-04-06 09:10 +0200
                Re: utf Stefan Monnier <monnier@iro.umontreal.ca> - 2018-04-05 19:00 +0200
                Re: utf rhkramer@gmail.com - 2018-04-05 20:00 +0200
              Re: utf Stefan Monnier <monnier@iro.umontreal.ca> - 2018-04-03 23:40 +0200
        Re: utf Henrique de Moraes Holschuh <hmh@debian.org> - 2018-04-04 14:20 +0200

csiph-web