Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!goblin1!goblin.stu.neva.ru!uio.no!news.tele.dk!news.tele.dk!small.news.tele.dk!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <drsalists@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.002
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'messages.': 0.05; 'tree': 0.05; 'binary': 0.07; 'complicate': 0.09; 'filenames': 0.09; 'subject:files': 0.09; 'cc:addr:python-list': 0.11; 'python': 0.11; '"is': 0.16; 'caches': 0.16; 'cc:name:python list': 0.16; 'else"': 0.16; 'erroneously': 0.16; 'folder.': 0.16; 'loop.': 0.16; 'maildir': 0.16; 'matters,': 0.16; 'numbered': 0.16; 'os.walk': 0.16; 'preceded': 0.16; 'script,': 0.16; 'subject:between': 0.16; 'text",': 0.16; 'to:addr:python.list': 0.16; 'to:addr:tim.thechases.com': 0.16; 'to:name:tim chase': 0.16; 'wrote:': 0.18; 'aug': 0.22; 'cc:addr:python.org': 0.22; '31,': 0.24; 'directory.': 0.24; 'fine': 0.24; 'file.': 0.24; 'cc:2**0': 0.24; 'subject:/': 0.26; 'header:In-Reply-To:1': 0.27; 'am,': 0.29; 'possibility': 0.29; 'reporting': 0.29; 'tim': 0.29; 'message-id:@mail.gmail.com': 0.30; 'easier': 0.31; 'too.': 0.31; 'chase': 0.31; 'flags': 0.31; 'them?': 0.31; 'file': 0.32; 'this.': 0.32; 'probably': 0.32; "i'd": 0.34; 'could': 0.34; 'knows': 0.35; 'something': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'there': 0.35; 'c++': 0.36; 'data,': 0.36; 'false': 0.36; 'subject:?': 0.36; 'being': 0.38; 'sometimes': 0.38; 'handle': 0.38; 'files': 0.38; 'little': 0.38; 'structure': 0.39; 'how': 0.40; 'tell': 0.60; 'full': 0.61; 'such': 0.63; 'myself': 0.63; 'more': 0.64; 'holding': 0.65; 'walk': 0.74; '11:45': 0.84; 'alongside': 0.84; 'directories,': 0.84; 'distinguish': 0.84; 'much,': 0.84; 'negatives,': 0.84
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=kPu9dyII4ziFpaCFRV1/fqkLwSFNjuz+QFP2q/zDKDk=; b=TapGIXL0eczzNNsShMNK3bPvNQCBNKnKygb6fEVNxhXNGGpzR3RbOEpl/nG+ob3Ov9 bn8ySK5SAqiFgYD27r2nttQuV77BoqZQ/wiQQxEKo373X3WQ1+1I1DlEwFMRn4ZRHs7j g2PoQWXt9dZKoJXRnK8AxeBVSiswSNyjkDXmQWxxv139rzmIMujBbBP3lEtG3NKNGf/6 k23pSC20KYO3hHf+EDyUe+Q59JsyO+qKV2YgdikvbDO+ruvaNYMt6V9DRvTdZx149bQ+ 95twtIl/a/lRSxL9s1Yx700KR1eWPHQtq38jv/adAEO/RKVNq9M5s015QhVYNR62VxEp vfcw==
MIME-Version: 1.0
X-Received: by 10.112.60.33 with SMTP id e1mr22747159lbr.36.1409518747846; Sun, 31 Aug 2014 13:59:07 -0700 (PDT)
In-Reply-To: <20140831134525.25a4321e@bigbox.christie.dr>
References: <20140831134525.25a4321e@bigbox.christie.dr>
Date: Sun, 31 Aug 2014 13:59:07 -0700
Subject: Re: Distinguishing between maildir, mbox, and MH files/directories?
From: Dan Stromberg <drsalists@gmail.com>
To: Tim Chase <python.list@tim.thechases.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Cc: Python List <python-list@python.org>
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.13664.1409518750.18130.python-list@python.org>
Lines: 35
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1409518750 news.xs4all.nl 2894 [2001:888:2000:d::a6]:50112
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:77355

On Sun, Aug 31, 2014 at 11:45 AM, Tim Chase
<python.list@tim.thechases.com> wrote:
> Tinkering around with a little script, I found myself with the need
> to walk a directory tree and process mail messaged found within.
> Sometimes these end up being mbox files (with multiple messages
> within), sometimes it's a Maildir structure with messages in each
> individual file and extra holding directories, and sometimes it's a
> MH directory.  To complicate matters, there's also the possibility of
> non-{mbox,maildir,mh) files such as binary MUA caches appearing
> alongside these messages.
>
> Python knows how to handle each just fine as long as I tell it what
> type of file to expect.  But is there a straight-forward way to
> distinguish them?  (FWIW, the *nix "file" utility is just reporting
> "ASCII text", sometimes "with very long lines", and sometimes
> erroneously flags them as C or C++ files=E2=80=BD).
>
> All I need is "is it maildir, mbox, mh, or something else" (I don't
> have to get more complex for the "something else") inside an os.walk
> loop.

If you find a directory full of numbered files (and optionally,
numbered filenames preceded by commas), that's probably an MH folder.
I don't like regexes that much, but I'd probably use one for this.

If you find a directory full of Maildir-style files, that's probably
Maildir.  You could probably match this with a regex too.

If you find a file with lots of '^From " in it, that's probably an
mbox file.  However, you could have an mbox file with only one '^From
', so watch out.

This will probably give some false postives and/or false negatives,
depending on your data, but perhaps it's easier than classifying
things manually.