Path: csiph.com!news.swapon.de!newsfeed.fsmpi.rwth-aachen.de!newsfeed.straub-nv.de!feeder.erje.net!1.eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed7.news.xs4all.nl!nzpost1.xs4all.net!not-for-mail
Return-Path: <random832@fastmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.000
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'subject:: [': 0.03; 'encoded': 0.05; '21,': 0.07; 'bytes.': 0.07; 'constructor': 0.07; 'present,': 0.07; 'valueerror:': 0.07; 'cc:addr:python-list': 0.09; 'bytes,': 0.09; 'encode': 0.09; 'received:internal': 0.09; 'def': 0.13; 'ignore': 0.14; 'section.': 0.15; '39,': 0.16; 'alexander': 0.16; 'bit.': 0.16; 'c():': 0.16; 'compatible.': 0.16; 'optionally': 0.16; 'printout': 0.16; 'received:10.202': 0.16; 'received:10.202.2': 0.16; 'received:66.111': 0.16; 'received:66.111.4': 0.16; 'received:messagingengine.com': 0.16; 'saying.': 0.16; 'versatile': 0.16; 'later': 0.16; 'string': 0.17; 'byte': 0.18; 'bytes': 0.18; 'accepting': 0.18; 'subject:] ': 0.19; '>>>': 0.20; 'versions': 0.20; 'cc:addr:python.org': 0.20; 'cc:2**2': 0.20; 'saying': 0.22; '"",': 0.22; 'subject:skip:i 10': 0.22; 'pass': 0.22; 'bit': 0.23; '(or': 0.23; 'originally': 0.23; '(most': 0.24; 'cc:addr:gmail.com': 0.24; 'tim': 0.24; 'written': 0.24; 'header:In-Reply-To:1': 0.24; 'header:User-Agent:1': 0.26; "doesn't": 0.26; 'question': 0.27; '14,': 0.27; 'object,': 0.27; "skip:' 10": 0.28; 'actual': 0.28; 'pickle': 0.29; 'that.': 0.30; 'class.': 0.30; 'option': 0.31; "can't": 0.32; 'skip:_ 10': 0.32; 'changed': 0.33; 'class': 0.33; 'right?': 0.33; 'traceback': 0.33; 'case,': 0.34; 'file': 0.34; 'previous': 0.34; 'skip:d 20': 0.34; 'skip:( 30': 0.35; 'could': 0.35; 'skip:p 30': 0.35; 'but': 0.36; 'should': 0.36; '(and': 0.36; 'subject:" ': 0.36; 'subject:?': 0.36; 'received:10': 0.37; 'itself': 0.38; 'received:66': 0.38; 'anything': 0.38; 'format': 0.39; 'does': 0.39; 'enough': 0.39; 'subject:-': 0.39; 'rather': 0.39; 'future': 0.60; 'care': 0.60; 'claim': 0.61; 'back': 0.62; 'subject:there': 0.66; 'talking': 0.67; 'hour': 0.69; 'study': 0.70; '_still_': 0.84; 'about,': 0.84; 'consisted': 0.84; 'subject:any': 0.84; 'subject:Are': 0.95
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.com; h=cc :content-type:date:from:in-reply-to:message-id:mime-version :references:subject:to:x-sasl-enc:x-sasl-enc; s=mesmtp; bh=+93Vj Fk0J6NHvvmxDLnihBHWypI=; b=D9hLqS/HYmQO5z64k5Gn7OgW1SX/5rzJI1OMh ckDYn2DWHcO26c2jdQOAky10UEfDIKoUdl37ErEoqcM0DLSi9GxlFQ0cVcVxopJq aaF0TcRvlZ8myWJMPMmj/l5gUTNkePq24lmMFsxnTDkMgnxo6wMR5wKI8OkNy/+A FFjVTo=
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-sasl-enc :x-sasl-enc; s=smtpout; bh=+93VjFk0J6NHvvmxDLnihBHWypI=; b=t0eCZ h3N2yFoZaTO5tngszPHUpbodJVb9kKRQqHOp6pyu4ukq47j57R4YOB8d6qyfytwU i8rcuRO+4OUWMLeFsrhuXMys4vWVQ54jTx5eKfztKRvyx4TVJg+22Pqu7n9y88Sg e5V82Jwgm6j/SZq7afG2c7G1XAAMoiAz1ulElk=
X-Sasl-enc: QpiH4z1S276GtgI80qrWaeJgffaA7KkvpYP73OExCkhw 1442282942
From: Random832 <random832@fastmail.com>
To: Alexander Belopolsky <alexander.belopolsky@gmail.com>
Cc: Tim Peters <tim.peters@gmail.com>, Python-List <python-list@python.org>, datetime-sig <datetime-sig@python.org>
Subject: Re: [Datetime-SIG] Are there any "correct" implementations of tzinfo?
References: <m2h9mzqyy7.fsf@fastmail.com> <201509131224.t8DCOXHO004891@fido.openend.se> <CAExdVNkN465=bWeY61PFAxpiUQ-u0p2zaqtwH8PQWyQO4Jw9Mg@mail.gmail.com> <201509131600.t8DG07e0025688@fido.openend.se> <CAExdVNm3e43mJ3tqcUc9175WssV4zeuO024svJbMTjrTab=Qew@mail.gmail.com> <201509132031.t8DKVTwJ028027@fido.openend.se> <CAExdVNkeRVgV8CXLugMgqhUSuXU=qHYSFUo24Xw83X=8tVBjCg@mail.gmail.com> <201509140827.t8E8RPqb001076@fido.openend.se> <CAExdVNn2wM8YW=Jg=aM86X6RaaVYPTioTSJ2d1gGp0k76CN3mg@mail.gmail.com> <1442257996.253100.383441705.7A0986C7@webmail.messagingengine.com> <CAExdVN=s_V_uz9mSOtp6b6+fKjqLZHArXKUQ-ty5EYkLM5V2qw@mail.gmail.com> <1442260714.263025.383475777.4728D768@webmail.messagingengine.com> <CAExdVN=OWrFxYiuNZrWuVHfeT74VyXaUOXLKj0Q1Ai1SoupqbQ@mail.gmail.com> <1442262425.268793.383506657.0443601E@webmail.messagingengine.com> <CAExdVNmNL4oiGWjokOmvKaWvqPdsp2kuGKYGJ-aERRB+irQU8A@mail.gmail.com> <1442265800.280460.383547057.16B65298@webmail.messagingengine.com> <CAExdVNnJG3-wz1fLtV2wLb+cbHQstc0Zh-KGxBFj6Q5CeqQpEw@mail.gmail.com> <1442267635.287083.383576201.0990DAA7@webmail.messagingengine.com> <CAExdVNkLmcCNHa20Y3NVe1w8U+GDXhxLZ8e9Zt74i0WAwi9iWA@mail.gmail.com> <1442279996.198469.383712497.36F9DE26@webmail.messagingengine.com> <EC9B5098-6691-455A-8251-9C3849F599E8@gmail.com>
Date: Mon, 14 Sep 2015 22:08:58 -0400
In-Reply-To: <EC9B5098-6691-455A-8251-9C3849F599E8@gmail.com> (Alexander Belopolsky's message of "Mon, 14 Sep 2015 21:42:00 -0400")
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (darwin)
MIME-Version: 1.0
Content-Type: text/plain
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.20+
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.581.1442282944.8327.python-list@python.org>
Lines: 49
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1442282944 news.xs4all.nl 23772 [2001:888:2000:d::a6]:54860
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:96615

Alexander Belopolsky <alexander.belopolsky@gmail.com> writes:

> No credit for anything other than the "extra credit" section.  Partial
> credit for that.  Study that printout and you should understand what
> Tim was saying.

My original claim was that the pickler can't know and doesn't care if a
byte string value merely happens to be 10 bytes long for this object, or
is 10 bytes long every time, it will encode it with SHORT_BINBYTES ["C",
count of bytes, string of bytes] regardless.

The datetime constructor itself does care what value is passed to it,
but what I was saying was the class could have been written originally
to accept optionally longer strings and ignore the extra values, so that
future versions could pickle as longer strings and be compatible.

In such a case, the actual pickle format would _still_ have consisted of
__reduce__() == (datetime, (b"..........", [optional tzinfo])), just
with the option of accepting (and ignoring) longer byte strings encoded
by later versions of the datetime class.

The pickle format is versatile enough to pass any (pickleable) value at
all to a constructor (or to __setstate__). Designing the datetime
constructor/setstate in the past to be able to accept a byte string of a
length other than exactly 10 would have allowed the representation to be
extended in the present, rather than smuggling a single extra bit into
one of the existing bytes. But it would not have changed the actual
representation that would have been produced by pickle back then, not
one bit.

And, now, to answer my own question from a previous message...
>>> class C():
...  def __reduce__(self):
...   return (datetime, (b"\x07\xdf\t\x0e\x155'\rA\xb2",))
...
>>> pickle.loads(pickle.dumps(C()))
datetime.datetime(2015, 9, 14, 21, 53, 39, 868786)
>>> class C():
...  def __reduce__(self):
...   return (datetime, (b"\x07\xdf\t\x0e\x955'\rA\xb2",))
...
>>> pickle.loads(pickle.dumps(C()))
datetime.datetime(2015, 9, 14, 149, 53, 39, 868786)
>>> datetime.strftime(pickle.loads(pickle.dumps(C())), '%Y%m%d%H%M%S')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: hour out of range

That was the bit we were talking about, right?