Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed2.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <python-python-list@m.gmane.org>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.000
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'binary': 0.07; 'column': 0.07; 'postgresql': 0.07; 'python:': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'strings.': 0.09; 'python': 0.11; '(1,': 0.16; '(code': 0.16; 'cleaner': 0.16; 'compares': 0.16; 'object)': 0.16; 'received:80.91.229.3': 0.16; 'received:plane.gmane.org': 0.16; 'storing': 0.16; 'subject:Problem': 0.16; 'appropriate': 0.16; 'first.': 0.19; 'solution.': 0.20; '>>>': 0.22; 'hack': 0.22; 'byte': 0.24; 'string,': 0.24; 'versions': 0.24; 'question': 0.24; 'compare': 0.26; 'values': 0.27; 'header:X-Complaints-To:1': 0.27; 'xml': 0.29; 'code': 0.31; "skip:' 10": 0.31; 'int,': 0.31; 'anyone': 0.31; 'table': 0.34; 'problem': 0.35; 'subject:with': 0.35; 'equal': 0.35; 'but': 0.35; 'add': 0.35; 'false': 0.36; 'server': 0.38; 'to:addr:python-list': 0.38; 'list,': 0.38; 'that,': 0.38; 'does': 0.39; 'to:addr:python.org': 0.39; 'received:org': 0.40; 'more': 0.64; 'here': 0.66; 'frank': 0.68
X-Injected-Via-Gmane: http://gmane.org/
To: python-list@python.org
From: "Frank Millman" <frank@chagford.com>
Subject: Problem with psycopg2, bytea, and memoryview
Date: Wed, 31 Jul 2013 11:44:01 +0200
X-Gmane-NNTP-Posting-Host: 197.87.30.34
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.3790.4657
X-RFC2646: Format=Flowed; Original
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.4913
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.4.1375263849.1251.python-list@python.org>
Lines: 43
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1375263849 news.xs4all.nl 15891 [2001:888:2000:d::a6]:42512
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:51635

Hi all

I don't know if this question is more appropriate for the psycopg2 list, but 
I thought I would ask here first.

I have some binary data (a gzipped xml object) that I want to store in a 
database. For PostgreSQL I use a column with datatype 'bytea', which is 
their recommended way of storing binary strings.

I use psycopg2 to access the database. It returns binary data in the form of 
a python 'memoryview'.

My problem is that, after a roundtrip to the database and back, the object 
no longer compares equal to the original.

>>> memoryview(b'abcdef') == b'abcdef'
True
>>> cur.execute('create table fmtemp (code int, xml bytea)')
>>> cur.execute('insert into fmtemp values (%s, %s)', (1, b'abcdef'))
>>> cur.execute('select * from fmtemp where code =1')
>>> row = cur.fetchone()
>>> row
(1, <memory at 0xb725f77c>)
>>> row[1] == b'abcdef'
False
>>> row[1].tobytes() == b'abcdef'
True
>>>

Using MS SQL Server and pyodbc, it returns a byte string, not a memoryview, 
and it does compare equal with the original.

I can hack my program to use tobytes(), but it would add complication, and 
it would be database-specific. I would prefer a cleaner solution.

Does anyone have any suggestions?

Versions - Python: 3.3.2  PostgreSQL: 9.2.4  psycopg2: 2.5

Frank Millman