Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #102624 > unrolled thread

What's the best/neatest way to get Unicode data from a database into a grid cell?

Back to article view | Back to comp.lang.python

Contents

  What's the best/neatest way to get Unicode data from a database into a grid cell? cl@isbd.net - 2016-02-07 11:19 +0000
    Re: What's the best/neatest way to get Unicode data from a database into a grid cell? Chris Angelico <rosuav@gmail.com> - 2016-02-07 22:56 +1100
      Re: What's the best/neatest way to get Unicode data from a database into a grid cell? cl@isbd.net - 2016-02-07 12:42 +0000
        Re: What's the best/neatest way to get Unicode data from a database into a grid cell? Chris Angelico <rosuav@gmail.com> - 2016-02-08 00:11 +1100
          Re: What's the best/neatest way to get Unicode data from a database into a grid cell? cl@isbd.net - 2016-02-07 13:59 +0000
    Re: What's the best/neatest way to get Unicode data from a database into a grid cell? Dietmar Schwertberger <maillist@schwertberger.de> - 2016-02-07 16:06 +0100
      Re: What's the best/neatest way to get Unicode data from a database into a grid cell? cl@isbd.net - 2016-02-08 10:22 +0000
        Re: What's the best/neatest way to get Unicode data from a database into a grid cell? Chris Angelico <rosuav@gmail.com> - 2016-02-08 21:57 +1100
    Re: What's the best/neatest way to get Unicode data from a database into a grid cell? Vlastimil Brom <vlastimil.brom@gmail.com> - 2016-02-07 19:22 +0100
      Re: What's the best/neatest way to get Unicode data from a database into a grid cell? cl@isbd.net - 2016-02-08 10:42 +0000
        Re: What's the best/neatest way to get Unicode data from a database into a grid cell? Chris Angelico <rosuav@gmail.com> - 2016-02-08 21:55 +1100
        Re: What's the best/neatest way to get Unicode data from a database into a grid cell? Vlastimil Brom <vlastimil.brom@gmail.com> - 2016-02-08 16:00 +0100

#102624 — What's the best/neatest way to get Unicode data from a database into a grid cell?

I'm using this as a starting point for creating a grid to view and
edit some sqlite data:-
    http://www.salstat.com/news/linking-wxgrid-to-sqlite-database-in-python-an-example

I can actually understand most of it which is a good start.  

However my database has quite a lot of Unicode data as there are
French (and other) names with accents etc.  What's the right way to
handle this reasonably neatly?  At the moment it traps an error at
line 37:-
    self.SetCellValue(row_num, i, str(cells[i]))

I realise why this fails (I think) but how should one program this so
that:-
    1 - the accented characters are displayed correctly in the grid cell
    2 - One can edit the cell with accented characters

(I'll get round how to get the accented characters back to the
database later!)

My system (xubuntu 15.10) is all UTF8 so accented characters are
handled by the display, in terminals, etc. correctly.  I'm currently
using python 2.7 for this but would be quite happy to move to 3.4 if
this handles UTF8 better (I seem to remember it does maybe).

-- 
Chris Green
·

[toc] | [next] | [standalone]

#102626

On Sun, Feb 7, 2016 at 10:19 PM,  <cl@isbd.net> wrote:
> My system (xubuntu 15.10) is all UTF8 so accented characters are
> handled by the display, in terminals, etc. correctly.  I'm currently
> using python 2.7 for this but would be quite happy to move to 3.4 if
> this handles UTF8 better (I seem to remember it does maybe).

As a general rule, yes, Python 3 handles Unicode somewhat better than
Python 2 does. The main reason for this is that the native string type
is now a Unicode string, instead of the native string type being a
byte string; so "abcd" is not a string of bytes that happen to be
encoded as UTF-8, it is actually a string of Unicode characters. For
the most part, you'll be able to ignore character encodings and such,
and just work with text.

So the question then becomes: Under Python 3, can you use regular
string objects for both the things you're working with? I can confirm
that the inbuilt sqlite3 module works just fine with Unicode text; so
all you need to do is try out your GUI code under Python 3. I've no
idea how good wx support in Py3 is, so you might find you need to
switch GUI toolkits to get everything working; but whether it's with
wxWidgets, GTK, QT, or some other library, you should be able to put
something together under Python 3 that "just works" as regards
Unicode. Caveat: I haven't done any serious GUI programming using
Python.

(Note that "just works" can actually be a lot of hassle if you truly
want to support *all* of Unicode. I've seen programs that don't behave
correctly in the presence of combining characters, or right-to-left
text, or zero-width characters, or non-BMP characters; but you can get
far better support for the same amount of effort if you use Py3 and
Unicode than if you try to do things manually under Py2.)

ChrisA

[toc] | [prev] | [next] | [standalone]

#102627

Chris Angelico <rosuav@gmail.com> wrote:
> 
> So the question then becomes: Under Python 3, can you use regular
> string objects for both the things you're working with? I can confirm
> that the inbuilt sqlite3 module works just fine with Unicode text; so
> all you need to do is try out your GUI code under Python 3. I've no
> idea how good wx support in Py3 is, so you might find you need to
> switch GUI toolkits to get everything working; but whether it's with
> wxWidgets, GTK, QT, or some other library, you should be able to put
> something together under Python 3 that "just works" as regards
> Unicode. Caveat: I haven't done any serious GUI programming using
> Python.
> 
Sadly wxpython doesn't work at all in Python 3.  There's a project
called Phoenix to move it across to Python 3 but it's nowhere near
stable yet, which is rather a pity.

So the question is do I stay with 2.7 and live with the Unicode
difficulties or do I try and move to another GUI that does work with
Python 3.  

Are there any Python 3 GUIs that would be reasonably easy to move to?
E.g. ones which have a grid object and which work in the same sort of
way as wxpython in general?

-- 
Chris Green
·

[toc] | [prev] | [next] | [standalone]

#102628

On Sun, Feb 7, 2016 at 11:42 PM,  <cl@isbd.net> wrote:
>
> Are there any Python 3 GUIs that would be reasonably easy to move to?
> E.g. ones which have a grid object and which work in the same sort of
> way as wxpython in general?

Grid object? I'm not sure. But all of my GUI work of late has been
with GTK, and I'm pretty happy with it. (Granted, that's with Pike,
not Python, so the exact API is slightly different; but I know there
is good support for Python 3.) You may also want to look into Tk /
tkinter, although its Unicode support is (or was, last I heard)
limited to the Basic Multilingual Plane - the 64K most commonly used
characters.

ChrisA

[toc] | [prev] | [next] | [standalone]

#102629

Chris Angelico <rosuav@gmail.com> wrote:
> On Sun, Feb 7, 2016 at 11:42 PM,  <cl@isbd.net> wrote:
> >
> > Are there any Python 3 GUIs that would be reasonably easy to move to?
> > E.g. ones which have a grid object and which work in the same sort of
> > way as wxpython in general?
> 
> Grid object? I'm not sure. But all of my GUI work of late has been
> with GTK, and I'm pretty happy with it. (Granted, that's with Pike,
> not Python, so the exact API is slightly different; but I know there
> is good support for Python 3.) You may also want to look into Tk /
> tkinter, although its Unicode support is (or was, last I heard)
> limited to the Basic Multilingual Plane - the 64K most commonly used
> characters.
> 
Tkinter is certainly a possibility.  Basic Unicode support will be
fine, I'm only interested in handling European accented characters.

-- 
Chris Green
·

[toc] | [prev] | [next] | [standalone]

#102631

On 07.02.2016 12:19, cl@isbd.net wrote:
> However my database has quite a lot of Unicode data as there are
> French (and other) names with accents etc.  What's the right way to
> handle this reasonably neatly?  At the moment it traps an error at
> line 37:-
>      self.SetCellValue(row_num, i, str(cells[i]))
The unicode versions of wxPython should have no problems with handling 
unicode strings.
The problem that you see here is that str(...) tries to convert your 
unicode data into a non-unicode string, which of course fails.
Do something like:

value = cells[i]
if not isinstance(value, basestring):
     value = str(value)
self.SetCellValue(row_num, i, value)

Once you switch to Python 3 and Phoenix you have to modify this 
slightly, e.g. by adding this to the top of your code:

try:
     basestring
except:
     basestring = (bytes,str)


Regards,

Dietmar

[toc] | [prev] | [next] | [standalone]

#102653

Dietmar Schwertberger <maillist@schwertberger.de> wrote:
> On 07.02.2016 12:19, cl@isbd.net wrote:
> > However my database has quite a lot of Unicode data as there are
> > French (and other) names with accents etc.  What's the right way to
> > handle this reasonably neatly?  At the moment it traps an error at
> > line 37:-
> >      self.SetCellValue(row_num, i, str(cells[i]))
> The unicode versions of wxPython should have no problems with handling 
> unicode strings.
> The problem that you see here is that str(...) tries to convert your 
> unicode data into a non-unicode string, which of course fails.
> Do something like:
> 
> value = cells[i]
> if not isinstance(value, basestring):
>     value = str(value)
> self.SetCellValue(row_num, i, value)
> 
Thanks, worked perfectly!  I had been trying to produce something
similar in tkinter but grids are a bit clumsy in tkinter.  This wx
implementation will do me nicely.


> Once you switch to Python 3 and Phoenix you have to modify this 
> slightly, e.g. by adding this to the top of your code:
> 
> try:
>     basestring
> except:
>     basestring = (bytes,str)
> 
Everything else is 3 compatible so moving should be fairly painless
if/when phoenix becomes available.


-- 
Chris Green
·

[toc] | [prev] | [next] | [standalone]

#102657

On Mon, Feb 8, 2016 at 9:22 PM,  <cl@isbd.net> wrote:
>> Once you switch to Python 3 and Phoenix you have to modify this
>> slightly, e.g. by adding this to the top of your code:
>>
>> try:
>>     basestring
>> except:
>>     basestring = (bytes,str)
>>
> Everything else is 3 compatible so moving should be fairly painless
> if/when phoenix becomes available.

Small point: Even for code as small as this, I would be inclined to
catch only the one exception you care about - in this case, NameError.
I try to avoid having a bare except clause anywhere other than a "log
and continue" or "react and reraise" kind of situation.

ChrisA

[toc] | [prev] | [next] | [standalone]

#102634

2016-02-07 12:19 GMT+01:00  <cl@isbd.net>:
> I'm using this as a starting point for creating a grid to view and
> edit some sqlite data:-
>     http://www.salstat.com/news/linking-wxgrid-to-sqlite-database-in-python-an-example
>
> I can actually understand most of it which is a good start.
>
> However my database has quite a lot of Unicode data as there are
> French (and other) names with accents etc.  What's the right way to
> handle this reasonably neatly?  At the moment it traps an error at
> line 37:-
>     self.SetCellValue(row_num, i, str(cells[i]))
>
> I realise why this fails (I think) but how should one program this so
> that:-
>     1 - the accented characters are displayed correctly in the grid cell
>     2 - One can edit the cell with accented characters
>
> (I'll get round how to get the accented characters back to the
> database later!)
>
> My system (xubuntu 15.10) is all UTF8 so accented characters are
> handled by the display, in terminals, etc. correctly.  I'm currently
> using python 2.7 for this but would be quite happy to move to 3.4 if
> this handles UTF8 better (I seem to remember it does maybe).
>
> --
> Chris Green
> ·
> --
> https://mail.python.org/mailman/listinfo/python-list

Hi,
your code in
http://www.salstat.com/news/linking-wxgrid-to-sqlite-database-in-python-an-example

seems to work for me after small changes with both python 2.7 and 3.4
(using wx Phoenix)
the changes I made are:
- encoding declaration at the beginning of the file (mainly for py 2,
if utf-8 is used):

#! Python
# -*- coding: utf-8 -*-

- removing the str(...) call in the offending line you mentioned:
 self.SetCellValue(row_num, i, cells[i])
(what was the intent of the conversion to str?)

- corrected event name (insted of EVT_GRID_CELL_CHANGE)
self.Bind(gridlib.EVT_GRID_CELL_CHANGED, self.CellContentsChanged)

- normal App() call (instead of the PySimpleApp)

app = wx.App()

I randomly tried some characters using Latin, Greek etc. characters
(within BMP) as well as Gothic - beyond the FFFF range - the cell
content seems to be saved and and newly loaded from the sqlite db on
subsequent calls of the app.

regards,
    vbr

[toc] | [prev] | [next] | [standalone]

#102654

Vlastimil Brom <vlastimil.brom@gmail.com> wrote:
> 
> Hi,
> your code in
> http://www.salstat.com/news/linking-wxgrid-to-sqlite-database-in-python-an-example
> 
> seems to work for me after small changes with both python 2.7 and 3.4
> (using wx Phoenix)

Where are you getting phoenix from?  It's not in the Ubuntu
repositories and when I search on Google I seem to go in ever
decreasing circles and always end up finding wxPython for 2.7.

Did you build it from source yourself (I can do that but only if it's
necessary).


> the changes I made are:
> - encoding declaration at the beginning of the file (mainly for py 2,
> if utf-8 is used):
> 
> #! Python
> # -*- coding: utf-8 -*-
> 
> - removing the str(...) call in the offending line you mentioned:
>  self.SetCellValue(row_num, i, cells[i])
> (what was the intent of the conversion to str?)
> 
I've no idea what the intent is/was, I just copied the code as I found
it.  I guess the cell value could be numeric.


> - corrected event name (insted of EVT_GRID_CELL_CHANGE)
> self.Bind(gridlib.EVT_GRID_CELL_CHANGED, self.CellContentsChanged)
> 
> - normal App() call (instead of the PySimpleApp)
> 
> app = wx.App()
> 
> I randomly tried some characters using Latin, Greek etc. characters
> (within BMP) as well as Gothic - beyond the FFFF range - the cell
> content seems to be saved and and newly loaded from the sqlite db on
> subsequent calls of the app.
> 
Thanks for your help, very useful.

-- 
Chris Green
·

[toc] | [prev] | [next] | [standalone]

#102656

On Mon, Feb 8, 2016 at 9:42 PM,  <cl@isbd.net> wrote:
> Vlastimil Brom <vlastimil.brom@gmail.com> wrote:
>>
>> Hi,
>> your code in
>> http://www.salstat.com/news/linking-wxgrid-to-sqlite-database-in-python-an-example
>>
>> seems to work for me after small changes with both python 2.7 and 3.4
>> (using wx Phoenix)
>
> Where are you getting phoenix from?  It's not in the Ubuntu
> repositories and when I search on Google I seem to go in ever
> decreasing circles and always end up finding wxPython for 2.7.
>
> Did you build it from source yourself (I can do that but only if it's
> necessary).
>
>

http://wiki.wxpython.org/ProjectPhoenix has links to the source code
and to daily snapshots. I suspect the easiest way to get it will be
from source.

ChrisA

[toc] | [prev] | [next] | [standalone]

#102674

2016-02-08 11:42 GMT+01:00  <cl@isbd.net>:
> Vlastimil Brom <vlastimil.brom@gmail.com> wrote:
>>
>> Hi,
>> your code in
>> http://www.salstat.com/news/linking-wxgrid-to-sqlite-database-in-python-an-example
>>
>> seems to work for me after small changes with both python 2.7 and 3.4
>> (using wx Phoenix)
>
> Where are you getting phoenix from?  It's not in the Ubuntu
> repositories and when I search on Google I seem to go in ever
> decreasing circles and always end up finding wxPython for 2.7.
>
> Did you build it from source yourself (I can do that but only if it's
> necessary).
>
>
...
Hi,
I'm glad it helped a bit. However, I have no experiences with running
wx phoenix on linux - I use it with windows (7) - where the mentioned
snapshot builds are available (e.g. as wheel files installable with
pip).
http://wxpython.org/Phoenix/snapshot-builds/
I believe, building and installing on linux should be doable, but I
can't supply any details, see e.g.:
http://wxpython.org/Phoenix/docs/html/#linux-unix-etc
the source files are available along with the snapshot builds:
http://wxpython.org/Phoenix/snapshot-builds/
hth,
 vbr

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web