Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #110983

Re: Well, I finally ran into a Python Unicode problem, sort of

Show key headers only | View raw

On Sun, Jul 3, 2016 at 4:58 PM, John Ladasky <john_ladasky@sbcglobal.net> wrote:
> Up until today, every character I've tried has been accepted by the Python interpreter as a legitimate character for inclusion in a variable name.  Now I'm copying a formula which defines a gradient.  The nabla symbol (∇) is used in the naming of gradients.  Python isn't having it.  The interpreter throws a "SyntaxError: invalid character in identifier" when it encounters the ∇.
>
> I am now wondering what constitutes a valid character for an identifier, and how they were chosen.  Obviously, the Western alphabet and standard Greek letters work.  I just tried a few very weird characters from the Latin Extended range, and some Cyrillic characters.  These are also fine.
>

Very good question! The detaily answer is here:

https://docs.python.org/3/reference/lexical_analysis.html#identifiers

> A philosophical question.  Why should any character be excluded from a variable name, besides the fact that it might also be an operator?
>

In a way, that's exactly what's happening here. Python permits certain
categories of character as identifiers, leaving other categories
available for operators. Even though there aren't any non-ASCII
operators in a vanilla CPython, it's plausible that someone could
create a Python-based language with more operators (eg ≠ NOT EQUAL TO
as an alternative to !=), and I'm sure you'd agree that saying "≠ = 1"
is nonsensical.

> This might be a problem I can solve, I'm not sure.  Is there a file that the Python interpreter refers to which defines the accepted variable name characters?  Perhaps I could just add ∇.
>

The key here is its Unicode category:

>>> unicodedata.category("∇")
'Sm'

You could probably hack CPython to include Sm, and maybe Sc, Sk, and
So, as valid identifier characters. I'm not sure where, though, and
I've just spent a good bit of time delving (it's based on the
XID_Start and XID_Continue derived properties, but I have no idea
where they're defined - Tools/unicode/makeunicodedata.py looks
promising, but even there, I can't find it). And - or maybe instead -
you could appeal to the core devs to have the category/ies in question
added to the official Python spec. Symbols like that are a bit of a
grey area, so you may find that you're starting a huge debate :)

Have fun.

ChrisA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Well, I finally ran into a Python Unicode problem, sort of John Ladasky <john_ladasky@sbcglobal.net> - 2016-07-02 23:58 -0700
  Re: Well, I finally ran into a Python Unicode problem, sort of Lawrence D’Oliveiro <lawrencedo99@gmail.com> - 2016-07-03 00:07 -0700
    Re: Well, I finally ran into a Python Unicode problem, sort of Marko Rauhamaa <marko@pacujo.net> - 2016-07-03 10:26 +0300
      Re: Well, I finally ran into a Python Unicode problem, sort of Lawrence D’Oliveiro <lawrencedo99@gmail.com> - 2016-07-03 01:38 -0700
        Re: Well, I finally ran into a Python Unicode problem, sort of Marko Rauhamaa <marko@pacujo.net> - 2016-07-03 12:01 +0300
          Re: Well, I finally ran into a Python Unicode problem, sort of Chris Angelico <rosuav@gmail.com> - 2016-07-04 01:14 +1000
          Re: Well, I finally ran into a Python Unicode problem, sort of Lawrence D’Oliveiro <lawrencedo99@gmail.com> - 2016-07-03 14:21 -0700
    Re: Well, I finally ran into a Python Unicode problem, sort of John Ladasky <john_ladasky@sbcglobal.net> - 2016-07-03 09:24 -0700
  Re: Well, I finally ran into a Python Unicode problem, sort of Rustom Mody <rustompmody@gmail.com> - 2016-07-03 00:26 -0700
  Re: Well, I finally ran into a Python Unicode problem, sort of Jussi Piitulainen <jussi.piitulainen@helsinki.fi> - 2016-07-03 10:29 +0300
    Re: Well, I finally ran into a Python Unicode problem, sort of Robert Kern <robert.kern@gmail.com> - 2016-07-03 10:16 +0100
  Re: Well, I finally ran into a Python Unicode problem, sort of Chris Angelico <rosuav@gmail.com> - 2016-07-03 17:41 +1000
    Re: Well, I finally ran into a Python Unicode problem, sort of John Ladasky <john_ladasky@sbcglobal.net> - 2016-07-03 11:39 -0700
      Re: Well, I finally ran into a Python Unicode problem, sort of MRAB <python@mrabarnett.plus.com> - 2016-07-03 20:41 +0100
      Re: Well, I finally ran into a Python Unicode problem, sort of Lawrence D’Oliveiro <lawrencedo99@gmail.com> - 2016-07-03 14:28 -0700
        Re: Well, I finally ran into a Python Unicode problem, sort of Steven D'Aprano <steve@pearwood.info> - 2016-07-04 12:33 +1000
          Re: Well, I finally ran into a Python Unicode problem, sort of Rustom Mody <rustompmody@gmail.com> - 2016-07-03 22:14 -0700
            Re: Well, I finally ran into a Python Unicode problem, sort of Lawrence D’Oliveiro <lawrencedo99@gmail.com> - 2016-07-04 01:11 -0700
              Re: Well, I finally ran into a Python Unicode problem, sort of Jussi Piitulainen <jussi.piitulainen@helsinki.fi> - 2016-07-04 13:10 +0300
              Re: Well, I finally ran into a Python Unicode problem, sort of Marko Rauhamaa <marko@pacujo.net> - 2016-07-04 12:04 +0300
            Re: Well, I finally ran into a Python Unicode problem, sort of Jussi Piitulainen <jussi.piitulainen@helsinki.fi> - 2016-07-04 09:08 +0300
  Re: Well, I finally ran into a Python Unicode problem, sort of Alain Ketterlin <alain@universite-de-strasbourg.fr.invalid> - 2016-07-03 11:53 +0200
    Re: Well, I finally ran into a Python Unicode problem, sort of Marko Rauhamaa <marko@pacujo.net> - 2016-07-03 14:01 +0300
      Re: Well, I finally ran into a Python Unicode problem, sort of Christian Gollwitzer <auriocus@gmx.de> - 2016-07-03 13:16 +0200
        Re: Well, I finally ran into a Python Unicode problem, sort of Marko Rauhamaa <marko@pacujo.net> - 2016-07-03 14:22 +0300
          Re: Well, I finally ran into a Python Unicode problem, sort of Christian Gollwitzer <auriocus@gmx.de> - 2016-07-03 16:05 +0200
            Re: Well, I finally ran into a Python Unicode problem, sort of Marko Rauhamaa <marko@pacujo.net> - 2016-07-03 17:39 +0300
          Re: Well, I finally ran into a Python Unicode problem, sort of Random832 <random832@fastmail.com> - 2016-07-03 16:49 -0400
            Re: Well, I finally ran into a Python Unicode problem, sort of Marko Rauhamaa <marko@pacujo.net> - 2016-07-04 00:25 +0300
      Re: Well, I finally ran into a Python Unicode problem, sort of BartC <bc@freeuk.com> - 2016-07-03 12:50 +0100
        Re: Well, I finally ran into a Python Unicode problem, sort of Lawrence D’Oliveiro <lawrencedo99@gmail.com> - 2016-07-03 14:26 -0700
  Re: Well, I finally ran into a Python Unicode problem, sort of eryk sun <eryksun@gmail.com> - 2016-07-03 23:46 +0000
    Re: Well, I finally ran into a Python Unicode problem, sort of Lawrence D’Oliveiro <lawrencedo99@gmail.com> - 2016-07-03 17:00 -0700
      Re: Well, I finally ran into a Python Unicode problem, sort of BartC <bc@freeuk.com> - 2016-07-04 01:17 +0100
        Re: Well, I finally ran into a Python Unicode problem, sort of Lawrence D’Oliveiro <lawrencedo99@gmail.com> - 2016-07-03 17:24 -0700
          Re: Well, I finally ran into a Python Unicode problem, sort of BartC <bc@freeuk.com> - 2016-07-04 01:39 +0100
            Re: Well, I finally ran into a Python Unicode problem, sort of Lawrence D’Oliveiro <lawrencedo99@gmail.com> - 2016-07-03 18:15 -0700
              Re: Well, I finally ran into a Python Unicode problem, sort of Random832 <random832@fastmail.com> - 2016-07-03 22:20 -0400
              Re: Well, I finally ran into a Python Unicode problem, sort of Rustom Mody <rustompmody@gmail.com> - 2016-07-04 05:16 -0700
              Re: Well, I finally ran into a Python Unicode problem, sort of BartC <bc@freeuk.com> - 2016-07-04 11:26 +0100
        Re: Well, I finally ran into a Python Unicode problem, sort of Steven D'Aprano <steve@pearwood.info> - 2016-07-04 12:30 +1000
          Re: Well, I finally ran into a Python Unicode problem, sort of Ned Batchelder <ned@nedbatchelder.com> - 2016-07-04 05:47 -0700
            Re: Well, I finally ran into a Python Unicode problem, sort of BartC <bc@freeuk.com> - 2016-07-04 15:36 +0100
              Re: Well, I finally ran into a Python Unicode problem, sort of Ned Batchelder <ned@nedbatchelder.com> - 2016-07-04 07:46 -0700
                Re: Well, I finally ran into a Python Unicode problem, sort of BartC <bc@freeuk.com> - 2016-07-04 17:16 +0100
                Re: Well, I finally ran into a Python Unicode problem, sort of Marko Rauhamaa <marko@pacujo.net> - 2016-07-04 19:44 +0300
          Re: Well, I finally ran into a Python Unicode problem, sort of BartC <bc@freeuk.com> - 2016-07-04 11:05 +0100
            Re: Well, I finally ran into a Python Unicode problem, sort of Jussi Piitulainen <jussi.piitulainen@helsinki.fi> - 2016-07-04 13:26 +0300
      Re: Well, I finally ran into a Python Unicode problem, sort of Random832 <random832@fastmail.com> - 2016-07-03 22:16 -0400

csiph-web