Re: Python NBSP DWIM

Path	csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed1a.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path	<random832@fastmail.us>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.001
X-Spam-Evidence	'H': 1.00; 'S': 0.00; 'ideally': 0.04; 'subject:Python': 0.05; 'lines.': 0.07; 'reason,': 0.07; 'width': 0.07; 'pixels': 0.09; 'received:internal': 0.09; 'yeah,': 0.09; 'wed,': 0.15; 'argument': 0.15; 'file,': 0.15; 'represents': 0.15; 'blank,': 0.16; 'distinct': 0.16; 'iirc': 0.16; 'in-memory': 0.16; 'message-id:@webmail.messagingengine.com': 0.16; 'needless': 0.16; 'received:10.202': 0.16; 'received:10.202.2': 0.16; 'received:10.202.2.212': 0.16; 'received:66.111': 0.16; 'received:66.111.4': 0.16; 'received:messagingengine.com': 0.16; 'wrote:': 0.16; 'byte': 0.18; 'say,': 0.18; "shouldn't": 0.18; 'windows': 0.20; '(the': 0.22; 'space.': 0.22; 'visible': 0.22; 'player': 0.23; "i've": 0.24; 'header:In-Reply-To:1': 0.24; 'chris': 0.26; 'points': 0.27; 'appear': 0.29; 'looks': 0.29; 'fighting': 0.29; 'symbols': 0.29; 'regardless': 0.31; 'code': 0.31; 'another': 0.34; 'file': 0.34; 'to:addr:python-list': 0.35; 'files,': 0.35; 'something': 0.35; "isn't": 0.35; 'represent': 0.35; 'but': 0.36; 'text': 0.36; 'totally': 0.36; 'two': 0.37; 'should': 0.37; 'received:10': 0.37; 'subject:: ': 0.37; 'world,': 0.38; 'rather': 0.38; 'received:66': 0.38; 'to:addr:python.org': 0.39; 'takes': 0.39; 'mark': 0.40; 'why': 0.40; 'some': 0.40; 'your': 0.60; 'behavior': 0.61; 'from:no real name:2**0': 0.61; 'real': 0.61; 'skip:u 10': 0.62; 'header:Message-Id:1': 0.62; 'media': 0.67; 'letters': 0.67; 'rendering': 0.72; 'nice,': 0.84; 'notable': 0.91; 'from.': 0.93
DKIM-Signature	v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.us; h= content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-sasl-enc :x-sasl-enc; s=mesmtp; bh=pCkwXQXV1k0Hh5w2XfN0AD6h8+k=; b=lQ7tBy wVsQ591yy1iXCDPwDR+Ll+aWrgsP1SGOTvY0l70Rf7Ng3FqShkzDB1TvsW3AFvU6 VQR7aefwUWCqOlRNfuIaaqz6K4NvVC2G1eDX9qKt4ho+r9jVIW/9AynCHJybfi3L XEGMFEJyeVBV0DTlauCgPgyliMi69x82G939Q=
DKIM-Signature	v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-sasl-enc:x-sasl-enc; s=smtpout; bh=pCkwXQXV1k0Hh5w 2XfN0AD6h8+k=; b=kHf4CrA59kscmpN6g7pUI0+owScocLGjmTgis0l/7fcqirT E+MBEKK1/fwRjlviBblC4NHOmI/haigJJiK1xnERVfN4Nv7TEz6SsCUsndTpp7xK MN6CTjKypx2njFO66dN4BHXayRYSKE3GPGgPT4h2Hkag99h2QqsReoy+bksM=
X-Sasl-Enc	5VvvdL0jwSHDpaG5hV0XpRodfjJLP+kd1AFgjFI47+nY 1433984536
From	random832@fastmail.us
To	python-list@python.org
MIME-Version	1.0
Content-Transfer-Encoding	7bit
Content-Type	text/plain
X-Mailer	MessagingEngine.com Webmail Interface - ajax-ff004c4b
In-Reply-To	<CAPTjJmqS-sx2yxrPAcN6iv625hUOQKpM8bqUqBrNfcHvyzm8AQ@mail.gmail.com>
References	<20150610082812.2ce887c3@bigbox.christie.dr> <mailman.344.1433946513.13271.python-list@python.org> <55786fd5$0$13003$c3e8da3$5496439d@news.astraweb.com> <CAPTjJmqS-sx2yxrPAcN6iv625hUOQKpM8bqUqBrNfcHvyzm8AQ@mail.gmail.com>
Subject	Re: Python NBSP DWIM
Date	Wed, 10 Jun 2015 21:02:16 -0400
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.20+
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.372.1433984539.13271.python-list@python.org> (permalink)
Lines	29
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1433984539 news.xs4all.nl 2833 [2001:888:2000:d::a6]:37787
X-Complaints-To	abuse@xs4all.nl
Xref	csiph.com comp.lang.python:92439

Show key headers only | View raw

On Wed, Jun 10, 2015, at 20:09, Chris Angelico wrote:
> And U+FEFF "ZERO WIDTH NO-BREAK SPACE", notable because it's also used as
> the byte-order mark (as its counterpart, U+FFFE, is unallocated). I've
> been
> fighting with VLC Media Player over the font it uses for subtitles; for
> some bizarre reason, that font represents U+FEFF not with zero pixels of
> emptiness, but with a box containing the letters "ZWN" "BSP" on two
> lines.
> Yeah, because that totally takes up zero width and looks like blank
> space.

As I understand it, the proper behavior is that the ZWNBSP that is the
byte order mark shall never appear in an in-memory representation of the
first line of a BOM-encoded file, or any other line of the concatenation
of two BOM-encoded files, but should "vanish" when the file is opened
and first read from. So it shouldn't be showing up in your subtitles
regardless of its rendering behavior.

The real world, needless to say, isn't so nice.

IIRC there's also a font in MS windows that uses various glyphs which
are zero-width, but are not blank, to represent ZWJ, ZWNJ, RLM, and LRM.
Good for seeing what is happening, bad for actually rendering text
that's intended to contain these characters. Though there's another
argument that ideally a rendering engine should not render any such
glyph unless something like "visible controls" has been selected (the
real world, again, isn't so nice, which is why most symbols intended for
visible control style rendering have their own distinct code points
rather than using those of the control characters they represent).

Thread

Re: Python NBSP DWIM Skip Montanaro <skip.montanaro@gmail.com> - 2015-06-10 09:28 -0500
  Re: Python NBSP DWIM Steven D'Aprano <steve@pearwood.info> - 2015-06-11 03:11 +1000
    Re: Python NBSP DWIM random832@fastmail.us - 2015-06-10 21:02 -0400
    Re: Python NBSP DWIM Chris Angelico <rosuav@gmail.com> - 2015-06-11 11:09 +1000
    Re: Python NBSP DWIM Steven D'Aprano <steve@pearwood.info> - 2015-06-11 12:26 +1000
      Re: Python NBSP DWIM Chris Angelico <rosuav@gmail.com> - 2015-06-11 13:05 +1000
        Re: Python NBSP DWIM Steven D'Aprano <steve@pearwood.info> - 2015-06-11 13:27 +1000
          Re: Python NBSP DWIM Chris Angelico <rosuav@gmail.com> - 2015-06-11 13:37 +1000
      Re: Python NBSP DWIM random832@fastmail.us - 2015-06-10 23:18 -0400
      Re: Python NBSP DWIM Chris Angelico <rosuav@gmail.com> - 2015-06-11 13:28 +1000

csiph-web