Path: csiph.com!eternal-september.org!feeder.eternal-september.org!mx02.eternal-september.org!.POSTED!not-for-mail
From: Unknown <dog@gmail.com>
Newsgroups: comp.lang.postscript
Subject: Re: *nix: pdftotext messes quotes; mouse-pastes OK?
Date: Thu, 24 Sep 2015 13:27:58 +0000 (UTC)
Organization: A noiseless patient Spider
Lines: 42
Message-ID: <pan.2015.09.24.13.31.07@gmail.com>
References: <pan.2015.09.09.05.37.39@gmail.com> <c39e0a23-63af-407a-b6ab-a8178fc33469@googlegroups.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Injection-Date: Thu, 24 Sep 2015 13:27:58 +0000 (UTC)
Injection-Info: mx02.eternal-september.org; posting-host="7b1d528b43483062edd45af9f036f23c"; logging-data="9548"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX1969SZty2rLS7NfwO3wX3/d90mh8EXCjRI="
User-Agent: Pan/0.133 (House of Butterflies)
Cancel-Lock: sha1:s2gtV83UJb5Lqju7sDy84u51ULc=
Xref: csiph.com comp.lang.postscript:2348

On Sat, 12 Sep 2015 22:18:53 -0700, luser- -droog wrote:

> On Wednesday, September 9, 2015 at 12:36:28 AM UTC-5, Unknown wrote:
>> How does mouse-pasting from `xpdf <file.pdf>` manage to do the quote/
>> apostrophe correctly; when `pdftotext` can't do it: but translates the
>> single-quote to 3 non-ASCII-chars ?!
>> 
>> I'm guessing, that working backwards from the screen: screen
>> <-FrameBufrPixels <-Char & Font Well yes,the mouse current X,Y position
>> is/must be mappable to the corresponding char.
>> Whereas `pdftotext` only sees the bytes of the <File.pdf>. Ie. the
>> mouse works with wysiwyg, where as `pdftotext` misses out several
>> translation steps.
>> 
>> Is that right?
> 
> Not sure exactly what problem you're encountering, but both programs
> "only see the bytes of the file". xpdf is not looking at the pixels.
> Even to do things like mouse selection of text, it stores the
> bounding-boxes of strings as it sets them. Then it checks the (x,y)
> coordinates of the mouse position against the list of boxes.
> 
> There is no "working backwards from the screen". Instead, both programs
> work with the encoded representation stored in the pdf file. Often this
> can be a non-standard custom encoding devised just to store the specific
> text of the document. If the font is also embedded, it can be a recoded
> subset matching the custom encoding and all information about the source
> encoding is lost.
> 
> It can be the case that a glyph will look the same as an ascii character
> but really be some other Unicode code point. In this case, all gui tools
> may seamlessly show the same character, but an ascii-only text editor
> will show the UTF-8 multi-byte encoding as several gibberish characters.
> So it could be it really is a Right-Single-Quote instead of an ascii
> single quote, and you're just not seeing it.

Since I wrote this, I've discovered:
 `pdftotext <option for ASCII> <args>`
which apparently replaces the 3-byte-crap with <singleQuote>.

So then the mouse-pasting does that by default.