Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #224422 > unrolled thread

Using .XCompose

Back to article view | Back to linux.debian.user

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-06 07:40 +0200
    Re: Using .XCompose Greg Wooledge <wooledg@eeg.ccf.org> - 2020-07-06 14:30 +0200
    Re: Using .XCompose Zenaan Harkness <zenaan@freedbms.net> - 2020-07-06 14:50 +0200
    Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-06 21:20 +0200
      Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-06 22:40 +0200
        Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-07 17:20 +0200
      Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-07 08:40 +0200
        Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-07 17:20 +0200
          Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-09 23:50 +0200
            Re: Using .XCompose [resend of xev_output_annotated.txt] davidson <davidson@freevolt.org> - 2020-07-10 03:40 +0200
            Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-10 10:50 +0200
              Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-11 10:40 +0200
                Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-11 12:30 +0200
                  Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-12 17:00 +0200
                    Re: Using .XCompose David Wright <deblis@lionunicorn.co.uk> - 2020-07-13 06:00 +0200
                    Re: Using .XCompose Greg Wooledge <wooledg@eeg.ccf.org> - 2020-07-13 14:00 +0200
                      Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-13 20:30 +0200
                        Re: Using .XCompose Greg Wooledge <wooledg@eeg.ccf.org> - 2020-07-13 20:40 +0200
                          Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-13 20:50 +0200
                            Re: Using .XCompose David Wright <deblis@lionunicorn.co.uk> - 2020-07-13 21:50 +0200
                              Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-14 05:40 +0200
                          Fw: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-15 16:50 +0200
                          Fw: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-15 17:10 +0200
                          Fw: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-15 17:20 +0200
                    Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-16 01:30 +0200
                      Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-16 02:40 +0200
                      Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-16 20:40 +0200
                        how u mine 4 utf8 [was Re: Using .XCompose] davidson <davidson@freevolt.org> - 2020-07-18 03:20 +0200
                          UTF-8 Everywhere -- Re: how u mine 4 utf8 [was Re: Using .XCompose] Zenaan Harkness <zenaan@freedbms.net> - 2020-07-18 07:40 +0200
                          Re: how u mine 4 utf8 [was Re: Using .XCompose] davidson <davidson@freevolt.org> - 2020-07-18 09:20 +0200
                            Re: how u mine 4 utf8 [was Re: Using .XCompose] Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-19 13:00 +0200
                          Fw: how u mine 4 utf8 [was Re: Using .XCompose] Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-19 12:50 +0200
                            Re: Fw: how u mine 4 utf8 [was Re: Using .XCompose] Tom Browder <tom.browder@gmail.com> - 2020-07-19 13:10 +0200
                              Re: how u mine 4 utf8 [was Re: Using .XCompose] Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-19 13:50 +0200
                        sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] davidson <davidson@freevolt.org> - 2020-07-18 12:10 +0200
                          Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] Zenaan Harkness <zenaan@freedbms.net> - 2020-07-18 12:50 +0200
                            Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] William Michels <wjm1@caa.columbia.edu> - 2020-07-18 22:00 +0200
                          Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] Andrei POPESCU <andreimpopescu@gmail.com> - 2020-07-18 13:20 +0200
                            Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] songbird <songbird@anthive.com> - 2020-07-18 21:00 +0200
                          Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] <tomas@tuxteam.de> - 2020-07-18 16:00 +0200
                            Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] mick crane <mick.crane@gmail.com> - 2020-07-18 22:40 +0200
                              Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] <tomas@tuxteam.de> - 2020-07-18 23:10 +0200
                                Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] mick crane <mick.crane@gmail.com> - 2020-07-18 23:30 +0200
                                  Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] <tomas@tuxteam.de> - 2020-07-19 10:40 +0200
                                    Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] mick crane <mick.crane@gmail.com> - 2020-07-20 13:20 +0200
                          Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] David Christensen <dpchrist@holgerdanske.com> - 2020-07-19 01:50 +0200
                            Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-19 12:50 +0200
                              Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] David Christensen <dpchrist@holgerdanske.com> - 2020-07-19 23:30 +0200
                          Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-19 13:40 +0200
                            Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] William Michels <wjm1@caa.columbia.edu> - 2020-07-19 21:50 +0200
                            Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] David Wright <deblis@lionunicorn.co.uk> - 2020-07-23 04:00 +0200
                              Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] songbird <songbird@anthive.com> - 2020-07-23 12:50 +0200
                    Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-16 02:00 +0200
                      Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-16 20:40 +0200
                Re: Using .XCompose David Wright <deblis@lionunicorn.co.uk> - 2020-07-11 17:00 +0200
                  Tex characters Teemu Likonen <tlikonen@iki.fi> - 2020-07-11 18:20 +0200
                  Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-11 22:40 +0200
                    Re: Using .XCompose Greg Wooledge <wooledg@eeg.ccf.org> - 2020-07-13 14:00 +0200
                      Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-16 07:50 +0200
                Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-11 18:10 +0200
                Re: Using .XCompose Greg Wooledge <wooledg@eeg.ccf.org> - 2020-07-20 15:30 +0200
                Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-20 15:30 +0200
                  Re: Using .XCompose David Wright <deblis@lionunicorn.co.uk> - 2020-07-20 18:20 +0200
            Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-10 12:50 +0200
      Re: [offlist] posting style [was Re: Using .XCompose] Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-08 05:00 +0200
        Re: [offlist] posting style [was Re: Using .XCompose] Andrei POPESCU <andreimpopescu@gmail.com> - 2020-07-08 07:00 +0200
        Re: [not-so-offlist] posting style [was Re: Using .XCompose] davidson <davidson@freevolt.org> - 2020-07-08 07:10 +0200
      Re: [offlist] Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-08 05:00 +0200
        Re: [offlist] Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-08 05:20 +0200
        Re: [offlist] Re: Using .XCompose Zenaan Harkness <zenaan@freedbms.net> - 2020-07-08 05:40 +0200
          Re: [offlist] Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-08 08:00 +0200
    Re: Using .XCompose David Wright <deblis@lionunicorn.co.uk> - 2020-07-07 00:40 +0200
      tangent on man page locations [was Re: Using .XCompose] davidson <davidson@freevolt.org> - 2020-07-07 01:10 +0200
        Re: tangent on man page locations [was Re: Using .XCompose] David Wright <deblis@lionunicorn.co.uk> - 2020-07-07 03:50 +0200
      Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-07 11:50 +0200
        Re: Using .XCompose Greg Wooledge <wooledg@eeg.ccf.org> - 2020-07-07 13:40 +0200
          Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-08 16:30 +0200
        Re: Using .XCompose David Wright <deblis@lionunicorn.co.uk> - 2020-07-09 02:20 +0200
          Re: Using .XCompose Zenaan Harkness <zenaan@freedbms.net> - 2020-07-09 04:20 +0200
    Re: Using .XCompose Zenaan Harkness <zenaan@freedbms.net> - 2020-07-08 01:50 +0200
      Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-08 02:30 +0200
      Fw: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-08 05:10 +0200
      Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-08 18:40 +0200
        Re: Using .XCompose Zenaan Harkness <zenaan@freedbms.net> - 2020-07-09 04:00 +0200
          Re: Using .XCompose Zenaan Harkness <zenaan@freedbms.net> - 2020-07-10 02:40 +0200
            Re: Using .XCompose The Wanderer <wanderer@fastmail.fm> - 2020-07-10 02:50 +0200
        Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-10 05:00 +0200

Page 2 of 5 — ← Prev page 1 [2] 3 4 5  Next page →

#224779

Hi David,

> And I assume they're not in the files that you wrote from scratch,
> like .XCompose. Would that be right?

Yes, I couldn't find any in my XCompose file and in the layout file.


> Kate appears to be a normal text editor. The third sentence of its
> official description (https://kate-editor.org/) says "Choose it for
> […], editing configuration files". It doesn't seem to me to be
> like writing text files in, say, Word (a word processor).

I am not sure I understand this correctly.  I know that Kate is plain text editor, not something like Word, if that is what you mean

> It would be nice not to get sidetracked by this, and to find out
> whether the way that the Indic Script input system (InScript) works is
> compatible with the composition of characters, which is the purpose of
> .XCompose. (I'm assuming that linux tries to give the >1billion people
> concerned an InScript system that's compatible with other platforms).

I tried thrice to send a reply to your previous message. Those messages are shown as being sent, but does not get delivered to me through debian list.
(Earlier, I got a message that there was a bounce and that I would be removed if it happens again. I don't know if I have been removed from the list or not.)
I want to repeat that my problem of .XCompose not working is not lmited to Indic scripts. It is happening with English too.
BTW, InScript is one layout among many layouts for the various Indian Scripts.


Thanks,
ajith

[toc] | [prev] | [next] | [standalone]

#224854

[Multipart message — attachments visible in raw view] — view raw

I was trying to send this mail for the past few days. It was not getting distributed probably because it is lengthy. So, I am trying to send it as two emails. This first part is being continued in the next email 



Hi David,

> OK. I wonder whether the problem you're having with using XCompose
> is that although those three characters <U0D19> <U0D4D> <U0D19>
> look independent of each other in the file, the keystrokes that
> generate them might not be. Not having your layout, I don't think
> I can test whether you get the behaviour I think you do, that
> when you put the cursor at the beginning of a *typed* line that
> looks like the next one and press Delete once:
> ങ്ങ
> you get
> ങ
> whereas I get
> ്ങ
> Is that right?

I am sorry if I didn't explain properly and for not attaching the keyboard layout (I assumed no one would want to  go through the rather boring lines. I am attaching my .XCompose, the layout file in (the variant I wrote is named mal_puthuniraA) , and the keyboard file.

To get the ligated conjunct using this layout, I type L while holding down Shift followed by f (or j) without shift and then L while holding down Shift. When these characters are typed, the program will show the ligated conjunct form if its font supports the form or else it displays the three characters separately. So, when you go to the beginning of ങ്ങ and press delete once, you should get ്ങ or the entire ങ്ങ is deleted (based on how the program treats the ligated conjnct form). There are issues in displaying ligated forms by various programs and I assume that by extension there will be problems while deleting also. More over, the < ് > is a combining mark. So, some programs will treat the charcter preceeding < ് > and the < ് > as one character.

In Konsole, the ligated conjunct is formed correctly, but the width calculated for display is slightly off and so the cursor is placed over the character. When I use Home key to go to the beginning of the line and then press Delete, I get ങ്. When I get to the end and press backspace also, I get the same result. If I press back space a second time, the ങ് is deleted. Note that both characters are deleted with one back space. If I move to the beginning of ങ്ങ and then press right arrow once and then press space once, I get ങ് ങ.

In Kate, deleting from the beginning deletes the entire ങ്ങ, backspace from the end deletes the entire one character at one time giving ങ് followed by ങ.Moving to beginning and inserting a space adds the space after ങ്ങ.

I don't think these variations in handling Indic scripts is related to the problem of Composing. But, please do check the keyboard layout I am attaching.

If the reason why the single line <W> : "a long sentence" in .XCompose is not working as expected is found out, I think my problem would be solved.

[toc] | [prev] | [next] | [standalone]

#224856

This is the second part of my email. 

> Note that the whitespace in your *attached* file (mixed tabs and
> saces) matched my own, whereas the file here in your post does not.
> That suggests that the 0xc2 0xa0 sequences may be a result of your
> copy/paste operation.

By "the file here in your post" do you mean the text of the email per se?

> again just reflects the additions in my /e/d/k.

Good to know that there are no problems in my files.


> … because it doesn't contain any lines with \x00A0 in them.
Ok. I understood my mistake. Now, grep $'\xc2\xa0' .XCompose doesn't return the line.

> \xHH  the eight-bit character whose value is the hexadecimal value
>        HH (one or two hex digits)
>  \uHHHH the Unicode (ISO/IEC 10646) character whose value is the
>        hexadecimal value HHHH (one to four hex digits)
>
> So you were mixing up those two constructions, perhaps.

Yes, I am. Also, I don't understand the conversion between the two. The \x tells that two charcters that follow it are hexadecimals and \u tells that the following four hexdecimals are to be interpreted as unicode values. But, how do you derive \xc2\xa0 for \u00A0?



> hexdump -C filename  will reveal exactly what's
> in a file, hex to the left, and corresponding characters to the right.



hexdump -C .XCompose gives
00000000  3c 57 3e 20 3a 20 22 54  68 69 73 20 72 65 70 6c  |<W> : "This repl|
00000010  61 63 65 73 20 57 22 0a                           |aces W".|
00000018

I am attaching the .XCompose file,; so, I am not redirecting the output to a file and attaching.

Thanks again for all your time and effort,
ajith

[toc] | [prev] | [next] | [standalone]

#224857

[Multipart message — attachments visible in raw view] — view raw

This is the third part - the layout file is comprssed and attached


ajith

[toc] | [prev] | [next] | [standalone]

#224865

[Multipart message — attachments visible in raw view] — view raw

> Hi All,
>
> I will start with what I did in sequence
>
> Using Konsole from my home directory
>
> 1)executed setxkbmap -layout us

Naturally, since it is after all the topic of your thread, you are
primarily interested in troubleshooting the XCompose mechanism.

And so you have set up known environmental conditions for subsequent
tests of that mechanism.

Because I am dim/one-track-minded, it took me a while to understand
that this is the purpose of step (1) above.

> 2)executed xmodmap which gave the following output
>
> xmodmap:  up to 4 keys per modifier, (keycodes in parentheses):
>
> shift       Shift_L (0x32),  Shift_R (0x3e)
> lock        Caps_Lock (0x42)
> control     Control_L (0x25),  Control_R (0x69)
> mod1        Alt_L (0x40),  Alt_R (0x6c),  Meta_L (0xcd)
> mod2        Num_Lock (0x4d)
> mod3      
> mod4        Super_L (0x85),  Super_R (0x86),  Super_L (0xce),  Hyper_L (0xcf)
> mod5        ISO_Level3_Shift (0x5c),  Mode_switch (0xcb)

I notice in passing that your modifier key setup here is identical to
my own, with one exception,

   lock        ISO_Next_Group (0x42)

which permits me to toggle my keyboard layout between two alternatives
("us" and "ru") by striking capslock.

I also notice that clobbering my "ru" alternate layout with the
command you issued in step (1) above does not change the output I get
when I issue the command in step (2).

So I am prone to conclude that whatever your layout may have been
prior to step (1), it was not a dual-layout setup.

(This relates more to my own curiosity than to your primary concern.)

> 3)executed xmodmap -pk > xmm
>
> The file named xmm is attached

In that keymap table I can find no keys that enter nonbreaking spaces.

And though it does not have to do with the test you are conducting
here, I do remain curious about whether your day-to-day keymap table
*does* include such mappings. It is entirely possible that it does.

I imagine at this point you may be able to work this out for yourself,
if it interests you.

> 4) executed setxkbmap -print which gave the following 
>
> xkb_keymap {
>         xkb_keycodes  { include "evdev+aliases(qwerty)" };
>         xkb_types     { include "complete"      };
>         xkb_compat    { include "complete"      };
>         xkb_symbols   { include "pc+us+inet(evdev)"     };
>         xkb_geometry  { include "pc(pc104)"     };
> };
>
> 5) execute grep $'\x00A0' .XCompose. All lines from the .XCompose file
> were listed.

Others have pointed out that here you were simply printing *all* lines
in the file.

> So, I replaced the .XCompose file and retyped the three lines (with
> only space typed using the space bar of the kyboardbetween the
> letters) and executed the grep command. Again it returned all
> lines. So, I replaced .XCompose file with just the W line. Again
> that line was reported by grep. So, I abandoned Kate and built one
> using cat > .XCompose followed by the line <W> : "This replaces W",
> followed by Enter and Ctrl+D. The grep command returned the line.

The ineffective grep command in step (5) seems to be a creation of
your own. Perhaps it is a synthesis of

  $ grep $'\xc2\xa0' sometextfile

together with various information independently learned elsewhere.

As others pointed out, it was *nearly* (but crucially not quite) the
effective replacement

  $ grep $'\u00a0' sometextfile

In APPENDICES at the bottom of this message, I make a few observations
about the two (effective) grep commands immediately above.

> I noticed that there is $ sign before the search string, which I
> couldn't understand. I removed it and re-executed the new grep
> command grep '\x00A0' .XCompose. Now it doesn't return the line

Others have given concise explanations. See appendices below for a
longer one.

It is cool you experiment with what you don't understand. That is how
mistakes are made, and mistakes are the best teacher of all.

This is probably a good spot to recommend a coherent and
comprehensive, well-curated beginner's guide to bash:

  BashGuide - Greg's Wiki
  http://mywiki.wooledge.org/BashGuide

It is a shame if good learning resources are not used.

> 6) The command grep "W" .XCompose | tr $'\xc2\xa0' \! returns 
> grep "W" .XCompose | tr $'\xc2\xa0' \!

This does not seem to make any sense. Where is the output?

It looks to me like you may have pasted a copy of the command, where
you meant to paste its output.

What you seem to be *showing* us, is that the command produced as
output a copy of itself. But you don't seem to be *saying* this, or
reasoning as if that were so, since whatever the output was, further
below you seem inclined to think that it indicated the file contained
no nonbreaking spaces.

Anyways, for a number of reasons that particular command line is a
poor method for rendering nonbreaking spaces visible. Greg explained
one of the reasons: tr maps bytes (octets of bits), and doesn't
understand that some characters are composed of more than one octet.

It is good to know your prefered text editor can display nonbreaking
spaces distinctively, and that you know how to turn on that option.

In case you some time want to do a mass translation of nonbreaking
spaces to some other character (let's say '%' this time) from the
shell command line, then doing

  $ sed 'y/\xc2\xa0/%/' somefile

should print contents of somefile, but with each nonbreaking space
(which is represented in that example as a pair of C-style
byte-constants, \xc2 and \xa0, which sed correctly understands denote
together a single character) replaced with a '%' character.

Note also that in that command, you can just substitute a literal
nonbreaking space for the sequence '\xc2\xa0' if you know (or can
figure out) how to type a literal nonbreaking space with your
keyboard.

(However, one advantage of using the C-style byte-constants (\xHH)
instead is that it is easy for everyone to see what they are, the web
archive won't replace them with normal spaces, etc.)

Anyways, it is good to know that sed seems to understand multi-octet
characters, unlike tr.

Also, notice that sed itself --unlike tr and grep-- can interpret a
\xHH byte-constant. We don't need to arrange for the shell to do this
on sed's behalf. See the discussion in the appendices below, if that
sounds mysterious. (sed does not, however, understand character
constants of the form \uHHHH, as far as I can tell from local
experiment and reading its documentation.)

If you have both the package called "info" and sed installed, like I do

  $ dpkg-query -l
  Desired=Unknown/Install/Remove/Purge/Hold
  | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
  |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
  ||/ Name                Version           Architecture Description
  +++-===================-=================-============-=================================================
  ii  info                6.5.0.dfsg.1-4+b1 amd64        Standalone GNU Info documentation browser
  ii  sed                 4.7-1             amd64        GNU stream editor for filtering/transforming text
  ii  texinfo-doc-nonfree 6.5.0-1           all          texinfo and info documentation that is non-free

then you can do

  $ info sed

to read some documentation about sed. Do

  $ info sed "sed scripts" "other commands"

to jump straight to a brief description of the 'y///' sed operation
used in that example.

However ---and believe me I realise this sounds completely ludicrous
(because it is indeed completely ludicrous)--- comfortable use of the
info command itself is somewhat of an acquired skill, and in order to
read locally the full documentation meant to go with info one must
install the package

  texinfo-doc-nonfree - texinfo and info documentation that is non-free
    This package provides documentation in info and html format for the
    texinfo and info packages. See the respective packages for more
    explanation.

from the "non-free" component of debian.

Once that package is installed, one can do

  $ info info # Without texinfo-doc-nonfree, a nearly pointless man page.

And by this point, we are truly in Douglas Adams' Hitchhiker's Guide
to the Galaxy territory:

  "But the plans were on display..."
  "On display? I eventually had to go down to the cellar to find them."
  "That's the display department."
  "With a flashlight."
  "Ah, well, the lights had probably gone."
  "So had the stairs."
  "But look, you found the notice, didn't you?"
  "Yes," said Arthur, "yes I did. It was on display in the bottom of a
  locked filing cabinet stuck in a disused lavatory with a sign on the
  door saying 'Beware of the Leopard'."

So, maybe you would like to forget all of that and just read the sed
info pages in html form on the web:

  https://www.gnu.org/software/sed/manual/
  https://www.gnu.org/software/sed/manual/html_node/index.html

Direct link to synopsis of what y/// does:

  https://www.gnu.org/software/sed/manual/html_node/Other-Commands.html#Other-Commands

Also, with sed installed, you will have on your system the directory
/usr/share/doc/sed, which contains at least the compressed text
document "sedfaq.txt.gz", which you can read with

   $ zless /usr/share/doc/sed/sedfaq.txt.gz # note the 'z' in "zless"

Near the top of that file is probably a url for an online html version
of the same document. You might find it more comfortable to navigate
as html in a web browser.

Regarding the "info" tool. Many GNU project tools use it for user
documentation, and if you are going to use those tools you may find it
worthwhile spending --at some point in time-- the time necessary to
get acquainted with use of the info browser.

But it is hilarious --truly Hitchhiker's Guide level hilarity-- how
baroque a gatekeeper the info browser seems to be, in the beginning,
when one merely wants to get on with reading the documentation
published in info pages about grub, or sed, or bash, etc.

> 7) I typed W (holding down shift and pressing w). The Konsole (the
> instance from which I issued the command as well as a new instance)
> did nothing except displaying the W. I typed W in Kate with the same
> result.

Interesting. I wonder what's going on.

> Is it safe to say that the .XCompose file doesn't have the nonbreaking
> spaces?

You have more evidence than we do, regarding this. You saw output that
somehow did not make it into your message to the list.

But more importantly, you have learned how to tell your prefered text
editor to show you nonbreaking spaces. So you can check for yourself.

> > You can print all lines of sometextfile which contain them by doing
> > this:
>
> > $ grep $'\xc2\xa0' sometextfile
>
> Is the $ just before the search string in single quotes in the grep
> command intentional? If so, what is its significance?

Covered this in Appendix B below.

> > '!' marks the spot of nonbreaking spaces that made it into OP's first
> > report of odd behavior, upon testing the white scissors XCompose rule:
>
> Could you please see if any of the lines in this message too behaves
> similarly?

I'll send a second reply illuminating them, sure. (To have shoehorned
that into this message would have been non-consensually psychedelic.)

> If so, isn't possible that the mail system adds them? 

Anything is possible. Nothing wrong with keeping an open mind until
the facts are illuminated. Healthier, probably.

> BTW, I understand that OP refers to me; but, what exactly does it
> stand for? Original Petitioner?

David Wright I believe explained this one already. But I do agree with
you that Original Petitioner would be much funnier.

   Wednesday, 6 July 2020. Now comes A****, Original Petitioner from
   K****a, with a humble request that The Committee hear that one's
   plea --for a limited waiver of The Keymap Hygiene Decree of 1972--
   pertaining to use of a personal device for strictly non-public
   purpose.

As a post-hoc justification of the practice, an OP is frequently a
newer personality, in the experience of a reader of the mailing
list. What is most salient about an OP in a given discussion, and them
alone, is their maximal proximity (in some sense at least) to the
subject of that discussion.

In other words, when you know fifty other things about somebody, their
name begins to take on meaning. Until then, it's just a handle.

OP, on the other hand, always conveys meaning in this sort of forum.

> > So those examples, fi and ½, illustrate the difference between
> > modification on output and input in English. I can't judge how the
> > OP views this, nor whether they are contravening some conventions
> > in their own computer culture by trying to make their changes.

But OP, wHaT WoUlD tHe CoMmUnItY tHiNk about your keyboard layout?

Have you petitioned your local Keyboard Zoning Board for a custom
keymap easement and filed the necessary declarations?

[snip]
> > May I ask how you happened to find the post about providing linux
> > support for the Breton keyboard
> > https://dominiko.livejournal.com/20206.html
>
> I found it by searching the net

Well, cool. All this has helped me make a new stab at reading the
unicode standard. Abstract material can become a more compelling
subject when you have concrete examples in mind.

> > Try again, for firefox-esr (and with a ~/.XCompose file that is not
> > befouled with nonbreaking spaces).
>
> > But make one change to the procedure. When you launch firefox-esr,
> > do so like this:
>
> > $ env GTK_IM_MODULE=xim firefox-esr
> > Let us know how that goes.
>
> Yet to do this. Will inform

Cool.


APPENDICES

TLDR: I waste some space here and point out a few things for fun:

   A. grep uses regexes, but you can tell it not to
   B. any command (grep included) gets its arguments from the shell
   C. the two equivalent grep command lines above
         i.  rely on the shell's interpretation of their first arg
 	ii. need not interpret their first arg as a regex to work

My aim here is to clarify a few specific things about how the grep
command

  $ grep $'\xc2\xa0' sometextfile

works, since OP seems intent to know that sort of thing.

[begin overly looong exposition]

%% A. grep uses Regular Expressions, but you can tell it not to

(TLDR: If you do not intend the pattern in grep's first argument to be
a Regular Expression, then use the -F option to avoid surprises.)

I do not know whether you are familiar with what are called "Regular
Expressions", often abbreviated as regex.

If you *are*, then good: grep typically uses them as a pattern to
filter a file line-by-line, usually so that it can print only those
lines that match the expression.

grep will assume that its first argument is such an expression, unless
you tell it otherwise.

But if you are *not* familiar with Regular Expressions, until you do
become familiar with them you should know that while *most* characters
that can appear in a regular expression do stand for themselves, not
all characters do (and the precise set of these last tends to vary
from regex-handling tool to regex-handling tool). Rather, a select few
are interpreted as special operators. (For example the character '.'
stands for any character.)

You can tell grep *not* to treat its first (non-option) argument as a
regular expression by giving it the option flag "-F" (which grep's
manual suggests is mnemonic for "Fixed-string mode"). For example:

  $ grep abc.efg # get lines that contain "abc" + any character + "efg"

  $ grep -F abc.efg # get lines that contain the string "abc.efg"

I have refrained from explaining here much of anything about regexes,
except how to tell grep you don't intend to give it one. But I
recommend learning more about them, if you have not already. They are
a useful abstraction when processing text. Many tools use them.


%% B. Any command (grep included) gets its arguments from the shell

That is to say, the shell gets the command line you enter first
--before grep ever sees it-- and will replace any parts that have
special meaning to the shell before it hands any arguments over to
grep.

In this command we rely on that very fact

  $ grep $'\xc2\xa0' sometextfile

to pass grep a single nonbreaking space as its filtering pattern.

Basically, when bash encounters

  $'somestuff'

it puts on its C costume and performs "C-style backslash
expansion". It replaces inside the sequence

  somestuff

certain backslash escape sequences with a corresponding
character:

  $ echo $'wello\n\thurled' # Expand \n to newline, \t to tab
  wello
  	hurled
  $ echo $'wello\x0a\x09hurled' # Same octets as above
  wello
  	hurled
  $

And if it hits a null byte (like '\x00' in C-style hex notation or
'\000' in octal) before encountering the single-quote that delimits
the end of somestuff, it considers the sequence complete, and ties off
the string.

  $ echo $'well\0\x0a\x09hurled' # \0 expands here to null octet/byte
  well
  $

You now ask: *Which* backslash escape sequences? *Which* corresponding
characters?

And all that is above my pay grade. So I'll simply quote here from
bash(1), under the section QUOTING, this passage:

   Words of the form $'string' are treated specially.  The word expands
   to string, with backslash-escaped characters replaced as specified
   by the ANSI C standard.  Backslash escape sequences, if present, are
   decoded as follows:

     \a     alert (bell)
     \b     backspace
     \e
     \E     an escape character
     \f     form feed
     \n     new line
     \r     carriage return
     \t     horizontal tab
     \v     vertical tab
     \\     backslash
     \'     single quote
     \"     double quote
     \?     question mark
     \nnn   the eight-bit character whose value is the octal value nnn (one
          to three octal digits)
     \xHH   the eight-bit character whose value is the hexadecimal value HH
          (one or two hex digits)
     \uHHHH the Unicode (ISO/IEC 10646) character whose value is the
          hexadecimal value HHHH (one to four hex digits)
     \UHHHHHHHH
          the Unicode (ISO/IEC 10646) character whose value is the
          hexadecimal value HHHHHHHH (one to eight hex digits)
     \cx    a control-x character

   The expanded result is single-quoted, as if the dollar sign had not been present.

And that *last* sentence means that aside from the backslash
expansions listed above, bash performs no further interpretation of
the enclosed string.

Why does it mean that? Because a sequence enclosed in bare single
quotes, with no prefixed '$' character, is protected from *any*
interpretation by bash.

So if you want bash to pass --as an argument to a command-- something
you enter on the command line without any doing interpretation first,
enclose that something in single quotes.

  $ grep -F '&amp;' somefile.txt  # without quotes, '&' and ';' chars would have special meaning to shell

I believe you've already been pointed to

  BashGuide/SpecialCharacters - Greg's Wiki
  http://mywiki.wooledge.org/BashGuide/SpecialCharacters

which discusses single quotes, among other characters bash treats
specially.


%% C. Tying (A) and (B) together with the present example

Notice that since nonbreaking space is just a normal character and not
a special operator inside a regex, these two amount to the same thing:

   $ grep -F $'\xc2\xa0' sometextfile
   $ grep $'\xc2\xa0' sometextfile

When you *know* you don't want (or need) grep to interpret a pattern
as a regex (for whatever reason), -F is a sensible option to use. In
this case, it was not necessary, but wouldn't have hurt anything
either.

[end of overly looong exposition]


-- 
Ce qui est important est rarement urgent
et ce qui est urgent est rarement important
-- Dwight David Eisenhower

[toc] | [prev] | [next] | [standalone]

#224867

On Wed, 15 Jul 2020 davidson wrote:

[snip]
> If you have both the package called "info" and sed installed, like I do
>
> $ dpkg-query -l

Yes, that is correct. I somehow manage to run a system with just three
packages installed... I'm *that* good.

$ dpkg-query -l info sed texinfo-doc-nonfree # Correction

> Desired=Unknown/Install/Remove/Purge/Hold
> | 
> Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
> |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
> ||/ Name                Version           Architecture Description
> +++-===================-=================-============-=================================================
> ii  info                6.5.0.dfsg.1-4+b1 amd64        Standalone GNU Info documentation browser
> ii  sed                 4.7-1             amd64        GNU stream editor for filtering/transforming text
> ii  texinfo-doc-nonfree 6.5.0-1           all          texinfo and info documentation that is non-free
>
> then you can do
[snip]

-- 
Ce qui est important est rarement urgent
et ce qui est urgent est rarement important
-- Dwight David Eisenhower

[toc] | [prev] | [next] | [standalone]

#224885

On Thursday 16 July 2020 4:54:09 AM IST davidson wrote:




> And so you have set up known environmental conditions for subsequent

> tests of that mechanism.

Yes, that was my intention




> which permits me to toggle my keyboard layout between two alternatives

> ("us" and "ru") by striking capslock.




I will do something similar once I get my  keyboard layout working properly.  




> So I am prone to conclude that whatever your layout may have been

> prior to step (1), it was not a dual-layout setup.

> (This relates more to my own curiosity than to your primary concern.)




No it is not. As of now, I am manually loading my Malayalam layout using 

setxkbmap




> And though it does not have to do with the test you are conducting

> here, I do remain curious about whether your day-to-day keymap table

> *does* include such mappings. It is entirely possible that it does.

> 

> I imagine at this point you may be able to work this out for yourself,

> if it interests you.




I checked and did not find the character.




> It is cool you experiment with what you don't understand. That is how

> mistakes are made, and mistakes are the best teacher of all.

That is true




> This is probably a good spot to recommend a coherent and

> comprehensive, well-curated beginner's guide to bash:

> 

>   BashGuide - Greg's Wiki

>   http://mywiki.wooledge.org/BashGuide

> 

> It is a shame if good learning resources are not used.

Will go through







> > 6) The command grep "W" .XCompose | tr $'\xc2\xa0' \! returns 

> > grep "W" .XCompose | tr $'\xc2\xa0' \!

> 

> This does not seem to make any sense. Where is the output?




Sorry for the mistake. The output is 

<W> : "This replaces W"







> It looks to me like you may have pasted a copy of the command, where

> you meant to paste its output.

Yes, that was the mistake.







> What you seem to be *showing* us, is that the command produced as

> output a copy of itself. But you don't seem to be *saying* this, or

> reasoning as if that were so, since whatever the output was, further

> below you seem inclined to think that it indicated the file contained

> no nonbreaking spaces.




Yes, that was my conclusion.




>   $ sed 'y/\xc2\xa0/%/' somefile

An off topic question: of sed, awk and perl, if I am to chose one to learn, 

which would you suggest. I wanted to do some substitutions. I read about them 

and decided on PERL because from what I understood, it has better support for 

regular expressions and do almost everything that sed and awk could do. Have I 

made the right decision?







> (However, one advantage of using the C-style byte-constants (\xHH)

> instead is that it is easy for everyone to see what they are, the web

> archive won't replace them with normal spaces, etc.)




Using the Unicode sequence also gives the same advantages, doesn't it?

I find it difficult to get the "translation" between the unicode code values and 

the hexadecimal/octal representation.







> David Wright I believe explained this one already. But I do agree with

> you that Original Petitioner would be much funnier.

Funny it is. To me, poster was (until a few emails back) a paper with 

something written in it and stuck on wall .




> OP, on the other hand, always conveys meaning in this sort of forum.

Sure, it does.







> But OP, wHaT WoUlD tHe CoMmUnItY tHiNk about your keyboard layout?

> 

> Have you petitioned your local Keyboard Zoning Board for a custom

> keymap easement and filed the necessary declarations?

With your sense of humor, I am unsure of the meaning of the weird casing of 

your sentence. On a serious note, I am unaware of anything like a local 

Keyboard Zoning Board for Kerala / India. Even if one is there, I will tackle 

it later. Anyway, there is no need for any formalities for customising my 

computer.




> I do not know whether you are familiar with what are called "Regular

> Expressions", often abbreviated as regex.

Yes, I know a bit about regex though not an expert. And I had read about sed 

earlier, but haven't used it favouring perl.




> %% B. Any command (grep included) gets its arguments from the shell

....

>   BashGuide/SpecialCharacters - Greg's Wiki

>   http://mywiki.wooledge.org/BashGuide/SpecialCharacters

This isn't something that I was familiar with. Will read. 




> [end of overly looong exposition]

Thanks for taking the time and the effort to explain

[toc] | [prev] | [next] | [standalone]

#224912 — how u mine 4 utf8 [was Re: Using .XCompose]

[Multipart message — attachments visible in raw view] — view raw

On Thu, 16 Jul 2020 Ajith R wrote:
> On Thursday 16 July 2020 4:54:09 AM IST davidson wrote:

[snip]
>>    $ sed 'y/\xc2\xa0/%/' somefile
>
> An off topic question: of sed, awk and perl, if I am to chose one to
> learn, which would you suggest. I wanted to do some substitutions. I
> read about them and decided on PERL because from what I understood,
> it has better support for regular expressions and do almost
> everything that sed and awk could do. Have I made the right
> decision?

[interesting question. postponing my answer]

>> (However, one advantage of using the C-style byte-constants (\xHH)
>> instead is that it is easy for everyone to see what they are, the
>> web archive won't replace them with normal spaces, etc.)
>
> Using the Unicode sequence also gives the same advantages, doesn't
> it?

Sure. The unicode code points for characters are a slightly more
abstract value for "what they are" than whatever bytes represent them
in some file, it seems to me. Depending on the purpose, this could be
a good difference.

> I find it difficult to get the "translation" between the unicode
> code values and the hexadecimal/octal representation.

In this case "the hexadecimal/octal representation" is the UTF-8
representation of unicode code points.

My understanding of text handling has a lot of weaknesses and fuzzy
spots that these discussions have made more apparent to me. And the
map from unicode code points to UTF-8 is one of them.

Or at least it was, until yesterday.

I recommend two references I found helpful, though I didn't read
either one especially carefully. I mostly just stared at the templates
(the UTF-8 fighters below), repeatedly asked myself "why did they do
it that way?" and then skimmed to confirm guesses.

First, utf-8(7).

  $ man 7 utf-8

Second, Wikipedia has a not-bad article about UTF-8.

Anyways, here is my ELI5/explainer how to represent a unicode code
point in UTF-8.


Mapping unicode to UTF-8

[If you don't/can't/won't display this with a fixed-width font, like
courier or something, you might prefer to stop reading here. The
references above are pretty good anyways.]

It helps me to notice that the UTF-8 encoding of unicode permits you
to know --whenever you grab some octet of bits-- where that byte
belongs in the following triage based on its Most Significant Bits
(MSBs).

The octet...

  1. represents an entire character on its own (MSB is 0)
  2. is a non-initial byte of a multi-byte character (MSBs are 10)
  3. is the initial byte of an N-byte character with N>1 (MSBs are N 1s trailed by a 0)

(Notice that you could rephrase (1-3) above in terms of the position
of the most significant zero bit.)

So given a unicode code point, do this:

  1. discard its leading 0 bits,
  2. count how many bits you have left (which is how many x you will
     need from the templates below), and
  3. pick your UTF-8 fighter:

1 octet  (7 bits)  0 x x x  x x x x
2 octets (11 bits) 1 1 0 x  x x x x  1 0 x x  x x x x
3 octets (16 bits) 1 1 1 0  x x x x  1 0 x x  x x x x  1 0 x x  x x x x
4 octets (21 bits) 1 1 1 1  0 x x x  1 0 x x  x x x x  1 0 x x  x x x x  1 0 x x  x x x x

Plug the unicode code point's bits into the rightward slots marked
with x in the template, and fill in any remaining more significant
(leftward) bits with zeros.

Synopsis of worked examples below:
[unicode code point]
[unicode representation, stripped of leading zero bits]
[smallest appropriate UTF-8 template]
[UTF-8 representation]

Example 1. Nonbreaking space character

U+00A0

A        0
1 0 1 0  0 0 0 0

1 1 0 x  x x x x  1 0 x x  x x x x

C        2        A        0
1 1 0 0  0 0 1 0  1 0 1 0  0 0 0 0


Example 2. Don't know the name of this one. I tell myself it
represents a velar nasal stop-initial syllable, but I would appreciate
correction.

U+0D19

1 1 0 1  0 0 0 1  1 0 0 1

So, we need to fit the 12 bits above into one of the four UTF-8
templates.

The smallest two can only fit 7 and 11 bits respectively, so we'll
need the next larger template, the three-octet one:

1 1 1 0  x x x x  1 0 x x  x x x x  1 0 x x  x x x x

E        0        B        4        9        9
1 1 1 0  x x x x  1 0 1 1  0 1 0 0  1 0 0 1  1 0 0 1


Example 3. a malayalam virama (sp?)

U+0D4D

1 1 0 1  0 1 0 0  1 1 0 1


1 1 1 0  x x x x  1 0 x x  x x x x  1 0 x x  x x x x

E     	 0     	  B   	   5   	    8 	     D
1 1 1 0  0 0 0 0  1 0 1 1  0 1 0 1  1 0 0 0  1 1 0 1

-- 
Ce qui est important est rarement urgent
et ce qui est urgent est rarement important
-- Dwight David Eisenhower

[toc] | [prev] | [next] | [standalone]

#224915 — UTF-8 Everywhere -- Re: how u mine 4 utf8 [was Re: Using .XCompose]

Just in case anyone missed the memo, essential reading [significantly beautified since last I looked, looks like some of the structure intent from the link at bottom has been usefully incorporated]:

   UTF-8 Everywhere
     Manifesto
   http://utf8everywhere.org/



Found in this dark and dingy corner of the dungeons:
   https://zenaan.github.io/zen/javadoc/zen/lang/string.html

[toc] | [prev] | [next] | [standalone]

#224916 — Re: how u mine 4 utf8 [was Re: Using .XCompose]

On Sat, 18 Jul 2020 davidson wrote:
> On Thu, 16 Jul 2020 Ajith R wrote:
>> On Thursday 16 July 2020 4:54:09 AM IST davidson wrote:

[snip]
> I recommend two references I found helpful, though I didn't read
> either one especially carefully. I mostly just stared at the templates
> (the UTF-8 fighters below), repeatedly asked myself "why did they do
> it that way?" and then skimmed to confirm guesses.
>
> First, utf-8(7).
>
> $ man 7 utf-8

These two too:

RFC 3629 - UTF-8, a transformation format of ISO 10646 (14 pages)
https://tools.ietf.org/html/rfc3629

The Unicode Consortium. The Unicode Standard.
http://www.unicode.org/versions/latest/

In present version 13.0.0 of the latter,

  * A brief feature summary of UTF-8 (in chapter 2, General Structure)
    on pages 37-38

  * on page 124 (in section 3.9) is a chart that I think makes the
    mapping from unicode code points to UTF-8 especially clear.

  * Section 3.9 "Unicode Encoding Forms". Formal definitions, if one
    enjoys that kind of thing.

[snip]
> So given a unicode code point, do this:
>
> 1. discard its leading 0 bits,
> 2. count how many bits you have left (which is how many x you will
>    need from the templates below), and
> 3. pick your UTF-8 fighter:

And pick the shortest one that works. Choosing a longer one will give
invalid result.

>
> 1 octet  (7 bits)  0xxx  xxxx
> 2 octets (11 bits) 110x  xxxx  10xx  xxxx
> 3 octets (16 bits) 1110  xxxx  10xx  xxxx  10xx  xxxx
> 4 octets (21 bits) 1111  0xxx  10xx  xxxx  10xx  xxxx  10xx  xxxx
[snip]

-- 
Ce qui est important est rarement urgent
et ce qui est urgent est rarement important
-- Dwight David Eisenhower

[toc] | [prev] | [next] | [standalone]

#224966 — Re: how u mine 4 utf8 [was Re: Using .XCompose]

Hi,


> These two too:

> RFC 3629 - UTF-8, a transformation format of ISO 10646 (14 pages)
> https://tools.ietf.org/html/rfc3629
> 
> The Unicode Consortium. The Unicode Standard.
> http://www.unicode.org/versions/latest/
> 
> In present version 13.0.0 of the latter,
>   * A brief feature summary of UTF-8 (in chapter 2, General Structure)
>   on pages 37-38
> 
>   * on page 124 (in section 3.9) is a chart that I think makes the
>    mapping from unicode code points to UTF-8 especially clear.

>   * Section 3.9 "Unicode Encoding Forms". Formal definitions, if one
>   enjoys that kind of thing.

Thanks for the references especially for giving the page number from the unicode standard (which is a pretty long document).

ajith

[toc] | [prev] | [next] | [standalone]

#224964 — Fw: how u mine 4 utf8 [was Re: Using .XCompose]

Hi,


> Mapping unicode to UTF-8
Thanks for the explanation and the references. I am starting to understand the idea. Hopefully, after going through the references everyone has given, I will understand it clearly.



> Example 2. Don't know the name of this one. I tell myself it
> represents a velar nasal stop-initial syllable, but I would appreciate
> correction.

> U+0D19


Yes, it is the velar nasal stop, though I don't understand how 'initial syllable' qualifies it . It is the same sound represented by 'ng' in king / sing. 

Thanks,
ajith

[toc] | [prev] | [next] | [standalone]

#224967 — Re: Fw: how u mine 4 utf8 [was Re: Using .XCompose]

[Multipart message — attachments visible in raw view] — view raw

On Sun, Jul 19, 2020 at 05:47 Ajith R <ajithramayyan@yahoo.co.in> wrote:

> Hi,
> > Mapping unicode to UTF-8


Ajith, if you deal with unicode scripting you really should look into Raku.
One of its many strengths is it is unicode (utf8) by default and has many
aids built-in for unicode grapheme handling including regexes and grapheme
introspection.

Cheers!

-Tom

[toc] | [prev] | [next] | [standalone]

#224969 — Re: how u mine 4 utf8 [was Re: Using .XCompose]

Hi,

Thanks for the suggestion.
No, I don't do unicode scripting regularly. 
My requirment was a one time requirment to analyse frequency of Malayalam text to design a keyboard layout.
But sure, I will keep your suggestion in mind if such requirment comes up frequently.

ajith

[toc] | [prev] | [next] | [standalone]

#224918 — sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose]

[Multipart message — attachments visible in raw view] — view raw

On Thu, 16 Jul 2020 Ajith R wrote:
> On Thursday 16 July 2020 4:54:09 AM IST davidson wrote:

[snip]
>>    $ sed 'y/\xc2\xa0/%/' somefile
>
> An off topic question: of sed, awk and perl, if I am to chose one to
> learn, which would you suggest. I wanted to do some substitutions. I
> read about them and decided on PERL because from what I understood,
> it has better support for regular expressions and do almost
> everything that sed and awk could do. Have I made the right
> decision?

This is sort of two questions in one.

First, there is a somewhat specific question about unspecified
substitutions. For all I know about these substitutions, you might
actually need XSLT to do them properly.

But then there is a more general invitation for advice on where to
invest time.

The second one is a little like one mechanic asking another:

   So erm, Carl. What's your signature ugly hack that you like to use
   to fix broken stuff in a totally unprofessional way? Glue? Duct
   tape?  Chewing gum?  I've been using this epoxy stuff for a while
   now, but I've heard some good things about chewing gum. Wanna tell
   me about it?

But first of all, I should reiterate that I lack expertise.

I lack expertise.

Second, I have no useful knowledge about real programming languages
(such as perl) to declare.

But as a peer, albeit a thick one who is not a perl-knowing brain
wizard, this is my experience:

IF

  1. I can fit it in less than 100 characters,
  2. it's a one-off task and
  3. I'm doing it from an interactive unix shell,

then I will sometimes use sed, most frequently to edit configuration
files. But I consider it a strictly write-only language.

Once I've written it, reviewed it carefully, tested that it does what
I intend, then I use it, check the results and forget I ever saw it.

If it gets more complex than that, and especially if I will ever have
to look at it again, then awk is a likely candidate if pattern-action
templates are the way to go and if the input is mere plain text (not
xml or something like that).

But wait. Let's back up a little.

First of all, why am I using sed to edit config files? Seriously, why?

This is like asking why I'm licking the window. Must be because I LIKE
TO LICK THE WINDOW. No accounting for taste, so they say, and they are
correct. Windows are just so delicious.

Yo-yo mastery results from dedication to an absurd hobby, but hey,
people enjoy it. And more power to 'em. It's harmless, which is more
than can be said for sed abuse.

I would probably be better off if I kicked the sed habit and took up
the yo-yo. Maybe learning some awk was a step in the right direction
after all. But what is interesting to me is that it almost seems like
the *more* I learn about unix, the *less* use I find for either one of
sed or awk.

And for whatever my opinion is worth (spoiler: literally less than
zero) I think you should absolutely use perl if it makes you happy.

I suspect, though, that it could be worthwhile to ask yourself why you
*need* to use it, whenever it seems like you do. For example, I ended
up using sed 'y///' because tr is currently *broken*, when faced with
multi-byte characters.

Unix has a pretty interesting collection of various small tools (which
"do one thing and do it well" as you may hear), and shells facilitate
hooking up their outputs and inputs. Almost as if they were made to do
just that.

This has been tonight's edition of Sweeping Generalities. I'm Richard
Blowhard. Thank you for tuning in.

Good night.

-- 
Ce qui est important est rarement urgent
et ce qui est urgent est rarement important
-- Dwight David Eisenhower

[toc] | [prev] | [next] | [standalone]

#224919 — Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose]

I use perl as a "slightly better sed", in particular the following commands I just cut and paste and modify for use, for many years now, to do (multi-)file search and replace:

```
# simple version:
perl -p -i[.bak] -e 's/xxx/yyy/[g];' $(readlink somefile) # readlink is necessary do not clobber symlinks

# hierarchical "find", handle ALL possible interesting filenames, including those that won't display:
find target/dir -type f -print0 | xargs -0 perl -p -i[.bak] -e 's/xxx/yyy/[g];'
```

So I am a regular user of perl, and know almost nothing about it :)

Perl is really great, even though I only know literally how to "sed" with it, and nothing else.

[toc] | [prev] | [next] | [standalone]

#224938 — Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose]

On Sat, Jul 18, 2020 at 3:40 AM Zenaan Harkness wrote:
> # simple version:
> perl -p -i[.bak] -e 's/xxx/yyy/[g];' $(readlink somefile) # readlink is necessary do not clobber symlinks

Speaking as not-an-expert on Raku (née Perl6), you'd write the above
something like:

user@mbook:~$ # One-liner flag set to line-by-line_&_autoprint (-pe)
user@mbook:~$ # "s///" substitute adverb set to global (:g)
user@mbook:~$ echo "aaa xxx ccc" | raku -pe 's:g/xxx/yyy/;'
aaa yyy ccc
user@mbook:~$

On Sat, Jul 18, 2020 at 3:40 AM Zenaan Harkness wrote:
> # hierarchical "find", handle ALL possible interesting filenames, including those that won't display:
> find target/dir -type f -print0 | xargs -0 perl -p -i[.bak] -e 's/xxx/yyy/[g];'
> ```

Again speaking as not-an-expert on Raku (née Perl6), the second half
of the call above doesn't appear too different from your first
example. However, if you want to try your hand at matching a filename
pattern in a directory, you can try the Raku script posted at:

https://examples.p6c.dev/categories/cookbook/09directories/09-06-filenames-matching-pattern.html
https://github.com/Raku/examples/blob/master/categories/cookbook/09directories/09-06-filenames-matching-pattern.p6

YMMV.

HTH, Bill.

[toc] | [prev] | [next] | [standalone]

#224920 — Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose]

[Multipart message — attachments visible in raw view] — view raw

On Sb, 18 iul 20, 09:59:46, davidson wrote:
> 
> But first of all, I should reiterate that I lack expertise.
> 
> I lack expertise.
> 
> Second, I have no useful knowledge about real programming languages
> (such as perl) to declare.

Same disclaimers apply to me.
 
> But as a peer, albeit a thick one who is not a perl-knowing brain
> wizard, this is my experience:

In my experience you will encounter sed (more often than awk) in shell 
scripts, either because some things are difficult or even impossible to 
do in pure shell script, or just because the author didn't know any 
better.

Learning perl could be useful if you *also* need a real programing 
language in which case you might as well consider python (should be 
easier to learn).

Kind regards,
Andrei
-- 
http://wiki.debian.org/FAQsFromDebianUser

[toc] | [prev] | [next] | [standalone]

#224933 — Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose]

Andrei POPESCU wrote:
> On Sb, 18 iul 20, 09:59:46, davidson wrote:
>>=20
>> But first of all, I should reiterate that I lack expertise.
>>=20
>> I lack expertise.
>>=20
>> Second, I have no useful knowledge about real programming languages
>> (such as perl) to declare.
>
> Same disclaimers apply to me.
>=20
>> But as a peer, albeit a thick one who is not a perl-knowing brain
>> wizard, this is my experience:
>
> In my experience you will encounter sed (more often than awk) in shell=20
> scripts, either because some things are difficult or even impossible to=20
> do in pure shell script, or just because the author didn't know any=20
> better.
>
> Learning perl could be useful if you *also* need a real programing=20
> language in which case you might as well consider python (should be=20
> easier to learn).
>
> Kind regards,
> Andrei

  except if you look at python3 pattern matching
stuff and throw up in disgust.

  i saw something recently that might make me 
revisit the topic in python, but until that 
actually gets implemented i will not be talking
much about python and pattern matching.

  i do use sed and awk and other unix standard
tools.  perl has not been needed for anything i've
done but if someone makes a useful tool and it is
done in perl i won't complain as long as it works
and does what the man page says it should do.


  songbird

[toc] | [prev] | [next] | [standalone]

#224928 — Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose]

[Multipart message — attachments visible in raw view] — view raw

On Sat, Jul 18, 2020 at 09:59:46AM +0000, davidson wrote:
> On Thu, 16 Jul 2020 Ajith R wrote:
> >On Thursday 16 July 2020 4:54:09 AM IST davidson wrote:
> 
> [snip]
> >>   $ sed 'y/\xc2\xa0/%/' somefile
> >
> >An off topic question: of sed, awk and perl, if I am to chose one to
> >learn, which would you suggest.

Sed (and awk) are much lighter than perl. So if you find yourself
peppering your shell scripts with lots of little calls to "that
other program", you'll be better off with those two.

Perl will be useful if you are doing the heavy lifting in perl
itself. Then it amortizes (I think this goes for both: computer
run time and your brain time).

Cheers
-- t

[toc] | [prev] | [next] | [standalone]

Page 2 of 5 — ← Prev page 1 [2] 3 4 5  Next page →

Back to top | Article view | linux.debian.user

csiph-web