Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #193310 > unrolled thread

Does bash have a tool ?

Back to article view | Back to linux.debian.user

Contents

  Does bash have a tool ? Richard Owlett <rowlett@cloud85.net> - 2018-03-04 16:30 +0100
    Re: Does bash have a tool ? rhkramer@gmail.com - 2018-03-04 17:10 +0100
      Re: Does bash have a tool ? Richard Owlett <rowlett@cloud85.net> - 2018-03-04 17:30 +0100
        Re: Does bash have a tool ? rhkramer@gmail.com - 2018-03-04 18:10 +0100
          Re: Does bash have a tool ? Richard Owlett <rowlett@cloud85.net> - 2018-03-04 20:10 +0100
            Re: Does bash have a tool ? Richard Hector <richard@walnut.gen.nz> - 2018-03-05 03:20 +0100
              Re: Does bash have a tool ? Richard Owlett <rowlett@cloud85.net> - 2018-03-05 04:10 +0100
                Re: Does bash have a tool ? John Hasler <jhasler@newsguy.com> - 2018-03-05 04:30 +0100
                  Re: Does bash have a tool ? Richard Owlett <rowlett@cloud85.net> - 2018-03-05 10:20 +0100
                    Re: Does bash have a tool ? John Hasler <jhasler@newsguy.com> - 2018-03-05 15:30 +0100
                      Re: Does bash have a tool ? Curt <curty@free.fr> - 2018-03-05 16:20 +0100
                        Re: Does bash have a tool ? John Hasler <jhasler@newsguy.com> - 2018-03-05 18:10 +0100
              Re: Does bash have a tool ? rhkramer@gmail.com - 2018-03-05 05:40 +0100
        Re: Does bash have a tool ? John Hasler <jhasler@newsguy.com> - 2018-03-04 19:20 +0100
          Re: Does bash have a tool ? David Wright <deblis@lionunicorn.co.uk> - 2018-03-05 02:00 +0100
            Re: Does bash have a tool ? Richard Owlett <rowlett@cloud85.net> - 2018-03-05 04:20 +0100
        Re: Does bash have a tool ? Erik Christiansen <dvalin@internode.on.net> - 2018-03-05 06:50 +0100
          Re: Does bash have a tool ? Richard Owlett <rowlett@cloud85.net> - 2018-03-05 10:10 +0100
            Re: Does bash have a tool ? Erik Christiansen <dvalin@internode.on.net> - 2018-03-05 11:20 +0100
    Re: Does bash have a tool ? Curt <curty@free.fr> - 2018-03-04 18:30 +0100
      Re: Does bash have a tool ? Richard Owlett <rowlett@cloud85.net> - 2018-03-05 11:30 +0100
        A recomended guide to vim - was {Re: Does bash have a tool ?} Richard Owlett <rowlett@cloud85.net> - 2018-03-07 16:40 +0100
          Re: A recomended guide to vim - was {Re: Does bash have a tool ?} Curt <curty@free.fr> - 2018-03-07 22:10 +0100
            Re: A recomended guide to vim - was {Re: Does bash have a tool ?} Richard Owlett <rowlett@cloud85.net> - 2018-03-09 13:40 +0100
          Re: A recomended guide to vim - was {Re: Does bash have a tool ?} Cousin Stanley <cousinstanley@gmail.com> - 2018-03-09 14:20 +0100
    An answer - was [Re: Does bash have a tool ?] Richard Owlett <rowlett@cloud85.net> - 2018-03-05 01:00 +0100
      Re: An answer - was [Re: Does bash have a tool ?] Forest Dean Feighner <forest.feighner@gmail.com> - 2018-03-05 02:00 +0100
    Re: Does bash have a tool ? Gregory Seidman <gsslist+debian@anthropohedron.net> - 2018-03-05 22:50 +0100

Page 1 of 2  [1] 2  Next page →

#193310 — Does bash have a tool ?

My eventual goal is to create a personalized FAQ.
To that end I've collected all my outgoing mail which DOES NOT have 
"Re:" in the Subject into a single file {used standard SeaMonkey tools}.

Using a text editor's search&replace function I've placed "KEY1" at the 
beginning of the body of each message. Similarly, I've placed "KEY2" at 
the end of each body.

Searches led to <http://www.dsl.org/cookbook/cookbook_16.html> which 
describes tools to do word frequency tasks, primarily with bash builtins.

First I need to eliminate the irrelevant text between "KEY2" of the 
previous message and "KEY1" of the message of interest. It should be 
straight forard to do in BASIC.

But is there an already tested function for that?
TIA

[toc] | [next] | [standalone]

#193312

On Sunday, March 04, 2018 10:26:51 AM Richard Owlett wrote:
> First I need to eliminate the irrelevant text between "KEY2" of the
> previous message and "KEY1" of the message of interest. It should be
> straight forard to do in BASIC.
> 
> But is there an already tested function for that?

Well, I would tend to use awk, sed, or perl, or even a "decent" text editor 
(or word processor) with decent support for regular expressions in the search 
and replace function.  Multiline regexes will be helpful, at the moment I 
don't remember if they are supported in kate / kwrite.  

I have done stuff like this in nedit, even saving a series of search and 
replace functions as, essentially, a macro / script (I forget what nedit calls 
it).

If you expect to do this often, I would think that developing a perl script 
would be worth the effort.

[toc] | [prev] | [next] | [standalone]

#193313

On 03/04/2018 10:02 AM, rhkramer@gmail.com wrote:
> On Sunday, March 04, 2018 10:26:51 AM Richard Owlett wrote:
>> First I need to eliminate the irrelevant text between "KEY2" of the
>> previous message and "KEY1" of the message of interest. It should be
>> straight forard to do in BASIC.
>>
>> But is there an already tested function for that?
> 
> Well, I would tend to use awk, sed, or perl, or even a "decent" text editor
> (or word processor) with decent support for regular expressions in the search
> and replace function.  Multiline regexes will be helpful, at the moment I
> don't remember if they are supported in kate / kwrite.
> 
> I have done stuff like this in nedit, even saving a series of search and
> replace functions as, essentially, a macro / script (I forget what nedit calls
> it).
> 
> If you expect to do this often, I would think that developing a perl script
> would be worth the effort.
> 
> 

I don't have any background in Perl and the last formal course in 
programming was in the 60's.

However awk and/or sed may be what I'm looking for and are well 
documented. Your description of nedit is interesting. I'll investigate. 
I'm on my way out at the moment.

Thank you.

[toc] | [prev] | [next] | [standalone]

#193314

On Sunday, March 04, 2018 11:28:55 AM Richard Owlett wrote:
> I don't have any background in Perl and the last formal course in
> programming was in the 60's.

Mine was very early 70's ('71)

> However awk and/or sed may be what I'm looking for and are well
> documented. Your description of nedit is interesting. I'll investigate.
> I'm on my way out at the moment.

Oh, I forgot, nedit doesn't handle UTF-8 (or any form of Unicode)--that could 
be a problem for you.

[toc] | [prev] | [next] | [standalone]

#193320

On 03/04/2018 11:02 AM, rhkramer@gmail.com wrote:
> On Sunday, March 04, 2018 11:28:55 AM Richard Owlett wrote:
>> I don't have any background in Perl and the last formal course in
>> programming was in the 60's.
> 
> Mine was very early 70's ('71)
> 
>> However awk and/or sed may be what I'm looking for and are well
>> documented. Your description of nedit is interesting. I'll investigate.
>> I'm on my way out at the moment.
> 
> Oh, I forgot, nedit doesn't handle UTF-8 (or any form of Unicode)--that could
> be a problem for you.
> 
> 

You youngsters. KSR-35 and 026 are my speed ;}

[toc] | [prev] | [next] | [standalone]

#193339

[Multipart message — attachments visible in raw view] — view raw

On 05/03/18 07:59, Richard Owlett wrote:
> On 03/04/2018 11:02 AM, rhkramer@gmail.com wrote:
>> On Sunday, March 04, 2018 11:28:55 AM Richard Owlett wrote:
>>> I don't have any background in Perl and the last formal course in
>>> programming was in the 60's.
>>
>> Mine was very early 70's ('71)
>>
>>> However awk and/or sed may be what I'm looking for and are well
>>> documented. Your description of nedit is interesting. I'll investigate.
>>> I'm on my way out at the moment.
>>
>> Oh, I forgot, nedit doesn't handle UTF-8 (or any form of
>> Unicode)--that could
>> be a problem for you.
>>
>>
> 
> You youngsters. KSR-35 and 026 are my speed ;}

A KSR-35 seems to be a sniper rifle ... Ah, but an IBM 026 might be a
key punch? So I guess the KSR-35 is for punching paper tape at a
distance. Communication was primitive in those days, wasn't it :-)

Richard

[toc] | [prev] | [next] | [standalone]

#193343

On 03/04/2018 08:18 PM, Richard Hector wrote:
> [snip ;]
>> You youngsters. KSR-35 and 026 are my speed ;}
> 
> A KSR-35 seems to be a sniper rifle ... Ah, but an IBM 026 might be a
> key punch? So I guess the KSR-35 is for punching paper tape at a
> distance. Communication was primitive in those days, wasn't it :-)
> 

*ROFL*
Do youngsters here recognize they are being teased?
a KSR-35 wrote/read paper tape
a 026 punched Hollerith(sp?) cards
We'll leave to their anemic imagination what "high speed paper tape" 
might be! *ROFL*

[toc] | [prev] | [next] | [standalone]

#193346

Richard Owlett writes:
> Do youngsters here recognize they are being teased?
> a KSR-35 wrote/read paper tape
> a 026 punched Hollerith(sp?) cards
> We'll leave to their anemic imagination what "high speed paper tape" might
> be! *ROFL*

You're describing an ASR (Automatic Send Receive) teleprinter.  KSR
means Keyboard Send Receive: no tape.

The Model 19 was more fun.
-- 
John Hasler 
jhasler@newsguy.com
Elmwood, WI USA

[toc] | [prev] | [next] | [standalone]

#193358

On 03/04/2018 09:28 PM, John Hasler wrote:
> Richard Owlett writes:
>> Do youngsters here recognize they are being teased?
>> a KSR-35 wrote/read paper tape
>> a 026 punched Hollerith(sp?) cards
>> We'll leave to their anemic imagination what "high speed paper tape" might
>> be! *ROFL*
> 
> You're describing an ASR (Automatic Send Receive) teleprinter.  KSR
> means Keyboard Send Receive: no tape.

You may be correct. It has been the the better part of a half-century. I 
recall two almost identical 20mA devices adjacent to PDP11's, 
with/without paper tape readers on the side.
> 
> The Model 19 was more fun.

Don't recognize the model.

[toc] | [prev] | [next] | [standalone]

#193368

I wrote:
> The Model 19 was more fun.

Richard Owlett writes:
> Don't recognize the model.

And here I thought you went way back.  The Model 19 did 5-level baudot.
We used them for amateur radio teletype in the 60s.  The machine had
some real heft to it.  It was driven by a 1/4 hp motor.  Lots of
rotating parts and a heavy cast-iron carriage the jumped up and down
when the machine shifted.
-- 
John Hasler 
jhasler@newsguy.com
Elmwood, WI USA

[toc] | [prev] | [next] | [standalone]

#193374

On 2018-03-05, John Hasler <jhasler@newsguy.com> wrote:
> I wrote:
>> The Model 19 was more fun.
>
> Richard Owlett writes:
>> Don't recognize the model.
>
> And here I thought you went way back.  The Model 19 did 5-level baudot.
> We used them for amateur radio teletype in the 60s.  The machine had
> some real heft to it.  It was driven by a 1/4 hp motor.  Lots of
> rotating parts and a heavy cast-iron carriage the jumped up and down
> when the machine shifted.

This thing (or something like it)?

http://jproc.ca/crypto/teletype/19asr_b.jpg

-- 
Bah, the latest news, the latest news is not the last.
Samuel Beckett

[toc] | [prev] | [next] | [standalone]

#193388

That looks like a real Model 19 ASR.
-- 
John Hasler 
jhasler@newsguy.com
Elmwood, WI USA

[toc] | [prev] | [next] | [standalone]

#193349

On Sunday, March 04, 2018 09:18:56 PM Richard Hector wrote:
> A KSR-35 seems to be a sniper rifle ... Ah, but an IBM 026 might be a
> key punch? So I guess the KSR-35 is for punching paper tape at a
> distance. Communication was primitive in those days, wasn't it :-)

LOL!!

[toc] | [prev] | [next] | [standalone]

#193316

Richard Owlett writes:
> I don't have any background in Perl and the last formal course in
> programming was in the 60's.

Perl is just bash, ed, sed, awk, grep, etc all smushed together into
one.

> However awk and/or sed may be what I'm looking for and are well
> documented.

It's exactly what those tools are for.  Don't forget the rest of the
toolkit.
-- 
John Hasler 
jhasler@newsguy.com
Elmwood, WI USA

[toc] | [prev] | [next] | [standalone]

#193337

On Sun 04 Mar 2018 at 12:14:36 (-0600), John Hasler wrote:
> Richard Owlett writes:
> > I don't have any background in Perl and the last formal course in
> > programming was in the 60's.
> 
> Perl is just bash, ed, sed, awk, grep, etc all smushed together into
> one.
> 
> > However awk and/or sed may be what I'm looking for and are well
> > documented.
> 
> It's exactly what those tools are for.  Don't forget the rest of the
> toolkit.

There's a case here for using specialist tools as we know we're
dealing with emails.

I'd start by using a client like mutt to sort out which messages to
process. The easiest way of doing this is¹ to copy the outbox, open
the copy and sort the index by Subject. (o s in mutt.)

Because Re: is normally ignored in mutt when sorting thus, type
:set reply_regexp=""
to temporarily defeat that. As a result, all the Re: messages now
sort together.

On the first Re: in the list, press and hold down the <d> key to
mark them as deleted. $ will now purge them. Quit.

Now I would run formail on the shrunken mailbox using the -s option
to run itself recursively, thus:

$ formail -s formail -I "" < path/to/shrunken-mailbox > file-of-concatenated-bodies

The first formail splits the mailbox and pipes each message to the
second one which strips the headers, pipes it back to the first which
pipes it into the output file.

¹the process may differ from client to client.

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#193344

On 03/04/2018 06:51 PM, David Wright wrote:
> On Sun 04 Mar 2018 at 12:14:36 (-0600), John Hasler wrote:
>> Richard Owlett writes:
>>> I don't have any background in Perl and the last formal course in
>>> programming was in the 60's.
>>
>> Perl is just bash, ed, sed, awk, grep, etc all smushed together into
>> one.
>>
>>> However awk and/or sed may be what I'm looking for and are well
>>> documented.
>>
>> It's exactly what those tools are for.  Don't forget the rest of the
>> toolkit.
> 
> There's a case here for using specialist tools as we know we're
> dealing with emails.
> 
> I'd start by using a client like mutt to sort out which messages to
> process. The easiest way of doing this is¹ to copy the outbox, open
> the copy and sort the index by Subject. (o s in mutt.)
> 
> Because Re: is normally ignored in mutt when sorting thus, type
> :set reply_regexp=""
> to temporarily defeat that. As a result, all the Re: messages now
> sort together.
> 
> On the first Re: in the list, press and hold down the <d> key to
> mark them as deleted. $ will now purge them. Quit.
> 
> Now I would run formail on the shrunken mailbox using the -s option
> to run itself recursively, thus:
> 
> $ formail -s formail -I "" < path/to/shrunken-mailbox > file-of-concatenated-bodies
> 
> The first formail splits the mailbox and pipes each message to the
> second one which strips the headers, pipes it back to the first which
> pipes it into the output file.
> 
> ¹the process may differ from client to client.
> 
> Cheers,
> David.
> 
> 


IOW I should become educated

IOW "I should think"
I did not recognize appropriate direction

[toc] | [prev] | [next] | [standalone]

#193352

On 04.03.18 10:28, Richard Owlett wrote:
> 
> I don't have any background in Perl and the last formal course in
> programming was in the 60's.
> 
> However awk and/or sed may be what I'm looking for and are well documented.
> Your description of nedit is interesting. I'll investigate. I'm on my way
> out at the moment.

Awk and Sed were created in the unix big bang, and might indeed be just
be your speed. (Some enhancements have been added to Awk since, such as
co-processes, but the current use case won't need that.)

It seems you're onto some doco, but this is hard to beat:
The Addison Wesley "The AWK programming Language" is a slender concise
exposition of the tool's use and capabilities, written by Awk's authors,
Aho, Weinberger, and Kernighan. The permuted index greatly eases finding
stuff in it.

This is a quick start: www.gnu.org/software/gawk/manual/gawk.html
Like the Addison Wesley dead-tree book, its "Sample Programs" shows that
Awk has a C-like syntax, supercharged with posix regexes, associative
arrays, etc.

A quick intro to regular expressions can be found in:
$ man regex
(Awk defaults to the more compact and easily read EREs, rather than
obsolete BREs.)

The awk manpage is a good reference, but as with all manpages, not a
tutorial.

Over the last three decades or more, I've found that Awk and shell have
more than adequately handled all my text munging requirements, right up
to a C-code generator taking structured English input to create
heterogenous communities of interacting state machines.

Erik

[toc] | [prev] | [next] | [standalone]

#193357

On 03/04/2018 11:40 PM, Erik Christiansen wrote:
> On 04.03.18 10:28, Richard Owlett wrote:
>>
>> I don't have any background in Perl and the last formal course in
>> programming was in the 60's.
>>
>> However awk and/or sed may be what I'm looking for and are well documented.
>> Your description of nedit is interesting. I'll investigate. I'm on my way
>> out at the moment.
> 
> Awk and Sed were created in the unix big bang, and might indeed be just
> be your speed. (Some enhancements have been added to Awk since, such as
> co-processes, but the current use case won't need that.)
> 
> It seems you're onto some doco, but this is hard to beat:
> The Addison Wesley "The AWK programming Language" is a slender concise
> exposition of the tool's use and capabilities, written by Awk's authors,
> Aho, Weinberger, and Kernighan. The permuted index greatly eases finding
> stuff in it.
> 
> This is a quick start: www.gnu.org/software/gawk/manual/gawk.html
> Like the Addison Wesley dead-tree book, its "Sample Programs" shows that
> Awk has a C-like syntax, supercharged with posix regexes, associative
> arrays, etc.
> 
> A quick intro to regular expressions can be found in:
> $ man regex
> (Awk defaults to the more compact and easily read EREs, rather than
> obsolete BREs.)
> 
> The awk manpage is a good reference, but as with all manpages, not a
> tutorial.
> 
> Over the last three decades or more, I've found that Awk and shell have
> more than adequately handled all my text munging requirements, right up
> to a C-code generator taking structured English input to create
> heterogenous communities of interacting state machines.
> 
> Erik

As to "manpages not a tutorial" <snicker> <snicker> *ROFL*
I'll admit content is there, but ...  ....
I've been referred to vim. Although awk and cousins are probably under 
the surface, vim.org is fascinating and accessible to end users such as 
myself.

[toc] | [prev] | [next] | [standalone]

#193360

On 05.03.18 03:06, Richard Owlett wrote:
> 
> As to "manpages not a tutorial" <snicker> <snicker> *ROFL*
> I'll admit content is there, but ...  ....
> I've been referred to vim. Although awk and cousins are probably under the
> surface, vim.org is fascinating and accessible to end users such as myself.

A good text editor (and it's hard to be as good as Vim) will reduce the
workload of repeated edits which are not consistent enough to be easily
automated with Sed or Awk.

It's amazing how much can be done by repeating the last search with 'n',
then deciding whether to hit '.' to repeat the last edit (e.g. a
change-word or delete-to some pattern)

There's also the ability to put any sequence of actions onto a key.
If you arc up vim and type ":help q", the on-line help will describe how
to start keystroke recording with "qx", where x is the key you'll use to
re-run the macro, via "@x". Although both search and action can be
packed into the macro, it's usually better to repeat the search (for
next edit target) with 'n', so you get to see the "before", then hit @x
to run the macro, then eye the results before repeating.

Vim is modal editor, with a normal mode and a couple of editing modes.
Thinking you're in insert mode while in normal mode, can in a few
keystrokes wreak random havoc, requiring a stab at Escape then a few
stabs at 'u', the undo button, to restore order. (If too many undos,
then ^r is redo.) To reduce the need to rely on memory, a line with

set showmode

in ~/.vimrc will display the current mode in bold text. (Blank for
normal mode) If you also add these lines:

" Cursor Appearance and behaviour:                                           !!!
" (Insert_Mode == Green, Normal_Mode == Red)
if &term =~ "xterm"
   let &t_SI = "\<Esc>]12;green\x7"
   let &t_EI = "\<Esc>]12;red\x7"
endif

the cursor will also serve as a reminder, so you don't need to glance to
the bottom of the screen.

What serves best varies not only with the task and the tweaks you've
made to the tools, but what techniques you have absorbed on the
text-whacking journey.

Erik

[toc] | [prev] | [next] | [standalone]

#193315

On 2018-03-04, Richard Owlett <rowlett@cloud85.net> wrote:
> My eventual goal is to create a personalized FAQ.
> To that end I've collected all my outgoing mail which DOES NOT have 
> "Re:" in the Subject into a single file {used standard SeaMonkey tools}.
>
> Using a text editor's search&replace function I've placed "KEY1" at the 
> beginning of the body of each message. Similarly, I've placed "KEY2" at 
> the end of each body.
>
> Searches led to <http://www.dsl.org/cookbook/cookbook_16.html> which 
> describes tools to do word frequency tasks, primarily with bash builtins.
>
> First I need to eliminate the irrelevant text between "KEY2" of the 
> previous message and "KEY1" of the message of interest. It should be 
> straight forard to do in BASIC.

vim (hit escape to get into command mode)

:/KEY2/+1;/KEY1/-1d 

will delete everything between KEY2 and KEY1, excluding the 
matching lines.

I think. Well, I just gave it a trivial workout and it worked (for a
trivial case).

> But is there an already tested function for that?
> TIA
>
>
>
>


-- 
Bah, the latest news, the latest news is not the last.
Samuel Beckett

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | linux.debian.user

csiph-web