Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.debian.user > #225015
| From | David Christensen <dpchrist@holgerdanske.com> |
|---|---|
| Newsgroups | linux.debian.user |
| Subject | Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] |
| Date | 2020-07-19 23:30 +0200 |
| Message-ID | <Aup2F-2Tl-13@gated-at.bofh.it> (permalink) |
| References | (11 earlier) <AsZ0B-7St-5@gated-at.bofh.it> <AtgXw-1P5-13@gated-at.bofh.it> <AtRX3-8jD-1@gated-at.bofh.it> <Au4KC-7nI-1@gated-at.bofh.it> <Auf3j-5dk-9@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
One last point I missed in my previous post -- big files. Many Perl
functions and/or libraries expect to do everything in RAM. This becomes
a problem when I want to compress, encrypt, save, and checksum 14 GiB
system drive images using a live drive and a Perl program in computers
without large amounts of memory (64 GiB?). So, I ended up writing a
Perl program that is truly a script -- it formulates and invokes
gzip(1), ccrypt(1), md5sum(1), and sha256sum(1) commands as required.
Traditional Bourne shell scripts are ideally suited for such tasks, and
the syntax can be cleaner than Perl in simple cases.
On 2020-07-19 03:46, Ajith R wrote:
>> I seem to recall that he puts Perl at the top of the
>> heap, and notes that Perl compatible regular expressions (PCRE) are
>> available via libraries in other programming languages.
>
> Thanks for confirming that I didn't make a wrong choice. Programs that claim to use PCRE don't support everything that PERL does.
My use of regular expressions is primarily via grep/ egrep and Perl. I
am curious to see what Erlang offers.
> I wanted to clean many documents (Wikipedia dump) to analyse the Malayalam content. As I was not comfortable with scripting, I was looking for some prorgam that could remove the foreign language text from the files. As, I could find none that could do the job, I had to use a Perl script with the line below (among others)
>
> s/[^\p{Block: Malayalam}\p{Block: Basic_Latin}\p{Block: General_Punctuation}\s]//g; # remove characters outside the specified unicode blocks.
>
> As of now, the simple substitute command of perl is sufficient for my requirements. Even that one command appears powerful compared to others.
That sounds like you need lexing and parsing, followed by your desired
processing. In simple cases, Perl regular expressions can accomplish
two or three of those tasks. But as complexity grows, you will need
more and more code. Writing a lexer/ parser in any language is a
non-trivial task. I have used the Perl 'LWP' library to parse HTML 4
pages, but I have not tried to parse HTML 5 and/or Wikipedia pages. I
would look for a library:
https://metacpan.org/search?q=parse
https://metacpan.org/search?q=wikipedia
I spent a little time with Raku (formerly Perl 6). AIUI improving
regular expressions was a design goal of Perl 6, and features were added
specifically for parsing. There is a book dedicated to the subject. I
have a friend who put some time into this area, and he seemed impressed:
https://www.apress.com/us/book/9781484232279
David
Back to linux.debian.user | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-06 07:40 +0200
Re: Using .XCompose Greg Wooledge <wooledg@eeg.ccf.org> - 2020-07-06 14:30 +0200
Re: Using .XCompose Zenaan Harkness <zenaan@freedbms.net> - 2020-07-06 14:50 +0200
Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-06 21:20 +0200
Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-06 22:40 +0200
Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-07 17:20 +0200
Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-07 08:40 +0200
Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-07 17:20 +0200
Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-09 23:50 +0200
Re: Using .XCompose [resend of xev_output_annotated.txt] davidson <davidson@freevolt.org> - 2020-07-10 03:40 +0200
Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-10 10:50 +0200
Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-11 10:40 +0200
Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-11 12:30 +0200
Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-12 17:00 +0200
Re: Using .XCompose David Wright <deblis@lionunicorn.co.uk> - 2020-07-13 06:00 +0200
Re: Using .XCompose Greg Wooledge <wooledg@eeg.ccf.org> - 2020-07-13 14:00 +0200
Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-13 20:30 +0200
Re: Using .XCompose Greg Wooledge <wooledg@eeg.ccf.org> - 2020-07-13 20:40 +0200
Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-13 20:50 +0200
Re: Using .XCompose David Wright <deblis@lionunicorn.co.uk> - 2020-07-13 21:50 +0200
Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-14 05:40 +0200
Fw: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-15 16:50 +0200
Fw: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-15 17:10 +0200
Fw: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-15 17:20 +0200
Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-16 01:30 +0200
Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-16 02:40 +0200
Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-16 20:40 +0200
how u mine 4 utf8 [was Re: Using .XCompose] davidson <davidson@freevolt.org> - 2020-07-18 03:20 +0200
UTF-8 Everywhere -- Re: how u mine 4 utf8 [was Re: Using .XCompose] Zenaan Harkness <zenaan@freedbms.net> - 2020-07-18 07:40 +0200
Re: how u mine 4 utf8 [was Re: Using .XCompose] davidson <davidson@freevolt.org> - 2020-07-18 09:20 +0200
Re: how u mine 4 utf8 [was Re: Using .XCompose] Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-19 13:00 +0200
Fw: how u mine 4 utf8 [was Re: Using .XCompose] Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-19 12:50 +0200
Re: Fw: how u mine 4 utf8 [was Re: Using .XCompose] Tom Browder <tom.browder@gmail.com> - 2020-07-19 13:10 +0200
Re: how u mine 4 utf8 [was Re: Using .XCompose] Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-19 13:50 +0200
sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] davidson <davidson@freevolt.org> - 2020-07-18 12:10 +0200
Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] Zenaan Harkness <zenaan@freedbms.net> - 2020-07-18 12:50 +0200
Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] William Michels <wjm1@caa.columbia.edu> - 2020-07-18 22:00 +0200
Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] Andrei POPESCU <andreimpopescu@gmail.com> - 2020-07-18 13:20 +0200
Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] songbird <songbird@anthive.com> - 2020-07-18 21:00 +0200
Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] <tomas@tuxteam.de> - 2020-07-18 16:00 +0200
Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] mick crane <mick.crane@gmail.com> - 2020-07-18 22:40 +0200
Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] <tomas@tuxteam.de> - 2020-07-18 23:10 +0200
Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] mick crane <mick.crane@gmail.com> - 2020-07-18 23:30 +0200
Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] <tomas@tuxteam.de> - 2020-07-19 10:40 +0200
Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] mick crane <mick.crane@gmail.com> - 2020-07-20 13:20 +0200
Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] David Christensen <dpchrist@holgerdanske.com> - 2020-07-19 01:50 +0200
Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-19 12:50 +0200
Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] David Christensen <dpchrist@holgerdanske.com> - 2020-07-19 23:30 +0200
Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-19 13:40 +0200
Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] William Michels <wjm1@caa.columbia.edu> - 2020-07-19 21:50 +0200
Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] David Wright <deblis@lionunicorn.co.uk> - 2020-07-23 04:00 +0200
Re: sed:awk:perl::rock:paper:chainsaw [was Re: Using .XCompose] songbird <songbird@anthive.com> - 2020-07-23 12:50 +0200
Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-16 02:00 +0200
Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-16 20:40 +0200
Re: Using .XCompose David Wright <deblis@lionunicorn.co.uk> - 2020-07-11 17:00 +0200
Tex characters Teemu Likonen <tlikonen@iki.fi> - 2020-07-11 18:20 +0200
Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-11 22:40 +0200
Re: Using .XCompose Greg Wooledge <wooledg@eeg.ccf.org> - 2020-07-13 14:00 +0200
Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-16 07:50 +0200
Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-11 18:10 +0200
Re: Using .XCompose Greg Wooledge <wooledg@eeg.ccf.org> - 2020-07-20 15:30 +0200
Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-20 15:30 +0200
Re: Using .XCompose David Wright <deblis@lionunicorn.co.uk> - 2020-07-20 18:20 +0200
Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-10 12:50 +0200
Re: [offlist] posting style [was Re: Using .XCompose] Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-08 05:00 +0200
Re: [offlist] posting style [was Re: Using .XCompose] Andrei POPESCU <andreimpopescu@gmail.com> - 2020-07-08 07:00 +0200
Re: [not-so-offlist] posting style [was Re: Using .XCompose] davidson <davidson@freevolt.org> - 2020-07-08 07:10 +0200
Re: [offlist] Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-08 05:00 +0200
Re: [offlist] Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-08 05:20 +0200
Re: [offlist] Re: Using .XCompose Zenaan Harkness <zenaan@freedbms.net> - 2020-07-08 05:40 +0200
Re: [offlist] Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-08 08:00 +0200
Re: Using .XCompose David Wright <deblis@lionunicorn.co.uk> - 2020-07-07 00:40 +0200
tangent on man page locations [was Re: Using .XCompose] davidson <davidson@freevolt.org> - 2020-07-07 01:10 +0200
Re: tangent on man page locations [was Re: Using .XCompose] David Wright <deblis@lionunicorn.co.uk> - 2020-07-07 03:50 +0200
Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-07 11:50 +0200
Re: Using .XCompose Greg Wooledge <wooledg@eeg.ccf.org> - 2020-07-07 13:40 +0200
Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-08 16:30 +0200
Re: Using .XCompose David Wright <deblis@lionunicorn.co.uk> - 2020-07-09 02:20 +0200
Re: Using .XCompose Zenaan Harkness <zenaan@freedbms.net> - 2020-07-09 04:20 +0200
Re: Using .XCompose Zenaan Harkness <zenaan@freedbms.net> - 2020-07-08 01:50 +0200
Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-08 02:30 +0200
Fw: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-08 05:10 +0200
Re: Using .XCompose Ajith R <ajithramayyan@yahoo.co.in> - 2020-07-08 18:40 +0200
Re: Using .XCompose Zenaan Harkness <zenaan@freedbms.net> - 2020-07-09 04:00 +0200
Re: Using .XCompose Zenaan Harkness <zenaan@freedbms.net> - 2020-07-10 02:40 +0200
Re: Using .XCompose The Wanderer <wanderer@fastmail.fm> - 2020-07-10 02:50 +0200
Re: Using .XCompose davidson <davidson@freevolt.org> - 2020-07-10 05:00 +0200
csiph-web