Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > linux.debian.user > #262540 > unrolled thread

can you parse and "tail" at once? (and if you can't why not?)

Started byAlbretch Mueller <lbrtchx@gmail.com>
First post2023-10-22 18:40 +0200
Last post2023-10-24 00:20 +0200
Articles 17 — 10 participants

Back to article view | Back to linux.debian.user


Contents

  can you parse and "tail" at once? (and if you can't why not?) Albretch Mueller <lbrtchx@gmail.com> - 2023-10-22 18:40 +0200
    Re: can you parse and "tail" at once? (and if you can't why not?) Charles Curley <charlescurley@charlescurley.com> - 2023-10-22 19:10 +0200
      Re: can you parse and "tail" at once? (and if you can't why not?) Greg Wooledge <greg@wooledge.org> - 2023-10-22 19:50 +0200
        Re: can you parse and "tail" at once? (and if you can't why not?) Albretch Mueller <lbrtchx@gmail.com> - 2023-10-22 23:00 +0200
          Re: can you parse and "tail" at once? (and if you can't why not?) Andy Smith <andy@strugglers.net> - 2023-10-22 23:20 +0200
            Re: can you parse and "tail" at once? (and if you can't why not?) gene heskett <gheskett@shentel.net> - 2023-10-23 00:30 +0200
              Re: can you parse and "tail" at once? (and if you can't why not?) Albretch Mueller <lbrtchx@gmail.com> - 2023-10-23 03:40 +0200
                Re: can you parse and "tail" at once? (and if you can't why not?) Greg Wooledge <greg@wooledge.org> - 2023-10-23 04:20 +0200
                  Re: can you parse and "tail" at once? (and if you can't why not?) <tomas@tuxteam.de> - 2023-10-23 06:50 +0200
                  Shell-script variable name case [WAS: Re: can you parse and "tail" at  once? (and if you can't why not?)] David <bouncingcats@gmail.com> - 2023-10-23 16:00 +0200
                Re: can you parse and "tail" at once? (and if you can't why not?) Andy Smith <andy@strugglers.net> - 2023-10-23 22:10 +0200
          Re: can you parse and "tail" at once? (and if you can't why not?) Patrick Wiseman <pwiseman@gmail.com> - 2023-10-23 04:50 +0200
    Re: can you parse and "tail" at once? (and if you can't why not?) Roberto C. Sánchez <roberto@debian.org> - 2023-10-22 19:50 +0200
    Re: can you parse and "tail" at once? (and if you can't why not?) Andy Smith <andy@strugglers.net> - 2023-10-22 20:00 +0200
    Re: can you parse and "tail" at once? (and if you can't why not?) Pankaj Jangid <pankaj@codeisgreat.org> - 2023-10-23 10:00 +0200
      Re: can you parse and "tail" at once? (and if you can't why not?) Greg Wooledge <greg@wooledge.org> - 2023-10-23 14:20 +0200
        Re: can you parse and "tail" at once? (and if you can't why not?) Albretch Mueller <lbrtchx@gmail.com> - 2023-10-24 00:20 +0200

#262540 — can you parse and "tail" at once? (and if you can't why not?)

FromAlbretch Mueller <lbrtchx@gmail.com>
Date2023-10-22 18:40 +0200
Subjectcan you parse and "tail" at once? (and if you can't why not?)
Message-ID<HrJEB-5ma-7@gated-at.bofh.it>
 After generating a file with lines of text, I have to:
 1) parse some of those lines based on a pattern I know (no regex
necessary, just a FS path)
 2) once parsed from those lines I need the last n characters only
 I am trying to use a one liner like:
 cat "${IFL}" | grep "${DESC_DIR}" | tail -c+$_DESC_DIR
 but this one liner repeats the output and the tail

 Say the lines of text in IFL are:
[info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Uppsala_Algebra/45_16.3_45MEB5h5H9Y.description
[info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Uppsala_Algebra/46_17.1_LE_gl5RcDnY.description
[info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/17_9jKzwoBGFs8.description
[info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/18_plOUIOo91lI.description
[info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/19_TJ7yAiq8gEw.description
[info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/20_UMBEgb14uqw.description

 I know the prefix:
[info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/

 I need as output:

Uppsala_Algebra/45_16.3_45MEB5h5H9Y
Uppsala_Algebra/46_17.1_LE_gl5RcDnY
Göttsche/17_9jKzwoBGFs8
Göttsche/18_plOUIOo91lI
Göttsche/19_TJ7yAiq8gEw
Göttsche/20_UMBEgb14uqw
~
 lbrtchx

[toc] | [next] | [standalone]


#262543

FromCharles Curley <charlescurley@charlescurley.com>
Date2023-10-22 19:10 +0200
Message-ID<HrK7E-5LX-5@gated-at.bofh.it>
In reply to#262540
On Sun, 22 Oct 2023 16:33:12 +0000
Albretch Mueller <lbrtchx@gmail.com> wrote:

> After generating a file with lines of text, I have to:
>  1) parse some of those lines based on a pattern I know (no regex
> necessary, just a FS path)
>  2) once parsed from those lines I need the last n characters only
>  I am trying to use a one liner like:
>  cat "${IFL}" | grep "${DESC_DIR}" | tail -c+$_DESC_DIR
>  but this one liner repeats the output and the tail

Bash has built in operators to extract this sort of thing. Greg
Wooledge will know what they are called. Something like:

fspec="/exp/home1/abc.txt" 
filename="${fspec##*/}"  # get filename
dirname="${fspec%/*}" # get directory/path name

That doesn't do exactly what you want, but comes close.

-- 
Does anybody read signatures any more?

https://charlescurley.com
https://charlescurley.com/blog/

[toc] | [prev] | [next] | [standalone]


#262552

FromGreg Wooledge <greg@wooledge.org>
Date2023-10-22 19:50 +0200
Message-ID<HrKKl-60o-3@gated-at.bofh.it>
In reply to#262543
On Sun, Oct 22, 2023 at 04:33:12PM +0000, Albretch Mueller wrote:
>  I am trying to use a one liner like:
>  cat "${IFL}" | grep "${DESC_DIR}" | tail -c+$_DESC_DIR
>  but this one liner repeats the output and the tail

Brevity is not the measure of goodness.  When you write a shell script,
it's *OK* to use more than one line.

Also, please stop using ALL_CAPS variable names for internal script
variables.  ALL_CAPS names are by convention reserved for the environment.
Names with a leading underscore are allowed, but should be used only
when you need something special.  There's nothing special in the code
above to merit a variable name like _DESC_DIR.

Also also, useless use of cat <https://mywiki.wooledge.org/BashFAQ/119>.

Finally, mixing up _DESC_DIR and DESC_DIR is probably not helping you.

>  Say the lines of text in IFL are:
> [info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Uppsala_Algebra/45_16.3_45MEB5h5H9Y.description
> [info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Uppsala_Algebra/46_17.1_LE_gl5RcDnY.description
> [info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/17_9jKzwoBGFs8.description
> [info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/18_plOUIOo91lI.description
> [info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/19_TJ7yAiq8gEw.description
> [info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/20_UMBEgb14uqw.description
> 
>  I know the prefix:
> [info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/
> 
>  I need as output:
> 
> Uppsala_Algebra/45_16.3_45MEB5h5H9Y
> Uppsala_Algebra/46_17.1_LE_gl5RcDnY
> Göttsche/17_9jKzwoBGFs8
> Göttsche/18_plOUIOo91lI
> Göttsche/19_TJ7yAiq8gEw
> Göttsche/20_UMBEgb14uqw

Yay!  Sample input!  That makes it nice and easy.


#!/bin/bash

inputfile=${1-/whatever}
prefix='[info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/'

while IFS= read -r line; do
    printf '%s\n' "${line#"$prefix"}"
done < <(grep -F -- "$prefix" "$inputfile")


Yes, I could make it shorter.  I could even use that functional approach
where you pipe grep and sed together, which some people think is the only
way to write shell code.  It could actually be done in a single sed
command (but you'd have to convert the prefix string to a regular
expression first).

I chose not to do any of those, for two reasons:

1) To make it easier to read and understand.
2) To make it possible to do more than simply *printing* the suffixes.

If your script wants to do something with those suffixes, more than
simply printing them on the screen, then the script I wrote is a much
better starting point.  You can stick additional commands inside the
loop body to do whatever you need.


On Sun, Oct 22, 2023 at 11:07:46AM -0600, Charles Curley wrote:
> Bash has built in operators to extract this sort of thing. Greg
> Wooledge will know what they are called. Something like:
> 
> fspec="/exp/home1/abc.txt" 
> filename="${fspec##*/}"  # get filename
> dirname="${fspec%/*}" # get directory/path name
> 
> That doesn't do exactly what you want, but comes close.

Those are called Parameter Expansions.  For a beginner-friendly intro,
please see <https://mywiki.wooledge.org/BashFAQ/100>.

[toc] | [prev] | [next] | [standalone]


#262564

FromAlbretch Mueller <lbrtchx@gmail.com>
Date2023-10-22 23:00 +0200
Message-ID<HrNIe-7Oc-9@gated-at.bofh.it>
In reply to#262552
 OK, Greg's suggestion once again "made my day".
 I know at some point I will have to code everything in some
programming language, but for now I will just get things done as
quickly as possible.
 Also, Greg, please, I would like for you to understand that it is not
my intention to upset you about what you find visually upsetting. We
have talked about that before.
 I am sure you could certainly understand that there are reasons why
some people build certain habits and that not everyone gets upset
about the same kinds of "secondary accidents" (as Aristotle would put
it). Some people may have a hard time even noticing what upsets you.
 lbrtchx

[toc] | [prev] | [next] | [standalone]


#262565

FromAndy Smith <andy@strugglers.net>
Date2023-10-22 23:20 +0200
Message-ID<HrO1z-8aO-1@gated-at.bofh.it>
In reply to#262564
Hello,

On Sun, Oct 22, 2023 at 08:50:55PM +0000, Albretch Mueller wrote:
> Greg, please, I would like for you to understand that it is not my
> intention to upset you about what you find visually upsetting. We
> have talked about that before.

Most of the points Greg makes to you are matters of correctness, not
matters of taste.

I don't know how many times you will have the opportunity to
continue talking with knowledgable people who try to help you, if
you dismiss their points as them just being "upset" over aesthetics.

>  I am sure you could certainly understand that there are reasons why
> some people build certain habits and that not everyone gets upset
> about the same kinds of "secondary accidents" (as Aristotle would put
> it). Some people may have a hard time even noticing what upsets you.

Most of the time when asking for help from people with a lot more
knowledge than me in a given subject, I tend to assume that if I
find something they say to be oddly picky or unnecessarily strict
then it's most likely because of a gap in my own knowledge, rather
than them trying to make me slaves to their own personal style.

Thanks,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

[toc] | [prev] | [next] | [standalone]


#262568

Fromgene heskett <gheskett@shentel.net>
Date2023-10-23 00:30 +0200
Message-ID<HrP7j-8Oc-1@gated-at.bofh.it>
In reply to#262565
On 10/22/23 17:11, Andy Smith wrote:
> Hello,
> 
> On Sun, Oct 22, 2023 at 08:50:55PM +0000, Albretch Mueller wrote:
>> Greg, please, I would like for you to understand that it is not my
>> intention to upset you about what you find visually upsetting. We
>> have talked about that before.
> 
> Most of the points Greg makes to you are matters of correctness, not
> matters of taste.
> 
> I don't know how many times you will have the opportunity to
> continue talking with knowledgable people who try to help you, if
> you dismiss their points as them just being "upset" over aesthetics.
> 
>>   I am sure you could certainly understand that there are reasons why
>> some people build certain habits and that not everyone gets upset
>> about the same kinds of "secondary accidents" (as Aristotle would put
>> it). Some people may have a hard time even noticing what upsets you.
> 
> Most of the time when asking for help from people with a lot more
> knowledge than me in a given subject, I tend to assume that if I
> find something they say to be oddly picky or unnecessarily strict
> then it's most likely because of a gap in my own knowledge, rather
> than them trying to make me slaves to their own personal style.
> 
> Thanks,
> Andy
> 
Very well said Andy.


Cheers, Gene Heskett.
-- 
"There are four boxes to be used in defense of liberty:
  soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author, 1940)
If we desire respect for the law, we must first make the law respectable.
  - Louis D. Brandeis

[toc] | [prev] | [next] | [standalone]


#262575

FromAlbretch Mueller <lbrtchx@gmail.com>
Date2023-10-23 03:40 +0200
Message-ID<HrS5b-azX-1@gated-at.bofh.it>
In reply to#262568
On 10/22/23, Andy Smith <andy@strugglers.net> wrote:
> So you might consider telling us what you will do next with the
> suffix of each line.

 Now you have gone into mind reading mode.

On 10/22/23, Andy Smith <andy@strugglers.net> wrote:
> Most of the points Greg makes to you are matters of correctness, not
> matters of taste.

 that you called "matters of correctness" may be "visual things" to
other people. this is something I see on a daily basis I don't try to
"correct" my students ways, but help them learn, see things for
themselves.

> I don't know how many times you will have the opportunity to
> continue talking with knowledgable people who try to help you, if
> you dismiss their points as them just being "upset" over aesthetics.

 and I don't understand your adhominen cr@p. For one, Greg understood
right away my problem which I explicitly explained, so from where do
you take the time to keep talking about any of it?

On 10/22/23, gene heskett <gheskett@shentel.net> wrote:
> Very well said Andy.
> "There are four boxes to be used in defense of liberty:
>   soap, ballot, jury, and ammo. Please use in that order."
> -Ed Howdershelt (Author, 1940)
> If we desire respect for the law, we must first make the law respectable.
>   - Louis D. Brandeis

 Nor do I understand what support from "the international community"
(tm) has to do with some Linux bash script.

 lbrtchx

[toc] | [prev] | [next] | [standalone]


#262576

FromGreg Wooledge <greg@wooledge.org>
Date2023-10-23 04:20 +0200
Message-ID<HrSHT-b8Q-1@gated-at.bofh.it>
In reply to#262575
On Mon, Oct 23, 2023 at 01:32:08AM +0000, Albretch Mueller wrote:
> On 10/22/23, Andy Smith <andy@strugglers.net> wrote:
> > Most of the points Greg makes to you are matters of correctness, not
> > matters of taste.
> 
>  that you called "matters of correctness" may be "visual things" to
> other people. this is something I see on a daily basis I don't try to
> "correct" my students ways, but help them learn, see things for
> themselves.

Was I not sufficiently clear in my explanations?  Sorry.

Your original post contained this line of code:

    cat "${IFL}" | grep "${DESC_DIR}" | tail -c+$_DESC_DIR

In the interests of learning, let's analyze this line *in full*.  From
left to right:

1) Useless use of cat.  In this particular example, the result is still
   correct.  The code is simply longer and slower than it needs to be.

2) All-caps variable name IFL.  All-caps variable names are reserved, by
   convention, for environment variables (e.g. PATH) and special shell
   variables (e.g. IFS).  I assume your variable IFL stands for "input
   file".  But note how close it is to IFS.  I can easily imagine that
   at some point in the future, if you continue using all-caps variable
   names inappropriately, you might accidentally use one that collides
   with an environment variable or a special shell variable.

   Accidentally overwriting IFS is going to break your scripts faster
   than you can imagine.

3) grep "${DESC_DIR}".  Another all-caps variable name, but I won't repeat
   that point.  $DESC_DIR is being passed to a command that expects a
   regular expression, but in your example, it's a literal string.
   Therefore you should use grep -F.  Or use a regular expression in
   DESC_DIR, but that wouldn't be my preferred choice.

4) The tail command is not correct here.  This is the primary error.
   The tail needs to be replaced to make the script work.  Tail operates
   on whole files, not on each line of input, regardless of what options
   you use.

5) You used the _DESC_DIR variable and the DESC_DIR variable together in
   the same command.  I can only imagine one of them was a typo for the
   other.

6) The _DESC_DIR variable expansion is not quoted.  Assuming that it was
   meant to be DESC_DIR which contains a string with spaces and glob
   characters in it, it *must* be quoted, or else it gets passed as an
   unguessable number of separate arguments.

Those are the actual issues I spotted.  I omitted a few in my previous
post, because I wasn't trying to be fully pedantic.  I am now.

Beyond that, I pointed out (by inference) that the code you wrote was
too *limited*.  If the errors are corrected, it will print the output
that you asked for.  But it can't be extended easily to do anything *more*
than that.

Assuming that you actually wanted to *do* something with each line's
suffix, it makes more sense to include some sort of loop, in which you
can add arbitrary commands to do whatever task is needed.

That's why I replaced the tail with a "while read" loop instead of a
single self-contained command, such as sed, which could have done the
printing task more concisely.

Finally, one might wonder why I used this form:

    while IFS= read -r line; do
        ....
    done < <(grep ...)

instead of the simpler form:

    grep ... |
    while IFS= read -r line; do
        ...
    done

This was another conscious choice to offer more flexibility for extending
the script.  In the simpler form, the while loop is executed in a subshell
(because of the pipeline).  When the loop terminates, any changes made in
the subshell are discarded.  This makes it impossible to set persistent
variables inside the loop, and then use them afterward.

The form that I used, with the process substition <(...), runs the loop
in the main shell process, not in a subshell.  With this form, it's
possible to set variables within the loop, and refer to them afterward.

If you have additional questions, feel free to ask them.

[toc] | [prev] | [next] | [standalone]


#262585

From<tomas@tuxteam.de>
Date2023-10-23 06:50 +0200
Message-ID<HrV33-cyT-1@gated-at.bofh.it>
In reply to#262576

[Multipart message — attachments visible in raw view] — view raw

On Sun, Oct 22, 2023 at 10:13:26PM -0400, Greg Wooledge wrote:

[...]

Good points all over -- I envy your clarity.

To drive home Greg's point about naming conventions a bit
more: bash (in general, shells) have very little protection
for mis-naming variables (you don' have to declare them,
they have dynamic, not lexical bindings, etc.).

That's why naming conventions are more important there.

In shells, there's more: shell variables and environment
variables (those you "export") are quite different beasts,
so making them visually differentiable is the more important.

See, a programming language is a communication tool. You
talk to other people with it. Observing conventions is
part of a language.

Cheers
-- 
t

[toc] | [prev] | [next] | [standalone]


#262599 — Shell-script variable name case [WAS: Re: can you parse and "tail" at once? (and if you can't why not?)]

FromDavid <bouncingcats@gmail.com>
Date2023-10-23 16:00 +0200
SubjectShell-script variable name case [WAS: Re: can you parse and "tail" at once? (and if you can't why not?)]
Message-ID<Hs3Dj-hUG-3@gated-at.bofh.it>
In reply to#262576
On Mon, 23 Oct 2023 at 13:25, Eduardo M KALINOWSKI
<eduardo@kalinowski.com.br> wrote:
> On 22/10/2023 23:13, Greg Wooledge wrote:

> > 2) All-caps variable name IFL.  All-caps variable names are reserved,
> >    by convention, for environment variables (e.g. PATH) and special
> >    shell variables (e.g. IFS).

> While I don't disagree with the suggestion of using lower case for
> variables (and have even started doing so myself), it seems that this
> "convention" is far from universal.

> I did a quick search on the bash manual page and found no suggestion on
> how to name user variables, or that uppercase names are reserved (but it
> was a very quick search - I might have skipped something).

> Even an internet search shows that people seem to be divided:

Hi, for your info, this convention is specified by POSIX:
  https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html

Which says:
  Environment variable names used by the utilities in the Shell and
  Utilities volume of POSIX.1-2017 consist solely of uppercase letters,
  digits, and the <underscore> ( '_' ) from the characters defined in
  Portable Character Set and do not begin with a digit.

And:
  The name space of environment variable names containing lowercase letters
  is reserved for applications. Applications can define any environment
  variables with names from this name space without modifying the behavior
  of the standard utilities.

[toc] | [prev] | [next] | [standalone]


#262627

FromAndy Smith <andy@strugglers.net>
Date2023-10-23 22:10 +0200
Message-ID<Hs9pn-mfk-7@gated-at.bofh.it>
In reply to#262575
Hello,

On Mon, Oct 23, 2023 at 01:32:08AM +0000, Albretch Mueller wrote:
> On 10/22/23, Andy Smith <andy@strugglers.net> wrote:
> > So you might consider telling us what you will do next with the
> > suffix of each line.
> 
>  Now you have gone into mind reading mode.

I've gone into "avoid the need for mind reading" mode. Because I
could see that from your very limited provision of context, you were
going to get a lot of different solutions from different people,
and that you were likely to disregard most of those due to context
you've not supplied. So I've suggested you supply more contact so
that people DON'T have to read your mind.

> On 10/22/23, Andy Smith <andy@strugglers.net> wrote:
> > Most of the points Greg makes to you are matters of correctness, not
> > matters of taste.
> 
>  that you called "matters of correctness" may be "visual things" to
> other people.

I think it's important to you understand that often only the
experienced person can tell the difference.

> from where do you take the time to keep talking about any of it?

Okay, message received and understood. I won't be troubling you
again.

Thanks,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

[toc] | [prev] | [next] | [standalone]


#262580

FromPatrick Wiseman <pwiseman@gmail.com>
Date2023-10-23 04:50 +0200
Message-ID<HrTaV-bjU-5@gated-at.bofh.it>
In reply to#262564

[Multipart message — attachments visible in raw view] — view raw

On Sun, Oct 22, 2023, 10:39 PM Albretch Mueller <lbrtchx@gmail.com> wrote:

>  OK, Greg's suggestion once again "made my day".
>  I know at some point I will have to code everything in some
> programming language, but for now I will just get things done as
> quickly as possible.
>  Also, Greg, please, I would like for you to understand that it is not
> my intention to upset you about what you find visually upsetting. We
> have talked about that before.
>  I am sure you could certainly understand that there are reasons why
> some people build certain habits and that not everyone gets upset
> about the same kinds of "secondary accidents" (as Aristotle would put
> it). Some people may have a hard time even noticing what upsets you.
>  lbrtchx
>
> Context? Totally lost. No need to reply, I haven't been following anyway.

[toc] | [prev] | [next] | [standalone]


#262553

FromRoberto C. Sánchez <roberto@debian.org>
Date2023-10-22 19:50 +0200
Message-ID<HrKKl-60o-5@gated-at.bofh.it>
In reply to#262540
On Sun, Oct 22, 2023 at 04:33:12PM +0000, Albretch Mueller wrote:
>  After generating a file with lines of text, I have to:
>  1) parse some of those lines based on a pattern I know (no regex
> necessary, just a FS path)
>  2) once parsed from those lines I need the last n characters only
>  I am trying to use a one liner like:
>  cat "${IFL}" | grep "${DESC_DIR}" | tail -c+$_DESC_DIR
>  but this one liner repeats the output and the tail
> 
The way that -c operates for tail is based on 'each file' provided as
input:

       -c, --bytes=[+]NUM
              output the last NUM bytes; or use -c +NUM to output starting with byte NUM of
              each file

However, when piping input in the way you are above, the entire output
of grep is taken as one file and fed into the input of tail. You could
instead loop over the lines coming out of grep, but that feels somewhat
ugly.

Essentially, your pipeline probably needs to be something more like
this:

sed -r -e "s+.*${DESC_DIR}++" "${IFL}" | tail

Or drop the tail entirely if what you really want is the last part of
each line, regardless of how many there are.

Regards,

-Roberto

-- 
Roberto C. Sánchez

[toc] | [prev] | [next] | [standalone]


#262556

FromAndy Smith <andy@strugglers.net>
Date2023-10-22 20:00 +0200
Message-ID<HrKU1-63W-5@gated-at.bofh.it>
In reply to#262540
Hello,

On Sun, Oct 22, 2023 at 04:33:12PM +0000, Albretch Mueller wrote:
>  After generating a file with lines of text, I have to:
>  1) parse some of those lines based on a pattern I know (no regex
> necessary, just a FS path)
>  2) once parsed from those lines I need the last n characters only

There's a large number of ways to achieve what you want, but it's
hard to believe that simply getting the suffix is all that you want.
I think it's likely that you'll reject most of the otherwise correct
answers because they don't suit what you're about to do next.

So you might consider telling us what you will do next with the
suffix of each line.

Thanks,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

[toc] | [prev] | [next] | [standalone]


#262588

FromPankaj Jangid <pankaj@codeisgreat.org>
Date2023-10-23 10:00 +0200
Message-ID<HrY0V-em0-3@gated-at.bofh.it>
In reply to#262540
Albretch Mueller <lbrtchx@gmail.com> writes:

>  After generating a file with lines of text, I have to:
>  1) parse some of those lines based on a pattern I know (no regex
> necessary, just a FS path)
>  2) once parsed from those lines I need the last n characters only
>  I am trying to use a one liner like:
>  cat "${IFL}" | grep "${DESC_DIR}" | tail -c+$_DESC_DIR
>  but this one liner repeats the output and the tail
>
>  Say the lines of text in IFL are:
> [info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Uppsala_Algebra/45_16.3_45MEB5h5H9Y.description
> [info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Uppsala_Algebra/46_17.1_LE_gl5RcDnY.description
> [info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/17_9jKzwoBGFs8.description
> [info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/18_plOUIOo91lI.description
> [info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/19_TJ7yAiq8gEw.description
> [info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/20_UMBEgb14uqw.description
>
>  I know the prefix:
> [info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/
>
>  I need as output:
>
> Uppsala_Algebra/45_16.3_45MEB5h5H9Y
> Uppsala_Algebra/46_17.1_LE_gl5RcDnY
> Göttsche/17_9jKzwoBGFs8
> Göttsche/18_plOUIOo91lI
> Göttsche/19_TJ7yAiq8gEw
> Göttsche/20_UMBEgb14uqw
> ~
>  lbrtchx

Another way to do it is,

cat "${IFL}"| cut -d "/" -f7-|sed 's/.description//'

because you already know the prefix, you can count the fields. So "-f7-"
i.e. 7 onwards. Then use sed to remove the extension.

[toc] | [prev] | [next] | [standalone]


#262597

FromGreg Wooledge <greg@wooledge.org>
Date2023-10-23 14:20 +0200
Message-ID<Hs24x-h75-1@gated-at.bofh.it>
In reply to#262588
On Mon, Oct 23, 2023 at 01:21:43PM +0530, Pankaj Jangid wrote:
> Another way to do it is,
> 
> cat "${IFL}"| cut -d "/" -f7-|sed 's/.description//'
> 
> because you already know the prefix, you can count the fields. So "-f7-"
> i.e. 7 onwards. Then use sed to remove the extension.

Whoops!  I completely missed the requirement of removing ".description"
in my original answer.

Here's a corrected version:


#!/bin/bash

inputfile=${1-/whatever}
prefix='[info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/'

while IFS= read -r line; do
    line=${line#"$prefix"}
    printf '%s\n' "${line%.description}"
done < <(grep -F -- "$prefix" "$inputfile")


Pankaj's answer is not wrong, but it dropped the "grep" which filters
the lines containing the desired prefix.  I'd also add an $ anchor to
the sed regex, and escape the . with backslash so that it becomes
literal.  This answer also requires counting the fields by hand each
time the prefix changes, which may or may not be a concern.

[toc] | [prev] | [next] | [standalone]


#262635

FromAlbretch Mueller <lbrtchx@gmail.com>
Date2023-10-24 00:20 +0200
Message-ID<Hsbrb-ntG-1@gated-at.bofh.it>
In reply to#262597
 Thank you very much, Greg!
 Since ".description" is constant (an extension used by youtube) I chose to go:
 ydx=".description"
 ydxL=${#ydx}
 ...
 yut=${yl:-${ydxL}}
...
 where yl is the line read in in the way you suggested.
 lbrtchx

[toc] | [prev] | [standalone]


Back to top | Article view | linux.debian.user


csiph-web