Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #188979 > unrolled thread

Editing Problem with bash script and sed

Back to article view | Back to linux.debian.user

Contents

  Editing Problem with bash script and sed Thomas George <lists@tomgeorge.info> - 2017-11-14 17:20 +0100
    Re: Editing Problem with bash script and sed Roberto C. Sánchez <roberto@debian.org> - 2017-11-14 17:30 +0100
    Re: Editing Problem with bash script and sed "Thomas Schmitt" <scdbackup@gmx.net> - 2017-11-14 19:30 +0100
      Re: Editing Problem with bash script and sed "Thomas Schmitt" <scdbackup@gmx.net> - 2017-11-14 19:40 +0100
        Re: Editing Problem with bash script and sed Greg Wooledge <wooledg@eeg.ccf.org> - 2017-11-14 19:50 +0100
      Re: Editing Problem with bash script and sed Greg Wooledge <wooledg@eeg.ccf.org> - 2017-11-14 19:50 +0100
    Re: Editing Problem with bash script and sed David Wright <deblis@lionunicorn.co.uk> - 2017-11-14 20:50 +0100
    Re: Editing Problem with bash script and sed RESOLVED Thomas George <lists@tomgeorge.info> - 2017-11-14 21:20 +0100
    Re: Editing Problem with bash script and sed Rusi Mody <rustompmody@gmail.com> - 2017-11-19 06:10 +0100

#188979 — Editing Problem with bash script and sed

[Multipart message — attachments visible in raw view] — view raw

The problem is editing long file names to shorten them. An example group 
of file names is attached.

The bash script copied from BashScripting is attached. This script works 
perfectly with simple deletions, for example TrimLine.sh "College" "" 
will remove College from each line in File.txt.

After experimenting with regular expressions with sed I found ls | sed 
-e s/S.*-// reduced the file names in File.txt to just the names of the 
Carols as shown in sed.txt. Used like this sed leaves the original file 
unchanged.

Trim_Line.sh "S.*-" "" fails to actually change the file names. Assuming 
Trim_Line.sh does not accept the regex as a pattern, I edited it to 
replace the $1 in the sed command with the regex expression. This does 
not work, the original file is left unchanged.  I also tried  replacing 
$1 with ^.*- which also works with the test ls | sed -e s/^.*-// but 
still the file names are unchanged.

I would appreciate any help or comments.

Tom George

[toc] | [next] | [standalone]

#188980

On Tue, Nov 14, 2017 at 11:07:47AM -0500, Thomas George wrote:
> 
> After experimenting with regular expressions with sed I found ls | sed -e
> s/S.*-// reduced the file names in File.txt to just the names of the Carols
> as shown in sed.txt. Used like this sed leaves the original file unchanged.
> 
You probably want the file-rename command from the rename package.  It
is a handy wrapper around the Perl File::Rename module.  It accepts Perl
RE, which is very similar to what you would use with sed.  In fact, that
particular RE that you are using should just work unmodified with
file-rename.

Regards,

-Roberto

-- 
Roberto C. Sánchez

[toc] | [prev] | [next] | [standalone]

#188982

Hi,

i see at least one problem in Trim_Line.sh :
It uses "$1" as shell parser input and as sed regular expression.

With "S.*-" as "$1", the meaning differs in both contexts.
The shell parser input of

  for filename in *$1*

will look for files with a text snippet "S.".
The expression in 

  s/$1//

will not insist in seeing the "." after "S". It rather understands the "."
as instruction to match any character.

Looking at File.txt, i'd say "St.*-" could match the "St. John's College
Choir, Cambridge -" files in both interpretations.


In general, sed goes better with grep than with the shell parser.
Try

  ls -d * | grep "$1" | while read filename
  do
     ...
                  n=$(echo "$fname" | sed -e s/"$1"//)

Regrettably i found no way to make this safe against newlines in filenames.
I'd do it in C language if such names were to be expected.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#188983

Hi,

after reading man 1 "read", i have to add option "-r" to my proposal:

  ls -d * | grep "$1" | while read -r filename


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#188986

On Tue, Nov 14, 2017 at 07:37:34PM +0100, Thomas Schmitt wrote:
> Hi,
> 
> after reading man 1 "read", i have to add option "-r" to my proposal:
> 
>   ls -d * | grep "$1" | while read -r filename

That fixes one problem, but there are plenty more still unfixed.

If the user input ("$1") is to be treated as an ERE, then you could
do something like this:

for filename in *; do
    [[ $filename =~ $1 ]] || continue
    ...
done

But your grep is treating "$1" as a BRE, not an ERE, and bash doesn't
have any implementation of BREs.

In any case, I do NOT recommend writing your own script to replace and
augment rename(1).  Just run a bunch of rename commands from the shell
prompt to make the filenames look how you want them.

[toc] | [prev] | [next] | [standalone]

#188985

On Tue, Nov 14, 2017 at 07:21:55PM +0100, Thomas Schmitt wrote:
>   ls -d * | grep "$1" | while read filename

Eww.  No.  That code is broken in multiple ways.

>                   n=$(echo "$fname" | sed -e s/"$1"//)
> 
> Regrettably i found no way to make this safe against newlines in filenames.
> I'd do it in C language if such names were to be expected.

Well, the whole thing is a shambles.  Just use rename(1) which is
designed for this.

  rename 's/Badstring//' *Badstring*

The problems with using sed for this are numerous.  There's no way to
tell sed that the string you're passing should be treated as an argument
to a command.  All you can do is inject the user input directly into a
sed command.  This means that if the user input contains any characters
that sed uses as syntax, you lose.

If you had to do this on a legacy Unix system without perl, then see
<http://mywiki.wooledge.org/BashFAQ/030>

[toc] | [prev] | [next] | [standalone]

#188988

On Tue 14 Nov 2017 at 11:07:47 (-0500), Thomas George wrote:

> The problem is editing long file names to shorten them.
> An example group of file names is attached.

I thought someone might mention MC for doing this.

The sequence of operations I would use here is:

Select the files you want to change. Usually it's sufficient to
select all the files in the directory because any stragglers that
don't match won't get changed.

Leave the cursor on any one of the files.

Press F6 (rename).

In the "to:" field, type *

With the mouse, drag across the text you want to eliminate.

In the "Move <N> files with source mask:" field, paste the
selected text and add * at the end.

It will now look something like:

Move <N> files with source mask:
St. John's College Choir, Cambridge - [19*

to:
*

Press Return and the matching files will have that string
removed.

Repeat, pasting:
88  421 022-2 10DC] Christmas Weekend (16 Fav*

and:
ourite Carols) - *

With such a long initial string, you will probably not be able
to see all of it at the bottom of the screen so I've demonstrated
how to do it in parts. Of course, you can Copy the entire initial
string from a normal ls command's output.

I leave the digits at the start of the carols' names so that
they play in the expected order when globbed.

MC can of course do more flexible selection of and changes to
filenames than just this simple case.

One minor point: I find it safer and more reliable to use MC
with --nomouse so that the mouse can be used for selecting and
pasting text (without shift), but not for pressing buttons etc.

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#188989 — Re: Editing Problem with bash script and sed RESOLVED

Thanks to all for very helpful responses. I have been trying to reinvent 
the wheel, rename does the job nicely and the perl regex's are more 
reliable. ^.*?-. allows removing the to the first - so repeated 
applications remove to the track numbers. The comments regarding the 
Trim_Line script were an education explaining why it did not work as 
expected.


On 11/14/2017 11:07 AM, Thomas George wrote:
> The problem is editing long file names to shorten them. An example 
> group of file names is attached.
>
> The bash script copied from BashScripting is attached. This script 
> works perfectly with simple deletions, for example TrimLine.sh 
> "College" "" will remove College from each line in File.txt.
>
> After experimenting with regular expressions with sed I found ls | sed 
> -e s/S.*-// reduced the file names in File.txt to just the names of 
> the Carols as shown in sed.txt. Used like this sed leaves the original 
> file unchanged.
>
> Trim_Line.sh "S.*-" "" fails to actually change the file names. 
> Assuming Trim_Line.sh does not accept the regex as a pattern, I edited 
> it to replace the $1 in the sed command with the regex expression. 
> This does not work, the original file is left unchanged.  I also 
> tried  replacing $1 with ^.*- which also works with the test ls | sed 
> -e s/^.*-// but still the file names are unchanged.
>
> I would appreciate any help or comments.
>
> Tom George
>

[toc] | [prev] | [next] | [standalone]

#189115

On Tuesday, November 14, 2017 at 9:50:06 PM UTC+5:30, Thomas George wrote:
> The problem is editing long file names to shorten them. An example group 
> of file names is attached.
> 
> The bash script copied from BashScripting is attached. This script works 
> perfectly with simple deletions, for example TrimLine.sh "College" "" 
> will remove College from each line in File.txt.
> 
> After experimenting with regular expressions with sed I found ls | sed 
> -e s/S.*-// reduced the file names in File.txt to just the names of the 
> Carols as shown in sed.txt. Used like this sed leaves the original file 
> unchanged.
> 
> Trim_Line.sh "S.*-" "" fails to actually change the file names. Assuming 
> Trim_Line.sh does not accept the regex as a pattern, I edited it to 
> replace the $1 in the sed command with the regex expression. This does 
> not work, the original file is left unchanged.  I also tried  replacing 
> $1 with ^.*- which also works with the test ls | sed -e s/^.*-// but 
> still the file names are unchanged.
> 
> I would appreciate any help or comments.

For such work I find emacs dired useful  [dired is emacs' file editor]

For more convenience add:
1. Menu → Immediate → Edit File names (shortcut Ctrl-x Ctrl-q)
combined with
2. column-editing mode of cua https://vimeo.com/1168225

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web