Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #195145 > unrolled thread

List words separated by comma and without duplicates

Back to article view | Back to linux.debian.user

Contents

  List words separated by comma and without duplicates Rodolfo Medina <rodolfo.medina@gmail.com> - 2018-04-29 19:50 +0200
    Re: List words separated by comma and without duplicates rhkramer@gmail.com - 2018-04-29 20:00 +0200
    Re: List words separated by comma and without duplicates "Antonio A. Rendina" <arendina13@gmail.com> - 2018-04-29 20:20 +0200
      Re: List words separated by comma and without duplicates David Margerison <deltam@electrosonics.com.au> - 2018-04-30 01:40 +0200
        Re: List words separated by comma and without duplicates Bob McGowan <ramjr0915@gmail.com> - 2018-05-01 05:50 +0200
    Re: List words separated by comma and without duplicates "Antonio A. Rendina" <arendina13@gmail.com> - 2018-04-29 20:30 +0200
      Re: List words separated by comma and without duplicates Abdullah Ramazanoglu <ar018@yahoo.com> - 2018-04-29 20:40 +0200
    Re: List words separated by comma and without duplicates <tomas@tuxteam.de> - 2018-04-29 20:30 +0200
      Re: List words separated by comma and without duplicates Curt <curty@free.fr> - 2018-04-29 22:20 +0200
        Re: List words separated by comma and without duplicates <tomas@tuxteam.de> - 2018-04-29 22:40 +0200
          Re: List words separated by comma and without duplicates Gene Heskett <gheskett@shentel.net> - 2018-04-30 00:30 +0200
            [OT - long running shows] Re: List words separated by comma and  without duplicates Richard Hector <richard@walnut.gen.nz> - 2018-04-30 02:30 +0200
              Re: [OT - long running shows] Re: List words separated by comma and without duplicates Gene Heskett <gheskett@shentel.net> - 2018-04-30 03:00 +0200
                Re: [OT - long running shows] Re: List words separated by comma and  without duplicates Richard Hector <richard@walnut.gen.nz> - 2018-04-30 03:30 +0200
                  Re: [OT - long running shows] Re: List words separated by comma and without duplicates Gene Heskett <gheskett@shentel.net> - 2018-04-30 05:00 +0200
    Re: List words separated by comma and without duplicates Rodolfo Medina <rodolfo.medina@gmail.com> - 2018-05-18 21:00 +0200
      Re: List words separated by comma and without duplicates <tomas@tuxteam.de> - 2018-05-18 21:20 +0200

#195145 — List words separated by comma and without duplicates

Hi all.

Suppose I have a file made by a list of names and surnames, e.g.:

Arvo Part
Harold Pinter
Lucio Battisti
Antonio Amurri
Eduardo De Filippo
Eduardo De Filippo

and that I want them listed all on one line, separated by a comma and without
duplicates, i.e.:

Arvo Part, Harold Pinter, Lucio Battisti, Antonio Amurri, Eduardo De Filippo

How can I do that with proper Unix commands?

Thanks for any help.

Rodolfo

[toc] | [next] | [standalone]

#195146

Is this a homework assignment?

How many names are in the list?

On Sunday, April 29, 2018 01:43:34 PM Rodolfo Medina wrote:
> Suppose I have a file made by a list of names and surnames, e.g.:
> 
> Arvo Part
> Harold Pinter
> Lucio Battisti
> Antonio Amurri
> Eduardo De Filippo
> Eduardo De Filippo
> 
> and that I want them listed all on one line, separated by a comma and
> without duplicates, i.e.:
> 
> Arvo Part, Harold Pinter, Lucio Battisti, Antonio Amurri, Eduardo De
> Filippo
> 
> How can I do that with proper Unix commands?
> 
> Thanks for any help.
> 
> Rodolfo

[toc] | [prev] | [next] | [standalone]

#195149

On Sun, 2018-04-29 at 19:43 +0200, Rodolfo Medina wrote:
> Hi all.
> 
> Suppose I have a file made by a list of names and surnames, e.g.:
> 
> Arvo Part
> Harold Pinter
> Lucio Battisti
> Antonio Amurri
> Eduardo De Filippo
> Eduardo De Filippo
> 
> and that I want them listed all on one line, separated by a comma and
> without
> duplicates, i.e.:
> 
> Arvo Part, Harold Pinter, Lucio Battisti, Antonio Amurri, Eduardo De
> Filippo
> 
> How can I do that with proper Unix commands?
> 
> Thanks for any help.
> 
> Rodolfo
> 

I would do something like that:
cat list-file | sed s/$/,/ | tr '\n' ' '

If you want to improve your bash skills you can read:
http://www.tldp.org/LDP/Bash-Beginners-Guide/html/

It is really useful.

-- 
Antonio

[toc] | [prev] | [next] | [standalone]

#195161

On 30 April 2018 at 04:12, Antonio A. Rendina <arendina13@gmail.com> wrote:
>
> If you want to improve your bash skills you can read:
> http://www.tldp.org/LDP/Bash-Beginners-Guide/html/

I suggest that to avoid poor and ancient, it would be
better to read current documents written by active experts.

http://mywiki.wooledge.org/
http://wiki.bash-hackers.org/

[toc] | [prev] | [next] | [standalone]

#195234

On 04/29/2018 03:45 PM, David Margerison wrote:
> On 30 April 2018 at 04:12, Antonio A. Rendina <arendina13@gmail.com> wrote:
>> If you want to improve your bash skills you can read:
>> http://www.tldp.org/LDP/Bash-Beginners-Guide/html/
> I suggest that to avoid poor and ancient, it would be
> better to read current documents written by active experts.
>
> http://mywiki.wooledge.org/
> http://wiki.bash-hackers.org/
>
And, along the same lines, why do all the solutions I saw us I/O 
redirection in the shell to get input to commands that, at least in 
modern versions, can all read a file directly?

Bob

[toc] | [prev] | [next] | [standalone]

#195150

On Sun, 2018-04-29 at 19:43 +0200, Rodolfo Medina wrote:
> Hi all.
> 
> Suppose I have a file made by a list of names and surnames, e.g.:
> 
> Arvo Part
> Harold Pinter
> Lucio Battisti
> Antonio Amurri
> Eduardo De Filippo
> Eduardo De Filippo
> 
> and that I want them listed all on one line, separated by a comma and without
> duplicates, i.e.:
> 
> Arvo Part, Harold Pinter, Lucio Battisti, Antonio Amurri, Eduardo De Filippo
> 
> How can I do that with proper Unix commands?
> 
> Thanks for any help.
> 
> Rodolfo
> 

Sorry, forgot uniq...

cat list-file | uniq | sed s/$/,/ | tr '\n' ' '

[toc] | [prev] | [next] | [standalone]

#195152

On Sun, 29 Apr 2018 19:24:12 +0100 Antonio A. Rendina said:

> cat list-file | uniq | sed s/$/,/ | tr '\n' ' '

But uniq eliminates consecutive (neighboring) duplicates only. If, for example,
"Eduardo De Filippo" also occurs at 2nd line, then there would still be 2
separate Eduardo's left. So, sort + uniq is necessary as in,

sort list-file | uniq | sed s/$/,/ | tr '\n' ' '

or you can combine them into one (as tomas already suggested)

sort -u list-file | sed s/$/,/ | tr '\n' ' '

Regards
-- 
Abdullah Ramazanoglu

[toc] | [prev] | [next] | [standalone]

#195151

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sun, Apr 29, 2018 at 07:43:34PM +0200, Rodolfo Medina wrote:
> Hi all.
> 
> Suppose I have a file made by a list of names and surnames, e.g.:
> 
> Arvo Part
> Harold Pinter
> Lucio Battisti
> Antonio Amurri
> Eduardo De Filippo
> Eduardo De Filippo
> 
> and that I want them listed all on one line, separated by a comma and without
> duplicates, i.e.:
> 
> Arvo Part, Harold Pinter, Lucio Battisti, Antonio Amurri, Eduardo De Filippo
> 
> How can I do that with proper Unix commands?

The "classical":

  tomas@trotzki:/tmp$ cat foo
  Arvo Part
  Harold Pinter
  Lucio Battisti
  Antonio Amurri
  Eduardo De Filippo
  Eduardo De Filippo

tomas@trotzki:/tmp$ sort -u < foo | sed -e 's/$/,/g' | tr '\n' ' '
Antonio Amurri, Arvo Part, Eduardo De Filippo, Harold Pinter, Lucio Battisti, 

Note that (a) the list ends with ", " (comma-space) and has no newline at
the end, but I preferred to present a simple proto-solution easier for you
to munge.

The tr is there because sed goes line-wise and thus can't replace newlines.
Except that it can; you pay for it, though [1], [2].

Finally, all this assumes that you're OK with the result being sorted
lexicographically: if you want to keep the original order, you'll probably
have to resort to awk or perl (unless you want to earn some obfuscated
Unix ninja price or something ;-)

For a Perl example which doesn't sort (sorry for the long line):

  tomas@trotzki:/tmp$ perl -e 'while(<STDIN>) {chop; push(@result, $_) unless $seen{$_}++;} print(join(", ", @result), "\n");' < foo
Arvo Part, Harold Pinter, Lucio Battisti, Antonio Amurri, Eduardo De Filippo

Note that the sort/sed/tr example above might cope well with a file
wich doesn't fit in your RAM, while the perl example... less so.
Times have changed :-)

(to be fair, the quoted examples below seem also to rely on sucking
all the whole file into a (potentially giant) line).

Cheers
[1] https://stackoverflow.com/questions/1251999/how-can-i-replace-a-newline-n-using-sed
[2] https://unix.stackexchange.com/questions/114943/can-sed-replace-new-line-characters

- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlrmDLwACgkQBcgs9XrR2kbqlACdEvT/8Mdf58Y2X+liEyVAScNC
DxoAnjsB0+cE5eLzLc7+RjjbRP0BhNVw
=fpGE
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

#195157

On 2018-04-29, <tomas@tuxteam.de> <tomas@tuxteam.de> wrote:
>
>> 
>> How can I do that with proper Unix commands?
>
> The "classical":
>
>   tomas@trotzki:/tmp$ cat foo
>   Arvo Part
>   Harold Pinter
>   Lucio Battisti
>   Antonio Amurri
>   Eduardo De Filippo
>   Eduardo De Filippo
>
> tomas@trotzki:/tmp$ sort -u < foo | sed -e 's/$/,/g' | tr '\n' ' '
> Antonio Amurri, Arvo Part, Eduardo De Filippo, Harold Pinter, Lucio Battisti, 
>
> Note that (a) the list ends with ", " (comma-space) and has no newline at
> the end, but I preferred to present a simple proto-solution easier for you
> to munge.
>
> The tr is there because sed goes line-wise and thus can't replace newlines.
> Except that it can; you pay for it, though [1], [2].
>

Not knowing anything I came up with (after some research):

sort -u < test.txt | tr '\n' ',' 

No sed. 

My list has a comma at the beginning as well as at the end, though (with
no newline, either).

But if we were going to the moon with this I would be economizing that
supernumerary process.

;-)

-- 
"Three prisoners were locked in a cell. When the largest of them finished his
food, he immediately ate the others. Too bad. An apostrophe in the right place
might have prevented a horrible crime." Joe Gunn

[toc] | [prev] | [next] | [standalone]

#195158

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sun, Apr 29, 2018 at 08:06:47PM +0000, Curt wrote:

[...]

> Not knowing anything I came up with (after some research):
> 
> sort -u < test.txt | tr '\n' ',' 
> 
> No sed. 

If you don't need the space after the comma then yes, sed is
superfluous. Reminds me that half of my work as a programmer
is done talking to the customer ;-)

> My list has a comma at the beginning as well as at the end, though (with
> no newline, either).
> 
> But if we were going to the moon with this I would be economizing that
> supernumerary process.
> 
> ;-)

:-))

Remember the size of the computers which went to the moon 1969. For
tr there's space (without unicode). For sed... hmmm. For Perl? Hell,
no!

Cheers
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlrmLEoACgkQBcgs9XrR2kaJ6QCfZLKgwpCnBXit01v/+JzyXpx+
ZOIAniY2EpMhbzsE0bYN2rsp0+MLwIUY
=h88w
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

#195160

[Multipart message — attachments visible in raw view] — view raw

On Sunday 29 April 2018 16:34:18 tomas@tuxteam.de wrote:

> On Sun, Apr 29, 2018 at 08:06:47PM +0000, Curt wrote:
>
> [...]
>
> > Not knowing anything I came up with (after some research):
> >
> > sort -u < test.txt | tr '\n' ','
> >
> > No sed.
>
> If you don't need the space after the comma then yes, sed is
> superfluous. Reminds me that half of my work as a programmer
> is done talking to the customer ;-)
>
> > My list has a comma at the beginning as well as at the end, though
> > (with no newline, either).
> >
> > But if we were going to the moon with this I would be economizing
> > that supernumerary process.
> >
> > ;-)
> >
> :-))
>
> Remember the size of the computers which went to the moon 1969. For
> tr there's space (without unicode). For sed... hmmm. For Perl? Hell,
> no!
>
For those that are too "new", that was AFAIK, and RCA 1802 with 4k of static ram. Very slow, and very low power. But having experience codeing for it in the later 70's, it was both stable and if the coder did his job, bulletproof. The program I wrote for it was still being used 20+ times a day by the production people at KRCR-tv in Redding CA, in 1996. Considering I wrote it in 1978, that 18 years of non-stop use at a tv station, its like The Simpsons, who just passed Gunsmoke as the longest running program ever. I suspect it was still in use during the day of June 30th 2008, the last day we broadcast in NTSC. For a program that made liberal use of self-modifying code, I'd say that was amazing. Lets also say that the 1802's architecture was also different from every other cpu in the 8/16 bit field then. The TI 9900 was the only simpler one, stripping any internal register that addressed memory directly.

-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#195162 — [OT - long running shows] Re: List words separated by comma and without duplicates

[Multipart message — attachments visible in raw view] — view raw

On 30/04/18 10:29, Gene Heskett wrote:
> The Simpsons, who just passed Gunsmoke as the longest running program ever.

I can't figure out what criteria are required to make that work.

Richard

[toc] | [prev] | [next] | [standalone]

#195163 — Re: [OT - long running shows] Re: List words separated by comma and without duplicates

On Sunday 29 April 2018 20:27:56 Richard Hector wrote:

> On 30/04/18 10:29, Gene Heskett wrote:
> > The Simpsons, who just passed Gunsmoke as the longest running
> > program ever.
>
> I can't figure out what criteria are required to make that work.
>
> Richard

Number of episodes aired. Gunsmoke ran for IIRC 22 years, 861 episodes. 
Yesterdays Simpsons made 862 episodes. Gunsmoke started in b&w, shot 
with image orthicon cameras, switched to vidicon's about the same time I 
was helping make the vidicon cameras that went down into the mohole on 
the Trieste in Fed '60. Yeah, I'm that old an old fart. :)


-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#195165 — Re: [OT - long running shows] Re: List words separated by comma and without duplicates

[Multipart message — attachments visible in raw view] — view raw

On 30/04/18 12:56, Gene Heskett wrote:
> On Sunday 29 April 2018 20:27:56 Richard Hector wrote:
> 
>> On 30/04/18 10:29, Gene Heskett wrote:
>>> The Simpsons, who just passed Gunsmoke as the longest running
>>> program ever.
>>
>> I can't figure out what criteria are required to make that work.
>>
>> Richard
> 
> Number of episodes aired. Gunsmoke ran for IIRC 22 years, 861 episodes. 
> Yesterdays Simpsons made 862 episodes. Gunsmoke started in b&w, shot 
> with image orthicon cameras, switched to vidicon's about the same time I 
> was helping make the vidicon cameras that went down into the mohole on 
> the Trieste in Fed '60. Yeah, I'm that old an old fart. :)

CBS News: 16,400+
General Hospital: 14,000+

https://en.wikipedia.org/wiki/List_of_longest-running_United_States_television_series

Other countries don't necessarily seem to count episodes, but also have
long-running series.

Richard

[toc] | [prev] | [next] | [standalone]

#195168 — Re: [OT - long running shows] Re: List words separated by comma and without duplicates

On Sunday 29 April 2018 21:28:56 Richard Hector wrote:

> On 30/04/18 12:56, Gene Heskett wrote:
> > On Sunday 29 April 2018 20:27:56 Richard Hector wrote:
> >> On 30/04/18 10:29, Gene Heskett wrote:
> >>> The Simpsons, who just passed Gunsmoke as the longest running
> >>> program ever.
> >>
> >> I can't figure out what criteria are required to make that work.
> >>
> >> Richard
> >
> > Number of episodes aired. Gunsmoke ran for IIRC 22 years, 861
> > episodes. Yesterdays Simpsons made 862 episodes. Gunsmoke started in
> > b&w, shot with image orthicon cameras, switched to vidicon's about
> > the same time I was helping make the vidicon cameras that went down
> > into the mohole on the Trieste in Fed '60. Yeah, I'm that old an old
> > fart. :)
>
> CBS News: 16,400+
> General Hospital: 14,000+
>
Yes, true, but those are daily, this was once a week.

> https://en.wikipedia.org/wiki/List_of_longest-running_United_States_te
>levision_series
>
> Other countries don't necessarily seem to count episodes, but also
> have long-running series.
>
> Richard



-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#195880

Rodolfo Medina <rodolfo.medina@gmail.com> writes:

> Suppose I have a file made by a list of names and surnames, e.g.:
>
> Arvo Part
> Harold Pinter
> Lucio Battisti
> Antonio Amurri
> Eduardo De Filippo
> Eduardo De Filippo
>
> and that I want them listed all on one line, separated by a comma and without
> duplicates, i.e.:
>
> Arvo Part, Harold Pinter, Lucio Battisti, Antonio Amurri, Eduardo De Filippo
>
> How can I do that with proper Unix commands?


Thanks to all who helped.  I adopted Tomas' solution:

perl -e 'while(<STDIN>) {chop; push(@result, $_) unless $seen{$_}++;} print(join(", ", @result), "\n");' < authors.auth > authors.txt

It seems to suit my needs.

Cheers,

Rodolfo

[toc] | [prev] | [next] | [standalone]

#195881

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, May 18, 2018 at 08:50:48PM +0200, Rodolfo Medina wrote:

[...]

> Thanks to all who helped.  I adopted Tomas' solution:

Hey, glad you liked it :-)

Cheers
- -- t
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlr/JnMACgkQBcgs9XrR2kZnkQCfZq2w0OFMvCq6yKjoFZxKGvG8
RzwAn1i96PaVqgvJD15R8IzBooAh7x2b
=Zr9+
-----END PGP SIGNATURE-----

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web