Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #261765

Re: [a bit OT] Automate a (G o o g l e) search from a list of strings

Show all headers | View raw

On Tue, Sep 19, 2023 at 10:42:14AM -0400, steve wrote:
> I have a list of 200 keywords and would like for every one to launch a
> search on a specific website and put the result(s) in a file [...]

Take a list of words and turn it into a single Google query matching any
of them:

    me% cat keywords
    corsairs
    buccaneers
    privateers

Desired query:

    https://www.google.com/search?q=corsairs+OR+buccaneers+OR+privateers

Script:

    me% cat search
    #!/bin/sh
    export PATH=/usr/local/bin:/bin:/usr/bin
    set -o nounset
    umask 022

    query=$(tr "[:space:]" "+" < keywords |
            sed -e 's/ /+OR+/g' -e 's/+$//' -e 's/+/+OR+/g')

    curl -s -L -o pirate.htm "https://www.google.com/search?q=${query}"
    ls -l pirate.htm
    exit 0

Results:

    me% ./search
    -rw-r--r-- 1 vogelke 220487 19-Sep-2023 21:33:21 pirate.htm

If you want results in text form, lots of programs can do that:

    me% lynx -cookies -accept_all_cookies -dump -width 80 pirate.htm
    Tampa Bay Buccaneers: Buccaneers Home
    www.buccaneers.com
    The official source of the latest Bucs headlines, news, videos, photos,
    tickets, rosters, stats, schedule and gameday information.
  
    People also ask
    What is the difference between a pirate and a corsair?
    What kind of pirate is a corsair?
    What were Spanish pirates called?
    What is the history of corsairs?

    Pirates, Privateers, Corsairs, Buccaneers: What's the Difference?
    www.britannica.com > Demystified > Geography & Travel
    Corsairs were essentially privateers, although the term corsair carried
    an added religious connotation because the conflict was between Muslim
    and Christian...

HTH.

-- 
Karl Vogel                      I don't speak for the USAF or my company

Leonardo diCaprio in Titanic: Despite acting like a plank of wood,
he still can't float.  --"Top Ten Most Welcome Movie Deaths", Stylus Magazine

Back to linux.debian.user | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[a bit OT] Automate a (G o o g l e) search from a list of strings steve <dlist@bluewin.ch> - 2023-09-19 16:50 +0200
  Re: [a bit OT] Automate a (G o o g l e) search from a list of strings Michael Kjörling <2695bd53d63c@ewoof.net> - 2023-09-19 16:50 +0200
  Re: [a bit OT] Automate a (G o o g l e) search from a list of strings Nicolas George <george@nsup.org> - 2023-09-19 17:00 +0200
    Re: [a bit OT] Automate a (G o o g l e) search from a list of strings Jason <jason@7748229.xyz> - 2023-09-19 19:30 +0200
    Re: [a bit OT] Automate a (G o o g l e) search from a list of strings steve <dlist@bluewin.ch> - 2023-09-20 08:20 +0200
      Re: [a bit OT] Automate a (G o o g l e) search from a list of strings Andy Smith <andy@strugglers.net> - 2023-09-20 10:50 +0200
        Re: [a bit OT] Automate a (G o o g l e) search from a list of strings steve <dlist@bluewin.ch> - 2023-09-20 11:40 +0200
        Re: [a bit OT] Automate a (G o o g l e) search from a list of strings Tom Browder <tom.browder@gmail.com> - 2023-09-20 17:50 +0200
          Re: [a bit OT] Automate a (G o o g l e) search from a list of strings Nicolas George <george@nsup.org> - 2023-09-20 20:40 +0200
            Re: [a bit OT] Automate a (G o o g l e) search from a list of strings <tomas@tuxteam.de> - 2023-09-20 21:20 +0200
            Re: [a bit OT] Automate a (G o o g l e) search from a list of strings Tom Browder <tom.browder@gmail.com> - 2023-09-20 23:00 +0200
              Re: [a bit OT] Automate a (G o o g l e) search from a list of strings The Wanderer <wanderer@fastmail.fm> - 2023-09-21 00:00 +0200
                Re: [a bit OT] Automate a (G o o g l e) search from a list of strings Tamas Sohamar <sohamartamas@gmail.com> - 2023-09-21 07:10 +0200
  Re: [a bit OT] Automate a (G o o g l e) search from a list of strings Karl Vogel <vogelke@pobox.com> - 2023-09-20 04:10 +0200
    Re: [a bit OT] Automate a (G o o g l e) search from a list of strings Greg Wooledge <greg@wooledge.org> - 2023-09-20 04:40 +0200

csiph-web