Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #11510 > unrolled thread

Interesting utilities

Back to article view | Back to comp.lang.c

Contents

  Interesting utilities jacob navia <jacob@spamsink.net> - 2011-10-11 11:57 +0200
    Re: Interesting utilities Roberto Waltman <usenet@rwaltman.com> - 2011-10-15 00:35 -0400
      Re: Interesting utilities jacob navia <jacob@spamsink.net> - 2011-10-15 13:06 +0200
        Re: Interesting utilities Geoff <geoff@invalid.invalid> - 2011-10-15 10:26 -0700
        [OT] Interesting utilities Roberto Waltman <usenet@rwaltman.com> - 2011-10-15 16:14 -0400
        Re: Interesting utilities Kaz Kylheku <kaz@kylheku.com> - 2011-10-16 01:24 +0000
          Re: Interesting utilities jacob navia <jacob@spamsink.net> - 2011-10-16 09:45 +0200
          Re: Interesting utilities jacob navia <jacob@spamsink.net> - 2011-10-16 14:05 +0200
            Re: Interesting utilities jacob navia <jacob@spamsink.net> - 2011-10-16 14:39 +0200
              Re: Interesting utilities Kaz Kylheku <kaz@kylheku.com> - 2011-10-17 16:38 +0000
    Re: Interesting utilities Kaz Kylheku <kaz@kylheku.com> - 2011-10-16 01:18 +0000
      Re: Interesting utilities Ian Collins <ian-news@hotmail.com> - 2011-10-16 14:34 +1300
    Re: Interesting utilities XeCycle <xecycle@gmail.com> - 2011-10-16 10:42 +0800
      Re: Interesting utilities Kaz Kylheku <kaz@kylheku.com> - 2011-10-16 03:53 +0000
        Re: Interesting utilities XeCycle <xecycle@gmail.com> - 2011-10-16 21:25 +0800

#11510 — Interesting utilities

I am developing in the Mac OS X (10.7) system, and it has some nice 
features that I hadn't seen in any windows/unix system.

One of them is the "mdfind" utility that is incredibly powerful:

mdfind some-identifier

will look in the whole machine for "some-identifier" and write you in 
standard output the name of all files that have a match. It is like

grep -R "some-identifier" /*.*

but it comes back in seconds instead of hours like grep.

It is very useful, and I never found something even remotely equivalent 
under linux/sun os

Another nifty thing is the splint code checker. (This one is probably
present under linux also but I never saw it under windows)

The problem with it is that it is incredibly verbose, swamping the
few interesting diagonstics under a mountain of rubbish. After a long
session with it I have trimmed it to give me reasonable short output.

I put all options in a shell script:

/bin/sh
splint -formatconst -dependenttrans -globstate -compdef -mayaliasunique 
-paramuse +boolint -immediatetrans -retvalother -noeffect 
-unqualifiedtrans -compdestroy -temptrans -branchstate -usereleased 
-kepttrans -compmempass -nullstate -nullpass -nullret -boolops 
-mustfreeonly -incondefs +charint -retvalint -predboolint -realcompare $1

[toc] | [next] | [standalone]

#11717

jacob navia  wrote:
>I am developing in the Mac OS X (10.7) system, and it has some nice 
>features that I hadn't seen in any windows/unix system.
>
>mdfind some-identifier ...
>but it comes back in seconds instead of hours like grep.

That probably means that it looks into a keyword database, built by
indexing the files in the background, over many hours instead of
seconds.
And yes, the same functionality is available under Linux and Windows.

>Another nifty thing is the splint code checker. (This one is probably
>present under linux also but I never saw it under windows)

A windows installer is available at the home page:
http://www.splint.org/

--
Roberto Waltman

[ Please reply to the group,
  return address is invalid ]

[toc] | [prev] | [next] | [standalone]

#11727

Le 15/10/11 06:35, Roberto Waltman a écrit :
> jacob navia  wrote:
>> I am developing in the Mac OS X (10.7) system, and it has some nice
>> features that I hadn't seen in any windows/unix system.
>>
>> mdfind some-identifier ...
>> but it comes back in seconds instead of hours like grep.
>
> That probably means that it looks into a keyword database, built by
> indexing the files in the background, over many hours instead of
> seconds.
> And yes, the same functionality is available under Linux and Windows.


Sorry but I have never even heard about something like that under
windows/linux.
Maybe you have an example?

>
>> Another nifty thing is the splint code checker. (This one is probably
>> present under linux also but I never saw it under windows)
>
> A windows installer is available at the home page:
> http://www.splint.org/
>

Thanks, I overlooked that one.

[toc] | [prev] | [next] | [standalone]

#11734

On Sat, 15 Oct 2011 13:06:46 +0200, jacob navia <jacob@spamsink.net>
wrote:

>Sorry but I have never even heard about something like that under
>windows/linux.
>Maybe you have an example?

It's called Spotlight on OS/X and Windows Search since the release of
Windows XP. I can't remember what it was called in Windows 2000.
Settings are available in the Windows Control panel applet named
"Indexing Options" on Windows 7. 

[toc] | [prev] | [next] | [standalone]

#11739 — [OT] Interesting utilities

jacob navia  wrote:

[ re: File indexing / search applications ]
> ... something like that under
>windows/linux.
>Maybe you have an example?

http://en.wikipedia.org/wiki/Windows_Search
http://www.microsoft.com/download/en/details.aspx?id=23
http://www.vandenoever.info/software/strigi/akademy2006.pdf	
http://www.wikinfo.org/index.php/Comparison_of_desktop_search_software
http://en.wikipedia.org/wiki/Beagle_%28software%29

The last page says the Beagle project, (only one I used under Linux,)
is not being developed anymore. I was not aware of that.

--
Roberto Waltman

[ Please reply to the group,
  return address is invalid ]

[toc] | [prev] | [next] | [standalone]

#11749

On 2011-10-15, jacob navia <jacob@spamsink.net> wrote:
> Le 15/10/11 06:35, Roberto Waltman a écrit :
>> jacob navia  wrote:
>>> I am developing in the Mac OS X (10.7) system, and it has some nice
>>> features that I hadn't seen in any windows/unix system.
>>>
>>> mdfind some-identifier ...
>>> but it comes back in seconds instead of hours like grep.
>>
>> That probably means that it looks into a keyword database, built by
>> indexing the files in the background, over many hours instead of
>> seconds.
>> And yes, the same functionality is available under Linux and Windows.
>
> Sorry but I have never even heard about something like that under
> windows/linux.
> Maybe you have an example?

http://freshmeat.net/search?q=file+indexing&submit=Search

45 hits, some of them relevant-looking.

http://freshmeat.net/search?q=file+search&submit=Search

251 hits.

Man, you really are still a goof, sorry to say.

Windows has dynamic file indexing since Vista, if not before.

[toc] | [prev] | [next] | [standalone]

#11757

Le 16/10/11 03:24, Kaz Kylheku a écrit :
> On 2011-10-15, jacob navia<jacob@spamsink.net>  wrote:
>> Le 15/10/11 06:35, Roberto Waltman a écrit :
>>> jacob navia  wrote:
>>>> I am developing in the Mac OS X (10.7) system, and it has some nice
>>>> features that I hadn't seen in any windows/unix system.
>>>>
>>>> mdfind some-identifier ...
>>>> but it comes back in seconds instead of hours like grep.
>>>
>>> That probably means that it looks into a keyword database, built by
>>> indexing the files in the background, over many hours instead of
>>> seconds.
>>> And yes, the same functionality is available under Linux and Windows.
>>
>> Sorry but I have never even heard about something like that under
>> windows/linux.
>> Maybe you have an example?
>
> http://freshmeat.net/search?q=file+indexing&submit=Search
>
> 45 hits, some of them relevant-looking.
>
> http://freshmeat.net/search?q=file+search&submit=Search
>
> 251 hits.
>
> Man, you really are still a goof, sorry to say.
>
> Windows has dynamic file indexing since Vista, if not before.

Sure, but it doesn't index everything, as I have seen lately.
Only some selected directories. SUre you can change that but still,
it doesn't match the easy of use of not doing anything
:-)

I used the find utility in windows XP, and the terrible interface
with the dog just convinced me to avoid it completely. In any case it
wasn't a background process as it is in the mac


Under linux there *could* be an indexing utility around of course,
but it is not automatic like mdfind.

What is interesting in the mac is the number of automatic utilities
like mdfind, for instance the indredibly useful backup utility, that
work out of the box without any need for big configurations, etc.

[toc] | [prev] | [next] | [standalone]

#11762

Le 16/10/11 03:24, Kaz Kylheku a écrit :
>> Maybe you have an example?
>
> http://freshmeat.net/search?q=file+indexing&submit=Search
>
> 45 hits, some of them relevant-looking.
>
> http://freshmeat.net/search?q=file+search&submit=Search
>
> 251 hits.
>
> Man, you really are still a goof, sorry to say.
>
> Windows has dynamic file indexing since Vista, if not before.

Sure sure....

In Wikipedia the program "tracker" is listed as a search utility for
linux.

Downloaded the package from
http://projects.gnome.org/tracker
Then:

bzip2 -d tracker-0.1.12.bz2
untarred result
changed to the direzctory and typed ./configure
./configure fails with:
configure: error: glib-compile-schemas not found.

Searched google for that.

After much searching I found out that actually I have that package
installed in /usr/bin.

Apparently I am not the only one havinng that problem, there are many
hits about that error message. But there are no solutions. Some of
the solutions point to upgrading to a newer version of the OS...
but I am running already the latest Ubuntu (I try to keep current since
under linux if you aren't upgrading every 2-3 months the OS nothing
will work any more)

The problem is that I have the package but the
configure script doesn't fdind it (it is in /usr/bin... I really do not
understand why that directory is not searched)

I edit the configure script and see that there is an environment
variable that should contain the path to those "compile-schemas"
I set the variable, rerun, nothing, same error.

Now I discover that I am not really a shell/ configure / language
expert. I can't figure out why the thing isn't found I give up.

Wasted time: Approx 1 hour.

mdfind came with the mac already, nothing to do for me.

But of course I am a "goofer"...

Look Mr Kaz, I am well aware that an indexing utility could
exist for linux even one that after several hours of work
could even work better than mdfind.

But... really, I do not feel like wasting more time in  it.
Yes, it is free, I did not pay for my Ubuntu version. But I
have learned the hard way that unless you want to specialize in
linux administration you will be wasting so much time with it that it
is just not worth trying to do things like this for somebody
that is not a linux admin...


I wonder how would I explain the usage of the linux utility to
my wife...

"Look Annie, just download that source package"

Then, there is no install program, you have to type
./configure

etc...
`
Good luck with that

:-(

jacob

[toc] | [prev] | [next] | [standalone]

#11764

Well, there was another way, sorry.

The ubuntu distribution has a package for tracker binaries.
Excellent!

I download the package and apparently installs without problems.

Obviously there isn't any documentation at all. After some searching
I discover that it installed in "Accessories". I click in the tracker
line and a window appears with a text field called "Search"

I type "iList" and press return. Since I downloaded the source code
of the containers library in a subdirectory under /home/jacob
I know that this word appears.

Nothing, nothing changes, everything remains empty. Mmmmmm...
Maybe I haven't configured something?

Open a shell "man tracker" gives no answer, no documentation.

I type

tracker-<TAB>

and see a list of tracker programs. I call the configuration
program and... yes, my home directory is listed under the
directories to be searched recursively.

I see another utility: tracker-search. I write

$ tracker-search iList
Results: 0

Ahhh that is why everything is empty. It doesn't find anything.
Why?

Mistery. The configuration program says that it has written the
configuration and I see /home/jacob in the recursive directory
list.

I go to /home/jacob/ccl and write:

$ grep iList *.c
containers.h: extern ListInterface iList;

Yes, I am not dreaming...

Well, I do not know what is going on. There must be some configuration
option I did not find out. Probably after some hours of work I could
get this stuff running, who knows?

[toc] | [prev] | [next] | [standalone]

#11816

On 2011-10-16, jacob navia <jacob@spamsink.net> wrote:
> Well, there was another way, sorry.
>
> The ubuntu distribution has a package for tracker binaries.
> Excellent!

Yes, I just found it in three seconds while looking at your
other post:

Is this it?

http://packages.ubuntu.com/lucid/tracker

I would not try to compile from scratch packages that have any
complex dependencies. (Not as a Ubuntu user, that is.)

> I download the package and apparently installs without problems.
> Obviously there isn't any documentation at all. After some searching

I searched the web instead to find the home of this thing.

http://projects.gnome.org/tracker/documentation.html

"All the documentation is on the live wiki"

http://live.gnome.org/Tracker/Documentation/GettingStarted

Much of the documentation is developer-oriented. This looks like an
experimental new project of some sorts.

> Well, I do not know what is going on. There must be some configuration
> option I did not find out. Probably after some hours of work I could
> get this stuff running, who knows?

The wiki page says there is is a:

"tracker-miner-fs | The file system crawler to mine data from local files"

I doubt that you will find anything if this has not run and done its job.

[toc] | [prev] | [next] | [standalone]

#11747

On 2011-10-11, jacob navia <jacob@spamsink.net> wrote:
> I am developing in the Mac OS X (10.7) system, and it has some nice 
> features that I hadn't seen in any windows/unix system.
>
> One of them is the "mdfind" utility that is incredibly powerful:
>
> mdfind some-identifier
>
> will look in the whole machine for "some-identifier" and write you in 
> standard output the name of all files that have a match. It is like
>
> grep -R "some-identifier" /*.*
>
> but it comes back in seconds instead of hours like grep.

So it is not more powerful than grep, since it computes the same thing. The
word you are looking for is "faster". 

Can it find any regular expression in seconds?

> It is very useful, and I never found something even remotely equivalent 
> under linux/sun os

How hard do you look?

I use id-utils and ctags for doing this over programming projects.
In id-utils there is a "mkid" utility which indexes the identifiers found
in a directory tree. This is gathered into a big file called ID. 
The ID file is queried with "lid".

I just set up an Ubuntu system where "apt-get install id-utils"
got me mkid in seconds.

> Another nifty thing is the splint code checker. (This one is probably
> present under linux also but I never saw it under windows)

There is a direct pointer to a Windows download page on the Splint
home page! This kind of thing is not going to come preinstalled on a Wintel PC
from Best Buy.

> The problem with it is that it is incredibly verbose, swamping the
> few interesting diagonstics under a mountain of rubbish. After a long
> session with it I have trimmed it to give me reasonable short output.

This is by design. To get value out of splint you have to read the
documentation and start using the annotations.

Splint can statically check things like that malloced objects are freed
somewhere, at least for some programs (maybe not in general).

[toc] | [prev] | [next] | [standalone]

#11750

On 10/16/11 02:18 PM, Kaz Kylheku wrote:
> On 2011-10-11, jacob navia<jacob@spamsink.net>  wrote:
>> I am developing in the Mac OS X (10.7) system, and it has some nice
>> features that I hadn't seen in any windows/unix system.
>>
>> One of them is the "mdfind" utility that is incredibly powerful:
>>
>> mdfind some-identifier
>>
>> will look in the whole machine for "some-identifier" and write you in
>> standard output the name of all files that have a match. It is like
>>
>> grep -R "some-identifier" /*.*
>>
>> but it comes back in seconds instead of hours like grep.
>
> So it is not more powerful than grep, since it computes the same thing. The
> word you are looking for is "faster".
>
> Can it find any regular expression in seconds?
>
>> It is very useful, and I never found something even remotely equivalent
>> under linux/sun os
>
> How hard do you look?
>
> I use id-utils and ctags for doing this over programming projects.
> In id-utils there is a "mkid" utility which indexes the identifiers found
> in a directory tree. This is gathered into a big file called ID.
> The ID file is queried with "lid".
>
> I just set up an Ubuntu system where "apt-get install id-utils"
> got me mkid in seconds.

These utilities are all well and good on a relatively static filesystem, 
but the fun begins when you want to keep the database as up to date as 
possible.

I know all about the fun involved, I have to maintain an index over a 
couple of multi-TB filesystems with tens of millions of files in each!

-- 
Ian Collins

[toc] | [prev] | [next] | [standalone]

#11752

jacob navia <jacob@spamsink.net> writes:

> I am developing in the Mac OS X (10.7) system, and it has some
> nice features that I hadn't seen in any windows/unix system.
>
> One of them is the "mdfind" utility that is incredibly powerful:
>
> mdfind some-identifier
>
> will look in the whole machine for "some-identifier" and write
> you in standard output the name of all files that have a
> match. It is like
>
> grep -R "some-identifier" /*.*
>
> but it comes back in seconds instead of hours like grep.
>
> It is very useful, and I never found something even remotely
> equivalent under linux/sun os

What about `locate' from GNU Findutils?

-- 
Carl Lei (XeCycle)
Department of Physics, Shanghai Jiao Tong University
OpenPGP public key: 7795E591
Fingerprint: 1FB6 7F1F D45D F681 C845 27F7 8D71 8EC4 7795 E591

[toc] | [prev] | [next] | [standalone]

#11753

On 2011-10-16, XeCycle <xecycle@gmail.com> wrote:
> jacob navia <jacob@spamsink.net> writes:
>> It is very useful, and I never found something even remotely
>> equivalent under linux/sun os
>
> What about `locate' from GNU Findutils?

Locate searches only file names, not their contents.

That's been a staple of Linux distros as far as I can remember (which could
potentially takes us back as far as 1993).

Long before operating systems "for the rest of them" started getting
any kind of indexing.

[toc] | [prev] | [next] | [standalone]

#11767

[Multipart message — attachments visible in raw view] — view raw

Kaz Kylheku <kaz@kylheku.com> writes:

> On 2011-10-16, XeCycle <xecycle@gmail.com> wrote:
>> jacob navia <jacob@spamsink.net> writes:
>>> It is very useful, and I never found something even remotely
>>> equivalent under linux/sun os
>>
>> What about `locate' from GNU Findutils?
>
> Locate searches only file names, not their contents.
>
> That's been a staple of Linux distros as far as I can remember (which could
> potentially takes us back as far as 1993).
>
> Long before operating systems "for the rest of them" started getting
> any kind of indexing.

I remembered something else, if you don't mind a GUI that comes along:

Pinot: http://pinot.berlios.de/
Recoll: http://www.lesbonscomptes.com/recoll/index.html.en

I have tried Pinot, which does the work well, however I stopped it since
I didn't use it very much, and it takes quite some resources.

-- 
Carl Lei (XeCycle)
Department of Physics, Shanghai Jiao Tong University
OpenPGP public key: 7795E591
Fingerprint: 1FB6 7F1F D45D F681 C845 27F7 8D71 8EC4 7795 E591

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.c

csiph-web