Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.sys.mac.advocacy > #137548
| From | Paul <nospam@needed.invalid> |
|---|---|
| Newsgroups | comp.sys.mac.advocacy, alt.comp.os.windows-10, alt.comp.os.windows-11 |
| Subject | Re: It is stunning when you see how badly Windows operates: indexing |
| Date | 2025-10-06 19:51 -0400 |
| Organization | A noiseless patient Spider |
| Message-ID | <10c1kms$knib$1@dont-email.me> (permalink) |
| References | <1078llp$1he7o$3@dont-email.me> <mki30dF49p4U1@mid.individual.net> |
Cross-posted to 3 groups.
On Mon, 10/6/2025 10:45 AM, Brian Gregory wrote:
> On 10/08/2025 00:30, Alan wrote:
>> I'm going some tech support for my brother today, and that's meant looking "under the hood" of his Windows 10 machine.
>>
>> And it is astounding how poorly Windows handles indexing content.
>>
>> I had to uninstall OneDrive and set it up again from scratch and in the course of that I moved the contents that had previously been synchronized to two folders with "(Old)" added to make sure that nothing got lost when we did it.
>>
>> And I wasn't surprised when resetting his connections to his OneDrive store and the company Sharepoint store set off a flurry of indexing. I wasn't even surprised that that took a bit of time to finish; I was re- downloading a lot of files which from the perspective of Windows, all needed to be indexed anew.
>>
>> What was completely surprising was that simply moving the those two "Old" folders immediately caused Windows to re-index that content! I decided to clean up the two separate "Old" stores into a single folder, and all of a sudden, the indexing which was at 0 in Settings:Search:Searching Windows
>>
>> Windows isn't smart enough to recognize that these are all files that its already indexed and they're now just in a new location!
>>
>> About 80,000 files were move about 30 minutes ago, and the re-indexing isn't even 25% done!
>>
>> If I do a similar thing on macOS, it's done almost before the files have finished the move.
>
> Surely most people just turn indexing off.
> It's pretty useless anyway.
> But you can leave it on if you have an SSD without it causing too much of a problem.
>
How the Indexer works, is going to depend on how the Change Journal works.
The Indexer is not a 60 line program and a "tight ball of code". There
are three running processes in Task Manager for the operation of it. It's
rather big, as packaging goes. The processes even run with different
priorities (one runs at a lower priority setting than the others and
gets trashed and restarted every once in a while).
*******
Doing operations on the file system, developers are able to get different
levels of performance. The NFI.exe utility from Microsoft, is my "go to"
utility when I must know what is in a partition. It was written in the
year 2003, and badly needs an update to be written (in particular, the
$FILE_NAME entry, the string stored in there needs to be printed on
the screen and that is missing information). Files which have multiple
$FILE_NAME, are hardlinked files. Notice here, how I am able to find
the Windows 11 Search Indexer inverted index file. It is stored on the
file system as Filenum 23838 and is lightly fragmented.
.\nfi C: > nfi-C-out.txt # And, 145 seconds later... 210MB listing 2,771,720 lines
File 23838
\ProgramData\Microsoft\Search\Data\Applications\Windows\Windows.db
$STANDARD_INFORMATION (resident)
$FILE_NAME (resident)
$DATA (nonresident)
logical sectors 33709448-33712863 (0x2025d88-0x2026adf)
... (30 fragments total) (early search Index had hundreds of fragments)
logical sectors 91984216-91984239 (0x57b9158-0x57b916f)
The Indexer file is 2.35MB in size, as the Indexer in this machine is not configured
to index anything..
Now compare that to a VoidTool Everything.exe attempt to list a file system.
Everything.exe does an "initial index" at machine startup, and you can tap into
that by using the utiity without all of its baggage ("one-shot" utility usage).
Now, it's not always this fast, but this is a good demo.
.\Everything.exe -create-filelist every_c.txt "C:" # Examine the text file, to see the metadata
# they keep in there. This emulates the initial
# indexing process. You cannot list the WsL1 tree
# files this way, they are ignored. Neither would
# it pretend to access System Volume Information files.
# There are safety reasons for staying out of there.
7 seconds, 56,985,788 bytes output 360,000 lines or so. Entries for five files shown.
Filename, Size, Date Modified, Date Created, Attributes
"C:\Users\paul\Downloads\Windows-universal-samples-main--includes-OCR-sample.zip",28899211,133659001969022138,134034795503869343,32
"C:\Users\paul\Downloads\windows10.0-kb5034232-x64_SafeOS-WinRE-example.cab",7530358,133696434360300569,134034795502291984,32
"C:\Users\paul\Downloads\Windows10Upgrade9252.exe",3343496,133195821885381788,134034795502453969,32
"C:\Users\paul\Downloads\Windows11Upgrade_EN.zip",409341,132937705786434792,134034795502762944,32
"C:\Users\paul\Downloads\Windows6.1-KB3004394-v2-x64.msu",2442957,132894276659630869,134034795501816348,32
That code then, has different objectives, but the code is 20x faster in the example.
The very first version of the Voidtools work, did that job in *2* seconds, but back then
all you got was a filename absolute path (no numbers on the end).
The Content Indexing that Microsoft does, is a mondo bloat thing to do.
It's orders of magnitude slower, and it takes all day to Content Index my
"NAS collection".
The Content Indexer is recommended to not be indexing more than 1 million files.
That's what Microsoft specs as a practical limit. You can drive it harder
than that, but expect some aspect to be slower.
Summary: No one in their right mind uses the search box in File Explorer.
How many times has it missed stuff ? You decide.
But there are signs that maintainers still fiddle around with
the implementation. In Win10 it uses Windows.edb "Jet Blue" database.
In Win11 it uses Windows.db SQLite3 database. This makes no different
at the user level (and isn't this why we write code???).
I think if we had a Change Journal expert here, we could get an answer
on how file moves are encoded, how many entries such an activity would
span, and whether an architecture would allow optimization for it.
It's the same when a naive person watches the miserably slow file deletion.
The person would say "hey, why not stop the computer for a second,
total up all the changes to the $MFT and just blast them in as a huge edit".
and that causes a huge explanation of why we can't/won't be doing that.
If that was even remotely possible, we'd have done it by now. Or at least,
there would be a credible explanation of how we could do it.
Paul
Back to comp.sys.mac.advocacy | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-09 16:30 -0700
Re: It is stunning when you see how badly Windows operates: indexing Hank Rogers <Hank@nospam.invalid> - 2025-08-09 18:55 -0500
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-09 17:08 -0700
Re: It is stunning when you see how badly Windows operates: indexing Hank Rogers <invalid@nospam.com> - 2025-08-10 03:22 +0000
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-09 23:52 -0400
Re: It is stunning when you see how badly Windows operates: indexing "David B." <BD@hotmail.co.uk> - 2025-08-10 09:54 +0100
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-09 23:28 -0400
Re: It is stunning when you see how badly Windows operates: indexing "Mr. Man-wai Chang" <toylet.toylet@gmail.com> - 2025-08-10 12:34 +0800
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-10 11:25 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-10 12:37 -0700
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-10 19:46 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-10 17:00 -0700
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-11 00:04 -0400
Re: It is stunning when you see how badly Windows operates: indexing Marion <marion@facts.com> - 2025-08-11 06:13 +0000
Re: It is stunning when you see how badly Windows operates: indexing Hank Rogers <invalid@nospam.com> - 2025-08-11 06:23 +0000
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-11 11:26 -0700
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-12 01:15 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-12 08:54 -0700
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-12 13:41 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-12 10:54 -0700
Re: It is stunning when you see how badly Windows operates: indexing Tom Elam <thomas.e.elam@gmail.com> - 2025-10-01 14:56 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-01 14:08 -0700
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-11 10:57 -0700
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-12 08:52 -0700
Re: It is stunning when you see how badly Windows operates: indexing "Mr. Man-wai Chang" <toylet.toylet@gmail.com> - 2025-08-11 20:22 +0800
Re: It is stunning when you see how badly Windows operates: indexing Marion <marion@facts.com> - 2025-08-11 19:16 +0000
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-11 17:03 -0700
Re: It is stunning when you see how badly Windows operates: indexing "Mr. Man-wai Chang" <toylet.toylet@gmail.com> - 2025-08-12 12:36 +0800
Re: It is stunning when you see how badly Windows operates: indexing Brian Gregory <void-invalid-dead-dontuse@email.invalid> - 2025-10-06 15:49 +0100
Re: It is stunning when you see how badly Windows operates: indexing WolfFan <akwolffan@zoho.com> - 2025-08-10 18:33 -0400
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-11 02:12 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-11 11:25 -0700
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-11 18:02 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-11 17:01 -0700
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-12 01:24 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-12 08:49 -0700
Re: It is stunning when you see how badly Windows operates: indexing Marion <marion@facts.com> - 2025-08-12 17:10 +0000
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-12 08:55 -0700
Re: It is stunning when you see how badly Windows operates: indexing Tom Elam <thomas.e.elam@gmail.com> - 2025-08-17 16:03 -0400
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-17 18:15 -0400
Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-01 22:19 +0200
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-01 14:18 -0700
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-05 15:01 -0700
Re: It is stunning when you see how badly Windows operates: indexing Lars Poulsen <lars@cleo.beagle-ears.com> - 2025-10-06 02:46 +0000
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-05 21:16 -0700
Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-06 19:23 +1100
Re: It is stunning when you see how badly Windows operates: indexing MikeS <MikeS@fred.com> - 2025-10-06 09:56 +0100
Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-06 20:54 +1100
Re: It is stunning when you see how badly Windows operates: indexing Brian Gregory <void-invalid-dead-dontuse@email.invalid> - 2025-10-06 15:59 +0100
Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-06 22:35 +0200
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 13:53 -0700
Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-08 20:12 +1100
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:18 -0700
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 08:59 -0700
Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-08 20:18 +1100
Re: It is stunning when you see how badly Windows operates: indexing "J. P. Gilliver" <G6JPG@255soft.uk> - 2025-10-08 11:16 +0100
Re: It is stunning when you see how badly Windows operates: indexing MikeS <MikeS@fred.com> - 2025-10-06 17:28 +0100
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 09:33 -0700
Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-08 20:23 +1100
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:20 -0700
Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-06 14:22 +0200
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 09:00 -0700
Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-08 20:30 +1100
Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-08 11:48 +0200
Re: It is stunning when you see how badly Windows operates: indexing "J. P. Gilliver" <G6JPG@255soft.uk> - 2025-10-08 11:25 +0100
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:22 -0700
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:21 -0700
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:20 -0700
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 08:58 -0700
Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-06 22:44 +0200
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 13:56 -0700
Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-06 23:22 +0200
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 16:51 -0700
Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-08 20:35 +1100
Re: It is stunning when you see how badly Windows operates: indexing "J. P. Gilliver" <G6JPG@255soft.uk> - 2025-10-08 11:38 +0100
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:23 -0700
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-10-06 04:52 -0400
Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-06 14:17 +0200
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 09:12 -0700
Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-06 22:47 +0200
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 14:12 -0700
Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-06 23:25 +0200
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 16:55 -0700
Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-08 21:21 +1100
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:24 -0700
Re: It is stunning when you see how badly Windows operates: indexing Brian Gregory <void-invalid-dead-dontuse@email.invalid> - 2025-10-06 15:45 +0100
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 09:13 -0700
Re: It is stunning when you see how badly Windows operates: indexing Brian Gregory <void-invalid-dead-dontuse@email.invalid> - 2025-10-06 17:48 +0100
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 10:08 -0700
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-10-06 19:51 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 17:27 -0700
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-10-06 21:47 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 22:13 -0700
Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-08 22:01 +1100
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-10-08 10:45 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:25 -0700
csiph-web