Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.sys.mac.advocacy > #137552
| From | Alan <nuh-uh@nope.com> |
|---|---|
| Newsgroups | comp.sys.mac.advocacy, alt.comp.os.windows-10, alt.comp.os.windows-11 |
| Subject | Re: It is stunning when you see how badly Windows operates: indexing |
| Date | 2025-10-06 17:27 -0700 |
| Organization | A noiseless patient Spider |
| Message-ID | <10c1mor$l2dr$1@dont-email.me> (permalink) |
| References | <1078llp$1he7o$3@dont-email.me> <mki30dF49p4U1@mid.individual.net> <10c1kms$knib$1@dont-email.me> |
Cross-posted to 3 groups.
On 2025-10-06 16:51, Paul wrote:
> On Mon, 10/6/2025 10:45 AM, Brian Gregory wrote:
>> On 10/08/2025 00:30, Alan wrote:
>>> I'm going some tech support for my brother today, and that's meant looking "under the hood" of his Windows 10 machine.
>>>
>>> And it is astounding how poorly Windows handles indexing content.
>>>
>>> I had to uninstall OneDrive and set it up again from scratch and in the course of that I moved the contents that had previously been synchronized to two folders with "(Old)" added to make sure that nothing got lost when we did it.
>>>
>>> And I wasn't surprised when resetting his connections to his OneDrive store and the company Sharepoint store set off a flurry of indexing. I wasn't even surprised that that took a bit of time to finish; I was re- downloading a lot of files which from the perspective of Windows, all needed to be indexed anew.
>>>
>>> What was completely surprising was that simply moving the those two "Old" folders immediately caused Windows to re-index that content! I decided to clean up the two separate "Old" stores into a single folder, and all of a sudden, the indexing which was at 0 in Settings:Search:Searching Windows
>>>
>>> Windows isn't smart enough to recognize that these are all files that its already indexed and they're now just in a new location!
>>>
>>> About 80,000 files were move about 30 minutes ago, and the re-indexing isn't even 25% done!
>>>
>>> If I do a similar thing on macOS, it's done almost before the files have finished the move.
>>
>> Surely most people just turn indexing off.
>> It's pretty useless anyway.
>> But you can leave it on if you have an SSD without it causing too much of a problem.
>>
>
> How the Indexer works, is going to depend on how the Change Journal works.
> The Indexer is not a 60 line program and a "tight ball of code". There
> are three running processes in Task Manager for the operation of it. It's
> rather big, as packaging goes. The processes even run with different
> priorities (one runs at a lower priority setting than the others and
> gets trashed and restarted every once in a while).
>
> *******
>
> Doing operations on the file system, developers are able to get different
> levels of performance. The NFI.exe utility from Microsoft, is my "go to"
> utility when I must know what is in a partition. It was written in the
> year 2003, and badly needs an update to be written (in particular, the
> $FILE_NAME entry, the string stored in there needs to be printed on
> the screen and that is missing information). Files which have multiple
> $FILE_NAME, are hardlinked files. Notice here, how I am able to find
> the Windows 11 Search Indexer inverted index file. It is stored on the
> file system as Filenum 23838 and is lightly fragmented.
>
> .\nfi C: > nfi-C-out.txt # And, 145 seconds later... 210MB listing 2,771,720 lines
>
> File 23838
> \ProgramData\Microsoft\Search\Data\Applications\Windows\Windows.db
>
> $STANDARD_INFORMATION (resident)
> $FILE_NAME (resident)
> $DATA (nonresident)
> logical sectors 33709448-33712863 (0x2025d88-0x2026adf)
> ... (30 fragments total) (early search Index had hundreds of fragments)
> logical sectors 91984216-91984239 (0x57b9158-0x57b916f)
>
> The Indexer file is 2.35MB in size, as the Indexer in this machine is not configured
> to index anything..
>
> Now compare that to a VoidTool Everything.exe attempt to list a file system.
> Everything.exe does an "initial index" at machine startup, and you can tap into
> that by using the utiity without all of its baggage ("one-shot" utility usage).
> Now, it's not always this fast, but this is a good demo.
>
> .\Everything.exe -create-filelist every_c.txt "C:" # Examine the text file, to see the metadata
> # they keep in there. This emulates the initial
> # indexing process. You cannot list the WsL1 tree
> # files this way, they are ignored. Neither would
> # it pretend to access System Volume Information files.
> # There are safety reasons for staying out of there.
>
> 7 seconds, 56,985,788 bytes output 360,000 lines or so. Entries for five files shown.
>
> Filename, Size, Date Modified, Date Created, Attributes
> "C:\Users\paul\Downloads\Windows-universal-samples-main--includes-OCR-sample.zip",28899211,133659001969022138,134034795503869343,32
> "C:\Users\paul\Downloads\windows10.0-kb5034232-x64_SafeOS-WinRE-example.cab",7530358,133696434360300569,134034795502291984,32
> "C:\Users\paul\Downloads\Windows10Upgrade9252.exe",3343496,133195821885381788,134034795502453969,32
> "C:\Users\paul\Downloads\Windows11Upgrade_EN.zip",409341,132937705786434792,134034795502762944,32
> "C:\Users\paul\Downloads\Windows6.1-KB3004394-v2-x64.msu",2442957,132894276659630869,134034795501816348,32
>
> That code then, has different objectives, but the code is 20x faster in the example.
> The very first version of the Voidtools work, did that job in *2* seconds, but back then
> all you got was a filename absolute path (no numbers on the end).
>
> The Content Indexing that Microsoft does, is a mondo bloat thing to do.
> It's orders of magnitude slower, and it takes all day to Content Index my
> "NAS collection".
>
> The Content Indexer is recommended to not be indexing more than 1 million files.
> That's what Microsoft specs as a practical limit. You can drive it harder
> than that, but expect some aspect to be slower.
>
> Summary: No one in their right mind uses the search box in File Explorer.
> How many times has it missed stuff ? You decide.
>
> But there are signs that maintainers still fiddle around with
> the implementation. In Win10 it uses Windows.edb "Jet Blue" database.
> In Win11 it uses Windows.db SQLite3 database. This makes no different
> at the user level (and isn't this why we write code???).
>
> I think if we had a Change Journal expert here, we could get an answer
> on how file moves are encoded, how many entries such an activity would
> span, and whether an architecture would allow optimization for it.
>
> It's the same when a naive person watches the miserably slow file deletion.
> The person would say "hey, why not stop the computer for a second,
> total up all the changes to the $MFT and just blast them in as a huge edit".
> and that causes a huge explanation of why we can't/won't be doing that.
> If that was even remotely possible, we'd have done it by now. Or at least,
> there would be a credible explanation of how we could do it.
>
> Paul
Paul, you're the idiot that claimed moving a file resulted in a
"deletefile" in the USN Journal...
...and you were spectacularly wrong.
Back to comp.sys.mac.advocacy | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-09 16:30 -0700
Re: It is stunning when you see how badly Windows operates: indexing Hank Rogers <Hank@nospam.invalid> - 2025-08-09 18:55 -0500
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-09 17:08 -0700
Re: It is stunning when you see how badly Windows operates: indexing Hank Rogers <invalid@nospam.com> - 2025-08-10 03:22 +0000
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-09 23:52 -0400
Re: It is stunning when you see how badly Windows operates: indexing "David B." <BD@hotmail.co.uk> - 2025-08-10 09:54 +0100
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-09 23:28 -0400
Re: It is stunning when you see how badly Windows operates: indexing "Mr. Man-wai Chang" <toylet.toylet@gmail.com> - 2025-08-10 12:34 +0800
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-10 11:25 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-10 12:37 -0700
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-10 19:46 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-10 17:00 -0700
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-11 00:04 -0400
Re: It is stunning when you see how badly Windows operates: indexing Marion <marion@facts.com> - 2025-08-11 06:13 +0000
Re: It is stunning when you see how badly Windows operates: indexing Hank Rogers <invalid@nospam.com> - 2025-08-11 06:23 +0000
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-11 11:26 -0700
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-12 01:15 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-12 08:54 -0700
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-12 13:41 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-12 10:54 -0700
Re: It is stunning when you see how badly Windows operates: indexing Tom Elam <thomas.e.elam@gmail.com> - 2025-10-01 14:56 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-01 14:08 -0700
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-11 10:57 -0700
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-12 08:52 -0700
Re: It is stunning when you see how badly Windows operates: indexing "Mr. Man-wai Chang" <toylet.toylet@gmail.com> - 2025-08-11 20:22 +0800
Re: It is stunning when you see how badly Windows operates: indexing Marion <marion@facts.com> - 2025-08-11 19:16 +0000
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-11 17:03 -0700
Re: It is stunning when you see how badly Windows operates: indexing "Mr. Man-wai Chang" <toylet.toylet@gmail.com> - 2025-08-12 12:36 +0800
Re: It is stunning when you see how badly Windows operates: indexing Brian Gregory <void-invalid-dead-dontuse@email.invalid> - 2025-10-06 15:49 +0100
Re: It is stunning when you see how badly Windows operates: indexing WolfFan <akwolffan@zoho.com> - 2025-08-10 18:33 -0400
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-11 02:12 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-11 11:25 -0700
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-11 18:02 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-11 17:01 -0700
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-12 01:24 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-12 08:49 -0700
Re: It is stunning when you see how badly Windows operates: indexing Marion <marion@facts.com> - 2025-08-12 17:10 +0000
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-12 08:55 -0700
Re: It is stunning when you see how badly Windows operates: indexing Tom Elam <thomas.e.elam@gmail.com> - 2025-08-17 16:03 -0400
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-17 18:15 -0400
Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-01 22:19 +0200
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-01 14:18 -0700
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-05 15:01 -0700
Re: It is stunning when you see how badly Windows operates: indexing Lars Poulsen <lars@cleo.beagle-ears.com> - 2025-10-06 02:46 +0000
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-05 21:16 -0700
Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-06 19:23 +1100
Re: It is stunning when you see how badly Windows operates: indexing MikeS <MikeS@fred.com> - 2025-10-06 09:56 +0100
Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-06 20:54 +1100
Re: It is stunning when you see how badly Windows operates: indexing Brian Gregory <void-invalid-dead-dontuse@email.invalid> - 2025-10-06 15:59 +0100
Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-06 22:35 +0200
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 13:53 -0700
Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-08 20:12 +1100
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:18 -0700
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 08:59 -0700
Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-08 20:18 +1100
Re: It is stunning when you see how badly Windows operates: indexing "J. P. Gilliver" <G6JPG@255soft.uk> - 2025-10-08 11:16 +0100
Re: It is stunning when you see how badly Windows operates: indexing MikeS <MikeS@fred.com> - 2025-10-06 17:28 +0100
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 09:33 -0700
Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-08 20:23 +1100
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:20 -0700
Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-06 14:22 +0200
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 09:00 -0700
Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-08 20:30 +1100
Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-08 11:48 +0200
Re: It is stunning when you see how badly Windows operates: indexing "J. P. Gilliver" <G6JPG@255soft.uk> - 2025-10-08 11:25 +0100
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:22 -0700
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:21 -0700
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:20 -0700
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 08:58 -0700
Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-06 22:44 +0200
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 13:56 -0700
Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-06 23:22 +0200
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 16:51 -0700
Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-08 20:35 +1100
Re: It is stunning when you see how badly Windows operates: indexing "J. P. Gilliver" <G6JPG@255soft.uk> - 2025-10-08 11:38 +0100
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:23 -0700
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-10-06 04:52 -0400
Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-06 14:17 +0200
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 09:12 -0700
Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-06 22:47 +0200
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 14:12 -0700
Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-06 23:25 +0200
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 16:55 -0700
Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-08 21:21 +1100
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:24 -0700
Re: It is stunning when you see how badly Windows operates: indexing Brian Gregory <void-invalid-dead-dontuse@email.invalid> - 2025-10-06 15:45 +0100
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 09:13 -0700
Re: It is stunning when you see how badly Windows operates: indexing Brian Gregory <void-invalid-dead-dontuse@email.invalid> - 2025-10-06 17:48 +0100
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 10:08 -0700
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-10-06 19:51 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 17:27 -0700
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-10-06 21:47 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 22:13 -0700
Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-08 22:01 +1100
Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-10-08 10:45 -0400
Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:25 -0700
csiph-web