Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.sys.mac.advocacy > #137586

Re: It is stunning when you see how badly Windows operates: indexing

From Paul <nospam@needed.invalid>
Newsgroups comp.sys.mac.advocacy, alt.comp.os.windows-10, alt.comp.os.windows-11
Subject Re: It is stunning when you see how badly Windows operates: indexing
Date 2025-10-08 10:45 -0400
Organization A noiseless patient Spider
Message-ID <10c5tdl$1lg3l$1@dont-email.me> (permalink)
References (1 earlier) <mki30dF49p4U1@mid.individual.net> <10c1kms$knib$1@dont-email.me> <10c1mor$l2dr$1@dont-email.me> <10c1rg5$m0om$1@dont-email.me> <10c5ga0$1hrih$1@dont-email.me>

Cross-posted to 3 groups.

Show all headers | View raw


On Wed, 10/8/2025 7:01 AM, Daniel70 wrote:

> 
> Surely this can't be so .... or has Copilot just not completed the process??

You can make up cases and ask the AI for a result.

The idea of making my stupid cases, is just to get it
to dump the names of a representative set of possibilities.

To move a file can:

1) Require deleting it with a change journal event,
   then create it with a second change journal event.
   The AI tells us there is no single-event Move type for the job.
   There is a tendency in computer systems designs to use
   atomic events rather than "do fancy stuff". There is a lot of
   fancy stuff we could do, which in the end we don't do.

2) Moves can be orchestrated by the Name Change set of change journal events.

Now, you have to prepare your set of file names, and then
figure out which of those cases apply. Take the actual move
command that demonstrated a problem and get the AI description
of how that would be coded.

The journal dumps I've tried, at some point the design must
have changed. I didn't make a fetish out of this. Like at
one time, you could see a write-file event dumped, for
every 256KB or so of data. I don't think that scheme was
in the latest try I made of a dump.

******************

CoPilot Question

   In the NTFS change journal, has the design of the journal events been a
   constant with time or did the change journal design or representation change over the years ?

CoPilot Answer

The NTFS Change Journal - also known as the USN Journal - has evolved over time,
though its core purpose has remained consistent: to efficiently track
changes to files and directories on an NTFS volume.

### 🕰️ Evolution of the Change Journal Design

Here’s how its design and representation have changed over the years:

#### 1. **Initial Introduction (Windows 2000 / NTFS 3.0)**
- The Change Journal was introduced with NTFS 3.0.
- It stored records in a metafile called `$Extend\$UsnJrnl`.
- Each record included:
  - A 64-bit Update Sequence Number (USN)
  - File reference number
  - Reason flags (bitmask indicating the type of change)
  - Timestamp
  - File name

#### 2. **USN_RECORD Structure Versions**
- Over time, Microsoft introduced new versions of the `USN_RECORD` structure:
  - **V2**: Used in early implementations (Windows 2000, XP, etc.)
  - **V3 and V4**: Introduced in later versions of Windows to support new features and metadata
- These versions added fields like extended reason flags, source info, and more precise timestamps.

#### 3. **Bit Flags Expansion**
- The `Reason` field uses bit flags like `USN_REASON_DATA_OVERWRITE`, `USN_REASON_FILE_CREATE`, etc.
- New flags were added over time to reflect more granular changes (e.g., file renames, ACL changes).

#### 4. **Performance and Scalability Enhancements**
- As file systems grew to support millions of files, the journal’s internal mechanisms were optimized for speed and reduced overhead.
- Journals became more configurable (e.g., size limits, retention policies).

#### 5. **Integration with New Features**
- Windows features like **File History**, **Windows Search**, and **Backup utilities** began leveraging the Change Journal.
- This led to more robust APIs and better documentation for developers.
---
### 🔍 Summary

While the fundamental concept of logging file system changes has stayed the same,
the **structure**, **metadata**, and **integration points** of the NTFS Change Journal
have evolved to meet the demands of modern systems. It’s a great example of a
foundational technology quietly adapting behind the scenes.

Would you like to dive into the specifics of the USN_RECORD structure versions
or how backup tools use the journal today?
**********************************8

    Paul

Back to comp.sys.mac.advocacy | Previous | NextPrevious in thread | Next in thread | Find similar | Unroll thread


Thread

It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-09 16:30 -0700
  Re: It is stunning when you see how badly Windows operates: indexing Hank Rogers <Hank@nospam.invalid> - 2025-08-09 18:55 -0500
    Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-09 17:08 -0700
      Re: It is stunning when you see how badly Windows operates: indexing Hank Rogers <invalid@nospam.com> - 2025-08-10 03:22 +0000
        Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-09 23:52 -0400
          Re: It is stunning when you see how badly Windows operates: indexing "David B." <BD@hotmail.co.uk> - 2025-08-10 09:54 +0100
  Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-09 23:28 -0400
  Re: It is stunning when you see how badly Windows operates: indexing "Mr. Man-wai Chang" <toylet.toylet@gmail.com> - 2025-08-10 12:34 +0800
    Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-10 11:25 -0400
      Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-10 12:37 -0700
        Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-10 19:46 -0400
          Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-10 17:00 -0700
            Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-11 00:04 -0400
              Re: It is stunning when you see how badly Windows operates: indexing Marion <marion@facts.com> - 2025-08-11 06:13 +0000
                Re: It is stunning when you see how badly Windows operates: indexing Hank Rogers <invalid@nospam.com> - 2025-08-11 06:23 +0000
                Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-11 11:26 -0700
                Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-12 01:15 -0400
                Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-12 08:54 -0700
                Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-12 13:41 -0400
                Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-12 10:54 -0700
                Re: It is stunning when you see how badly Windows operates: indexing Tom Elam <thomas.e.elam@gmail.com> - 2025-10-01 14:56 -0400
                Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-01 14:08 -0700
              Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-11 10:57 -0700
                Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-12 08:52 -0700
      Re: It is stunning when you see how badly Windows operates: indexing "Mr. Man-wai Chang" <toylet.toylet@gmail.com> - 2025-08-11 20:22 +0800
        Re: It is stunning when you see how badly Windows operates: indexing Marion <marion@facts.com> - 2025-08-11 19:16 +0000
          Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-11 17:03 -0700
          Re: It is stunning when you see how badly Windows operates: indexing "Mr. Man-wai Chang" <toylet.toylet@gmail.com> - 2025-08-12 12:36 +0800
    Re: It is stunning when you see how badly Windows operates: indexing Brian Gregory <void-invalid-dead-dontuse@email.invalid> - 2025-10-06 15:49 +0100
  Re: It is stunning when you see how badly Windows operates: indexing WolfFan <akwolffan@zoho.com> - 2025-08-10 18:33 -0400
    Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-11 02:12 -0400
      Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-11 11:25 -0700
        Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-11 18:02 -0400
          Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-11 17:01 -0700
            Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-12 01:24 -0400
              Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-12 08:49 -0700
              Re: It is stunning when you see how badly Windows operates: indexing Marion <marion@facts.com> - 2025-08-12 17:10 +0000
          Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-08-12 08:55 -0700
  Re: It is stunning when you see how badly Windows operates: indexing Tom Elam <thomas.e.elam@gmail.com> - 2025-08-17 16:03 -0400
    Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-08-17 18:15 -0400
  Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-01 22:19 +0200
    Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-01 14:18 -0700
    Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-05 15:01 -0700
      Re: It is stunning when you see how badly Windows operates: indexing Lars Poulsen <lars@cleo.beagle-ears.com> - 2025-10-06 02:46 +0000
        Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-05 21:16 -0700
          Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-06 19:23 +1100
            Re: It is stunning when you see how badly Windows operates: indexing MikeS <MikeS@fred.com> - 2025-10-06 09:56 +0100
              Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-06 20:54 +1100
                Re: It is stunning when you see how badly Windows operates: indexing Brian Gregory <void-invalid-dead-dontuse@email.invalid> - 2025-10-06 15:59 +0100
                Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-06 22:35 +0200
                Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 13:53 -0700
                Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-08 20:12 +1100
                Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:18 -0700
                Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 08:59 -0700
                Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-08 20:18 +1100
                Re: It is stunning when you see how badly Windows operates: indexing "J. P. Gilliver" <G6JPG@255soft.uk> - 2025-10-08 11:16 +0100
                Re: It is stunning when you see how badly Windows operates: indexing MikeS <MikeS@fred.com> - 2025-10-06 17:28 +0100
                Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 09:33 -0700
                Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-08 20:23 +1100
                Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:20 -0700
            Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-06 14:22 +0200
              Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 09:00 -0700
              Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-08 20:30 +1100
                Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-08 11:48 +0200
                Re: It is stunning when you see how badly Windows operates: indexing "J. P. Gilliver" <G6JPG@255soft.uk> - 2025-10-08 11:25 +0100
                Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:22 -0700
                Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:21 -0700
                Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:20 -0700
            Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 08:58 -0700
              Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-06 22:44 +0200
                Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 13:56 -0700
                Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-06 23:22 +0200
                Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 16:51 -0700
              Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-08 20:35 +1100
                Re: It is stunning when you see how badly Windows operates: indexing "J. P. Gilliver" <G6JPG@255soft.uk> - 2025-10-08 11:38 +0100
                Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:23 -0700
          Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-10-06 04:52 -0400
          Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-06 14:17 +0200
            Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 09:12 -0700
              Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-06 22:47 +0200
                Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 14:12 -0700
                Re: It is stunning when you see how badly Windows operates: indexing "Carlos E.R." <robin_listas@es.invalid> - 2025-10-06 23:25 +0200
                Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 16:55 -0700
                Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-08 21:21 +1100
                Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:24 -0700
  Re: It is stunning when you see how badly Windows operates: indexing Brian Gregory <void-invalid-dead-dontuse@email.invalid> - 2025-10-06 15:45 +0100
    Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 09:13 -0700
      Re: It is stunning when you see how badly Windows operates: indexing Brian Gregory <void-invalid-dead-dontuse@email.invalid> - 2025-10-06 17:48 +0100
        Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 10:08 -0700
    Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-10-06 19:51 -0400
      Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 17:27 -0700
        Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-10-06 21:47 -0400
          Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-06 22:13 -0700
          Re: It is stunning when you see how badly Windows operates: indexing Daniel70 <daniel47@nomail.afraid.org> - 2025-10-08 22:01 +1100
            Re: It is stunning when you see how badly Windows operates: indexing Paul <nospam@needed.invalid> - 2025-10-08 10:45 -0400
              Re: It is stunning when you see how badly Windows operates: indexing Alan <nuh-uh@nope.com> - 2025-10-08 08:25 -0700

csiph-web