Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > alt.comp.os.windows-11 > #17793 > unrolled thread

MS Shadow Copy service

Back to article view | Back to alt.comp.os.windows-11

Contents

  MS Shadow Copy service micky <NONONOmisc07@fmguy.com> - 2025-03-16 17:02 -0400
    Re: MS Shadow Copy service User One <noreply@invalid.com> - 2025-03-16 22:03 +0000
      Re: MS Shadow Copy service Stan Brown <the_stan_brown@fastmail.fm> - 2025-03-16 20:42 -0700
      Re: MS Shadow Copy service micky <NONONOmisc07@fmguy.com> - 2025-03-29 00:50 -0400
        Re: MS Shadow Copy service micky <NONONOmisc07@fmguy.com> - 2025-03-29 01:38 -0400
    Re: MS Shadow Copy service micky <NONONOmisc07@fmguy.com> - 2025-03-16 21:49 -0400
      Re: MS Shadow Copy service Paul <nospam@needed.invalid> - 2025-03-17 00:46 -0400
        Re: MS Shadow Copy service "Carlos E.R." <robin_listas@es.invalid> - 2025-03-17 13:16 +0100
          Re: MS Shadow Copy service Paul <nospam@needed.invalid> - 2025-03-17 11:25 -0400
            Re: MS Shadow Copy service "Carlos E.R." <robin_listas@es.invalid> - 2025-03-17 19:50 +0100
              Re: MS Shadow Copy service Paul <nospam@needed.invalid> - 2025-03-17 15:27 -0400
                Re: MS Shadow Copy service "Carlos E.R." <robin_listas@es.invalid> - 2025-03-17 21:34 +0100
                  Re: MS Shadow Copy service Frank Slootweg <this@ddress.is.invalid> - 2025-03-18 14:45 +0000
                    Re: MS Shadow Copy service Paul <nospam@needed.invalid> - 2025-03-18 20:13 -0400
      Re: MS Shadow Copy service Paul <nospam@needed.invalid> - 2025-03-17 01:07 -0400
        Re: MS Shadow Copy service micky <NONONOmisc07@fmguy.com> - 2025-03-27 01:00 -0400
          Re: MS Shadow Copy service Paul <nospam@needed.invalid> - 2025-03-27 02:51 -0400
    Re: MS Shadow Copy service Newyana2 <newyana@invalid.nospam> - 2025-03-17 08:41 -0400
      Re: MS Shadow Copy service Frank Slootweg <this@ddress.is.invalid> - 2025-03-17 16:45 +0000
        Re: MS Shadow Copy service Paul <nospam@needed.invalid> - 2025-03-17 15:32 -0400
          Re: MS Shadow Copy service Newyana2 <newyana@invalid.nospam> - 2025-03-17 17:27 -0400
            Re: MS Shadow Copy service Paul <nospam@needed.invalid> - 2025-03-17 18:16 -0400
            Re: MS Shadow Copy service Frank Slootweg <this@ddress.is.invalid> - 2025-03-18 13:49 +0000
    Re: MS Shadow Copy service Frank Slootweg <this@ddress.is.invalid> - 2025-03-17 16:36 +0000
      Re: MS Shadow Copy service micky <NONONOmisc07@fmguy.com> - 2025-03-27 01:05 -0400
        Re: MS Shadow Copy service Char Jackson <none@none.invalid> - 2025-03-28 00:27 -0500
          Re: MS Shadow Copy service micky <NONONOmisc07@fmguy.com> - 2025-03-28 12:17 -0400
    Re: MS Shadow Copy service ...w¡ñ§±¤ñ  <winstonmvp@gmail.com> - 2025-03-17 10:44 -0700
      Re: MS Shadow Copy service Newyana2 <newyana@invalid.nospam> - 2025-03-17 17:28 -0400
        Re: MS Shadow Copy service Paul <nospam@needed.invalid> - 2025-03-17 18:50 -0400
        Re: MS Shadow Copy service ...w¡ñ§±¤ñ  <winstonmvp@gmail.com> - 2025-03-20 11:52 -0700
          Re: MS Shadow Copy service micky <NONONOmisc07@fmguy.com> - 2025-03-27 01:09 -0400

Page 1 of 2  [1] 2  Next page →

#17793 — MS Shadow Copy service

So for the first time, I tried copying and moving files within win11. It
took about 5 minutes to copy one file, and about 20 minutes to move 10
directories.  This is, how do they say it?, Unacceptable.

The bussiest task was MS Shadow Copy service.  None of these files were
in use.  They were just sitting in the dat or downloads directories. I'd
just turn off the service -- should I? -- but it might well come in
handy if it means I could back up files from Forte Agent while it was
running. 

I wasn't using File Explorer, but PowerDesk, a 34d-party, but I assume
it just formats the same commands tht File Explorer does. 

Does this "feature" bite everyone who gets win11? 

[toc] | [next] | [standalone]

#17794

On 16/03/2025 21:02, micky wrote:
> So for the first time, I tried copying and moving files within win11. It
> took about 5 minutes to copy one file, and about 20 minutes to move 10
> directories.  This is, how do they say it?, Unacceptable.
> 
> The bussiest task was MS Shadow Copy service.  None of these files were
> in use.  They were just sitting in the dat or downloads directories. I'd
> just turn off the service -- should I? -- but it might well come in
> handy if it means I could back up files from Forte Agent while it was
> running.
> 
> I wasn't using File Explorer, but PowerDesk, a 34d-party, but I assume
> it just formats the same commands tht File Explorer does.
> 
> Does this "feature" bite everyone who gets win11?
> 
> 

Just use robocopy cmd and it will be faster than your "PowerDesk, a 
34d-party".

It is time that you grow up (mentally, of course)  and stop using 
crap-ware you find on the net. Microsoft tools should be the first thing 
you should always try first.

The simplest cmd to use is:

robocopy D:\Ventoy F:\ *.iso /MOV

This command will move all iso files to a new drive called F:\

You can also use the mir switch but the destination will be wiped clean 
and a mirror image of source created. This may not be what a user want 
all the time because they might want to keep some files in the 
destination folder, and only the new files copied.

[toc] | [prev] | [next] | [standalone]

#17796

On Sun, 16 Mar 2025 22:03:47 +0000, User One wrote:
> On 16/03/2025 21:02, micky wrote:
> > [quoted text muted]
> > 
> > Does this "feature" bite everyone who gets win11? 
> 
> Just use robocopy cmd and it will be faster than your "PowerDesk, a 
> 34d-party".
> 
> It is time that you grow up (mentally, of course)  and stop using 
> crap-ware you find on the net. Microsoft tools should be the first thing 
> you should always try first.
> 
> The simplest cmd to use is:
> 
> robocopy D:\Ventoy F:\ *.iso /MOV
> 
> This command will move all iso files to a new drive called F:\
> 
> You can also use the mir switch but the destination will be wiped clean 
> and a mirror image of source created. This may not be what a user want 
> all the time because they might want to keep some files in the 
> destination folder, and only the new files copied.

And a very nice feature of Robocopy is the /L option, which shows you 
what _would_ be done but doesn't actually do it. Whenever I have the 
slightest doubt about the Robocopy command I've typed, I append a 
space and /L before hitting Enter or Return, then examine the log 
carefully. If all is well, I hit the up arrow, backspace twice to get 
rid of /L, and hit Enter or Return.

-- 
Stan Brown, Tehachapi, California, USA         https://BrownMath.com/
Shikata ga nai...

[toc] | [prev] | [next] | [standalone]

#18033

In alt.comp.os.windows-11, on Sun, 16 Mar 2025 22:03:47 +0000, User One
<noreply@invalid.com> wrote:

>On 16/03/2025 21:02, micky wrote:
>> So for the first time, I tried copying and moving files within win11. It
>> took about 5 minutes to copy one file, and about 20 minutes to move 10
>> directories.  This is, how do they say it?, Unacceptable.
>> 
>> The bussiest task was MS Shadow Copy service.  None of these files were
>> in use.  They were just sitting in the dat or downloads directories. I'd
>> just turn off the service -- should I? -- but it might well come in
>> handy if it means I could back up files from Forte Agent while it was
>> running.
>> 
>> I wasn't using File Explorer, but PowerDesk, a 34d-party, but I assume
>> it just formats the same commands tht File Explorer does.
>> 
>> Does this "feature" bite everyone who gets win11?
>> 
>> 
>
>Just use robocopy cmd and it will be faster than your "PowerDesk, a 
>34d-party".

Not faster than when Powerdesk was working right.   Which was the same
time that File Explorer was working right. 
>
>It is time that you grow up (mentally, of course)  and stop using 

You're so sweet.   You really know how to win friends and influence
people.    How come you haven't posted here for more than 2 years?  Do
you use another name when you're not being insulting.  

>crap-ware you find on the net.

My version of PowerDesk is definitely a good program, with featurs MS FE
doesn't have.  I tried lots of other file managers and this is my
favorite. 

> Microsoft tools should be the first thing 
>you should always try first.

I did try MS FE first and I didn't like it.  I still don't, although it
does have one advantage.  Why do you think there are so many competing
file managers?  Other people don't like it either.   You seem to have
reservations or you wouldn't be pushing Robocopy. 

I've been using PowerDesk for 20 or 25 years.  I like its extra
features, and it's never given me trouble, until this month, at the same
time that File Explorer didn't work either.  (Now they both work. Go
figure.) 

Worth noting that it's version 5, no longer distributed afaik, not
version 8 or 9 that they are promoting now.   It's a free version
requiring no installation code.  If anyone wants a copy, I'll send it to
him.    

>The simplest cmd to use is:
>
>robocopy D:\Ventoy F:\ *.iso /MOV
>
>This command will move all iso files to a new drive called F:\

Yes, I know, but a) I've never wanted to do this, and b) surely you
don't expect me to come up with a command line for every copy and move I
want to do.  The reason they invented GUIs is to make things go more
quickly. 

>You can also use the mir switch but the destination will be wiped clean 
>and a mirror image of source created. This may not be what a user want 
>all the time because they might want to keep some files in the 
>destination folder, and only the new files copied.

[toc] | [prev] | [next] | [standalone]

#18036

In alt.comp.os.windows-11, on Sat, 29 Mar 2025 00:50:08 -0400, micky
<NONONOmisc07@fmguy.com> wrote:

>
>I've been using PowerDesk for 20 or 25 years.  I like its extra
>features, and it's never given me trouble, until this month, at the same
>time that File Explorer didn't work either.  (Now they both work. Go
>figure.) 
>
>Worth noting that it's version 5, no longer distributed afaik, not
>version 8 or 9 that they are promoting now.   It's a free version
>requiring no installation code.  If anyone wants a copy, I'll send it to
>him.    

It's version 9 they are selling now**, for $40, and they don't seem to
have an intro version, like they once did.  It was bought by Avanquest,
but version 5 was still written by V-Com.  it's guaranteed for 30 days.
5 or 10 years ago I bought a new version from Avanquest and it had 2
substantial flaws that made using it not worth it.  They did give me my
money back.   they've had plenty of time to fix the flaws, but I'm happy
with version 5. 

[toc] | [prev] | [next] | [standalone]

#17795

In alt.comp.os.windows-11, on Sun, 16 Mar 2025 17:02:37 -0400, micky
<NONONOmisc07@fmguy.com> wrote:

>So for the first time, I tried copying and moving files within win11. It
>took about 5 minutes to copy one file, and about 20 minutes to move 10
>directories.  This is, how do they say it?, Unacceptable.
>
>The bussiest task was MS Shadow Copy service.  None of these files were
>in use.  They were just sitting in the dat or downloads directories. I'd
>just turn off the service -- should I? -- but it might well come in
>handy if it means I could back up files from Forte Agent while it was
>running. 
>
>I wasn't using File Explorer, but PowerDesk, a 3d-party, but I assume
>it just formats the same commands tht File Explorer does.

a) I thought I should still test it with File Explorer and the problem
is the same, as I was sure it would be. 

Using FE, I moved one file and it took 4 minutes.  As the other times,
the graph immediately goes 99% of the way across and then stops.   A
line in the box says it's "Calculating".  

b) I thought I should be able to reverse the move with Ctrl-Z, but it
did nothing.  Ctrl-Z used to reverse moves.  Has that been stopped or is
this another problem? 

c) I decided the file was expendable so I deleted it and the lnk, and
each time it took 50 seconds to send it to the recycle bin.  This time I
started from Everything, but I'm sure it doesn't matter where the
instruction is issued from. 



>
>Does this "feature" bite everyone who gets win11? 
>

[toc] | [prev] | [next] | [standalone]

#17797

On Sun, 3/16/2025 9:49 PM, micky wrote:
> In alt.comp.os.windows-11, on Sun, 16 Mar 2025 17:02:37 -0400, micky
> <NONONOmisc07@fmguy.com> wrote:
> 
>> So for the first time, I tried copying and moving files within win11. It
>> took about 5 minutes to copy one file, and about 20 minutes to move 10
>> directories.  This is, how do they say it?, Unacceptable.
>>
>> The bussiest task was MS Shadow Copy service.  None of these files were
>> in use.  They were just sitting in the dat or downloads directories. I'd
>> just turn off the service -- should I? -- but it might well come in
>> handy if it means I could back up files from Forte Agent while it was
>> running. 
>>
>> I wasn't using File Explorer, but PowerDesk, a 3d-party, but I assume
>> it just formats the same commands tht File Explorer does.
> 
> a) I thought I should still test it with File Explorer and the problem
> is the same, as I was sure it would be. 
> 
> Using FE, I moved one file and it took 4 minutes.  As the other times,
> the graph immediately goes 99% of the way across and then stops.   A
> line in the box says it's "Calculating".  
> 
> b) I thought I should be able to reverse the move with Ctrl-Z, but it
> did nothing.  Ctrl-Z used to reverse moves.  Has that been stopped or is
> this another problem? 
> 
> c) I decided the file was expendable so I deleted it and the lnk, and
> each time it took 50 seconds to send it to the recycle bin.  This time I
> started from Everything, but I'm sure it doesn't matter where the
> instruction is issued from. 
> 
>> Does this "feature" bite everyone who gets win11? 

Sysinternals Process Monitor, it will collect a trace of what is
going on in the machine, while you do your copy.

When a file is copied, a fixed size buffer can be used
by the copy code. The ProcMon trace would look like this

    ReadFile      1MB       time=1.001sec
    WriteFile     1MB       time=1.009sec
    ReadFile      1MB       time=1.017sec
    WriteFile     1MB       time=1.025sec
    ReadFile   78342 bytes  time=1.026sec
    Writefile  78342 bytes  time=1.027sec
    Progname      Exit()

the idea is, the read buffer is filled and emptied in
a loop, until a fraction of a cluster (or whatever)
is left. When the fractional one is finally done, the
program doing the copying will indicate that it is
exiting.

By studying what *other* programs are doing,
like if an ignorant program sandwiches itself
into the stream of activity, that can slow down
the progress of the copy.

This is why you use a Process Monitor program,
to see how all the players on the team are
performing. Are the programs all rowing the
boat in the same direction, or, are they wasting
your time.

You decide.

*******

I wrote my own copy program.
Now, who hasn't done this, right ?

.\ghettocopy test.bin out.bin
file_size = 1073741824
Starting run...
some_size = 1073741824
Read  1073741824 bytes in 000.369524 seconds    [about 3GB/sec or so]
Wrote 1073741824 bytes in 000.354558 seconds    [about 3GB/sec or so]
File copied successfully.

I got CoPilot to write the basic program.
I added my timing code I use, for those
not-so-accurate timestamps (there can be a 10%
error in timestamping on Windows!).

Now, I run the command again.

.\ghettocopy test.bin out.bin
file_size = 1073741824
Starting run...
some_size = 1073741824
Read  1073741824 bytes in 000.342627 seconds
Wrote 1073741824 bytes in 000.360895 seconds
File copied successfully.

How annoying. The speed didn't change :-)

I told CoPilot to read the source file, the
whole thing, into memory in one shot. And
write it out, the whole thing, in one shot.
This is obviously not scalable (it's a bad
way to write a program). But notice how, at the
moment, the read and write time are almost the
same. If I were to run this on a hard drive,
it would read at 200MB/sec and write at 200MB/sec.

Perhaps I should test that.

# Win10, hard drive
Read  1073741824 bytes in 000.525854 seconds   <=== once the file is in the System Read cache, it reads faster
Wrote 1073741824 bytes in 006.470807 seconds

# Win11, hard drive
Read  1073741824 bytes in 006.400220 seconds   <=== not in the system read cache
Wrote 1073741824 bytes in 007.081455 seconds

Read  1073741824 bytes in 000.748078 seconds   <=== NOW in the system read cache
Wrote 1073741824 bytes in 013.870901 seconds        [grrr...]

Read  1073741824 bytes in 000.731090 seconds
Wrote 1073741824 bytes in 008.017847 seconds

Not only is the write time variable,
the write LED is still lit after the program
exits, so the total write time is actually longer
than shown. The 6.5 second number is likely to
be the most reasonable result.

While I'm running it on Win11, it suddenly
decided it needs to look for more AV definitions
and scan something. It can't be the test file,
as that is all zeros inside and makes for a
boring AV scan.

So while I didn't run ProcMon for a look, I still
had some fun.

I need to compile ghettocopy.exe as a 64-bit program,
and I suspect it will then start handling files larger
than 4GB properly. That's one reason I can't share the
program with you right now, it's not properly tested.
It only works on smaller files.

Everyone thinks they can write a copy program,
and I am a member of that camp. You can see my OS
isn't being very nice to me. Even though I wrote
grease-lightning code that uses lots of RAM. Only
one invocation is shown above running at 3GB/sec.

   Paul

[toc] | [prev] | [next] | [standalone]

#17799

On 2025-03-17 05:46, Paul wrote:

...

> Everyone thinks they can write a copy program,
> and I am a member of that camp. You can see my OS
> isn't being very nice to me. Even though I wrote
> grease-lightning code that uses lots of RAM. Only
> one invocation is shown above running at 3GB/sec.

Sure. It was part of me learning Pascal back in 1985 or thereabouts :-)

The size of the buffer mattered a lot, I was using floppies.


-- 
Cheers, Carlos.

[toc] | [prev] | [next] | [standalone]

#17801

On Mon, 3/17/2025 8:16 AM, Carlos E.R. wrote:
> On 2025-03-17 05:46, Paul wrote:
> 
> ...
> 
>> Everyone thinks they can write a copy program,
>> and I am a member of that camp. You can see my OS
>> isn't being very nice to me. Even though I wrote
>> grease-lightning code that uses lots of RAM. Only
>> one invocation is shown above running at 3GB/sec.
> 
> Sure. It was part of me learning Pascal back in 1985 or thereabouts :-)
> 
> The size of the buffer mattered a lot, I was using floppies.
> 
> 

So I cross-compiled to a 64-bit program, because the *definition*
of fwrite() says I can write 5GB if I want.

It turns out, there is a *bug* in the C runtime, that prevents
the 5GB fwrite() from completing. It goes into an infinite loop
and the program stops. I had to kill it with a control-C.

The bug was filed in 2012, it is 2025, so obviously the internal
status of the bug report was "WONT FIX", even though no human
would say that out loud. If you're a dev, doesn't your skin crawl
when you do stuff like this ? Then, why isn't the *documentation*
for fwrite() modified ? I would put it like this.

   "Even though fwrite64() is implied by

    -D_FILE_OFFSET_BITS=64   # on the command line, to the compiler, early #define

    and you would hope 64bit code is 64bit code, the Microsoft
    fwrite64() does not work properly for amounts of buffer
    larger than 4GB. Thus the name of the routine has been
    changed to

    fwontwrite64()
   "

If Raymond was still there, he'd slap someone.

    Paul

[toc] | [prev] | [next] | [standalone]

#17805

On 2025-03-17 16:25, Paul wrote:
> On Mon, 3/17/2025 8:16 AM, Carlos E.R. wrote:
>> On 2025-03-17 05:46, Paul wrote:
>>
>> ...
>>
>>> Everyone thinks they can write a copy program,
>>> and I am a member of that camp. You can see my OS
>>> isn't being very nice to me. Even though I wrote
>>> grease-lightning code that uses lots of RAM. Only
>>> one invocation is shown above running at 3GB/sec.
>>
>> Sure. It was part of me learning Pascal back in 1985 or thereabouts :-)
>>
>> The size of the buffer mattered a lot, I was using floppies.
>>
>>
> 
> So I cross-compiled to a 64-bit program, because the *definition*
> of fwrite() says I can write 5GB if I want.
> 
> It turns out, there is a *bug* in the C runtime, that prevents
> the 5GB fwrite() from completing. It goes into an infinite loop
> and the program stops. I had to kill it with a control-C.
> 
> The bug was filed in 2012, it is 2025, so obviously the internal
> status of the bug report was "WONT FIX", even though no human
> would say that out loud. If you're a dev, doesn't your skin crawl
> when you do stuff like this ? Then, why isn't the *documentation*
> for fwrite() modified ? I would put it like this.
> 
>     "Even though fwrite64() is implied by
> 
>      -D_FILE_OFFSET_BITS=64   # on the command line, to the compiler, early #define
> 
>      and you would hope 64bit code is 64bit code, the Microsoft
>      fwrite64() does not work properly for amounts of buffer
>      larger than 4GB. Thus the name of the routine has been
>      changed to
> 
>      fwontwrite64()
>     "

Sigh. Reir para no llorar. Spanish saying. Laugh so as not to cry.

> 
> If Raymond was still there, he'd slap someone.
> 
>      Paul
> 
> 
> 


-- 
Cheers, Carlos.

[toc] | [prev] | [next] | [standalone]

#17806

On Mon, 3/17/2025 2:50 PM, Carlos E.R. wrote:
> On 2025-03-17 16:25, Paul wrote:

>>      fwontwrite64()
> 
> Sigh. Reir para no llorar. Spanish saying. Laugh so as not to cry.
> 
To be clear, this is unlikely to affect anyone, except
I can find questions from developers regarding the status
of the issue.

My purpose in using it, is as a "student exercise" in
why we don't do it this way. It's possible (I haven't checked
yet), that as a single kernel call, it could "lock" the OS and
not play nicely. But then, as a developer, if you choose to
write your code that way, that's your choice. The compiler people
know it is not their job, to do engineering for people. The
option should be left open, to make it any size you might want,
as the size_t implies. Then if you don't like the dynamic
response of the thing, you'll change your code.

I'm a bit curious how Windows Defender handles the
shadowing of a large request like that. Does it
bug out ? Dunno. It's one of the reasons for testing
and investigating... if it had worked.

   Paul

[toc] | [prev] | [next] | [standalone]

#17808

On 2025-03-17 20:27, Paul wrote:
> On Mon, 3/17/2025 2:50 PM, Carlos E.R. wrote:
>> On 2025-03-17 16:25, Paul wrote:
> 
>>>       fwontwrite64()
>>
>> Sigh. Reir para no llorar. Spanish saying. Laugh so as not to cry.
>>
> To be clear, this is unlikely to affect anyone, except
> I can find questions from developers regarding the status
> of the issue.
> 
> My purpose in using it, is as a "student exercise" in
> why we don't do it this way. It's possible (I haven't checked
> yet), that as a single kernel call, it could "lock" the OS and
> not play nicely. But then, as a developer, if you choose to
> write your code that way, that's your choice. The compiler people
> know it is not their job, to do engineering for people. The
> option should be left open, to make it any size you might want,
> as the size_t implies. Then if you don't like the dynamic
> response of the thing, you'll change your code.
> 
> I'm a bit curious how Windows Defender handles the
> shadowing of a large request like that. Does it
> bug out ? Dunno. It's one of the reasons for testing
> and investigating... if it had worked.

You can make a file copy in Linux with dd, and assign a huge cache:

bs=16G

It will read up to 16 gigs into ram, them write it :-D

I learned it as I saw the system I had then starting to swap and sweat. 
It surely obeyed my orders :-)


-- 
Cheers, Carlos.

[toc] | [prev] | [next] | [standalone]

#17817

Carlos E.R. <robin_listas@es.invalid> wrote:
[...]

> You can make a file copy in Linux with dd, and assign a huge cache:
> 
> bs=16G
> 
> It will read up to 16 gigs into ram, them write it :-D
> 
> I learned it as I saw the system I had then starting to swap and sweat. 
> It surely obeyed my orders :-)

  We already used bs=... with large block sizes in the 80s, to keep tape
drives (9-track reel-to-reel, CS80, DDS, etc.) streaming, instead of
going through very slow and damaging read/write, stop, back up,
read/write ... cycles. The default block size (512 bytes?) was way too
small for these devices.

[toc] | [prev] | [next] | [standalone]

#17822

On Tue, 3/18/2025 10:45 AM, Frank Slootweg wrote:
> Carlos E.R. <robin_listas@es.invalid> wrote:
> [...]
> 
>> You can make a file copy in Linux with dd, and assign a huge cache:
>>
>> bs=16G
>>
>> It will read up to 16 gigs into ram, them write it :-D
>>
>> I learned it as I saw the system I had then starting to swap and sweat. 
>> It surely obeyed my orders :-)
> 
>   We already used bs=... with large block sizes in the 80s, to keep tape
> drives (9-track reel-to-reel, CS80, DDS, etc.) streaming, instead of
> going through very slow and damaging read/write, stop, back up,
> read/write ... cycles. The default block size (512 bytes?) was way too
> small for these devices.
> 

I use "dd" (the chrysocome windows one) for making test files for
my copy program I just wrote (to test with). I almost forget to check
the program actually copied a file, and discovered before running
the test case, the program didn't actually copy. It's because I
bungled the buffer handling (which was one of the reasons I got
CoPilot to write me one without a buffer loop in the first place).

   .\dd if=/dev/random of=test.bin bs=1048576 count=20480

Because the file is random, if any of the material is transferred out
of order, the sha1sum hash check on the file will be different, and then
I would know the program wasn't actually copying properly. The above
is a 20GB file. Checking for "choking" during copy.

.\ghettocopy64 test.bin out.bin

file_size = 21474836480
Prepare buffer...
Starting run...
Read  21474836480 bytes in 009.604756 seconds
Wrote 21474836480 bytes in 007.306034 seconds
File copied successfully.

The problem Mickey is seeing, could quite well be
VSS, if... there are enough shadows in this command.

   vssadmin list shadows

There is the "slow COW (Copy On Write)" effect with
VSS, but more likely if a lot of shadows are stacked
on a partition.

In my tests with my copy program, I didn't see much in
the way of Windows Defender interference.

   [Picture]

    https://i.postimg.cc/Njt32WsQ/copy-test-single-file-W11.gif

   Paul

[toc] | [prev] | [next] | [standalone]

#17798

On Sun, 3/16/2025 9:49 PM, micky wrote:
> In alt.comp.os.windows-11, on Sun, 16 Mar 2025 17:02:37 -0400, micky
> <NONONOmisc07@fmguy.com> wrote:
> 
>> So for the first time, I tried copying and moving files within win11. It
>> took about 5 minutes to copy one file, and about 20 minutes to move 10
>> directories.  This is, how do they say it?, Unacceptable.
>>
>> The bussiest task was MS Shadow Copy service.  None of these files were
>> in use.  They were just sitting in the dat or downloads directories. I'd
>> just turn off the service -- should I? -- but it might well come in
>> handy if it means I could back up files from Forte Agent while it was
>> running. 
>>
>> I wasn't using File Explorer, but PowerDesk, a 3d-party, but I assume
>> it just formats the same commands tht File Explorer does.
> 
> a) I thought I should still test it with File Explorer and the problem
> is the same, as I was sure it would be. 
> 
> Using FE, I moved one file and it took 4 minutes.  As the other times,
> the graph immediately goes 99% of the way across and then stops.   A
> line in the box says it's "Calculating".  
> 
> b) I thought I should be able to reverse the move with Ctrl-Z, but it
> did nothing.  Ctrl-Z used to reverse moves.  Has that been stopped or is
> this another problem? 
> 
> c) I decided the file was expendable so I deleted it and the lnk, and
> each time it took 50 seconds to send it to the recycle bin.  This time I
> started from Everything, but I'm sure it doesn't matter where the
> instruction is issued from. 
> 
>> Does this "feature" bite everyone who gets win11? 

It looks like my Win10 OS has set a shadow on this machine.
Normally, I'm in Win11 and the machine name is WALLACE then.
H: is the OS drive for Win10.

# Administrator window of some sort...

vssadmin list shadows

vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2013 Microsoft Corp.

Contents of shadow copy set ID: {b4f5227e-cfef-41de-b82e-6276d469498b}
   Contained 1 shadow copies at creation time: Fri, 3/14/2025 4:05:48 AM
      Shadow Copy ID: {4213196c-852e-4d87-8138-5eacbf55641f}
         Original Volume: (H:)\\?\Volume{bd4d69bc-6fbb-4f12-b804-f32c6b9e828c}\
         Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1
         Originating Machine: TEN
         Service Machine: TEN
         Provider: 'Microsoft Software Shadow Copy provider 1.0'
         Type: ClientAccessibleWriters
         Attributes: Persistent, Client-accessible, No auto release, Differential, Auto recovered

WALLACE has Macrium 7.3.6391 installed. And the System Protection is OFF.
TEN has a Macrium 7.2 installed. And the System Protection is ON.

That's what made the persistent shadow. Once I turned off System Protection
in Win10, the shadow was gone. Reboot into Win11, I'm now free of shadows.

There have been cases, that are pretty close to the 64 shadow limit.
This is caused by a certain backup product that drops shadows like
crazy, and presumably they are the persistent ones.

I don't think I've ever had more than about three shadows.

The shadows on WinXP were not persistent. Persistence is available
on the later OSes. In addition, on Windows Servers, apparently
a shadow (the storage part) can be put on another disk. The logic of
which, escapes me. Doing that can't be very fast.

   Paul

[toc] | [prev] | [next] | [standalone]

#17979

In alt.comp.os.windows-11, on Mon, 17 Mar 2025 01:07:04 -0400, Paul
<nospam@needed.invalid> wrote:

>On Sun, 3/16/2025 9:49 PM, micky wrote:
>> In alt.comp.os.windows-11, on Sun, 16 Mar 2025 17:02:37 -0400, micky
>> <NONONOmisc07@fmguy.com> wrote:

Sorry I hven't gotten bck to this in 10 days.  Something else took my
time. 
 
>>> So for the first time, I tried copying and moving files within win11. It
>>> took about 5 minutes to copy one file, and about 20 minutes to move 10
>>> directories. 

None of which were very big.  150 files totaling under a gig in the 10
directories.   Went no faster using File Explorer than with PowerDesk. 

>>> This is, how do they say it?, Unacceptable.
>>>
>>> The busiest task was MS Shadow Copy service.  None of these files were
>>> in use.  They were just sitting in the data or downloads directories. I'd
>>> just turn off the service -- should I? -- but it might well come in
>>> handy if it means I could back up files from Forte Agent while it was
>>> running. 
>>>
>>> I wasn't using File Explorer, but PowerDesk, a 3d-party, but I assume
>>> it just formats the same commands tht File Explorer does.
>> 
>> a) I thought I should still test it with File Explorer and the problem
>> is the same, as I was sure it would be. 

Oh, I did say already that the problem was the same. 
 
>> Using FE, I moved one file and it took 4 minutes.  As the other times,
>> the graph immediately goes 99% of the way across and then stops.   A
>> line in the box says it's "Calculating".  
>> 
>> b) I thought I should be able to reverse the move with Ctrl-Z, but it
>> did nothing.  Ctrl-Z used to reverse moves.  Has that been stopped or is
>> this another problem? 
>> 
>> c) I decided the file was expendable so I deleted it and the lnk, and
>> each time it took 50 seconds to send it to the recycle bin.  This time I
>> started from Everything, but I'm sure it doesn't matter where the
>> instruction is issued from. 
>> 
>>> Does this "feature" bite everyone who gets win11? 
>
>It looks like my Win10 OS has set a shadow on this machine.
>Normally, I'm in Win11 and the machine name is WALLACE then.
>H: is the OS drive for Win10.
>
># Administrator window of some sort...
>
>vssadmin list shadows

Thanks for this command.

>vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
>(C) Copyright 2001-2013 Microsoft Corp.
>
>Contents of shadow copy set ID: {b4f5227e-cfef-41de-b82e-6276d469498b}
>   Contained 1 shadow copies at creation time: Fri, 3/14/2025 4:05:48 AM
>      Shadow Copy ID: {4213196c-852e-4d87-8138-5eacbf55641f}
>         Original Volume: (H:)\\?\Volume{bd4d69bc-6fbb-4f12-b804-f32c6b9e828c}\
>         Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1
>         Originating Machine: TEN
>         Service Machine: TEN
>         Provider: 'Microsoft Software Shadow Copy provider 1.0'
>         Type: ClientAccessibleWriters
>         Attributes: Persistent, Client-accessible, No auto release, Differential, Auto recovered

If these 10 lines represent a shadow, then I have 6 of them, made
between March 6 and March 24.  I don't understand if any are still
active or if they're just sitting there.   

For example: 
Contents of shadow copy set ID: {249dc25e-d207-44a0-ad7c-bd0353d63c26}
   Contained 1 shadow copies at creation time: 3/6/2025 11:07:32 PM
      Shadow Copy ID: {cf8d690a-bdf7-4983-9ed6-a3aeeb3f8534}
         Original Volume:
(C:)\\?\Volume{0a40503f-33d8-4db3-b218-34e0167033de}\
         Shadow Copy Volume:
\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy2
         Originating Machine: DESKTOP-VJCM7SG
         Service Machine: DESKTOP-VJCM7SG
         Provider: 'Microsoft Software Shadow Copy provider 1.0'
         Type: ClientAccessibleWriters
         Attributes: Persistent, Client-accessible, No auto release,
Differential, Auto recovered

All entries are similar with the same last line:   Attributes:
Persistent, Client-accessible, No auto release, Differential, Auto
recovered.   

Does Differential refer to a differential backup by Macrium Reflect? 

I should go back a bit and say that during the original problem I had 5
occurrences over 3 sessions of Windows, when copying or deleting took an
exceptionally long time, and if you recall, Shadow Copy was using a lot
of the CPU then, even though I believe was not running any kind of a
backup then. but that today, the problem is gone.  Copies are almost
instantaneous.    More details to follow in replies to other posts. 

>WALLACE has Macrium 7.3.6391 installed. And the System Protection is OFF.
>TEN has a Macrium 7.2 installed. And the System Protection is ON.
>
>That's what made the persistent shadow. Once I turned off System Protection
>in Win10, the shadow was gone. Reboot into Win11, I'm now free of shadows.

I only have win11 on this machine.  Should I turn off Real Time
Protection, or all of the protections, and Shutdown and then Reboot?
Will that make something stop running?  Or only erase these entries? 

It doesn't seem important if it's only to erase these entries, but maybe
that would save me a lot of space?  

Or maybe I shouldn't delete  these until the next time I run a full
backup??????    Not sure why a full backup has not run again already. 

>There have been cases, that are pretty close to the 64 shadow limit.
>This is caused by a certain backup product that drops shadows like
>crazy, and presumably they are the persistent ones.
>
>I don't think I've ever had more than about three shadows.

I've got six.  Nyah, nyah, nyah!
>
>The shadows on WinXP were not persistent. Persistence is available
>on the later OSes. In addition, on Windows Servers, apparently
>a shadow (the storage part) can be put on another disk. The logic of
>which, escapes me. Doing that can't be very fast.
>
>   Paul

[toc] | [prev] | [next] | [standalone]

#17982

On Thu, 3/27/2025 1:00 AM, micky wrote:

> I only have win11 on this machine.  Should I turn off Real Time
> Protection, or all of the protections, and Shutdown and then Reboot?
> Will that make something stop running?  Or only erase these entries? 
> 
> It doesn't seem important if it's only to erase these entries, but maybe
> that would save me a lot of space?  
> 
> Or maybe I shouldn't delete  these until the next time I run a full
> backup??????    Not sure why a full backup has not run again already. 

The command:

   vssadmin list shadows

tells us they exist, but it might not identify who "set" them.

What I was doing, was using my powers of guessing.

You might remember in WinXP, there were "Restore Points". If
you installed a driver, as long as System Protection was enabled,
the OS would make a "Restore Point" before the driver was installed.
You could roll back the Restore Point, to return a portion of the
file system, to its previous state.

If you turn off System Restore, in the System Protection setting

    [Picture]

     https://i.postimg.cc/28qnbHpG/restore-points-use-shadows.gif

... that has nothing to do with Windows Defender (Microsoft Defender)
and RealTime AV scanning. Windows Defender doesn't use shadows :-)
Thank goodness.

Your backup software can also use shadows. and you are right to identify
certain types of backup activity, as the source of some shadows.
I do "Fulls" and I don't think those leave too much cruft (the shadow
or snapshot can be removed after the backup is finished).
I would guess "Differential" and "Incremental" *might* use shadows.

Your shrewd guess that Macrium is involved, is probably it.
While it could be the System Protection in the picture above,
if you're not in need of the Restore Points, you can click
the Delete button, and the shadows that currently exist
should be removed, and then, if and when more Restore Points
get added, they would be created again. In WinXP days, an RP
was created every day. The W10/W11 approach is different, in
that "safety" RPs are created at a much lower frequency, and
there will be more of the "on-demand" type, where you're installing
software or are doing Windows Update, and a new RP is created when
that is starting.

For the average person, the "suspect list" is pretty short.
Restore Points might be a source of shadow activity.
And backup software might do it too. For some modes
of operation.

*******

I got to see something weird the other day. I was doing some
cleanup on the other machine. I figured "OK, let's do a defrag".
The defragment is running. I'm looking at the Optimize panel.
Normally, you see "Defrag" "Consolidate" "Defrag" "Consolidate"
and the Pass number increments.

It was up to around ten passes of that, and the disk drive LED
is flashing as you would expect for each phase. Then on the
eleventh pass, just "Consolidate" appears and the percentage
number starts to increment. I look over and *the disk LED is not flashing*.
It does another pass. It Consolidates (in theory, moves large
blocks of defragmented data to elsewhere on the disk). Yet,
again, the disk light will not flash! It must have done
around six passes of fake data movement, each time wasting
my time by pretending the percentage indicator was doing
real work.

Was that a shadow ? What was that ? I haven't a clue myself.
But it's the first time I've ever seen one like that.

   Paul

[toc] | [prev] | [next] | [standalone]

#17800

On 3/16/2025 5:02 PM, micky wrote:
> So for the first time, I tried copying and moving files within win11. It
> took about 5 minutes to copy one file, and about 20 minutes to move 10
> directories.  This is, how do they say it?, Unacceptable.
> 
> The bussiest task was MS Shadow Copy service.  None of these files were
> in use.  They were just sitting in the dat or downloads directories. I'd
> just turn off the service -- should I? -- but it might well come in
> handy if it means I could back up files from Forte Agent while it was
> running.
> 
> I wasn't using File Explorer, but PowerDesk, a 34d-party, but I assume
> it just formats the same commands tht File Explorer does.
> 
> Does this "feature" bite everyone who gets win11?
> 
> 
You didn't mention how big the file is. :)

   I've had Volume Shadow Copy disabled for as long as I
can remember. Disk image backup is much better than
system restore. But that doesn't explain the lag, unless
the file was gigantic. If you have SSDs and it takes a long
time to move files then something is wrong. (And it's not
something that will be solved by going back to command line
days with Robocopy. Explorer should handle it just fine.)

   I think Paul suggested the obvious first step: After getting
rid of PowerDesk, if it still doesn't work then figure out
what the heck is running that's holding things up. Maybe
3 antivirus programs? Maybe there's a hardware problem?
Something is wrong.

[toc] | [prev] | [next] | [standalone]

#17803

Newyana2 <newyana@invalid.nospam> wrote:
[...]

$DRIFT ON

>    I've had Volume Shadow Copy disabled for as long as I
> can remember. Disk image backup is much better than
> system restore. ...

  Volume Shadow Copy is not only used for System Restore, but also by
any decent backup program, including disk/partition backup programs.

  If a disk/partition backup program does not use Volume Shadow Copy to
make a snapshot of the disk/partitions, it can not do make an on-line
backup, nor do a verify after the backup.

$DRIFT OFF

[toc] | [prev] | [next] | [standalone]

#17807

On Mon, 3/17/2025 12:45 PM, Frank Slootweg wrote:
> Newyana2 <newyana@invalid.nospam> wrote:
> [...]
> 
> $DRIFT ON
> 
>>    I've had Volume Shadow Copy disabled for as long as I
>> can remember. Disk image backup is much better than
>> system restore. ...
> 
>   Volume Shadow Copy is not only used for System Restore, but also by
> any decent backup program, including disk/partition backup programs.
> 
>   If a disk/partition backup program does not use Volume Shadow Copy to
> make a snapshot of the disk/partitions, it can not do make an on-line
> backup, nor do a verify after the backup.
> 
> $DRIFT OFF
> 

A backup cannot back up open file handles, without the shadow service.
(Most) open file handles, will be handled properly using VSS. Only the file
handles that won't quiesce will not be handled. Whether you have
VSS or not, pagefile.sys cannot be backed up (well, as you would expect --
nobody really wants their pagefile.sys backed up!).

For some reason, the SearchIndexer file Windows.edb (or on Windows 11
the Windows.db file), do not get backed up. Not only is SearchIndexer
a persistent pest process in Task Manager, even the file it uses
is a pest (cannot be backed up). Since the file is regenerated, any
time it goes missing, it's not a big deal, it's a moderate deal
(your cloned disk might not search as fast in Windows Explorer
as it did previously... until the index is rebuilt).

   Paul

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | alt.comp.os.windows-11

csiph-web