Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.forth > #14790 > unrolled thread

Implementing virtual memory on cassette tape

Back to article view | Back to comp.lang.forth

Contents

  Implementing virtual memory on cassette tape chitselb <chitselb@gmail.com> - 2012-08-07 06:21 -0700
    Re: Implementing virtual memory on cassette tape Andrew Haley <andrew29@littlepinkcloud.invalid> - 2012-08-07 08:44 -0500
    Re: Implementing virtual memory on cassette tape anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2012-08-07 14:01 +0000
    Re: Implementing virtual memory on cassette tape Mark Wills <markrobertwills@yahoo.co.uk> - 2012-08-07 07:24 -0700
    Re: Implementing virtual memory on cassette tape Stan Barr <plan.b@dsl.pipex.com> - 2012-08-07 15:30 +0000
      Re: Implementing virtual memory on cassette tape Stan Barr <plan.b@dsl.pipex.com> - 2012-08-07 17:36 +0000
    Re: Implementing virtual memory on cassette tape Jason Damisch <jasondamisch@yahoo.com> - 2012-08-07 11:52 -0700
      Re: Implementing virtual memory on cassette tape Paul Rubin <no.email@nospam.invalid> - 2012-08-07 12:39 -0700
        Re: Implementing virtual memory on cassette tape Paul Rubin <no.email@nospam.invalid> - 2012-08-07 12:55 -0700
          Re: Implementing virtual memory on cassette tape Bernd Paysan <bernd.paysan@gmx.de> - 2012-08-07 22:00 +0200
            Re: Implementing virtual memory on cassette tape Paul Rubin <no.email@nospam.invalid> - 2012-08-08 00:27 -0700
              Re: Implementing virtual memory on cassette tape Mark Wills <markrobertwills@yahoo.co.uk> - 2012-08-08 01:26 -0700
                Re: Implementing virtual memory on cassette tape Alex McDonald <blog@rivadpm.com> - 2012-08-08 02:31 -0700
                Re: Implementing virtual memory on cassette tape Paul Rubin <no.email@nospam.invalid> - 2012-08-08 02:46 -0700
              Re: Implementing virtual memory on cassette tape Alex McDonald <blog@rivadpm.com> - 2012-08-08 02:23 -0700
                Re: Implementing virtual memory on cassette tape anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2012-08-08 10:57 +0000
                  Re: Implementing virtual memory on cassette tape Alex McDonald <blog@rivadpm.com> - 2012-08-08 04:59 -0700
                    Re: Implementing virtual memory on cassette tape anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2012-08-08 12:24 +0000
                      Re: Implementing virtual memory on cassette tape Alex McDonald <blog@rivadpm.com> - 2012-08-08 11:10 -0700
                        Re: Implementing virtual memory on cassette tape Bernd Paysan <bernd.paysan@gmx.de> - 2012-08-09 00:13 +0200
                          Re: Implementing virtual memory on cassette tape Alex McDonald <blog@rivadpm.com> - 2012-08-08 16:05 -0700
                            Re: Implementing virtual memory on cassette tape Paul Rubin <no.email@nospam.invalid> - 2012-08-08 17:30 -0700
                            Re: Implementing virtual memory on cassette tape Bernd Paysan <bernd.paysan@gmx.de> - 2012-08-09 03:26 +0200
                              Re: Implementing virtual memory on cassette tape Alex McDonald <blog@rivadpm.com> - 2012-08-09 05:30 -0700
                                Re: Implementing virtual memory on cassette tape Bernd Paysan <bernd.paysan@gmx.de> - 2012-08-09 19:21 +0200
                                  Re: Implementing virtual memory on cassette tape Alex McDonald <blog@rivadpm.com> - 2012-08-09 13:30 -0700
                                    Re: Implementing virtual memory on cassette tape Bernd Paysan <bernd.paysan@gmx.de> - 2012-08-10 01:27 +0200
                          Re: Implementing virtual memory on cassette tape vandys@vsta.org - 2012-08-09 00:32 +0000
                            Re: Implementing virtual memory on cassette tape Bernd Paysan <bernd.paysan@gmx.de> - 2012-08-09 03:33 +0200
                        Re: Implementing virtual memory on cassette tape anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2012-08-09 06:00 +0000
                          Re: Implementing virtual memory on cassette tape Alex McDonald <blog@rivadpm.com> - 2012-08-09 05:26 -0700
                            Re: Implementing virtual memory on cassette tape anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2012-08-09 13:44 +0000
                              Re: Implementing virtual memory on cassette tape Alex McDonald <blog@rivadpm.com> - 2012-08-09 10:21 -0700
                                Re: Implementing virtual memory on cassette tape Bernd Paysan <bernd.paysan@gmx.de> - 2012-08-09 19:50 +0200
                                  Re: Implementing virtual memory on cassette tape Alex McDonald <blog@rivadpm.com> - 2012-08-09 12:32 -0700
                                    Re: Implementing virtual memory on cassette tape Bernd Paysan <bernd.paysan@gmx.de> - 2012-08-09 22:07 +0200
                                      Re: Implementing virtual memory on cassette tape Alex McDonald <blog@rivadpm.com> - 2012-08-09 13:58 -0700
                                    Re: Implementing virtual memory on cassette tape Paul Rubin <no.email@nospam.invalid> - 2012-08-09 17:36 -0700
                                      Re: Implementing virtual memory on cassette tape Alex McDonald <blog@rivadpm.com> - 2012-08-10 04:13 -0700
                                        Re: Implementing virtual memory on cassette tape Paul Rubin <no.email@nospam.invalid> - 2012-08-11 20:27 -0700
                                Re: Implementing virtual memory on cassette tape anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2012-08-10 15:57 +0000
                                  Re: Implementing virtual memory on cassette tape Alex McDonald <blog@rivadpm.com> - 2012-08-13 05:23 -0700
                                    Re: Implementing virtual memory on cassette tape anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2012-08-15 15:13 +0000
                                      Re: Implementing virtual memory on cassette tape Alex McDonald <blog@rivadpm.com> - 2012-08-15 11:57 -0700
        Re: Implementing virtual memory on cassette tape anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2012-08-08 07:08 +0000
          Re: Implementing virtual memory on cassette tape chitselb <chitselb@gmail.com> - 2012-08-08 06:25 -0700
        Re: Implementing virtual memory on cassette tape Mark Wills <markrobertwills@yahoo.co.uk> - 2012-08-08 01:23 -0700
        Re: Implementing virtual memory on cassette tape kenney@cix.compulink.co.uk - 2012-08-08 05:06 -0500
    Re: Implementing virtual memory on cassette tape Percy <percival.andrews@gmail.com> - 2012-08-08 21:11 -0700
      Re: Implementing virtual memory on cassette tape chitselb <chitselb@gmail.com> - 2012-08-08 21:30 -0700
        Re: Implementing virtual memory on cassette tape percival.andrews@gmail.com - 2012-08-08 23:50 -0700
          Re: Implementing virtual memory on cassette tape chitselb <chitselb@gmail.com> - 2012-08-09 03:54 -0700
            Re: Implementing virtual memory on cassette tape Paul Rubin <no.email@nospam.invalid> - 2012-08-09 09:07 -0700
              Re: Implementing virtual memory on cassette tape chitselb <chitselb@gmail.com> - 2012-08-09 12:20 -0700
    Re: Implementing virtual memory on cassette tape Mat <dambere@web.de> - 2012-08-10 13:41 -0700
      Re: Implementing virtual memory on cassette tape Coos Haak <chforth@hccnet.nl> - 2012-08-10 23:54 +0200
        Re: Implementing virtual memory on cassette tape dambere@web.de - 2012-08-10 15:41 -0700
          Re: Implementing virtual memory on cassette tape Coos Haak <chforth@hccnet.nl> - 2012-08-11 01:47 +0200
      Re: Implementing virtual memory on cassette tape Andrew Haley <andrew29@littlepinkcloud.invalid> - 2012-08-11 03:50 -0500
        Re: Implementing virtual memory on cassette tape anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2012-08-11 09:03 +0000
          Re: Implementing virtual memory on cassette tape Andrew Haley <andrew29@littlepinkcloud.invalid> - 2012-08-11 16:08 -0500
      Re: Implementing virtual memory on cassette tape "Elizabeth D. Rather" <erather@forth.com> - 2012-08-11 21:51 -1000

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

#14912

Alex McDonald <blog@rivadpm.com> writes:
>On Aug 9, 2:44=A0pm, an...@mips.complang.tuwien.ac.at (Anton Ertl)
>wrote:
>> Alex McDonald <b...@rivadpm.com> writes:
>> >On Aug 9, 7:00=3DA0am, an...@mips.complang.tuwien.ac.at (Anton Ertl)
>> >wrote:
>> >> Sounds like you swallowed some horror stories some people like to
>> >> spin. =3DA0Why should spin down exacerbate these problems?
>>
>> >Several reasons.
>>
>> >Rated start/stop cycles; 250 average on/off cycles per year at the
>> >expected population AFR of 0.55% (Seagate Cheetah 15.7, enterprise
>> >class drive).
>>
>> What does AFR have to do with the horror stories about corrupted data?
>
>AFR includes corrupted data.

It includes other failure modes, so this says nothing about spin-down
exacerbating disk corruption.

>> And anyone who uses "enterprise class" drives for backup has too much
>> money.
>
>Why? Since many operations value data integrity greater than the cost,
>this is an economic argument, not one of wealth causing stupidity.

In backup and also in RAIDs, we increase safety/reliability through
redundancy.  For a given amount of money, we get more
safety/reliability by using more cheap drives instead of fewer
expensive drives.

>And AFR includes corrupted data. I'm mystified; where did I say that
>corrupted data was the only issue?

You spun horror stories about data corruption as if it was the main
issue.  In my experience it's a minor issue.

>> >Drives vary; SATA drives at 5k RPM spin up faster than high RPM SAS
>> >drives at 15K, which may take minutes to stabilize at operating speed.
>> >During that time, the disk isn't usable, and I stand by my assertion
>> >that spin up wastes as much power as several minutes of full
>> >operation.
>>
>> Sure, if a drive takes several minutes to spin up, it will consume as
>> much power as several minutes of full operation.
>>
>> But who in his right mind uses an expensive and power-hungry high-RPM
>> drive that takes forever to spin up for a storage solution that
>> requires low power and fast spin-up? =A0Ok, a sales guy selling to a
>> clueless and rich customer will do it, but not because of technical
>> merit.
>
>I was giving an example of slow spin up to counterpoint the "10
>seconds and you're good to go" example you gave.

It's an irrelevant example, because nobody in his right mind will use
such drives for such a design.  The 10s example is an ordinary 7200rpm
drive.  If somebody wanted to use special drives for a spin-down
system and spin-up time is of any relevance, they will choose drives
that spin up at least as fast as the one I measured.

>To spin up a RAID group of say 14 drives on a shelf of disks will
>require that the drives are turned on serially in small groups.

No.  If the hardware cannot spin them up at the same time, one will
not choose such a large RAID group.  Conversely, if RAID groups of 14
disks are desired, the hardware should be designed so that the group
can be spun up at the same time.  For a system that contains 480 disk
drives, dimensioning the power supply such that it can spin up 14
drives at once should be no problem.

>> >I don't know where you got the idea that 480 tape drives was the
>> >equivalent to 480 disk drives, but it's not an assertion I made and
>> >certainly qualifies as insane.
>>
>> You claimed that lots of disks had to be spun up for bandwidth
>>
>> reasons, and you wrote:
>>
>> |It's the economics of competing with tape; big power supplies to
>> |support 480 disks packed in a single rack cost lots of money.
>>
>> which suggest that you think that a backup solution needs 480 disks
>> spun up for bandwidth reasons.
>
>No, that was the COPAN solution. (IIRC it was the smallest COPAN
>system you could buy.)

And you claimed that their solution was insufficient because they
could only spin 25% of the disks, and that that was insufficient
because it limits the bandwidth too much.

>> It's nonsense, because we are backing up to disks with a total of 10s
>> of TB, and it's workable, and if we wanted to back up to more disks,
>> we would just use more disks. =A0And the main bandwidth limit is, as you
>> write, getting the data off the main storage.
>
>That was my point. If you want off-server backup, then the bandwidth
>off the server is the issue.

For our servers, the bandwidth off the server disks is the limit most
of the time, because there is a lot of seeking during backups, and
also, the data is already compressed when it goes off the server.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2012: http://www.euroforth.org/ef12/

[toc] | [prev] | [next] | [standalone]

#14948

On Aug 10, 4:57 pm, an...@mips.complang.tuwien.ac.at (Anton Ertl)
wrote:
> Alex McDonald <b...@rivadpm.com> writes:
> >On Aug 9, 2:44=A0pm, an...@mips.complang.tuwien.ac.at (Anton Ertl)
> >wrote:
> >> Alex McDonald <b...@rivadpm.com> writes:
> >> >On Aug 9, 7:00=3DA0am, an...@mips.complang.tuwien.ac.at (Anton Ertl)
> >> >wrote:
> >> >> Sounds like you swallowed some horror stories some people like to
> >> >> spin. =3DA0Why should spin down exacerbate these problems?
>
> >> >Several reasons.
>
> >> >Rated start/stop cycles; 250 average on/off cycles per year at the
> >> >expected population AFR of 0.55% (Seagate Cheetah 15.7, enterprise
> >> >class drive).
>
> >> What does AFR have to do with the horror stories about corrupted data?
>
> >AFR includes corrupted data.
>
> It includes other failure modes, so this says nothing about spin-down
> exacerbating disk corruption.
>
> >> And anyone who uses "enterprise class" drives for backup has too much
> >> money.
>
> >Why? Since many operations value data integrity greater than the cost,
> >this is an economic argument, not one of wealth causing stupidity.
>
> In backup and also in RAIDs, we increase safety/reliability through
> redundancy.  For a given amount of money, we get more
> safety/reliability by using more cheap drives instead of fewer
> expensive drives.
>
> >And AFR includes corrupted data. I'm mystified; where did I say that
> >corrupted data was the only issue?
>
> You spun horror stories about data corruption as if it was the main
> issue.  In my experience it's a minor issue.
>
>
>
>
>
>
>
>
>
> >> >Drives vary; SATA drives at 5k RPM spin up faster than high RPM SAS
> >> >drives at 15K, which may take minutes to stabilize at operating speed.
> >> >During that time, the disk isn't usable, and I stand by my assertion
> >> >that spin up wastes as much power as several minutes of full
> >> >operation.
>
> >> Sure, if a drive takes several minutes to spin up, it will consume as
> >> much power as several minutes of full operation.
>
> >> But who in his right mind uses an expensive and power-hungry high-RPM
> >> drive that takes forever to spin up for a storage solution that
> >> requires low power and fast spin-up? =A0Ok, a sales guy selling to a
> >> clueless and rich customer will do it, but not because of technical
> >> merit.
>
> >I was giving an example of slow spin up to counterpoint the "10
> >seconds and you're good to go" example you gave.
>
> It's an irrelevant example, because nobody in his right mind will use
> such drives for such a design.  The 10s example is an ordinary 7200rpm
> drive.  If somebody wanted to use special drives for a spin-down
> system and spin-up time is of any relevance, they will choose drives
> that spin up at least as fast as the one I measured.
>
> >To spin up a RAID group of say 14 drives on a shelf of disks will
> >require that the drives are turned on serially in small groups.
>
> No.  If the hardware cannot spin them up at the same time, one will
> not choose such a large RAID group.  Conversely, if RAID groups of 14
> disks are desired, the hardware should be designed so that the group
> can be spun up at the same time.  For a system that contains 480 disk
> drives, dimensioning the power supply such that it can spin up 14
> drives at once should be no problem.

The power supplies are relatively small, and serve a disk shelf that
is (commonly) organised in groups that can be fitted in a 19inch rack
system. 14 drives or more is a common number, although very dense 48
drive systems are also available. In total, the loading on a rack of
such shelves (which may be in the mid hundreds to more of drives)
cannot exceed certain limits in terms of amperage due to the heat
generated; 15KW or more heat from a rack is difficult to dissipate.
Scaling power supplies on a shelf to support 14 drives power-up
simultaneously means that most of the time supplies are operating at
low loads, which is where power supplies are very inefficient; running
them near their maximum rating is preferable, when conversion rates
can be 90% or better.

RAID group size is (relatively) small for RAID-5 type schemes;
normally no more than 6+1 parity or so. Dual parity schemes may employ
12+2 up to around 16+2. Much higher than these limits, and the RAID
rebuild times become prohibitively expensive and riskier due to
failures during rebuild; much lower, and the total space efficiency
and performance due to loss of parallelism is compromised.

>
>
>
>
>
>
>
>
>
> >> >I don't know where you got the idea that 480 tape drives was the
> >> >equivalent to 480 disk drives, but it's not an assertion I made and
> >> >certainly qualifies as insane.
>
> >> You claimed that lots of disks had to be spun up for bandwidth
>
> >> reasons, and you wrote:
>
> >> |It's the economics of competing with tape; big power supplies to
> >> |support 480 disks packed in a single rack cost lots of money.
>
> >> which suggest that you think that a backup solution needs 480 disks
> >> spun up for bandwidth reasons.
>
> >No, that was the COPAN solution. (IIRC it was the smallest COPAN
> >system you could buy.)
>
> And you claimed that their solution was insufficient because they
> could only spin 25% of the disks, and that that was insufficient
> because it limits the bandwidth too much.

It may do. Bandwidth is a problem during massively parallel backups
and due to the design of the shelves. Many systems employ a bus into
which the disks are plugged; disks are addressed via two or more fibre
channel arbitrated loops or a multi-path SAS arrangement (even for
SATA disks). Getting parallelism on such a system requires many
shelves to be active, and the RAID groups are sometimes split across
them, since a single shelf doesn't have max-bandwidth = (disk
bandwidth * number of disks). Hence why 25% of the shelves powered on
in the COPAN system limited bandwidth. (Some system employ "active"
servers supporting the 14+ disks that make up a shelf and can drive
higher sequential (but not random) bandwidth rates, but they are very
power hungry indeed.)

>
> >> It's nonsense, because we are backing up to disks with a total of 10s
> >> of TB, and it's workable, and if we wanted to back up to more disks,
> >> we would just use more disks. =A0And the main bandwidth limit is, as you
> >> write, getting the data off the main storage.
>
> >That was my point. If you want off-server backup, then the bandwidth
> >off the server is the issue.
>
> For our servers, the bandwidth off the server disks is the limit most
> of the time, because there is a lot of seeking during backups, and
> also, the data is already compressed when it goes off the server.

A smart backup program can reduce seeks by sorting, say, a snapshot of
th disk, to reduce the seeks and read blocks serially. Enterprise
class disks often support "skip read" semantics that can reduce the
requirement to seek when reading data from a single track. The order
in which the blocks are read & sent is immaterial to the construction
of a backup on the target.

>
> - anton
> --
> M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
> comp.lang.forth FAQs:http://www.complang.tuwien.ac.at/forth/faq/toc.html
>      New standard:http://www.forth200x.org/forth200x.html
>    EuroForth 2012:http://www.euroforth.org/ef12/

[toc] | [prev] | [next] | [standalone]

#14969

Alex McDonald <blog@rivadpm.com> writes:
>On Aug 10, 4:57=A0pm, an...@mips.complang.tuwien.ac.at (Anton Ertl)
>wrote:
>> Alex McDonald <b...@rivadpm.com> writes:
>> >On Aug 9, 2:44=3DA0pm, an...@mips.complang.tuwien.ac.at (Anton Ertl)
>> >wrote:
>> >> Alex McDonald <b...@rivadpm.com> writes:
>> >> >On Aug 9, 7:00=3D3DA0am, an...@mips.complang.tuwien.ac.at (Anton Ertl=
>)
>> >> >wrote:


>> >To spin up a RAID group of say 14 drives on a shelf of disks will
>> >require that the drives are turned on serially in small groups.
>>
>> No. =A0If the hardware cannot spin them up at the same time, one will
>> not choose such a large RAID group. =A0Conversely, if RAID groups of 14
>> disks are desired, the hardware should be designed so that the group
>> can be spun up at the same time. =A0For a system that contains 480 disk
>> drives, dimensioning the power supply such that it can spin up 14
>> drives at once should be no problem.
>
>The power supplies are relatively small, and serve a disk shelf that
>is (commonly) organised in groups that can be fitted in a 19inch rack
>system. 14 drives or more is a common number, although very dense 48
>drive systems are also available. In total, the loading on a rack of
>such shelves (which may be in the mid hundreds to more of drives)
>cannot exceed certain limits in terms of amperage due to the heat
>generated; 15KW or more heat from a rack is difficult to dissipate.

If I have several power supplies, each powering a bunch of drives, I
would distribute the RAID group across these bunches, ideally one
drive per bunch.  A nice side benefit is that the system can now
survive a power supply failure without needing any additional power
supply redundancy (not sure if the following rebuilding of lots of
RAID groups on power supply failure is practical, though, but if it
isn't, then we'll just have to bite the bullet and provide power
supply redundancy after all).  So, to spin up a whole RAID group at
the same time, each power supply only needs to be able to support
spinning up one drive.

15KW would allow spinning up 480 drives at the same time (and would
also be necessary to let 480 LTO-5 tape drives work at the same time).

>Scaling power supplies on a shelf to support 14 drives power-up
>simultaneously means that most of the time supplies are operating at
>low loads, which is where power supplies are very inefficient; running
>them near their maximum rating is preferable, when conversion rates
>can be 90% or better.

Typical power supplies are relatively efficient across a pretty wide
range, and the highest efficiency is not at maximum load.  E.g.,
looking at
http://www.anandtech.com/show/6013/350450w-roundup-11-cheap-psus/3,
i.e., even looking at a cheap power supply, there is relatively little
efficiency variation between 20% and 110% load, and the highest
efficiency is at 50% load.  I also looked at the next one in the test
(FSP OEM 400W) and find the same pattern there.

>> For our servers, the bandwidth off the server disks is the limit most
>> of the time, because there is a lot of seeking during backups, and
>> also, the data is already compressed when it goes off the server.
>
>A smart backup program can reduce seeks by sorting, say, a snapshot of
>th disk, to reduce the seeks and read blocks serially. Enterprise
>class disks often support "skip read" semantics that can reduce the
>requirement to seek when reading data from a single track.

Any commodity drive I or my students have measured in the last 15
years or so has cached the data of several tracks for reading (I guess
but have not confirmed that in particular they cache data that they
read while waiting for the disk to rotate to the target sector, but if
there was no request right afterward, probably also the rest of the
track), and that's why some OS-side optimizations we (and others) did
were not as effective as I expected: the drives already did part of
them for us.  Anyway, I had not heard that this is a marketing feature
for enterprise drives, and Google has not heard about "skip read
semantics", either.

Concerning our backup program, we are just using tar instead of a
smart one.  It's good enough for our needs, but it does not optimize
disk reads the way you suggest (it would need to more about file
systems than I find comfortable to do that), so there's quite a bit of
waiting for disk seeks despite drive caches.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2012: http://www.euroforth.org/ef12/

[toc] | [prev] | [next] | [standalone]

#14974

On Aug 15, 4:13 pm, an...@mips.complang.tuwien.ac.at (Anton Ertl)
wrote:
> Alex McDonald <b...@rivadpm.com> writes:
> >On Aug 10, 4:57=A0pm, an...@mips.complang.tuwien.ac.at (Anton Ertl)
> >wrote:
> >> Alex McDonald <b...@rivadpm.com> writes:
> >> >On Aug 9, 2:44=3DA0pm, an...@mips.complang.tuwien.ac.at (Anton Ertl)
> >> >wrote:
> >> >> Alex McDonald <b...@rivadpm.com> writes:
> >> >> >On Aug 9, 7:00=3D3DA0am, an...@mips.complang.tuwien.ac.at (Anton Ertl=
> >)
> >> >> >wrote:
> >> >To spin up a RAID group of say 14 drives on a shelf of disks will
> >> >require that the drives are turned on serially in small groups.
>
> >> No. =A0If the hardware cannot spin them up at the same time, one will
> >> not choose such a large RAID group. =A0Conversely, if RAID groups of 14
> >> disks are desired, the hardware should be designed so that the group
> >> can be spun up at the same time. =A0For a system that contains 480 disk
> >> drives, dimensioning the power supply such that it can spin up 14
> >> drives at once should be no problem.
>
> >The power supplies are relatively small, and serve a disk shelf that
> >is (commonly) organised in groups that can be fitted in a 19inch rack
> >system. 14 drives or more is a common number, although very dense 48
> >drive systems are also available. In total, the loading on a rack of
> >such shelves (which may be in the mid hundreds to more of drives)
> >cannot exceed certain limits in terms of amperage due to the heat
> >generated; 15KW or more heat from a rack is difficult to dissipate.
>
> If I have several power, each powering a bunch of drives, I
> would distribute the RAID group across these bunches, ideally one
> drive per bunch.

As a practice, that leads to issues with failure modes at the shelf
level. For instance, a failure of a single shelf with something as
simple as a tripped power supply, where the disks in that shelf
contribute to (say) 10 RAID groups may cause 10 simultaneous RAID
rebuilds requiring the involvement of several hundred drives. I have
seen this happen on an HP EVA, where their vdisk RAID supports such a
scheme (although it was not recommended); the resulting mess is not
pretty. It also requires a very large number of spare drives for such
a rebuild.

> A nice side benefit is that the system can now
> survive a power supply failure without needing any additional power
> supply redundancy (not sure if the following rebuilding of lots of
> RAID groups on power supply failure is practical, though, but if it
> isn't, then we'll just have to bite the bullet and provide power
> supply redundancy after all).  So, to spin up a whole RAID group at
> the same time, each power supply only needs to be able to support
> spinning up one drive.
>
> 15KW would allow spinning up 480 drives at the same time (and would
> also be necessary to let 480 LTO-5 tape drives work at the same time).
>
> >Scaling power supplies on a shelf to support 14 drives power-up
> >simultaneously means that most of the time supplies are operating at
> >low loads, which is where power supplies are very inefficient; running
> >them near their maximum rating is preferable, when conversion rates
> >can be 90% or better.
>
> Typical power supplies are relatively efficient across a pretty wide
> range, and the highest efficiency is not at maximum load.  E.g.,
> looking athttp://www.anandtech.com/show/6013/350450w-roundup-11-cheap-psus/3,
> i.e., even looking at a cheap power supply, there is relatively little
> efficiency variation between 20% and 110% load, and the highest
> efficiency is at 50% load.  I also looked at the next one in the test
> (FSP OEM 400W) and find the same pattern there.

Here's an analysis of power efficiencies in a data center you might
find interesting. It confirms your measurements.
http://www.thegreengrid.org/~/media/WhitePapers/White_Paper_16_-_Quantitative_Efficiency_Analysis_30DEC08.pdf?lang=en.

Take a 14 disk system with 2 power supplies. For one power supply to
support all 14 drives plus spinning up 1 requires around 17 disks
worth of power (approx; e.g. 20W spin-up as opposed to 7W in use for a
single drive) from a single PSU running at 100%. Two running in steady
state has 1 supporting 7 drives, and the PSUs are now running at
approx 45% load or less. For a single power supply to support 14 spin-
ups simultaneously would have a pair of PSUs running nearer the 20%
mark, where they are less efficient.

>
> >> For our servers, the bandwidth off the server disks is the limit most
> >> of the time, because there is a lot of seeking during backups, and
> >> also, the data is already compressed when it goes off the server.
>
> >A smart backup program can reduce seeks by sorting, say, a snapshot of
> >th disk, to reduce the seeks and read blocks serially. Enterprise
> >class disks often support "skip read" semantics that can reduce the
> >requirement to seek when reading data from a single track.
>
> Any commodity drive I or my students have measured in the last 15
> years or so has cached the data of several tracks for reading (I guess
> but have not confirmed that in particular they cache data that they
> read while waiting for the disk to rotate to the target sector, but if
> there was no request right afterward, probably also the rest of the
> track), and that's why some OS-side optimizations we (and others) did
> were not as effective as I expected: the drives already did part of
> them for us.  Anyway, I had not heard that this is a marketing feature
> for enterprise drives, and Google has not heard about "skip read
> semantics", either.

See http://ps-2.kev009.com/rs6000/manuals/SAN/ESS/2105_Model_ExxFxx/ESS_SCSI_Command_Reference_ExxFxx_SC26-7297-01.PDF
for an example (page 66) and http://www.ibmsystemsmag.com/getattachment/d5e03906-aa31-40c2-88f6-adce31c776ab/
for a diagram of 1 command with skip vs 2 commands with no skip.

This kind of feature is only available on enterprise class drives.

>
> Concerning our backup program, we are just using tar instead of a
> smart one.  It's good enough for our needs, but it does not optimize
> disk reads the way you suggest (it would need to more about file
> systems than I find comfortable to do that), so there's quite a bit of
> waiting for disk seeks despite drive caches.
>
> - anton
> --
> M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
> comp.lang.forth FAQs:http://www.complang.tuwien.ac.at/forth/faq/toc.html
>      New standard:http://www.forth200x.org/forth200x.html
>    EuroForth 2012:http://www.euroforth.org/ef12/

[toc] | [prev] | [next] | [standalone]

#14820

Paul Rubin <no.email@nospam.invalid> writes:
>The PET and microcomputer cassette interfaces were before my time but
>looking at
>
>  http://en.wikipedia.org/wiki/Datassette
>
>was interesting.  It was apparently a standard cassette recorder with
>some adc/dac's and a special edge connector.  In particular the article
>doesn't say whether the computer could start and stop the tape transport
>or rewind the tape under software control.  Do you know if that was
>possible?  

IIRC it was not.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2012: http://www.euroforth.org/ef12/

[toc] | [prev] | [next] | [standalone]

#14847

> 
> >  http://en.wikipedia.org/wiki/Datassette
> 
> >
> 
> >was interesting.  It was apparently a standard cassette recorder with
> 
> >some adc/dac's and a special edge connector.  In particular the article
> 
> >doesn't say whether the computer could start and stop the tape transport
> 
> >or rewind the tape under software control.  Do you know if that was
> 
> >possible?  
> 
> 
> 
> IIRC it was not.
> 
> 
> 
> - anton


The user has to push the buttons on the drive. The PET can start/stop either tape motor under program control. 

[toc] | [prev] | [next] | [standalone]

#14827

On Aug 7, 8:39 pm, Paul Rubin <no.em...@nospam.invalid> wrote:
> Jason Damisch <jasondami...@yahoo.com> writes:
> > You can assume that for the sake of the simulation, that you didn't
> > get your computer from K-Mart or Toys-R-Us and didn't have enough to
> > pay for a disk drive so went with the tape drive. LOL
>
> The PET and microcomputer cassette interfaces were before my time but
> looking at
>
>  http://en.wikipedia.org/wiki/Datassette
>
> was interesting.  It was apparently a standard cassette recorder with
> some adc/dac's and a special edge connector.  In particular the article
> doesn't say whether the computer could start and stop the tape transport
> or rewind the tape under software control.  Do you know if that was
> possible?
>
> If it wasn't, I doubt think the datasette was likely usable for much
> other than a program loader.  In particular, the usual things one did
> with magtape, such as merge sorting, required controlling the transport.
> Merge sorting involved reading blocks of data from 2 drives, merging
> them, and writing the output to a third drive.  That meant you had to
> write to the output drive twice as fast as you were reading from the
> individual input drives, so you had to control the drive speed or be
> able to start and stop the tape when you read a block.
>
> I did see some 9-track magtape drives back in the day and they had
> powerful motors and tape slack mechanisms to be able to start and stop
> the tape quickly.  I don't think audio recorders were built anything
> like that.

It could start and stop the tape, but nothing else. No forward and
rewing via software. That had to be done by a human. Loading from
cassette tape was a big part of my computing life as a young child. I
shudder when I think of it now!

[toc] | [prev] | [next] | [standalone]

#14835

In article <7xipcuija4.fsf@ruckus.brouhaha.com>, no.email@nospam.invalid 
(Paul Rubin) wrote:

>  Do you know if that was
> possible?  

 Don't know about the Pet but it was certainly possible to start stop 
the tape on my Video Genie. However the only file system supported was 
sequential access. Personally I would abandon the idea of using blocks 
on a tape system. Using a file system should make it possible to use 
routines in the Pet ROM to control the cassette.

 Ken Young

[toc] | [prev] | [next] | [standalone]

#14880

I'm not sure how determined you are to use original cassette hardware, but my PET (4032) had a robust 8-bit user-port that you could access with a suitable PCB edge-connector.  How would you feel about building a small microcontroller project to interface with the user-port and provide permanent storage?  

On the PET side you'd need to develop a very small set of routines in assembler that implement a basic command protocol for communictaion with the microcontroller.  You could make the command protocol closely aligned with the FORTH BLOCK word-set (i.e., load and save a block, etc.) and let the microcontroller do all the work of actually fetching and saving.   The microcontroller could use an SD-card for storing the data.  You would not need to implement a proper file system since it's all just blocks, just very basic commands for SD-card interface in SPI mode.  If you do something like that helpful resources are available from a very impressive website here:  http://elm-chan.org/docs/mmc/mmc_e.html

[toc] | [prev] | [next] | [standalone]

#14881

My reasons for using the PET's C2N cassette are:
1) It's the hardware most PET owners had
2) It's the hardware I have already
3) It feels like a cool problem to solve.  Reminds me of reading those Knuth books on tape sorting

Put another way: I am very unlikely in 2012 to go shopping for an 8050 dual floppy drive (or even a 4040) and some diskettes to put in it.

http://www.bitfixer.com/bf/PETdisk  <-- however, this is very appealing




On Thursday, August 9, 2012 12:11:23 AM UTC-4, Percy wrote:
> I'm not sure how determined you are to use original cassette hardware, but my PET (4032) had a robust 8-bit user-port that you could access with a suitable PCB edge-connector.  How would you feel about building a small microcontroller project to interface with the user-port and provide permanent storage?  
> 
> 
> 
> On the PET side you'd need to develop a very small set of routines in assembler that implement a basic command protocol for communictaion with the microcontroller.  You could make the command protocol closely aligned with the FORTH BLOCK word-set (i.e., load and save a block, etc.) and let the microcontroller do all the work of actually fetching and saving.   The microcontroller could use an SD-card for storing the data.  You would not need to implement a proper file system since it's all just blocks, just very basic commands for SD-card interface in SPI mode.  If you do something like that helpful resources are available from a very impressive website here:  http://elm-chan.org/docs/mmc/mmc_e.html

[toc] | [prev] | [next] | [standalone]

#14882

The bitfixer PETdisk looks great!  Thank you for sharing the link.   Maybe you can develop and test your FORTH using the PETdisk as a stop-gap storage solution to support you while you port the FORTH tape drive as phase 2?

[toc] | [prev] | [next] | [standalone]

#14890

On Thursday, August 9, 2012 2:50:59 AM UTC-4, (unknown) wrote:
> The bitfixer PETdisk looks great!  Thank you for sharing the link.   Maybe you can develop and test your FORTH using the PETdisk as a stop-gap storage solution to support you while you port the FORTH tape drive as phase 2?

I do most of the development and testing on my VAIO netbook which mostly runs Ubuntu 12.04 Precise.  One problem is there is currently only support for a single tape drive in the xpet emulator.  In a rare historical inversion, the viceteam.org xpet code came *after* the x64 and that's where the tape code came from.

chitselb@pakora:~$ dpkg -l vice xa65
||/ Name           Version        Description
+++-==============-==============-============================================
ii  vice           2.3.dfsg-2     Versatile Commodore Emulator
ii  xa65           2.3.5-1        cross-assembler and utility suite for 65xx/6

[toc] | [prev] | [next] | [standalone]

#14894

chitselb <chitselb@gmail.com> writes:
> One problem is there is currently only support for a single tape drive
> in the xpet emulator.  In a rare historical inversion, the
> viceteam.org xpet code came *after* the x64 and that's where the tape
> code came from.

Did anyone actually use multiple cassette drives with a PET?  Given what
I've seen about the datasette's transfer speed, running any of those
multi-tape algorithms sounds utterly unbearable.  Old-fashioned magtape
transferred fast enough to fill fill main memory in a few seconds on
computers of the era.  At 50 bytes/sec the datasette would take over 2
minutes to fill 8k on a PET.  Also, it sounds like the dataset didn't
have a way to switch from writing to reading or vice versa, without the
user having to push buttons on the drive.

I think you have to treat the datasette as more like a paper tape
reader/punch, than like a magtape-like device.

It looks like Commodore floppies are pretty cheap:

  http://www.ebay.com/itm/350581937903

Getting one is probably more in the retro spirit, than trying to do
contorted things with cassette drives that nobody ever(?) considered on
real systems of the day.

[toc] | [prev] | [next] | [standalone]

#14899

On Thursday, August 9, 2012 12:07:05 PM UTC-4, Paul Rubin wrote:
> chitselb  writes:
> 
> > One problem is there is currently only support for a single tape drive
> 
> > in the xpet emulator.  In a rare historical inversion, the
> 
> > viceteam.org xpet code came *after* the x64 and that's where the tape
> 
> > code came from.
> 
> 
> 
> Did anyone actually use multiple cassette drives with a PET?  Given what
> 
> I've seen about the datasette's transfer speed, running any of those
> 
> multi-tape algorithms sounds utterly unbearable.  Old-fashioned magtape
> 
> transferred fast enough to fill fill main memory in a few seconds on
> 
> computers of the era.  At 50 bytes/sec the datasette would take over 2
> 
> minutes to fill 8k on a PET.  Also, it sounds like the dataset didn't
> 
> have a way to switch from writing to reading or vice versa, without the
> 
> user having to push buttons on the drive.
> 
> 
> 
> I think you have to treat the datasette as more like a paper tape
> 
> reader/punch, than like a magtape-like device.
> 
> 
> 
> It looks like Commodore floppies are pretty cheap:
> 
> 
> 
>   http://www.ebay.com/itm/350581937903
> 
> 
> 
> Getting one is probably more in the retro spirit, than trying to do
> 
> contorted things with cassette drives that nobody ever(?) considered on
> 
> real systems of the day.

Yes, I have used two cassettes on the PET.  Circa 1981 I wrote a program in Microsoft BASIC to catalog my record (now we say "vinyls") collection.  There was a screen for each album with artist, title, and a track listing.  I had a few hundred albums which exceeded my 32K of RAM, so I resorted to datafiles on tape, and I used two cassette drives (not unlike the "Plan 1" from the original post on this thread).  Yes, it was slow.

Scanning through even a 30-minute tape to find the 10th program was slow too, so the first program on every cassette was a short menu program that would show a listing of the programs on the rest of the tape.  After selecting, say, #7, the user would be prompted to press fast-forward, then the tape motor would stop after 1517 jiffies (25.28 seconds).  The user would then press stop, play, and the tape would be cued up in the right position to load the 7th program.

The reason I was going for blocks is two-fold:
1) I want to make PETTIL (anachronistically) Forth-83 compliant.  BLOCK is in the required wordset
2) Forth is (to me) both a high-level and a bare-metal language that gives the programmer maximum control over low-resource hardware. 16-bit words are ideal within a 64K address space.  It has a tradition and a feel, and BLOCK-based virtual memory is a part of that.

A primary goal is to leverage and present as much of the power of the PET as possible to the programmer.  Particularly, I want to mine the ROMs for value.  Floating point math would use the BASIC floating point routines.  The screen editor will be the native 25x40 screen editor already familiar to the PET BASIC programmer, with the RVS/OFF as a meta key to invoke editor commands. The PET 2001 and 2001-N don't have a "Control" or "Alt" button, so I have to do something else to let the programmer escape out of the editor and replace control-meta-cokebottle type keysequences.

Another of my goals is to have PETTIL eventually become self-compiling, with the entire language written in itself.  The Klein-bottle nature of Forth was a huge part of its initial appeal to me when I discovered it in R.G. Loeliger's "Threaded Interpretive Languages" book in 1982.  I hadn't considered how this could be done in native source files, but I suppose it would work.

The way the design is evolving, I'll add a set of words to use the Kernal routines for OPEN, CLOSE, CMDIN and CMDOUT etc... using Ballantyne's excellent Blazin' Forth as a model, implement sequential files on top of that, and that should be good enough.  To get all the way to FORTH-83 compliance I would just implement BLOCK, FLUSH, etc... on top of that scheme.

[toc] | [prev] | [next] | [standalone]

#14914

Hello,
I don't understand why you burden your forth system (and possible
users) with block accessing on cassette storage. In my opinion it
would be both simplier and better to stream the whole memory back to
cassette at demand as load and save times would be slow anyway.

Implementing threading dispatch for 6502 class cpu's is a bad idea in
my opinion. It would be better to implement a simple native-code
compiler for these processor type.

But please fiish your project and show me I'm wrong with these two
cents.

chitselb schrieb:
> I'm working on a retro computing project, a 6502 Forth implementation for the Commodore PET 2001.  https://github.com/chitselb/pettil if you're curious.  The goal is for the language to be fast, tight, and capable of running on the actual hardware. For development I'm using the viceteam.org PET emulator with the xa65 cross-assembler, on Linux.
>
> Since most of us back then (1980) didn't have disk drives, I am going to use the cassette tape for mass storage.  These are a few ways I'm considering:
>
> 1) Simulate random access using two cassette decks and copy/merge
>
> The PET cassette had two file types, sequential(data) and program.
> a) For program files, there's a long tone followed by a short header block containing the filename, and then a shorter tone followed by one continuous block of memory (two byte load address followed by the data)
> b) for data files, there's the same long tone/file name header, followed by zero or more short tone/192-byte data blocks
>
> On the PET (not the VIC-20 or C=64) there were two datassette ports, and I have two drives.  Using the sequential file format and both decks, FLUSH would copy the entire virtual memory from one tape to the other in 1024-byte blocks (preceded by a 16-bit unsigned block number), inserting and replacing blocks from the memory buffers.  Then rewind both tapes and go the other way.  Slow, tedious, cumbersome.  Welcome to my world in 1980.
>
> 2) Historically accurate
>
> Some Forth implementations from back then implemented tape storage.  I have been unable to locate one for the PET but yesterday I found tape images for Datatronic Forth on the C=64 and another thing called "C=64 Forth". Both of these appear to implement some type of mass storage on tape.
>
> I'd be very interested to know what other Forth implementations of that era did as far as tape storage.  What Forth words, what did they do, etc...
>
> 3) Save source code as sequential files
>
> Using native named files instead of blocks.  Not very Forth-like, but possibly the most expedient.
>
> I'm very grateful for the help of this community with my earlier design considerations (circa 2010) on this project, particularly the hashed dictionary and the incredibly fast inner interpreter.  Check the project link above if you're curious to see how those parts turned out.
>
> Charlie

[toc] | [prev] | [next] | [standalone]

#14915

Op Fri, 10 Aug 2012 13:41:09 -0700 (PDT) schreef Mat:

> Hello,
> I don't understand why you burden your forth system (and possible
> users) with block accessing on cassette storage. In my opinion it
> would be both simplier and better to stream the whole memory back to
> cassette at demand as load and save times would be slow anyway.
> 
> Implementing threading dispatch for 6502 class cpu's is a bad idea in
> my opinion. It would be better to implement a simple native-code
> compiler for these processor type.
> 
> But please fiish your project and show me I'm wrong with these two
> cents.
> 
What if you have 100 blocks (100 KB) data, how would you load that many in
one goto into the memory of a 8 bit computer?

-- 
Coos

CHForth, 16 bit DOS applications
http://home.hccnet.nl/j.j.haak/forth.html 

[toc] | [prev] | [next] | [standalone]

#14916

Am Freitag, 10. August 2012 23:54:32 UTC+2 schrieb Coos Haak:
> > What if you have 100 blocks (100 KB) data, how would you load that many in
> 
> one goto into the memory of a 8 bit computer?

Typical PET's had 8 KB ram ! How would you processing 100 KB of data with such a platform ? You would process these data in chunks of some KB. This would also be possible with a forth system streaming it's state back to tape, because that do not pretend words managing to load and save processed data at demand. Instead of block accessing without motor control, resulting in at-hand position adjustments for each data block regardless of its use you would gain freedom to format a tape well suited for a specific task so all needed for processing would be to press the play and stop keys.

[toc] | [prev] | [next] | [standalone]

#14919

Op Fri, 10 Aug 2012 15:41:45 -0700 (PDT) schreef dambere@web.de:

> Am Freitag, 10. August 2012 23:54:32 UTC+2 schrieb Coos Haak:
>>> What if you have 100 blocks (100 KB) data, how would you load that many in
>> 
>> one goto into the memory of a 8 bit computer?
> 
> Typical PET's had 8 KB ram ! How would you processing 100 KB of data with such a platform ? You would process these data in chunks of some KB. This would also be possible with a forth system streaming it's state back to tape, because that do not pretend words managing to load and save processed data at demand. Instead of block accessing without motor control, resulting in at-hand position adjustments for each data block regardless of its use you would gain freedom to format a tape well suited for a specific task so all needed for processing would be to press the play and stop keys.

Of course, that's what I meant. Blocks are neat for this sort of work. I've
used a casette drive with C90 casettes in 1981, but not for long. My ZX
Spectrum had two microdrives that I could control from within my own Forth.
Much fast and simpler than pressing knobs with the previous meant computer.

-- 
Coos

CHForth, 16 bit DOS applications
http://home.hccnet.nl/j.j.haak/forth.html 

[toc] | [prev] | [next] | [standalone]

#14921

Mat <dambere@web.de> wrote:

> Implementing threading dispatch for 6502 class cpu's is a bad idea in
> my opinion. It would be better to implement a simple native-code
> compiler for these processor type.

I'm sure you're right about speed, but there isn't much memory, and
for that reason all the language implementations I came across at the
time used some some sort of interpretation, whether Forth or Pascal.
6502 Pascal was even more compact than Forth, using a very tight
bytecode.  Maybe a JSR-threaded Forth would be OK, but that's still a
considerable code expansion.

Andrew.

[toc] | [prev] | [next] | [standalone]

#14922

Andrew Haley <andrew29@littlepinkcloud.invalid> writes:
>Mat <dambere@web.de> wrote:
>
>> Implementing threading dispatch for 6502 class cpu's is a bad idea in
>> my opinion. It would be better to implement a simple native-code
>> compiler for these processor type.
>
>I'm sure you're right about speed,

A JSR-RTS pair is 12 cycles, and the OP mentioned 17 cycles for his
NEXT, so yes, subroutine threading would be a little faster.

> but there isn't much memory, and
>for that reason all the language implementations I came across at the
>time used some some sort of interpretation, whether Forth or Pascal.
>6502 Pascal was even more compact than Forth, using a very tight
>bytecode.  Maybe a JSR-threaded Forth would be OK, but that's still a
>considerable code expansion.

Going in the other direction, if the target machine has only 8KB,
there probably won't be more than 256 words anyway, so one could
represent words with bytes in interpreted code.  NEXT would probably
be a little slower, though.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2012: http://www.euroforth.org/ef12/

[toc] | [prev] | [next] | [standalone]

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

Back to top | Article view | comp.lang.forth

csiph-web