Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.sys.apple2.programmer > #2328 > unrolled thread

Streamlining Applesoft - The garbage collector

Back to article view | Back to comp.sys.apple2.programmer

Contents

  Streamlining Applesoft - The garbage collector gids.rs@sasktel.net - 2016-02-20 20:33 -0800
    Re: Streamlining Applesoft - The garbage collector Michael Pohoreski <michael.pohoreski@gmail.com> - 2016-02-22 13:58 -0800
      Re: Streamlining Applesoft - The garbage collector "Anton Treuenfels" <teamtempest@yahoo.com> - 2016-02-22 17:55 -0600
      Re: Streamlining Applesoft - The garbage collector gids.rs@sasktel.net - 2016-02-22 21:46 -0800
        Re: Streamlining Applesoft - The garbage collector Benoit0123 <bgilon@free.fr> - 2016-02-23 15:33 +0000
          Re: Streamlining Applesoft - The garbage collector gids.rs@sasktel.net - 2016-02-23 08:09 -0800
            Re: Streamlining Applesoft - The garbage collector Benoit0123 <bgilon@free.fr> - 2016-02-23 18:08 +0000
              Re: Streamlining Applesoft - The garbage collector Michael J. Mahon <mjmahon@aol.com> - 2016-02-23 12:52 -0600
                Re: Streamlining Applesoft - The garbage collector Michael Pohoreski <michael.pohoreski@gmail.com> - 2016-02-23 12:15 -0800
                  Re: Streamlining Applesoft - The garbage collector Michael J. Mahon <mjmahon@aol.com> - 2016-02-24 00:46 -0600
              Re: Streamlining Applesoft - The garbage collector gids.rs@sasktel.net - 2016-02-23 21:47 -0800
        Re: Streamlining Applesoft - The garbage collector Brian Patrie <bpatrie@bellsouth.spamisicky.net> - 2016-02-28 04:51 -0600
          Re: Streamlining Applesoft - The garbage collector D Finnigan <dog_cow@macgui.com> - 2016-02-29 23:09 +0000
            Re: Streamlining Applesoft - The garbage collector Michael Pohoreski <michael.pohoreski@gmail.com> - 2016-03-01 09:05 -0800
              Re: Streamlining Applesoft - The garbage collector D Finnigan <dog_cow@macgui.com> - 2016-03-01 18:04 +0000
                Re: Streamlining Applesoft - The garbage collector Michael Pohoreski <michael.pohoreski@gmail.com> - 2016-03-01 12:22 -0800
      Re: Streamlining Applesoft - The garbage collector Michael J. Mahon <mjmahon@aol.com> - 2016-02-23 12:40 -0600

#2328 — Streamlining Applesoft - The garbage collector

Another version of garbage collection?  Or, just a better one?

There are a couple of ways to go about this.  And I would like to implement both ways.

The first way is to create a fairly compact garbage collector that would not be much bigger than the applesoft version, but have a fairly substantial speed increase.

I believed understanding the applesoft version was key to keeping it compact and identifying the loops that cause the delays.

What I found was that the time was spent in searching for the variable that had the highest string in memory.  This was very time consuming to search through hundreds of variables just to find one, move its string to the top of string space, then go back through the variable table to find the next, and so on.

I resolved that if through each search of the variable table, one could collect a page worth of pointers to the highest strings, which would account for 128 strings, this would make the search 128 times faster.

Only a little overhead would be needed to sort these 128 strings.  And there are some very fast sorting algorythms out there.  But in keeping the code compact, I chose the insertion sort.  Very simple to implement, and very compact.

It works by taking an input, searching through a table (or adding an entry if none exists), comparing the location of the input variables' string to the ones already in the table, and inserting the input by moving the table up to make room for it.  The comparison is made until the input variables' string is higher than the one being compared-to in the table.  This way, the table is always sorted with the highest string being at the beginning of the table.

The table we will use is, 2 of the 4 page buffer above himem that is reserved for Prodos use.  But since we are not doing any file manipulation, it is safe to use.  And the Basic.system version of the Garbage collector uses it as well.

Now, with one pass through the variable tables, string variables are tested to see if they have already been collected, and if so they are skipped.  If not, they are compared with the sorted table and added to the table if there is room.  When the sorted table is full, the ptrs that scroll off the end of the table will be collected on the next round.  Only the highest 128 strings are contained in the sorted table.  Or, to be more precise, it is pointers to the variables that point to the highest strings.  Only 2 bytes per ptr is needed.

The strings for these 128 variables are then moved up to the start of string space (which started out as himem), and moves down as strings are added.  Each variable has its string pointer adjusted to the new location, and will be skipped on the next search.  Once the 128 variables have their strings collected, then we go back to the variable space and collect another 128.  This goes on until the last variable, which also has the address of the bottom of strings before collecting, are equal.  Then we are finished.

As previously stated, the sorted table requires 2 byte pointers and 2 byte ptrs always can be broken down into a hi-byte and lo-byte.  With very little overhead, we can implement a scheme where we can have hi-byte pointers in one page and lo-byte pointers in another.  This would give us and allow for 256 string pointers through each pass through the variable tables.  I apologize for mis-leading you to think we are only collecting 128 strings at a time, but such is the suspense of story telling.

The size and speed differences of the different garbage collector versions are:
Applesoft - $113 (275) bytes - collection time infinity (I have never sat through one, have you?)
This ver - $174 (372) bytes - collection time @1Mhz / 6 secs (256x faster than infinity)
Basic.system ver - $1DD (477) bytes - collection time @1 Mhz / 2 secs

I believe this to be a good trade-off between size and speed to other collectors that are out there.  And this version has two advantages over the Basic.system version.  It is 105 bytes smaller, and it does not require a 4 page reserved space area at the bottom of strings, so can use the entire free space for strings.

If you have understood all that so far, then all that is left is the code.  A demo applesoft program at the end of this post will show the speed of this garbage collector.

Sorry to mislead you again.  But I wanted to explain my second way of garbage collection that is even more advantagous than having a faster garbage collector.  In theory it should be very easy to implement and require very little code and almost no overhead.

First, a quick look into how applesoft stores strings and then concatenates them with a sample line.

]5 D$=CHR$(4)
]10 A$="Hi" + CHR$(32): B$="there" + CHR$(32) : C$=A$+B$

Each string is added to string space in order that it is encountered in an applesoft program.  Having a GOTO at the beginning of the program will skip all lines between LINE #1 and the GOTO'd line and start collecting strings at this GOTO line.

The length of the string that follows the D$= variable is subtracted from the "start of string space" ptr, then the string is copied into string space, and the pointers for the string are stored in the variable D$, which is stored in variable space.  And lastly, the "start of string space" pointer is updated with this same address and is an indicator for the bottom of strings.  D$'s pointer now points to $95FF with a length of 1

When saving A$ to string space, the CHR$(32) is given a temp posn first, with a length of 1 and the "start of string space" is updated.

A$ has a length of 2 plus the space = 3, which is subtracted from the new "start of string space" to give its storage address, and the "start of string space" is again updated to this new address.
But as you can see below, the space is duplicated, which is not necessary since that temporary string will never be used again.  we could have saved 1 byte by just adding the string literal "Hi" to the space.  A$ has a pointer of $95FB and a length of 3

B$ has a length of 5 plus the space and again, a temp string was created for the space, which gets duplicated when conatenation is performed.  The "start of string space" ptr is also updated each time as before.  The var B$ points to $95F4 and has a length of 6

C$ concatentas the two previous strings and instead of making use of either previous string, it makes room equivalent to the total length of the strings put together.  So room for another 9 bytes was inserted, then the text for those two strings are copied into that room.

All that needed to be done, was a check to see if the last string of the concatenation (in this case B$) was already at the "start of string space"  address, and if it was, then only enough room was needed for A$ (3 bytes) for a savings of 6 bytes.  As you can see, the memory savings can be quite signifcant and may even prevent garbage collection ever happening at all.

There are, however, some cases where this cannot be avoided, especially with multiple concatenations.

CALL-151
*95EB.95FF
85EB:48 69 20 74 68 - "Hi th"
95F0:65 72 65 20 74 68 65 72 65 20 20 48 69 20 20 04 - "ere there  Hi  ."


Hopefully, this adds more insight to how applesoft could be greatly improved as a language.

* On to the good stuff, the souce code
* skip to the bottom for a quick copy and paste demo and for
* the worst applesoft routine that creates garbage collection

* SHOVE ALL REFERENCED STRINGS AS HIGH AS POSSIBLE
* IN MEMORY (STARTING AT HIMEM), FREEING UP SPACE
* BELOW STRING AREA AND ABOVE ARRAY SPACE (KNOWN AS STRingEND)

* $E484...$E597 = $113 bytes
* PRODOS VERSION = $1DD bytes
* This version = $170 bytes

* This uses a search and insort routine for the garbage collector

BLTU2 EQU $D39A

 ORG $9000
 SHT

GARBAGE EQU *
L9000 LDX $74
 LDY $73
 STX $70
 STY $6f
 BEQ OK
 INX
 LDY #0
OK STY $87       ; Himem for lo-bytes
 STX $88
 STY $8D       ; Himem + $100 for hi-bytes
 INX
 STX $8E

***
* START BY COLLECTING TEMPORARIES
* max 3 temp strings
* created by STR$ or FN
* from $55.5D, $52 is the counter
***

FNDHISTR LDA #3
 STA $8C
 LDA #$55
 LDX #0
 STA $5E
 STX $5F
 STX $D9       ; counter for string ptr list
]LP CMP $52       ; FINISHED WITH TEMPS YET?
 BEQ DOSIMPL   ; YES, NOW DO SIMPLE VARIABLES
 JSR CHK.VAR   ; DO A TEMP
 BNE ]LP       ; always

***
* NOW COLLECT SIMPLE VARIABLES
***

DOSIMPL LDA $69       ; START AT BEGINNING OF VARTAB
 LDX $6A
 STA $5E
 STX $5F
]LP CPX $6C       ; FINISHED WITH SIMPLE VARIABLES?
 BCC HISTR4
 CMP $6B       ; MAYBE, CHECK LO-BYTE
 BCS DOARRY    ; YES, NOW DO ARRAYS
HISTR4 JSR CHKSMPLVAR
 BNE ]LP       ; always

ARRYRTN RTS

***
* NOW COLLECT ARRAY VARIABLES
***

DOARRY STA $94
 STX $95

CHKARYEND LDA $94
 CPX $6E       ; end of arrays
 BCC MOREARY
 CMP $6D
 BCS FIXSTR

MOREARY STA $5E       ; SET UP PNTR TO START OF ARRAY
 STX $5F

 LDY #3        ; DESCR IN ARRAYS ARE 3-BYTES
 STY $8C
 DEY
 LDA ($5E),Y   ; OFFSET TO NEXT ARRAY
 ADC $94       ; (CARRY always CLEAR)
 STA $94
 INY
 LDA ($5E),Y   ; HI-BYTE OF OFFSET
 ADC $95
 TAX
 STX $95

 LDY #1
 LDA ($5E),Y
 BPL CHKARYEND
 DEY
 LDA ($5E),Y
 BMI CHKARYEND

 LDY #4
 LDA ($5E),Y   ; # OF DIMENSIONS FOR THIS ARRAY
 ASL           ; PREAMBLE SIZE = 2 * # OF DIMS + 5
 ADC #5
 ADC $5E       ; MAKE $5E POINT AT FIRST ELEMENT
 STA $5E       ; IN THE ARRAY
 BCC *+4
 INC $5F

]LP JSR CHK.VAR   ; PROCESS THE ARRAY
 CPX $95       ; ARRAY DONE?
 BCC ]LP       ; NO, PROCESS NEXT ELEMENT
 CMP $94       ; MAYBE, CHECK LO-BYTE
 BCC ]LP       ; YES, MOVE TO NEXT ARRAY
 BEQ CHKARYEND ; finished one array, do the next

***
* Strings are sorted in himem, now fix them
* Move the strings to the start of string storage in order of the list
* The adjust the string variables to the new location
***
* $9B.9C enter with the address of the highest string
* $8A.8B hold the address of the string ptr

FIXSTR LDY #0
 STY $8F       ; COUNTER FOR STRING VARS
]LP LDA ($87),Y
 STA $8A
 LDA ($8D),Y
 STA $8B

 LDY #2
 LDA ($8A),Y
 STA $9C
 DEY
 LDA ($8A),Y
 STA $9B
 DEY
 LDA ($8A),Y   ; LENGTH OF STRING

 CLC
 ADC $9B
 STA $96       ; STRING IS BTWN ($9B) AND ($96)
 LDA $9C
 ADC #0
 STA $97
 LDA $6F       ; HIGH END DESTINATION
 LDX $70
 STA $94
 STX $95
 JSR BLTU2     ; MOVE STRING UP

 LDY #1        ; FIX ITS DESCRIPTOR
 LDA $94       ; STORE NEW ADDRESS
 STA ($8A),Y
 STA $6F
 INY
 INC $95       ; CORRECT BLTU"S OVERSHOOT
 LDA $95
 STA ($8A),Y
 STA $70
 INC $8F
 LDY $8F
 CPY $D9       ; MAX COUNT OF STRING VARS FOR THIS ROUND
 BCC ]LP
 BEQ ]LP       ; max for $D9 is $FF
 INY           ; IF NOT #$FF THEN DONE
 BNE *+5
 JMP FNDHISTR
 RTS         

***
* PROCESS A SIMPLE VARIABLE
***

* compare each variables' ptr with each ptr in himem to see if its string is the
*  highest.  If yes, then move the table up in memory.  Ptrs will then be in
*  descending order with the highest string ptr at the beginning of himem.

CHKSMPLVAR LDA #7
 STA $8C
 LDY #1
 LDA ($5E),Y   ; LOOK AT 2ND CHAR OF NAME
 BPL BUMP      ; NOT A STRING VARIABLE
 DEY
 LDA ($5E),Y   ; LOOK AT 1ST CHAR OF NAME
 BMI BUMP      ; NOT A STRING VARIABLE

 LDA $5E       ; CARRY IS ALREADY CLEAR
 ADC #2        ; GET PAST VARIABLE NAME
 STA $5E
 BCC *+4
 INC $5F

 LDA #5
 STA $8C

***
* Sort each string ptr in the himem work page
***

CHK.VAR LDY #0
 LDA ($5E),Y   ; GET LENGTH OF STRING
 BEQ BUMP      ; IGNORE STRING IF LENGTH IS ZERO

 LDY #2
 LDA ($5E),Y   ; GET ADDRESS OF STRING
 DEY
 CMP $70       ; CHECK IF ALREADY COLLECTED
 BCC CHKVAR1   ; NO, BELOW FRETOP
 BNE BUMP      ; YES, ABOVE FRETOP
 LDA ($5E),Y
 CMP $6F       ; MAYBE, CHECK LO-BYTE
 BCS BUMP      ; YES, ABOVE FRETOP

* store each string in the table of sorted addresses starting at Himem
* one page for hi-byte and one for lo-byte allows for 256 strings

CHKVAR1 DEY           ; Y=0
 LDA $D9
 BEQ ADD2END
 STY $8F
]LOOP LDA ($8D),Y
 STA $9C
 LDA ($87),Y
 STA $9B
 LDY #2
]LP LDA ($9B),Y   ; Check hi-byte first then lo-byte
 CMP ($5E),Y
 BEQ *+6
 BCC MOVEUP    ; above current ptr, so give this one priority
 BCS *+5       ; branch if below current ptr
 DEY
 BNE ]LP

 INC $8F
 LDY $8F
 CPY $D9       ; chk for end of list
 BNE ]LOOP     ; no, go again
 INY           ; chk for end of table
 BEQ BUMP      ; yes, so do not change table
 DEY           ; add ptrs to end of list without shifting up 
 BNE ADD2END   ; always

MOVEUP LDY $D9
]LP LDA ($87),Y
 INY
 STA ($87),Y
 DEY
 LDA ($8D),Y
 INY
 STA ($8D),Y
 DEY
 BEQ ADD2END   ; add to beginning of ptr list
 DEY
 CPY $8F       ; move up until the posn where to insert is reached
 BCS ]LP

* This is the location of the variable that has the ptr to the highest strings

ADD2END LDA $5E
 STA ($87),Y
 LDA $5F
 STA ($8D),Y

 LDA $D9       ; increment Y counter
 CMP #$FE      ; some pointers get left out of the table and are
 BEQ *+4       ; processed in the next round
 INC $D9

BUMP CLC
 LDA $5E
 ADC $8C       ; 3, 5 or 7
 STA $5E
 BCC *+4
 INC $5F
 LDX $5F
 RTS
 
*** Copy and paste all below here into any emulator at the applesoft prompt and the demo will automatically run ***

CALL-151
2000:A6 74 A4 73 86 70 84 6F F0 03 E8 A0 00 84 87 86
2010:88 84 8D E8 86 8E A9 03 85 8C A9 55 A2 00 85 5E
2020:86 5F 86 D9 C5 52 F0 05 20 FE 90 D0 F7 A5 69 A6
2030:6A 85 5E 86 5F E4 6C 90 04 C5 6B B0 06 20 E1 90
2040:D0 F3 60 85 94 86 95 A5 94 E4 6E 90 04 C5 6D B0
2050:3E 85 5E 86 5F A0 03 84 8C 88 B1 5E 65 94 85 94
2060:C8 B1 5E 65 95 AA 86 95 A0 01 B1 5E 10 D9 88 B1
2070:5E 30 D4 A0 04 B1 5E 0A 69 05 65 5E 85 5E 90 02
2080:E6 5F 20 FE 90 E4 95 90 F9 C5 94 90 F5 F0 B8 A0
2090:00 84 8F B1 87 85 8A B1 8D 85 8B A0 02 B1 8A 85
20A0:9C 88 B1 8A 85 9B 88 B1 8A 18 65 9B 85 96 A5 9C
20B0:69 00 85 97 A5 6F A6 70 85 94 86 95 20 9A D3 A0
20C0:01 A5 94 91 8A 85 6F C8 E6 95 A5 95 91 8A 85 70
20D0:E6 8F A4 8F C4 D9 90 BB F0 B9 C8 D0 03 4C 16 90
20E0:60 A9 07 85 8C A0 01 B1 5E 10 7B 88 B1 5E 30 76
20F0:A5 5E 69 02 85 5E 90 02 E6 5F A9 05 85 8C A0 00
2100:B1 5E F0 62 A0 02 B1 5E 88 C5 70 90 08 D0 57 B1
2110:5E C5 6F B0 51 88 A5 D9 F0 3C 84 8F B1 8D 85 9C
2120:B1 87 85 9B A0 02 B1 9B D1 5E F0 04 90 13 B0 03
2130:88 D0 F3 E6 8F A4 8F C4 D9 D0 E1 C8 F0 28 88 D0
2140:15 A4 D9 B1 87 C8 91 87 88 B1 8D C8 91 8D 88 F0
2150:05 88 C4 8F B0 ED A5 5E 91 87 A5 5F 91 8D A5 D9
2160:C9 FE F0 02 E6 D9 18 A5 5E 65 8C 85 5E 90 02 E6
2170:5F A6 5F 60 00 
3D0G
BSAVE GARB.SORT,A$2000,L$174
 5  LOMEM: 4096: REM  is the garbage routine start + $200
 6  PRINT  CHR$ (4)"BLOAD GARB.SORT,A3584"
 7  POKE 3584 + 42,14: POKE 3584 + 63,14: POKE 3584 + 132,14: POKE 3584 + 223,14
 8  POKE 1014,0: POKE 1015,14
 10 AM = 32
 1000 N = 500: DIM A$(N): REM  create 500 strings
 1010  FOR I = 1 TO N: FOR J = 1 TO 5 +  INT ( RND (1) * 5):A$(I) = A$(I) +  CHR$ ( INT ( RND (1) * 96) + AM): NEXT : VTAB 23: PRINT I"  ": NEXT
 1015  PRINT : PRINT "Available memory: "( PEEK (112) * 256 +  PEEK (111)) - ( PEEK (110) * 256 +  PEEK (109))
 1020  FOR I = 1 TO 10: PRINT A$(I),: NEXT
 1030  &  : REM replace with ?CHR$(4)"FRE"  to initiate the Basic.system version for comparison
 1035  PRINT : PRINT "Available memory: "( PEEK (112) * 256 +  PEEK (111)) - ( PEEK (110) * 256 +  PEEK (109))
 1040  FOR I = 1 TO 10: PRINT A$(I),: NEXT 

SAVE GARB.TEST
RUN

[toc] | [next] | [standalone]

#2357

On Saturday, February 20, 2016 at 9:33:11 PM UTC-7, gid...@sasktel.net wrote:
> Another version of garbage collection?  Or, just a better one?

This is great work!

>  This would give us and allow for 256 string pointers through each pass through the variable tables.  I apologize for mis-leading you to think we are only collecting 128 strings at a time, but such is the suspense of story telling. 

Question that is tangential to this:

Is there a limit on the number of strings Applesoft can handle, aside from a memory one? 256 max strings? i.e. if LOMEM was set to something ridiculous so that Applesoft had ~32K available, is there a hard limit to the number of strings or is strictly memory limited?

Since variables only use the first 2 characters of the identifiers:

a) What are valid identifier names? 
b) The first letter has to be an uppercase or lowercase letter, right?

c) Let's assume you can use the characters $40 .. $7F which is around $40 choices for the first character of the identifier.  Ignoring that you can use numbers for the second character of the identifier, does this mean Applesoft can process $40 * $40 = $1000 strings?


I probably should re-read this consolidated Applesoft disassembly again to see where/how  strings are handled ...

http://jamtronix.com/files/applesoft.html

[toc] | [prev] | [next] | [standalone]

#2359

Memory limited. You can have arrays of strings, which means that the limited 
number of identifiers is irrelevant to how many total strings there can be.

"Michael Pohoreski" <michael.pohoreski@gmail.com> wrote in message 
news:6f8f2b6f-3c25-4030-893a-a5573dcad824@googlegroups.com...
On Saturday, February 20, 2016 at 9:33:11 PM UTC-7, gid...@sasktel.net 
wrote:
> Another version of garbage collection?  Or, just a better one?

This is great work!

>  This would give us and allow for 256 string pointers through each pass 
> through the variable tables.  I apologize for mis-leading you to think we 
> are only collecting 128 strings at a time, but such is the suspense of 
> story telling.

Question that is tangential to this:

Is there a limit on the number of strings Applesoft can handle, aside from a 
memory one? 256 max strings? i.e. if LOMEM was set to something ridiculous 
so that Applesoft had ~32K available, is there a hard limit to the number of 
strings or is strictly memory limited?

Since variables only use the first 2 characters of the identifiers:

a) What are valid identifier names?
b) The first letter has to be an uppercase or lowercase letter, right?

c) Let's assume you can use the characters $40 .. $7F which is around $40 
choices for the first character of the identifier.  Ignoring that you can 
use numbers for the second character of the identifier, does this mean 
Applesoft can process $40 * $40 = $1000 strings?


I probably should re-read this consolidated Applesoft disassembly again to 
see where/how  strings are handled ...

http://jamtronix.com/files/applesoft.html

[toc] | [prev] | [next] | [standalone]

#2360

> This is great work!
> 
> >  This would give us and allow for 256 string pointers through each pass through the variable tables.  I apologize for mis-leading you to think we are only collecting 128 strings at a time, but such is the suspense of story telling. 
> 
> Question that is tangential to this:
> 
> Is there a limit on the number of strings Applesoft can handle, aside from a memory one? 256 max strings? i.e. if LOMEM was set to something ridiculous so that Applesoft had ~32K available, is there a hard limit to the number of strings or is strictly memory limited?


Applesoft limits each array indice to 65536.  So the array NAME$(65536,65536,65536) is a valid array and would allow for, lets just say that is a really big number of strings.  The array itself would not even fit in memory, let alone any strings.  But even if it did have the memory, this is much larger than the 16 Mb file limit of Prodos.

> 
> Since variables only use the first 2 characters of the identifiers:
> 
> a) What are valid identifier names? 

The first character of a variable must be a letter, and the 2nd can be a letter or number.  No punctuation or symbols.


> b) The first letter has to be an uppercase or lowercase letter, right?

Lowercase letters can be entered in, but are always converted to uppercase.


> c) Let's assume you can use the characters $40 .. $7F which is around $40 choices for the first character of the identifier.  Ignoring that you can use numbers for the second character of the identifier, does this mean Applesoft can process $40 * $40 = $1000 strings?
 
You wouldn't enter strings that way, giving each string a variable name.  Arrays are much more advanced under applesoft than they were under Integer Basic.  A single string array with a single indice can have up to 65536 strings.

An indice is the number of groups an array can have.  i.e NA$(65536) has one indice.  NA$(65536,65536) has two indices and can have pointers to 65536x65536 strings.  The example in the first question has 3 indices.  I could not find any restriction in Applesoft on the number of indices one can have, but memory has a big part in restricting the size of the array.


> I probably should re-read this consolidated Applesoft disassembly again to see where/how strings are handled ...


A little hint of things to come, a Virtual Memory for strings that will allow string arrays have up to the 16 Mb limit of Prodos, and support for multiple Virtual Memory files at the same time.

[toc] | [prev] | [next] | [standalone]

#2362

gids.rs wrote:
>> This is great work!
>> 
>> >  This would give us and allow for 256 string pointers through each pass
>> >  through the variable tables.  I apologize for mis-leading you to think
>> >  we are only collecting 128 strings at a time, but such is the suspense
>> >  of story telling.
>> 
>> Question that is tangential to this:
>> 
>> Is there a limit on the number of strings Applesoft can handle, aside
>> from a memory one? 256 max strings? i.e. if LOMEM was set to something
>> ridiculous so that Applesoft had ~32K available, is there a hard limit to
>> the number of strings or is strictly memory limited?
> 
> 
> Applesoft limits each array indice to 65536.  So the array
> NAME$(65536,65536,65536) is a valid array and would allow for, lets just
> say that is a really big number of strings.  The array itself would not
> even fit in memory, let alone any strings.  But even if it did have the
> memory, this is much larger than the 16 Mb file limit of Prodos.
Small adjustment: Plain vanilla Applesoft accepts indice values from 1 to
32767 while handling DIM statements and from 0 to 32767 within array element
lookup.
For instance,
DIM A%(32768) will lead to an "?ILLEGAL QUANTITY ERROR" raised at the time
the indice value is evaluated..
DIM A%(32767) will lead to an "?OUT OF MEMORY ERROR" instead, raised
afterwards when time is coming for ensuring enough RAM available within the
heap.
At the time of Applesoft release, this was a sensible choice as adressable
mémory was limited to 64K and smallest element size for arrays was the
16bits (2 bytes wide) integer (65536 / 2 = 32768)...

 


-- 
Growing old is mandatory..
growing up is optional..
But the other way round is as true.

[toc] | [prev] | [next] | [standalone]

#2363

On Tuesday, February 23, 2016 at 9:36:53 AM UTC-6, Benoit0123 wrote:
> gids.rs wrote:
> >> This is great work!
> >> 
> >> >  This would give us and allow for 256 string pointers through each pass
> >> >  through the variable tables.  I apologize for mis-leading you to think
> >> >  we are only collecting 128 strings at a time, but such is the suspense
> >> >  of story telling.
> >> 
> >> Question that is tangential to this:
> >> 
> >> Is there a limit on the number of strings Applesoft can handle, aside
> >> from a memory one? 256 max strings? i.e. if LOMEM was set to something
> >> ridiculous so that Applesoft had ~32K available, is there a hard limit to
> >> the number of strings or is strictly memory limited?
> > 
> > 
> > Applesoft limits each array indice to 65536.  So the array
> > NAME$(65536,65536,65536) is a valid array and would allow for, lets just
> > say that is a really big number of strings.  The array itself would not
> > even fit in memory, let alone any strings.  But even if it did have the
> > memory, this is much larger than the 16 Mb file limit of Prodos.
> Small adjustment: Plain vanilla Applesoft accepts indice values from 1 to
> 32767 while handling DIM statements and from 0 to 32767 within array element
> lookup.
> For instance,
> DIM A%(32768) will lead to an "?ILLEGAL QUANTITY ERROR" raised at the time
> the indice value is evaluated..
> DIM A%(32767) will lead to an "?OUT OF MEMORY ERROR" instead, raised
> afterwards when time is coming for ensuring enough RAM available within the
> heap.
> At the time of Applesoft release, this was a sensible choice as adressable
> mémory was limited to 64K and smallest element size for arrays was the
> 16bits (2 bytes wide) integer (65536 / 2 = 32768)...
> 


Thanks for the correction.  I always thought applesoft limited the max dimension size to 32767 but I thought that was only for integer variables, and I could not find where it was limited at, in my applesoft dissassembly.  I finally found it.  The limitation was not in the dimensioning routine, but instead was in the conversion from floating point to an integer in applesoft.  And it affects all array variables, string, floating point and integer.

This limitation will be removed in my version of a streamlined applesoft since there will be no FP to INT conversions.

[toc] | [prev] | [next] | [standalone]

#2364

gids.rs wrote:
> On Tuesday, February 23, 2016 at 9:36:53 AM UTC-6, Benoit0123 wrote:
>> gids.rs wrote:
>> >> This is great work!
>> >> 
>> >> >  This would give us and allow for 256 string pointers through each
>> >> >  pass
>> >> >  through the variable tables.  I apologize for mis-leading you to
>> >> >  think
>> >> >  we are only collecting 128 strings at a time, but such is the
>> >> >  suspense
>> >> >  of story telling.
>> >> 
>> >> Question that is tangential to this:
>> >> 
>> >> Is there a limit on the number of strings Applesoft can handle, aside
>> >> from a memory one? 256 max strings? i.e. if LOMEM was set to something
>> >> ridiculous so that Applesoft had ~32K available, is there a hard limit
>> >> to
>> >> the number of strings or is strictly memory limited?
>> > 
>> > 
>> > Applesoft limits each array indice to 65536.  So the array
>> > NAME$(65536,65536,65536) is a valid array and would allow for, lets
>> > just
>> > say that is a really big number of strings.  The array itself would not
>> > even fit in memory, let alone any strings.  But even if it did have the
>> > memory, this is much larger than the 16 Mb file limit of Prodos.
>> Small adjustment: Plain vanilla Applesoft accepts indice values from 1 to
>> 32767 while handling DIM statements and from 0 to 32767 within array
>> element
>> lookup.
>> For instance,
>> DIM A%(32768) will lead to an "?ILLEGAL QUANTITY ERROR" raised at the
>> time
>> the indice value is evaluated..
>> DIM A%(32767) will lead to an "?OUT OF MEMORY ERROR" instead, raised
>> afterwards when time is coming for ensuring enough RAM available within
>> the
>> heap.
>> At the time of Applesoft release, this was a sensible choice as
>> adressable
>> mémory was limited to 64K and smallest element size for arrays was the
>> 16bits (2 bytes wide) integer (65536 / 2 = 32768)...
>> 
> 
> 
> Thanks for the correction.  I always thought applesoft limited the max
> dimension size to 32767 but I thought that was only for integer variables,
> and I could not find where it was limited at, in my applesoft
> dissassembly.
> I finally found it.  The limitation was not in the dimensioning routine,
> but instead was in the conversion from floating point to an integer in
> applesoft.  And it affects all array variables, string, floating point and
> integer.
> 
> This limitation will be removed in my version of a streamlined applesoft
> since there will be no FP to INT conversions.

If you are interested in enhancing Applesoft in the array management domain,
may I suggest that you take a look of an article by John R. Vokey Ph.D who
wrote a very interesting article for managing Applesoft arrays live from
ProDOS files (sort of virtual memory, hence the article title) and published
in Nibble magazine?
Likewise the array size limitation barrier is pushed some kilobytes further
away ;-) if that is your goal (I understand "letting Applesoft evolve/mutate
according to 'new' Apple // hardware platform capabilities").
It appeared in the August 1990 issue of Nibble magazine. The utility runs
under ProDOS 8 (BASIC.SYSTEM) and a provided demo (provided on the disk)
illustrates well what this utility could be used for.
The disk section of the magazine site's address is given below:
http://www.nibblemagazine.com/nibble_disks.htm

By the way, thanks to the author of this utility, M. Vokey Ph.D, for having
shared his works and experiments, to M. Harvey for having managed such a
great magazine at the time and to Stavros Karatsoridis for having prepared
the Nibble disks now available to the community.



-- 
Growing old is mandatory..
growing up is optional..
But the other way round is as true.

[toc] | [prev] | [next] | [standalone]

#2367

Benoit0123 <bgilon@free.fr> wrote:
> gids.rs wrote:
>> On Tuesday, February 23, 2016 at 9:36:53 AM UTC-6, Benoit0123 wrote:
>>> gids.rs wrote:
>>>>> This is great work!
>>>>> 
>>>>>> This would give us and allow for 256 string pointers through each
>>>>>> pass
>>>>>> through the variable tables.  I apologize for mis-leading you to
>>>>>> think
>>>>>> we are only collecting 128 strings at a time, but such is the
>>>>>> suspense
>>>>>> of story telling.
>>>>> 
>>>>> Question that is tangential to this:
>>>>> 
>>>>> Is there a limit on the number of strings Applesoft can handle, aside
>>>>> from a memory one? 256 max strings? i.e. if LOMEM was set to something
>>>>> ridiculous so that Applesoft had ~32K available, is there a hard limit
>>>>> to
>>>>> the number of strings or is strictly memory limited?
>>>> 
>>>> 
>>>> Applesoft limits each array indice to 65536.  So the array
>>>> NAME$(65536,65536,65536) is a valid array and would allow for, lets
>>>> just
>>>> say that is a really big number of strings.  The array itself would not
>>>> even fit in memory, let alone any strings.  But even if it did have the
>>>> memory, this is much larger than the 16 Mb file limit of Prodos.
>>> Small adjustment: Plain vanilla Applesoft accepts indice values from 1 to
>>> 32767 while handling DIM statements and from 0 to 32767 within array
>>> element
>>> lookup.
>>> For instance,
>>> DIM A%(32768) will lead to an "?ILLEGAL QUANTITY ERROR" raised at the
>>> time
>>> the indice value is evaluated..
>>> DIM A%(32767) will lead to an "?OUT OF MEMORY ERROR" instead, raised
>>> afterwards when time is coming for ensuring enough RAM available within
>>> the
>>> heap.
>>> At the time of Applesoft release, this was a sensible choice as
>>> adressable
>>> mémory was limited to 64K and smallest element size for arrays was the
>>> 16bits (2 bytes wide) integer (65536 / 2 = 32768)...
>>> 
>> 
>> 
>> Thanks for the correction.  I always thought applesoft limited the max
>> dimension size to 32767 but I thought that was only for integer variables,
>> and I could not find where it was limited at, in my applesoft
>> dissassembly.
>> I finally found it.  The limitation was not in the dimensioning routine,
>> but instead was in the conversion from floating point to an integer in
>> applesoft.  And it affects all array variables, string, floating point and
>> integer.
>> 
>> This limitation will be removed in my version of a streamlined applesoft
>> since there will be no FP to INT conversions.
> 
> If you are interested in enhancing Applesoft in the array management domain,
> may I suggest that you take a look of an article by John R. Vokey Ph.D who
> wrote a very interesting article for managing Applesoft arrays live from
> ProDOS files (sort of virtual memory, hence the article title) and published
> in Nibble magazine?
> Likewise the array size limitation barrier is pushed some kilobytes further
> away ;-) if that is your goal (I understand "letting Applesoft evolve/mutate
> according to 'new' Apple // hardware platform capabilities").
> It appeared in the August 1990 issue of Nibble magazine. The utility runs
> under ProDOS 8 (BASIC.SYSTEM) and a provided demo (provided on the disk)
> illustrates well what this utility could be used for.
> The disk section of the magazine site's address is given below:
> http://www.nibblemagazine.com/nibble_disks.htm
> 
> By the way, thanks to the author of this utility, M. Vokey Ph.D, for having
> shared his works and experiments, to M. Harvey for having managed such a
> great magazine at the time and to Stavros Karatsoridis for having prepared
> the Nibble disks now available to the community.

I've written several Applesoft programs that use ProDOS binary files as
"virtual memory". BLOAD and BSAVE with (B)yte offset and (L)ength
parameters essentially make binary files directly "addressable".  And
ProDOS's implementation of sparse files makes many simple techniques
practical. 

Add a little LRU caching and it can be reasonably efficient. ;-)
-- 
-michael - NadaNet 3.1 and AppleCrate II:  http://michaeljmahon.com

[toc] | [prev] | [next] | [standalone]

#2368

On Tuesday, February 23, 2016 at 11:52:22 AM UTC-7, mjm...@aol.com wrote:
> I've written several Applesoft programs that use ProDOS binary files as
> "virtual memory". BLOAD and BSAVE with (B)yte offset and (L)ength
> parameters essentially make binary files directly "addressable".  And
> ProDOS's implementation of sparse files makes many simple techniques
> -michael - NadaNet 3.1 and AppleCrate II:  http://michaeljmahon.com

If only DOS 3.3 wasn't dumb and store the meta information (Binary Address and Binary Length) in the file itself ... :-/

[toc] | [prev] | [next] | [standalone]

#2372

Michael Pohoreski <michael.pohoreski@gmail.com> wrote:
> On Tuesday, February 23, 2016 at 11:52:22 AM UTC-7, mjm...@aol.com wrote:
>> I've written several Applesoft programs that use ProDOS binary files as
>> "virtual memory". BLOAD and BSAVE with (B)yte offset and (L)ength
>> parameters essentially make binary files directly "addressable".  And
>> ProDOS's implementation of sparse files makes many simple techniques
>> -michael - NadaNet 3.1 and AppleCrate II:  http://michaeljmahon.com
> 
> If only DOS 3.3 wasn't dumb and store the meta information (Binary
> Address and Binary Length) in the file itself ... :-/

That wouldn't be a problem since it's just a constant offset hidden by the
file system. 

But you can't do anything general without byte offsets (from start of
logical file).  We're talking about files potentially much larger than RAM.

-- 
-michael - NadaNet 3.1 and AppleCrate II:  http://michaeljmahon.com

[toc] | [prev] | [next] | [standalone]

#2371

On Tuesday, February 23, 2016 at 12:11:29 PM UTC-6, Benoit0123 wrote:
> gids.rs wrote:
> > On Tuesday, February 23, 2016 at 9:36:53 AM UTC-6, Benoit0123 wrote:
> >> gids.rs wrote:
> >> >> This is great work!
> >> >> 
> >> >> >  This would give us and allow for 256 string pointers through each
> >> >> >  pass
> >> >> >  through the variable tables.  I apologize for mis-leading you to
> >> >> >  think
> >> >> >  we are only collecting 128 strings at a time, but such is the
> >> >> >  suspense
> >> >> >  of story telling.
> >> >> 
> >> >> Question that is tangential to this:
> >> >> 
> >> >> Is there a limit on the number of strings Applesoft can handle, aside
> >> >> from a memory one? 256 max strings? i.e. if LOMEM was set to something
> >> >> ridiculous so that Applesoft had ~32K available, is there a hard limit
> >> >> to
> >> >> the number of strings or is strictly memory limited?
> >> > 
> >> > 
> >> > Applesoft limits each array indice to 65536.  So the array
> >> > NAME$(65536,65536,65536) is a valid array and would allow for, lets
> >> > just
> >> > say that is a really big number of strings.  The array itself would not
> >> > even fit in memory, let alone any strings.  But even if it did have the
> >> > memory, this is much larger than the 16 Mb file limit of Prodos.
> >> Small adjustment: Plain vanilla Applesoft accepts indice values from 1 to
> >> 32767 while handling DIM statements and from 0 to 32767 within array
> >> element
> >> lookup.
> >> For instance,
> >> DIM A%(32768) will lead to an "?ILLEGAL QUANTITY ERROR" raised at the
> >> time
> >> the indice value is evaluated..
> >> DIM A%(32767) will lead to an "?OUT OF MEMORY ERROR" instead, raised
> >> afterwards when time is coming for ensuring enough RAM available within
> >> the
> >> heap.
> >> At the time of Applesoft release, this was a sensible choice as
> >> adressable
> >> mémory was limited to 64K and smallest element size for arrays was the
> >> 16bits (2 bytes wide) integer (65536 / 2 = 32768)...
> >> 
> > 
> > 
> > Thanks for the correction.  I always thought applesoft limited the max
> > dimension size to 32767 but I thought that was only for integer variables,
> > and I could not find where it was limited at, in my applesoft
> > dissassembly.
> > I finally found it.  The limitation was not in the dimensioning routine,
> > but instead was in the conversion from floating point to an integer in
> > applesoft.  And it affects all array variables, string, floating point and
> > integer.
> > 
> > This limitation will be removed in my version of a streamlined applesoft
> > since there will be no FP to INT conversions.
> 
> If you are interested in enhancing Applesoft in the array management domain,
> may I suggest that you take a look of an article by John R. Vokey Ph.D who
> wrote a very interesting article for managing Applesoft arrays live from
> ProDOS files (sort of virtual memory, hence the article title) and published
> in Nibble magazine?
> Likewise the array size limitation barrier is pushed some kilobytes further
> away ;-) if that is your goal (I understand "letting Applesoft evolve/mutate
> according to 'new' Apple // hardware platform capabilities").
> It appeared in the August 1990 issue of Nibble magazine. The utility runs
> under ProDOS 8 (BASIC.SYSTEM) and a provided demo (provided on the disk)
> illustrates well what this utility could be used for.
> The disk section of the magazine site's address is given below:
> http://www.nibblemagazine.com/nibble_disks.htm
> 
> By the way, thanks to the author of this utility, M. Vokey Ph.D, for having
> shared his works and experiments, to M. Harvey for having managed such a
> great magazine at the time and to Stavros Karatsoridis for having prepared
> the Nibble disks now available to the community.


Thanks!  I have that nibble article.  That was my inspiration in showing me that it can be done.

I have the layout on paper on how I believe VM could work with strings, and my reason for delving into this garbage collector routine.

I believe I now have the knowledge to pull it off.

[toc] | [prev] | [next] | [standalone]

#2380

On 2016-02-22 23:46, gids.rs@sasktel.net wrote:
>> b) The first letter has to be an uppercase or lowercase letter, right?
>
> Lowercase letters can be entered in, but are always converted to uppercase.

This is only true of the IIc and later (incl the Enhanced IIe). The 
original IIe and earlier firmwares do not convert to uppercase. (Of 
course the II, and II+ keyboards don't produce lowercase, but there were 
software ways.)

And lowercase letters are are not valid in AppleSoft (or Integer BASIC) 
variable names.

[toc] | [prev] | [next] | [standalone]

#2391

Brian Patrie wrote:
> On 2016-02-22 23:46, gids.rs@sasktel.net wrote:
>>> b) The first letter has to be an uppercase or lowercase letter, right?
>>
>> Lowercase letters can be entered in, but are always converted to
>> uppercase.
> 
> This is only true of the IIc and later (incl the Enhanced IIe). The 
> original IIe and earlier firmwares do not convert to uppercase. (Of 
> course the II, and II+ keyboards don't produce lowercase, but there were 
> software ways.)

Also, don't forget the feature that the Apple IIe had that would allow you
to type the Applesoft tokens in uppercase, but make strings between double
quotes (") in lower-case. What key command activated this? I think it was
Escape-Control-R

Yes, it was called Uppercase Restrict.

-- 
]DF$
The Marina IP stack for Apple II--
http://marina.a2hq.com/

[toc] | [prev] | [next] | [standalone]

#2404

On Monday, February 29, 2016 at 3:12:36 PM UTC-8, D Finnigan wrote:
> Also, don't forget the feature that the Apple IIe had that would allow you
> to type the Applesoft tokens in uppercase, but make strings between double
> quotes (") in lower-case. What key command activated this? I think it was
> Escape-Control-R
> 
> Yes, it was called Uppercase Restrict.

Oh wow, didn't know that.  Is there a FAQ / list with all the supported Escape codes on the ][+, //e, and //c ?

[toc] | [prev] | [next] | [standalone]

#2405

Michael Pohoreski wrote:
> On Monday, February 29, 2016 at 3:12:36 PM UTC-8, D Finnigan wrote:
>> Also, don't forget the feature that the Apple IIe had that would allow
>> you
>> to type the Applesoft tokens in uppercase, but make strings between
>> double
>> quotes (") in lower-case. What key command activated this? I think it was
>> Escape-Control-R
>> 
>> Yes, it was called Uppercase Restrict.
> 
> Oh wow, didn't know that.  Is there a FAQ / list with all the supported
> Escape codes on the ][+, //e, and //c ?
>

Do you own a book called The New Apple II User's Guide? It's in there.

[toc] | [prev] | [next] | [standalone]

#2410

On Tuesday, March 1, 2016 at 10:07:34 AM UTC-8, D Finnigan wrote:
> Michael Pohoreski wrote:
> > Oh wow, didn't know that.  Is there a FAQ / list with all the supported
> > Escape codes on the ][+, //e, and //c ?
> Do you own a book called The New Apple II User's Guide? It's in there.

I have your book. ;-)  Ah, Table E-4, Page 662 of Appendix E: Tables and Conversion Charts.
Thanks!

P.S.
Let me know if you ever plan to do a 2nd Ed so we can fix some of the oversights. :-)

[toc] | [prev] | [next] | [standalone]

#2366

Michael Pohoreski <michael.pohoreski@gmail.com> wrote:
> On Saturday, February 20, 2016 at 9:33:11 PM UTC-7, gid...@sasktel.net wrote:
>> Another version of garbage collection?  Or, just a better one?
> 
> This is great work!
> 
>> This would give us and allow for 256 string pointers through each pass
>> through the variable tables.  I apologize for mis-leading you to think
>> we are only collecting 128 strings at a time, but such is the suspense of story telling. 
> 
> Question that is tangential to this:
> 
> Is there a limit on the number of strings Applesoft can handle, aside
> from a memory one? 256 max strings? i.e. if LOMEM was set to something
> ridiculous so that Applesoft had ~32K available, is there a hard limit to
> the number of strings or is strictly memory limited?
> 
> Since variables only use the first 2 characters of the identifiers:
> 
> a) What are valid identifier names? 
> b) The first letter has to be an uppercase or lowercase letter, right?
> 
> c) Let's assume you can use the characters $40 .. $7F which is around $40
> choices for the first character of the identifier.  Ignoring that you can
> use numbers for the second character of the identifier, does this mean
> Applesoft can process $40 * $40 = $1000 strings?
> 
> 
> I probably should re-read this consolidated Applesoft disassembly again
> to see where/how  strings are handled ...
> 
> http://jamtronix.com/files/applesoft.html
> 
> 

Don't forget string arrays. 

Memory is the only limit. 
-- 
-michael - NadaNet 3.1 and AppleCrate II:  http://michaeljmahon.com

[toc] | [prev] | [standalone]

Back to top | Article view | comp.sys.apple2.programmer

csiph-web