Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.sys.apple2.programmer > #2085 > unrolled thread

New discovery with Uthernet II/W5100 buffers

Back to article view | Back to comp.sys.apple2.programmer

Contents

  New discovery with Uthernet II/W5100 buffers D Finnigan <dog_cow@macgui.com> - 2016-01-09 22:04 +0000
    Re: New discovery with Uthernet II/W5100 buffers ol.sc@web.de (Oliver Schmidt) - 2016-01-19 22:20 +0000
      Re: New discovery with Uthernet II/W5100 buffers ol.sc@web.de (Oliver Schmidt) - 2016-01-20 18:33 +0000
        Re: New discovery with Uthernet II/W5100 buffers D Finnigan <dog_cow@macgui.com> - 2016-01-20 19:34 +0000
          Re: New discovery with Uthernet II/W5100 buffers ol.sc@web.de (Oliver Schmidt) - 2016-01-21 18:20 +0000
            Re: New discovery with Uthernet II/W5100 buffers D Finnigan <dog_cow@macgui.com> - 2016-01-22 23:47 +0000
              Re: New discovery with Uthernet II/W5100 buffers ol.sc@web.de (Oliver Schmidt) - 2016-01-24 01:01 +0000
                Re: New discovery with Uthernet II/W5100 buffers D Finnigan <dog_cow@macgui.com> - 2016-01-25 16:03 +0000
                  Re: New discovery with Uthernet II/W5100 buffers Oliver Schmidt <ol.sc@web.de> - 2016-04-13 05:49 -0700

#2085 — New discovery with Uthernet II/W5100 buffers

While spending more time debugging the new send sequence algorithm for the
Uthernet II driver in Marina, I discovered a really excellent feature in the
W5100 that as far as I know is not mentioned in any of the documentation, or
anywhere else that I've read.

I was trying to debug the code that resets the W5100 address pointer back to
the start of the RX buffer. I inserted some debug lines to print the present
value of the W5100 address pointer *before* I reset it back to $6000.

Well imagine my surprise when I saw that it was printing out $6001! So not
only did I have an off-by-one error in my code, but somehow the address was
wrapping around before my code reset it!

So I poked around in the Monitor, manually setting and testing the address
ports and the reading the data port.

And now, here is my good discovery:

The W5100 will automatically wrap the address pointer from $7FFF back to
$6000, -AND- it will also automatically wrap the address pointer from $5FFF
back to $4000.

And there was much rejoicing. :-D

What this means is that if you're coding for the case where you're only
using 1 socket (in any mode-- TCP, UDP, Raw, it doesn't matter) and your
buffer sizes are both 8 KB (why wouldn't they be???) then you don't need to
worry about buffer wrap around at all. Not a thing to concern yourself with.
Just make sure your reads or writes start at the correct address in the
W5100, and away you go!

These two wrap around addresses seem to be hardwired, and as far as I can
tell, can't be changed even if you change the RX and TX buffer sizes. So if
you're writing code that uses more than 1 socket, you can't use this
wonderful technique.


Another discovery I made, is less important, but also mildly interesting:
What lies beyond $7FFF, the official end of the memory space in the W5100?
Well it turns out that internally it is treated as wrapping around back to
0. Address auto-increment still works, so you set the pointer to $8000, read
or write, and it advances to $8001. What happens when you get to $FFFF? It
wraps back to $E000! I don't recommend writing any programs that read or
write beyond $7FFF, of course! :-)

-- 
]DF$
The Marina IP stack for Apple II--
http://marina.a2hq.com/

[toc] | [next] | [standalone]

#2158

Hi David,

>The W5100 will automatically wrap the address pointer from $7FFF back to
>$6000, -AND- it will also automatically wrap the address pointer from $5FFF
>back to $4000.

Amazing. I'll try to reproduce that asap!

>What this means is that if you're coding for the case where you're only
>using 1 socket (in any mode-- TCP, UDP, Raw, it doesn't matter) and your
>buffer sizes are both 8 KB (why wouldn't they be???) [...]

Actually I see (or should I now say "I saw" ?) a pretty viable use
case:

You may have read that David Schmidt is about to complete the
implementation of an ADTPro VEDRIVE using the Uthernet II built-in
TCP/IP stack. Doing so will allow the VEDRIVE to be "patched into"
ProDOS like the VSDRIVE. This will turn the VEDRIVE from a mere toy to

something really cool - likely even the Uthernet II killer app.

But what happens when the VEDRIVE is active and a "foreground" program
using the Uthernet II gets loaded? This issue could be solved very
elegantly by sharing the Uthernet II on the hardware level. One socket
for the VEDRIVE and one socket for the foreground program.

The foreground program might want to use the W5100 MAC raw mode which
is only possible with socket 0. Therefore the VEDRIVE uses socket 1.
Each socket uses 4kB for both RX and TX.

The foreground program check if the W5100 is already initialized by
the VEDRIVE (i.e. by checking the state of socket 1). Then it does
_not_ reset the W5100 but only initializes socket 0. The MAC address
to use is read from the W5100.

The code in
https://raw.githubusercontent.com/oliverschmidt/ip65/master/supplement/w5100_udp.s
I provided to David Schmidt for VEDRIVE already sets up the W5100
memory to use socket 0 and 1. It uses socket 1 as described above.

My plan is update the W5100 code used by Contiki and IP65 to ckeck for
a running VEDRIVE as soon as David Schmidt has finalized VEDRIVE. By
now ADTPro uses the stock IP65 so it could is the updated W5100 code
right away.

I would of by happy to see you (and David Schmenk) support this
approach too and adapt the Marina (and PLASMA) code cooperate with an
active VEDRIVE.

Regards,
Oliver

[toc] | [prev] | [next] | [standalone]

#2164

Hi David,

>>The W5100 will automatically wrap the address pointer from $7FFF back to
>>$6000, -AND- it will also automatically wrap the address pointer from $5FFF
>>back to $4000.
>
>Amazing. I'll try to reproduce that asap!

I could reproduce what you describe - a big thank-you for sharing this
discovery!

In fact the W5100 indirect bus mode autoincrement pointer divides its
64 kB address space into 8 segments with 8 kB each and always stays in
the current segment.

>>What this means is that if you're coding for the case where you're only
>>using 1 socket (in any mode-- TCP, UDP, Raw, it doesn't matter) and your
>>buffer sizes are both 8 KB (why wouldn't they be???) [...]
>
>Actually I see (or should I now say "I saw" ?) a pretty viable use
>case: [...]

Given that using the W5100 with only a single socket results in
faster/smaller code I'm pretty unsure if there's a momentum to
actually implement/agree on the W5100 sharing between a driver being
part of the OS and and a foreground program running on the OS using
two sockets (as I described it in my previous post).

What are your opinions, gents?

Regards,
Oliver

[toc] | [prev] | [next] | [standalone]

#2165

Oliver Schmidt wrote:
> Hi David,
> 
>>>The W5100 will automatically wrap the address pointer from $7FFF back to
>>>$6000, -AND- it will also automatically wrap the address pointer from
>>>$5FFF
>>>back to $4000.
>>
>>Amazing. I'll try to reproduce that asap!
> 
> I could reproduce what you describe - a big thank-you for sharing this
> discovery!
> 
> In fact the W5100 indirect bus mode autoincrement pointer divides its
> 64 kB address space into 8 segments with 8 kB each and always stays in
> the current segment.

I need to put out the updated Uthernet II manual tomorrow, because that's
got full coverage of this undocumented wraparound feature. You can advance
the address pointer past $8000 but it's a mirror of the beginning of the
address space, as you no doubt encountered in your tests.

> 
> Given that using the W5100 with only a single socket results in
> faster/smaller code I'm pretty unsure if there's a momentum to
> actually implement/agree on the W5100 sharing between a driver being
> part of the OS and and a foreground program running on the OS using
> two sockets (as I described it in my previous post).
> 
> What are your opinions, gents?

It's hard to say, but I'm thinking about writing a tcplib that does three
things:
1.) Provides a simplified interface to open and interact with up to 4 UDP or
TCP sockets,
2.) Provides a DNS resolution service,
3.) Provides a DHCP client.

2 and 3 are essentially done at this point through my past efforts with the
Marina project. 1 is fairly close. I have no time-frame for when I could
complete this, but even having it going by March is not unrealistic, I
think.

-- 
]DF$
The Marina IP stack for Apple II--
http://marina.a2hq.com/

[toc] | [prev] | [next] | [standalone]

#2166

Hi David,

>> In fact the W5100 indirect bus mode autoincrement pointer divides its
>> 64 kB address space into 8 segments with 8 kB each and always stays in
>> the current segment.
>
>[...]You can advance
>the address pointer past $8000 but it's a mirror of the beginning of the
>address space, as you no doubt encountered in your tests.

Maybe a question of definitions:

- What you describe has to do with the question what is mapped where
in the address space. The upper 32kB are a mirror of the lower 32kB.

- What I describe has to do with the question how the pointer to the
address space handles autoincrement. This is totally independent from
the question what is mapped where. Therefore I'm talking about 64kB,
not 32kB.

>> [...] I'm pretty unsure if there's a momentum to
>> actually implement/agree on the W5100 sharing between a driver being
>> part of the OS and and a foreground program running on the OS using
>> two sockets [...].
>> 
>> What are your opinions, gents?
>
>It's hard to say, but I'm thinking about writing a tcplib that does three
>things [...]

Thanks for sharing your plans. However I can't see a relation to my
question above. Trying to re-phrase it...

I understand that your tcplib doesn't use the W5100 TCP/IP stack,
rather it uses the MAC raw socket. Based on that presumption: Would
you be interested/willing to implement your tcplib in a way that it
uses only 4kB of the W5100 TX buffer and only 4kB of the W5100 RX
buffer. The other 2 x 4kB could then be used by the VEDRIVE being
active while a program using your tcplib runs. As the VEDRIVE would be
already active when your tcplib is initialized it would need to detect
the presence of VEDRIVE (in example by looking at the socket 1 status)
and in that case keep clear from reseting/configuring the W5100 but
rather only open socket 0. Instead of setting your MAC address in the
W5100 you'd read the current MAC from the W5100 and use that for the
tcplib.

Regards,
Oliver

[toc] | [prev] | [next] | [standalone]

#2167

Oliver Schmidt wrote:
> 
> I understand that your tcplib doesn't use the W5100 TCP/IP stack,
> rather it uses the MAC raw socket.

No, only Marina uses the MAC raw mode.

Tcplib will use only the UDP or TCP sockets given to us by the W5100,
depending on what the programmer wants. Now that Uthernet II is here with
built-in TCP, I don't want to program any more TCP/IP stacks for it using
MAC raw mode!


> Based on that presumption: Would
> you be interested/willing to implement your tcplib in a way that it
> uses only 4kB of the W5100 TX buffer and only 4kB of the W5100 RX
> buffer. The other 2 x 4kB could then be used by the VEDRIVE being
> active while a program using your tcplib runs. As the VEDRIVE would be
> already active when your tcplib is initialized it would need to detect
> the presence of VEDRIVE (in example by looking at the socket 1 status)
> and in that case keep clear from reseting/configuring the W5100 but
> rather only open socket 0. Instead of setting your MAC address in the
> W5100 you'd read the current MAC from the W5100 and use that for the
> tcplib.

Yes that could be done.

-- 
]DF$
The Marina IP stack for Apple II--
http://marina.a2hq.com/

[toc] | [prev] | [next] | [standalone]

#2168

Hi David,

>> I understand that your tcplib doesn't use the W5100 TCP/IP stack,
>> rather it uses the MAC raw socket.
>
>No, only Marina uses the MAC raw mode.

Okay, _NOW_ I see. So far I thought you'd be talking about "your
tcplib named 'Marina'"...

>Tcplib will use only the UDP or TCP sockets given to us by the W5100,
>depending on what the programmer wants. Now that Uthernet II is here with
>built-in TCP, I don't want to program any more TCP/IP stacks for it using
>MAC raw mode!

Now I fully understand - and look forward to it :-)

>> [...] The other 2 x 4kB could then be used by the VEDRIVE being
>> active [...}
>
>Yes that could be done.

So far there were "only" Contiki/IP65/Marina using one (MAC raw)
socket. So it was easy to say. Let's just split the memory resources
into two equal pieces. 50% for the forground app using the MAC raw
socket und 50% for the VEDRIVE.

With tcplib things become more complex, don't they?

1.
So far my idea is that the VEDRIVE uses socket 1 (socket 0 needs to be
reserved for the foregorund app because MAC raw only works with socket
0 - so Contiki/IP65/Marina need socket 0). But given that tcplib can
make use of up to 4 sockets it might be "cleaner" to have VEDRIVE use
socket 3 instead of socket 1. The W5100 data sheet page 23 shows that
there's no option to explicitly assign 0 kB to a socket. So it
wouldn't be possible to use i.e. only socket 0 for the tcplib and
socket 3 for the VEDRIVE. From that perspective the only viable
solution seems to be that VEDRIVE continues to use socket 1.

2.
So far my idea is that 4 kB are assigned to both socket 0 and socket
1. But this would mean that tcplib can't open socket 2 and/or socket 3
as there's no memory left. On the other hand the memory configuration
for socket 1 can't be changed after VEDRIVE as been initialized - and
even at initalization time of VEDRIVE it might be had to allow for
different memory configurations. The VEDRIVE code needs to be as small
as possible and may therefore need the socket 1 memory area/size to be
hardcoded.

VEDRIVE uses UDP and on top of that a stop-and-wait scheme. So both
the server and the client won't send more than one packet before
receiving the answer. From that perspective the socket 1 doesn't
benefit from "a lot of" memory. On the other hand for W5100-TCP
connections the TCP ACKs are managed by the W5100 so the larger the
memory assigned to a TCP socket the larger the TCP window size.

Right now I see the following viable options for the memory assignment
to sockets 0 to 3 given in kB:

a) 2/2/2/2 - this is the W5100 default. But it means that
Contiki/IP65/Marina as well as tcplib using a single socket (which I
presume to be the most usual scenario) only make use of 2kB.

b) 4/2/1/1 - this gives Contiki/IP65/Marina and tcplib with a single
socket the maximum memory and makes socket 2 and 3 "secondary" sockets
for tcplib.

c) 4/2/x/x - according to my current understanding of the W5100 it is
generally possible to change the memory configuration while sockets
are open as long as it doesn't change memory locations for the already
open sockets. If tests should validate my presumption tcplib could
allow the user to choose between a maximum of two sockets or three
sockets for tcplib. In the former case tcplib would configure 4/2/2/0
and in the latter case tcplib would configure 4/2/1/1.

What do you think?

Regards,
Oliver

[toc] | [prev] | [next] | [standalone]

#2179

Oliver Schmidt wrote:
> 
> b) 4/2/1/1 - this gives Contiki/IP65/Marina and tcplib with a single
> socket the maximum memory and makes socket 2 and 3 "secondary" sockets
> for tcplib.
> 
> c) 4/2/x/x - according to my current understanding of the W5100 it is
> generally possible to change the memory configuration while sockets
> are open as long as it doesn't change memory locations for the already
> open sockets. If tests should validate my presumption tcplib could
> allow the user to choose between a maximum of two sockets or three
> sockets for tcplib. In the former case tcplib would configure 4/2/2/0
> and in the latter case tcplib would configure 4/2/1/1.
> 
> What do you think?

Options b and c seem reasonable to me.

[toc] | [prev] | [next] | [standalone]

#2688

Am Montag, 25. Januar 2016 17:06:00 UTC+1 schrieb D Finnigan:

> > [...]
> > b) 4/2/1/1 - this gives Contiki/IP65/Marina and tcplib with a single
> > socket the maximum memory and makes socket 2 and 3 "secondary" sockets
> > for tcplib.
> > [...]

> Options b and c seem reasonable to me.

Thanks for your positive feedback.

In my last proposal I wrote that I see VEDRIVE use socket 1. However in the
meantime I noticed that VEDRIVE doesn't benefit from 2kB over 1kB. Therfore it
is reasonable to always go for the 4/2/1/1 setup and have VEDRIVE use socket 3
leaving socket 0, 1 und 2 for tcplib.

https://github.com/a2retrosystems/mxp/wiki/W5100-Shared-Access is supposed to be
a spec for this 'W5100 Shared Access'. I already modified the W5100 driver used
by Contiki and IP65 to follow this spec.

BTW: I moved all my W5100 test/demo code to:
https://github.com/a2retrosystems/mxp/tree/master/w5100/demo
and my W5100 findings to:
https://github.com/a2retrosystems/mxp/wiki

[toc] | [prev] | [standalone]

Back to top | Article view | comp.sys.apple2.programmer

csiph-web