Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > linux.kernel > #1310052 > unrolled thread

Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4

Started byMark Rutland <mark.rutland@arm.com>
First post2016-01-15 12:10 +0100
Last post2016-01-15 20:20 +0100
Articles 20 on this page of 43 — 11 participants

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.


Contents

  Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 Mark Rutland <mark.rutland@arm.com> - 2016-01-15 12:10 +0100
    Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 "H. Nikolaus Schaller" <hns@goldelico.com> - 2016-01-15 16:10 +0100
      Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 Andrey Vostrikov <andrey.vostrikov@cogentembedded.com> - 2016-01-15 16:50 +0100
        Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 "H. Nikolaus Schaller" <hns@goldelico.com> - 2016-01-15 17:10 +0100
          Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 Peter Hurley <peter@hurleysoftware.com> - 2016-01-15 18:20 +0100
            Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 "H. Nikolaus Schaller" <hns@goldelico.com> - 2016-01-15 18:40 +0100
              Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 Peter Hurley <peter@hurleysoftware.com> - 2016-01-15 18:50 +0100
                Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 "H. Nikolaus Schaller" <hns@goldelico.com> - 2016-01-15 19:00 +0100
                  Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 Peter Hurley <peter@hurleysoftware.com> - 2016-01-15 20:30 +0100
                    Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 "H. Nikolaus Schaller" <hns@goldelico.com> - 2016-01-15 22:30 +0100
            Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 Rob Herring <robherring2@gmail.com> - 2016-01-15 23:50 +0100
              Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 Vostrikov Andrey <andrey.vostrikov@cogentembedded.com> - 2016-01-16 08:40 +0100
                Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 Rob Herring <robh@kernel.org> - 2016-01-17 00:40 +0100
                  Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 "H. Nikolaus Schaller" <hns@goldelico.com> - 2016-01-17 10:00 +0100
                    Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> - 2016-01-17 15:30 +0100
                      Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 "H. Nikolaus Schaller" <hns@goldelico.com> - 2016-01-17 19:00 +0100
                        Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> - 2016-01-17 20:40 +0100
                          Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 "H. Nikolaus Schaller" <hns@goldelico.com> - 2016-01-18 09:20 +0100
                            Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 Andrey Vostrikov <andrey.vostrikov@cogentembedded.com> - 2016-01-18 10:00 +0100
                              Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 "H. Nikolaus Schaller" <hns@goldelico.com> - 2016-01-18 13:00 +0100
                            Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> - 2016-01-18 12:30 +0100
                              Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 "H. Nikolaus Schaller" <hns@goldelico.com> - 2016-01-18 22:00 +0100
                                Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> - 2016-01-18 23:10 +0100
                                  Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 "H. Nikolaus Schaller" <hns@goldelico.com> - 2016-01-18 23:40 +0100
                                    Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> - 2016-01-19 15:30 +0100
                                      Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 "H. Nikolaus Schaller" <hns@goldelico.com> - 2016-01-20 18:40 +0100
                                    Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 "H. Nikolaus Schaller" <hns@goldelico.com> - 2016-01-20 17:20 +0100
                                      Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> - 2016-01-20 18:50 +0100
                                        Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 "H. Nikolaus Schaller" <hns@goldelico.com> - 2016-01-20 19:10 +0100
                                          Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 Tomeu Vizoso <tomeu@tomeuvizoso.net> - 2016-01-22 17:00 +0100
                                            Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 Rob Herring <robherring2@gmail.com> - 2016-01-22 18:00 +0100
                                            Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> - 2016-01-22 21:20 +0100
                                              Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 Andreas Kemnade <andreas@kemnade.info> - 2016-01-23 08:50 +0100
                                              Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 "H. Nikolaus Schaller" <hns@goldelico.com> - 2016-01-23 13:20 +0100
                                                Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> - 2016-01-23 18:30 +0100
                                                  Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 "H. Nikolaus Schaller" <hns@goldelico.com> - 2016-01-23 23:10 +0100
                                                    Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> - 2016-01-24 18:20 +0100
                                                      Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 "H. Nikolaus Schaller" <hns@goldelico.com> - 2016-01-25 11:40 +0100
                                  Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 Andreas Kemnade <andreas@kemnade.info> - 2016-01-19 07:40 +0100
                Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 Dmitry Torokhov <dmitry.torokhov@gmail.com> - 2016-01-20 20:40 +0100
                  Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 Vostrikov Andrey <andrey.vostrikov@cogentembedded.com> - 2016-01-20 21:10 +0100
      Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 Mark Rutland <mark.rutland@arm.com> - 2016-01-15 17:20 +0100
        Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4 "H. Nikolaus Schaller" <hns@goldelico.com> - 2016-01-15 20:20 +0100

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →


#1311487

FromOne Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>
Date2016-01-18 12:30 +0100
Message-ID<qSgat-8nF-1@gated-at.bofh.it>
In reply to#1311367
> I have not counted how often I have explained that, but I am happy to do it
> again.
> 
> * the GTA04 device is an open hackable smartphone platform where power saving is highest priority

Yep I did real the manual before replying last time.

> * the wi2wi,w2sg0084 chip is a GPS chip (it does not understand HCI protocol, it just sends NMEA records if powered on)
> * we want to provide NMEA data through /dev/ttyO1 (it uses an OMAP UART) because a tty port is the most common interface for GPS devices (e.g. a bluetooth GPS mouse is also presented as a tty)
> * we want the chip to automatically power up as soon (but not before) as any gps client opens /dev/ttyO1 (or activates the DTR mctrl)
> * we want the chip to automatically power down if no process uses /dev/ttyO1 any more
> The standard logic of GPS daemons and applications is to receive
> NMEA records through some serial /dev/tty.
> 
> Please tell me how power on/off management can be done without intercepting
> somewhere in the kernel that /dev/ttyO1 is opened/closed (which is not the same
> as suspend/resume).

Your user space can do it (as most Android does).

> Secondly, the chip has a very special logic that it may end up in the opposite
> power state than the kernel driver thinks. Especially after boot it simply can not
> know the state. The chip might be powered up/send records or might not.
> 
> A driver can only detect such a discrepancy if it thinks the GPS chip is powered off,
> but there is still data coming through the UART.

To play devils advocate a moment: so can user space.

> Please tell me how this situation can be detected without monitoring the data stream
> in the kernel going to /dev/ttyO1 - from the UART behind it - even if /dev/ttyO1 is closed.
> 
> This are *our* requirements.

The Linux kernel runs on billions of devices. If we put kernel hooks in
random glue layers for every weird little platform corner case it would
collapse in a heap. Those are *our* requirements. Thus it is good to push
back on stuff that can be done in user space just as well.

> Other people think that our approach helps to solve their driver architecture as well
> and have added their requirements on top. This is why I attempt to make the API
> more general than just for our own use-cases.

If it's going to be generic then it needs to be dealing with this at the
tty/tty_port level not uart. uart isn't a general serial abstraction,
tty_port and tty are. uart is just a helper library for some types of
port.

In practise that ought to be a small distinction. If you have to bind
some kind of device logic to the port activation/deactivation then bind
it to tty_port not uart. uart open/close is basically an implementation
of the tty port->ops->activate() and port->ops->shutdown() method.

> 
> > 
> > The 8686 is already working in serial mode  with no kernel hackery on
> > other boards.
> 
> You appear to mix the chips we are talking about. We have the w2sg0084 gps
> chip and a w2cbw003, which is a combo of an 8686 and a CSR BT serial device.
> 
> Both chips need somehow to be powered on or off if not used. Ideally automatically
> at the moment no user space client is using them any more. For the bluetooth side
> the moment to power off is when a hciattach is killed.
> 
> Other 8686 boards appear not to have such critical power restrictions and then they
> just leave power on.

The ones I am familiar with either have the userspace managing it via
sysfs (which has some latency advantages when doing clever stuff) or
wired the power control to the carrier signal (or that is declared the
gpio that controls it to be the carrier).

> No, the opposite: it can be any open source application, which by principle
> could be changed in any way. But we can't because we as the hardware
> platform+kernel developers have no (and don't want to have) control over
> the the user space. So we have to fit into the standards of the user space.

Well the standard of user space today *is* managing via sysfs or the
carrier trick. Take a look at all the Android devices out there - they
all tackle it that way. Some of them do very aggressive power management
as you can imagine. But yes that's ugly 8)

> Here, we need to solve the problem to power down the chip if NO
> tty port is open any more.

port->ops->shutdown in the tty layer.

> > You can just report EBUSY in your open method. You don't need to touch
> > the serial core layer. It's quite sufficient to do
> > 
> > 	if (busy)
> > 		return -EBUSY;
> > 
> > at the top of your uart open method.
> 
> Hm. I am not writing a new UART driver or touch them. I am using the existing
> ones (omap-serial). They all call this uart_add_one_port() in their probe() function.

At the moment. But they may not, or they may get folded together.

> If I would modify just omap-serial, people would for sure complain that the solution
> is not generic enough.

I would say two things

1. If you modify omap-serial it's not that generic, but it doesn't mess
with library code it shouldn't. That is better than messing with uart
layer code. It also solves your problem and localises the solution. That
to me is a win.

2. If not then hook tty_port_shutdown() and tty_port_open() because those
are the right abstraction point. Everything in the kernel that is a tty
is a tty_port.

> > The uart layer is part of the tty layer. It's just a glue library to make
> > writing some tty drivers a bit easier.
> 
> Yes, and exactly that is helpful to solve this problem. We attach only to the
> glue layer and everything is done.

It's the wrong place - it's not the abstraction.

What I am trying to say is that if you do this generically then add the
needed method calls into tty_port_open and tty_port_shutdown, make them
run after port->ops->activate and before port->ops->shutdown so there is
a sensible ordering if you need to do something to the port itself, and
also so on open it only runs if port->ops->activate succeeded.

Something like

       if (!test_bit(ASYNCB_INITIALIZED, &port->flags)) {
                clear_bit(TTY_IO_ERROR, &tty->flags);
                if (port->ops->activate) {
                        int retval = port->ops->activate(port, tty);
                        if (retval) {
                                mutex_unlock(&port->mutex);
                                return retval;
                        }
                }
		/* Wake the device if we have one tied to us */
		if (port->ops->activate_slave)
			port->ops->activate_slave(port, tty);
                set_bit(ASYNCB_INITIALIZED, &port->flags);
	}

Then all you need is the (possibly device specific) small patches to check
the device tree for the bindings on init, and if so set the port->ops
methods according to the binding.

> > It's tied deeply to the tty_port
> > implementation. One day it might even cease to exist replaced by more
> > generic tty_port helpers.
> 
> Is there a concrete plan to change that? Is anyone working on it now?

Nobody wants to lose the ability to do so or to move stuff around.

> If not, I would not worry about this, because it is not specific to the problem
> we want to solve today (to be precise: since 3 years).

You solve it today you block other things for the next 20 years. The
kernel has to deal in long terms.

> > The tty layer is also an *abstract* concept. There is no real world tie
> > between physical collections of shift registers that dribble bits to one
> > another and tty devices in the kernel. You only need a tty driver for
> > certain types of user interaction.
> 
> And we need it to process the GPS data by standard applications and tools.

Ok - no problem with that, and for what you've explained that bit makes
sense.

Alan

[toc] | [prev] | [next] | [standalone]


#1311802

From"H. Nikolaus Schaller" <hns@goldelico.com>
Date2016-01-18 22:00 +0100
Message-ID<qSp47-5SG-15@gated-at.bofh.it>
In reply to#1311487
Hi Alan,

Am 18.01.2016 um 12:19 schrieb One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>:

>> I have not counted how often I have explained that, but I am happy to do it
>> again.
>> 
>> * the GTA04 device is an open hackable smartphone platform where power saving is highest priority
> 
> Yep I did real the manual before replying last time.

That is good to know, because it allows to refer to it if necessary.

> 
>> * the wi2wi,w2sg0084 chip is a GPS chip (it does not understand HCI protocol, it just sends NMEA records if powered on)
>> * we want to provide NMEA data through /dev/ttyO1 (it uses an OMAP UART) because a tty port is the most common interface for GPS devices (e.g. a bluetooth GPS mouse is also presented as a tty)
>> * we want the chip to automatically power up as soon (but not before) as any gps client opens /dev/ttyO1 (or activates the DTR mctrl)
>> * we want the chip to automatically power down if no process uses /dev/ttyO1 any more
>> The standard logic of GPS daemons and applications is to receive
>> NMEA records through some serial /dev/tty.
>> 
>> Please tell me how power on/off management can be done without intercepting
>> somewhere in the kernel that /dev/ttyO1 is opened/closed (which is not the same
>> as suspend/resume).
> 
> Your user space can do it (as most Android does).

How can it do it in automatically in a standardized way without need for daemon support?

> 
>> Secondly, the chip has a very special logic that it may end up in the opposite
>> power state than the kernel driver thinks. Especially after boot it simply can not
>> know the state. The chip might be powered up/send records or might not.
>> 
>> A driver can only detect such a discrepancy if it thinks the GPS chip is powered off,
>> but there is still data coming through the UART.
> 
> To play devils advocate a moment: so can user space.

* how can it be done without permanently running a daemon that monitors RX data (/dev/tty*)
and attaches to some /sysfs control?
* how can the daemon present another /dev/tty so that applications expecting such
an interface can attach to it (maybe through pty - but isn't that an overkill?)
* Who makes sure that this daemon is installed and running right after boot up on *any* Linux system
so that it can always react in case that the chip did start when it should be off?

More generally: what is a kernel good for? Why do we need kernel drivers?

You can do almost everything in user space if you want to: we can expose
every wire to /sys/class/gpio and clever user space daemons can bitbang on
them to implement any protocol.

Well, only in theory: this is too slow and needs too much energy because the CPU
runs at 100%.

Just think about waking up the daemon process if a character is received. This is
much more costly than calling a notification function in the kernel driver which might
have to execute just 4 or 5 assembler instructions to decide what to do.

So kernel drivers are sometimes the best solution and more efficient and controllable
than a user space daemon.

That is why we propose a kernel solution for this problem.

> 
>> Please tell me how this situation can be detected without monitoring the data stream
>> in the kernel going to /dev/ttyO1 - from the UART behind it - even if /dev/ttyO1 is closed.
>> 
>> This are *our* requirements.
> 
> The Linux kernel runs on billions of devices. If we put kernel hooks in
> random glue layers for every weird little platform corner case it would
> collapse in a heap. Those are *our* requirements. Thus it is good to push
> back on stuff that can be done in user space just as well.

I understand that. But what is Linux good for? For it's own sake or for
users and platforms using it? Isn't it that we take it from the community
and contribute improvements back to it?

And it appears that this is not a little platform corner case but there is some
more general need for drivers to access the UART layer and not a higher level
abstraction. The networking stack also has mechanisms to access all layers
because not everything can be done on every layer (some functions of
lower layers are hidden when accessing higher layers - e.g. you can't
get the Ethernet MAC from the TCP/socket layer) and it might be more
performant to directly go to a lower layer.

So to me it appears that such a kernel feature is missing. Therefore we
are discussing it.

> 
>> Other people think that our approach helps to solve their driver architecture as well
>> and have added their requirements on top. This is why I attempt to make the API
>> more general than just for our own use-cases.
> 
> If it's going to be generic then it needs to be dealing with this at the
> tty/tty_port level not uart. uart isn't a general serial abstraction,
> tty_port and tty are.

well I think most driver projects that have expressed that they want to have
such a solution just want an UART abstraction and not a general tty/serial
interface with all bells and whistles.

> uart is just a helper library for some types of
> port.

Yes, this is the type of port, our peer devices are directly connected to.

> In practise that ought to be a small distinction. If you have to bind
> some kind of device logic to the port activation/deactivation then bind
> it to tty_port not uart. uart open/close is basically an implementation
> of the tty port->ops->activate() and port->ops->shutdown() method.

I am not sure if that still exists for UART based tty_ports (but I am not
understanding everything of the tty layer and have not followed recent
changes since I have focussed on the uart_port and serial-core):

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/drivers/tty/serial/serial_core.c?id=9e31364fc3272073ec8c5fac18a3e01d4f013418

Independently of that, doing the port open/close could work that way, if
it were the only topic to be solved.

But we need several components to make everything we need work.

All of them need to be present.

Another question if we discuss moving the hooks into the tty_port layer:

is it possible from tty layer to activate the UART without a user-space open()
and keep it activated after a close()?

> 
>> 
>>> 
>>> The 8686 is already working in serial mode  with no kernel hackery on
>>> other boards.
>> 
>> You appear to mix the chips we are talking about. We have the w2sg0084 gps
>> chip and a w2cbw003, which is a combo of an 8686 and a CSR BT serial device.
>> 
>> Both chips need somehow to be powered on or off if not used. Ideally automatically
>> at the moment no user space client is using them any more. For the bluetooth side
>> the moment to power off is when a hciattach is killed.
>> 
>> Other 8686 boards appear not to have such critical power restrictions and then they
>> just leave power on.
> 
> The ones I am familiar with either have the userspace managing it via
> sysfs (which has some latency advantages when doing clever stuff) or
> wired the power control to the carrier signal (or that is declared the
> gpio that controls it to be the carrier).

How many of these special drivers are in mainline? Our target is to get full support
by mainline and not run our own kernel branch forever. Because we are not
yet-another-android-thing-that-just-needs-to-work-for-6-months-and-nobody-cares-
about-updates-and-GPL.

In all mainlining attempts I know, the user-space control approach was rejected
because it was said that the kernel should take care of and not a /sysfs or some
other artificial protocol.

Especially as the information when to power on the chip is already known inside the
kernel and does not need a new side-band control mechanism. Other similar (GPS)
devices don't need it as well. This makes our devices unnecessarily a big exception
on user-side, especially as we have proven that it can be solved inside the kernel.

We just have not found an architecture that is accepted by mainline.

Some years ago all this started with this proposal:
* make the driver expose an gpio
* use mctrl-gpios for the DTR line
* connect them in device tree

http://neil.brown.name/blog/20120724060722

This was rejected because a chip driver is not a gpio controller and virtual gpios
are not gpios. In other words: sysfs managing and its related device tree representation
was rejected for mainlining as well.

But anyway, it would not solve the device RX data stream monitoring problem.
It just could be a solution to know when to turn the chip on or off.

Neil's proposal to monitor the RX line was rejected because it was doing
nasty tricks with switching pinmux states and setting a temporary interrupt
on the UART RX line.

Therefore we now want to monitor the RX line "behind" the UART, i.e. on the
byte stream coming from the UART shift registers.

This did lead to tty/uart slaves concept and everybody wanted it. Now as we
have code proposals it should go back to become a user space daemon...

> 
>> No, the opposite: it can be any open source application, which by principle
>> could be changed in any way. But we can't because we as the hardware
>> platform+kernel developers have no (and don't want to have) control over
>> the the user space. So we have to fit into the standards of the user space.
> 
> Well the standard of user space today *is* managing via sysfs or the
> carrier trick. Take a look at all the Android devices out there - they
> all tackle it that way. Some of them do very aggressive power management
> as you can imagine. But yes that's ugly 8)

And these solutions are not mainline. Or is any of these?

> 
>> Here, we need to solve the problem to power down the chip if NO
>> tty port is open any more.
> 
> port->ops->shutdown in the tty layer.
> 
>>> You can just report EBUSY in your open method. You don't need to touch
>>> the serial core layer. It's quite sufficient to do
>>> 
>>> 	if (busy)
>>> 		return -EBUSY;
>>> 
>>> at the top of your uart open method.
>> 
>> Hm. I am not writing a new UART driver or touch them. I am using the existing
>> ones (omap-serial). They all call this uart_add_one_port() in their probe() function.
> 
> At the moment. But they may not, or they may get folded together.

Are they? If you have a clear plan for changes, we can update our hooks to what
you have planned.

Otherwise we are digging in the dark.

> 
>> If I would modify just omap-serial, people would for sure complain that the solution
>> is not generic enough.
> 
> I would say two things
> 
> 1. If you modify omap-serial it's not that generic, but it doesn't mess
> with library code it shouldn't.

And tomorrow comes someone who connects the same chip to an i.MX.
Next week to a Samsung SoC. Every time we add the same patches
to the device specific UART driver.

> That is better than messing with uart
> layer code. It also solves your problem and localises the solution. That
> to me is a win.


Neil had proposed that a while ago and there was code in the kernel, but it
was removed last year, because it is not general enough and we did not
get the driver into mainline that would have used it: 985bfd54

> 
> 2. If not then hook tty_port_shutdown() and tty_port_open() because those
> are the right abstraction point. Everything in the kernel that is a tty
> is a tty_port.

What is in your view the right abstraction point for a peer device driver to get
notified about rx characters (even if the tty is currently not open)?

> 
>>> The uart layer is part of the tty layer. It's just a glue library to make
>>> writing some tty drivers a bit easier.
>> 
>> Yes, and exactly that is helpful to solve this problem. We attach only to the
>> glue layer and everything is done.
> 
> It's the wrong place - it's not the abstraction.

Maybe we think about different levels of abstractions.

I think on a level of abstraction of all the different UART drivers. Which is sufficient
to access the UART by the peer driver. This is why I call it UART-peer (and
not serial or tty peer).

I understand the peer device driver requirements that were mentioned so far,
that they all will be happy with basic UART functionality (receive characters,
send characters, activate/deactivate, set baud rate).

You want to abstract from UART and add all tty bells and whistles. We need them
for the GPS chip's data stream (because our unknown GPS applications might
use them) - but other low level UART peer drivers do not even need them.

> 
> What I am trying to say is that if you do this generically then add the
> needed method calls into tty_port_open and tty_port_shutdown, make them
> run after port->ops->activate and before port->ops->shutdown so there is
> a sensible ordering if you need to do something to the port itself, and
> also so on open it only runs if port->ops->activate succeeded.
> 
> Something like
> 
>       if (!test_bit(ASYNCB_INITIALIZED, &port->flags)) {
>                clear_bit(TTY_IO_ERROR, &tty->flags);
>                if (port->ops->activate) {
>                        int retval = port->ops->activate(port, tty);
>                        if (retval) {
>                                mutex_unlock(&port->mutex);
>                                return retval;
>                        }
>                }
> 		/* Wake the device if we have one tied to us */
> 		if (port->ops->activate_slave)
> 			port->ops->activate_slave(port, tty);
>                set_bit(ASYNCB_INITIALIZED, &port->flags);
> 	}
> 
> Then all you need is the (possibly device specific) small patches to check
> the device tree for the bindings on init, and if so set the port->ops
> methods according to the binding.

For me it appears to only solves a small part of the whole problem.

1. it must be possible to start the UART before any user-space open()
2. the UART must be kept running as long as the peer driver wants to
monitor rx data
3. we need a hook to monitor: that (and which) data is incoming

Another aspect is that on a physical RS232 the DTR line is usually
used to power on/off a remote device (the DCE). Therefore I prefer
to mimic that in software by intercepting the mctrl changes. This
additionally allows a client to turn off power of the chip through a "standard"
protocol, even while the tty file is kept open.

> 
>>> It's tied deeply to the tty_port
>>> implementation. One day it might even cease to exist replaced by more
>>> generic tty_port helpers.
>> 
>> Is there a concrete plan to change that? Is anyone working on it now?
> 
> Nobody wants to lose the ability to do so or to move stuff around.

Indeed. But why would you loose this ability at all?

> 
>> If not, I would not worry about this, because it is not specific to the problem
>> we want to solve today (to be precise: since 3 years).
> 
> You solve it today you block other things for the next 20 years. The
> kernel has to deal in long terms.

i understand what you mean but I don't think we are blocking anything.

This is a too black&white view for me. Especially since we would probably still
have all the APIs of ca. Linux 1.3 (1996) if things were blocked for 20 years
by any new piece of hardware. I am quite sure that a lot of things has been
completely turned upside down in the past 20 years.

If significant architectural changes are to be done, you can deprecate this
API and ask all users (driver authors) to replace it with something new.
And if they aren't adjusted within some time or no new interface is proposed,
they get removed. The *I* have the problem back on my table and not you :)

Isn't this a standard way to handle such situations?

Things are gradually changed every now and then. And other parts have to
follow. BTW: most of my work to update a production kernel to a new -rc1 is
to adjust non-mainline (or not-yet) drivers to such API changes.

So this is daily life for someone who maintains a working kernel for a specific
device and therefore does not come unexpected.

So it is more you blocking an adequate solution for our devices than we blocking
Linux development...

> 
>>> The tty layer is also an *abstract* concept. There is no real world tie
>>> between physical collections of shift registers that dribble bits to one
>>> another and tty devices in the kernel. You only need a tty driver for
>>> certain types of user interaction.
>> 
>> And we need it to process the GPS data by standard applications and tools.
> 
> Ok - no problem with that, and for what you've explained that bit makes
> sense.
> 
> Alan

BR,
Nikolaus

[toc] | [prev] | [next] | [standalone]


#1311819

FromOne Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>
Date2016-01-18 23:10 +0100
Message-ID<qSq9Q-6Tz-7@gated-at.bofh.it>
In reply to#1311802
> > Your user space can do it (as most Android does).
> 
> How can it do it in automatically in a standardized way without need for daemon support?

You don't need to - it can be device specific. In Android it usually is.
I've never understood why low end devices don't also abstract user space
descriptions of power control into DT nodes as well as kernel properties ?

> * how can the daemon present another /dev/tty so that applications expecting such
> an interface can attach to it (maybe through pty - but isn't that an overkill?)

You don't need to - you can monitor the rx/tx stats via procfs too. Not
pretty but given the daemon can do both the decoding and the monitoring
for your device it all seems a strawman. I'd argue given the sheer range
of ways people PM a GPS that you ought to have those power abstractions
in the user space daemon. They might use kernel methods, they might not.

> * Who makes sure that this daemon is installed and running right after boot up on *any* Linux system
> so that it can always react in case that the chip did start when it should be off?

Whoever puts the distribution together. The kernel runs init. Beyond that
we don't care. Not our problem. You can boot into emacs if you want.

> More generally: what is a kernel good for? Why do we need kernel drivers?

To maintain security, to manage hardware elements that cannot be managed
in user space, and to provide abstractions where the abstraction is
necessary and meaningful to a large number of users (not a corner case
phone device unless the abstraction can be made meaningful and widely
useful).

> Just think about waking up the daemon process if a character is received. This is
> much more costly than calling a notification function in the kernel driver which might
> have to execute just 4 or 5 assembler instructions to decide what to do.

The logical continuation of that is of course not to have user space but
write your entire phone device in ring 0. Point being it is always a
trade off.

Your only way currently to do that is to open the tty and set a line
discipline which does your monitoring then hold it open. We can't stack
ldiscs either. If you want to monitor the line state with the physical
uart receiver powered down then this won't work at all.

> So kernel drivers are sometimes the best solution and more efficient and controllable
> than a user space daemon.

Sometimes but not always. And even if a kernel device is the right
solution to your device that doesn't mean it's general purpose enough to
justify being stuffed upstream. Android is full of "interesting" device
specific tweaks the sum of which would sink the kernel project.

> I understand that. But what is Linux good for? For it's own sake or for
> users and platforms using it? Isn't it that we take it from the community
> and contribute improvements back to it?

Only when the community benefits overall.

> So to me it appears that such a kernel feature is missing. Therefore we
> are discussing it.

I'm glad - because it raises some hard questions and while I don't agree
with some of your starting points (like needing to "open" a uart without
user space) I do agree that

- There isn't a nice way to bind extra non device specific behaviour to
  open and close (but we have the right places to add one)

- There isn't a way to monitor rx data (and this is *really* hard to
  sort especially when the uart is powered down)

- Monitoring mctrl via a nice abstraction is going to need the mctrl
  ops moving from tty to tty_port if you want to use that to trigger tty
  slave behaviour. Doable but a change.

> well I think most driver projects that have expressed that they want to have
> such a solution just want an UART abstraction and not a general tty/serial
> interface with all bells and whistles.

There isn't any difference at the proper abstraction layer. If you wanted
your monitor to power on and off when you open a console that's the same
problem space.

> > uart is just a helper library for some types of
> > port.
> 
> Yes, this is the type of port, our peer devices are directly connected to.

Think about the bigger picture. The high level abstraction is the tty and
the tty_port. But see below as I think your mental model is perhaps wrong
and this is a point of confusion ?

> Another question if we discuss moving the hooks into the tty_port layer:
> 
> is it possible from tty layer to activate the UART without a user-space open()
> and keep it activated after a close()?

No the entire tty layer from top to bottom assumes that you activate a
device by opening it from user space. uart, tty, the lot. The only
corner case weird exception is the serial console and that is a horrible
hack for output only - so anything that fixes that assumption should also
be made to fix the console case.

It's never really mattered because the obscure corner cases where you
want a tty held open the cost is minimal because you just let systemd or
similar own the file handle along with everything else it is hanging on
to.

> > The ones I am familiar with either have the userspace managing it via
> > sysfs (which has some latency advantages when doing clever stuff) or
> > wired the power control to the carrier signal (or that is declared the
> > gpio that controls it to be the carrier).
> 
> How many of these special drivers are in mainline? Our target is to get full support
> by mainline and not run our own kernel branch forever. Because we are not
> yet-another-android-thing-that-just-needs-to-work-for-6-months-and-nobody-cares-
> about-updates-and-GPL.

Both of those techniques work in mainline without kernel changes (at
least on devices where the right gpio sysfs nodes exist). Not pretty and
it would be better to abstract it - although ACPI often abstracts it by
having the kernel power down the uart which calls into ACPI which pokes a
GPIO line so presumably devicetree could learn the same tricks if it got
better at describing power management.

> In all mainlining attempts I know, the user-space control approach was rejected
> because it was said that the kernel should take care of and not a /sysfs or some
> other artificial protocol.

I'm not sure who said that but they obviously didn't look at the real
world 8)

> But anyway, it would not solve the device RX data stream monitoring problem.
> It just could be a solution to know when to turn the chip on or off.

This I think is actually the really hard and interesting part of the
problem. The "tell me about open and close" case is simple and can be
done via tty_port today with minimal extra hooks. There is a small
question about how you set those hooks from a DT binding - mostly because
we want the tty_port method table to be const for security reasons.

The data one is much harder because the abstractions in the tty_port
never see data. It goes direct from the hardware interface (possibly via
a support library liek uart but not always) on to the core of the tty
code. It's also a very hot path on things like older USB 3G modems that
use AT commands. Peter has done minor miracles on making it more
efficient but a USB modem can still really stress a small CPU without any
further callbacks appearing.

> Neil's proposal to monitor the RX line was rejected because it was doing
> nasty tricks with switching pinmux states and setting a temporary interrupt
> on the UART RX line.

For some hardware that is the only way I know to do this because the
power hungry uart receiver is physically powered down. I would have to
check but I *think* that is true even on a modern x86 PC that supports
wakeups via serial - although it may be well hidden in ACPI and firmware.

> Therefore we now want to monitor the RX line "behind" the UART, i.e. on the
> byte stream coming from the UART shift registers.
> 
> This did lead to tty/uart slaves concept and everybody wanted it. Now as we
> have code proposals it should go back to become a user space daemon...

I'm not personally opoosed to the tty slave idea providing it ends up
attached to the tty_port not just uart.

> > 2. If not then hook tty_port_shutdown() and tty_port_open() because those
> > are the right abstraction point. Everything in the kernel that is a tty
> > is a tty_port.
> 
> What is in your view the right abstraction point for a peer device driver to get
> notified about rx characters (even if the tty is currently not open)?

I don't think you have one. A lot of hardware has the receiver and
transceiver physically powered off when the tty is closed. You can't even
touch the registers in some cases (eg a PCI port in D3 state).

I certainly have no good ideas for that specific case because apart from
things buried in stuff like ACPI I don't know of any general purpose way
to ask "how do I make some other piece of hardware peek at the level on
that line and/or trigger on rising or falling edges"

> I think on a level of abstraction of all the different UART drivers. Which is sufficient
> to access the UART by the peer driver. This is why I call it UART-peer (and
> not serial or tty peer).

Not all uarts use the uart layer. It's an optional helper.

> You want to abstract from UART and add all tty bells and whistles. We need them
> for the GPS chip's data stream (because our unknown GPS applications might
> use them) - but other low level UART peer drivers do not even need them.

Ok I think the model you have may be wrong.

The lowest level physical interface abstraction in the whole tty stack is
the tty_port. Every object which can be opened as a tty contains a
tty_port, and the tty_port lifetime is the lifetime of the "physical"
interface to which it is attached.

tty_port requires the device provide a set of methods.

uart optionally sits on top of tty_port. It provides the tty_port methods
and provides a differentish set of low level abstractions more suited to
byte oriended serial port devices. Console doesn't use it, sdio serial
doesn't use it, some uarts don't use it, usb doesn't use it (including
on-board HSIC and SSIC devices), most jtag serials don't use it either.

The tty layer talks to the tty_port layer and also directly to tty
methods provided by the physical device. Conveniently for you the
tty_port abstracts opening and closing the device. Inconveniently for you
it doesn't really mediate receiving and sending data although it does hold
the buffers used.

> > Then all you need is the (possibly device specific) small patches to check
> > the device tree for the bindings on init, and if so set the port->ops
> > methods according to the binding.
> 
> For me it appears to only solves a small part of the whole problem.
> 
> 1. it must be possible to start the UART before any user-space open()

There is no tty layer support at any level to do this, nor any reason
I've seen advanced to justify all the extra complexity needed.

> 2. the UART must be kept running as long as the peer driver wants to
> monitor rx data

Correct, otherwise in the general case the uart may be physically powered
off and the kernel will in fact try aggressively to do so.

> 3. we need a hook to monitor: that (and which) data is incoming

That isn't tty_port related - but it's certainly hard.

> Another aspect is that on a physical RS232 the DTR line is usually
> used to power on/off a remote device (the DCE). Therefore I prefer
> to mimic that in software by intercepting the mctrl changes. This
> additionally allows a client to turn off power of the chip through a "standard"
> protocol, even while the tty file is kept open.

The mctrl currently goes via tty->ops->tiocmset()/tiocmget. That IMHO is
actually an oddity and moving it to the tty_port would make sense anyway
because it's state and behaviour that is persistent at hardware level.

I don't know what Peter thinks about that however.

> > Nobody wants to lose the ability to do so or to move stuff around.
> 
> Indeed. But why would you loose this ability at all?

Because all your hooks would break.

> If significant architectural changes are to be done, you can deprecate this
> API and ask all users (driver authors) to replace it with something new.
> And if they aren't adjusted within some time or no new interface is proposed,
> they get removed. The *I* have the problem back on my table and not you :)
> 
> Isn't this a standard way to handle such situations?
> 
> Things are gradually changed every now and then. And other parts have to
> follow. BTW: most of my work to update a production kernel to a new -rc1 is
> to adjust non-mainline (or not-yet) drivers to such API changes.

What usually happens is the creator the tangle has despite best
intentions ended up somewhere else on another project, their users whine
if we break it, someone screams regression and it all gets dumped on the
maintainers.

Alan

[toc] | [prev] | [next] | [standalone]


#1311834

From"H. Nikolaus Schaller" <hns@goldelico.com>
Date2016-01-18 23:40 +0100
Message-ID<qSqCS-79p-13@gated-at.bofh.it>
In reply to#1311819
Just a first short answer (can't work 7/24 :).

Am 18.01.2016 um 23:03 schrieb One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>:

>>> Your user space can do it (as most Android does).
>> 
>> How can it do it in automatically in a standardized way without need for daemon support?
> 
> You don't need to - it can be device specific. In Android it usually is.
> I've never understood why low end devices don't also abstract user space
> descriptions of power control into DT nodes as well as kernel properties ?
> 
>> * how can the daemon present another /dev/tty so that applications expecting such
>> an interface can attach to it (maybe through pty - but isn't that an overkill?)
> 
> You don't need to - you can monitor the rx/tx stats via procfs too. Not
> pretty but given the daemon can do both the decoding and the monitoring
> for your device it all seems a strawman. I'd argue given the sheer range
> of ways people PM a GPS that you ought to have those power abstractions
> in the user space daemon. They might use kernel methods, they might not.
> 
>> * Who makes sure that this daemon is installed and running right after boot up on *any* Linux system
>> so that it can always react in case that the chip did start when it should be off?
> 
> Whoever puts the distribution together. The kernel runs init. Beyond that
> we don't care. Not our problem. You can boot into emacs if you want.

Well, it is my big problem which goes contrary to our goal to have the best
supported platform... We would have to provide and maintain such things
so that they are compatible to a plethora of unknown runtime environments.

> 
> 
>> More generally: what is a kernel good for? Why do we need kernel drivers?
> 
> To maintain security, to manage hardware elements that cannot be managed
> in user space, and to provide abstractions where the abstraction is
> necessary and meaningful to a large number of users (not a corner case
> phone device unless the abstraction can be made meaningful and widely
> useful).
> 
>> Just think about waking up the daemon process if a character is received. This is
>> much more costly than calling a notification function in the kernel driver which might
>> have to execute just 4 or 5 assembler instructions to decide what to do.
> 
> The logical continuation of that is of course not to have user space but
> write your entire phone device in ring 0. Point being it is always a
> trade off.
> 
> Your only way currently to do that is to open the tty and set a line
> discipline which does your monitoring then hold it open. We can't stack
> ldiscs either. If you want to monitor the line state with the physical
> uart receiver powered down then this won't work at all.
> 
>> So kernel drivers are sometimes the best solution and more efficient and controllable
>> than a user space daemon.
> 
> Sometimes but not always. And even if a kernel device is the right
> solution to your device that doesn't mean it's general purpose enough to
> justify being stuffed upstream. Android is full of "interesting" device
> specific tweaks the sum of which would sink the kernel project.
> 
>> I understand that. But what is Linux good for? For it's own sake or for
>> users and platforms using it? Isn't it that we take it from the community
>> and contribute improvements back to it?
> 
> Only when the community benefits overall.
> 
>> So to me it appears that such a kernel feature is missing. Therefore we
>> are discussing it.
> 
> I'm glad - because it raises some hard questions and while I don't agree
> with some of your starting points (like needing to "open" a uart without
> user space) I do agree that
> 
> - There isn't a nice way to bind extra non device specific behaviour to
>  open and close (but we have the right places to add one)
> 
> - There isn't a way to monitor rx data (and this is *really* hard to
>  sort especially when the uart is powered down)

Exactly. This is why we already work 3 years on this topic...

The solution is to optionally keep it powered up - as long as the peer
device asks for.

> 
> - Monitoring mctrl via a nice abstraction is going to need the mctrl
>  ops moving from tty to tty_port if you want to use that to trigger tty
>  slave behaviour. Doable but a change.
> 
>> well I think most driver projects that have expressed that they want to have
>> such a solution just want an UART abstraction and not a general tty/serial
>> interface with all bells and whistles.
> 
> There isn't any difference at the proper abstraction layer. If you wanted
> your monitor to power on and off when you open a console that's the same
> problem space.
> 
>>> uart is just a helper library for some types of
>>> port.
>> 
>> Yes, this is the type of port, our peer devices are directly connected to.
> 
> Think about the bigger picture. The high level abstraction is the tty and
> the tty_port. But see below as I think your mental model is perhaps wrong
> and this is a point of confusion ?
> 
>> Another question if we discuss moving the hooks into the tty_port layer:
>> 
>> is it possible from tty layer to activate the UART without a user-space open()
>> and keep it activated after a close()?
> 
> No the entire tty layer from top to bottom assumes that you activate a
> device by opening it from user space. uart, tty, the lot. The only
> corner case weird exception is the serial console and that is a horrible
> hack for output only - so anything that fixes that assumption should also
> be made to fix the console case.
> 
> It's never really mattered because the obscure corner cases where you
> want a tty held open the cost is minimal because you just let systemd or
> similar own the file handle along with everything else it is hanging on
> to.
> 
>>> The ones I am familiar with either have the userspace managing it via
>>> sysfs (which has some latency advantages when doing clever stuff) or
>>> wired the power control to the carrier signal (or that is declared the
>>> gpio that controls it to be the carrier).
>> 
>> How many of these special drivers are in mainline? Our target is to get full support
>> by mainline and not run our own kernel branch forever. Because we are not
>> yet-another-android-thing-that-just-needs-to-work-for-6-months-and-nobody-cares-
>> about-updates-and-GPL.
> 
> Both of those techniques work in mainline without kernel changes (at
> least on devices where the right gpio sysfs nodes exist). Not pretty and
> it would be better to abstract it - although ACPI often abstracts it by
> having the kernel power down the uart which calls into ACPI which pokes a
> GPIO line so presumably devicetree could learn the same tricks if it got
> better at describing power management.
> 
>> In all mainlining attempts I know, the user-space control approach was rejected
>> because it was said that the kernel should take care of and not a /sysfs or some
>> other artificial protocol.
> 
> I'm not sure who said that but they obviously didn't look at the real
> world 8)
> 
>> But anyway, it would not solve the device RX data stream monitoring problem.
>> It just could be a solution to know when to turn the chip on or off.
> 
> This I think is actually the really hard and interesting part of the
> problem. The "tell me about open and close" case is simple and can be
> done via tty_port today with minimal extra hooks. There is a small
> question about how you set those hooks from a DT binding - mostly because
> we want the tty_port method table to be const for security reasons.
> 
> The data one is much harder because the abstractions in the tty_port
> never see data. It goes direct from the hardware interface (possibly via
> a support library liek uart but not always) on to the core of the tty
> code. It's also a very hot path on things like older USB 3G modems that
> use AT commands. Peter has done minor miracles on making it more
> efficient but a USB modem can still really stress a small CPU without any
> further callbacks appearing.
> 
>> Neil's proposal to monitor the RX line was rejected because it was doing
>> nasty tricks with switching pinmux states and setting a temporary interrupt
>> on the UART RX line.
> 
> For some hardware that is the only way I know to do this because the
> power hungry uart receiver is physically powered down. I would have to
> check but I *think* that is true even on a modern x86 PC that supports
> wakeups via serial - although it may be well hidden in ACPI and firmware.
> 
>> Therefore we now want to monitor the RX line "behind" the UART, i.e. on the
>> byte stream coming from the UART shift registers.
>> 
>> This did lead to tty/uart slaves concept and everybody wanted it. Now as we
>> have code proposals it should go back to become a user space daemon...
> 
> I'm not personally opoosed to the tty slave idea providing it ends up
> attached to the tty_port not just uart.
> 
>>> 2. If not then hook tty_port_shutdown() and tty_port_open() because those
>>> are the right abstraction point. Everything in the kernel that is a tty
>>> is a tty_port.
>> 
>> What is in your view the right abstraction point for a peer device driver to get
>> notified about rx characters (even if the tty is currently not open)?
> 
> I don't think you have one. A lot of hardware has the receiver and
> transceiver physically powered off when the tty is closed. You can't even
> touch the registers in some cases (eg a PCI port in D3 state).

This is why I want to keep the UART open as long as the peer driver
tells to do so.

> 
> I certainly have no good ideas for that specific case because apart from
> things buried in stuff like ACPI I don't know of any general purpose way
> to ask "how do I make some other piece of hardware peek at the level on
> that line and/or trigger on rising or falling edges"
> 
>> I think on a level of abstraction of all the different UART drivers. Which is sufficient
>> to access the UART by the peer driver. This is why I call it UART-peer (and
>> not serial or tty peer).
> 
> Not all uarts use the uart layer. It's an optional helper.
> 
>> You want to abstract from UART and add all tty bells and whistles. We need them
>> for the GPS chip's data stream (because our unknown GPS applications might
>> use them) - but other low level UART peer drivers do not even need them.
> 
> Ok I think the model you have may be wrong.
> 
> The lowest level physical interface abstraction in the whole tty stack is
> the tty_port. Every object which can be opened as a tty contains a
> tty_port, and the tty_port lifetime is the lifetime of the "physical"
> interface to which it is attached.
> 
> tty_port requires the device provide a set of methods.
> 
> uart optionally sits on top of tty_port. It provides the tty_port methods
> and provides a differentish set of low level abstractions more suited to
> byte oriended serial port devices. Console doesn't use it, sdio serial
> doesn't use it, some uarts don't use it, usb doesn't use it (including
> on-board HSIC and SSIC devices), most jtag serials don't use it either.
> 
> The tty layer talks to the tty_port layer and also directly to tty
> methods provided by the physical device.

Ok, I see. My picture was more a layer scheme based on the path how
data flows from the hardware specific uart driver to the uart layer and
then to the tty_port things.

> Conveniently for you the
> tty_port abstracts opening and closing the device. Inconveniently for you
> it doesn't really mediate receiving and sending data although it does hold
> the buffers used.

Yes.

> 
>>> Then all you need is the (possibly device specific) small patches to check
>>> the device tree for the bindings on init, and if so set the port->ops
>>> methods according to the binding.
>> 
>> For me it appears to only solves a small part of the whole problem.
>> 
>> 1. it must be possible to start the UART before any user-space open()
> 
> There is no tty layer support at any level to do this, nor any reason
> I've seen advanced to justify all the extra complexity needed.

This is why I want to do it on the uart_port.

> 
>> 2. the UART must be kept running as long as the peer driver wants to
>> monitor rx data
> 
> Correct, otherwise in the general case the uart may be physically powered
> off and the kernel will in fact try aggressively to do so.
> 
>> 3. we need a hook to monitor: that (and which) data is incoming
> 
> That isn't tty_port related - but it's certainly hard.

Not, if we do it on uart_port level... It already works with two not very
big patches.

> 
>> Another aspect is that on a physical RS232 the DTR line is usually
>> used to power on/off a remote device (the DCE). Therefore I prefer
>> to mimic that in software by intercepting the mctrl changes. This
>> additionally allows a client to turn off power of the chip through a "standard"
>> protocol, even while the tty file is kept open.
> 
> The mctrl currently goes via tty->ops->tiocmset()/tiocmget. That IMHO is
> actually an oddity and moving it to the tty_port would make sense anyway
> because it's state and behaviour that is persistent at hardware level.
> 
> I don't know what Peter thinks about that however.
> 
>>> Nobody wants to lose the ability to do so or to move stuff around.
>> 
>> Indeed. But why would you loose this ability at all?
> 
> Because all your hooks would break.

Only if you do big changes. That is why I ask if such big changes are planned
or can be anticipated. They probably don't come unexpectedly around the corner.

And if you reject other changes like ours, there won't be any in the next 20 years :)

> 
>> If significant architectural changes are to be done, you can deprecate this
>> API and ask all users (driver authors) to replace it with something new.
>> And if they aren't adjusted within some time or no new interface is proposed,
>> they get removed. The *I* have the problem back on my table and not you :)
>> 
>> Isn't this a standard way to handle such situations?
>> 
>> Things are gradually changed every now and then. And other parts have to
>> follow. BTW: most of my work to update a production kernel to a new -rc1 is
>> to adjust non-mainline (or not-yet) drivers to such API changes.
> 
> What usually happens is the creator the tangle has despite best
> intentions ended up somewhere else on another project, their users whine
> if we break it, someone screams regression and it all gets dumped on the
> maintainers.

That is of course something we want to avoid. This is why I persistently try
to discuss it with you every detail. Fortunately a good discussion has now started.

And it looks that we either need a kernel solution or a user space solution - but
none makes the responsible party happy... Although that is not a criterion for
a whole system architecture. There the more efficient solution should be done.
Efficient in terms of LOC, speed, maintenance requirements.

BR & GN,
Nikolaus

[toc] | [prev] | [next] | [standalone]


#1312282

FromOne Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>
Date2016-01-19 15:30 +0100
Message-ID<qSFse-Eb-17@gated-at.bofh.it>
In reply to#1311834
> > Whoever puts the distribution together. The kernel runs init. Beyond that
> > we don't care. Not our problem. You can boot into emacs if you want.
> 
> Well, it is my big problem which goes contrary to our goal to have the best
> supported platform... We would have to provide and maintain such things
> so that they are compatible to a plethora of unknown runtime environments.

Every platform does this and has to do this. GTA04 userspace is different
to Raspberry PI userspace is different to PC userspace etc. Your graphics
is different to other people, you don't have a keyboard. All these
require there is some customisation going on.

> > - There isn't a nice way to bind extra non device specific behaviour to
> >  open and close (but we have the right places to add one)
> > 
> > - There isn't a way to monitor rx data (and this is *really* hard to
> >  sort especially when the uart is powered down)
> 
> Exactly. This is why we already work 3 years on this topic...
> 
> The solution is to optionally keep it powered up - as long as the peer
> device asks for.

That won't work on a lot of platforms. They need to power down the uart
to get the power savings and they expect some kind of other monitor like
GPIO. Eg some x86 power states are not achievable on certain SoCs with the
uart powered up. We are already using GPIO triggers on lots of devices,
even if people haven't noticed what is going on because the firmware
hides it all or it's done in user space on the device.

For that to work generically we would need a way to go from a serial port
to a gpio or other monitor setting, described via ACPI and/or DT. We'd
also need a way to open a port in powered off mode, or perhaps to be able
to make open() block for an event on the downed port (just like today you
can block for carrier), but do it before bringing the port active and thus
powering it on.

I don't like the open case because it then means you can't use poll() on
a set of ports to wait cleanly for an event to power them on but the
alternative is really hard because you would have to know that no other
thread of execution or IRQ handler or timer for the port could touch the
hardware

- you were the only thread of execution in the driver
- you held sufficient locks to prevent any other thread of execution
  entering your tty code and touching the hardware

and then in effect do

	tty_port_shutdown	// power down
	port->ops->monitor_rx() (some new method)
	tty_port_open		// power up

We don't normally get into the situation where we have a userspace or
kernel reference open to a device which may be physically powered off.

In that sense having the gpio monitoring separate and the relationship
described to user space (or to some gpio/monitoring driver) by DT is a
lot cleaner IMHO.

The open case can be made to work so that opening the tty can block with
it powered off until an event happens, then powers up the uart. That one
would be doable.

> >> What is in your view the right abstraction point for a peer device driver to get
> >> notified about rx characters (even if the tty is currently not open)?
> > 
> > I don't think you have one. A lot of hardware has the receiver and
> > transceiver physically powered off when the tty is closed. You can't even
> > touch the registers in some cases (eg a PCI port in D3 state).
> 
> This is why I want to keep the UART open as long as the peer driver
> tells to do so.

For the cases that works this is the same as user space keeping the tty
open. You might want a peer driver just to avoid continually waking up
user space, but that could probably also be an ldisc.
 
> >> For me it appears to only solves a small part of the whole problem.
> >> 
> >> 1. it must be possible to start the UART before any user-space open()
> > 
> > There is no tty layer support at any level to do this, nor any reason
> > I've seen advanced to justify all the extra complexity needed.
> 
> This is why I want to do it on the uart_port.

The problem is pretty much the same for uart and tty_port. There is also
no reason you've yet given that to me justifies need to open it from
kernel space.

However doing the open/close hooks is the same in both cases. The termios
one for modem state monitoring is a tiny shift of a method between two
places (and the ripple of the change through all the drivers). The one
that causes all the trouble is monitoring receive data because that goes
directly via the tty flip buffers if the device is powered on, so there
is nowhere to intercept it cleanly. Not only that but when the port is
present but not bound to a userspace tty most devices don't even read the
bytes from the hardware interface and queue them.

If we take your patch then

The hook to uart_update_mctrl becomes a hook to the tty_port termios
method (which right now is for historical reasons still in tty but can
move)

The hook to uart_port_startup becomes the hook I posted to the
tty_port_open method. Probably the method needs to be a
tty_port->slave_ops and not tty_port->ops because ops is const and shared
between physical interfaces. That's a detail.

The hook to uart_port_shutdown becomes a hook to tty_port_shutdown.

Those three are fairly trivial.

Your patch to uart_insert_char doesn't work anyway as that is an
*optional* helper that only some uart drivers use.

That would mean
- Firmware describes the port relationship
- The firmware parsing installs the tty_port->slave_ops pointer at boot
  time or when the module for the serial port is loaded. (Care needed for
  serial console that gets one), and does the right thing on hot
  unplugging, and has the right locking for unplug while scanning

that is much as you have in the patch now, but with the right abstraction
layer and I think some locking tweaks.

> >> 2. the UART must be kept running as long as the peer driver wants to
> >> monitor rx data
> > 
> > Correct, otherwise in the general case the uart may be physically powered
> > off and the kernel will in fact try aggressively to do so.
> > 
> >> 3. we need a hook to monitor: that (and which) data is incoming
> > 
> > That isn't tty_port related - but it's certainly hard.
> 
> Not, if we do it on uart_port level... It already works with two not very
> big patches.

If you require the uart to be powered up then your "peer" is just a line
discipline pushed onto the port by a userspace process holding the tty
open.

So I'd do

	open /dev/ttyWhatever
	set the ldisc to a new "don't wake me until X happens" ldisc
	poll (blocks until X happens)
	set the ldisc to N_TTY
	do stuff
	close

> > Because all your hooks would break.
> 
> Only if you do big changes. That is why I ask if such big changes are planned
> or can be anticipated. They probably don't come unexpectedly around the corner.

They often do. 

> And it looks that we either need a kernel solution or a user space solution - but
> none makes the responsible party happy... Although that is not a criterion for
> a whole system architecture. There the more efficient solution should be done.
> Efficient in terms of LOC, speed, maintenance requirements.

As measured for the sum of the kernel community not just Gta04 users.

Alan

[toc] | [prev] | [next] | [standalone]


#1313403

From"H. Nikolaus Schaller" <hns@goldelico.com>
Date2016-01-20 18:40 +0100
Message-ID<qT4TE-1oo-11@gated-at.bofh.it>
In reply to#1312282
Next mail.

Am 19.01.2016 um 15:25 schrieb One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>:

>>> Whoever puts the distribution together. The kernel runs init. Beyond that
>>> we don't care. Not our problem. You can boot into emacs if you want.
>> 
>> Well, it is my big problem which goes contrary to our goal to have the best
>> supported platform... We would have to provide and maintain such things
>> so that they are compatible to a plethora of unknown runtime environments.
> 
> Every platform does this and has to do this. GTA04 userspace is different
> to Raspberry PI userspace is different to PC userspace etc. Your graphics
> is different to other people, you don't have a keyboard. All these
> require there is some customisation going on.

Not necessarily. If it is well designed and every problem is solved at the right
place.

In the case of our GPS chip some customization is already there in standard
user space code. Users can just choose or configure some /dev/tty name for
their standard navigation apps and daemons and it everything else should be
done by the kernel.

Neil did describe the goal long ago:

http://neil.brown.name/blog/20120724060722

> 
>>> - There isn't a nice way to bind extra non device specific behaviour to
>>> open and close (but we have the right places to add one)
>>> 
>>> - There isn't a way to monitor rx data (and this is *really* hard to
>>> sort especially when the uart is powered down)
>> 
>> Exactly. This is why we already work 3 years on this topic...
>> 
>> The solution is to optionally keep it powered up - as long as the peer
>> device asks for.
> 
> That won't work on a lot of platforms. They need to power down the uart
> to get the power savings and they expect some kind of other monitor like
> GPIO. Eg some x86 power states are not achievable on certain SoCs with the
> uart powered up. We are already using GPIO triggers on lots of devices,
> even if people haven't noticed what is going on because the firmware
> hides it all or it's done in user space on the device.

Our proposal is completely optional. If there is no UART peer, everything
runs as it does today. Only in the case the device driver needs the UART
to stay powered on it remains powered on.

> 
> For that to work generically we would need a way to go from a serial port
> to a gpio or other monitor setting, described via ACPI and/or DT.

In other words you ask for a device driver... A device driver that knows
which serial port and which gpio. And its exact setting are defined by DT.

This is exactly what we propose. We just need the connection from a
serial interface to that driver to do the monitoring.

> We'd
> also need a way to open a port in powered off mode, or perhaps to be able
> to make open() block for an event on the downed port (just like today you
> can block for carrier), but do it before bringing the port active and thus
> powering it on.
> 
> I don't like the open case because it then means you can't use poll() on
> a set of ports to wait cleanly for an event to power them on but the
> alternative is really hard because you would have to know that no other
> thread of execution or IRQ handler or timer for the port could touch the
> hardware
> 
> - you were the only thread of execution in the driver
> - you held sufficient locks to prevent any other thread of execution
>  entering your tty code and touching the hardware
> 
> and then in effect do
> 
> 	tty_port_shutdown	// power down
> 	port->ops->monitor_rx() (some new method)
> 	tty_port_open		// power up
> 
> We don't normally get into the situation where we have a userspace or
> kernel reference open to a device which may be physically powered off.
> 
> In that sense having the gpio monitoring separate and the relationship
> described to user space (or to some gpio/monitoring driver) by DT is a
> lot cleaner IMHO.
> 
> The open case can be made to work so that opening the tty can block with
> it powered off until an event happens, then powers up the uart. That one
> would be doable.

In our solution it simply powers up the UART and then it sets the mctrl DTR line
This will make the driver wake up the device. I.e. the problem you describe
does not exist.

> 
>>>> What is in your view the right abstraction point for a peer device driver to get
>>>> notified about rx characters (even if the tty is currently not open)?
>>> 
>>> I don't think you have one. A lot of hardware has the receiver and
>>> transceiver physically powered off when the tty is closed. You can't even
>>> touch the registers in some cases (eg a PCI port in D3

Ok, I think it is time to summarize (a little exaggerated) the discussion:

1. technical side:
* you prefer to attach to the tty layer
* you can solve the open/close issue
* you can not solve the rx monitoring issue
* we attach to the uart_port layer
* we have a working solution
* we avoid to handle rx monitoring and rx data processing separately

2. non-techncial side ("belief" level)
* you see the GTA04 as a small corner case which is not worth considering
* you do not account for others who have expressed that they are looking for something very similar
* you point out alternatives:
	* user space daemon
	* /sysfs control
	* virtual gpio
  but we are sitting between all chairs because all subsystems are directing us elsewhere
* you don't wan't anyone to touch uart_port because it may get magically removed
* you don't want to accept new interfaces since you fear that nobody maintains them
* and deprecating is also not a way to go because users will whine instead of following

This is impossible to argue against on technical level (except for the alternatives
because they are differently power efficient and fast - but they are also sort of
"beliefs" unless someone really benchmarks them).

Unfortunately how this discussion turned, breaks my believe that our project is something
exceptional and really welcome :(

Especially as I don't understand your role in this discussion (you are not listed as a MAINTAINER
of tty/serial-core).

And we see that almost all other smartphone projects who use Linux don't even think about
mainlining their things (just a note: our special Community is called "Open Pho(n)e (w. Li)nux"
and therefore *is* Linux (=kernel) based).

They just *use* Linux but don't contribute. If we are happy, they publish the sources
and some volunteer takes the time to integrate it into mainline. But may face the
same barriers as we do.

How this discussion went is a little disillusioning for me about the openness of
open source projects. It is open to discuss, but needs too much energy to really
contribute to the official project. Energy that is urgently missing elsewhere.

So I am now thinking about a third option (because daemon is not a good
option for me): we run our own kernel branch where we maintain these UART peer
patches. And rebase on top of every Linux release that comes. So it will
be available and maintained by the GTA04 project. But not on kernel.org.

That this is the outcome makes me sad because our dream is to be able
to say to everyone who wants "the" kernel: look at kernel.org. But this needs
acceptance. We were not able to get over 3 years of trying.

Going this way saves me a lot of enthusiasm to be put into pushing forwards
other things. And is not much work to maintain. Much less than rewriting the
code according to your proposals.

From time to time we will tease LKML that there is something. And maybe
as time passes, you might reconsider everything.

BR and thanks for your PoV,
Nikolaus

[toc] | [prev] | [next] | [standalone]


#1313331

From"H. Nikolaus Schaller" <hns@goldelico.com>
Date2016-01-20 17:20 +0100
Message-ID<qT3Ee-yP-17@gated-at.bofh.it>
In reply to#1311834
Hi Alan,
here the missing answers.

Am 18.01.2016 um 23:32 schrieb H. Nikolaus Schaller <hns@goldelico.com>:

> Just a first short answer (can't work 7/24 :).
> 
> Am 18.01.2016 um 23:03 schrieb One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>:
> 
>>>> Your user space can do it (as most Android does).
>>> 
>>> How can it do it in automatically in a standardized way without need for daemon support?
>> 
>> You don't need to - it can be device specific. In Android it usually is.
>> I've never understood why low end devices don't also abstract user space
>> descriptions of power control into DT nodes as well as kernel properties ?

I don't know.

But what I know is that I don't want to solve someone else's problems...

>> 
>>> * how can the daemon present another /dev/tty so that applications expecting such
>>> an interface can attach to it (maybe through pty - but isn't that an overkill?)
>> 
>> You don't need to - you can monitor the rx/tx stats via procfs too. Not
>> pretty but given the daemon can do both the decoding and the monitoring
>> for your device it all seems a strawman. I'd argue given the sheer range
>> of ways people PM a GPS that you ought to have those power abstractions
>> in the user space daemon. They might use kernel methods, they might not.

The problem is that *I* have no control over user space. But I also don't want
to say to my users "that is not my problem - get it solved yourself". This does
not help them.

Especially as I can help them with the kernel based solutions we have.

>> 
>>> * Who makes sure that this daemon is installed and running right after boot up on *any* Linux system
>>> so that it can always react in case that the chip did start when it should be off?
>> 
>> Whoever puts the distribution together. The kernel runs init. Beyond that
>> we don't care. Not our problem. You can boot into emacs if you want.
> 
> Well, it is my big problem which goes contrary to our goal to have the best
> supported platform... We would have to provide and maintain such things
> so that they are compatible to a plethora of unknown runtime environments.
> 
>> 
>> 
>>> More generally: what is a kernel good for? Why do we need kernel drivers?
>> 
>> To maintain security, to manage hardware elements that cannot be managed
>> in user space,

Ahem. Most hardware elements can be manages in user space. Use FUSE for
everything and you will only need 10% of the kernel code.

So why are you always arguing with exaggerations?

>> and to provide abstractions where the abstraction is
>> necessary and meaningful to a large number of users (not a corner case
>> phone device

well, phone devices in general outnumber all other devices in sheer numbers.

Yes, the GTA04 has low quantities. But the main hurdle I always hear is that
the kernel is not good enough.

>> unless the abstraction can be made meaningful and widely
>> useful).

And, most device drivers are corner cases since they are special solutions
for singular platforms.

>> 
>>> Just think about waking up the daemon process if a character is received. This is
>>> much more costly than calling a notification function in the kernel driver which might
>>> have to execute just 4 or 5 assembler instructions to decide what to do.
>> 
>> The logical continuation of that is of course not to have user space but
>> write your entire phone device in ring 0. Point being it is always a
>> trade off.
>> 
>> Your only way currently to do that is to open the tty and set a line
>> discipline which does your monitoring then hold it open. We can't stack
>> ldiscs either. If you want to monitor the line state with the physical
>> uart receiver powered down then this won't work at all.
>> 
>>> So kernel drivers are sometimes the best solution and more efficient and controllable
>>> than a user space daemon.
>> 
>> Sometimes but not always.

I am only interested in this case and here it obviously is.

>> And even if a kernel device is the right
>> solution to your device that doesn't mean it's general purpose enough to
>> justify being stuffed upstream. Android is full of "interesting" device
>> specific tweaks the sum of which would sink the kernel project.

This is of course an argument which we have to overcome.

>> 
>>> I understand that. But what is Linux good for? For it's own sake or for
>>> users and platforms using it? Isn't it that we take it from the community
>>> and contribute improvements back to it?
>> 
>> Only when the community benefits overall.

Yes, it benefits because there are several requests for such tty/uart slaves/peers.
From this I deduce that there is community demand.

I brought up this discussion again, because Andrey and Tomeu recently asked
for progress.

>> 
>>> So to me it appears that such a kernel feature is missing. Therefore we
>>> are discussing it.
>> 
>> I'm glad - because it raises some hard questions and while I don't agree
>> with some of your starting points (like needing to "open" a uart without
>> user space

If have an idea how to turn off the device at boot time, before any user space
daemon is running, we can of course ignore that.

>> ) I do agree that
>> 
>> - There isn't a nice way to bind extra non device specific behaviour to
>> open and close (but we have the right places to add one)
>> 
>> - There isn't a way to monitor rx data (and this is *really* hard to
>> sort especially when the uart is powered down)
> 
> Exactly. This is why we already work 3 years on this topic...
> 
> The solution is to optionally keep it powered up - as long as the peer
> device asks for.
> 
>> 
>> - Monitoring mctrl via a nice abstraction is going to need the mctrl
>> ops moving from tty to tty_port if you want to use that to trigger tty
>> slave behaviour. Doable but a change.

Yes, any new function or improvement is a change.

>> 
>>> well I think most driver projects that have expressed that they want to have
>>> such a solution just want an UART abstraction and not a general tty/serial
>>> interface with all bells and whistles.
>> 
>> There isn't any difference at the proper abstraction layer. If you wanted
>> your monitor to power on and off when you open a console that's the same
>> problem space.
>> 
>>>> uart is just a helper library for some types of
>>>> port.
>>> 
>>> Yes, this is the type of port, our peer devices are directly connected to.
>> 
>> Think about the bigger picture. The high level abstraction is the tty and
>> the tty_port.

Yes, it is indeed.

>> But see below as I think your mental model is perhaps wrong
>> and this is a point of confusion ?

Maybe you do not accept that I want to keep as low level as reasonable (for me).

>> 
>>> Another question if we discuss moving the hooks into the tty_port layer:
>>> 
>>> is it possible from tty layer to activate the UART without a user-space open()
>>> and keep it activated after a close()?
>> 
>> No the entire tty layer from top to bottom assumes that you activate a
>> device by opening it from user space. uart, tty, the lot. The only
>> corner case weird exception is the serial console and that is a horrible
>> hack for output only - so anything that fixes that assumption should also
>> be made to fix the console case.

Reasonable.

>> 
>> It's never really mattered because the obscure corner cases where you
>> want a tty held open the cost is minimal because you just let systemd or
>> similar own the file handle along with everything else it is hanging on
>> to.
>> 
>>>> The ones I am familiar with either have the userspace managing it via
>>>> sysfs (which has some latency advantages when doing clever stuff) or
>>>> wired the power control to the carrier signal (or that is declared the
>>>> gpio that controls it to be the carrier).
>>> 
>>> How many of these special drivers are in mainline? Our target is to get full support
>>> by mainline and not run our own kernel branch forever. Because we are not
>>> yet-another-android-thing-that-just-needs-to-work-for-6-months-and-nobody-cares-
>>> about-updates-and-GPL.
>> 
>> Both of those techniques work in mainline without kernel changes (at
>> least on devices where the right gpio sysfs nodes exist

they do not exist...

>> ). Not pretty and
>> it would be better to abstract it - although ACPI often abstracts it by
>> having the kernel power down the uart which calls into ACPI which pokes a
>> GPIO line so presumably devicetree could learn the same tricks if it got
>> better at describing power management.

I do not want to wait until that happens.

>> 
>>> In all mainlining attempts I know, the user-space control approach was rejected
>>> because it was said that the kernel should take care of and not a /sysfs or some
>>> other artificial protocol.
>> 
>> I'm not sure who said that but they obviously didn't look at the real
>> world 8)
>> 
>>> But anyway, it would not solve the device RX data stream monitoring problem.
>>> It just could be a solution to know when to turn the chip on or off.
>> 
>> This I think is actually the really hard and interesting part of the
>> problem. The "tell me about open and close" case is simple and can be
>> done via tty_port today with minimal extra hooks. There is a small
>> question about how you set those hooks from a DT binding

tty has no binding. An UART hardware has. Another reason for me to
start with UARTs.

>> - mostly because
>> we want the tty_port method table to be const for security reasons.
>> 
>> The data one is much harder because the abstractions in the tty_port
>> never see data. It goes direct from the hardware interface (possibly via
>> a support library liek uart but not always) on to the core of the tty
>> code. It's also a very hot path on things like older USB 3G modems that
>> use AT commands. Peter has done minor miracles on making it more
>> efficient but a USB modem can still really stress a small CPU without any
>> further callbacks appearing.
>> 
>>> Neil's proposal to monitor the RX line was rejected because it was doing
>>> nasty tricks with switching pinmux states and setting a temporary interrupt
>>> on the UART RX line.
>> 
>> For some hardware that is the only way I know to do this because the
>> power hungry uart receiver is physically powered down. I would have to
>> check but I *think* that is true even on a modern x86 PC that supports
>> wakeups via serial - although it may be well hidden in ACPI and firmware.

Yes, agreed. But the gpio + interrupt solution was not mainlineable as well.

>> 
>>> Therefore we now want to monitor the RX line "behind" the UART, i.e. on the
>>> byte stream coming from the UART shift registers.
>>> 
>>> This did lead to tty/uart slaves concept and everybody wanted it. Now as we
>>> have code proposals it should go back to become a user space daemon...
>> 
>> I'm not personally opoosed to the tty slave idea providing it ends up
>> attached to the tty_port not just uart.

Well if you can tell us how to handle the data path I have no problems with it
to attach to the tty level.

>> 
>>>> 2. If not then hook tty_port_shutdown() and tty_port_open() because those
>>>> are the right abstraction point. Everything in the kernel that is a tty
>>>> is a tty_port.
>>> 
>>> What is in your view the right abstraction point for a peer device driver to get
>>> notified about rx characters (even if the tty is currently not open)?
>> 
>> I don't think you have one. A lot of hardware has the receiver and
>> transceiver physically powered off when the tty is closed. You can't even
>> touch the registers in some cases (eg a PCI port in D3 state).
> 
> This is why I want to keep the UART open as long as the peer driver
> tells to do so.

If there is no peer driver or the peer drivers does not need the UART
it is exactly the same as today.

> 
>> 
>> I certainly have no good ideas for that specific case because apart from
>> things buried in stuff like ACPI I don't know of any general purpose way
>> to ask "how do I make some other piece of hardware peek at the level on
>> that line and/or trigger on rising or falling edges"
>> 
>>> I think on a level of abstraction of all the different UART drivers. Which is sufficient
>>> to access the UART by the peer driver. This is why I call it UART-peer (and
>>> not serial or tty peer).
>> 
>> Not all uarts use the uart layer. It's an optional helper.

I have researched and it appears that struct uart_port originally came in 2002
in release 2.4 for the sa1100 uart. Since then it has been extended significantly
and is now used by some uart hardware drivers.

But this is again a black&white argument. You recommended to be either general
or patch the omap-serial driver only that we have. I said that i want it a little
more general - which does not mean that I want to rewrite everything. I.e.
for my purposes it would suffice to support those UARTs that already use the
uart_port layer. If a hardware doesn't use it, it could be useful to rewrite the
UART device driver as well. But that is not something I feel I must do and
not now.

>> 
>>> You want to abstract from UART and add all tty bells and whistles. We need them
>>> for the GPS chip's data stream (because our unknown GPS applications might
>>> use them) - but other low level UART peer drivers do not even need them.
>> 
>> Ok I think the model you have may be wrong.
>> 
>> The lowest level physical interface abstraction in the whole tty stack is
>> the tty_port. Every object which can be opened as a tty contains a
>> tty_port, and the tty_port lifetime is the lifetime of the "physical"
>> interface to which it is attached.
>> 
>> tty_port requires the device provide a set of methods.
>> 
>> uart optionally sits on top of tty_port. It provides the tty_port methods
>> and provides a differentish set of low level abstractions more suited to
>> byte oriended serial port devices. Console doesn't use it, sdio serial
>> doesn't use it, some uarts don't use it, usb doesn't use it (including
>> on-board HSIC and SSIC devices), most jtag serials don't use it either.
>> 
>> The tty layer talks to the tty_port layer and also directly to tty
>> methods provided by the physical device.
> 
> Ok, I see. My picture was more a layer scheme based on the path how
> data flows from the hardware specific uart driver to the uart layer and
> then to the tty_port things.
> 
>> Conveniently for you the
>> tty_port abstracts opening and closing the device. Inconveniently for you
>> it doesn't really mediate receiving and sending data although it does hold
>> the buffers used.
> 
> Yes.
> 
>> 
>>>> Then all you need is the (possibly device specific) small patches to check
>>>> the device tree for the bindings on init, and if so set the port->ops
>>>> methods according to the binding.
>>> 
>>> For me it appears to only solves a small part of the whole problem.
>>> 
>>> 1. it must be possible to start the UART before any user-space open()
>> 
>> There is no tty layer support at any level to do this, nor any reason
>> I've seen advanced to justify all the extra complexity needed.
> 
> This is why I want to do it on the uart_port.
> 
>> 
>>> 2. the UART must be kept running as long as the peer driver wants to
>>> monitor rx data
>> 
>> Correct, otherwise in the general case the uart may be physically powered
>> off and the kernel will in fact try aggressively to do so.
>> 
>>> 3. we need a hook to monitor: that (and which) data is incoming
>> 
>> That isn't tty_port related - but it's certainly hard.
> 
> Not, if we do it on uart_port level... It already works with two not very
> big patches.
> 
>> 
>>> Another aspect is that on a physical RS232 the DTR line is usually
>>> used to power on/off a remote device (the DCE). Therefore I prefer
>>> to mimic that in software by intercepting the mctrl changes. This
>>> additionally allows a client to turn off power of the chip through a "standard"
>>> protocol, even while the tty file is kept open.
>> 
>> The mctrl currently goes via tty->ops->tiocmset()/tiocmget. That IMHO is
>> actually an oddity and moving it to the tty_port would make sense anyway
>> because it's state and behaviour that is persistent at hardware level.
>> 
>> I don't know what Peter thinks about that however.
>> 
>>>> Nobody wants to lose the ability to do so or to move stuff around.
>>> 
>>> Indeed. But why would you loose this ability at all?
>> 
>> Because all your hooks would break.
> 
> Only if you do big changes. That is why I ask if such big changes are planned
> or can be anticipated. They probably don't come unexpectedly around the corner.
> 
> And if you reject other changes like ours, there won't be any in the next 20 years :)
> 
>> 
>>> If significant architectural changes are to be done, you can deprecate this
>>> API and ask all users (driver authors) to replace it with something new.
>>> And if they aren't adjusted within some time or no new interface is proposed,
>>> they get removed. The *I* have the problem back on my table and not you :)
>>> 
>>> Isn't this a standard way to handle such situations?
>>> 
>>> Things are gradually changed every now and then. And other parts have to
>>> follow. BTW: most of my work to update a production kernel to a new -rc1 is
>>> to adjust non-mainline (or not-yet) drivers to such API changes.
>> 
>> What usually happens is the creator the tangle has despite best
>> intentions ended up somewhere else on another project, their users whine
>> if we break it, someone screams regression and it all gets dumped on the
>> maintainers.
> 
> That is of course something we want to avoid. This is why I persistently try
> to discuss it with you every detail. Fortunately a good discussion has now started.
> 
> And it looks that we either need a kernel solution or a user space solution - but
> none makes the responsible party happy... Although that is not a criterion for
> a whole system architecture. There the more efficient solution should be done.
> Efficient in terms of LOC, speed, maintenance requirements.

BR,
Nikolaus

[toc] | [prev] | [next] | [standalone]


#1313413

FromOne Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>
Date2016-01-20 18:50 +0100
Message-ID<qT53k-1s5-17@gated-at.bofh.it>
In reply to#1313331
> The problem is that *I* have no control over user space. But I also don't want
> to say to my users "that is not my problem - get it solved yourself". This does
> not help them.

Stuffing things into the kernel because the user space of a given
platform can't get itself organised isn't helpful to the other billion
plus Linux devices out there.

> And, most device drivers are corner cases since they are special solutions
> for singular platforms.

Actually that is quite a small percentage - and the corner cases hide in
drivers not in the core code, which is really important for
maintainability.

> >> I'm glad - because it raises some hard questions and while I don't agree
> >> with some of your starting points (like needing to "open" a uart without
> >> user space
> 
> If have an idea how to turn off the device at boot time, before any user space
> daemon is running, we can of course ignore that.

Your early user space is responsible for it. If you can't accept that
then I don't see any point continuing the conversation.

> >> But see below as I think your mental model is perhaps wrong
> >> and this is a point of confusion ?
> 
> Maybe you do not accept that I want to keep as low level as reasonable (for me).

It's always "for me". No the kernel project is not "for me"

> >> Both of those techniques work in mainline without kernel changes (at
> >> least on devices where the right gpio sysfs nodes exist
> 
> they do not exist...

For most they do because they are gpio lines so exportable to userspace.

> >> This I think is actually the really hard and interesting part of the
> >> problem. The "tell me about open and close" case is simple and can be
> >> done via tty_port today with minimal extra hooks. There is a small
> >> question about how you set those hooks from a DT binding
> 
> tty has no binding. An UART hardware has. Another reason for me to
> start with UARTs.

Every uart is a tty_port, every non uart is a tty_port. There is no
reason you can't bind to a non uart device. Your current patches create
bindings for the uart layer.

> >> For some hardware that is the only way I know to do this because the
> >> power hungry uart receiver is physically powered down. I would have to
> >> check but I *think* that is true even on a modern x86 PC that supports
> >> wakeups via serial - although it may be well hidden in ACPI and firmware.
> 
> Yes, agreed. But the gpio + interrupt solution was not mainlineable as well.

That I am unsure about - at some point it is going to have to be sorted
because it is increasingly common (if currently mostly invisible)
 
> >> I'm not personally opoosed to the tty slave idea providing it ends up
> >> attached to the tty_port not just uart.
> 
> Well if you can tell us how to handle the data path I have no problems with it
> to attach to the tty level.

If your port is closed you have no data path. If you are using uart you
have no data path because while your patch hooks a helper that some uarts
use some of the time it's optional and a lot of uarts don't use it, so
its not even uart generic.

Alan

[toc] | [prev] | [next] | [standalone]


#1313421

From"H. Nikolaus Schaller" <hns@goldelico.com>
Date2016-01-20 19:10 +0100
Message-ID<qT5mG-1Ph-9@gated-at.bofh.it>
In reply to#1313413
Am 20.01.2016 um 18:46 schrieb One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>:

>> The problem is that *I* have no control over user space. But I also don't want
>> to say to my users "that is not my problem - get it solved yourself". This does
>> not help them.
> 
> Stuffing things into the kernel because the user space of a given
> platform can't get itself organised isn't helpful to the other billion
> plus Linux devices out there.

The assumption that there is  "the" user space of a given platform is wrong.

> 
>> And, most device drivers are corner cases since they are special solutions
>> for singular platforms.
> 
> Actually that is quite a small percentage - and the corner cases hide in
> drivers not in the core code, which is really important for
> maintainability.
> 
>>>> I'm glad - because it raises some hard questions and while I don't agree
>>>> with some of your starting points (like needing to "open" a uart without
>>>> user space
>> 
>> If have an idea how to turn off the device at boot time, before any user space
>> daemon is running, we can of course ignore that.
> 
> Your early user space is responsible for it. If you can't accept that
> then I don't see any point continuing the conversation.

Exactly. There are two reasons:
* we want to make sure that it works for any user space
* it should be done as early as possible

> 
>>>> But see below as I think your mental model is perhaps wrong
>>>> and this is a point of confusion ?
>> 
>> Maybe you do not accept that I want to keep as low level as reasonable (for me).
> 
> It's always "for me". No the kernel project is not "for me"
> 
>>>> Both of those techniques work in mainline without kernel changes (at
>>>> least on devices where the right gpio sysfs nodes exist
>> 
>> they do not exist...
> 
> For most they do because they are gpio lines so exportable to userspace.
> 
>>>> This I think is actually the really hard and interesting part of the
>>>> problem. The "tell me about open and close" case is simple and can be
>>>> done via tty_port today with minimal extra hooks. There is a small
>>>> question about how you set those hooks from a DT binding
>> 
>> tty has no binding. An UART hardware has. Another reason for me to
>> start with UARTs.
> 
> Every uart is a tty_port, every non uart is a tty_port. There is no
> reason you can't bind to a non uart device. Your current patches create
> bindings for the uart layer.

Yes and no. The &uart { compatible = "something"; } already exists.

> 
>>>> For some hardware that is the only way I know to do this because the
>>>> power hungry uart receiver is physically powered down. I would have to
>>>> check but I *think* that is true even on a modern x86 PC that supports
>>>> wakeups via serial - although it may be well hidden in ACPI and firmware.
>> 
>> Yes, agreed. But the gpio + interrupt solution was not mainlineable as well.
> 
> That I am unsure about - at some point it is going to have to be sorted
> because it is increasingly common (if currently mostly invisible)
> 
>>>> I'm not personally opoosed to the tty slave idea providing it ends up
>>>> attached to the tty_port not just uart.
>> 
>> Well if you can tell us how to handle the data path I have no problems with it
>> to attach to the tty level.
> 
> If your port is closed you have no data path. If you are using uart you
> have no data path because while your patch hooks a helper that some uarts
> use some of the time it's optional and a lot of uarts don't use

I wasn't aware that lots of uart's don't use it. At least one is using it. I would have
to check which percentage is using it and which isn't. Thanks for pointing this
out.

> it, so
> its not even uart generic.

Understood. I wasn't aware of that.

I just was under the false impression that this is the recommended common
and a well designed (object oriented) interface. struct uart_port
being the object and the uart_ops assigned to it, being the list of methods
that can be applied to an uart_port.

http://lxr.free-electrons.com/source/Documentation/serial/driver#L14
http://lxr.free-electrons.com/source/include/linux/serial_core.h#L45
http://lxr.free-electrons.com/source/include/linux/serial_core.h#L235

BR,
Nikolaus

[toc] | [prev] | [next] | [standalone]


#1315090

FromTomeu Vizoso <tomeu@tomeuvizoso.net>
Date2016-01-22 17:00 +0100
Message-ID<qTMhZ-62g-21@gated-at.bofh.it>
In reply to#1313421
On 20 January 2016 at 19:03, H. Nikolaus Schaller <hns@goldelico.com> wrote:
>
> Am 20.01.2016 um 18:46 schrieb One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>:
>
>>> The problem is that *I* have no control over user space. But I also don't want
>>> to say to my users "that is not my problem - get it solved yourself". This does
>>> not help them.
>>
>> Stuffing things into the kernel because the user space of a given
>> platform can't get itself organised isn't helpful to the other billion
>> plus Linux devices out there.
>
> The assumption that there is  "the" user space of a given platform is wrong.

I'm a bit surprised at the arguments being exchanged here regarding
why the kernel may or may not deal with the detail that a (say) BT
chip is behind a uart.

I would have expected that the main (and IMO sufficient) reason why
the kernel should do it is because the particular bus used to connect
a BT chip to the CPU is a hw detail that a kernel that does its job
should keep to itself. Same as userspace not needing to care if a BT
chip is behind SDIO or USB, why does it have to tell the kernel behind
which UART a BT chip is sitting?

Regards,

Tomeu

[toc] | [prev] | [next] | [standalone]


#1315131

FromRob Herring <robherring2@gmail.com>
Date2016-01-22 18:00 +0100
Message-ID<qTNe3-6HG-9@gated-at.bofh.it>
In reply to#1315090
On Fri, Jan 22, 2016 at 9:45 AM, Tomeu Vizoso <tomeu@tomeuvizoso.net> wrote:
> On 20 January 2016 at 19:03, H. Nikolaus Schaller <hns@goldelico.com> wrote:
>>
>> Am 20.01.2016 um 18:46 schrieb One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>:
>>
>>>> The problem is that *I* have no control over user space. But I also don't want
>>>> to say to my users "that is not my problem - get it solved yourself". This does
>>>> not help them.
>>>
>>> Stuffing things into the kernel because the user space of a given
>>> platform can't get itself organised isn't helpful to the other billion
>>> plus Linux devices out there.
>>
>> The assumption that there is  "the" user space of a given platform is wrong.
>
> I'm a bit surprised at the arguments being exchanged here regarding
> why the kernel may or may not deal with the detail that a (say) BT
> chip is behind a uart.
>
> I would have expected that the main (and IMO sufficient) reason why
> the kernel should do it is because the particular bus used to connect
> a BT chip to the CPU is a hw detail that a kernel that does its job
> should keep to itself. Same as userspace not needing to care if a BT
> chip is behind SDIO or USB, why does it have to tell the kernel behind
> which UART a BT chip is sitting?

Thanks for writing exactly what I had not gotten around to writing.
You are exactly right. While historically UART devices where not
probe-able like SDIO (in theory) and USB are and maybe that was a
reason to handle in userspace, we do have the technology to describe
the UART connections now.

Adding Marcel who has also previously commented on the reasons why
this belongs in the kernel [1].

Rob

[1] https://lists.linuxfoundation.org/pipermail/ksummit-discuss/2015-August/002212.html

[toc] | [prev] | [next] | [standalone]


#1315250

FromOne Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>
Date2016-01-22 21:20 +0100
Message-ID<qTQlA-Ac-3@gated-at.bofh.it>
In reply to#1315090
> I would have expected that the main (and IMO sufficient) reason why
> the kernel should do it is because the particular bus used to connect
> a BT chip to the CPU is a hw detail that a kernel that does its job
> should keep to itself. Same as userspace not needing to care if a BT
> chip is behind SDIO or USB, why does it have to tell the kernel behind
> which UART a BT chip is sitting?

Lots of reasons, some historic some not

1. Different BT chips have different interfaces, especially when it gets
to stuff like firmware reprogramming

2. In many cases we don't know at the kernel level where there are BT
uarts. It's improving with recent ACPI but for many systems it's simply
not available to the OS

3. The power management for a lot of BT (especially on device tree) is
not actually expressed, so you need a slightly customised daemon for each
device - that one is ugly but the serial and bt layers can't fix it.

4. Because you don't want to just automatically load and turn on
bluetooth just because it is there - it burns power


There is lots of stuff we probe and bind via user space - most things
these days in fact. That's much of why we have notifiers and udev. It's
frequently a win in flexibility, security and configurability to do stuff
via user daemons. We do it for example with all the volume management,
raid and disk encryption.

Alan

[toc] | [prev] | [next] | [standalone]


#1315532

FromAndreas Kemnade <andreas@kemnade.info>
Date2016-01-23 08:50 +0100
Message-ID<qU17k-7Wf-9@gated-at.bofh.it>
In reply to#1315250

[Multipart message — attachments visible in raw view] — view raw

On Fri, 22 Jan 2016 20:12:29 +0000
One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> wrote:

> > I would have expected that the main (and IMO sufficient) reason why
> > the kernel should do it is because the particular bus used to connect
> > a BT chip to the CPU is a hw detail that a kernel that does its job
> > should keep to itself. Same as userspace not needing to care if a BT
> > chip is behind SDIO or USB, why does it have to tell the kernel behind
> > which UART a BT chip is sitting?
> 
> Lots of reasons, some historic some not
> 
> 1. Different BT chips have different interfaces, especially when it gets
> to stuff like firmware reprogramming
> 
> 2. In many cases we don't know at the kernel level where there are BT
> uarts. It's improving with recent ACPI but for many systems it's simply
> not available to the OS
> 
Same is true for i2c devices. The solution there is that you have various
methods for providing the information to the kernel, some 
are autoprobed, some are via board files and you can also tell via sysfs
that there is one device.

> 3. The power management for a lot of BT (especially on device tree) is
> not actually expressed, so you need a slightly customised daemon for each
> device - that one is ugly but the serial and bt layers can't fix it.
> 
That boils down to a circular it is not there because it is not there.
If we express the power management, it can be done in kernel.

> 4. Because you don't want to just automatically load and turn on
> bluetooth just because it is there - it burns power
> 
Exactly the same is true for wifi and for many other devices for
which drivers are automatically handled in kernel, too.
Well, do you have a list of devices which do not burn power?
I would be highly interested in those.

Regards,
Andreas

[toc] | [prev] | [next] | [standalone]


#1315580

From"H. Nikolaus Schaller" <hns@goldelico.com>
Date2016-01-23 13:20 +0100
Message-ID<qU5kC-2uF-1@gated-at.bofh.it>
In reply to#1315250
Am 22.01.2016 um 21:12 schrieb One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>:

>> I would have expected that the main (and IMO sufficient) reason why
>> the kernel should do it is because the particular bus used to connect
>> a BT chip to the CPU is a hw detail that a kernel that does its job
>> should keep to itself. Same as userspace not needing to care if a BT
>> chip is behind SDIO or USB, why does it have to tell the kernel behind
>> which UART a BT chip is sitting?
> 
> Lots of reasons, some historic some not
> 
> 1. Different BT chips have different interfaces, especially when it gets
> to stuff like firmware reprogramming

HCI protocol is quite standardized. And firmware reprogramming is
rarely done. If it is needed, each chip type can have its own driver
module that knows how to inject firmware.

This just needs a decent kernel API for a chip driver to communicate
with the chip.

For SDIO connected WiFi chips it appears that firmware download done
by kernel modules is a standard solution.

> 
> 2. In many cases we don't know at the kernel level where there are BT
> uarts. It's improving with recent ACPI but for many systems it's simply
> not available to the OS

We just need to describe the connection of some peer driver to some
UART by means of DT. Then, kernel level can know.

> 
> 3. The power management for a lot of BT (especially on device tree) is
> not actually expressed, so you need a slightly customised daemon for each
> device - that one is ugly but the serial and bt layers can't fix it.

The peer chip driver (not the serial or bt layers) could fix it. With help
from bt and serial layers.

> 
> 4. Because you don't want to just automatically load and turn on
> bluetooth just because it is there - it burns power

That is obviously something nobody wants.

If the chip must be turned on (or turns on) during boot, the driver
can turn it off right after initialization and make the subsystem sleep
until activated again. Then it does not burn power although it
loads automatically.

> 
> 
> There is lots of stuff we probe and bind via user space - most things
> these days in fact. That's much of why we have notifiers and udev. It's
> frequently a win in flexibility, security and configurability to do stuff
> via user daemons. We do it for example with all the volume management,
> raid and disk encryption.

Because volumes are something users really want to configure. They
can change their hardware configuration every now and then. And
there are removable media to be considered.

In our UART cases the underlaying hardware can't be reconfigured. So
there is no need to load this burden of config to the user.

For BT or GPS I just want it to work the same on all devices (independently
on how the specific chip is connected). Kernel should unify such things.
Or it would not be a Un(iplexed)ix.

-- hns

[toc] | [prev] | [next] | [standalone]


#1315705

FromOne Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>
Date2016-01-23 18:30 +0100
Message-ID<qUaaC-5YV-3@gated-at.bofh.it>
In reply to#1315580
> > There is lots of stuff we probe and bind via user space - most things
> > these days in fact. That's much of why we have notifiers and udev. It's
> > frequently a win in flexibility, security and configurability to do stuff
> > via user daemons. We do it for example with all the volume management,
> > raid and disk encryption.  
> 
> Because volumes are something users really want to configure. They
> can change their hardware configuration every now and then. And
> there are removable media to be considered.

Like USB bluetooth dongles, like systems with external SPI ports, or plug
in SPI devices, or plug in gps devices on other interfaces ?

> In our UART cases the underlaying hardware can't be reconfigured. So
> there is no need to load this burden of config to the user.

Plenty of uarts it can be or the BT can be muxed with other device
endpoints.

> For BT or GPS I just want it to work the same on all devices (independently
> on how the specific chip is connected). Kernel should unify such things.
> Or it would not be a Un(iplexed)ix.

I think you are confusing Unix and Multics.

Unix is nothing to do with Linux and Unix was about creating a beautiful
system not by having a huge crap filled kernel, but by having only the
minimum necessary in the kernel. Unix was not about what was put in but
what was left out.

"We used to sit around in the Unix Room saying, 'What can we throw out?
Why is there this option?" - Doug McIlroy

GPS is a train wreck for commonality. Most GPS requires custom binary
only user space code often obfuscated in order to meet the regulations
governing GPS technology to stop third parties using it for missile
guidance.


Alan

[toc] | [prev] | [next] | [standalone]


#1315763

From"H. Nikolaus Schaller" <hns@goldelico.com>
Date2016-01-23 23:10 +0100
Message-ID<qUexA-3nL-7@gated-at.bofh.it>
In reply to#1315705
Am 23.01.2016 um 18:28 schrieb One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>:

>>> There is lots of stuff we probe and bind via user space - most things
>>> these days in fact. That's much of why we have notifiers and udev. It's
>>> frequently a win in flexibility, security and configurability to do stuff
>>> via user daemons. We do it for example with all the volume management,
>>> raid and disk encryption.  
>> 
>> Because volumes are something users really want to configure. They
>> can change their hardware configuration every now and then. And
>> there are removable media to be considered.
> 
> Like USB bluetooth dongles, like systems with external SPI ports, or plug
> in SPI devices, or plug in gps devices on other interfaces ?
> 
>> In our UART cases the underlaying hardware can't be reconfigured. So
>> there is no need to load this burden of config to the user.
> 
> Plenty of uarts it can be or the BT can be muxed with other device
> endpoints.

Please give examples where the user can configure such a chip that is
soldered on the same PCB as the SoC.

> 
>> For BT or GPS I just want it to work the same on all devices (independently
>> on how the specific chip is connected). Kernel should unify such things.
>> Or it would not be a Un(iplexed)ix.
> 
> I think you are confusing Unix and Multics.

No, If I write Unix I mean Unix. The "Un" stands for "Uniplexed" which is
a pun of course. But it alludes to "Unification" giving the impression of
"Simplification".

> 
> Unix is nothing to do with Linux and Unix was about creating a beautiful
> system not by having a huge crap filled kernel,

and no crap filled user space.

> but by having only the
> minimum necessary in the kernel. Unix

I think you are confusing it with the goals of microkernels (e.g. Mach or Hurd).

> was not about what was put in but
> what was left out.

Exactly. It left out complexity. E.g. everything is a file. You can use devices
like files. Just open some /dev/tty and get GPS NMEA records...

> 
> "We used to sit around in the Unix Room saying, 'What can we throw out?
> Why is there this option?" - Doug McIlroy

Fine. Let's throw out the idea to configure power on/off a GPS or BT device
by user space daemons for hard wired chips.

> 
> GPS is a train wreck for commonality. Most GPS requires custom binary
> only user space code often obfuscated in order to meet the regulations
> governing GPS technology to stop third parties using it for missile
> guidance.

Most GPS receivers I came across are modules which spit out NMEA
records with serial 9600 bit/s. Either through RS232 or Bluetooth SPP. There
may be others, but I don't want to have all problems of the world solved
at once.

-- hns

[toc] | [prev] | [next] | [standalone]


#1315937

FromOne Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>
Date2016-01-24 18:20 +0100
Message-ID<qUwut-81d-3@gated-at.bofh.it>
In reply to#1315763
> > but by having only the
> > minimum necessary in the kernel. Unix  
> 
> I think you are confusing it with the goals of microkernels (e.g. Mach or Hurd).

No and the quote below is from Doug McIlroy - who amongst other thing
invented pipes.

> > 
> > "We used to sit around in the Unix Room saying, 'What can we throw out?
> > Why is there this option?" - Doug McIlroy  

> Most GPS receivers I came across are modules which spit out NMEA
> records with serial 9600 bit/s. Either through RS232 or Bluetooth SPP. There
> may be others, but I don't want to have all problems of the world solved
> at once.

The kernel lives in the big world, not your personal fiefdom

*PLONK*

[toc] | [prev] | [next] | [standalone]


#1316447

From"H. Nikolaus Schaller" <hns@goldelico.com>
Date2016-01-25 11:40 +0100
Message-ID<qUMIV-2z3-13@gated-at.bofh.it>
In reply to#1315937
Hi Alan,

Am 24.01.2016 um 18:10 schrieb One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>:

>>> but by having only the
>>> minimum necessary in the kernel. Unix  
>> 
>> I think you are confusing it with the goals of microkernels (e.g. Mach or Hurd).
> 
> No and the quote below is from Doug McIlroy - who amongst other thing
> invented pipes.

Which is a brilliant concept.

> 
>>> 
>>> "We used to sit around in the Unix Room saying, 'What can we throw out?
>>> Why is there this option?" - Doug McIlroy  

Maybe I am interpreting differently, but "throwing out unnecessary things from
Unix" is logically not the same as "having the minimum necessary in the kernel".

The latter appears to be implying to do *everything* which can be done in user space.
This is what I mean that the goal of throwing out everything not necessary from the
kernel leads to micro-kernel concepts. And neither Unix nor Linux did throw out
enough to become a microkernel. So they still have components which could be
thrown out and moved to user space.

And Unix is kernel + user space. So the citation could also mean that they did
throw out (and simplify) concepts in both areas. Not only from the kernel. So they
did balance and optimize the whole system (which is what I want to achieve as well).

Unfortunately I never had a chance to meet one of the Unix inventors, so I
can't ask them what they really meant and how they did work.

But anyways, this is discussing the wrong problem on the wrong level.

People out there (and me included) would like to hide or better, easier and
more systematically access devices directly connected on the same PCB to
a specific UART. And can't. Or can do only with ugly or inefficient solutions.

From that we come to this fundamental discussion level because you say that
Linux should be a kernel which should only support the minimum. Therefore our
requests are rejected, because it could be solved by blowing up the user space.

This is IMHO your very valid opinion, but it is not a guideline that I observe when
going through the source tree. There are tons of things that could be done
in user space (e.g. what are all those I2C device drivers good for? /dev/i2c should
suffice to control every device from user space. What is iio good for in the kernel?
a library could translate everything into iio interfaces).

So this discrepancy is what I criticize because I don't see a basic rule or
design principle behind what is acceptable and what is not.

If it exists, please point me to it. I am open to learn about that.

> 
>> Most GPS receivers I came across are modules which spit out NMEA
>> records with serial 9600 bit/s. Either through RS232 or Bluetooth SPP. There
>> may be others, but I don't want to have all problems of the world solved
>> at once.
> 
> The kernel lives in the big world, not your personal fiefdom

Every contribution initially looks at the area, the contributor is aware of and
knows best and does not find a solution using existing hooks. Then discussion
starts.

But what a contributor can't (and should never) accept is if his problem is
simply declared to be a non-problem without providing a *better* alternative
solution for it. Better for everybody including the contributor.

> 
> *PLONK*

Thanks.

Well, I have to apologize that I was not yet aware who hides behind the
"one thousand gnomes" e-mail address... Because I judge discussion partners
by what they say now and how they help to solve my actual problem for the future
(and not what they have done in the past).

So you do have a lot of experience with Linux history. And you are amongst
the handful of persons who know every detail of the Linux kernel. That is good
because it shows that you can really influence how this discussion ends.

But sometimes experience could be a little blocking to new ideas and experiences
not related to Linux (e.g. close to hardware or embedded design) brought in by
newcomers.

And it collides with my belief that everything can be done, if people want.

Now, how can we help those who want (for IMHO very understandable reasons)
to access a specific UART from kernel modules?

-- hns

[toc] | [prev] | [next] | [standalone]


#1311957

FromAndreas Kemnade <andreas@kemnade.info>
Date2016-01-19 07:40 +0100
Message-ID<qSy7n-3RB-13@gated-at.bofh.it>
In reply to#1311819

[Multipart message — attachments visible in raw view] — view raw

On Mon, 18 Jan 2016 22:03:19 +0000
One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> wrote:

> > > Your user space can do it (as most Android does).
> > 
> > How can it do it in automatically in a standardized way without need for daemon support?
> 
> You don't need to - it can be device specific. In Android it usually is.
> I've never understood why low end devices don't also abstract user space
> descriptions of power control into DT nodes as well as kernel properties ?
> 
Well, on these android devices, they are only intended to run android and
have another abstraction layer where you can hide things.
If doing actions on opening or closing a tty should not be implemented in
kernel space, then an alternative I see would be to at least have proper
rfkill support for bluetooth and gps in kernel. Then userspace can at least
can talk to a standardised interface. So userspace just has to implement
generic things.

That would mean for the kernel drivers needed:


W2CBW003/bluetooth:
   map the rfkill to just a simple regulator

W2SG0004/gps:
   being notified about incoming data
     a) the nice way: getting it from the tty/tty_port layer
 	(but requiring changes at generic kernel code)
        or
     b) the ugly way:
        remux the data line as a gpio and simply look for state
        changes during rfkill call (probably only during unblock)
	The first characters after power on would probably be lost
        toggle a gpio depending on the desired and detected state

Regards,
Andreas

[toc] | [prev] | [next] | [standalone]


#1313462

FromDmitry Torokhov <dmitry.torokhov@gmail.com>
Date2016-01-20 20:40 +0100
Message-ID<qT6LM-2E2-19@gated-at.bofh.it>
In reply to#1310902
On Fri, Jan 15, 2016 at 11:34 PM, Vostrikov Andrey
<andrey.vostrikov@cogentembedded.com> wrote:
>
> Yes, such implementation will help. There is a need for interface like UART BUS that will probe devices without user space.
> Serial I/O for input subsystem defines new type of bus and uses dedicated line discipline, but it still unable to start driver by itself and requires call from 'inputattach' to open port, assign line discipline and go to forever wait on 'read'.

That was done mainly because almost none of the serial protocols could
be auto-probed, so device initialization/setup was moved out of kernel
and thus we have the separate line discipline and inputattach utility.

Thanks.

-- 
Dmitry

[toc] | [prev] | [next] | [standalone]


Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

Back to top | Article view | linux.kernel


csiph-web