Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1256823 > unrolled thread

[RFD] Functional dependencies between devices

Back to article view | Back to linux.kernel

Contents

  [RFD] Functional dependencies between devices "Rafael J. Wysocki" <rjw@rjwysocki.net> - 2015-10-27 16:00 +0100
    Re: [RFD] Functional dependencies between devices Tomeu Vizoso <tomeu.vizoso@collabora.com> - 2015-10-27 16:30 +0100
      Re: [RFD] Functional dependencies between devices "Rafael J. Wysocki" <rjw@rjwysocki.net> - 2015-10-28 02:50 +0100
        Re: [RFD] Functional dependencies between devices Tomeu Vizoso <tomeu.vizoso@collabora.com> - 2015-10-28 15:30 +0100
          Re: [RFD] Functional dependencies between devices "Rafael J. Wysocki" <rjw@rjwysocki.net> - 2015-10-28 16:30 +0100
            Re: [RFD] Functional dependencies between devices Mark Brown <broonie@kernel.org> - 2015-10-29 01:20 +0100
            Re: [RFD] Functional dependencies between devices Tomeu Vizoso <tomeu.vizoso@collabora.com> - 2015-10-29 15:20 +0100
              Re: [RFD] Functional dependencies between devices Alan Stern <stern@rowland.harvard.edu> - 2015-10-29 15:40 +0100
                Re: [RFD] Functional dependencies between devices "Rafael J. Wysocki" <rjw@rjwysocki.net> - 2015-10-31 03:00 +0100
    Re: [RFD] Functional dependencies between devices Mark Brown <broonie@kernel.org> - 2015-10-29 01:20 +0100
      Re: [RFD] Functional dependencies between devices "Rafael J. Wysocki" <rjw@rjwysocki.net> - 2015-10-31 02:50 +0100
        Re: [RFD] Functional dependencies between devices Mark Brown <broonie@kernel.org> - 2015-10-31 03:50 +0100
    Re: [RFD] Functional dependencies between devices Linus Walleij <linus.walleij@linaro.org> - 2015-10-30 11:00 +0100
    Re: [RFD] Functional dependencies between devices Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-10-31 00:00 +0100

#1256823 — [RFD] Functional dependencies between devices

Hi All,

As discussed in the recent "On-demand device probing" thread and in a Kernel
Summit session earlier today, there is a problem with handling cases where
functional dependencies between devices are involved.

What I mean by a "functional dependency" is when the driver of device B needs
both device A and its driver to be present and functional to be able to work.
This implies that the driver of A needs to be working for B to be probed
successfully and it cannot be unbound from the device before the B's driver.
This also has certain consequences for power management of these devices
(suspend/resume and runtime PM ordering).

So I want to be able to represent those functional dependencies between devices
and I'd like the driver core to track them and act on them in certain cases
where they matter.  The argument for doing that in the driver core is that
there are quite a few distinct use cases related to that, they are relatively
hard to get right in a driver (if one wants to address all of them properly)
and it only gets worse if multiplied by the number of drivers potentially
needing to do it.  Morever, at least one case (asynchronous system suspend/resume)
cannot be handled in a single driver at all, because it requires the driver of A
to wait for B to suspend (during system suspend) and the driver of B to wait for
A to resume (during system resume).

My idea is to represent a supplier-consumer dependency between devices (or
more precisely between device+driver combos) as a "link" object containing
pointers to the devices in question, a list node for each of them and some
additional information related to the management of those objects, ie.
something like:

struct device_link {
	struct device *supplier;
	struct list_head supplier_node;
	struct device *consumer;
	struct list_head consumer_node;
	<flags, status etc>
};

In general, there will be two lists of those things per device, one list
of links to consumers and one list of links to suppliers.

In that picture, links will be created by calling, say:

int device_add_link(struct device *me, struct device *my_supplier, unsigned int flags);

and they will be deleted by the driver core when not needed any more.  The
creation of a link should also cause dpm_list and the list used during shutdown
to be reordered if needed.

In principle, it seems usefult to consider two types of links, one created
at device registration time (when registering the second device from the linked
pair, whichever it is) and one created at probe time (of the consumer device).
I'll refer to them as "permanent" and "probe-time" links, respectively.

The permanent links (created at device registration time) will stay around
until one of the linked devices is unregistered (at which time the driver
core will drop the link along with the device going away).  The probe-time
ones will be dropped (automatically) at the consumer device driver unbind time.

There's a question about what if the supplier device is being unbound before
the consumer one (for example, as a result of a hotplug event).  My current
view on that is that the consumer needs to be force-unbound in that case too,
but I guess I may be persuaded otherwise given sufficiently convincing
arguments.  Anyway, there are reasons to do that, like for example it may
help with the synchronization.  Namely, if there's a rule that suppliers
cannot be unbound before any consumers linked to them, than the list of links
to suppliers for a consumer can only change at its registration/probe or
unbind/remove times (which simplifies things quite a bit).

With that, the permanent links existing at the probe time for a consumer
device can be used to check whether or not to defer the probing of it
even before executing its probe callback.  In turn, system suspend
synchronization should be a matter of calling device_pm_wait_for_dev()
for all consumers of a supplier device, in analogy with dpm_wait_for_children(),
and so on.

Of course, the new lists have to be stable during those operations and ensuring
that is going to be somewhat tricky (AFAICS right now at least), but apart from
that the whole concept looks reasonably straightforward to me.

So, the question to everybody is whether or not this sounds reasonable or there
are concerns about it and if so what they are.  At this point I mostly need to
know if I'm not overlooking anything fundamental at the general level.

Thanks,
Rafael

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1256850

On 27 October 2015 at 16:24, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
> Hi All,
>
> As discussed in the recent "On-demand device probing" thread and in a Kernel
> Summit session earlier today, there is a problem with handling cases where
> functional dependencies between devices are involved.
>
> What I mean by a "functional dependency" is when the driver of device B needs
> both device A and its driver to be present and functional to be able to work.
> This implies that the driver of A needs to be working for B to be probed
> successfully and it cannot be unbound from the device before the B's driver.
> This also has certain consequences for power management of these devices
> (suspend/resume and runtime PM ordering).
>
> So I want to be able to represent those functional dependencies between devices
> and I'd like the driver core to track them and act on them in certain cases
> where they matter.  The argument for doing that in the driver core is that
> there are quite a few distinct use cases related to that, they are relatively
> hard to get right in a driver (if one wants to address all of them properly)
> and it only gets worse if multiplied by the number of drivers potentially
> needing to do it.  Morever, at least one case (asynchronous system suspend/resume)
> cannot be handled in a single driver at all, because it requires the driver of A
> to wait for B to suspend (during system suspend) and the driver of B to wait for
> A to resume (during system resume).
>
> My idea is to represent a supplier-consumer dependency between devices (or
> more precisely between device+driver combos) as a "link" object containing
> pointers to the devices in question, a list node for each of them and some
> additional information related to the management of those objects, ie.
> something like:
>
> struct device_link {
>         struct device *supplier;
>         struct list_head supplier_node;
>         struct device *consumer;
>         struct list_head consumer_node;
>         <flags, status etc>
> };
>
> In general, there will be two lists of those things per device, one list
> of links to consumers and one list of links to suppliers.
>
> In that picture, links will be created by calling, say:
>
> int device_add_link(struct device *me, struct device *my_supplier, unsigned int flags);
>
> and they will be deleted by the driver core when not needed any more.  The
> creation of a link should also cause dpm_list and the list used during shutdown
> to be reordered if needed.
>
> In principle, it seems usefult to consider two types of links, one created
> at device registration time (when registering the second device from the linked
> pair, whichever it is) and one created at probe time (of the consumer device).
> I'll refer to them as "permanent" and "probe-time" links, respectively.
>
> The permanent links (created at device registration time) will stay around
> until one of the linked devices is unregistered (at which time the driver
> core will drop the link along with the device going away).  The probe-time
> ones will be dropped (automatically) at the consumer device driver unbind time.
>
> There's a question about what if the supplier device is being unbound before
> the consumer one (for example, as a result of a hotplug event).  My current
> view on that is that the consumer needs to be force-unbound in that case too,
> but I guess I may be persuaded otherwise given sufficiently convincing
> arguments.  Anyway, there are reasons to do that, like for example it may
> help with the synchronization.  Namely, if there's a rule that suppliers
> cannot be unbound before any consumers linked to them, than the list of links
> to suppliers for a consumer can only change at its registration/probe or
> unbind/remove times (which simplifies things quite a bit).
>
> With that, the permanent links existing at the probe time for a consumer
> device can be used to check whether or not to defer the probing of it
> even before executing its probe callback.  In turn, system suspend
> synchronization should be a matter of calling device_pm_wait_for_dev()
> for all consumers of a supplier device, in analogy with dpm_wait_for_children(),
> and so on.
>
> Of course, the new lists have to be stable during those operations and ensuring
> that is going to be somewhat tricky (AFAICS right now at least), but apart from
> that the whole concept looks reasonably straightforward to me.
>
> So, the question to everybody is whether or not this sounds reasonable or there
> are concerns about it and if so what they are.  At this point I mostly need to
> know if I'm not overlooking anything fundamental at the general level.

Sounds really great to me at the conceptual level, but wonder if you
have already thought of how the permanent links will be inferred.

When I looked at computing dependencies of a device before it's
probed, the concern was that the code that finds the dependencies
duplicated part of the logic when looking resources up. Because each
subsystem has its own code for looking up dependencies for potentially
each of DT, ACPI and board files, it will be a bit of a big task to
refactor things to avoid that duplication. Fwnode could help with
this, but it doesn't as of yet and I'm not sure if that's still the
plan.

Also wonder if you have considered setting the permanent links also
during probe, as the on-demand probe series did (device_add_link would
be "sprinkled" around as of_device_probe was). That would avoid the
problem with code duplication because the links would be established
from the functions that do resource lookup.

Thanks,

Tomeu

> Thanks,
> Rafael
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1257563

On Tuesday, October 27, 2015 04:20:51 PM Tomeu Vizoso wrote:
> On 27 October 2015 at 16:24, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
> > Hi All,
> >
> > As discussed in the recent "On-demand device probing" thread and in a Kernel
> > Summit session earlier today, there is a problem with handling cases where
> > functional dependencies between devices are involved.
> >
> > What I mean by a "functional dependency" is when the driver of device B needs
> > both device A and its driver to be present and functional to be able to work.
> > This implies that the driver of A needs to be working for B to be probed
> > successfully and it cannot be unbound from the device before the B's driver.
> > This also has certain consequences for power management of these devices
> > (suspend/resume and runtime PM ordering).
> >
> > So I want to be able to represent those functional dependencies between devices
> > and I'd like the driver core to track them and act on them in certain cases
> > where they matter.  The argument for doing that in the driver core is that
> > there are quite a few distinct use cases related to that, they are relatively
> > hard to get right in a driver (if one wants to address all of them properly)
> > and it only gets worse if multiplied by the number of drivers potentially
> > needing to do it.  Morever, at least one case (asynchronous system suspend/resume)
> > cannot be handled in a single driver at all, because it requires the driver of A
> > to wait for B to suspend (during system suspend) and the driver of B to wait for
> > A to resume (during system resume).
> >
> > My idea is to represent a supplier-consumer dependency between devices (or
> > more precisely between device+driver combos) as a "link" object containing
> > pointers to the devices in question, a list node for each of them and some
> > additional information related to the management of those objects, ie.
> > something like:
> >
> > struct device_link {
> >         struct device *supplier;
> >         struct list_head supplier_node;
> >         struct device *consumer;
> >         struct list_head consumer_node;
> >         <flags, status etc>
> > };
> >
> > In general, there will be two lists of those things per device, one list
> > of links to consumers and one list of links to suppliers.
> >
> > In that picture, links will be created by calling, say:
> >
> > int device_add_link(struct device *me, struct device *my_supplier, unsigned int flags);
> >
> > and they will be deleted by the driver core when not needed any more.  The
> > creation of a link should also cause dpm_list and the list used during shutdown
> > to be reordered if needed.
> >
> > In principle, it seems usefult to consider two types of links, one created
> > at device registration time (when registering the second device from the linked
> > pair, whichever it is) and one created at probe time (of the consumer device).
> > I'll refer to them as "permanent" and "probe-time" links, respectively.
> >
> > The permanent links (created at device registration time) will stay around
> > until one of the linked devices is unregistered (at which time the driver
> > core will drop the link along with the device going away).  The probe-time
> > ones will be dropped (automatically) at the consumer device driver unbind time.
> >
> > There's a question about what if the supplier device is being unbound before
> > the consumer one (for example, as a result of a hotplug event).  My current
> > view on that is that the consumer needs to be force-unbound in that case too,
> > but I guess I may be persuaded otherwise given sufficiently convincing
> > arguments.  Anyway, there are reasons to do that, like for example it may
> > help with the synchronization.  Namely, if there's a rule that suppliers
> > cannot be unbound before any consumers linked to them, than the list of links
> > to suppliers for a consumer can only change at its registration/probe or
> > unbind/remove times (which simplifies things quite a bit).
> >
> > With that, the permanent links existing at the probe time for a consumer
> > device can be used to check whether or not to defer the probing of it
> > even before executing its probe callback.  In turn, system suspend
> > synchronization should be a matter of calling device_pm_wait_for_dev()
> > for all consumers of a supplier device, in analogy with dpm_wait_for_children(),
> > and so on.
> >
> > Of course, the new lists have to be stable during those operations and ensuring
> > that is going to be somewhat tricky (AFAICS right now at least), but apart from
> > that the whole concept looks reasonably straightforward to me.
> >
> > So, the question to everybody is whether or not this sounds reasonable or there
> > are concerns about it and if so what they are.  At this point I mostly need to
> > know if I'm not overlooking anything fundamental at the general level.
> 
> Sounds really great to me at the conceptual level, but wonder if you
> have already thought of how the permanent links will be inferred.

In ACPI there is a mechanism for that already.  In DT it would require walking
the phandle dependency graph I suppose.

The point is, though, that it doesn't have to be mandatory to have any
permanent links created.  If you can find a dependency at device registration
time, great.  Create a permanent link for it and use it.  If you can't,
it's fine too.  You'll find it at probe time and create a link for it then.

> When I looked at computing dependencies of a device before it's
> probed, the concern was that the code that finds the dependencies
> duplicated part of the logic when looking resources up. Because each
> subsystem has its own code for looking up dependencies for potentially
> each of DT, ACPI and board files, it will be a bit of a big task to
> refactor things to avoid that duplication. Fwnode could help with
> this, but it doesn't as of yet and I'm not sure if that's still the
> plan.

That almost certainly is going to be a fair amount of work, but that
doesn't mean we should avoid doing it.  If it leads to better code
eventually, it's worth doing.

> Also wonder if you have considered setting the permanent links also
> during probe, as the on-demand probe series did (device_add_link would
> be "sprinkled" around as of_device_probe was). That would avoid the
> problem with code duplication because the links would be established
> from the functions that do resource lookup.

I have considered that, but at this point I have some concerns about
lifecycle management related to that.

In any case, if the given link is really permanent, there should be enough
information available to find it at device registration time, although that
may require some additional computations to be carried out.

Thanks,
Rafael

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1258150

On 28 October 2015 at 03:15, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
> On Tuesday, October 27, 2015 04:20:51 PM Tomeu Vizoso wrote:
>> On 27 October 2015 at 16:24, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
>> > Hi All,
>> >
>> > As discussed in the recent "On-demand device probing" thread and in a Kernel
>> > Summit session earlier today, there is a problem with handling cases where
>> > functional dependencies between devices are involved.
>> >
>> > What I mean by a "functional dependency" is when the driver of device B needs
>> > both device A and its driver to be present and functional to be able to work.
>> > This implies that the driver of A needs to be working for B to be probed
>> > successfully and it cannot be unbound from the device before the B's driver.
>> > This also has certain consequences for power management of these devices
>> > (suspend/resume and runtime PM ordering).
>> >
>> > So I want to be able to represent those functional dependencies between devices
>> > and I'd like the driver core to track them and act on them in certain cases
>> > where they matter.  The argument for doing that in the driver core is that
>> > there are quite a few distinct use cases related to that, they are relatively
>> > hard to get right in a driver (if one wants to address all of them properly)
>> > and it only gets worse if multiplied by the number of drivers potentially
>> > needing to do it.  Morever, at least one case (asynchronous system suspend/resume)
>> > cannot be handled in a single driver at all, because it requires the driver of A
>> > to wait for B to suspend (during system suspend) and the driver of B to wait for
>> > A to resume (during system resume).
>> >
>> > My idea is to represent a supplier-consumer dependency between devices (or
>> > more precisely between device+driver combos) as a "link" object containing
>> > pointers to the devices in question, a list node for each of them and some
>> > additional information related to the management of those objects, ie.
>> > something like:
>> >
>> > struct device_link {
>> >         struct device *supplier;
>> >         struct list_head supplier_node;
>> >         struct device *consumer;
>> >         struct list_head consumer_node;
>> >         <flags, status etc>
>> > };
>> >
>> > In general, there will be two lists of those things per device, one list
>> > of links to consumers and one list of links to suppliers.
>> >
>> > In that picture, links will be created by calling, say:
>> >
>> > int device_add_link(struct device *me, struct device *my_supplier, unsigned int flags);
>> >
>> > and they will be deleted by the driver core when not needed any more.  The
>> > creation of a link should also cause dpm_list and the list used during shutdown
>> > to be reordered if needed.
>> >
>> > In principle, it seems usefult to consider two types of links, one created
>> > at device registration time (when registering the second device from the linked
>> > pair, whichever it is) and one created at probe time (of the consumer device).
>> > I'll refer to them as "permanent" and "probe-time" links, respectively.
>> >
>> > The permanent links (created at device registration time) will stay around
>> > until one of the linked devices is unregistered (at which time the driver
>> > core will drop the link along with the device going away).  The probe-time
>> > ones will be dropped (automatically) at the consumer device driver unbind time.
>> >
>> > There's a question about what if the supplier device is being unbound before
>> > the consumer one (for example, as a result of a hotplug event).  My current
>> > view on that is that the consumer needs to be force-unbound in that case too,
>> > but I guess I may be persuaded otherwise given sufficiently convincing
>> > arguments.  Anyway, there are reasons to do that, like for example it may
>> > help with the synchronization.  Namely, if there's a rule that suppliers
>> > cannot be unbound before any consumers linked to them, than the list of links
>> > to suppliers for a consumer can only change at its registration/probe or
>> > unbind/remove times (which simplifies things quite a bit).
>> >
>> > With that, the permanent links existing at the probe time for a consumer
>> > device can be used to check whether or not to defer the probing of it
>> > even before executing its probe callback.  In turn, system suspend
>> > synchronization should be a matter of calling device_pm_wait_for_dev()
>> > for all consumers of a supplier device, in analogy with dpm_wait_for_children(),
>> > and so on.
>> >
>> > Of course, the new lists have to be stable during those operations and ensuring
>> > that is going to be somewhat tricky (AFAICS right now at least), but apart from
>> > that the whole concept looks reasonably straightforward to me.
>> >
>> > So, the question to everybody is whether or not this sounds reasonable or there
>> > are concerns about it and if so what they are.  At this point I mostly need to
>> > know if I'm not overlooking anything fundamental at the general level.
>>
>> Sounds really great to me at the conceptual level, but wonder if you
>> have already thought of how the permanent links will be inferred.
>
> In ACPI there is a mechanism for that already.  In DT it would require walking
> the phandle dependency graph I suppose.
>
> The point is, though, that it doesn't have to be mandatory to have any
> permanent links created.  If you can find a dependency at device registration
> time, great.  Create a permanent link for it and use it.  If you can't,
> it's fine too.  You'll find it at probe time and create a link for it then.
>
>> When I looked at computing dependencies of a device before it's
>> probed, the concern was that the code that finds the dependencies
>> duplicated part of the logic when looking resources up. Because each
>> subsystem has its own code for looking up dependencies for potentially
>> each of DT, ACPI and board files, it will be a bit of a big task to
>> refactor things to avoid that duplication. Fwnode could help with
>> this, but it doesn't as of yet and I'm not sure if that's still the
>> plan.
>
> That almost certainly is going to be a fair amount of work, but that
> doesn't mean we should avoid doing it.  If it leads to better code
> eventually, it's worth doing.
>
>> Also wonder if you have considered setting the permanent links also
>> during probe, as the on-demand probe series did (device_add_link would
>> be "sprinkled" around as of_device_probe was). That would avoid the
>> problem with code duplication because the links would be established
>> from the functions that do resource lookup.
>
> I have considered that, but at this point I have some concerns about
> lifecycle management related to that.

Hi Rafael,

could you extend on why do you prefer inferring the permanent links
before probe as opposed to collecting them during probe from the
lookup functions?

Also, have you considered that not only drivers request resources? For
example, the on-demand probing series would probe a device that is
needed by an initcall, simplifying synchronization.

Regards,

Tomeu

> In any case, if the given link is really permanent, there should be enough
> information available to find it at device registration time, although that
> may require some additional computations to be carried out.
>
> Thanks,
> Rafael
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1258243

On Wednesday, October 28, 2015 03:26:14 PM Tomeu Vizoso wrote:
> On 28 October 2015 at 03:15, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
> > On Tuesday, October 27, 2015 04:20:51 PM Tomeu Vizoso wrote:
> >> On 27 October 2015 at 16:24, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
> >> > Hi All,
> >> >
> >> > As discussed in the recent "On-demand device probing" thread and in a Kernel
> >> > Summit session earlier today, there is a problem with handling cases where
> >> > functional dependencies between devices are involved.
> >> >
> >> > What I mean by a "functional dependency" is when the driver of device B needs
> >> > both device A and its driver to be present and functional to be able to work.
> >> > This implies that the driver of A needs to be working for B to be probed
> >> > successfully and it cannot be unbound from the device before the B's driver.
> >> > This also has certain consequences for power management of these devices
> >> > (suspend/resume and runtime PM ordering).
> >> >
> >> > So I want to be able to represent those functional dependencies between devices
> >> > and I'd like the driver core to track them and act on them in certain cases
> >> > where they matter.  The argument for doing that in the driver core is that
> >> > there are quite a few distinct use cases related to that, they are relatively
> >> > hard to get right in a driver (if one wants to address all of them properly)
> >> > and it only gets worse if multiplied by the number of drivers potentially
> >> > needing to do it.  Morever, at least one case (asynchronous system suspend/resume)
> >> > cannot be handled in a single driver at all, because it requires the driver of A
> >> > to wait for B to suspend (during system suspend) and the driver of B to wait for
> >> > A to resume (during system resume).
> >> >
> >> > My idea is to represent a supplier-consumer dependency between devices (or
> >> > more precisely between device+driver combos) as a "link" object containing
> >> > pointers to the devices in question, a list node for each of them and some
> >> > additional information related to the management of those objects, ie.
> >> > something like:
> >> >
> >> > struct device_link {
> >> >         struct device *supplier;
> >> >         struct list_head supplier_node;
> >> >         struct device *consumer;
> >> >         struct list_head consumer_node;
> >> >         <flags, status etc>
> >> > };
> >> >
> >> > In general, there will be two lists of those things per device, one list
> >> > of links to consumers and one list of links to suppliers.
> >> >
> >> > In that picture, links will be created by calling, say:
> >> >
> >> > int device_add_link(struct device *me, struct device *my_supplier, unsigned int flags);
> >> >
> >> > and they will be deleted by the driver core when not needed any more.  The
> >> > creation of a link should also cause dpm_list and the list used during shutdown
> >> > to be reordered if needed.
> >> >
> >> > In principle, it seems usefult to consider two types of links, one created
> >> > at device registration time (when registering the second device from the linked
> >> > pair, whichever it is) and one created at probe time (of the consumer device).
> >> > I'll refer to them as "permanent" and "probe-time" links, respectively.
> >> >
> >> > The permanent links (created at device registration time) will stay around
> >> > until one of the linked devices is unregistered (at which time the driver
> >> > core will drop the link along with the device going away).  The probe-time
> >> > ones will be dropped (automatically) at the consumer device driver unbind time.
> >> >
> >> > There's a question about what if the supplier device is being unbound before
> >> > the consumer one (for example, as a result of a hotplug event).  My current
> >> > view on that is that the consumer needs to be force-unbound in that case too,
> >> > but I guess I may be persuaded otherwise given sufficiently convincing
> >> > arguments.  Anyway, there are reasons to do that, like for example it may
> >> > help with the synchronization.  Namely, if there's a rule that suppliers
> >> > cannot be unbound before any consumers linked to them, than the list of links
> >> > to suppliers for a consumer can only change at its registration/probe or
> >> > unbind/remove times (which simplifies things quite a bit).
> >> >
> >> > With that, the permanent links existing at the probe time for a consumer
> >> > device can be used to check whether or not to defer the probing of it
> >> > even before executing its probe callback.  In turn, system suspend
> >> > synchronization should be a matter of calling device_pm_wait_for_dev()
> >> > for all consumers of a supplier device, in analogy with dpm_wait_for_children(),
> >> > and so on.
> >> >
> >> > Of course, the new lists have to be stable during those operations and ensuring
> >> > that is going to be somewhat tricky (AFAICS right now at least), but apart from
> >> > that the whole concept looks reasonably straightforward to me.
> >> >
> >> > So, the question to everybody is whether or not this sounds reasonable or there
> >> > are concerns about it and if so what they are.  At this point I mostly need to
> >> > know if I'm not overlooking anything fundamental at the general level.
> >>
> >> Sounds really great to me at the conceptual level, but wonder if you
> >> have already thought of how the permanent links will be inferred.
> >
> > In ACPI there is a mechanism for that already.  In DT it would require walking
> > the phandle dependency graph I suppose.
> >
> > The point is, though, that it doesn't have to be mandatory to have any
> > permanent links created.  If you can find a dependency at device registration
> > time, great.  Create a permanent link for it and use it.  If you can't,
> > it's fine too.  You'll find it at probe time and create a link for it then.
> >
> >> When I looked at computing dependencies of a device before it's
> >> probed, the concern was that the code that finds the dependencies
> >> duplicated part of the logic when looking resources up. Because each
> >> subsystem has its own code for looking up dependencies for potentially
> >> each of DT, ACPI and board files, it will be a bit of a big task to
> >> refactor things to avoid that duplication. Fwnode could help with
> >> this, but it doesn't as of yet and I'm not sure if that's still the
> >> plan.
> >
> > That almost certainly is going to be a fair amount of work, but that
> > doesn't mean we should avoid doing it.  If it leads to better code
> > eventually, it's worth doing.
> >
> >> Also wonder if you have considered setting the permanent links also
> >> during probe, as the on-demand probe series did (device_add_link would
> >> be "sprinkled" around as of_device_probe was). That would avoid the
> >> problem with code duplication because the links would be established
> >> from the functions that do resource lookup.
> >
> > I have considered that, but at this point I have some concerns about
> > lifecycle management related to that.
> 
> Hi Rafael,
> 
> could you extend on why do you prefer inferring the permanent links
> before probe as opposed to collecting them during probe from the
> lookup functions?

Information that is already available at the device registration time should
be used at that time or it makes things harder to follow.

But that really is a tradeoff.  If collecting that information requires too
much effort, it may not be worth it.

> Also, have you considered that not only drivers request resources? For
> example, the on-demand probing series would probe a device that is
> needed by an initcall, simplifying synchronization.

You really need to explain what you mean here or maybe give an example.

Thanks,
Rafael

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1258498

[Multipart message — attachments visible in raw view] — view raw

On Wed, Oct 28, 2015 at 04:54:04PM +0100, Rafael J. Wysocki wrote:

> Information that is already available at the device registration time should
> be used at that time or it makes things harder to follow.

> But that really is a tradeoff.  If collecting that information requires too
> much effort, it may not be worth it.

For DT it's going to be a lot eaiser to reliably collect everything in
driver specific code, the property names to look at do follow
conventions but are driver defined.

[toc] | [prev] | [next] | [standalone]

#1258813

On 28 October 2015 at 16:54, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
> On Wednesday, October 28, 2015 03:26:14 PM Tomeu Vizoso wrote:
>> On 28 October 2015 at 03:15, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
>> > On Tuesday, October 27, 2015 04:20:51 PM Tomeu Vizoso wrote:
>> >> On 27 October 2015 at 16:24, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
>> >> > Hi All,
>> >> >
>> >> > As discussed in the recent "On-demand device probing" thread and in a Kernel
>> >> > Summit session earlier today, there is a problem with handling cases where
>> >> > functional dependencies between devices are involved.
>> >> >
>> >> > What I mean by a "functional dependency" is when the driver of device B needs
>> >> > both device A and its driver to be present and functional to be able to work.
>> >> > This implies that the driver of A needs to be working for B to be probed
>> >> > successfully and it cannot be unbound from the device before the B's driver.
>> >> > This also has certain consequences for power management of these devices
>> >> > (suspend/resume and runtime PM ordering).
>> >> >
>> >> > So I want to be able to represent those functional dependencies between devices
>> >> > and I'd like the driver core to track them and act on them in certain cases
>> >> > where they matter.  The argument for doing that in the driver core is that
>> >> > there are quite a few distinct use cases related to that, they are relatively
>> >> > hard to get right in a driver (if one wants to address all of them properly)
>> >> > and it only gets worse if multiplied by the number of drivers potentially
>> >> > needing to do it.  Morever, at least one case (asynchronous system suspend/resume)
>> >> > cannot be handled in a single driver at all, because it requires the driver of A
>> >> > to wait for B to suspend (during system suspend) and the driver of B to wait for
>> >> > A to resume (during system resume).
>> >> >
>> >> > My idea is to represent a supplier-consumer dependency between devices (or
>> >> > more precisely between device+driver combos) as a "link" object containing
>> >> > pointers to the devices in question, a list node for each of them and some
>> >> > additional information related to the management of those objects, ie.
>> >> > something like:
>> >> >
>> >> > struct device_link {
>> >> >         struct device *supplier;
>> >> >         struct list_head supplier_node;
>> >> >         struct device *consumer;
>> >> >         struct list_head consumer_node;
>> >> >         <flags, status etc>
>> >> > };
>> >> >
>> >> > In general, there will be two lists of those things per device, one list
>> >> > of links to consumers and one list of links to suppliers.
>> >> >
>> >> > In that picture, links will be created by calling, say:
>> >> >
>> >> > int device_add_link(struct device *me, struct device *my_supplier, unsigned int flags);
>> >> >
>> >> > and they will be deleted by the driver core when not needed any more.  The
>> >> > creation of a link should also cause dpm_list and the list used during shutdown
>> >> > to be reordered if needed.
>> >> >
>> >> > In principle, it seems usefult to consider two types of links, one created
>> >> > at device registration time (when registering the second device from the linked
>> >> > pair, whichever it is) and one created at probe time (of the consumer device).
>> >> > I'll refer to them as "permanent" and "probe-time" links, respectively.
>> >> >
>> >> > The permanent links (created at device registration time) will stay around
>> >> > until one of the linked devices is unregistered (at which time the driver
>> >> > core will drop the link along with the device going away).  The probe-time
>> >> > ones will be dropped (automatically) at the consumer device driver unbind time.
>> >> >
>> >> > There's a question about what if the supplier device is being unbound before
>> >> > the consumer one (for example, as a result of a hotplug event).  My current
>> >> > view on that is that the consumer needs to be force-unbound in that case too,
>> >> > but I guess I may be persuaded otherwise given sufficiently convincing
>> >> > arguments.  Anyway, there are reasons to do that, like for example it may
>> >> > help with the synchronization.  Namely, if there's a rule that suppliers
>> >> > cannot be unbound before any consumers linked to them, than the list of links
>> >> > to suppliers for a consumer can only change at its registration/probe or
>> >> > unbind/remove times (which simplifies things quite a bit).
>> >> >
>> >> > With that, the permanent links existing at the probe time for a consumer
>> >> > device can be used to check whether or not to defer the probing of it
>> >> > even before executing its probe callback.  In turn, system suspend
>> >> > synchronization should be a matter of calling device_pm_wait_for_dev()
>> >> > for all consumers of a supplier device, in analogy with dpm_wait_for_children(),
>> >> > and so on.
>> >> >
>> >> > Of course, the new lists have to be stable during those operations and ensuring
>> >> > that is going to be somewhat tricky (AFAICS right now at least), but apart from
>> >> > that the whole concept looks reasonably straightforward to me.
>> >> >
>> >> > So, the question to everybody is whether or not this sounds reasonable or there
>> >> > are concerns about it and if so what they are.  At this point I mostly need to
>> >> > know if I'm not overlooking anything fundamental at the general level.
>> >>
>> >> Sounds really great to me at the conceptual level, but wonder if you
>> >> have already thought of how the permanent links will be inferred.
>> >
>> > In ACPI there is a mechanism for that already.  In DT it would require walking
>> > the phandle dependency graph I suppose.
>> >
>> > The point is, though, that it doesn't have to be mandatory to have any
>> > permanent links created.  If you can find a dependency at device registration
>> > time, great.  Create a permanent link for it and use it.  If you can't,
>> > it's fine too.  You'll find it at probe time and create a link for it then.
>> >
>> >> When I looked at computing dependencies of a device before it's
>> >> probed, the concern was that the code that finds the dependencies
>> >> duplicated part of the logic when looking resources up. Because each
>> >> subsystem has its own code for looking up dependencies for potentially
>> >> each of DT, ACPI and board files, it will be a bit of a big task to
>> >> refactor things to avoid that duplication. Fwnode could help with
>> >> this, but it doesn't as of yet and I'm not sure if that's still the
>> >> plan.
>> >
>> > That almost certainly is going to be a fair amount of work, but that
>> > doesn't mean we should avoid doing it.  If it leads to better code
>> > eventually, it's worth doing.
>> >
>> >> Also wonder if you have considered setting the permanent links also
>> >> during probe, as the on-demand probe series did (device_add_link would
>> >> be "sprinkled" around as of_device_probe was). That would avoid the
>> >> problem with code duplication because the links would be established
>> >> from the functions that do resource lookup.
>> >
>> > I have considered that, but at this point I have some concerns about
>> > lifecycle management related to that.
>>
>> Hi Rafael,
>>
>> could you extend on why do you prefer inferring the permanent links
>> before probe as opposed to collecting them during probe from the
>> lookup functions?
>
> Information that is already available at the device registration time should
> be used at that time or it makes things harder to follow.
>
> But that really is a tradeoff.  If collecting that information requires too
> much effort, it may not be worth it.
>
>> Also, have you considered that not only drivers request resources? For
>> example, the on-demand probing series would probe a device that is
>> needed by an initcall, simplifying synchronization.
>
> You really need to explain what you mean here or maybe give an example.

There are initcalls that assume that a given resource is available.
Because of async probes, or because the resource's driver being built
as a module, or because the resource's driver gained a dependency
(direct or not), those initcalls break unexpectedly at times.

If resource getters could probe dependencies on-demand, those
initcalls would be more robust to changes in other parts of the
codebase.

AFAIUI, your proposal would help with a device's dependencies being
there when it's probed, but initcalls could still run into unfulfilled
dependencies.

I would personally prefer if those initcalls were turned into drivers
because that's the component model we have in the kernel, but that
doesn't seem to be a popular idea.

Regards,

Tomeu

> Thanks,
> Rafael
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1258829

Good grief, don't you guys ever trim unwanted material from your 
emails?  I had to erase more than 4 screens worth of useless stuff 
before getting to the relevant portions.

On Thu, 29 Oct 2015, Tomeu Vizoso wrote:

> >> Also, have you considered that not only drivers request resources? For
> >> example, the on-demand probing series would probe a device that is
> >> needed by an initcall, simplifying synchronization.

Did Rafael ever say that only drivers could create these functional
dependencies?  I don't recall seeing that anywhere.  Presumably any
part of the kernel will be allowed to do it.

> > You really need to explain what you mean here or maybe give an example.
> 
> There are initcalls that assume that a given resource is available.
> Because of async probes, or because the resource's driver being built
> as a module, or because the resource's driver gained a dependency
> (direct or not), those initcalls break unexpectedly at times.
> 
> If resource getters could probe dependencies on-demand, those
> initcalls would be more robust to changes in other parts of the
> codebase.
> 
> AFAIUI, your proposal would help with a device's dependencies being
> there when it's probed, but initcalls could still run into unfulfilled
> dependencies.

One possible approach is to have a "wait_for_driver" flag, along with a
timeout value (or perhaps using a fixed timeout value).  When a
dependency gets registered with this flag set, the function call
wouldn't return until the target device is bound to a driver or the
timeout has elapsed.

This would make it easy to insert dependencies at probe time without 
relying on deferred probing.

Alan Stern

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1259896

On Thursday, October 29, 2015 10:31:08 AM Alan Stern wrote:
> Good grief, don't you guys ever trim unwanted material from your 
> emails?  I had to erase more than 4 screens worth of useless stuff 
> before getting to the relevant portions.
> 
> On Thu, 29 Oct 2015, Tomeu Vizoso wrote:
> 
> > >> Also, have you considered that not only drivers request resources? For
> > >> example, the on-demand probing series would probe a device that is
> > >> needed by an initcall, simplifying synchronization.
> 
> Did Rafael ever say that only drivers could create these functional
> dependencies?  I don't recall seeing that anywhere.  Presumably any
> part of the kernel will be allowed to do it.

Right.

> > > You really need to explain what you mean here or maybe give an example.
> > 
> > There are initcalls that assume that a given resource is available.
> > Because of async probes, or because the resource's driver being built
> > as a module, or because the resource's driver gained a dependency
> > (direct or not), those initcalls break unexpectedly at times.

Well, I'm still unsure what initcalls are in question here.

> > If resource getters could probe dependencies on-demand, those
> > initcalls would be more robust to changes in other parts of the
> > codebase.
> > 
> > AFAIUI, your proposal would help with a device's dependencies being
> > there when it's probed, but initcalls could still run into unfulfilled
> > dependencies.
> 
> One possible approach is to have a "wait_for_driver" flag, along with a
> timeout value (or perhaps using a fixed timeout value).  When a
> dependency gets registered with this flag set, the function call
> wouldn't return until the target device is bound to a driver or the
> timeout has elapsed.
> 
> This would make it easy to insert dependencies at probe time without 
> relying on deferred probing.

I'm not sure about this to be honest.  It seems like implementing it might
be sort of tricky.

Thanks,
Rafael

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1258496

[Multipart message — attachments visible in raw view] — view raw

On Tue, Oct 27, 2015 at 04:24:14PM +0100, Rafael J. Wysocki wrote:

> So, the question to everybody is whether or not this sounds reasonable or there
> are concerns about it and if so what they are.  At this point I mostly need to
> know if I'm not overlooking anything fundamental at the general level.

This seems like a good plan to me however I am concerned that only
allowing links to be created at device registration time will prove
restrictive - it means we're going to ignore anything we figure out
later on in the boot sequence.  I would be very surprised if we didn't
need that, either from things that get missed or from things that get
allocated dynamically at runtime on systems with flexible hardware, and
it'd also mean that systems can start to benefit from this for suspend
and resume without needing the updates to the firmware parsing support.

[toc] | [prev] | [next] | [standalone]

#1259893

[Multipart message — attachments visible in raw view] — view raw

On Thursday, October 29, 2015 09:15:09 AM Mark Brown wrote:
> On Tue, Oct 27, 2015 at 04:24:14PM +0100, Rafael J. Wysocki wrote:
> 
> > So, the question to everybody is whether or not this sounds reasonable or there
> > are concerns about it and if so what they are.  At this point I mostly need to
> > know if I'm not overlooking anything fundamental at the general level.
> 
> This seems like a good plan to me however I am concerned that only
> allowing links to be created at device registration time will prove
> restrictive - it means we're going to ignore anything we figure out
> later on in the boot sequence.

That's not the plan, though. :-)

I'm talking about device registration time or device probe time (for
dependencies that aren't known at the registration time).

> I would be very surprised if we didn't
> need that, either from things that get missed or from things that get
> allocated dynamically at runtime on systems with flexible hardware, and
> it'd also mean that systems can start to benefit from this for suspend
> and resume without needing the updates to the firmware parsing support.

The reason why I think it should be restricted to the probe/remove and
registration/unregistration times is because of PM.  More specifically,
adding a link to a "supplier" causes additional actions to be taken during
PM operations (eg. if the supplier device is suspended at the consumer device
resume time, it needs to be resumed in the runtime PM case or waited for in
the system resume case) which may be regarded as a change in a way PM is
handled.  Such changes are only expected to happen at the points I mentioned
and may lead to rather undesirable outcomes if made elsewhere.

Thanks,
Rafael

[toc] | [prev] | [next] | [standalone]

#1259911

[Multipart message — attachments visible in raw view] — view raw

On Sat, Oct 31, 2015 at 03:13:09AM +0100, Rafael J. Wysocki wrote:
> On Thursday, October 29, 2015 09:15:09 AM Mark Brown wrote:

> > This seems like a good plan to me however I am concerned that only
> > allowing links to be created at device registration time will prove
> > restrictive - it means we're going to ignore anything we figure out
> > later on in the boot sequence.

> That's not the plan, though. :-)

> I'm talking about device registration time or device probe time (for
> dependencies that aren't known at the registration time).

Oh, so I think allowing things to be added at probe time addresses my
concerns - for some reason I'd parsed your mail as talking about
registration time only.

[toc] | [prev] | [next] | [standalone]

#1259368

On Tue, Oct 27, 2015 at 4:24 PM, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:

> My idea is to represent a supplier-consumer dependency between devices (or
> more precisely between device+driver combos) as a "link" object containing
> pointers to the devices in question, a list node for each of them and some
> additional information related to the management of those objects, ie.
> something like:
>
> struct device_link {
>         struct device *supplier;
>         struct list_head supplier_node;
>         struct device *consumer;
>         struct list_head consumer_node;
>         <flags, status etc>
> };
>
> In general, there will be two lists of those things per device, one list
> of links to consumers and one list of links to suppliers.

I like this idea. I earlier have written that the device core needs to know
the dependencies of devices as a graph. This mechanism does that.

IIUC the mechanism does not inheritly protect against creating
cyclic graphs (you can get stuck in a loop) but that is just the
nature of the things and even deferred probe has the ability
to shoot oneself in the foot. Besides we're not doing computer
science here, we're solving practical problems.

What I further like about the approach is that it can even be used
by archs still doing boardfiles and not using any HW description
mechanism: it is there for anyone. Those platforms can define
dependencies with good old code.

Yours,
Linus Walleij
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1259843

On Tue, Oct 27, 2015 at 04:24:14PM +0100, Rafael J. Wysocki wrote:
> My idea is to represent a supplier-consumer dependency between devices (or
> more precisely between device+driver combos) as a "link" object containing
> pointers to the devices in question, a list node for each of them and some
> additional information related to the management of those objects, ie.
> something like:
> 
> struct device_link {
> 	struct device *supplier;
> 	struct list_head supplier_node;
> 	struct device *consumer;
> 	struct list_head consumer_node;
> 	<flags, status etc>
> };
> 
> In general, there will be two lists of those things per device, one list
> of links to consumers and one list of links to suppliers.
> 
> In that picture, links will be created by calling, say:
> 
> int device_add_link(struct device *me, struct device *my_supplier, unsigned int flags);

At first glance, I like this, nice.  Now to see how well it can be
implemented :)

Again, nice job.

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web