Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1591273 > unrolled thread
| Started by | Sakari Ailus <sakari.ailus@iki.fi> |
|---|---|
| First post | 2017-03-02 17:50 +0100 |
| Last post | 2017-03-11 22:40 +0100 |
| Articles | 20 on this page of 86 — 11 participants |
Back to article view | Back to linux.kernel
This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by
below is the oldest one visible, not the original post.
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Sakari Ailus <sakari.ailus@iki.fi> - 2017-03-02 17:50 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-03 01:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-03 02:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-03 03:30 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Sakari Ailus <sakari.ailus@iki.fi> - 2017-03-03 20:20 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-03 23:50 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-04 00:20 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-04 01:40 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Sakari Ailus <sakari.ailus@iki.fi> - 2017-03-04 14:20 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-10 14:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-10 14:10 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-10 14:30 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-10 15:10 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-10 15:30 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-10 17:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Sakari Ailus <sakari.ailus@iki.fi> - 2017-03-10 23:40 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-11 12:30 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Pavel Machek <pavel@ucw.cz> - 2017-03-11 23:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-12 00:20 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-12 01:30 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Pavel Machek <pavel@ucw.cz> - 2017-03-12 22:30 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-12 23:40 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Pavel Machek <pavel@ucw.cz> - 2017-03-14 19:30 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Sakari Ailus <sakari.ailus@iki.fi> - 2017-03-13 13:50 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-14 04:50 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-14 09:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-14 11:30 +0100
media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) Pavel Machek <pavel@ucw.cz> - 2017-03-14 23:40 +0100
Re: media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-15 02:00 +0100
Re: media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) Philippe De Muyter <phdm@macq.eu> - 2017-03-15 12:10 +0100
Re: media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) Nicolas Dufresne <nicolas@ndufresne.ca> - 2017-03-15 20:00 +0100
Re: media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) Philipp Zabel <p.zabel@pengutronix.de> - 2017-03-16 10:30 +0100
Re: media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) Philippe De Muyter <phdm@macq.eu> - 2017-03-16 11:00 +0100
Re: media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) Philipp Zabel <p.zabel@pengutronix.de> - 2017-03-16 11:10 +0100
Re: media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) Philippe De Muyter <phdm@macq.eu> - 2017-03-16 11:30 +0100
Re: media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) Pavel Machek <pavel@ucw.cz> - 2017-03-15 19:10 +0100
Re: media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-15 21:30 +0100
Re: media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) Pavel Machek <pavel@ucw.cz> - 2017-03-16 23:20 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-20 14:30 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-20 16:50 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-20 17:20 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-20 18:40 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-17 12:50 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Sakari Ailus <sakari.ailus@linux.intel.com> - 2017-03-17 13:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Philipp Zabel <p.zabel@pengutronix.de> - 2017-03-17 15:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-17 15:50 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-20 14:20 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-20 16:20 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-17 15:10 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Pavel Machek <pavel@ucw.cz> - 2017-03-21 12:20 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-20 12:20 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Philipp Zabel <p.zabel@pengutronix.de> - 2017-03-17 13:10 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-17 13:30 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-17 19:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Pavel Machek <pavel@ucw.cz> - 2017-03-19 14:30 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Laurent Pinchart <laurent.pinchart@ideasonboard.com> - 2017-03-26 18:50 +0200
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-10 16:30 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-10 17:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-10 18:10 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-10 21:50 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Pavel Machek <pavel@ucw.cz> - 2017-03-10 23:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-10 16:10 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-11 12:40 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-11 14:20 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Sakari Ailus <sakari.ailus@iki.fi> - 2017-03-11 16:40 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-11 18:40 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-11 19:10 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-11 19:50 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-11 20:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-11 20:10 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-11 20:10 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-11 21:50 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-12 04:40 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-12 08:40 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-12 19:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-12 23:00 +0100
script to setup pipeline was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Pavel Machek <pavel@ucw.cz> - 2017-03-26 11:20 +0200
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-13 11:50 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-13 12:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-13 12:10 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-13 12:50 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-13 13:40 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Pavel Machek <pavel@ucw.cz> - 2017-03-12 19:20 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Pavel Machek <pavel@ucw.cz> - 2017-03-11 21:30 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-11 21:40 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Pavel Machek <pavel@ucw.cz> - 2017-03-11 22:40 +0100
Page 2 of 5 — ← Prev page 1 [2] 3 4 5 Next page →
| From | Pavel Machek <pavel@ucw.cz> |
|---|---|
| Date | 2017-03-12 22:30 +0100 |
| Message-ID | <tkjdU-7cP-7@gated-at.bofh.it> |
| In reply to | #1598505 |
[Multipart message — attachments visible in raw view] — view raw
On Sat 2017-03-11 23:14:56, Russell King - ARM Linux wrote: > On Sat, Mar 11, 2017 at 08:25:49AM -0300, Mauro Carvalho Chehab wrote: > > This situation is there since 2009. If I remember well, you tried to write > > such generic plugin in the past, but never finished it, apparently because > > it is too complex. Others tried too over the years. > > > > The last trial was done by Jacek, trying to cover just the exynos4 driver. > > Yet, even such limited scope plugin was not good enough, as it was never > > merged upstream. Currently, there's no such plugins upstream. > > > > If we can't even merge a plugin that solves it for just *one* driver, > > I have no hope that we'll be able to do it for the generic case. > > This is what really worries me right now about the current proposal for > iMX6. What's being proposed is to make the driver exclusively MC-based. > > What that means is that existing applications are _not_ going to work > until we have some answer for libv4l2, and from what you've said above, > it seems that this has been attempted multiple times over the last _8_ > years, and each time it's failed. Yeah. We need a mid-layer between legacy applications and MC devices. Such layer does not exist in userspace or in kernel. > Loading the problem onto the user in the hope that the user knows > enough to properly configure it also doesn't work - who is going to > educate the user about the various quirks of the hardware they're > dealing with? We have docs. Users can write shell scripts. Still, mid-layer would be nice. > So, the problem space we have here is absolutely huge, and merely > having a plugin that activates when you open a /dev/video* node > really doesn't solve it. > > All in all, I really don't think "lets hope someone writes a v4l2 > plugin to solve it" is ever going to be successful. I don't even > see that there will ever be a userspace application that is anything > more than a representation of the dot graphs that users can use to > manually configure the capture system with system knowledge. > > I think everyone needs to take a step back and think long and hard > about this from the system usability perspective - I seriously > doubt that we will ever see any kind of solution to this if we > continue to progress with "we'll sort it in userspace some day." Mid-layer is difficult... there are _hundreds_ of possible pipeline setups. If it should live in kernel or in userspace is a question... but I don't think having it in kernel helps in any way. Best regards, Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
[toc] | [prev] | [next] | [standalone]
| From | Mauro Carvalho Chehab <mchehab@s-opensource.com> |
|---|---|
| Date | 2017-03-12 23:40 +0100 |
| Message-ID | <tkkjE-7Sg-3@gated-at.bofh.it> |
| In reply to | #1598765 |
Em Sun, 12 Mar 2017 22:29:04 +0100 Pavel Machek <pavel@ucw.cz> escreveu: > Mid-layer is difficult... there are _hundreds_ of possible > pipeline setups. If it should live in kernel or in userspace is a > question... but I don't think having it in kernel helps in any way. Mid-layer is difficult, because we either need to feed some library with knowledge for all kernel drivers or we need to improve the MC API to provide more details. For example, several drivers used to expose entities via the generic MEDIA_ENT_T_DEVNODE to represent entities of different types. See, for example, entities 1, 5 and 7 (and others) at: https://mchehab.fedorapeople.org/mc-next-gen/igepv2_omap3isp.png A device-specific code could either be hardcoding the entity number or checking for the entity strings to add some logic to setup controls on those "unknown" entities, a generic app won't be able to do anything with them, as it doesn't know what function(s) such entity provide. Also, on some devices, like the analog TV decoder at: https://mchehab.fedorapeople.org/mc-next-gen/au0828_test/mc_nextgen_test-output.png May have pads with different signals on their output. In such case, pads 1 and 2 provide video, while pad 3 provides audio using a different type of output. The application needs to know such kind of things in order to be able to properly setup the pipeline [1]. [1] this specific device has a fixed pipeline, but I'm aware of SoC with flexible pipelines that have this kind of issue (nobody upstreamed the V4L part of those devices yet). Thanks, Mauro
[toc] | [prev] | [next] | [standalone]
| From | Pavel Machek <pavel@ucw.cz> |
|---|---|
| Date | 2017-03-14 19:30 +0100 |
| Message-ID | <tkZmO-47s-9@gated-at.bofh.it> |
| In reply to | #1598788 |
[Multipart message — attachments visible in raw view] — view raw
Hi! > > Mid-layer is difficult... there are _hundreds_ of possible > > pipeline setups. If it should live in kernel or in userspace is a > > question... but I don't think having it in kernel helps in any way. > > Mid-layer is difficult, because we either need to feed some > library with knowledge for all kernel drivers or we need to improve > the MC API to provide more details. > > For example, several drivers used to expose entities via the > generic MEDIA_ENT_T_DEVNODE to represent entities of different > types. See, for example, entities 1, 5 and 7 (and others) at: > > > https://mchehab.fedorapeople.org/mc-next-gen/igepv2_omap3isp.png Well... we provide enough information, so that device-specific code does not have to be in kernel. There are few types of ENT_T_DEVNODE there. "ISP CCP2" does not really provide any functionality to the user. It just has to be there, because the pipeline needs to be connected. "ISP Preview" provides format conversion. "ISP Resizer" provides rescaling. I'm not sure if it ever makes sense to use "ISP Preview output". Normally you take data for display from "ISP Resizer output". (Would there be some power advantage from that?) > A device-specific code could either be hardcoding the entity number > or checking for the entity strings to add some logic to setup > controls on those "unknown" entities, a generic app won't be able > to do anything with them, as it doesn't know what function(s) such > entity provide. Generic app should know if it wants RGGB10 data, then it can use "ISP CCDC output", or if it wants "cooked" data suitable for display, then it wants to use "ISP Resizer output". Once application knows what output it wants, there's just one path through the system. So being able to tell the ENT_T_DEVNODEs apart does not seem to be critical. OTOH, for useful camera application, different paths are needed at different phases. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
[toc] | [prev] | [next] | [standalone]
| From | Sakari Ailus <sakari.ailus@iki.fi> |
|---|---|
| Date | 2017-03-13 13:50 +0100 |
| Message-ID | <tkxAd-ys-13@gated-at.bofh.it> |
| In reply to | #1598303 |
Hi Mauro, On Sat, Mar 11, 2017 at 08:25:49AM -0300, Mauro Carvalho Chehab wrote: > Em Sat, 11 Mar 2017 00:37:14 +0200 > Sakari Ailus <sakari.ailus@iki.fi> escreveu: > > > Hi Mauro (and others), > > > > On Fri, Mar 10, 2017 at 12:53:42PM -0300, Mauro Carvalho Chehab wrote: > > > Em Fri, 10 Mar 2017 15:20:48 +0100 > > > Hans Verkuil <hverkuil@xs4all.nl> escreveu: > > > > > > > > > > > > As I've already mentioned, from talking about this with Mauro, it seems > > > > > Mauro is in agreement with permitting the control inheritence... I wish > > > > > Mauro would comment for himself, as I can't quote our private discussion > > > > > on the subject. > > > > > > > > I can't comment either, not having seen his mail and reasoning. > > > > > > The rationale is that we should support the simplest use cases first. > > > > > > In the case of the first MC-based driver (and several subsequent > > > ones), the simplest use case required MC, as it was meant to suport > > > a custom-made sophisticated application that required fine control > > > on each component of the pipeline and to allow their advanced > > > proprietary AAA userspace-based algorithms to work. > > > > The first MC based driver (omap3isp) supports what the hardware can do, it > > does not support applications as such. > > All media drivers support a subset of what the hardware can do. The > question is if such subset covers the use cases or not. Can you name a feature in the OMAP 3 ISP that is not and can not be supported using the current driver model (MC + V4L2 sub-device + V4L2) that could be even remotely useful? > > The current MC-based drivers (except for uvc) took a patch to offer a > more advanced API, to allow direct control to each IP module, as it was > said, by the time we merged the OMAP3 driver, that, for the N9/N900 camera > to work, it was mandatory to access the pipeline's individual components. > > Such approach require that some userspace software will have knowledge > about some hardware details, in order to setup pipelines and send controls > to the right components. That makes really hard to have a generic user > friendly application to use such devices. The effect you described above is true, but I disagree with the cause. The cause is the hardware is more complex and variable than what has been supported previously, and providing a generic interface for accessing such hardware will require more complex interface. The hardware we have today and the user cases we have today are more --- not less --- complex and nuanced than when the Media controller was merged back in 2010. Arguably there is thus more need for the functionality it provides, not less. > > Non-MC based drivers control the hardware via a portable interface with > doesn't require any knowledge about the hardware specifics, as either the > Kernel or some firmware at the device will set any needed pipelines. > > In the case of V4L2 controls, when there's no subdev API, the main > driver (e. g. the driver that creates the /dev/video nodes) sends a > multicast message to all bound I2C drivers. The driver(s) that need > them handle it. When the same control may be implemented on different > drivers, the main driver sends a unicast message to just one driver[1]. > > [1] There are several non-MC drivers that have multiple ways to > control some things, like doing scaling or adjust volume levels at > either the bridge driver or at a subdriver. > > There's nothing wrong with this approach: it works, it is simpler, > it is generic. So, if it covers most use cases, why not allowing it > for usecases where a finer control is not a requirement? Drivers are written to support hardware, not particular use case. Use case specific knowledge should be present only in applications, not in drivers. Doing it otherwise will lead to use case specific drivers and more driver code to maintain for any particular piece of hardware. An individual could possibly choose the right driver for his / her use case, but this approach could hardly work for Linux distribution kernels. The plain V4L2 interface is generic within its own scope: hardware can be supported within the hardware model assumed by the interface. However, on some devices this will end up being a small subset of what the hardware can do. Besides that, when writing the driver, you need to decide at detail level what kind of subset that might be. That's not something anyone writing a driver should need to confront. > > > Adding support to drivers for different "operation modes" --- this is > > essentially what is being asked for --- is not an approach which could serve > > either purpose (some functionality with simple interface vs. fully support > > what the hardware can do, with interfaces allowing that) adequately in the > > short or the long run. > > Why not? Let's suppose that the omap3isp driver provided an "operation mode" for more simple applications. Would you continue to have a V4L2 video device per DMA engine? Without Media controller it'd be rather confusing for applications since depending on which format (and level of processing) is requested defines the video node where the images is captured. Instead you'd probably want to have a single video node. For the driver to expose just a single device node, should that be a Kconfig option or a module parameter, for instance? I have to say I wouldn't be even particularly interested to know how much driver changes you'd have to implement to achieve that and how unmaintainable to end result would be. Consider inflicting the same on all drivers. That's just *one* of a large number of things you'd need to change in order to support plain V4L2 applications from the driver, while still continuing to support the current interface. With the help of a user space library, we can show the omap3isp device as a single video node with a number of inputs (sensors) that can provide some level of service to the user. I'm using "can", because it's just up to a missing implementation of such a library. It may be hardware specific or not. A hardware specific one may produce better results than best effort since it may use knowledge of the hardware not available through the kernel interfaces. > > > If we are missing pieces in the puzzle --- in this case the missing pieces > > in the puzzle are a generic pipeline configuration library and another > > library that, with the help of pipeline autoconfiguration would implement > > "best effort" service for regular V4L2 on top of the MC + V4L2 subdev + V4L2 > > --- then these pieces need to be impelemented. The solution is > > *not* to attempt to support different types of applications in each driver > > separately. That will make writing drivers painful, error prone and is > > unlikely ever deliver what either purpose requires. > > > > So let's continue to implement the functionality that the hardware supports. > > Making a different choice here is bound to create a lasting conflict between > > having to change kernel interface behaviour and the requirement of > > supporting new functionality that hasn't been previously thought of, pushing > > away SoC vendors from V4L2 ecosystem. This is what we all do want to avoid. > > This situation is there since 2009. If I remember well, you tried to write > such generic plugin in the past, but never finished it, apparently because > it is too complex. Others tried too over the years. I'd argue I know better what happened with that attempt than you do. I had a prototype of a generic pipeline configuration library but due to various reasons I haven't been able to continue working on that since around 2012. The prototype could figure out that the ccdc -> resizer path isn't usable with raw sensors due to the lack of common formats between the two, something that was argued to be too complex to implement. I'm not aware of anyone else who would have tried that. Are you? > > The last trial was done by Jacek, trying to cover just the exynos4 driver. > Yet, even such limited scope plugin was not good enough, as it was never > merged upstream. Currently, there's no such plugins upstream. > > If we can't even merge a plugin that solves it for just *one* driver, > I have no hope that we'll be able to do it for the generic case. I believe Jacek ceased to work on that plugin in his day job; other than that, there are some matters left to be addressed in his latest patchset. Having provided feedback on that patchset, I don't see additional technical problems that require solving before the patches can be merged. The remaining matters seem to be actually fairly trivial. > > That's why I'm saying that I'm OK on merging any patch that would allow > setting controls via the /dev/video interface on MC-based drivers when > compiled without subdev API. I may also consider merging patches allowing > to change the behavior on runtime, when compiled with subdev API. > > > As far as i.MX6 driver goes, it is always possible to implement i.MX6 plugin > > for libv4l to perform this. This should be much easier than getting the > > automatic pipe configuration library and the rest working, and as it is > > custom for i.MX6, the resulting plugin may make informed technical choices > > for better functionality. > > I wouldn't call "much easier" something that experienced media > developers failed to do over the last 8 years. > > It is just the opposite: broadcasting a control via I2C is very easy: > there are several examples about how to do that all over the media > drivers. That's one of the things Jacek's plugin actually does. This is still functionality that *only* some user applications wish to have, and as implemented in the plugin it works correctly without use case specific semantics --- please see my comments on the patch picking the controls from the pipeline. Drivers can also make use of v4l2_device_for_each_subdev() to distribute setting controls on different sub-devices. A few drivers actually do that. > > > Jacek has been working on such a plugin for > > Samsung Exynos hardware, but I don't think he has quite finished it yey. > > As Jacek answered when questioned about the merge status: > > Hi Hans, > > On 11/03/2016 12:51 PM, Hans Verkuil wrote: > > Hi all, > > > > Is there anything that blocks me from merging this? > > > > This plugin work has been ongoing for years and unless there are serious > > objections I propose that this is merged. > > > > Jacek, is there anything missing that would prevent merging this? > > There were issues raised by Sakari during last review, related to > the way how v4l2 control bindings are defined. That discussion wasn't > finished, so I stayed by my approach. Other than that - I've tested it > and it works fine both with GStreamer and my test app. > > After that, he sent a new version (v7.1), but never got reviews. There are a number of mutually agreed but unaddressed comments against v7. Also, v7.1 is just a single patch that does not address issues pointed out in v7, but something Jacek wanted to fix himself. In other words, there are comments to address but no patches to review. Let's see how to best address them. I could possibly fix at least some of those, but due to lack of hardware I have no ability to test the end result. > > > The original plan was and continues to be sound, it's just that there have > > always been too few hands to implement it. :-( > > If there are no people to implement a plan, it doesn't matter how good > the plan is, it won't work. Do you have other proposals than what we have commonly agreed on? I don't see other approaches that could satisfactorily address all the requirements going forward. That said, I do think we need to reinvigorate the efforts to get things rolling again on supporting plain V4L2 applications on devices that are controlled through the MC inteface. These matters have received undeservedly little attention in recent years. > > > > That's not true, for example, for the UVC driver. There, MC > > > is optional, as it should be. > > > > UVC is different. The device simply provides additional information through > > MC to the user but MC (or V4L2 sub-device interface) is not used for > > controlling the device. > > It is not different. If the Kernel is compiled without the V4L2 > subdev interface, the i.MX6 driver (or whatever other driver) > won't receive any control via the subdev interface. So, it has to > handle the control logic control via the only interface that > supports it, e. g. via the video devnode. Looking at the driver code and the Kconfig file, i.MX6 driver depends on CONFIG_MEDIA_CONTROLLER and uses the Media controller API. So if Media controller is disabled in kernel configuration, the i.MX6 IPU driver won't be compiled. -- Kind regards, Sakari Ailus e-mail: sakari.ailus@iki.fi XMPP: sailus@retiisi.org.uk
[toc] | [prev] | [next] | [standalone]
| From | Mauro Carvalho Chehab <mchehab@s-opensource.com> |
|---|---|
| Date | 2017-03-14 04:50 +0100 |
| Message-ID | <tkLDb-2Dk-9@gated-at.bofh.it> |
| In reply to | #1599303 |
Hi Sakari, I started preparing a long argument about it, but gave up in favor of a simpler one. Em Mon, 13 Mar 2017 14:46:22 +0200 Sakari Ailus <sakari.ailus@iki.fi> escreveu: > Drivers are written to support hardware, not particular use case. No, it is just the reverse: drivers and hardware are developed to support use cases. Btw, you should remember that the hardware is the full board, not just the SoC. In practice, the board do limit the use cases: several provide a single physical CSI connector, allowing just one sensor. > > This situation is there since 2009. If I remember well, you tried to write > > such generic plugin in the past, but never finished it, apparently because > > it is too complex. Others tried too over the years. > > I'd argue I know better what happened with that attempt than you do. I had a > prototype of a generic pipeline configuration library but due to various > reasons I haven't been able to continue working on that since around 2012. ... > > The last trial was done by Jacek, trying to cover just the exynos4 driver. > > Yet, even such limited scope plugin was not good enough, as it was never > > merged upstream. Currently, there's no such plugins upstream. > > > > If we can't even merge a plugin that solves it for just *one* driver, > > I have no hope that we'll be able to do it for the generic case. > > I believe Jacek ceased to work on that plugin in his day job; other than > that, there are some matters left to be addressed in his latest patchset. The two above basically summaries the issue: the task of doing a generic plugin on userspace, even for a single driver is complex enough to not cover within a reasonable timeline. From 2009 to 2012, you were working on it, but didn't finish it. Apparently, nobody worked on it between 2013-2014 (but I may be wrong, as I didn't check when the generic plugin interface was added to libv4l). In the case of Jacek's work, the first patch I was able to find was written in Oct, 2014: https://patchwork.kernel.org/patch/5098111/ (not sure what happened with the version 1). The last e-mail about this subject was issued in Dec, 2016. In summary, you had this on your task for 3 years for an OMAP3 plugin (where you have a good expertise), and Jacek for 2 years, for Exynos 4, where he should also have a good knowledge. Yet, with all that efforts, no concrete results were achieved, as none of the plugins got merged. Even if they were merged, if we keep the same mean time to develop a libv4l plugin, that would mean that a plugin for i.MX6 could take 2-3 years to be developed. There's a clear message on it: - we shouldn't keep pushing for a solution via libv4l. Thanks, Mauro
[toc] | [prev] | [next] | [standalone]
| From | Hans Verkuil <hverkuil@xs4all.nl> |
|---|---|
| Date | 2017-03-14 09:00 +0100 |
| Message-ID | <tkPx8-5ja-19@gated-at.bofh.it> |
| In reply to | #1600020 |
On 03/14/2017 04:45 AM, Mauro Carvalho Chehab wrote: > Hi Sakari, > > I started preparing a long argument about it, but gave up in favor of a > simpler one. > > Em Mon, 13 Mar 2017 14:46:22 +0200 > Sakari Ailus <sakari.ailus@iki.fi> escreveu: > >> Drivers are written to support hardware, not particular use case. > > No, it is just the reverse: drivers and hardware are developed to > support use cases. > > Btw, you should remember that the hardware is the full board, not just the > SoC. In practice, the board do limit the use cases: several provide a > single physical CSI connector, allowing just one sensor. > >>> This situation is there since 2009. If I remember well, you tried to write >>> such generic plugin in the past, but never finished it, apparently because >>> it is too complex. Others tried too over the years. >> >> I'd argue I know better what happened with that attempt than you do. I had a >> prototype of a generic pipeline configuration library but due to various >> reasons I haven't been able to continue working on that since around 2012. > > ... > >>> The last trial was done by Jacek, trying to cover just the exynos4 driver. >>> Yet, even such limited scope plugin was not good enough, as it was never >>> merged upstream. Currently, there's no such plugins upstream. >>> >>> If we can't even merge a plugin that solves it for just *one* driver, >>> I have no hope that we'll be able to do it for the generic case. >> >> I believe Jacek ceased to work on that plugin in his day job; other than >> that, there are some matters left to be addressed in his latest patchset. > > The two above basically summaries the issue: the task of doing a generic > plugin on userspace, even for a single driver is complex enough to > not cover within a reasonable timeline. > > From 2009 to 2012, you were working on it, but didn't finish it. > > Apparently, nobody worked on it between 2013-2014 (but I may be wrong, as > I didn't check when the generic plugin interface was added to libv4l). > > In the case of Jacek's work, the first patch I was able to find was > written in Oct, 2014: > https://patchwork.kernel.org/patch/5098111/ > (not sure what happened with the version 1). > > The last e-mail about this subject was issued in Dec, 2016. > > In summary, you had this on your task for 3 years for an OMAP3 > plugin (where you have a good expertise), and Jacek for 2 years, > for Exynos 4, where he should also have a good knowledge. > > Yet, with all that efforts, no concrete results were achieved, as none > of the plugins got merged. > > Even if they were merged, if we keep the same mean time to develop a > libv4l plugin, that would mean that a plugin for i.MX6 could take 2-3 > years to be developed. > > There's a clear message on it: > - we shouldn't keep pushing for a solution via libv4l. Or: - userspace plugin development had a very a low priority and never got the attention it needed. I know that's *my* reason. I rarely if ever looked at it. I always assumed Sakari and/or Laurent would look at it. If this reason is also valid for Sakari and Laurent, then it is no wonder nothing has happened in all that time. We're all very driver-development-driven, and userspace gets very little attention in general. So before just throwing in the towel we should take a good look at the reasons why there has been little or no development: is it because of fundamental design defects, or because nobody paid attention to it? I strongly suspect it is the latter. In addition, I suspect end-users of these complex devices don't really care about a plugin: they want full control and won't typically use generic applications. If they would need support for that, we'd have seen much more interest. The main reason for having a plugin is to simplify testing and if this is going to be used on cheap hobbyist devkits. An additional complication is simply that it is hard to find fully supported MC hardware. omap3 boards are hard to find these days, renesas boards are not easy to get, freescale isn't the most popular either. Allwinner, mediatek, amlogic, broadcom and qualcomm all have closed source implementations or no implementation at all. I know it took me a very long time before I had a working omap3. So I am not at all surprised that little progress has been made. Regards, Hans
[toc] | [prev] | [next] | [standalone]
| From | Mauro Carvalho Chehab <mchehab@s-opensource.com> |
|---|---|
| Date | 2017-03-14 11:30 +0100 |
| Message-ID | <tkRSh-74u-5@gated-at.bofh.it> |
| In reply to | #1600091 |
Em Tue, 14 Mar 2017 08:55:36 +0100 Hans Verkuil <hverkuil@xs4all.nl> escreveu: > On 03/14/2017 04:45 AM, Mauro Carvalho Chehab wrote: > > Hi Sakari, > > > > I started preparing a long argument about it, but gave up in favor of a > > simpler one. > > > > Em Mon, 13 Mar 2017 14:46:22 +0200 > > Sakari Ailus <sakari.ailus@iki.fi> escreveu: > > > >> Drivers are written to support hardware, not particular use case. > > > > No, it is just the reverse: drivers and hardware are developed to > > support use cases. > > > > Btw, you should remember that the hardware is the full board, not just the > > SoC. In practice, the board do limit the use cases: several provide a > > single physical CSI connector, allowing just one sensor. > > > >>> This situation is there since 2009. If I remember well, you tried to write > >>> such generic plugin in the past, but never finished it, apparently because > >>> it is too complex. Others tried too over the years. > >> > >> I'd argue I know better what happened with that attempt than you do. I had a > >> prototype of a generic pipeline configuration library but due to various > >> reasons I haven't been able to continue working on that since around 2012. > > > > ... > > > >>> The last trial was done by Jacek, trying to cover just the exynos4 driver. > >>> Yet, even such limited scope plugin was not good enough, as it was never > >>> merged upstream. Currently, there's no such plugins upstream. > >>> > >>> If we can't even merge a plugin that solves it for just *one* driver, > >>> I have no hope that we'll be able to do it for the generic case. > >> > >> I believe Jacek ceased to work on that plugin in his day job; other than > >> that, there are some matters left to be addressed in his latest patchset. > > > > The two above basically summaries the issue: the task of doing a generic > > plugin on userspace, even for a single driver is complex enough to > > not cover within a reasonable timeline. > > > > From 2009 to 2012, you were working on it, but didn't finish it. > > > > Apparently, nobody worked on it between 2013-2014 (but I may be wrong, as > > I didn't check when the generic plugin interface was added to libv4l). > > > > In the case of Jacek's work, the first patch I was able to find was > > written in Oct, 2014: > > https://patchwork.kernel.org/patch/5098111/ > > (not sure what happened with the version 1). > > > > The last e-mail about this subject was issued in Dec, 2016. > > > > In summary, you had this on your task for 3 years for an OMAP3 > > plugin (where you have a good expertise), and Jacek for 2 years, > > for Exynos 4, where he should also have a good knowledge. > > > > Yet, with all that efforts, no concrete results were achieved, as none > > of the plugins got merged. > > > > Even if they were merged, if we keep the same mean time to develop a > > libv4l plugin, that would mean that a plugin for i.MX6 could take 2-3 > > years to be developed. > > > > There's a clear message on it: > > - we shouldn't keep pushing for a solution via libv4l. > > Or: > > - userspace plugin development had a very a low priority and > never got the attention it needed. The end result is the same: we can't count on it. > > I know that's *my* reason. I rarely if ever looked at it. I always assumed > Sakari and/or Laurent would look at it. If this reason is also valid for > Sakari and Laurent, then it is no wonder nothing has happened in all that > time. > > We're all very driver-development-driven, and userspace gets very little > attention in general. So before just throwing in the towel we should take > a good look at the reasons why there has been little or no development: is > it because of fundamental design defects, or because nobody paid attention > to it? No. We should look it the other way: basically, there are patches for i.MX6 driver that sends control from videonode to subdevs. If we nack apply it, who will write the userspace plugin? When such change will be merged upstream? If we don't have answers to any of the above questions, we should not nack it. That's said, that doesn't prevent merging a libv4l plugin if/when someone can find time/interest to develop it. > I strongly suspect it is the latter. > > In addition, I suspect end-users of these complex devices don't really care > about a plugin: they want full control and won't typically use generic > applications. If they would need support for that, we'd have seen much more > interest. The main reason for having a plugin is to simplify testing and > if this is going to be used on cheap hobbyist devkits. What are the needs for a cheap hobbyist devkit owner? Do we currently satisfy those needs? I'd say that having a functional driver when compiled without the subdev API, that implements the ioctl's/controls that a generic application like camorama/google talk/skype/zbar... would work should be enough to make them happy, even if they need to add some udev rule and/or run some "prep" application that would setup the pipelines via MC and eventually rename the device with a working pipeline to /dev/video0. > > An additional complication is simply that it is hard to find fully supported > MC hardware. omap3 boards are hard to find these days, renesas boards are not > easy to get, freescale isn't the most popular either. Allwinner, mediatek, > amlogic, broadcom and qualcomm all have closed source implementations or no > implementation at all. I'd say that we should not care anymore on providing a solution for generic applications to run on boards like OMAP3[1]. For hardware that are currently available that have Kernel driver and boards developed to be used as "cheap hobbyist devkit", I'd say we should implement a Kernel solution that would allow them to be used without subdev API, e. g. having all ioctls needed by generic applications to work functional, after some external application sets the pipeline. [1] Yet, I might eventually do that for fun, an OMAP3 board with tvp5150 just arrived here last week. It would be nice to have xawtv3 running on it :-) So, if I have a lot of spare time (with is very unlikely), I might eventually do something for it to work. > I know it took me a very long time before I had a working omap3. My first OMAP3 board with working V4L2 source just arrived last week :-) > So I am not at all surprised that little progress has been made. I'm not surprised, but I'm disappointed, as I tried to push toward a solution for this problem since when we had our initial meetings about it. Thanks, Mauro
[toc] | [prev] | [next] | [standalone]
| From | Pavel Machek <pavel@ucw.cz> |
|---|---|
| Date | 2017-03-14 23:40 +0100 |
| Subject | media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) |
| Message-ID | <tl3gJ-6Qt-1@gated-at.bofh.it> |
| In reply to | #1600195 |
[Multipart message — attachments visible in raw view] — view raw
Hi! > > > Even if they were merged, if we keep the same mean time to develop a > > > libv4l plugin, that would mean that a plugin for i.MX6 could take 2-3 > > > years to be developed. > > > > > > There's a clear message on it: > > > - we shouldn't keep pushing for a solution via libv4l. > > > > Or: > > > > - userspace plugin development had a very a low priority and > > never got the attention it needed. > > The end result is the same: we can't count on it. > > > > > I know that's *my* reason. I rarely if ever looked at it. I always assumed > > Sakari and/or Laurent would look at it. If this reason is also valid for > > Sakari and Laurent, then it is no wonder nothing has happened in all that > > time. > > > > We're all very driver-development-driven, and userspace gets very little > > attention in general. So before just throwing in the towel we should take > > a good look at the reasons why there has been little or no development: is > > it because of fundamental design defects, or because nobody paid attention > > to it? > > No. We should look it the other way: basically, there are patches > for i.MX6 driver that sends control from videonode to subdevs. > > If we nack apply it, who will write the userspace plugin? When > such change will be merged upstream? Well, I believe first question is: what applications would we want to run on complex devices? Will sending control from video to subdevs actually help? mplayer is useful for testing... but that one already works (after you setup the pipeline, and configure exposure/gain). But thats useful for testing, not really for production. Image will be out of focus and with wrong white balance. What I would really like is an application to get still photos. For taking pictures with manual settings we need a) units for controls: user wants to focus on 1m, and take picture with ISO200, 1/125 sec. We should also tell him that lens is f/5.6 and focal length is 20mm with 5mm chip. But... autofocus/autogain would really be good to have. Thus we need: b) for each frame, we need exposure settings and focus position at time frame was taken. Otherwise autofocus/autogain will be too slow. At least focus position is going to be tricky -- either kernel would have to compute focus position for us (not trivial) or we'd need enough information to compute it in userspace. There are more problems: hardware-accelerated preview is not trivial to set up (and I'm unsure if it can be done in generic way). Still photos application needs to switch resolutions between preview and photo capture. Probably hardware-accelerated histograms are needed for white balance, auto gain and auto focus, .... It seems like there's a _lot_ of stuff to be done before we have useful support for complex cameras... (And I'm not sure... when application such as skype is running, is there some way to run autogain/autofocus/autowhitebalance? Is that something we want to support?) > If we don't have answers to any of the above questions, we should not > nack it. > > That's said, that doesn't prevent merging a libv4l plugin if/when > someone can find time/interest to develop it. I believe other question is: will not having same control on main video device and subdevs be confusing? Does it actually help userspace in any way? Yes, we can make controls accessible to old application, but does it make them more useful? > > In addition, I suspect end-users of these complex devices don't really care > > about a plugin: they want full control and won't typically use generic > > applications. If they would need support for that, we'd have seen much more > > interest. The main reason for having a plugin is to simplify testing and > > if this is going to be used on cheap hobbyist devkits. > > What are the needs for a cheap hobbyist devkit owner? Do we currently > satisfy those needs? I'd say that having a functional driver when > compiled without the subdev API, that implements the ioctl's/controls Having different interface based on config options... is just weird. What about poor people (like me) trying to develop complex applications? > [1] Yet, I might eventually do that for fun, an OMAP3 board with tvp5150 > just arrived here last week. It would be nice to have xawtv3 running on it :-) > So, if I have a lot of spare time (with is very unlikely), I might eventually > do something for it to work. > > > I know it took me a very long time before I had a working omap3. > > My first OMAP3 board with working V4L2 source just arrived last week > :-) You can get Nokia N900 on aliexpress. If not, they are still available between people :-). Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
[toc] | [prev] | [next] | [standalone]
| From | Mauro Carvalho Chehab <mchehab@s-opensource.com> |
|---|---|
| Date | 2017-03-15 02:00 +0100 |
| Subject | Re: media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) |
| Message-ID | <tl5se-8fK-13@gated-at.bofh.it> |
| In reply to | #1600939 |
Em Tue, 14 Mar 2017 23:32:54 +0100 Pavel Machek <pavel@ucw.cz> escreveu: > Hi! > > > > > Even if they were merged, if we keep the same mean time to develop a > > > > libv4l plugin, that would mean that a plugin for i.MX6 could take 2-3 > > > > years to be developed. > > > > > > > > There's a clear message on it: > > > > - we shouldn't keep pushing for a solution via libv4l. > > > > > > Or: > > > > > > - userspace plugin development had a very a low priority and > > > never got the attention it needed. > > > > The end result is the same: we can't count on it. > > > > > > > > I know that's *my* reason. I rarely if ever looked at it. I always assumed > > > Sakari and/or Laurent would look at it. If this reason is also valid for > > > Sakari and Laurent, then it is no wonder nothing has happened in all that > > > time. > > > > > > We're all very driver-development-driven, and userspace gets very little > > > attention in general. So before just throwing in the towel we should take > > > a good look at the reasons why there has been little or no development: is > > > it because of fundamental design defects, or because nobody paid attention > > > to it? > > > > No. We should look it the other way: basically, there are patches > > for i.MX6 driver that sends control from videonode to subdevs. > > > > If we nack apply it, who will write the userspace plugin? When > > such change will be merged upstream? > > Well, I believe first question is: what applications would we want to > run on complex devices? Will sending control from video to subdevs > actually help? I would say: camorama, xawtv3, zbar, google talk, skype. If it runs with those, it will likely run with any other application. > mplayer is useful for testing... but that one already works (after you > setup the pipeline, and configure exposure/gain). > > But thats useful for testing, not really for production. Image will be > out of focus and with wrong white balance. > > What I would really like is an application to get still photos. For > taking pictures with manual settings we need > > a) units for controls: user wants to focus on 1m, and take picture > with ISO200, 1/125 sec. We should also tell him that lens is f/5.6 and > focal length is 20mm with 5mm chip. > > But... autofocus/autogain would really be good to have. Thus we need: > > b) for each frame, we need exposure settings and focus position at > time frame was taken. Otherwise autofocus/autogain will be too > slow. At least focus position is going to be tricky -- either kernel > would have to compute focus position for us (not trivial) or we'd need > enough information to compute it in userspace. > > There are more problems: hardware-accelerated preview is not trivial > to set up (and I'm unsure if it can be done in generic way). Still > photos application needs to switch resolutions between preview and > photo capture. Probably hardware-accelerated histograms are needed for > white balance, auto gain and auto focus, .... > > It seems like there's a _lot_ of stuff to be done before we have > useful support for complex cameras... Taking still pictures using a hardware-accelerated preview is a sophisticated use case. I don't know any userspace application that does that. Ok, several allow taking snapshots, by simply storing the image of the current frame. > (And I'm not sure... when application such as skype is running, is > there some way to run autogain/autofocus/autowhitebalance? Is that > something we want to support?) Autofocus no. Autogain/Autowhite can be done via libv4l, provided that it can access the device's controls via /dev/video devnode. Other applications may be using some other similar algorithms. Ok, they don't use histograms provided by the SoC. So, they do it in software, with is slower. Still, it works fine when the light conditions don't change too fast. > > If we don't have answers to any of the above questions, we should not > > nack it. > > > > That's said, that doesn't prevent merging a libv4l plugin if/when > > someone can find time/interest to develop it. > > I believe other question is: will not having same control on main > video device and subdevs be confusing? Does it actually help userspace > in any way? Yes, we can make controls accessible to old application, > but does it make them more useful? Yes. As I said, libv4l (and some apps) have logic inside to adjust the image via bright, contrast and white balance controls, using the video devnode. They don't talk subdev API. So, if those controls aren't exported, they won't be able to provide a good quality image. > > > In addition, I suspect end-users of these complex devices don't really care > > > about a plugin: they want full control and won't typically use generic > > > applications. If they would need support for that, we'd have seen much more > > > interest. The main reason for having a plugin is to simplify testing and > > > if this is going to be used on cheap hobbyist devkits. > > > > What are the needs for a cheap hobbyist devkit owner? Do we currently > > satisfy those needs? I'd say that having a functional driver when > > compiled without the subdev API, that implements the ioctl's/controls > > Having different interface based on config options... is just > weird. What about poor people (like me) trying to develop complex > applications? Well, that could be done using other mechanisms, like a modprobe parameter or by switching the behaviour if a subdev interface is opened. I don't see much trouble on allowing accessing a control via both interfaces. > > > [1] Yet, I might eventually do that for fun, an OMAP3 board with tvp5150 > > just arrived here last week. It would be nice to have xawtv3 running on it :-) > > So, if I have a lot of spare time (with is very unlikely), I might eventually > > do something for it to work. > > > > > I know it took me a very long time before I had a working omap3. > > > > My first OMAP3 board with working V4L2 source just arrived last week > > :-) > > You can get Nokia N900 on aliexpress. If not, they are still available > between people :-) I have one. Unfortunately, I never had a chance to use it, as the display stopped working one week after I get it. Thanks, Mauro
[toc] | [prev] | [next] | [standalone]
| From | Philippe De Muyter <phdm@macq.eu> |
|---|---|
| Date | 2017-03-15 12:10 +0100 |
| Subject | Re: media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) |
| Message-ID | <tleYy-6NF-13@gated-at.bofh.it> |
| In reply to | #1600971 |
On Tue, Mar 14, 2017 at 09:54:31PM -0300, Mauro Carvalho Chehab wrote: > Em Tue, 14 Mar 2017 23:32:54 +0100 > Pavel Machek <pavel@ucw.cz> escreveu: > > > > > Well, I believe first question is: what applications would we want to > > run on complex devices? Will sending control from video to subdevs > > actually help? > > I would say: camorama, xawtv3, zbar, google talk, skype. If it runs > with those, it will likely run with any other application. > I would like to add the 'v4l2src' plugin of gstreamer, and on the imx6 its imx-specific counterpart 'imxv4l2videosrc' from the gstreamer-imx package at https://github.com/Freescale/gstreamer-imx, and 'v4l2-ctl'. Philippe -- Philippe De Muyter +32 2 6101532 Macq SA rue de l'Aeronef 2 B-1140 Bruxelles
[toc] | [prev] | [next] | [standalone]
| From | Nicolas Dufresne <nicolas@ndufresne.ca> |
|---|---|
| Date | 2017-03-15 20:00 +0100 |
| Subject | Re: media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) |
| Message-ID | <tlmjo-3g1-19@gated-at.bofh.it> |
| In reply to | #1601278 |
[Multipart message — attachments visible in raw view] — view raw
Le mercredi 15 mars 2017 à 11:50 +0100, Philippe De Muyter a écrit : > > I would say: camorama, xawtv3, zbar, google talk, skype. If it runs > > with those, it will likely run with any other application. > > > > I would like to add the 'v4l2src' plugin of gstreamer, and on the > imx6 its While it would be nice if somehow you would get v4l2src to work (in some legacy/emulation mode through libv4l2), the longer plan is to implement smart bin that handle several v4l2src, that can do the required interactions so we can expose similar level of controls as found in Android Camera HAL3, and maybe even further assuming userspace can change the media tree at run-time. We might be a long way from there, specially that some of the features depends on how much the hardware can do. Just being able to figure-out how to build the MC tree dynamically seems really hard when thinking of generic mechanism. Also, Request API will be needed. I think for this one, we'll need some userspace driver that enable the features (not hide them), and that's what I'd be looking for from libv4l2 in this regard. > imx-specific counterpart 'imxv4l2videosrc' from the gstreamer-imx > package > at https://github.com/Freescale/gstreamer-imx, and 'v4l2-ctl'. This one is specific to IMX hardware using vendor driver. You can probably ignore that. Nicolas
[toc] | [prev] | [next] | [standalone]
| From | Philipp Zabel <p.zabel@pengutronix.de> |
|---|---|
| Date | 2017-03-16 10:30 +0100 |
| Subject | Re: media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) |
| Message-ID | <tlzTj-4Bt-1@gated-at.bofh.it> |
| In reply to | #1601663 |
On Wed, 2017-03-15 at 14:55 -0400, Nicolas Dufresne wrote: > Le mercredi 15 mars 2017 à 11:50 +0100, Philippe De Muyter a écrit : > > > I would say: camorama, xawtv3, zbar, google talk, skype. If it runs > > > with those, it will likely run with any other application. > > > > > > > I would like to add the 'v4l2src' plugin of gstreamer, and on the > > imx6 its > > While it would be nice if somehow you would get v4l2src to work (in > some legacy/emulation mode through libv4l2), v4l2src works just fine, provided the pipeline is configured manually in advance via media-ctl. > the longer plan is to > implement smart bin that handle several v4l2src, that can do the > required interactions so we can expose similar level of controls as > found in Android Camera HAL3, and maybe even further assuming userspace > can change the media tree at run-time. We might be a long way from > there, specially that some of the features depends on how much the > hardware can do. Just being able to figure-out how to build the MC tree > dynamically seems really hard when thinking of generic mechanism. Also, > Request API will be needed. > > I think for this one, we'll need some userspace driver that enable the > features (not hide them), and that's what I'd be looking for from > libv4l2 in this regard. regards Philipp
[toc] | [prev] | [next] | [standalone]
| From | Philippe De Muyter <phdm@macq.eu> |
|---|---|
| Date | 2017-03-16 11:00 +0100 |
| Subject | Re: media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) |
| Message-ID | <tlAmn-4QJ-23@gated-at.bofh.it> |
| In reply to | #1602122 |
On Thu, Mar 16, 2017 at 10:26:00AM +0100, Philipp Zabel wrote: > On Wed, 2017-03-15 at 14:55 -0400, Nicolas Dufresne wrote: > > Le mercredi 15 mars 2017 à 11:50 +0100, Philippe De Muyter a écrit : > > > > I would say: camorama, xawtv3, zbar, google talk, skype. If it runs > > > > with those, it will likely run with any other application. > > > > > > > > > > I would like to add the 'v4l2src' plugin of gstreamer, and on the > > > imx6 its > > > > While it would be nice if somehow you would get v4l2src to work (in > > some legacy/emulation mode through libv4l2), > > v4l2src works just fine, provided the pipeline is configured manually in > advance via media-ctl. Including choosing the framerate ? Sorry, I have no time these days to test it myself. And I cited imxv4l2videosrc for its ability to provide the physical address of the image buffers for further processing by other (not necessarily next in gstreamer pipeline, or for all frames) hardware-accelerated plugins likes the h.264 video encoder. As I am stuck with fsl/nxp kernel and driver on that matter, I don't know how the interfaces have evolved in current linux kernels. BR Philippe -- Philippe De Muyter +32 2 6101532 Macq SA rue de l'Aeronef 2 B-1140 Bruxelles
[toc] | [prev] | [next] | [standalone]
| From | Philipp Zabel <p.zabel@pengutronix.de> |
|---|---|
| Date | 2017-03-16 11:10 +0100 |
| Subject | Re: media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) |
| Message-ID | <tlAw2-59w-15@gated-at.bofh.it> |
| In reply to | #1602145 |
On Thu, 2017-03-16 at 10:47 +0100, Philippe De Muyter wrote: > On Thu, Mar 16, 2017 at 10:26:00AM +0100, Philipp Zabel wrote: > > On Wed, 2017-03-15 at 14:55 -0400, Nicolas Dufresne wrote: > > > Le mercredi 15 mars 2017 à 11:50 +0100, Philippe De Muyter a écrit : > > > > > I would say: camorama, xawtv3, zbar, google talk, skype. If it runs > > > > > with those, it will likely run with any other application. > > > > > > > > > > > > > I would like to add the 'v4l2src' plugin of gstreamer, and on the > > > > imx6 its > > > > > > While it would be nice if somehow you would get v4l2src to work (in > > > some legacy/emulation mode through libv4l2), > > > > v4l2src works just fine, provided the pipeline is configured manually in > > advance via media-ctl. > > Including choosing the framerate ? Sorry, I have no time these days > to test it myself. No, the framerate is set with media-ctl on the CSI output pad. To really choose the framerate, the element would indeed need a deeper understanding of the pipeline, as the resulting framerate depends on at least the source v4l2_subdevice (sensor) framerate and the CSI frame skipping. > And I cited imxv4l2videosrc for its ability to provide the physical address > of the image buffers for further processing by other (not necessarily next > in gstreamer pipeline, or for all frames) hardware-accelerated plugins likes > the h.264 video encoder. As I am stuck with fsl/nxp kernel and driver on that > matter, I don't know how the interfaces have evolved in current linux kernels. The physical address of the image buffers is hidden from userspace by dma-buf objects, but those can be passed around to the next driver without copying the image data. regards Philipp
[toc] | [prev] | [next] | [standalone]
| From | Philippe De Muyter <phdm@macq.eu> |
|---|---|
| Date | 2017-03-16 11:30 +0100 |
| Subject | Re: media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) |
| Message-ID | <tlAPo-5jO-15@gated-at.bofh.it> |
| In reply to | #1602151 |
On Thu, Mar 16, 2017 at 11:01:56AM +0100, Philipp Zabel wrote: > On Thu, 2017-03-16 at 10:47 +0100, Philippe De Muyter wrote: > > On Thu, Mar 16, 2017 at 10:26:00AM +0100, Philipp Zabel wrote: > > > On Wed, 2017-03-15 at 14:55 -0400, Nicolas Dufresne wrote: > > > > Le mercredi 15 mars 2017 à 11:50 +0100, Philippe De Muyter a écrit : > > > > > > I would say: camorama, xawtv3, zbar, google talk, skype. If it runs > > > > > > with those, it will likely run with any other application. > > > > > > > > > > > > > > > > I would like to add the 'v4l2src' plugin of gstreamer, and on the > > > > > imx6 its > > > > > > > > While it would be nice if somehow you would get v4l2src to work (in > > > > some legacy/emulation mode through libv4l2), > > > > > > v4l2src works just fine, provided the pipeline is configured manually in > > > advance via media-ctl. > > > > Including choosing the framerate ? Sorry, I have no time these days > > to test it myself. > > No, the framerate is set with media-ctl on the CSI output pad. To really > choose the framerate, the element would indeed need a deeper > understanding of the pipeline, as the resulting framerate depends on at > least the source v4l2_subdevice (sensor) framerate and the CSI frame > skipping. Count me in than as a supporter of Steve's "v4l2-mc: add a function to inherit controls from a pipeline" patch > > of the image buffers for further processing by other (not necessarily next > > in gstreamer pipeline, or for all frames) hardware-accelerated plugins likes > > the h.264 video encoder. As I am stuck with fsl/nxp kernel and driver on that > > matter, I don't know how the interfaces have evolved in current linux kernels. > > The physical address of the image buffers is hidden from userspace by > dma-buf objects, but those can be passed around to the next driver > without copying the image data. OK thanks Philippe
[toc] | [prev] | [next] | [standalone]
| From | Pavel Machek <pavel@ucw.cz> |
|---|---|
| Date | 2017-03-15 19:10 +0100 |
| Subject | Re: media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) |
| Message-ID | <tllx0-2TA-7@gated-at.bofh.it> |
| In reply to | #1600971 |
[Multipart message — attachments visible in raw view] — view raw
Hi! > > Well, I believe first question is: what applications would we want to > > run on complex devices? Will sending control from video to subdevs > > actually help? > > I would say: camorama, xawtv3, zbar, google talk, skype. If it runs > with those, it will likely run with any other application. I'll take a look when I'm at better internet access. > > mplayer is useful for testing... but that one already works (after you > > setup the pipeline, and configure exposure/gain). > > > > But thats useful for testing, not really for production. Image will be > > out of focus and with wrong white balance. > > > > What I would really like is an application to get still photos. For > > taking pictures with manual settings we need > > > > a) units for controls: user wants to focus on 1m, and take picture > > with ISO200, 1/125 sec. We should also tell him that lens is f/5.6 and > > focal length is 20mm with 5mm chip. > > > > But... autofocus/autogain would really be good to have. Thus we need: > > > > b) for each frame, we need exposure settings and focus position at > > time frame was taken. Otherwise autofocus/autogain will be too > > slow. At least focus position is going to be tricky -- either kernel > > would have to compute focus position for us (not trivial) or we'd need > > enough information to compute it in userspace. > > > > There are more problems: hardware-accelerated preview is not trivial > > to set up (and I'm unsure if it can be done in generic way). Still > > photos application needs to switch resolutions between preview and > > photo capture. Probably hardware-accelerated histograms are needed for > > white balance, auto gain and auto focus, .... > > > > It seems like there's a _lot_ of stuff to be done before we have > > useful support for complex cameras... > > Taking still pictures using a hardware-accelerated preview is > a sophisticated use case. I don't know any userspace application > that does that. Ok, several allow taking snapshots, by simply > storing the image of the current frame. Well, there are applications that take still pictures. Android has one. Maemo has another. Then there's fcam-dev. Its open source; with modified kernel it is fully usable. I have version that runs on recent nearly-mainline on N900. So yes, I'd like solution for problems a) and b). > > (And I'm not sure... when application such as skype is running, is > > there some way to run autogain/autofocus/autowhitebalance? Is that > > something we want to support?) > > Autofocus no. Autogain/Autowhite can be done via libv4l, provided that > it can access the device's controls via /dev/video devnode. Other > applications may be using some other similar algorithms. > > Ok, they don't use histograms provided by the SoC. So, they do it in > software, with is slower. Still, it works fine when the light > conditions don't change too fast. I guess it is going to work well enough with higher CPU usage. Question is if camera without autofocus is usable. I'd say "not really". > > I believe other question is: will not having same control on main > > video device and subdevs be confusing? Does it actually help userspace > > in any way? Yes, we can make controls accessible to old application, > > but does it make them more useful? > > Yes. As I said, libv4l (and some apps) have logic inside to adjust > the image via bright, contrast and white balance controls, using the > video devnode. They don't talk subdev API. So, if those controls > aren't exported, they won't be able to provide a good quality image. Next question is if the libv4l will do the right thing if we just put all controls to one node. For example on N900 you have exposure/gain and brightness. But the brightness is applied at preview phase, so it is "basically useless". You really need to adjust the image using the exposure/gain. > > > > In addition, I suspect end-users of these complex devices don't really care > > > > about a plugin: they want full control and won't typically use generic > > > > applications. If they would need support for that, we'd have seen much more > > > > interest. The main reason for having a plugin is to simplify testing and > > > > if this is going to be used on cheap hobbyist devkits. > > > > > > What are the needs for a cheap hobbyist devkit owner? Do we currently > > > satisfy those needs? I'd say that having a functional driver when > > > compiled without the subdev API, that implements the ioctl's/controls > > > > Having different interface based on config options... is just > > weird. What about poor people (like me) trying to develop complex > > applications? > > Well, that could be done using other mechanisms, like a modprobe > parameter or by switching the behaviour if a subdev interface is > opened. I don't see much trouble on allowing accessing a control via > both interfaces. If we really want to go that way (is not modifying library to access the right files quite easy?), I believe non-confusing option would be to have '/dev/video0 -- omap3 camera for legacy applications' which would include all the controls. > > You can get Nokia N900 on aliexpress. If not, they are still available > > between people :-) > > I have one. Unfortunately, I never had a chance to use it, as the display > stopped working one week after I get it. Well, I guess the easiest option is to just get another one :-). But otoh -- N900 is quite usable without the screen. 0xffff tool can be used to boot the kernel, then you can use nfsroot and usb networking. It also has serial port (over strange connector). Connected over ssh over usb network is actually how I do most of the v4l work. Best regards, Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
[toc] | [prev] | [next] | [standalone]
| From | Mauro Carvalho Chehab <mchehab@s-opensource.com> |
|---|---|
| Date | 2017-03-15 21:30 +0100 |
| Subject | Re: media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) |
| Message-ID | <tlnIu-4nV-35@gated-at.bofh.it> |
| In reply to | #1601608 |
Em Wed, 15 Mar 2017 19:04:21 +0100 Pavel Machek <pavel@ucw.cz> escreveu: > Hi! > > > > Well, I believe first question is: what applications would we want to > > > run on complex devices? Will sending control from video to subdevs > > > actually help? > > > > I would say: camorama, xawtv3, zbar, google talk, skype. If it runs > > with those, it will likely run with any other application. > > I'll take a look when I'm at better internet access. Ok. > > > mplayer is useful for testing... but that one already works (after you > > > setup the pipeline, and configure exposure/gain). > > > > > > But thats useful for testing, not really for production. Image will be > > > out of focus and with wrong white balance. > > > > > > What I would really like is an application to get still photos. For > > > taking pictures with manual settings we need > > > > > > a) units for controls: user wants to focus on 1m, and take picture > > > with ISO200, 1/125 sec. We should also tell him that lens is f/5.6 and > > > focal length is 20mm with 5mm chip. > > > > > > But... autofocus/autogain would really be good to have. Thus we need: > > > > > > b) for each frame, we need exposure settings and focus position at > > > time frame was taken. Otherwise autofocus/autogain will be too > > > slow. At least focus position is going to be tricky -- either kernel > > > would have to compute focus position for us (not trivial) or we'd need > > > enough information to compute it in userspace. > > > > > > There are more problems: hardware-accelerated preview is not trivial > > > to set up (and I'm unsure if it can be done in generic way). Still > > > photos application needs to switch resolutions between preview and > > > photo capture. Probably hardware-accelerated histograms are needed for > > > white balance, auto gain and auto focus, .... > > > > > > It seems like there's a _lot_ of stuff to be done before we have > > > useful support for complex cameras... > > > > Taking still pictures using a hardware-accelerated preview is > > a sophisticated use case. I don't know any userspace application > > that does that. Ok, several allow taking snapshots, by simply > > storing the image of the current frame. > > Well, there are applications that take still pictures. Android has > one. Maemo has another. Then there's fcam-dev. Its open source; with > modified kernel it is fully usable. I have version that runs on recent > nearly-mainline on N900. Hmm... it seems that FCam is specific for N900: http://fcam.garage.maemo.org/ If so, then we have here just the opposite problem, if want it to be used as a generic application, as very likely it requires OMAP3-specific graph/subdevs. > So yes, I'd like solution for problems a) and b). > > > > (And I'm not sure... when application such as skype is running, is > > > there some way to run autogain/autofocus/autowhitebalance? Is that > > > something we want to support?) > > > > Autofocus no. Autogain/Autowhite can be done via libv4l, provided that > > it can access the device's controls via /dev/video devnode. Other > > applications may be using some other similar algorithms. > > > > Ok, they don't use histograms provided by the SoC. So, they do it in > > software, with is slower. Still, it works fine when the light > > conditions don't change too fast. > > I guess it is going to work well enough with higher CPU > usage. Yes. > Question is if camera without autofocus is usable. I'd say "not > really".qv4l2 That actually depends on the sensor and how focus is adjusted. I'm testing right now this camera module for RPi: https://www.raspberrypi.org/products/camera-module-v2/ I might be wrong, but this sensor doesn't seem to have auto-focus. Instead, it seems to use a wide-angle lens. So, except when the object is too close, the focus look OK. > > > I believe other question is: will not having same control on main > > > video device and subdevs be confusing? Does it actually help userspace > > > in any way? Yes, we can make controls accessible to old application, > > > but does it make them more useful? > > > > Yes. As I said, libv4l (and some apps) have logic inside to adjust > > the image via bright, contrast and white balance controls, using the > > video devnode. They don't talk subdev API. So, if those controls > > aren't exported, they won't be able to provide a good quality image. > > Next question is if the libv4l will do the right thing if we just put > all controls to one node. For example on N900 you have exposure/gain > and brightness. But the brightness is applied at preview phase, so it > is "basically useless". You really need to adjust the image using the > exposure/gain. I've no idea, but I suspect it shouldn't be hard to teach libv4l to prefer use an exposure/gain instead of brightness when available. > > > > > In addition, I suspect end-users of these complex devices don't really care > > > > > about a plugin: they want full control and won't typically use generic > > > > > applications. If they would need support for that, we'd have seen much more > > > > > interest. The main reason for having a plugin is to simplify testing and > > > > > if this is going to be used on cheap hobbyist devkits. > > > > > > > > What are the needs for a cheap hobbyist devkit owner? Do we currently > > > > satisfy those needs? I'd say that having a functional driver when > > > > compiled without the subdev API, that implements the ioctl's/controls > > > > > > Having different interface based on config options... is just > > > weird. What about poor people (like me) trying to develop complex > > > applications? > > > > Well, that could be done using other mechanisms, like a modprobe > > parameter or by switching the behaviour if a subdev interface is > > opened. I don't see much trouble on allowing accessing a control via > > both interfaces. > > If we really want to go that way (is not modifying library to access > the right files quite easy?), I believe non-confusing option would be > to have '/dev/video0 -- omap3 camera for legacy applications' which > would include all the controls. Yeah, keeping /dev/video0 reserved for generic applications is something that could work. Not sure how easy would be to implement it. > > > > You can get Nokia N900 on aliexpress. If not, they are still available > > > between people :-) > > > > I have one. Unfortunately, I never had a chance to use it, as the display > > stopped working one week after I get it. > > Well, I guess the easiest option is to just get another one :-). :-) Well, I guess very few units of N900 was sold in Brazil. Importing one is too expensive, due to taxes. > But otoh -- N900 is quite usable without the screen. 0xffff tool can > be used to boot the kernel, then you can use nfsroot and usb > networking. It also has serial port (over strange > connector). Connected over ssh over usb network is actually how I do > most of the v4l work. If you pass me the pointers, I can try it when I have some time. Anyway, I got myself an ISEE IGEPv2, with the expansion board: https://www.isee.biz/products/igep-processor-boards/igepv2-dm3730 https://www.isee.biz/products/igep-expansion-boards/igepv2-expansion The expansion board comes with a tvp5150 analog TV demod. So, with this device, I can simply connect it to a composite input signal. I have some sources here that I can use to test it. Thanks, Mauro
[toc] | [prev] | [next] | [standalone]
| From | Pavel Machek <pavel@ucw.cz> |
|---|---|
| Date | 2017-03-16 23:20 +0100 |
| Subject | Re: media / v4l2-mc: wishlist for complex cameras (was Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline) |
| Message-ID | <tlLUt-4QA-7@gated-at.bofh.it> |
| In reply to | #1601724 |
[Multipart message — attachments visible in raw view] — view raw
Hi! > > > > mplayer is useful for testing... but that one already works (after you > > > > setup the pipeline, and configure exposure/gain). > > > > > > > > But thats useful for testing, not really for production. Image will be > > > > out of focus and with wrong white balance. > > > > > > > > What I would really like is an application to get still photos. For > > > > taking pictures with manual settings we need > > > > > > > > a) units for controls: user wants to focus on 1m, and take picture > > > > with ISO200, 1/125 sec. We should also tell him that lens is f/5.6 and > > > > focal length is 20mm with 5mm chip. > > > > > > > > But... autofocus/autogain would really be good to have. Thus we need: > > > > > > > > b) for each frame, we need exposure settings and focus position at > > > > time frame was taken. Otherwise autofocus/autogain will be too > > > > slow. At least focus position is going to be tricky -- either kernel > > > > would have to compute focus position for us (not trivial) or we'd need > > > > enough information to compute it in userspace. > > > > > > > > There are more problems: hardware-accelerated preview is not trivial > > > > to set up (and I'm unsure if it can be done in generic way). Still > > > > photos application needs to switch resolutions between preview and > > > > photo capture. Probably hardware-accelerated histograms are needed for > > > > white balance, auto gain and auto focus, .... > > > > > > > > It seems like there's a _lot_ of stuff to be done before we have > > > > useful support for complex cameras... > > > > > > Taking still pictures using a hardware-accelerated preview is > > > a sophisticated use case. I don't know any userspace application > > > that does that. Ok, several allow taking snapshots, by simply > > > storing the image of the current frame. > > > > Well, there are applications that take still pictures. Android has > > one. Maemo has another. Then there's fcam-dev. Its open source; with > > modified kernel it is fully usable. I have version that runs on recent > > nearly-mainline on N900. > > Hmm... it seems that FCam is specific for N900: > http://fcam.garage.maemo.org/ > > If so, then we have here just the opposite problem, if want it to be > used as a generic application, as very likely it requires OMAP3-specific > graph/subdevs. Well... there's quick and great version on maemo.org. I do have local version (still somehow N900-specific), but it no longer uses hardware histogram/sharpness support. Should be almost generic. > > So yes, I'd like solution for problems a) and b). ...but it has camera parameters hardcoded (problem a) and slow (problem b). > > Question is if camera without autofocus is usable. I'd say "not > > really".qv4l2 > > That actually depends on the sensor and how focus is adjusted. > > I'm testing right now this camera module for RPi: > https://www.raspberrypi.org/products/camera-module-v2/ > > I might be wrong, but this sensor doesn't seem to have auto-focus. > Instead, it seems to use a wide-angle lens. So, except when the > object is too close, the focus look OK. Well, cameras without autofocus are somehow usable without autofocus. But cameras with autofocus don't work too well without one. > > If we really want to go that way (is not modifying library to access > > the right files quite easy?), I believe non-confusing option would be > > to have '/dev/video0 -- omap3 camera for legacy applications' which > > would include all the controls. > > Yeah, keeping /dev/video0 reserved for generic applications is something > that could work. Not sure how easy would be to implement it. Plus advanced applications would just ignore /dev/video0.. and not be confused. > > > > > > You can get Nokia N900 on aliexpress. If not, they are still available > > > > between people :-) > > > > > > I have one. Unfortunately, I never had a chance to use it, as the display > > > stopped working one week after I get it. > > > > Well, I guess the easiest option is to just get another one :-). > > :-) Well, I guess very few units of N900 was sold in Brazil. Importing > one is too expensive, due to taxes. Try to ask at local mailing list. Those machines were quite common. > > But otoh -- N900 is quite usable without the screen. 0xffff tool can > > be used to boot the kernel, then you can use nfsroot and usb > > networking. It also has serial port (over strange > > connector). Connected over ssh over usb network is actually how I do > > most of the v4l work. > > If you pass me the pointers, I can try it when I have some time. Ok, I guess I'll do that in private email. > Anyway, I got myself an ISEE IGEPv2, with the expansion board: > https://www.isee.biz/products/igep-processor-boards/igepv2-dm3730 > https://www.isee.biz/products/igep-expansion-boards/igepv2-expansion > > The expansion board comes with a tvp5150 analog TV demod. So, with > this device, I can simply connect it to a composite input signal. > I have some sources here that I can use to test it. Well... it looks like TV capture is a "solved" problem. Taking useful photos is what is hard... Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
[toc] | [prev] | [next] | [standalone]
| From | Hans Verkuil <hverkuil@xs4all.nl> |
|---|---|
| Date | 2017-03-20 14:30 +0100 |
| Message-ID | <tn5xN-5ik-51@gated-at.bofh.it> |
| In reply to | #1600195 |
On 03/14/2017 11:21 AM, Mauro Carvalho Chehab wrote: > Em Tue, 14 Mar 2017 08:55:36 +0100 > Hans Verkuil <hverkuil@xs4all.nl> escreveu: > >> On 03/14/2017 04:45 AM, Mauro Carvalho Chehab wrote: >>> Hi Sakari, >>> >>> I started preparing a long argument about it, but gave up in favor of a >>> simpler one. >>> >>> Em Mon, 13 Mar 2017 14:46:22 +0200 >>> Sakari Ailus <sakari.ailus@iki.fi> escreveu: >>> >>>> Drivers are written to support hardware, not particular use case. >>> >>> No, it is just the reverse: drivers and hardware are developed to >>> support use cases. >>> >>> Btw, you should remember that the hardware is the full board, not just the >>> SoC. In practice, the board do limit the use cases: several provide a >>> single physical CSI connector, allowing just one sensor. >>> >>>>> This situation is there since 2009. If I remember well, you tried to write >>>>> such generic plugin in the past, but never finished it, apparently because >>>>> it is too complex. Others tried too over the years. >>>> >>>> I'd argue I know better what happened with that attempt than you do. I had a >>>> prototype of a generic pipeline configuration library but due to various >>>> reasons I haven't been able to continue working on that since around 2012. >>> >>> ... >>> >>>>> The last trial was done by Jacek, trying to cover just the exynos4 driver. >>>>> Yet, even such limited scope plugin was not good enough, as it was never >>>>> merged upstream. Currently, there's no such plugins upstream. >>>>> >>>>> If we can't even merge a plugin that solves it for just *one* driver, >>>>> I have no hope that we'll be able to do it for the generic case. >>>> >>>> I believe Jacek ceased to work on that plugin in his day job; other than >>>> that, there are some matters left to be addressed in his latest patchset. >>> >>> The two above basically summaries the issue: the task of doing a generic >>> plugin on userspace, even for a single driver is complex enough to >>> not cover within a reasonable timeline. >>> >>> From 2009 to 2012, you were working on it, but didn't finish it. >>> >>> Apparently, nobody worked on it between 2013-2014 (but I may be wrong, as >>> I didn't check when the generic plugin interface was added to libv4l). >>> >>> In the case of Jacek's work, the first patch I was able to find was >>> written in Oct, 2014: >>> https://patchwork.kernel.org/patch/5098111/ >>> (not sure what happened with the version 1). >>> >>> The last e-mail about this subject was issued in Dec, 2016. >>> >>> In summary, you had this on your task for 3 years for an OMAP3 >>> plugin (where you have a good expertise), and Jacek for 2 years, >>> for Exynos 4, where he should also have a good knowledge. >>> >>> Yet, with all that efforts, no concrete results were achieved, as none >>> of the plugins got merged. >>> >>> Even if they were merged, if we keep the same mean time to develop a >>> libv4l plugin, that would mean that a plugin for i.MX6 could take 2-3 >>> years to be developed. >>> >>> There's a clear message on it: >>> - we shouldn't keep pushing for a solution via libv4l. >> >> Or: >> >> - userspace plugin development had a very a low priority and >> never got the attention it needed. > > The end result is the same: we can't count on it. > >> >> I know that's *my* reason. I rarely if ever looked at it. I always assumed >> Sakari and/or Laurent would look at it. If this reason is also valid for >> Sakari and Laurent, then it is no wonder nothing has happened in all that >> time. >> >> We're all very driver-development-driven, and userspace gets very little >> attention in general. So before just throwing in the towel we should take >> a good look at the reasons why there has been little or no development: is >> it because of fundamental design defects, or because nobody paid attention >> to it? > > No. We should look it the other way: basically, there are patches > for i.MX6 driver that sends control from videonode to subdevs. > > If we nack apply it, who will write the userspace plugin? When > such change will be merged upstream? > > If we don't have answers to any of the above questions, we should not > nack it. > > That's said, that doesn't prevent merging a libv4l plugin if/when > someone can find time/interest to develop it. I don't think this control inheritance patch will magically prevent you from needed a plugin. > >> I strongly suspect it is the latter. >> >> In addition, I suspect end-users of these complex devices don't really care >> about a plugin: they want full control and won't typically use generic >> applications. If they would need support for that, we'd have seen much more >> interest. The main reason for having a plugin is to simplify testing and >> if this is going to be used on cheap hobbyist devkits. > > What are the needs for a cheap hobbyist devkit owner? Do we currently > satisfy those needs? I'd say that having a functional driver when > compiled without the subdev API, that implements the ioctl's/controls > that a generic application like camorama/google talk/skype/zbar... > would work should be enough to make them happy, even if they need to > add some udev rule and/or run some "prep" application that would setup > the pipelines via MC and eventually rename the device with a working > pipeline to /dev/video0. This is literally the first time we have to cater to a cheap devkit. We were always aware of this issue, but nobody really needed it. > >> >> An additional complication is simply that it is hard to find fully supported >> MC hardware. omap3 boards are hard to find these days, renesas boards are not >> easy to get, freescale isn't the most popular either. Allwinner, mediatek, >> amlogic, broadcom and qualcomm all have closed source implementations or no >> implementation at all. > > I'd say that we should not care anymore on providing a solution for > generic applications to run on boards like OMAP3[1]. For hardware that > are currently available that have Kernel driver and boards developed > to be used as "cheap hobbyist devkit", I'd say we should implement > a Kernel solution that would allow them to be used without subdev > API, e. g. having all ioctls needed by generic applications to work > functional, after some external application sets the pipeline. I liked Russell's idea of having the DT set up an initial video path. This would (probably) make it much easier to provide a generic plugin since there is already an initial valid path when the driver is loaded, and it doesn't require custom code in the driver since this is left to the DT which really knows about the HW. > > [1] Yet, I might eventually do that for fun, an OMAP3 board with tvp5150 > just arrived here last week. It would be nice to have xawtv3 running on it :-) > So, if I have a lot of spare time (with is very unlikely), I might eventually > do something for it to work. > >> I know it took me a very long time before I had a working omap3. > > My first OMAP3 board with working V4L2 source just arrived last week :-) > >> So I am not at all surprised that little progress has been made. > > I'm not surprised, but I'm disappointed, as I tried to push toward a > solution for this problem since when we had our initial meetings about > it. So many things to do, so little time. Sounds corny, but really, that's what this is all about. There were always (and frankly, still are) more important things that needed to be done. Regards, Hans
[toc] | [prev] | [next] | [standalone]
| From | Mauro Carvalho Chehab <mchehab@s-opensource.com> |
|---|---|
| Date | 2017-03-20 16:50 +0100 |
| Message-ID | <tn7Jf-6KZ-13@gated-at.bofh.it> |
| In reply to | #1604568 |
Em Mon, 20 Mar 2017 14:24:25 +0100 Hans Verkuil <hverkuil@xs4all.nl> escreveu: > On 03/14/2017 11:21 AM, Mauro Carvalho Chehab wrote: > > Em Tue, 14 Mar 2017 08:55:36 +0100 > > Hans Verkuil <hverkuil@xs4all.nl> escreveu: > > > >> On 03/14/2017 04:45 AM, Mauro Carvalho Chehab wrote: > >>> Hi Sakari, > >>> > >> We're all very driver-development-driven, and userspace gets very little > >> attention in general. So before just throwing in the towel we should take > >> a good look at the reasons why there has been little or no development: is > >> it because of fundamental design defects, or because nobody paid attention > >> to it? > > > > No. We should look it the other way: basically, there are patches > > for i.MX6 driver that sends control from videonode to subdevs. > > > > If we nack apply it, who will write the userspace plugin? When > > such change will be merged upstream? > > > > If we don't have answers to any of the above questions, we should not > > nack it. > > > > That's said, that doesn't prevent merging a libv4l plugin if/when > > someone can find time/interest to develop it. > > I don't think this control inheritance patch will magically prevent you > from needed a plugin. Yeah, it is not just control inheritance. The driver needs to work without subdev API, e. g. mbus settings should also be done via the video devnode. Btw, Sakari made a good point on IRC: what happens if some app try to change the pipeline or subdev settings while another application is using the device? The driver should block such changes, maybe using the V4L2 priority mechanism. > This is literally the first time we have to cater to a cheap devkit. We > were always aware of this issue, but nobody really needed it. That's true. Now that we have a real need for that, we need to provide a solution. > > I'd say that we should not care anymore on providing a solution for > > generic applications to run on boards like OMAP3[1]. For hardware that > > are currently available that have Kernel driver and boards developed > > to be used as "cheap hobbyist devkit", I'd say we should implement > > a Kernel solution that would allow them to be used without subdev > > API, e. g. having all ioctls needed by generic applications to work > > functional, after some external application sets the pipeline. > > I liked Russell's idea of having the DT set up an initial video path. > This would (probably) make it much easier to provide a generic plugin since > there is already an initial valid path when the driver is loaded, and it > doesn't require custom code in the driver since this is left to the DT > which really knows about the HW. Setting the device via DT indeed makes easier (either for a kernel or userspace solution), but things like resolution changes should be possible without needing to change DT. Also, as MC and subdev changes should be blocked while a V4L2 app is using the device, using a mechanism like calling VIDIOC_S_PRIORITY ioctl via the V4l2 interface, Kernel will require changes, anyway. My suggestion is to touch on one driver to make it work with a generic application. As we currently have efforts and needs for the i.MX6 to do it, I'd say that the best would be to make it work on such driver. Once the work is done, we can see if the approach taken there would work at libv4l or not. In parallel, someone has to fix libv4l for it to be default on applications like gstreamer, e. g. adding support for DMABUF and fixing other issues that are preventing it to be used by default. Nicolas, Why libv4l is currently disabled at gstreamer's default settings? > > [1] Yet, I might eventually do that for fun, an OMAP3 board with tvp5150 > > just arrived here last week. It would be nice to have xawtv3 running on it :-) > > So, if I have a lot of spare time (with is very unlikely), I might eventually > > do something for it to work. > > > >> I know it took me a very long time before I had a working omap3. > > > > My first OMAP3 board with working V4L2 source just arrived last week :-) > > > >> So I am not at all surprised that little progress has been made. > > > > I'm not surprised, but I'm disappointed, as I tried to push toward a > > solution for this problem since when we had our initial meetings about > > it. > > So many things to do, so little time. Sounds corny, but really, that's what > this is all about. There were always (and frankly, still are) more important > things that needed to be done. What's most important for some developer may not be so important for another developer. My understanding here is that there are developers wanting/needing to have standard V4L2 apps support for (some) i.MX6 devices. Those are the ones that may/will allocate some time for it to happen. Thanks, Mauro
[toc] | [prev] | [next] | [standalone]
Page 2 of 5 — ← Prev page 1 [2] 3 4 5 Next page →
Back to top | Article view | linux.kernel
csiph-web