Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1591273 > unrolled thread
| Started by | Sakari Ailus <sakari.ailus@iki.fi> |
|---|---|
| First post | 2017-03-02 17:50 +0100 |
| Last post | 2017-03-11 22:40 +0100 |
| Articles | 12 on this page of 52 — 6 participants |
Back to article view | Back to linux.kernel
This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by
below is the oldest one visible, not the original post.
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Sakari Ailus <sakari.ailus@iki.fi> - 2017-03-02 17:50 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-03 01:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-03 02:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-03 03:30 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Sakari Ailus <sakari.ailus@iki.fi> - 2017-03-03 20:20 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-03 23:50 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-04 00:20 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-04 01:40 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Sakari Ailus <sakari.ailus@iki.fi> - 2017-03-04 14:20 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-10 14:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-10 14:10 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-10 14:30 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-10 15:10 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-10 15:30 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-10 17:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Sakari Ailus <sakari.ailus@iki.fi> - 2017-03-10 23:40 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-11 12:30 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Pavel Machek <pavel@ucw.cz> - 2017-03-11 23:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-12 00:20 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-12 01:30 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Pavel Machek <pavel@ucw.cz> - 2017-03-12 22:30 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-12 23:40 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Sakari Ailus <sakari.ailus@iki.fi> - 2017-03-13 13:50 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-10 16:30 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-10 17:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-10 18:10 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-10 21:50 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Pavel Machek <pavel@ucw.cz> - 2017-03-10 23:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-10 16:10 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-11 12:40 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-11 14:20 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Sakari Ailus <sakari.ailus@iki.fi> - 2017-03-11 16:40 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-11 18:40 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-11 19:10 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-11 19:50 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-11 20:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-11 20:10 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-11 20:10 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-11 21:50 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-12 04:40 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-12 08:40 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-12 19:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-12 23:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-13 11:50 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-13 12:00 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-13 12:10 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-13 12:50 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-13 13:40 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Pavel Machek <pavel@ucw.cz> - 2017-03-12 19:20 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Pavel Machek <pavel@ucw.cz> - 2017-03-11 21:30 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-11 21:40 +0100
Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Pavel Machek <pavel@ucw.cz> - 2017-03-11 22:40 +0100
Page 3 of 3 — ← Prev page 1 2 [3]
| From | Russell King - ARM Linux <linux@armlinux.org.uk> |
|---|---|
| Date | 2017-03-12 08:40 +0100 |
| Message-ID | <tk6gG-6FK-3@gated-at.bofh.it> |
| In reply to | #1598538 |
On Sat, Mar 11, 2017 at 07:31:18PM -0800, Steve Longerbeam wrote: > > > On 03/11/2017 10:59 AM, Russell King - ARM Linux wrote: > >On Sat, Mar 11, 2017 at 10:54:55AM -0800, Steve Longerbeam wrote: > >> > >> > >>On 03/11/2017 10:45 AM, Russell King - ARM Linux wrote: > >>>I really don't think expecting the user to understand and configure > >>>the pipeline is a sane way forward. Think about it - should the > >>>user need to know that, because they have a bayer-only CSI data > >>>source, that there is only one path possible, and if they try to > >>>configure a different path, then things will just error out? > >>> > >>>For the case of imx219 connected to iMX6, it really is as simple as > >>>"there is only one possible path" and all the complexity of the media > >>>interfaces/subdevs is completely unnecessary. Every other block in > >>>the graph is just noise. > >>> > >>>The fact is that these dot graphs show a complex picture, but reality > >>>is somewhat different - there's only relatively few paths available > >>>depending on the connected source and the rest of the paths are > >>>completely useless. > >>> > >> > >>I totally disagree there. Raw bayer requires passthrough yes, but for > >>all other media bus formats on a mipi csi-2 bus, and all other media > >>bus formats on 8-bit parallel buses, the conersion pipelines can be > >>used for scaling, CSC, rotation, and motion-compensated de-interlacing. > > > >... which only makes sense _if_ your source can produce those formats. > >We don't actually disagree on that. > > ...and there are lots of those sources! You should try getting out of > your imx219 shell some time, and have a look around! :) If you think that, you are insulting me. I've been thinking about this from the "big picture" point of view. If you think I'm only thinking about this from only the bayer point of view, you're wrong. Given what Mauro has said, I'm convinced that the media controller stuff is a complete failure for usability, and adding further drivers using it is a mistake. I counter your accusation by saying that you are actually so focused on the media controller way of doing things that you can't see the bigger picture here. So, tell me how the user can possibly use iMX6 video capture without resorting to opening up a terminal and using media-ctl to manually configure the pipeline. How is the user going to control the source device without using media-ctl to find the subdev node, and then using v4l2-ctl on it. How is the user supposed to know which /dev/video* node they should be opening with their capture application? If you can actually respond to the points that I've been raising about end user usability, then we can have a discussion. -- RMK's Patch system: http://www.armlinux.org.uk/developer/patches/ FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up according to speedtest.net.
[toc] | [prev] | [next] | [standalone]
| From | Steve Longerbeam <slongerbeam@gmail.com> |
|---|---|
| Date | 2017-03-12 19:00 +0100 |
| Message-ID | <tkfWF-4O6-3@gated-at.bofh.it> |
| In reply to | #1598557 |
On 03/11/2017 11:37 PM, Russell King - ARM Linux wrote: > On Sat, Mar 11, 2017 at 07:31:18PM -0800, Steve Longerbeam wrote: >> >> >> On 03/11/2017 10:59 AM, Russell King - ARM Linux wrote: >>> On Sat, Mar 11, 2017 at 10:54:55AM -0800, Steve Longerbeam wrote: >>>> >>>> >>>> On 03/11/2017 10:45 AM, Russell King - ARM Linux wrote: >>>>> I really don't think expecting the user to understand and configure >>>>> the pipeline is a sane way forward. Think about it - should the >>>>> user need to know that, because they have a bayer-only CSI data >>>>> source, that there is only one path possible, and if they try to >>>>> configure a different path, then things will just error out? >>>>> >>>>> For the case of imx219 connected to iMX6, it really is as simple as >>>>> "there is only one possible path" and all the complexity of the media >>>>> interfaces/subdevs is completely unnecessary. Every other block in >>>>> the graph is just noise. >>>>> >>>>> The fact is that these dot graphs show a complex picture, but reality >>>>> is somewhat different - there's only relatively few paths available >>>>> depending on the connected source and the rest of the paths are >>>>> completely useless. >>>>> >>>> >>>> I totally disagree there. Raw bayer requires passthrough yes, but for >>>> all other media bus formats on a mipi csi-2 bus, and all other media >>>> bus formats on 8-bit parallel buses, the conersion pipelines can be >>>> used for scaling, CSC, rotation, and motion-compensated de-interlacing. >>> >>> ... which only makes sense _if_ your source can produce those formats. >>> We don't actually disagree on that. >> >> ...and there are lots of those sources! You should try getting out of >> your imx219 shell some time, and have a look around! :) > > If you think that, you are insulting me. I've been thinking about this > from the "big picture" point of view. If you think I'm only thinking > about this from only the bayer point of view, you're wrong. No insult there, you have my utmost respect Russel. Me gives you the Ali-G "respec!" :) It was just a light-hearted attempt at suggesting you might be too entangled with the imx219 (or short on hardware access, which I can certainly understand). > > Given what Mauro has said, I'm convinced that the media controller stuff > is a complete failure for usability, and adding further drivers using it > is a mistake. > I do agree with you that MC places a lot of burden on the user to attain a lot of knowledge of the system's architecture. That's really why I included that control inheritance patch, to ease the burden somewhat. On the other hand, I also think this just requires that MC drivers have very good user documentation. And my other point is, I think most people who have a need to work with the media framework on a particular platform will likely already be quite familiar with that platform. > I counter your accusation by saying that you are actually so focused on > the media controller way of doing things that you can't see the bigger > picture here. > Yeah I've been too mired in the details of this driver. > So, tell me how the user can possibly use iMX6 video capture without > resorting to opening up a terminal and using media-ctl to manually > configure the pipeline. How is the user going to control the source > device without using media-ctl to find the subdev node, and then using > v4l2-ctl on it. How is the user supposed to know which /dev/video* > node they should be opening with their capture application? The media graph for imx6 is fairly self-explanatory in my opinion. Yes that graph has to be generated, but just with a simple 'media-ctl --print-dot', I don't see how that is difficult for the user. The graph makes it quite clear which subdev node belongs to which entity. As for which /dev/videoX node to use, I hope I made it fairly clear in the user doc what functions each node performs. But I will review the doc again and make sure it's been made explicitly clear. > > If you can actually respond to the points that I've been raising about > end user usability, then we can have a discussion. Right, I haven't added my input to the middle-ware discussions (libv4l, v4lconvert, and the auto-pipeline-configuration library work). I can only say at this point that v4lconvert does indeed sound broken w.r.t bayer formats from your description. But it also sounds like an isolated problem and it just needs a patch to allow passing bayer through without software conversion. I wish I had the IMX219 to help you debug these bayer issues. I don't have any bayer sources. In summary, I do like the media framework, it's a good abstraction of hardware pipelines. It does require a lot of system level knowledge to configure, but as I said that is a matter of good documentation. Steve
[toc] | [prev] | [next] | [standalone]
| From | Mauro Carvalho Chehab <mchehab@s-opensource.com> |
|---|---|
| Date | 2017-03-12 23:00 +0100 |
| Message-ID | <tkjGV-7nz-9@gated-at.bofh.it> |
| In reply to | #1598700 |
Em Sun, 12 Mar 2017 10:56:53 -0700 Steve Longerbeam <slongerbeam@gmail.com> escreveu: > On 03/11/2017 11:37 PM, Russell King - ARM Linux wrote: > > On Sat, Mar 11, 2017 at 07:31:18PM -0800, Steve Longerbeam wrote: > > Given what Mauro has said, I'm convinced that the media controller stuff > > is a complete failure for usability, and adding further drivers using it > > is a mistake. I never said that. The thing is that the V4L2 API was designed in 1999, when the video hardware were a way simpler: just one DMA engine on a PCI device, one video/audio input switch and a few video entries. On those days, setting up a pipeline on such devices is simple, and can be done via VIDIOC_*_INPUT ioctls. Nowadays hardware used on SoC devices are a way more complex. SoC devices comes with several DMA engines for buffer transfers, plus video transform blocks whose pipeline can be set dynamically. The MC API is a need to allow setting a complex pipeline, as VIDIOC_*_INPUT cannot work with such complexity. The subdev API solves a different issue. On a "traditional" device, we usually have a pipeline like: <video input> ==> <processing> ==> /dev/video0 Where <processing> controls something at the device (like bright and/or resolution) if you change something at the /dev/video0 node, it is clear that the <processing> block should handle it. On complex devices, with a pipeline like: <camera> ==> <processing0> ==> <CSI bus> ==> <processing1> ==> /dev/video0 If you send a command to adjust the something at /dev/video0, it is not clear for the device driver to do it at processing0 or at processing1. Ok, the driver can decide it, but this can be sub-optimal. Yet, several drivers do that. For example, with em28xx-based drivers several parameters can be adjusted either at the em28xx driver or at the video decoder driver (saa711x). There's a logic inside the driver that decides it. The pipeline there is fixed, though, so it is easy to hardcode a logic for that. So, I've no doubt that both MC and subdev APIs are needed when full hardware control is required. I don't know about how much flexibility the i.MX6 hardware gives, nor if all such flexibility is needed for most use case applications. If I were to code a driver for such hardware, though, I would try to provide a subset of the functionality that would work without the subdev API, allowing it to work with standard V4L applications. That doesn't sound hard to do, as the driver may limit the pipelines to a subset that would make sense, in order to make easier for the driver to take the right decision about to where send a control to setup some parameter. > I do agree with you that MC places a lot of burden on the user to > attain a lot of knowledge of the system's architecture. Setting up the pipeline is not the hard part. One could write a script to do that. > That's really why I included that control inheritance patch, to > ease the burden somewhat. IMHO, that makes sense, as, once some script sets the pipeline, any V4L2 application can work, if you forward the controls to the right I2C devices. > On the other hand, I also think this just requires that MC drivers have > very good user documentation. No, it is not a matter of just documentation. It is a matter of having to rewrite applications for each device, as the information exposed by MC are not enough for an application to do what's needed. For a generic application to work properly with MC, we need to have to add more stuff to MC, in order to allow applications to know more about the features of each subdevice and to things like discovering what kind of signal is present on each PAD. We're calling it as "properties API"[1]. [1] we discussed about that at the ML and at the MC workshop: https://linuxtv.org/news.php?entry=2015-08-17.mchehab Unfortunately, nobody sent any patches implementing it so far :-( > And my other point is, I think most people who have a need to work with > the media framework on a particular platform will likely already be > quite familiar with that platform. I disagree. The most popular platform device currently is Raspberry PI. I doubt that almost all owners of RPi + camera module know anything about MC. They just use Raspberry's official driver with just provides the V4L2 interface. I have a strong opinion that, for hardware like RPi, just the V4L2 API is enough for more than 90% of the cases. > The media graph for imx6 is fairly self-explanatory in my opinion. > Yes that graph has to be generated, but just with a simple 'media-ctl > --print-dot', I don't see how that is difficult for the user. Again, IMHO, the problem is not how to setup the pipeline, but, instead, the need to forward controls to the subdevices. To use a camera, the user needs to set up a set of controls for the image to make sense (bright, contrast, focus, etc). If the driver doesn't forward those controls to the subdevs, an application like "camorama" won't actually work for real, as the user won't be able to adjust those parameters via GUI. Thanks, Mauro
[toc] | [prev] | [next] | [standalone]
| From | Hans Verkuil <hverkuil@xs4all.nl> |
|---|---|
| Date | 2017-03-13 11:50 +0100 |
| Message-ID | <tkvI7-7IV-27@gated-at.bofh.it> |
| In reply to | #1598700 |
On 03/12/2017 06:56 PM, Steve Longerbeam wrote: > > > On 03/11/2017 11:37 PM, Russell King - ARM Linux wrote: >> On Sat, Mar 11, 2017 at 07:31:18PM -0800, Steve Longerbeam wrote: >>> >>> >>> On 03/11/2017 10:59 AM, Russell King - ARM Linux wrote: >>>> On Sat, Mar 11, 2017 at 10:54:55AM -0800, Steve Longerbeam wrote: >>>>> >>>>> >>>>> On 03/11/2017 10:45 AM, Russell King - ARM Linux wrote: >>>>>> I really don't think expecting the user to understand and configure >>>>>> the pipeline is a sane way forward. Think about it - should the >>>>>> user need to know that, because they have a bayer-only CSI data >>>>>> source, that there is only one path possible, and if they try to >>>>>> configure a different path, then things will just error out? >>>>>> >>>>>> For the case of imx219 connected to iMX6, it really is as simple as >>>>>> "there is only one possible path" and all the complexity of the media >>>>>> interfaces/subdevs is completely unnecessary. Every other block in >>>>>> the graph is just noise. >>>>>> >>>>>> The fact is that these dot graphs show a complex picture, but reality >>>>>> is somewhat different - there's only relatively few paths available >>>>>> depending on the connected source and the rest of the paths are >>>>>> completely useless. >>>>>> >>>>> >>>>> I totally disagree there. Raw bayer requires passthrough yes, but for >>>>> all other media bus formats on a mipi csi-2 bus, and all other media >>>>> bus formats on 8-bit parallel buses, the conersion pipelines can be >>>>> used for scaling, CSC, rotation, and motion-compensated de-interlacing. >>>> >>>> ... which only makes sense _if_ your source can produce those formats. >>>> We don't actually disagree on that. >>> >>> ...and there are lots of those sources! You should try getting out of >>> your imx219 shell some time, and have a look around! :) >> >> If you think that, you are insulting me. I've been thinking about this >> from the "big picture" point of view. If you think I'm only thinking >> about this from only the bayer point of view, you're wrong. > > No insult there, you have my utmost respect Russel. Me gives you the > Ali-G "respec!" :) > > It was just a light-hearted attempt at suggesting you might be too > entangled with the imx219 (or short on hardware access, which I can > certainly understand). > > >> >> Given what Mauro has said, I'm convinced that the media controller stuff >> is a complete failure for usability, and adding further drivers using it >> is a mistake. >> > > I do agree with you that MC places a lot of burden on the user to > attain a lot of knowledge of the system's architecture. That's really > why I included that control inheritance patch, to ease the burden > somewhat. > > On the other hand, I also think this just requires that MC drivers have > very good user documentation. > > And my other point is, I think most people who have a need to work with > the media framework on a particular platform will likely already be > quite familiar with that platform. > >> I counter your accusation by saying that you are actually so focused on >> the media controller way of doing things that you can't see the bigger >> picture here. >> > > Yeah I've been too mired in the details of this driver. > > >> So, tell me how the user can possibly use iMX6 video capture without >> resorting to opening up a terminal and using media-ctl to manually >> configure the pipeline. How is the user going to control the source >> device without using media-ctl to find the subdev node, and then using >> v4l2-ctl on it. How is the user supposed to know which /dev/video* >> node they should be opening with their capture application? > > The media graph for imx6 is fairly self-explanatory in my opinion. > Yes that graph has to be generated, but just with a simple 'media-ctl > --print-dot', I don't see how that is difficult for the user. > > The graph makes it quite clear which subdev node belongs to which > entity. > > As for which /dev/videoX node to use, I hope I made it fairly clear > in the user doc what functions each node performs. But I will review > the doc again and make sure it's been made explicitly clear. > > >> >> If you can actually respond to the points that I've been raising about >> end user usability, then we can have a discussion. > > Right, I haven't added my input to the middle-ware discussions (libv4l, > v4lconvert, and the auto-pipeline-configuration library work). I can > only say at this point that v4lconvert does indeed sound broken w.r.t > bayer formats from your description. But it also sounds like an isolated > problem and it just needs a patch to allow passing bayer through without > software conversion. > > I wish I had the IMX219 to help you debug these bayer issues. I don't > have any bayer sources. > > In summary, I do like the media framework, it's a good abstraction of > hardware pipelines. It does require a lot of system level knowledge to > configure, but as I said that is a matter of good documentation. And the reason we went into this direction is that the end-users that use these SoCs with complex pipelines actually *need* this functionality. Which is also part of the reason why work on improved userspace support gets little attention: they don't need to have a plugin that allows generic V4L2 applications to work (at least with simple scenarios). If they would need it, it would have been done (and paid for) before. And improving userspace support for this isn't even at the top of our prio list: getting the request API and stateless codec support in is our highest priority. And that's a big job as well. If you want to blame anyone for this, blame Nokia who set fire to their linux-based phones and thus to the funding for this work. Yes, I am very unhappy with the current state, but given the limited resources I understand why it is as it is. I will try to get time to work on this this summer, but there is no guarantee that that will be granted. If someone else is interested in doing this and can get funding for it, then that would be very welcome. Regards, Hans
[toc] | [prev] | [next] | [standalone]
| From | Russell King - ARM Linux <linux@armlinux.org.uk> |
|---|---|
| Date | 2017-03-13 12:00 +0100 |
| Message-ID | <tkvRL-7Mz-15@gated-at.bofh.it> |
| In reply to | #1599196 |
On Mon, Mar 13, 2017 at 11:44:50AM +0100, Hans Verkuil wrote: > On 03/12/2017 06:56 PM, Steve Longerbeam wrote: > > In summary, I do like the media framework, it's a good abstraction of > > hardware pipelines. It does require a lot of system level knowledge to > > configure, but as I said that is a matter of good documentation. > > And the reason we went into this direction is that the end-users that use > these SoCs with complex pipelines actually *need* this functionality. Which > is also part of the reason why work on improved userspace support gets > little attention: they don't need to have a plugin that allows generic V4L2 > applications to work (at least with simple scenarios). If you stop inheriting controls from the capture sensor to the v4l2 capture device, then this breaks - generic v4l2 applications are not going to be able to show the controls, because they're not visible at the v4l2 capture device anymore. They're only visible through the subdev interfaces, which these generic applications know nothing about. > If you want to blame anyone for this, blame Nokia who set fire to > their linux-based phones and thus to the funding for this work. No, I think that's completely unfair to Nokia. If the MC approach is the way you want to go, you should be thanking Nokia for the amount of effort that they have put in to it, and recognising that it was rather unfortunate that the market had changed, which meant that they weren't able to continue. No one has any right to require any of us to finish what we start coding up in open source, unless there is a contractual obligation in place. That goes for Nokia too. Nokia's decision had ramifications far and wide (resulting in knock on effects in TI and further afield), so don't think for a moment I wasn't affected by what happened in Nokia. Even so, it was a decision for Nokia to make, they had the right to make it, and we have no right to attribute "blame" to Nokia for having made that decision. To even suggest that Nokia should be blamed is absurd. Open source gives rights to everyone. It gives rights to contribute and use, but it also gives rights to walk away without notice (remember the "as is" and "no warranty" clauses?) -- RMK's Patch system: http://www.armlinux.org.uk/developer/patches/ FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up according to speedtest.net.
[toc] | [prev] | [next] | [standalone]
| From | Hans Verkuil <hverkuil@xs4all.nl> |
|---|---|
| Date | 2017-03-13 12:10 +0100 |
| Message-ID | <tkw1s-852-3@gated-at.bofh.it> |
| In reply to | #1599204 |
On 03/13/2017 11:58 AM, Russell King - ARM Linux wrote: > On Mon, Mar 13, 2017 at 11:44:50AM +0100, Hans Verkuil wrote: >> On 03/12/2017 06:56 PM, Steve Longerbeam wrote: >>> In summary, I do like the media framework, it's a good abstraction of >>> hardware pipelines. It does require a lot of system level knowledge to >>> configure, but as I said that is a matter of good documentation. >> >> And the reason we went into this direction is that the end-users that use >> these SoCs with complex pipelines actually *need* this functionality. Which >> is also part of the reason why work on improved userspace support gets >> little attention: they don't need to have a plugin that allows generic V4L2 >> applications to work (at least with simple scenarios). > > If you stop inheriting controls from the capture sensor to the v4l2 > capture device, then this breaks - generic v4l2 applications are not > going to be able to show the controls, because they're not visible at > the v4l2 capture device anymore. They're only visible through the > subdev interfaces, which these generic applications know nothing about. > >> If you want to blame anyone for this, blame Nokia who set fire to >> their linux-based phones and thus to the funding for this work. > > No, I think that's completely unfair to Nokia. If the MC approach is > the way you want to go, you should be thanking Nokia for the amount of > effort that they have put in to it, and recognising that it was rather > unfortunate that the market had changed, which meant that they weren't > able to continue. > > No one has any right to require any of us to finish what we start > coding up in open source, unless there is a contractual obligation in > place. That goes for Nokia too. > > Nokia's decision had ramifications far and wide (resulting in knock on > effects in TI and further afield), so don't think for a moment I wasn't > affected by what happened in Nokia. Even so, it was a decision for > Nokia to make, they had the right to make it, and we have no right to > attribute "blame" to Nokia for having made that decision. > > To even suggest that Nokia should be blamed is absurd. > > Open source gives rights to everyone. It gives rights to contribute > and use, but it also gives rights to walk away without notice (remember > the "as is" and "no warranty" clauses?) Sorry, unfortunate choice of words. While it lasted they did great work. But the reason why MC development stopped for quite some time (esp. the work on userspace software) was because the funding from Nokia dried up. Regards, Hans
[toc] | [prev] | [next] | [standalone]
| From | Mauro Carvalho Chehab <mchehab@s-opensource.com> |
|---|---|
| Date | 2017-03-13 12:50 +0100 |
| Message-ID | <tkwEa-8nq-19@gated-at.bofh.it> |
| In reply to | #1599204 |
Em Mon, 13 Mar 2017 10:58:42 +0000 Russell King - ARM Linux <linux@armlinux.org.uk> escreveu: > On Mon, Mar 13, 2017 at 11:44:50AM +0100, Hans Verkuil wrote: > > On 03/12/2017 06:56 PM, Steve Longerbeam wrote: > > > In summary, I do like the media framework, it's a good abstraction of > > > hardware pipelines. It does require a lot of system level knowledge to > > > configure, but as I said that is a matter of good documentation. > > > > And the reason we went into this direction is that the end-users that use > > these SoCs with complex pipelines actually *need* this functionality. Which > > is also part of the reason why work on improved userspace support gets > > little attention: they don't need to have a plugin that allows generic V4L2 > > applications to work (at least with simple scenarios). > > If you stop inheriting controls from the capture sensor to the v4l2 > capture device, then this breaks - generic v4l2 applications are not > going to be able to show the controls, because they're not visible at > the v4l2 capture device anymore. They're only visible through the > subdev interfaces, which these generic applications know nothing about. True. That's why IMHO, the best is to do control inheritance when there are use cases for generic applications and is possible for the driver to do it (e. g. when the pipeline is not too complex to prevent it to work). As Hans said, for the drivers currently upstreamed at drivers/media, there are currently very little interest on running generic apps there, as they're meant to be used inside embedded hardware using specialized applications. I don't have myself any hardware with i.MX6. Yet, I believe that a low cost board like SolidRun Hummingboard - with comes with a CSI interface compatible with RPi camera modules - will likely attract users who need to run generic applications on their devices. So, I believe that it makes sense for i.MX6 driver to inherit controls from video devnode. Thanks, Mauro
[toc] | [prev] | [next] | [standalone]
| From | Russell King - ARM Linux <linux@armlinux.org.uk> |
|---|---|
| Date | 2017-03-13 13:40 +0100 |
| Message-ID | <tkxqy-uQ-31@gated-at.bofh.it> |
| In reply to | #1599244 |
On Mon, Mar 13, 2017 at 08:42:15AM -0300, Mauro Carvalho Chehab wrote:
> I don't have myself any hardware with i.MX6. Yet, I believe that
> a low cost board like SolidRun Hummingboard - with comes with a
> CSI interface compatible with RPi camera modules - will likely
> attract users who need to run generic applications on their
> devices.
As you've previously mentioned about camorama, I've installed it (I
run Ubuntu 16.04 with "gnome-flashback-metacity" on the HB) and I'm
able to use camorama to view the IMX219 camera sensor.
There's some gotcha's though:
* you need to start it on the command line, manually specifying
which /dev/video device to use, as it always wants to use
/dev/video0. With the CODA mem2mem driver loaded, this may not
be a camera device:
$ v4l2-ctl -d 0 --all
Driver Info (not using libv4l2):
Driver name : coda
Card type : CODA960
Bus info : platform:coda
Driver version: 4.11.0
* camorama seems to use the v4lconvert library, and looking at the
resulting image quality, is rather pixelated - my guess is that
v4lconvert is using a basic algorithm to de-bayer the data. It
also appears to only manage 7fps at best. The gstreamer neon
debayer plugin appears to be faster and higher quality.
* it provides five controls - brightness/contrast/color/hue/white
balance, each of which are not supported by the hardware (IMX219
supports gain and analogue gain only.) These controls appear to
have no effect on the resulting image.
However, using qv4l2 (with the segfault bug in
GeneralTab::updateFrameSize() fixed - m_frameSize, m_frameWidth and
m_frameHeight can be NULL) provides access to all controls. This
can happen if GeneralTab::inputSection() is not called.
The USB uvcvideo camera achieves around 24fps with functional controls
in camorama (mainly because it provides those exact controls to
userspace.)
--
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.
[toc] | [prev] | [next] | [standalone]
| From | Pavel Machek <pavel@ucw.cz> |
|---|---|
| Date | 2017-03-12 19:20 +0100 |
| Message-ID | <tkgg1-5dS-5@gated-at.bofh.it> |
| In reply to | #1598557 |
[Multipart message — attachments visible in raw view] — view raw
On Sun 2017-03-12 07:37:45, Russell King - ARM Linux wrote: > On Sat, Mar 11, 2017 at 07:31:18PM -0800, Steve Longerbeam wrote: > > > > > > On 03/11/2017 10:59 AM, Russell King - ARM Linux wrote: > > >On Sat, Mar 11, 2017 at 10:54:55AM -0800, Steve Longerbeam wrote: > > >> > > >> > > >>On 03/11/2017 10:45 AM, Russell King - ARM Linux wrote: > > >>>I really don't think expecting the user to understand and configure > > >>>the pipeline is a sane way forward. Think about it - should the > > >>>user need to know that, because they have a bayer-only CSI data > > >>>source, that there is only one path possible, and if they try to > > >>>configure a different path, then things will just error out? > > >>> > > >>>For the case of imx219 connected to iMX6, it really is as simple as > > >>>"there is only one possible path" and all the complexity of the media > > >>>interfaces/subdevs is completely unnecessary. Every other block in > > >>>the graph is just noise. > > >>> > > >>>The fact is that these dot graphs show a complex picture, but reality > > >>>is somewhat different - there's only relatively few paths available > > >>>depending on the connected source and the rest of the paths are > > >>>completely useless. > > >>> > > >> > > >>I totally disagree there. Raw bayer requires passthrough yes, but for > > >>all other media bus formats on a mipi csi-2 bus, and all other media > > >>bus formats on 8-bit parallel buses, the conersion pipelines can be > > >>used for scaling, CSC, rotation, and motion-compensated de-interlacing. > > > > > >... which only makes sense _if_ your source can produce those formats. > > >We don't actually disagree on that. > > > > ...and there are lots of those sources! You should try getting out of > > your imx219 shell some time, and have a look around! :) > > If you think that, you are insulting me. I've been thinking about this > from the "big picture" point of view. If you think I'm only thinking > about this from only the bayer point of view, you're wrong. Can you stop that insults nonsense? > Given what Mauro has said, I'm convinced that the media controller stuff > is a complete failure for usability, and adding further drivers using it > is a mistake. Hmm. But you did not present any alternative. Seems some hardware is simply complex. So either we don't add complex drivers (_that_ would be a mistake), or some userspace solution will need to be done. Shell-script running media-ctl does not seem that hard. > So, tell me how the user can possibly use iMX6 video capture without > resorting to opening up a terminal and using media-ctl to manually > configure the pipeline. How is the user going to control the source > device without using media-ctl to find the subdev node, and then using > v4l2-ctl on it. How is the user supposed to know which /dev/video* > node they should be opening with their capture application? Complex hardware sometimes requires userspace configuration. Running a shell script on startup does not seem that hard. And maybe we could do some kind of default setup in kernel, but that does not really solve the problem. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
[toc] | [prev] | [next] | [standalone]
| From | Pavel Machek <pavel@ucw.cz> |
|---|---|
| Date | 2017-03-11 21:30 +0100 |
| Message-ID | <tjVOh-7UT-3@gated-at.bofh.it> |
| In reply to | #1598395 |
Hi! > >>I tend to agree with that. > > > >I agree as well. > > > >This is in line with how existing drivers behave, too. > > > Well, sounds like there is consensus on this topic. I guess I'll > go ahead and remove the control inheritance support. I suppose > having a control appear in two places (subdev and video nodes) can > be confusing. I guess that's way to go. It is impossible to change userland APIs once the patch is merged... Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
[toc] | [prev] | [next] | [standalone]
| From | Steve Longerbeam <slongerbeam@gmail.com> |
|---|---|
| Date | 2017-03-11 21:40 +0100 |
| Message-ID | <tjVXY-7Yc-5@gated-at.bofh.it> |
| In reply to | #1598455 |
On 03/11/2017 12:26 PM, Pavel Machek wrote: > Hi! > >>>> I tend to agree with that. >>> >>> I agree as well. >>> >>> This is in line with how existing drivers behave, too. >> >> >> Well, sounds like there is consensus on this topic. I guess I'll >> go ahead and remove the control inheritance support. I suppose >> having a control appear in two places (subdev and video nodes) can >> be confusing. > > I guess that's way to go. It is impossible to change userland APIs > once the patch is merged... Ok, not including myself, it's now 4 in favor of removing, 1 against... Steve
[toc] | [prev] | [next] | [standalone]
| From | Pavel Machek <pavel@ucw.cz> |
|---|---|
| Date | 2017-03-11 22:40 +0100 |
| Message-ID | <tjWU2-cq-27@gated-at.bofh.it> |
| In reply to | #1598317 |
[Multipart message — attachments visible in raw view] — view raw
Hi! > > > Ok, perhaps supporting both subdev API and V4L2 API at the same > > > time doesn't make much sense. We could disable one in favor of the > > > other, either at compilation time or at runtime. > > > > Right. If the subdev API is disabled, then you have to inherit the subdev > > controls in the bridge driver (how else would you be able to access them?). > > And that's the usual case. > > > > If you do have the subdev API enabled, AND you use the MC, then the > > intention clearly is to give userspace full control and inheriting controls > > no longer makes any sense (and is highly confusing IMHO). > > I tend to agree with that. Well, having different userspace interface according to config options is strange. I believe the right solution is to make complex drivers depend on CONFIG_VIDEO_V4L2_SUBDEV_API... Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
[toc] | [prev] | [standalone]
Page 3 of 3 — ← Prev page 1 2 [3]
Back to top | Article view | linux.kernel
csiph-web