Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1598396

Re: [PATCH v5 15/39] [media] v4l2: add a frame interval error event

From Steve Longerbeam <slongerbeam@gmail.com>
Newsgroups linux.kernel
Subject Re: [PATCH v5 15/39] [media] v4l2: add a frame interval error event
Date 2017-03-11 19:20 +0100
Message-ID <tjTMt-6AE-3@gated-at.bofh.it> (permalink)
References <tjkOJ-708-5@gated-at.bofh.it> <tjkOL-708-59@gated-at.bofh.it> <tjrwS-3LF-43@gated-at.bofh.it> <tjxCi-7Rw-23@gated-at.bofh.it> <tjNH3-2cd-7@gated-at.bofh.it>
Organization linux.* mail to news gateway

Show all headers | View raw


On 03/11/2017 03:39 AM, Hans Verkuil wrote:
> On 10/03/17 19:37, Steve Longerbeam wrote:
>> Hi Hans,
>>
>> On 03/10/2017 04:03 AM, Hans Verkuil wrote:
>>> On 10/03/17 05:52, Steve Longerbeam wrote:
>>>> Add a new FRAME_INTERVAL_ERROR event to signal that a video capture or
>>>> output device has measured an interval between the reception or transmit
>>>> completion of two consecutive frames of video that is outside the nominal
>>>> frame interval by some tolerance value.
>>>
>>> Reading back what was said on this I agree with Sakari that this doesn't
>>> belong here.
>>>
>>> Userspace can detect this just as easily (if not easier) with a timeout.
>>>
>>
>>
>> Unfortunately measuring frame intervals from userland is not accurate
>> enough for i.MX6.
>>
>> The issue here is that the IPUv3, specifically the CSI unit, can
>> permanently lose vertical sync if there are truncated frames sent
>> on the bt.656 bus. We have seen a single missing line of video cause
>> loss of vertical sync. The only way to correct this is to shutdown
>> the IPU capture hardware and restart, which can be accomplished
>> simply by restarting streaming from userland.
>>
>> There are no other indicators from the sensor about these short
>> frame events (believe me, we've exhausted all avenues with the ADV718x).
>> And the IPUv3 DMA engine has no status indicators for short frames
>> either. So the only way to detect them is by measuring frame intervals.
>>
>> The intervals have to be able to resolve a single line of missing video.
>> With a PAL video source that requires better than 58 usec accuracy.
>>
>> There is too much uncertainty to resolve this at user level. The
>> driver is able to resolve this by measuring intervals between hardware
>> interrupts as long as interrupt latency is reasonably low, and we
>> have another method using the i.MX6 hardware input capture support
>> that can measure these intervals very accurately with no errors
>> introduced by interrupt latency.
>>
>> I made this event a private event to imx-media driver in a previous
>> iteration, so I can return it to a private event, but this can't be
>> done at user level.
>
> It's fine to use an internal event as long as the end-user doesn't
> see it. But if you lose vsyncs, then you never capture another frame,
> right?

No, that's not correct. By loss of vertical sync, I mean the IPU
captures portions of two different frames, resulting in a permanent
"split image", with one frame containing portions of two consecutive
images. Or, the video rolls continuously, if you remember the old CRT
television sets of yore, it's the same rolling effect.


> So userspace can detect that (i.e. no new frames arrive) and
> it can timeout on that. Or you detect it in the driver and restart there,
> or call vb2_queue_error().
>

There is no timeout, the frames keep coming, but they are split images
or rolling.

> Anything really as long as this event isn't user-visible :-)

The event must be user visible, otherwise the user has no indication
the error, and can't correct it by stream restart.

Steve

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH v5 00/39] i.MX Media Driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
  [PATCH v5 26/39] media: imx: Add VDIC subdev driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
  [PATCH v5 38/39] media: imx: csi: fix crop rectangle reset in sink set_fmt Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
  [PATCH v5 30/39] media: imx: add support for bayer formats Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
  [PATCH v5 35/39] media: imx: csi/fim: add support for frame intervals Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
  [PATCH v5 20/39] platform: add video-multiplexer subdevice driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
  [PATCH v5 29/39] ARM: imx_v6_v7_defconfig: Enable staging video4linux drivers Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
  [PATCH v5 09/39] ARM: dts: imx6-sabresd: add OV5642 and OV5640 camera sensors Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
  [PATCH v5 10/39] ARM: dts: imx6-sabreauto: create i2cmux for i2c3 Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
  [PATCH v5 08/39] ARM: dts: imx6-sabrelite: add OV5642 and OV5640 camera sensors Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
  [PATCH v5 01/39] [media] dt-bindings: Add bindings for video-multiplexer device Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
  [PATCH v5 24/39] media: imx: Add Capture Device Interface Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
  [PATCH v5 32/39] media: imx: csi: fix crop rectangle changes in set_fmt Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
  [PATCH v5 33/39] media: imx: mipi-csi2: enable setting and getting of frame rates Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
  [PATCH v5 25/39] media: imx: Add CSI subdev driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
  [PATCH v5 05/39] ARM: dts: imx6qdl: Add mipi_ipu1/2 multiplexers, mipi_csi, and their connections Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
  [PATCH v5 11/39] ARM: dts: imx6-sabreauto: add reset-gpios property for max7310_b Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
  [PATCH v5 21/39] UAPI: Add media UAPI Kbuild file Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
    Re: [PATCH v5 21/39] UAPI: Add media UAPI Kbuild file Sakari Ailus <sakari.ailus@iki.fi> - 2017-03-11 14:50 +0100
      Re: [PATCH v5 21/39] UAPI: Add media UAPI Kbuild file Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-11 19:30 +0100
        Re: [PATCH v5 21/39] UAPI: Add media UAPI Kbuild file Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-13 11:00 +0100
  [PATCH v5 37/39] media: imx: csi: add frame skipping support Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
  [PATCH v5 22/39] media: Add userspace header file for i.MX Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
    Re: [PATCH v5 22/39] media: Add userspace header file for i.MX Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-10 12:50 +0100
    Re: [PATCH v5 22/39] media: Add userspace header file for i.MX Pavel Machek <pavel@ucw.cz> - 2017-03-11 00:40 +0100
  [PATCH v5 36/39] media: imx: redo pixel format enumeration and negotiation Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
  [PATCH v5 15/39] [media] v4l2: add a frame interval error event Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
    Re: [PATCH v5 15/39] [media] v4l2: add a frame interval error event Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-10 13:10 +0100
      Re: [PATCH v5 15/39] [media] v4l2: add a frame interval error event Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 19:40 +0100
        Re: [PATCH v5 15/39] [media] v4l2: add a frame interval error event Pavel Machek <pavel@ucw.cz> - 2017-03-11 00:40 +0100
          Re: [PATCH v5 15/39] [media] v4l2: add a frame interval error event Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-11 00:50 +0100
        Re: [PATCH v5 15/39] [media] v4l2: add a frame interval error event Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-11 12:50 +0100
          Re: [PATCH v5 15/39] [media] v4l2: add a frame interval error event Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-11 19:20 +0100
            Re: [PATCH v5 15/39] [media] v4l2: add a frame interval error event Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-11 20:00 +0100
              Re: [PATCH v5 15/39] [media] v4l2: add a frame interval error event Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-11 20:00 +0100
              Re: [PATCH v5 15/39] [media] v4l2: add a frame interval error event Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-11 20:10 +0100
            Re: [PATCH v5 15/39] [media] v4l2: add a frame interval error event Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-13 11:10 +0100
              Re: [PATCH v5 15/39] [media] v4l2: add a frame interval error event Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-13 11:50 +0100
                Re: [PATCH v5 15/39] [media] v4l2: add a frame interval error event Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-13 12:00 +0100
                Re: [PATCH v5 15/39] [media] v4l2: add a frame interval error event Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-13 18:10 +0100
                Re: [PATCH v5 15/39] [media] v4l2: add a frame interval error event Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-13 18:20 +0100
                Re: [PATCH v5 15/39] [media] v4l2: add a frame interval error event Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-13 22:50 +0100
  [PATCH v5 31/39] media: imx: csi: add support for bayer formats Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
  [PATCH v5 03/39] [media] dt/bindings: Add bindings for OV5640 Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:00 +0100
  [PATCH v5 14/39] add mux and video interface bridge entity functions Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:10 +0100
  [PATCH v5 07/39] ARM: dts: imx6qdl-sabrelite: remove erratum ERR006687 workaround Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:10 +0100
    Re: [PATCH v5 07/39] ARM: dts: imx6qdl-sabrelite: remove erratum  ERR006687 workaround Troy Kisky <troy.kisky@boundarydevices.com> - 2017-03-10 20:00 +0100
      Re: [PATCH v5 07/39] ARM: dts: imx6qdl-sabrelite: remove erratum  ERR006687 workaround Fabio Estevam <festevam@gmail.com> - 2017-03-10 20:20 +0100
        Re: [PATCH v5 07/39] ARM: dts: imx6qdl-sabrelite: remove erratum  ERR006687 workaround Pavel Machek <pavel@ucw.cz> - 2017-03-10 23:00 +0100
          Re: [PATCH v5 07/39] ARM: dts: imx6qdl-sabrelite: remove erratum  ERR006687 workaround Fabio Estevam <festevam@gmail.com> - 2017-03-10 23:10 +0100
  [PATCH v5 17/39] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:10 +0100
    Re: [PATCH v5 17/39] [media] v4l2-mc: add a function to inherit  controls from a pipeline Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-10 12:50 +0100
  [PATCH v5 12/39] ARM: dts: imx6-sabreauto: add pinctrl for gpt input capture Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:10 +0100
  [PATCH v5 18/39] [media] v4l: subdev: Add function to validate frame interval Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:10 +0100
    Re: [PATCH v5 18/39] [media] v4l: subdev: Add function to validate  frame interval Sakari Ailus <sakari.ailus@iki.fi> - 2017-03-11 14:50 +0100
      Re: [PATCH v5 18/39] [media] v4l: subdev: Add function to validate  frame interval Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-11 21:40 +0100
  [PATCH v5 04/39] ARM: dts: imx6qdl: Add compatible, clocks, irqs to MIPI CSI-2 node Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:10 +0100
  [PATCH v5 16/39] [media] v4l2: add a new-frame before end-of-frame event Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:10 +0100
    Re: [PATCH v5 16/39] [media] v4l2: add a new-frame before  end-of-frame event Hans Verkuil <hverkuil@xs4all.nl> - 2017-03-10 13:10 +0100
  [PATCH v5 06/39] ARM: dts: imx6qdl: add capture-subsystem device Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:10 +0100
  [PATCH v5 02/39] [media] dt-bindings: Add bindings for i.MX media driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-10 06:10 +0100
  Re: [PATCH v5 00/39] i.MX Media Driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-10 21:20 +0100
    Re: [PATCH v5 00/39] i.MX Media Driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-11 00:30 +0100
      Re: [PATCH v5 00/39] i.MX Media Driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-12 18:50 +0100
    Re: [PATCH v5 00/39] i.MX Media Driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-12 01:40 +0100
      Re: [PATCH v5 00/39] i.MX Media Driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-12 21:00 +0100
        Re: [PATCH v5 00/39] i.MX Media Driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-12 21:10 +0100
          Re: [PATCH v5 00/39] i.MX Media Driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-12 21:30 +0100
            Re: [PATCH v5 00/39] i.MX Media Driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-13 05:40 +0100
              Re: [PATCH v5 00/39] i.MX Media Driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-13 09:20 +0100
                Re: [PATCH v5 00/39] i.MX Media Driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-13 10:40 +0100
                Re: [PATCH v5 00/39] i.MX Media Driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-14 00:40 +0100
                Re: [PATCH v5 00/39] i.MX Media Driver Steve Longerbeam <steve_longerbeam@mentor.com> - 2017-03-14 00:40 +0100
  Re: [PATCH v5 00/39] i.MX Media Driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-12 19:00 +0100
    Re: [PATCH v5 00/39] i.MX Media Driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-12 20:30 +0100
      Re: [PATCH v5 00/39] i.MX Media Driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-12 20:40 +0100
        Re: [PATCH v5 00/39] i.MX Media Driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-12 20:50 +0100
          Re: [PATCH v5 00/39] i.MX Media Driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-12 21:20 +0100
            Re: [PATCH v5 00/39] i.MX Media Driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-12 21:40 +0100
              Re: [PATCH v5 00/39] i.MX Media Driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-12 21:40 +0100
              Re: [PATCH v5 00/39] i.MX Media Driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-12 21:50 +0100
                Re: [PATCH v5 00/39] i.MX Media Driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-12 22:20 +0100
  Re: [PATCH v5 00/39] i.MX Media Driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-12 20:50 +0100
    Re: [PATCH v5 00/39] i.MX Media Driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-12 21:10 +0100
    Re: [PATCH v5 00/39] i.MX Media Driver Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-12 22:00 +0100
      Re: [PATCH v5 00/39] i.MX Media Driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-03-12 22:20 +0100
        Re: [PATCH v5 00/39] i.MX Media Driver Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-03-12 23:20 +0100

csiph-web