Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1661646 > unrolled thread

Re: [RFC 00/10] V4L2 explicit synchronization support

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [RFC 00/10] V4L2 explicit synchronization support Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2017-06-08 22:20 +0200
    Re: [RFC 00/10] V4L2 explicit synchronization support Shuah Khan <shuahkhan@gmail.com> - 2017-06-08 23:40 +0200
      Re: [RFC 00/10] V4L2 explicit synchronization support Gustavo Padovan <gustavo@padovan.org> - 2017-06-09 08:30 +0200
        Re: [RFC 00/10] V4L2 explicit synchronization support Shuah Khan <shuahkh@osg.samsung.com> - 2017-06-09 18:20 +0200
    Re: [RFC 00/10] V4L2 explicit synchronization support Gustavo Padovan <gustavo@padovan.org> - 2017-06-09 08:20 +0200

#1661646 — Re: [RFC 00/10] V4L2 explicit synchronization support

Hi Gustavo,

Em Wed, 24 May 2017 21:31:01 -0300
Gustavo Padovan <gustavo@padovan.org> escreveu:

> Hi all,
> 
> I've been working on the v2 of this series, but I think I hit a blocker
> when trying to cover the case where the driver asks to requeue the
> buffer. It is related to the out-fence side.
> 
> In the current implementation we return on QBUF an out-fence fd that is not
> tied to any buffer, because we don't know the queueing order until the
> buffer is queued to the driver. Then when the buffer is queued we use
> the BUF_QUEUED event to notify userspace of the index of the buffer,
> so now userspace knows the buffer associated to the out-fence fd
> received earlier.
> 
> Userspace goes ahead and send a DRM Atomic Request to the kernel to
> display that buffer on the screen once the fence signals. If it is 
> a nonblocking request the fence waiting is past the check phase, thus
> it isn't allowed to fail anymore.
> 
> But now, what happens if the V4L2 driver calls buffer_done() asking
> to requeue the buffer. That means the operation failed and can't
> signal the fence, starving the DRM side.
> 
> We need to fix that. The only way I can see is to guarantee ordering of
> buffers when out-fences are used. Ordering is something that HAL3 needs
> to so maybe there is more than one reason to do it like this. I'm not
> a V4L2 expert, so I don't know all the consequences of such a change.
> 
> Any other ideas?
> 
> The current patchset is at:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/padovan/linux.git/log/?h=v4l2-fences

Currently, nothing warrants that buffers will be returned in order,
but that should be the case of most drivers. I guess the main
exception would be mem2mem codec drivers. On those, the driver
or the hardware may opt to reorder the buffers.

If this is a mandatory requirement for explicit fences to work, then
we'll need to be able to explicitly enable it, per driver, and
clearly document that drivers using it *should* warrant that the
dequeued buffer will follow the queued order.

We may need to modify VB2 in order to enforce it or return an
error if the order doesn't match.

-- 
Thanks,
Mauro

[toc] | [next] | [standalone]

#1661714

Hi Gustavo,

On Thu, Jun 8, 2017 at 2:17 PM, Mauro Carvalho Chehab
<mchehab@osg.samsung.com> wrote:
> Hi Gustavo,
>
> Em Wed, 24 May 2017 21:31:01 -0300
> Gustavo Padovan <gustavo@padovan.org> escreveu:
>
>> Hi all,
>>
>> I've been working on the v2 of this series, but I think I hit a blocker
>> when trying to cover the case where the driver asks to requeue the
>> buffer. It is related to the out-fence side.
>>
>> In the current implementation we return on QBUF an out-fence fd that is not
>> tied to any buffer, because we don't know the queueing order until the
>> buffer is queued to the driver. Then when the buffer is queued we use
>> the BUF_QUEUED event to notify userspace of the index of the buffer,
>> so now userspace knows the buffer associated to the out-fence fd
>> received earlier.
>>
>> Userspace goes ahead and send a DRM Atomic Request to the kernel to
>> display that buffer on the screen once the fence signals. If it is
>> a nonblocking request the fence waiting is past the check phase, thus
>> it isn't allowed to fail anymore.
>>
>> But now, what happens if the V4L2 driver calls buffer_done() asking
>> to requeue the buffer. That means the operation failed and can't
>> signal the fence, starving the DRM side.
>>
>> We need to fix that. The only way I can see is to guarantee ordering of
>> buffers when out-fences are used. Ordering is something that HAL3 needs
>> to so maybe there is more than one reason to do it like this. I'm not
>> a V4L2 expert, so I don't know all the consequences of such a change.
>>
>> Any other ideas?
>>
>> The current patchset is at:
>>
>> https://git.kernel.org/pub/scm/linux/kernel/git/padovan/linux.git/log/?h=v4l2-fences

Do you plan to send the v2 out? I did a quick review and have a few comments.

[media] vb2: split out queueing from vb_core_qbuf()

It changes the sequence a bit.

/* Fill buffer information for the userspace */
  if (pb)
  call_void_bufop(q, fill_user_buffer, vb, pb);

With the changes - user information is filled before __enqueue_in_driver(vb);

Anyway, it might be a good idea to send the v2 out for review and we can review
patches in detail. I am hoping to test your patch series on odroid-xu4
next week.
Could you please add me to the thread as well as include me when you send
v2 and subsequent versions.

thanks,
-- Shuah

[toc] | [prev] | [next] | [standalone]

#1661960

2017-06-08 Shuah Khan <shuahkhan@gmail.com>:

> Hi Gustavo,
> 
> On Thu, Jun 8, 2017 at 2:17 PM, Mauro Carvalho Chehab
> <mchehab@osg.samsung.com> wrote:
> > Hi Gustavo,
> >
> > Em Wed, 24 May 2017 21:31:01 -0300
> > Gustavo Padovan <gustavo@padovan.org> escreveu:
> >
> >> Hi all,
> >>
> >> I've been working on the v2 of this series, but I think I hit a blocker
> >> when trying to cover the case where the driver asks to requeue the
> >> buffer. It is related to the out-fence side.
> >>
> >> In the current implementation we return on QBUF an out-fence fd that is not
> >> tied to any buffer, because we don't know the queueing order until the
> >> buffer is queued to the driver. Then when the buffer is queued we use
> >> the BUF_QUEUED event to notify userspace of the index of the buffer,
> >> so now userspace knows the buffer associated to the out-fence fd
> >> received earlier.
> >>
> >> Userspace goes ahead and send a DRM Atomic Request to the kernel to
> >> display that buffer on the screen once the fence signals. If it is
> >> a nonblocking request the fence waiting is past the check phase, thus
> >> it isn't allowed to fail anymore.
> >>
> >> But now, what happens if the V4L2 driver calls buffer_done() asking
> >> to requeue the buffer. That means the operation failed and can't
> >> signal the fence, starving the DRM side.
> >>
> >> We need to fix that. The only way I can see is to guarantee ordering of
> >> buffers when out-fences are used. Ordering is something that HAL3 needs
> >> to so maybe there is more than one reason to do it like this. I'm not
> >> a V4L2 expert, so I don't know all the consequences of such a change.
> >>
> >> Any other ideas?
> >>
> >> The current patchset is at:
> >>
> >> https://git.kernel.org/pub/scm/linux/kernel/git/padovan/linux.git/log/?h=v4l2-fences
> 
> Do you plan to send the v2 out? I did a quick review and have a few comments.
> 
> [media] vb2: split out queueing from vb_core_qbuf()
> 
> It changes the sequence a bit.
> 
> /* Fill buffer information for the userspace */
>   if (pb)
>   call_void_bufop(q, fill_user_buffer, vb, pb);
> 
> With the changes - user information is filled before __enqueue_in_driver(vb);

Without my changes it also fills it before __enqueue_in_driver() when
start_streaming wasn't called yet. So I don't think it really matters.

> 
> Anyway, it might be a good idea to send the v2 out for review and we can review
> patches in detail. I am hoping to test your patch series on odroid-xu4
> next week.
> Could you please add me to the thread as well as include me when you send
> v2 and subsequent versions.

I will send a v2 as soon as I can, but from Thursday next week until
the 25th I'll be on vacation.

Gustavo

[toc] | [prev] | [next] | [standalone]

#1662596

On 06/09/2017 12:25 AM, Gustavo Padovan wrote:
> 2017-06-08 Shuah Khan <shuahkhan@gmail.com>:
> 
>> Hi Gustavo,
>>
>> On Thu, Jun 8, 2017 at 2:17 PM, Mauro Carvalho Chehab
>> <mchehab@osg.samsung.com> wrote:
>>> Hi Gustavo,
>>>
>>> Em Wed, 24 May 2017 21:31:01 -0300
>>> Gustavo Padovan <gustavo@padovan.org> escreveu:
>>>
>>>> Hi all,
>>>>
>>>> I've been working on the v2 of this series, but I think I hit a blocker
>>>> when trying to cover the case where the driver asks to requeue the
>>>> buffer. It is related to the out-fence side.
>>>>
>>>> In the current implementation we return on QBUF an out-fence fd that is not
>>>> tied to any buffer, because we don't know the queueing order until the
>>>> buffer is queued to the driver. Then when the buffer is queued we use
>>>> the BUF_QUEUED event to notify userspace of the index of the buffer,
>>>> so now userspace knows the buffer associated to the out-fence fd
>>>> received earlier.
>>>>
>>>> Userspace goes ahead and send a DRM Atomic Request to the kernel to
>>>> display that buffer on the screen once the fence signals. If it is
>>>> a nonblocking request the fence waiting is past the check phase, thus
>>>> it isn't allowed to fail anymore.
>>>>
>>>> But now, what happens if the V4L2 driver calls buffer_done() asking
>>>> to requeue the buffer. That means the operation failed and can't
>>>> signal the fence, starving the DRM side.
>>>>
>>>> We need to fix that. The only way I can see is to guarantee ordering of
>>>> buffers when out-fences are used. Ordering is something that HAL3 needs
>>>> to so maybe there is more than one reason to do it like this. I'm not
>>>> a V4L2 expert, so I don't know all the consequences of such a change.
>>>>
>>>> Any other ideas?
>>>>
>>>> The current patchset is at:
>>>>
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/padovan/linux.git/log/?h=v4l2-fences
>>
>> Do you plan to send the v2 out? I did a quick review and have a few comments.
>>
>> [media] vb2: split out queueing from vb_core_qbuf()
>>
>> It changes the sequence a bit.
>>
>> /* Fill buffer information for the userspace */
>>   if (pb)
>>   call_void_bufop(q, fill_user_buffer, vb, pb);
>>
>> With the changes - user information is filled before __enqueue_in_driver(vb);
> 
> Without my changes it also fills it before __enqueue_in_driver() when
> start_streaming wasn't called yet. So I don't think it really matters.

Right, with this change, it fills the buffer before __enqueue_in_driver()
when start_streaming is called. Is that an issue, I don't know for sure.
It might not be necessary perhaps if buffer is filled in the path when
stream is already called.

> 
>>
>> Anyway, it might be a good idea to send the v2 out for review and we can review
>> patches in detail. I am hoping to test your patch series on odroid-xu4
>> next week.
>> Could you please add me to the thread as well as include me when you send
>> v2 and subsequent versions.
> 
> I will send a v2 as soon as I can, but from Thursday next week until
> the 25th I'll be on vacation.

okay sounds good.

thanks,
-- Shuah

[toc] | [prev] | [next] | [standalone]

#1661956

Hi Mauro,

2017-06-08 Mauro Carvalho Chehab <mchehab@osg.samsung.com>:

> Hi Gustavo,
> 
> Em Wed, 24 May 2017 21:31:01 -0300
> Gustavo Padovan <gustavo@padovan.org> escreveu:
> 
> > Hi all,
> > 
> > I've been working on the v2 of this series, but I think I hit a blocker
> > when trying to cover the case where the driver asks to requeue the
> > buffer. It is related to the out-fence side.
> > 
> > In the current implementation we return on QBUF an out-fence fd that is not
> > tied to any buffer, because we don't know the queueing order until the
> > buffer is queued to the driver. Then when the buffer is queued we use
> > the BUF_QUEUED event to notify userspace of the index of the buffer,
> > so now userspace knows the buffer associated to the out-fence fd
> > received earlier.
> > 
> > Userspace goes ahead and send a DRM Atomic Request to the kernel to
> > display that buffer on the screen once the fence signals. If it is 
> > a nonblocking request the fence waiting is past the check phase, thus
> > it isn't allowed to fail anymore.
> > 
> > But now, what happens if the V4L2 driver calls buffer_done() asking
> > to requeue the buffer. That means the operation failed and can't
> > signal the fence, starving the DRM side.
> > 
> > We need to fix that. The only way I can see is to guarantee ordering of
> > buffers when out-fences are used. Ordering is something that HAL3 needs
> > to so maybe there is more than one reason to do it like this. I'm not
> > a V4L2 expert, so I don't know all the consequences of such a change.
> > 
> > Any other ideas?
> > 
> > The current patchset is at:
> > 
> > https://git.kernel.org/pub/scm/linux/kernel/git/padovan/linux.git/log/?h=v4l2-fences
> 
> Currently, nothing warrants that buffers will be returned in order,
> but that should be the case of most drivers. I guess the main
> exception would be mem2mem codec drivers. On those, the driver
> or the hardware may opt to reorder the buffers.
> 
> If this is a mandatory requirement for explicit fences to work, then
> we'll need to be able to explicitly enable it, per driver, and
> clearly document that drivers using it *should* warrant that the
> dequeued buffer will follow the queued order.

It is mandatory in the sense it won't work properly and make DRM fail an
atomic commit if a buffer is requeued. So it is fair to ask drivers to
guarantee ordering is a good thing. Then later we can deal with the
drivers that can't. 

> 
> We may need to modify VB2 in order to enforce it or return an
> error if the order doesn't match.

Yeah, I'll look into how to do this.

Gustavo

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web