Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1570567 > unrolled thread

Re: [PATCH v3 00/24] i.MX Media Driver

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH v3 00/24] i.MX Media Driver Philipp Zabel <p.zabel@pengutronix.de> - 2017-01-31 11:20 +0100
    Re: [PATCH v3 00/24] i.MX Media Driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-01-31 14:20 +0100
      Re: [PATCH v3 00/24] i.MX Media Driver Philipp Zabel <p.zabel@pengutronix.de> - 2017-01-31 14:40 +0100
        Re: [PATCH v3 00/24] i.MX Media Driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-01-31 15:10 +0100

#1570567 — Re: [PATCH v3 00/24] i.MX Media Driver

On Mon, 2017-01-30 at 13:06 +0000, Russell King - ARM Linux wrote:
> > The central issue seems to be that I think media pad links / media bus
> > formats should describe physical links, such as parallel or serial
> > buses, and the formats of pixels flowing through them, whereas Steve
> > would like to extend them to describe software transports and in-memory
> > formats.
> 
> This probably isn't the right place to attach this comment in this
> thread, but... the issue of media bus formats matching physical buses
> is an argument that I think is already lost.

It is unfortunate that the parallel format definitions have been reused
for the MIPI logical formats, but I suppose we have to live with that.
Still, I think this is not a reason to open the floodgates and start
describing in-memory formats using MEDIA_BUS_FMT_*

Does at least the combination of logical format and number of lanes
uniquiely describe the physical format?
For the 4-lane LVDS bus formats there are JEIDA/VESA variants where just
the bit ordering is different (MEDIA_BUS_FMT_RGB888_1X7X4_SPWG,
MEDIA_BUS_FMT_RGB888_1X7X4_JEIDA).

> For example, take the 10-bit bayer formats:
> 
> #define MEDIA_BUS_FMT_SBGGR10_1X10              0x3007
> #define MEDIA_BUS_FMT_SGBRG10_1X10              0x300e
> #define MEDIA_BUS_FMT_SGRBG10_1X10              0x300a
> #define MEDIA_BUS_FMT_SRGGB10_1X10              0x300f
> 
> These are commonly used on CSI serial buses (see the smiapp driver for
> example).  From the description at the top of the file, it says the
> 1X10 means that one pixel is transferred as one 10-bit sample.
>
> However, the format on wire is somewhat different - four pixels are
> transmitted over five bytes:
> 
> 	P0	P1	P2	P3	P0	P1	P2	P3
> 	8-bit	8-bit	8-bit	8-bit	2-bit	2-bit	2-bit	2-bit
> 
> This gives two problems:
> 1) it doesn't fit in any sensible kind of "one pixel transferred as
>    N M-bit samples" description because the pixel/sample values
>    (depending how you look at them) are broken up.
> 
> 2) changing this will probably be a user visible change, as things
>    like smiapp are already in use.
> 
> So, I think what we actually have is the media bus formats describing
> the _logical_ bus format.  Yes, one pixel is transferred as one 10-bit
> sample in this case.

Yes. I suppose one way to look at it is that the MIPI CSI-2 specified
formats are representations of corresponding parallel bus formats.

> To help illustrate my point, consider the difference between
> MEDIA_BUS_FMT_RGB565_1X16 and MEDIA_BUS_FMT_RGB565_2X8_BE or
> MEDIA_BUS_FMT_RGB565_2X8_LE.  RGB565_1X16 means 1 pixel over an effective
> 16-bit wide bus (if it's not 16-bit, then it has to be broken up into
> separate "samples".)  RGB565_2X8 means 1 pixel as two 8-bit samples.
> 
> So, the 10-bit bayer is 1 pixel as 1.25 bytes.  Or is it, over a serial
> bus.  Using the RGB565 case, 10-bit bayer over a 4 lane CSI bus becomes
> interesting:
> 
> 	first byte	2nd	3rd
> lane 1	P0 9:2		S0	P7 9:2
> lane 2	P1 9:2		P4 9:2	S1
> lane 3	P2 9:2		P5 9:2	P8 9:2
> lane 4	P3 9:2		P6 9:2	P9 9:2
> 
> S0 = P0/P1/P2/P3 least significant two bits
> S1 = P4/P5/P6/P7 least significant two bits
> 
> or 2 lane CSI:
> 	first byte	2nd	3rd	4th	5th
> lane 1	P0 9:2		P2	S0	P5	P7
> lane 2	P1 9:2		P3	P4	P6	S1
> 
> or 1 lane CSI:
> lane 1	P0 P1 P2 P3 S0 P4 P5 P6 P7 S1 P8 P9 ...

These do look like three different media bus formats to me.

regards
Philipp

[toc] | [next] | [standalone]

#1570729

On Tue, Jan 31, 2017 at 11:09:24AM +0100, Philipp Zabel wrote:
> On Mon, 2017-01-30 at 13:06 +0000, Russell King - ARM Linux wrote:
> > To help illustrate my point, consider the difference between
> > MEDIA_BUS_FMT_RGB565_1X16 and MEDIA_BUS_FMT_RGB565_2X8_BE or
> > MEDIA_BUS_FMT_RGB565_2X8_LE.  RGB565_1X16 means 1 pixel over an effective
> > 16-bit wide bus (if it's not 16-bit, then it has to be broken up into
> > separate "samples".)  RGB565_2X8 means 1 pixel as two 8-bit samples.
> > 
> > So, the 10-bit bayer is 1 pixel as 1.25 bytes.  Or is it, over a serial
> > bus.  Using the RGB565 case, 10-bit bayer over a 4 lane CSI bus becomes
> > interesting:
> > 
> > 	first byte	2nd	3rd
> > lane 1	P0 9:2		S0	P7 9:2
> > lane 2	P1 9:2		P4 9:2	S1
> > lane 3	P2 9:2		P5 9:2	P8 9:2
> > lane 4	P3 9:2		P6 9:2	P9 9:2
> > 
> > S0 = P0/P1/P2/P3 least significant two bits
> > S1 = P4/P5/P6/P7 least significant two bits
> > 
> > or 2 lane CSI:
> > 	first byte	2nd	3rd	4th	5th
> > lane 1	P0 9:2		P2	S0	P5	P7
> > lane 2	P1 9:2		P3	P4	P6	S1
> > 
> > or 1 lane CSI:
> > lane 1	P0 P1 P2 P3 S0 P4 P5 P6 P7 S1 P8 P9 ...
> 
> These do look like three different media bus formats to me.

This isn't limited to the serial side - the parallel bus side between
the CSI2 interface and CSI2IPU wrapper, and the CSI2IPU wrapper and
the CS0/1 interfaces is much the same with 10-bit bayer.

Think of the CSI2 <-> CSI2IPU bit as the 4-lane case, lane 0 ending
up on the least significant 8 bits of the 32-bit bus, lane 3 on the
top 8-bits.

Post CSI2IPU, it talks about a 16-bit bus in the diagrams, so that's
kind of the 2-lane case above...

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

[toc] | [prev] | [next] | [standalone]

#1570753

On Tue, 2017-01-31 at 13:14 +0000, Russell King - ARM Linux wrote:
> On Tue, Jan 31, 2017 at 11:09:24AM +0100, Philipp Zabel wrote:
> > On Mon, 2017-01-30 at 13:06 +0000, Russell King - ARM Linux wrote:
> > > To help illustrate my point, consider the difference between
> > > MEDIA_BUS_FMT_RGB565_1X16 and MEDIA_BUS_FMT_RGB565_2X8_BE or
> > > MEDIA_BUS_FMT_RGB565_2X8_LE.  RGB565_1X16 means 1 pixel over an effective
> > > 16-bit wide bus (if it's not 16-bit, then it has to be broken up into
> > > separate "samples".)  RGB565_2X8 means 1 pixel as two 8-bit samples.
> > > 
> > > So, the 10-bit bayer is 1 pixel as 1.25 bytes.  Or is it, over a serial
> > > bus.  Using the RGB565 case, 10-bit bayer over a 4 lane CSI bus becomes
> > > interesting:
> > > 
> > > 	first byte	2nd	3rd
> > > lane 1	P0 9:2		S0	P7 9:2
> > > lane 2	P1 9:2		P4 9:2	S1
> > > lane 3	P2 9:2		P5 9:2	P8 9:2
> > > lane 4	P3 9:2		P6 9:2	P9 9:2
> > > 
> > > S0 = P0/P1/P2/P3 least significant two bits
> > > S1 = P4/P5/P6/P7 least significant two bits
> > > 
> > > or 2 lane CSI:
> > > 	first byte	2nd	3rd	4th	5th
> > > lane 1	P0 9:2		P2	S0	P5	P7
> > > lane 2	P1 9:2		P3	P4	P6	S1
> > > 
> > > or 1 lane CSI:
> > > lane 1	P0 P1 P2 P3 S0 P4 P5 P6 P7 S1 P8 P9 ...
> > 
> > These do look like three different media bus formats to me.
> 
> This isn't limited to the serial side - the parallel bus side between
> the CSI2 interface and CSI2IPU wrapper, and the CSI2IPU wrapper and
> the CS0/1 interfaces is much the same with 10-bit bayer.
> 
> Think of the CSI2 <-> CSI2IPU bit as the 4-lane case, lane 0 ending
> up on the least significant 8 bits of the 32-bit bus, lane 3 on the
> top 8-bits.
> 
> Post CSI2IPU, it talks about a 16-bit bus in the diagrams, so that's
> kind of the 2-lane case above...

You are right, on the parallel buses the format most definitely is not
MEDIA_BUS_FMT_SBGGR10_1X10. We don't have any representation of the
32-bit bus between CSI2 host and CSI2IPU gasket because we model the two
as a single entity, but the four 16-bit parallel buses between the
CSI2IPU gasket and the IPU1/2 CSI0/1 probably should be set to a custom
format describing this accurately.

regards
Philipp

[toc] | [prev] | [next] | [standalone]

#1570767

On Tue, Jan 31, 2017 at 02:35:00PM +0100, Philipp Zabel wrote:
> On Tue, 2017-01-31 at 13:14 +0000, Russell King - ARM Linux wrote:
> > This isn't limited to the serial side - the parallel bus side between
> > the CSI2 interface and CSI2IPU wrapper, and the CSI2IPU wrapper and
> > the CS0/1 interfaces is much the same with 10-bit bayer.
> > 
> > Think of the CSI2 <-> CSI2IPU bit as the 4-lane case, lane 0 ending
> > up on the least significant 8 bits of the 32-bit bus, lane 3 on the
> > top 8-bits.
> > 
> > Post CSI2IPU, it talks about a 16-bit bus in the diagrams, so that's
> > kind of the 2-lane case above...
> 
> You are right, on the parallel buses the format most definitely is not
> MEDIA_BUS_FMT_SBGGR10_1X10. We don't have any representation of the
> 32-bit bus between CSI2 host and CSI2IPU gasket because we model the two
> as a single entity, but the four 16-bit parallel buses between the
> CSI2IPU gasket and the IPU1/2 CSI0/1 probably should be set to a custom
> format describing this accurately.

Yep.  I should also point out that there's a very odd transformation
going on somewhere, and I don't yet know where.

The sensor is definitely outputting GBRG format, but what seems to get
written into memory is RGGB format.  It's somewhere post CSI, because
when I was using the (broken) CSI compander with 10 bit bayer, the red
compander channel affected the red channel output from the camera, but
changed the green component written to memory... it's very much like
either the first line gets lost somewhere, or the odd/even lines are
transposed.  It could also be a gstreamer bug.  As I say, it's not
something I've looked into deeply enough yet... too many other issues
to chase down!

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web