Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > linux.kernel > #1582252 > unrolled thread

[PATCH v4 00/36] i.MX Media Driver

Started bySteve Longerbeam <slongerbeam@gmail.com>
First post2017-02-16 03:30 +0100
Last post2017-03-01 01:50 +0100
Articles 20 on this page of 98 — 8 participants

Back to article view | Back to linux.kernel


Contents

  [PATCH v4 00/36] i.MX Media Driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:30 +0100
    [PATCH v4 05/36] ARM: dts: imx6qdl-sabrelite: remove erratum ERR006687 workaround Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:30 +0100
    [PATCH v4 17/36] media: Add userspace header file for i.MX Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:30 +0100
      Re: [PATCH v4 17/36] media: Add userspace header file for i.MX Philipp Zabel <p.zabel@pengutronix.de> - 2017-02-16 12:40 +0100
        Re: [PATCH v4 17/36] media: Add userspace header file for i.MX Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-23 01:00 +0100
    [PATCH v4 23/36] media: imx: Add MIPI CSI-2 Receiver subdev driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:30 +0100
      Re: [PATCH v4 23/36] media: imx: Add MIPI CSI-2 Receiver subdev  driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-02-16 11:30 +0100
        Re: [PATCH v4 23/36] media: imx: Add MIPI CSI-2 Receiver subdev  driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 19:10 +0100
      Re: [PATCH v4 23/36] media: imx: Add MIPI CSI-2 Receiver subdev  driver Philipp Zabel <p.zabel@pengutronix.de> - 2017-02-17 12:00 +0100
        Re: [PATCH v4 23/36] media: imx: Add MIPI CSI-2 Receiver subdev  driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-02-17 12:20 +0100
          Re: [PATCH v4 23/36] media: imx: Add MIPI CSI-2 Receiver subdev  driver Philipp Zabel <p.zabel@pengutronix.de> - 2017-02-17 12:40 +0100
            Re: [PATCH v4 23/36] media: imx: Add MIPI CSI-2 Receiver subdev  driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-23 00:50 +0100
              Re: [PATCH v4 23/36] media: imx: Add MIPI CSI-2 Receiver subdev  driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-23 00:50 +0100
          Re: [PATCH v4 23/36] media: imx: Add MIPI CSI-2 Receiver subdev  driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-23 01:10 +0100
            Re: [PATCH v4 23/36] media: imx: Add MIPI CSI-2 Receiver subdev  driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-23 01:20 +0100
        Re: [PATCH v4 23/36] media: imx: Add MIPI CSI-2 Receiver subdev  driver Philipp Zabel <p.zabel@pengutronix.de> - 2017-02-17 15:20 +0100
          Re: [PATCH v4 23/36] media: imx: Add MIPI CSI-2 Receiver subdev  driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-17 19:30 +0100
    [PATCH v4 28/36] media: imx: csi: fix crop rectangle changes in set_fmt Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:30 +0100
      Re: [PATCH v4 28/36] media: imx: csi: fix crop rectangle changes in  set_fmt Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-02-16 12:10 +0100
        Re: [PATCH v4 28/36] media: imx: csi: fix crop rectangle changes in  set_fmt Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 19:20 +0100
    [PATCH v4 35/36] media: imx: csi: fix crop rectangle reset in sink set_fmt Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:30 +0100
    [PATCH v4 27/36] media: imx: csi: add support for bayer formats Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:30 +0100
    [PATCH v4 33/36] media: imx: redo pixel format enumeration and negotiation Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:30 +0100
      Re: [PATCH v4 33/36] media: imx: redo pixel format enumeration and  negotiation Philipp Zabel <p.zabel@pengutronix.de> - 2017-02-16 12:40 +0100
        Re: [PATCH v4 33/36] media: imx: redo pixel format enumeration and  negotiation Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-23 01:00 +0100
          Re: [PATCH v4 33/36] media: imx: redo pixel format enumeration and  negotiation Philipp Zabel <p.zabel@pengutronix.de> - 2017-02-23 10:20 +0100
            Re: [PATCH v4 33/36] media: imx: redo pixel format enumeration and  negotiation Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-24 02:50 +0100
    [PATCH v4 26/36] media: imx: add support for bayer formats Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:30 +0100
    [PATCH v4 07/36] ARM: dts: imx6-sabresd: add OV5642 and OV5640 camera sensors Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:30 +0100
      Re: [PATCH v4 07/36] ARM: dts: imx6-sabresd: add OV5642 and OV5640  camera sensors Fabio Estevam <festevam@gmail.com> - 2017-02-17 02:00 +0100
        Re: [PATCH v4 07/36] ARM: dts: imx6-sabresd: add OV5642 and OV5640  camera sensors Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-17 02:00 +0100
    [PATCH v4 10/36] ARM: dts: imx6-sabreauto: add pinctrl for gpt input capture Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:30 +0100
    [PATCH v4 06/36] ARM: dts: imx6-sabrelite: add OV5642 and OV5640 camera sensors Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:30 +0100
    [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and getting of frame rates Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:30 +0100
      Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and  getting of frame rates Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-18 02:20 +0100
        Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and  getting of frame rates Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-02-18 10:30 +0100
          Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and  getting of frame rates Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-18 18:30 +0100
            Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and  getting of frame rates Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-02-18 19:20 +0100
      Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and  getting of frame rates Steve Longerbeam <steve_longerbeam@mentor.com> - 2017-02-18 02:20 +0100
      Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and  getting of frame rates Sakari Ailus <sakari.ailus@iki.fi> - 2017-02-20 23:10 +0100
        Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and  getting of frame rates Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-21 00:00 +0100
          Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and  getting of frame rates Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-21 00:50 +0100
          Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and  getting of frame rates Sakari Ailus <sakari.ailus@iki.fi> - 2017-02-21 13:20 +0100
            Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and  getting of frame rates Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-21 23:30 +0100
              Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and  getting of frame rates Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-22 00:40 +0100
        Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and  getting of frame rates Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-21 01:20 +0100
          Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and  getting of frame rates Philipp Zabel <p.zabel@pengutronix.de> - 2017-02-21 10:00 +0100
        Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and  getting of frame rates Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-02-21 01:20 +0100
          Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and  getting of frame rates Sakari Ailus <sakari.ailus@iki.fi> - 2017-02-21 13:40 +0100
            Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and  getting of frame rates Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-02-21 14:30 +0100
              Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and  getting of frame rates Sakari Ailus <sakari.ailus@iki.fi> - 2017-02-21 16:40 +0100
                Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and  getting of frame rates Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-02-21 17:10 +0100
                  Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and  getting of frame rates Sakari Ailus <sakari.ailus@iki.fi> - 2017-02-21 17:20 +0100
    [PATCH v4 13/36] [media] v4l2: add a frame timeout event Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:30 +0100
    [PATCH v4 31/36] media: imx: csi: add __csi_get_fmt Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:30 +0100
    [PATCH v4 12/36] add mux and video interface bridge entity functions Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:30 +0100
      Re: [PATCH v4 12/36] add mux and video interface bridge entity  functions Pavel Machek <pavel@ucw.cz> - 2017-02-19 22:40 +0100
        Re: [PATCH v4 12/36] add mux and video interface bridge entity  functions Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-22 18:30 +0100
    [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit controls from a pipeline Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:30 +0100
      Re: [PATCH v4 14/36] [media] v4l2-mc: add a function to inherit  controls from a pipeline Pavel Machek <pavel@ucw.cz> - 2017-02-19 23:00 +0100
    [PATCH v4 15/36] platform: add video-multiplexer subdevice driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:30 +0100
      Re: [PATCH v4 15/36] platform: add video-multiplexer subdevice driver Pavel Machek <pavel@ucw.cz> - 2017-02-19 23:20 +0100
        Re: [PATCH v4 15/36] platform: add video-multiplexer subdevice  driver Philipp Zabel <p.zabel@pengutronix.de> - 2017-02-21 10:20 +0100
          Re: [PATCH v4 15/36] platform: add video-multiplexer subdevice driver Pavel Machek <pavel@ucw.cz> - 2017-02-24 21:10 +0100
      Re: [PATCH v4 15/36] platform: add video-multiplexer subdevice driver Rob Herring <robh@kernel.org> - 2017-02-27 15:50 +0100
        Re: [PATCH v4 15/36] platform: add video-multiplexer subdevice driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-01 01:30 +0100
    [PATCH v4 21/36] media: imx: Add VDIC subdev driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:30 +0100
    [PATCH v4 32/36] media: imx: csi/fim: add support for frame intervals Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:30 +0100
      Re: [PATCH v4 32/36] media: imx: csi/fim: add support for frame  intervals Steve Longerbeam <steve_longerbeam@mentor.com> - 2017-02-16 03:40 +0100
    [PATCH v4 09/36] ARM: dts: imx6-sabreauto: add reset-gpios property for max7310_b Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:30 +0100
    [PATCH v4 01/36] [media] dt-bindings: Add bindings for i.MX media driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:40 +0100
      Re: [PATCH v4 01/36] [media] dt-bindings: Add bindings for i.MX  media driver Philipp Zabel <p.zabel@pengutronix.de> - 2017-02-16 13:00 +0100
        Re: [PATCH v4 01/36] [media] dt-bindings: Add bindings for i.MX media  driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 20:30 +0100
      Re: [PATCH v4 01/36] [media] dt-bindings: Add bindings for i.MX  media driver Rob Herring <robh@kernel.org> - 2017-02-27 15:50 +0100
        Re: [PATCH v4 01/36] [media] dt-bindings: Add bindings for i.MX media  driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-01 01:40 +0100
    [PATCH v4 04/36] ARM: dts: imx6qdl: add capture-subsystem device Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 03:40 +0100
    Re: [PATCH v4 18/36] media: Add i.MX media core driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-02-16 11:30 +0100
      Re: [PATCH v4 18/36] media: Add i.MX media core driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 19:00 +0100
    Re: [PATCH v4 00/36] i.MX Media Driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-02-16 12:40 +0100
      Re: [PATCH v4 00/36] i.MX Media Driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 19:40 +0100
    Re: [PATCH v4 18/36] media: Add i.MX media core driver Philipp Zabel <p.zabel@pengutronix.de> - 2017-02-16 14:10 +0100
      Re: [PATCH v4 18/36] media: Add i.MX media core driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-02-16 14:50 +0100
      Re: [PATCH v4 18/36] media: Add i.MX media core driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-17 02:40 +0100
        Re: [PATCH v4 18/36] media: Add i.MX media core driver Philipp Zabel <p.zabel@pengutronix.de> - 2017-02-17 09:40 +0100
    Re: [PATCH v4 00/36] i.MX Media Driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-16 23:30 +0100
      Re: [PATCH v4 00/36] i.MX Media Driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-02-17 00:10 +0100
        Re: [PATCH v4 00/36] i.MX Media Driver Philipp Zabel <p.zabel@pengutronix.de> - 2017-02-17 11:50 +0100
          Re: [PATCH v4 00/36] i.MX Media Driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-02-17 12:00 +0100
            Re: [PATCH v4 00/36] i.MX Media Driver Philipp Zabel <p.zabel@pengutronix.de> - 2017-02-17 12:30 +0100
        Re: [PATCH v4 00/36] i.MX Media Driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-02-18 18:40 +0100
      Re: [PATCH v4 00/36] i.MX Media Driver Philipp Zabel <p.zabel@pengutronix.de> - 2017-02-17 12:50 +0100
        Re: [PATCH v4 00/36] i.MX Media Driver Sakari Ailus <sakari.ailus@iki.fi> - 2017-02-17 13:30 +0100
          Re: [PATCH v4 00/36] i.MX Media Driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-02-17 13:40 +0100
          Re: [PATCH v4 00/36] i.MX Media Driver Philipp Zabel <p.zabel@pengutronix.de> - 2017-02-17 16:10 +0100
            Re: [PATCH v4 00/36] i.MX Media Driver Sakari Ailus <sakari.ailus@iki.fi> - 2017-02-18 13:10 +0100
    Re: [PATCH v4 00/36] i.MX Media Driver Russell King - ARM Linux <linux@armlinux.org.uk> - 2017-02-16 23:30 +0100
    Re: [PATCH v4 24/36] [media] add Omnivision OV5640 sensor driver Rob Herring <robh@kernel.org> - 2017-02-27 15:50 +0100
      Re: [PATCH v4 24/36] [media] add Omnivision OV5640 sensor driver Steve Longerbeam <slongerbeam@gmail.com> - 2017-03-01 01:50 +0100

Page 2 of 5 — ← Prev page 1 [2] 3 4 5  Next page →


#1582257 — [PATCH v4 35/36] media: imx: csi: fix crop rectangle reset in sink set_fmt

FromSteve Longerbeam <slongerbeam@gmail.com>
Date2017-02-16 03:30 +0100
Subject[PATCH v4 35/36] media: imx: csi: fix crop rectangle reset in sink set_fmt
Message-ID<tbjZw-8cO-35@gated-at.bofh.it>
In reply to#1582252
From: Philipp Zabel <p.zabel@pengutronix.de>

The csi_try_crop call in set_fmt should compare the cropping rectangle
to the currently set input format, not to the previous input format.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
---
 drivers/staging/media/imx/imx-media-csi.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
index 6284f99..3e6b607 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -937,15 +937,13 @@ __csi_get_fmt(struct csi_priv *priv, struct v4l2_subdev_pad_config *cfg,
 static int csi_try_crop(struct csi_priv *priv,
 			struct v4l2_rect *crop,
 			struct v4l2_subdev_pad_config *cfg,
-			enum v4l2_subdev_format_whence which,
+			struct v4l2_mbus_framefmt *infmt,
 			struct imx_media_subdev *sensor)
 {
 	struct v4l2_of_endpoint *sensor_ep;
-	struct v4l2_mbus_framefmt *infmt;
 	v4l2_std_id std;
 	int ret;
 
-	infmt = __csi_get_fmt(priv, cfg, CSI_SINK_PAD, which);
 	sensor_ep = &sensor->sensor_ep;
 
 	crop->width = min_t(__u32, infmt->width, crop->width);
@@ -1142,8 +1140,7 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
 		crop.top = 0;
 		crop.width = sdformat->format.width;
 		crop.height = sdformat->format.height;
-		ret = csi_try_crop(priv, &crop, cfg,
-				   sdformat->which, sensor);
+		ret = csi_try_crop(priv, &crop, cfg, &sdformat->format, sensor);
 		if (ret)
 			return ret;
 
@@ -1225,6 +1222,7 @@ static int csi_set_selection(struct v4l2_subdev *sd,
 			     struct v4l2_subdev_selection *sel)
 {
 	struct csi_priv *priv = v4l2_get_subdevdata(sd);
+	struct v4l2_mbus_framefmt *infmt;
 	struct imx_media_subdev *sensor;
 	int ret;
 
@@ -1254,7 +1252,8 @@ static int csi_set_selection(struct v4l2_subdev *sd,
 		return 0;
 	}
 
-	ret = csi_try_crop(priv, &sel->r, cfg, sel->which, sensor);
+	infmt = __csi_get_fmt(priv, cfg, CSI_SINK_PAD, sel->which);
+	ret = csi_try_crop(priv, &sel->r, cfg, infmt, sensor);
 	if (ret)
 		return ret;
 
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]


#1582258 — [PATCH v4 27/36] media: imx: csi: add support for bayer formats

FromSteve Longerbeam <slongerbeam@gmail.com>
Date2017-02-16 03:30 +0100
Subject[PATCH v4 27/36] media: imx: csi: add support for bayer formats
Message-ID<tbjZw-8cO-41@gated-at.bofh.it>
In reply to#1582252
From: Russell King <rmk+kernel@armlinux.org.uk>

Bayer formats must be treated as generic data and passthrough mode must
be used.  Add the correct setup for these formats.

Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>

- added check to csi_link_validate() to verify that destination is
  IDMAC output pad when passthrough conditions exist: bayer formats
  and 16-bit parallel buses.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
---
 drivers/staging/media/imx/imx-media-csi.c | 50 ++++++++++++++++++++++++++-----
 1 file changed, 42 insertions(+), 8 deletions(-)

diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
index 0343fc3..ae24b42 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -2,6 +2,7 @@
  * V4L2 Capture CSI Subdev for Freescale i.MX5/6 SOC
  *
  * Copyright (c) 2014-2016 Mentor Graphics Inc.
+ * Copyright (C) 2017 Pengutronix, Philipp Zabel <kernel@pengutronix.de>
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -271,10 +272,11 @@ static int csi_idmac_setup_channel(struct csi_priv *priv)
 	struct imx_media_video_dev *vdev = priv->vdev;
 	struct v4l2_of_endpoint *sensor_ep;
 	struct v4l2_mbus_framefmt *infmt;
-	unsigned int burst_size;
 	struct ipu_image image;
+	u32 passthrough_bits;
 	dma_addr_t phys[2];
 	bool passthrough;
+	u32 burst_size;
 	int ret;
 
 	infmt = &priv->format_mbus[CSI_SINK_PAD];
@@ -301,15 +303,38 @@ static int csi_idmac_setup_channel(struct csi_priv *priv)
 	ipu_cpmem_set_burstsize(priv->idmac_ch, burst_size);
 
 	/*
-	 * If the sensor uses 16-bit parallel CSI bus, we must handle
-	 * the data internally in the IPU as 16-bit generic, aka
-	 * passthrough mode.
+	 * Check for conditions that require the IPU to handle the
+	 * data internally as generic data, aka passthrough mode:
+	 * - raw bayer formats
+	 * - the sensor bus is 16-bit parallel
 	 */
-	passthrough = (sensor_ep->bus_type != V4L2_MBUS_CSI2 &&
-		       sensor_ep->bus.parallel.bus_width >= 16);
+	switch (image.pix.pixelformat) {
+	case V4L2_PIX_FMT_SBGGR8:
+	case V4L2_PIX_FMT_SGBRG8:
+	case V4L2_PIX_FMT_SGRBG8:
+	case V4L2_PIX_FMT_SRGGB8:
+		burst_size = 8;
+		passthrough = true;
+		passthrough_bits = 8;
+		break;
+	case V4L2_PIX_FMT_SBGGR16:
+	case V4L2_PIX_FMT_SGBRG16:
+	case V4L2_PIX_FMT_SGRBG16:
+	case V4L2_PIX_FMT_SRGGB16:
+		burst_size = 4;
+		passthrough = true;
+		passthrough_bits = 16;
+		break;
+	default:
+		passthrough = (sensor_ep->bus_type != V4L2_MBUS_CSI2 &&
+			       sensor_ep->bus.parallel.bus_width >= 16);
+		passthrough_bits = 16;
+		break;
+	}
 
 	if (passthrough)
-		ipu_cpmem_set_format_passthrough(priv->idmac_ch, 16);
+		ipu_cpmem_set_format_passthrough(priv->idmac_ch,
+						 passthrough_bits);
 
 	/*
 	 * Set the channel for the direct CSI-->memory via SMFC
@@ -695,6 +720,7 @@ static int csi_link_validate(struct v4l2_subdev *sd,
 			     struct v4l2_subdev_format *sink_fmt)
 {
 	struct csi_priv *priv = v4l2_get_subdevdata(sd);
+	const struct imx_media_pixfmt *incc;
 	struct v4l2_of_endpoint *sensor_ep;
 	bool is_csi2;
 	int ret;
@@ -713,8 +739,16 @@ static int csi_link_validate(struct v4l2_subdev *sd,
 	}
 
 	sensor_ep = &priv->sensor->sensor_ep;
-
 	is_csi2 = (sensor_ep->bus_type == V4L2_MBUS_CSI2);
+	incc = priv->cc[CSI_SINK_PAD];
+
+	if (priv->dest != IPU_CSI_DEST_IDMAC &&
+	    (incc->bayer || (!is_csi2 &&
+			     sensor_ep->bus.parallel.bus_width >= 16))) {
+		v4l2_err(&priv->sd,
+			 "bayer/16-bit parallel buses must go to IDMAC pad\n");
+		return -EINVAL;
+	}
 
 	if (is_csi2) {
 		int vc_num = 0;
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]


#1582259 — [PATCH v4 33/36] media: imx: redo pixel format enumeration and negotiation

FromSteve Longerbeam <slongerbeam@gmail.com>
Date2017-02-16 03:30 +0100
Subject[PATCH v4 33/36] media: imx: redo pixel format enumeration and negotiation
Message-ID<tbjZw-8cO-37@gated-at.bofh.it>
In reply to#1582252
The previous API and negotiation of mbus codes and pixel formats
was broken, and has been completely redone.

The negotiation of media bus codes should be as follows:

CSI:

sink pad     direct src pad      IDMAC src pad
--------     ----------------    -------------
RGB (any)        IPU RGB           RGB (any)
YUV (any)        IPU YUV           YUV (any)
Bayer              N/A             must be same bayer code as sink

VDIC:

direct sink pad    IDMAC sink pad    direct src pad
---------------    --------------    --------------
IPU YUV only       YUV (any)         IPU YUV only

PRP:

direct sink pad    direct src pads
---------------    ---------------
IPU (any)          same as sink code

PRP ENC/VF:

direct sink pad    IDMAC src pads
---------------    --------------
IPU (any)          any RGB or YUV

Given the above, a new internal API is created:

enum codespace_sel {
       CS_SEL_YUV = 0, /* find or enumerate only YUV codes */
       CS_SEL_RGB,     /* find or enumerate only RGB codes */
       CS_SEL_ANY,     /* find or enumerate both YUV and RGB codes */
};

/* Find and enumerate fourcc pixel formats */
const struct imx_media_pixfmt *
imx_media_find_format(u32 fourcc, enum codespace_sel cs_sel);
int imx_media_enum_format(u32 *fourcc, u32 index, enum codespace_sel cs_sel);

/* Find and enumerate media bus codes */
const struct imx_media_pixfmt *
imx_media_find_mbus_format(u32 code, enum codespace_sel cs_sel,
                          bool allow_bayer);
int imx_media_enum_mbus_format(u32 *code, u32 index, enum codespace_sel cs_sel,
                              bool allow_bayer);

/* Find and enumerate IPU internal media bus codes */
const struct imx_media_pixfmt *
imx_media_find_ipu_format(u32 code, enum codespace_sel cs_sel);
int imx_media_enum_ipu_format(u32 *code, u32 index, enum codespace_sel cs_sel);

The tables have been split into separate tables for YUV and RGB formats
to support the implementation of the above.

The subdev's .enum_mbus_code() and .set_fmt() operations have
been rewritten using the above APIs.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
---
 drivers/staging/media/imx/imx-ic-prp.c        |  72 ++++--
 drivers/staging/media/imx/imx-ic-prpencvf.c   |  53 ++--
 drivers/staging/media/imx/imx-media-capture.c |  77 +++---
 drivers/staging/media/imx/imx-media-csi.c     | 107 +++++---
 drivers/staging/media/imx/imx-media-utils.c   | 357 +++++++++++++++++++-------
 drivers/staging/media/imx/imx-media-vdic.c    | 100 +++++---
 drivers/staging/media/imx/imx-media.h         |  27 +-
 7 files changed, 528 insertions(+), 265 deletions(-)

diff --git a/drivers/staging/media/imx/imx-ic-prp.c b/drivers/staging/media/imx/imx-ic-prp.c
index 3683f7c..b9ee8fb 100644
--- a/drivers/staging/media/imx/imx-ic-prp.c
+++ b/drivers/staging/media/imx/imx-ic-prp.c
@@ -96,16 +96,6 @@ static void prp_stop(struct prp_priv *priv)
 {
 }
 
-static int prp_enum_mbus_code(struct v4l2_subdev *sd,
-			      struct v4l2_subdev_pad_config *cfg,
-			      struct v4l2_subdev_mbus_code_enum *code)
-{
-	if (code->pad >= PRP_NUM_PADS)
-		return -EINVAL;
-
-	return imx_media_enum_ipu_format(NULL, &code->code, code->index, true);
-}
-
 static struct v4l2_mbus_framefmt *
 __prp_get_fmt(struct prp_priv *priv, struct v4l2_subdev_pad_config *cfg,
 	      unsigned int pad, enum v4l2_subdev_format_whence which)
@@ -118,6 +108,33 @@ __prp_get_fmt(struct prp_priv *priv, struct v4l2_subdev_pad_config *cfg,
 		return &priv->format_mbus[pad];
 }
 
+static int prp_enum_mbus_code(struct v4l2_subdev *sd,
+			      struct v4l2_subdev_pad_config *cfg,
+			      struct v4l2_subdev_mbus_code_enum *code)
+{
+	struct prp_priv *priv = sd_to_priv(sd);
+	struct v4l2_mbus_framefmt *infmt;
+	int ret = 0;
+
+	switch (code->pad) {
+	case PRP_SINK_PAD:
+		ret = imx_media_enum_ipu_format(&code->code, code->index,
+						CS_SEL_ANY);
+		break;
+	case PRP_SRC_PAD_PRPENC:
+	case PRP_SRC_PAD_PRPVF:
+		if (code->index != 0)
+			return -EINVAL;
+		infmt = __prp_get_fmt(priv, cfg, PRP_SINK_PAD, code->which);
+		code->code = infmt->code;
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	return ret;
+}
+
 static int prp_get_fmt(struct v4l2_subdev *sd,
 		       struct v4l2_subdev_pad_config *cfg,
 		       struct v4l2_subdev_format *sdformat)
@@ -152,23 +169,28 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
 	if (priv->stream_on)
 		return -EBUSY;
 
-	cc = imx_media_find_ipu_format(0, sdformat->format.code, true);
-	if (!cc) {
-		imx_media_enum_ipu_format(NULL, &code, 0, true);
-		cc = imx_media_find_ipu_format(0, code, true);
-		sdformat->format.code = cc->codes[0];
-	}
-
-	v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W,
-			      W_ALIGN, &sdformat->format.height,
-			      MIN_H, MAX_H, H_ALIGN, S_ALIGN);
-
-	/* Output pads mirror input pad */
-	if (sdformat->pad == PRP_SRC_PAD_PRPENC ||
-	    sdformat->pad == PRP_SRC_PAD_PRPVF) {
+	switch (sdformat->pad) {
+	case PRP_SINK_PAD:
+		v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W,
+				      W_ALIGN, &sdformat->format.height,
+				      MIN_H, MAX_H, H_ALIGN, S_ALIGN);
+
+		cc = imx_media_find_ipu_format(sdformat->format.code,
+					       CS_SEL_ANY);
+		if (!cc) {
+			imx_media_enum_ipu_format(&code, 0, CS_SEL_ANY);
+			cc = imx_media_find_ipu_format(code, CS_SEL_ANY);
+			sdformat->format.code = cc->codes[0];
+		}
+		break;
+	case PRP_SRC_PAD_PRPENC:
+	case PRP_SRC_PAD_PRPVF:
+		/* Output pads mirror input pad */
 		infmt = __prp_get_fmt(priv, cfg, PRP_SINK_PAD,
 				      sdformat->which);
+		cc = imx_media_find_ipu_format(infmt->code, CS_SEL_ANY);
 		sdformat->format = *infmt;
+		break;
 	}
 
 	if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY) {
@@ -364,7 +386,7 @@ static int prp_registered(struct v4l2_subdev *sd)
 			MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
 
 		/* set a default mbus format  */
-		imx_media_enum_ipu_format(NULL, &code, 0, true);
+		imx_media_enum_ipu_format(&code, 0, CS_SEL_YUV);
 		ret = imx_media_init_mbus_fmt(&priv->format_mbus[i],
 					      640, 480, code, V4L2_FIELD_NONE,
 					      &priv->cc[i]);
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c b/drivers/staging/media/imx/imx-ic-prpencvf.c
index 6e45975..dd9d499 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -689,20 +689,6 @@ static void prp_stop(struct prp_priv *priv)
 	prp_put_ipu_resources(priv);
 }
 
-static int prp_enum_mbus_code(struct v4l2_subdev *sd,
-			      struct v4l2_subdev_pad_config *cfg,
-			      struct v4l2_subdev_mbus_code_enum *code)
-{
-	if (code->pad >= PRPENCVF_NUM_PADS)
-		return -EINVAL;
-
-	if (code->pad == PRPENCVF_SRC_PAD)
-		return imx_media_enum_format(NULL, &code->code, code->index,
-					     true, false);
-
-	return imx_media_enum_ipu_format(NULL, &code->code, code->index, true);
-}
-
 static struct v4l2_mbus_framefmt *
 __prp_get_fmt(struct prp_priv *priv, struct v4l2_subdev_pad_config *cfg,
 	      unsigned int pad, enum v4l2_subdev_format_whence which)
@@ -715,6 +701,25 @@ __prp_get_fmt(struct prp_priv *priv, struct v4l2_subdev_pad_config *cfg,
 		return &priv->format_mbus[pad];
 }
 
+static int prp_enum_mbus_code(struct v4l2_subdev *sd,
+			      struct v4l2_subdev_pad_config *cfg,
+			      struct v4l2_subdev_mbus_code_enum *code)
+{
+	int ret;
+
+	if (code->pad >= PRPENCVF_NUM_PADS)
+		return -EINVAL;
+
+	if (code->pad == PRPENCVF_SINK_PAD)
+		ret = imx_media_enum_ipu_format(&code->code, code->index,
+						CS_SEL_ANY);
+	else
+		ret = imx_media_enum_mbus_format(&code->code, code->index,
+						 CS_SEL_ANY, false);
+
+	return ret;
+}
+
 static int prp_get_fmt(struct v4l2_subdev *sd,
 		       struct v4l2_subdev_pad_config *cfg,
 		       struct v4l2_subdev_format *sdformat)
@@ -754,11 +759,13 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
 		infmt = __prp_get_fmt(priv, cfg, PRPENCVF_SINK_PAD,
 				      sdformat->which);
 
-		cc = imx_media_find_format(0, sdformat->format.code,
-					   true, false);
+		cc = imx_media_find_mbus_format(sdformat->format.code,
+						CS_SEL_ANY, false);
 		if (!cc) {
-			imx_media_enum_format(NULL, &code, 0, true, false);
-			cc = imx_media_find_format(0, code, true, false);
+			imx_media_enum_mbus_format(&code, 0,
+						   CS_SEL_ANY, false);
+			cc = imx_media_find_mbus_format(code,
+							CS_SEL_ANY, false);
 			sdformat->format.code = cc->codes[0];
 		}
 
@@ -781,11 +788,11 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
 					      infmt->height / 4, MAX_H_SRC,
 					      H_ALIGN_SRC, S_ALIGN);
 	} else {
-		cc = imx_media_find_ipu_format(0, sdformat->format.code,
-					       true);
+		cc = imx_media_find_ipu_format(sdformat->format.code,
+					       CS_SEL_ANY);
 		if (!cc) {
-			imx_media_enum_ipu_format(NULL, &code, 0, true);
-			cc = imx_media_find_ipu_format(0, code, true);
+			imx_media_enum_ipu_format(&code, 0, CS_SEL_ANY);
+			cc = imx_media_find_ipu_format(code, CS_SEL_ANY);
 			sdformat->format.code = cc->codes[0];
 		}
 
@@ -1021,7 +1028,7 @@ static int prp_registered(struct v4l2_subdev *sd)
 	for (i = 0; i < PRPENCVF_NUM_PADS; i++) {
 		if (i == PRPENCVF_SINK_PAD) {
 			priv->pad[i].flags = MEDIA_PAD_FL_SINK;
-			imx_media_enum_ipu_format(NULL, &code, 0, true);
+			imx_media_enum_ipu_format(&code, 0, CS_SEL_YUV);
 		} else {
 			priv->pad[i].flags = MEDIA_PAD_FL_SOURCE;
 			code = 0;
diff --git a/drivers/staging/media/imx/imx-media-capture.c b/drivers/staging/media/imx/imx-media-capture.c
index fbf6067..9ef1cc2 100644
--- a/drivers/staging/media/imx/imx-media-capture.c
+++ b/drivers/staging/media/imx/imx-media-capture.c
@@ -85,12 +85,33 @@ static int vidioc_querycap(struct file *file, void *fh,
 static int capture_enum_fmt_vid_cap(struct file *file, void *fh,
 				    struct v4l2_fmtdesc *f)
 {
-	u32 fourcc;
+	struct capture_priv *priv = video_drvdata(file);
+	const struct imx_media_pixfmt *cc_src;
+	struct v4l2_subdev_format fmt_src;
+	u32 fourcc, cs_sel;
 	int ret;
 
-	ret = imx_media_enum_format(&fourcc, NULL, f->index, true, true);
-	if (ret)
+	fmt_src.pad = priv->src_sd_pad;
+	fmt_src.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	ret = v4l2_subdev_call(priv->src_sd, pad, get_fmt, NULL, &fmt_src);
+	if (ret) {
+		v4l2_err(priv->src_sd, "failed to get src_sd format\n");
 		return ret;
+	}
+	cc_src = imx_media_find_mbus_format(fmt_src.format.code,
+					    CS_SEL_ANY, true);
+	if (cc_src->bayer) {
+		if (f->index != 0)
+			return -EINVAL;
+		fourcc = cc_src->fourcc;
+	} else {
+		cs_sel = (cc_src->cs == IPUV3_COLORSPACE_YUV) ?
+			CS_SEL_YUV : CS_SEL_RGB;
+
+		ret = imx_media_enum_format(&fourcc, f->index, cs_sel);
+		if (ret)
+			return ret;
+	}
 
 	f->pixelformat = fourcc;
 
@@ -112,40 +133,36 @@ static int capture_try_fmt_vid_cap(struct file *file, void *fh,
 {
 	struct capture_priv *priv = video_drvdata(file);
 	struct v4l2_subdev_format fmt_src;
-	const struct imx_media_pixfmt *cc, *src_cc;
-	u32 fourcc;
+	const struct imx_media_pixfmt *cc, *cc_src;
 	int ret;
 
-	fourcc = f->fmt.pix.pixelformat;
-	cc = imx_media_find_format(fourcc, 0, true, true);
-	if (!cc) {
-		imx_media_enum_format(&fourcc, NULL, 0, true, true);
-		cc = imx_media_find_format(fourcc, 0, true, true);
-	}
-
-	/*
-	 * user frame dimensions are the same as src_sd's pad.
-	 */
 	fmt_src.pad = priv->src_sd_pad;
 	fmt_src.which = V4L2_SUBDEV_FORMAT_ACTIVE;
 	ret = v4l2_subdev_call(priv->src_sd, pad, get_fmt, NULL, &fmt_src);
 	if (ret)
 		return ret;
 
-	/*
-	 * but we can allow planar pixel formats if the src_sd's
-	 * pad configured a YUV format
-	 */
-	src_cc = imx_media_find_format(0, fmt_src.format.code, true, false);
-	if (src_cc->cs == IPUV3_COLORSPACE_YUV &&
-	    cc->cs == IPUV3_COLORSPACE_YUV) {
-		imx_media_mbus_fmt_to_pix_fmt(&f->fmt.pix,
-					      &fmt_src.format, cc);
+	cc_src = imx_media_find_mbus_format(fmt_src.format.code,
+					    CS_SEL_ANY, true);
+
+	if (cc_src->bayer) {
+		cc = cc_src;
 	} else {
-		imx_media_mbus_fmt_to_pix_fmt(&f->fmt.pix,
-					      &fmt_src.format, src_cc);
+		u32 fourcc, cs_sel;
+
+		cs_sel = (cc_src->cs == IPUV3_COLORSPACE_YUV) ?
+			CS_SEL_YUV : CS_SEL_RGB;
+		fourcc = f->fmt.pix.pixelformat;
+
+		cc = imx_media_find_format(fourcc, cs_sel);
+		if (!cc) {
+			imx_media_enum_format(&fourcc, 0, cs_sel);
+			cc = imx_media_find_format(fourcc, cs_sel);
+		}
 	}
 
+	imx_media_mbus_fmt_to_pix_fmt(&f->fmt.pix, &fmt_src.format, cc);
+
 	return 0;
 }
 
@@ -165,8 +182,8 @@ static int capture_s_fmt_vid_cap(struct file *file, void *fh,
 		return ret;
 
 	priv->vdev.fmt.fmt.pix = f->fmt.pix;
-	priv->vdev.cc = imx_media_find_format(f->fmt.pix.pixelformat, 0,
-					      true, true);
+	priv->vdev.cc = imx_media_find_format(f->fmt.pix.pixelformat,
+					      CS_SEL_ANY);
 
 	return 0;
 }
@@ -573,8 +590,8 @@ int imx_media_capture_device_register(struct imx_media_video_dev *vdev)
 	vdev->fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 	imx_media_mbus_fmt_to_pix_fmt(&vdev->fmt.fmt.pix,
 				      &fmt_src.format, NULL);
-	vdev->cc = imx_media_find_format(0, fmt_src.format.code,
-					 true, false);
+	vdev->cc = imx_media_find_format(vdev->fmt.fmt.pix.pixelformat,
+					 CS_SEL_ANY);
 
 	v4l2_info(sd, "Registered %s as /dev/%s\n", vfd->name,
 		  video_device_node_name(vfd));
diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
index 040cca6..1d4e746 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -877,15 +877,44 @@ static int csi_enum_mbus_code(struct v4l2_subdev *sd,
 			      struct v4l2_subdev_pad_config *cfg,
 			      struct v4l2_subdev_mbus_code_enum *code)
 {
-	if (code->pad >= CSI_NUM_PADS)
-		return -EINVAL;
+	struct csi_priv *priv = v4l2_get_subdevdata(sd);
+	const struct imx_media_pixfmt *incc;
+	struct v4l2_mbus_framefmt *infmt;
+	int ret = 0;
+	u32 cs_sel;
 
-	if (code->pad == CSI_SRC_PAD_DIRECT)
-		return imx_media_enum_ipu_format(NULL, &code->code,
-						 code->index, true);
+	infmt = __csi_get_fmt(priv, cfg, CSI_SINK_PAD, code->which);
+	incc = imx_media_find_mbus_format(infmt->code, CS_SEL_ANY, true);
 
-	return imx_media_enum_format(NULL, &code->code, code->index,
-				     true, false);
+	switch (code->pad) {
+	case CSI_SINK_PAD:
+		ret = imx_media_enum_mbus_format(&code->code, code->index,
+						 CS_SEL_ANY, true);
+		break;
+	case CSI_SRC_PAD_DIRECT:
+		cs_sel = (incc->cs == IPUV3_COLORSPACE_YUV) ?
+			CS_SEL_YUV : CS_SEL_RGB;
+		ret = imx_media_enum_ipu_format(&code->code, code->index,
+						cs_sel);
+		break;
+	case CSI_SRC_PAD_IDMAC:
+		if (incc->bayer) {
+			if (code->index != 0)
+				return -EINVAL;
+			code->code = infmt->code;
+		} else {
+			cs_sel = (incc->cs == IPUV3_COLORSPACE_YUV) ?
+				CS_SEL_YUV : CS_SEL_RGB;
+			ret = imx_media_enum_mbus_format(&code->code,
+							 code->index,
+							 cs_sel, false);
+		}
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	return ret;
 }
 
 static int csi_get_fmt(struct v4l2_subdev *sd,
@@ -917,7 +946,7 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
 	struct v4l2_mbus_framefmt *infmt;
 	struct imx_media_subdev *sensor;
 	struct v4l2_rect crop;
-	u32 code;
+	u32 code, cs_sel;
 	int ret;
 
 	if (sdformat->pad >= CSI_NUM_PADS)
@@ -932,14 +961,16 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
 		return PTR_ERR(sensor);
 	}
 
-	v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W,
-			      W_ALIGN, &sdformat->format.height,
-			      MIN_H, MAX_H, H_ALIGN, S_ALIGN);
-
 	switch (sdformat->pad) {
 	case CSI_SRC_PAD_DIRECT:
 	case CSI_SRC_PAD_IDMAC:
-		infmt = __csi_get_fmt(priv, cfg, CSI_SINK_PAD, sdformat->which);
+		infmt = __csi_get_fmt(priv, cfg, CSI_SINK_PAD,
+				      sdformat->which);
+		incc = imx_media_find_mbus_format(infmt->code,
+						  CS_SEL_ANY, true);
+
+		cs_sel = (incc->cs == IPUV3_COLORSPACE_YUV) ?
+			CS_SEL_YUV : CS_SEL_RGB;
 
 		if (sdformat->format.width < priv->crop.width * 3 / 4)
 			sdformat->format.width = priv->crop.width / 2;
@@ -952,32 +983,29 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
 			sdformat->format.height = priv->crop.height;
 
 		if (sdformat->pad == CSI_SRC_PAD_IDMAC) {
-			cc = imx_media_find_format(0, sdformat->format.code,
-						   true, false);
-			if (!cc) {
-				imx_media_enum_format(NULL, &code, 0,
-						      true, false);
-				cc = imx_media_find_format(0, code,
-							   true, false);
-				sdformat->format.code = cc->codes[0];
-			}
-
-			incc = priv->cc[CSI_SINK_PAD];
-			if (cc->cs != incc->cs) {
+			if (incc->bayer) {
 				sdformat->format.code = infmt->code;
-				cc = imx_media_find_format(
-					0, sdformat->format.code,
-					true, false);
+				cc = incc;
+			} else {
+				cc = imx_media_find_mbus_format(
+					sdformat->format.code, cs_sel, false);
+				if (!cc) {
+					imx_media_enum_mbus_format(
+						&code, 0, cs_sel, false);
+					cc = imx_media_find_mbus_format(
+						code, cs_sel, false);
+					sdformat->format.code = cc->codes[0];
+				}
 			}
 
 			if (sdformat->format.field != V4L2_FIELD_NONE)
 				sdformat->format.field = infmt->field;
 		} else {
-			cc = imx_media_find_ipu_format(0, sdformat->format.code,
-						       true);
+			cc = imx_media_find_ipu_format(sdformat->format.code,
+						       cs_sel);
 			if (!cc) {
-				imx_media_enum_ipu_format(NULL, &code, 0, true);
-				cc = imx_media_find_ipu_format(0, code, true);
+				imx_media_enum_ipu_format(&code, 0, cs_sel);
+				cc = imx_media_find_ipu_format(code, cs_sel);
 				sdformat->format.code = cc->codes[0];
 			}
 
@@ -999,6 +1027,9 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
 		}
 		break;
 	case CSI_SINK_PAD:
+		v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W,
+				      W_ALIGN, &sdformat->format.height,
+				      MIN_H, MAX_H, H_ALIGN, S_ALIGN);
 		crop.left = 0;
 		crop.top = 0;
 		crop.width = sdformat->format.width;
@@ -1008,11 +1039,13 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
 		if (ret)
 			return ret;
 
-		cc = imx_media_find_format(0, sdformat->format.code,
-					   true, false);
+		cc = imx_media_find_mbus_format(sdformat->format.code,
+						CS_SEL_ANY, true);
 		if (!cc) {
-			imx_media_enum_format(NULL, &code, 0, true, false);
-			cc = imx_media_find_format(0, code, true, false);
+			imx_media_enum_mbus_format(&code, 0,
+						   CS_SEL_ANY, false);
+			cc = imx_media_find_mbus_format(code,
+							CS_SEL_ANY, false);
 			sdformat->format.code = cc->codes[0];
 		}
 		break;
@@ -1157,7 +1190,7 @@ static int csi_registered(struct v4l2_subdev *sd)
 
 		code = 0;
 		if (i == CSI_SRC_PAD_DIRECT)
-			imx_media_enum_ipu_format(NULL, &code, 0, true);
+			imx_media_enum_ipu_format(&code, 0, CS_SEL_YUV);
 
 		/* set a default mbus format  */
 		ret = imx_media_init_mbus_fmt(&priv->format_mbus[i],
diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
index 6855560..a7fa84a 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -12,14 +12,13 @@
 #include "imx-media.h"
 
 /*
- * List of pixel formats for the subdevs. This must be a super-set of
- * the formats supported by the ipu image converter.
+ * List of supported pixel formats for the subdevs.
  *
- * The non-mbus formats (planar and BGR) must all fall at the end of
- * this table, otherwise enum_fmt() at media pads will stop before
- * seeing all the supported mbus formats.
+ * In all of these tables, the non-mbus formats (with no
+ * mbus codes) must all fall at the end of the table.
  */
-static const struct imx_media_pixfmt imx_media_formats[] = {
+
+static const struct imx_media_pixfmt yuv_formats[] = {
 	{
 		.fourcc	= V4L2_PIX_FMT_UYVY,
 		.codes  = {
@@ -36,13 +35,45 @@ static const struct imx_media_pixfmt imx_media_formats[] = {
 		},
 		.cs     = IPUV3_COLORSPACE_YUV,
 		.bpp    = 16,
+	},
+	/***
+	 * non-mbus YUV formats start here. NOTE! when adding non-mbus
+	 * formats, NUM_NON_MBUS_YUV_FORMATS must be updated below.
+	 ***/
+	{
+		.fourcc	= V4L2_PIX_FMT_YUV420,
+		.cs     = IPUV3_COLORSPACE_YUV,
+		.bpp    = 12,
+		.planar = true,
 	}, {
-		.fourcc = V4L2_PIX_FMT_YUV32,
-		.codes  = {MEDIA_BUS_FMT_AYUV8_1X32},
+		.fourcc = V4L2_PIX_FMT_YVU420,
 		.cs     = IPUV3_COLORSPACE_YUV,
-		.bpp    = 32,
-		.ipufmt = true,
+		.bpp    = 12,
+		.planar = true,
+	}, {
+		.fourcc = V4L2_PIX_FMT_YUV422P,
+		.cs     = IPUV3_COLORSPACE_YUV,
+		.bpp    = 16,
+		.planar = true,
+	}, {
+		.fourcc = V4L2_PIX_FMT_NV12,
+		.cs     = IPUV3_COLORSPACE_YUV,
+		.bpp    = 12,
+		.planar = true,
 	}, {
+		.fourcc = V4L2_PIX_FMT_NV16,
+		.cs     = IPUV3_COLORSPACE_YUV,
+		.bpp    = 16,
+		.planar = true,
+	},
+};
+
+#define NUM_NON_MBUS_YUV_FORMATS 5
+#define NUM_YUV_FORMATS ARRAY_SIZE(yuv_formats)
+#define NUM_MBUS_YUV_FORMATS (NUM_YUV_FORMATS - NUM_NON_MBUS_YUV_FORMATS)
+
+static const struct imx_media_pixfmt rgb_formats[] = {
+	{
 		.fourcc	= V4L2_PIX_FMT_RGB565,
 		.codes  = {MEDIA_BUS_FMT_RGB565_2X8_LE},
 		.cs     = IPUV3_COLORSPACE_RGB,
@@ -61,7 +92,9 @@ static const struct imx_media_pixfmt imx_media_formats[] = {
 		.cs     = IPUV3_COLORSPACE_RGB,
 		.bpp    = 32,
 		.ipufmt = true,
-	}, {
+	},
+	/*** raw bayer formats start here ***/
+	{
 		.fourcc = V4L2_PIX_FMT_SBGGR8,
 		.codes  = {MEDIA_BUS_FMT_SBGGR8_1X8},
 		.cs     = IPUV3_COLORSPACE_RGB,
@@ -130,7 +163,10 @@ static const struct imx_media_pixfmt imx_media_formats[] = {
 		.bpp    = 16,
 		.bayer  = true,
 	},
-	/*** non-mbus formats start here ***/
+	/***
+	 * non-mbus RGB formats start here. NOTE! when adding non-mbus
+	 * formats, NUM_NON_MBUS_RGB_FORMATS must be updated below.
+	 ***/
 	{
 		.fourcc	= V4L2_PIX_FMT_BGR24,
 		.cs     = IPUV3_COLORSPACE_RGB,
@@ -139,135 +175,256 @@ static const struct imx_media_pixfmt imx_media_formats[] = {
 		.fourcc	= V4L2_PIX_FMT_BGR32,
 		.cs     = IPUV3_COLORSPACE_RGB,
 		.bpp    = 32,
-	}, {
-		.fourcc	= V4L2_PIX_FMT_YUV420,
-		.cs     = IPUV3_COLORSPACE_YUV,
-		.bpp    = 12,
-		.planar = true,
-	}, {
-		.fourcc = V4L2_PIX_FMT_YVU420,
-		.cs     = IPUV3_COLORSPACE_YUV,
-		.bpp    = 12,
-		.planar = true,
-	}, {
-		.fourcc = V4L2_PIX_FMT_YUV422P,
-		.cs     = IPUV3_COLORSPACE_YUV,
-		.bpp    = 16,
-		.planar = true,
-	}, {
-		.fourcc = V4L2_PIX_FMT_NV12,
-		.cs     = IPUV3_COLORSPACE_YUV,
-		.bpp    = 12,
-		.planar = true,
-	}, {
-		.fourcc = V4L2_PIX_FMT_NV16,
+	},
+};
+
+#define NUM_NON_MBUS_RGB_FORMATS 2
+#define NUM_RGB_FORMATS ARRAY_SIZE(rgb_formats)
+#define NUM_MBUS_RGB_FORMATS (NUM_RGB_FORMATS - NUM_NON_MBUS_RGB_FORMATS)
+
+static const struct imx_media_pixfmt ipu_yuv_formats[] = {
+	{
+		.fourcc = V4L2_PIX_FMT_YUV32,
+		.codes  = {MEDIA_BUS_FMT_AYUV8_1X32},
 		.cs     = IPUV3_COLORSPACE_YUV,
-		.bpp    = 16,
-		.planar = true,
+		.bpp    = 32,
+		.ipufmt = true,
 	},
 };
 
-static const u32 imx_media_ipu_internal_codes[] = {
-	MEDIA_BUS_FMT_AYUV8_1X32, MEDIA_BUS_FMT_ARGB8888_1X32,
+#define NUM_IPU_YUV_FORMATS ARRAY_SIZE(ipu_yuv_formats)
+
+static const struct imx_media_pixfmt ipu_rgb_formats[] = {
+	{
+		.fourcc	= V4L2_PIX_FMT_RGB32,
+		.codes  = {MEDIA_BUS_FMT_ARGB8888_1X32},
+		.cs     = IPUV3_COLORSPACE_RGB,
+		.bpp    = 32,
+		.ipufmt = true,
+	},
 };
 
+#define NUM_IPU_RGB_FORMATS ARRAY_SIZE(ipu_rgb_formats)
+
 static inline u32 pixfmt_to_colorspace(const struct imx_media_pixfmt *fmt)
 {
 	return (fmt->cs == IPUV3_COLORSPACE_RGB) ?
 		V4L2_COLORSPACE_SRGB : V4L2_COLORSPACE_SMPTE170M;
 }
 
-static const struct imx_media_pixfmt *find_format(u32 fourcc, u32 code,
-						  bool allow_rgb,
-						  bool allow_planar,
-						  bool ipu_fmt_only)
+static const struct imx_media_pixfmt *find_format(u32 fourcc,
+						  u32 code,
+						  enum codespace_sel cs_sel,
+						  bool allow_non_mbus,
+						  bool allow_bayer)
 {
-	const struct imx_media_pixfmt *fmt, *ret = NULL;
+	const struct imx_media_pixfmt *array, *fmt, *ret = NULL;
+	u32 array_size;
 	int i, j;
 
-	for (i = 0; i < ARRAY_SIZE(imx_media_formats); i++) {
-		fmt = &imx_media_formats[i];
+	switch (cs_sel) {
+	case CS_SEL_YUV:
+		array_size = NUM_YUV_FORMATS;
+		array = yuv_formats;
+		break;
+	case CS_SEL_RGB:
+		array_size = NUM_RGB_FORMATS;
+		array = rgb_formats;
+		break;
+	case CS_SEL_ANY:
+		array_size = NUM_YUV_FORMATS + NUM_RGB_FORMATS;
+		array = yuv_formats;
+		break;
+	default:
+		return NULL;
+	}
+
+	for (i = 0; i < array_size; i++) {
+		if (cs_sel == CS_SEL_ANY && i >= NUM_YUV_FORMATS)
+			fmt = &rgb_formats[i - NUM_YUV_FORMATS];
+		else
+			fmt = &array[i];
 
-		if (ipu_fmt_only && !fmt->ipufmt)
+		if ((!allow_non_mbus && fmt->codes[0] == 0) ||
+		    (!allow_bayer && fmt->bayer))
 			continue;
 
-		if (fourcc && fmt->fourcc == fourcc &&
-		    (fmt->cs != IPUV3_COLORSPACE_RGB || allow_rgb) &&
-		    (!fmt->planar || allow_planar)) {
+		if (fourcc && fmt->fourcc == fourcc) {
 			ret = fmt;
 			goto out;
 		}
 
 		for (j = 0; code && fmt->codes[j]; j++) {
-			if (fmt->codes[j] == code && !fmt->planar &&
-			    (fmt->cs != IPUV3_COLORSPACE_RGB || allow_rgb)) {
+			if (code == fmt->codes[j]) {
 				ret = fmt;
 				goto out;
 			}
 		}
 	}
+
 out:
 	return ret;
 }
 
-const struct imx_media_pixfmt *imx_media_find_format(u32 fourcc, u32 code,
-						     bool allow_rgb,
-						     bool allow_planar)
+static int enum_format(u32 *fourcc, u32 *code, u32 index,
+		       enum codespace_sel cs_sel,
+		       bool allow_non_mbus,
+		       bool allow_bayer)
 {
-	return find_format(fourcc, code, allow_rgb, allow_planar, false);
+	const struct imx_media_pixfmt *fmt;
+	u32 mbus_yuv_sz = NUM_MBUS_YUV_FORMATS;
+	u32 mbus_rgb_sz = NUM_MBUS_RGB_FORMATS;
+	u32 yuv_sz = NUM_YUV_FORMATS;
+	u32 rgb_sz = NUM_RGB_FORMATS;
+
+	switch (cs_sel) {
+	case CS_SEL_YUV:
+		if (index >= yuv_sz ||
+		    (!allow_non_mbus && index >= mbus_yuv_sz))
+			return -EINVAL;
+		fmt = &yuv_formats[index];
+		break;
+	case CS_SEL_RGB:
+		if (index >= rgb_sz ||
+		    (!allow_non_mbus && index >= mbus_rgb_sz))
+			return -EINVAL;
+		fmt = &rgb_formats[index];
+		if (!allow_bayer && fmt->bayer)
+			return -EINVAL;
+		break;
+	case CS_SEL_ANY:
+		if (!allow_non_mbus) {
+			if (index >= mbus_yuv_sz) {
+				index -= mbus_yuv_sz;
+				if (index >= mbus_rgb_sz)
+					return -EINVAL;
+				fmt = &rgb_formats[index];
+				if (!allow_bayer && fmt->bayer)
+					return -EINVAL;
+			} else {
+				fmt = &yuv_formats[index];
+			}
+		} else {
+			if (index >= yuv_sz + rgb_sz)
+				return -EINVAL;
+			if (index >= yuv_sz) {
+				fmt = &rgb_formats[index - yuv_sz];
+				if (!allow_bayer && fmt->bayer)
+					return -EINVAL;
+			} else {
+				fmt = &yuv_formats[index];
+			}
+		}
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	if (fourcc)
+		*fourcc = fmt->fourcc;
+	if (code)
+		*code = fmt->codes[0];
+
+	return 0;
+}
+
+const struct imx_media_pixfmt *
+imx_media_find_format(u32 fourcc, enum codespace_sel cs_sel)
+{
+	return find_format(fourcc, 0, cs_sel, true, false);
 }
 EXPORT_SYMBOL_GPL(imx_media_find_format);
 
-const struct imx_media_pixfmt *imx_media_find_ipu_format(u32 fourcc,
-							 u32 code,
-							 bool allow_rgb)
+int imx_media_enum_format(u32 *fourcc, u32 index, enum codespace_sel cs_sel)
 {
-	return find_format(fourcc, code, allow_rgb, false, true);
+	return enum_format(fourcc, NULL, index, cs_sel, true, false);
 }
-EXPORT_SYMBOL_GPL(imx_media_find_ipu_format);
+EXPORT_SYMBOL_GPL(imx_media_enum_format);
 
-int imx_media_enum_format(u32 *fourcc, u32 *code, u32 index,
-			  bool allow_rgb, bool allow_planar)
+const struct imx_media_pixfmt *
+imx_media_find_mbus_format(u32 code, enum codespace_sel cs_sel,
+			   bool allow_bayer)
 {
-	const struct imx_media_pixfmt *fmt;
+	return find_format(0, code, cs_sel, false, allow_bayer);
+}
+EXPORT_SYMBOL_GPL(imx_media_find_mbus_format);
 
-	if (index >= ARRAY_SIZE(imx_media_formats))
-		return -EINVAL;
+int imx_media_enum_mbus_format(u32 *code, u32 index, enum codespace_sel cs_sel,
+			       bool allow_bayer)
+{
+	return enum_format(NULL, code, index, cs_sel, false, allow_bayer);
+}
+EXPORT_SYMBOL_GPL(imx_media_enum_mbus_format);
 
-	fmt = &imx_media_formats[index];
+const struct imx_media_pixfmt *
+imx_media_find_ipu_format(u32 code, enum codespace_sel cs_sel)
+{
+	const struct imx_media_pixfmt *array, *fmt, *ret = NULL;
+	u32 array_size;
+	int i, j;
 
-	if ((fmt->cs == IPUV3_COLORSPACE_RGB && !allow_rgb) ||
-	    (fmt->planar && !allow_planar))
-		return -EINVAL;
+	switch (cs_sel) {
+	case CS_SEL_YUV:
+		array_size = NUM_IPU_YUV_FORMATS;
+		array = ipu_yuv_formats;
+		break;
+	case CS_SEL_RGB:
+		array_size = NUM_IPU_RGB_FORMATS;
+		array = ipu_rgb_formats;
+		break;
+	case CS_SEL_ANY:
+		array_size = NUM_IPU_YUV_FORMATS + NUM_IPU_RGB_FORMATS;
+		array = ipu_yuv_formats;
+		break;
+	default:
+		return NULL;
+	}
 
-	if (code)
-		*code = fmt->codes[0];
-	if (fourcc)
-		*fourcc = fmt->fourcc;
+	for (i = 0; i < array_size; i++) {
+		if (cs_sel == CS_SEL_ANY && i >= NUM_IPU_YUV_FORMATS)
+			fmt = &ipu_rgb_formats[i - NUM_IPU_YUV_FORMATS];
+		else
+			fmt = &array[i];
 
-	return 0;
+		for (j = 0; code && fmt->codes[j]; j++) {
+			if (code == fmt->codes[j]) {
+				ret = fmt;
+				goto out;
+			}
+		}
+	}
+
+out:
+	return ret;
 }
-EXPORT_SYMBOL_GPL(imx_media_enum_format);
+EXPORT_SYMBOL_GPL(imx_media_find_ipu_format);
 
-int imx_media_enum_ipu_format(u32 *fourcc, u32 *code, u32 index,
-			      bool allow_rgb)
+int imx_media_enum_ipu_format(u32 *code, u32 index, enum codespace_sel cs_sel)
 {
-	const struct imx_media_pixfmt *fmt;
-	u32 lcode;
-
-	if (index >= ARRAY_SIZE(imx_media_ipu_internal_codes))
-		return -EINVAL;
-
-	lcode = imx_media_ipu_internal_codes[index];
-
-	fmt = find_format(0, lcode, allow_rgb, false, true);
-	if (!fmt)
+	switch (cs_sel) {
+	case CS_SEL_YUV:
+		if (index >= NUM_IPU_YUV_FORMATS)
+			return -EINVAL;
+		*code = ipu_yuv_formats[index].codes[0];
+		break;
+	case CS_SEL_RGB:
+		if (index >= NUM_IPU_RGB_FORMATS)
+			return -EINVAL;
+		*code = ipu_rgb_formats[index].codes[0];
+		break;
+	case CS_SEL_ANY:
+		if (index >= NUM_IPU_YUV_FORMATS + NUM_IPU_RGB_FORMATS)
+			return -EINVAL;
+		if (index >= NUM_IPU_YUV_FORMATS) {
+			index -= NUM_IPU_YUV_FORMATS;
+			*code = ipu_rgb_formats[index].codes[0];
+		} else {
+			*code = ipu_yuv_formats[index].codes[0];
+		}
+		break;
+	default:
 		return -EINVAL;
-
-	if (code)
-		*code = fmt->codes[0];
-	if (fourcc)
-		*fourcc = fmt->fourcc;
+	}
 
 	return 0;
 }
@@ -283,10 +440,14 @@ int imx_media_init_mbus_fmt(struct v4l2_mbus_framefmt *mbus,
 	mbus->height = height;
 	mbus->field = field;
 	if (code == 0)
-		imx_media_enum_format(NULL, &code, 0, true, false);
-	lcc = imx_media_find_format(0, code, true, false);
-	if (!lcc)
-		return -EINVAL;
+		imx_media_enum_mbus_format(&code, 0, CS_SEL_YUV, false);
+	lcc = imx_media_find_mbus_format(code, CS_SEL_ANY, false);
+	if (!lcc) {
+		lcc = imx_media_find_ipu_format(code, CS_SEL_ANY);
+		if (!lcc)
+			return -EINVAL;
+	}
+
 	mbus->code = code;
 	mbus->colorspace = pixfmt_to_colorspace(lcc);
 
@@ -304,7 +465,7 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix,
 	u32 stride;
 
 	if (!cc) {
-		cc = imx_media_find_format(0, mbus->code, true, false);
+		cc = imx_media_find_mbus_format(mbus->code, CS_SEL_ANY, true);
 		if (!cc)
 			return -EINVAL;
 	}
@@ -346,7 +507,7 @@ int imx_media_ipu_image_to_mbus_fmt(struct v4l2_mbus_framefmt *mbus,
 {
 	const struct imx_media_pixfmt *fmt;
 
-	fmt = imx_media_find_format(image->pix.pixelformat, 0, true, false);
+	fmt = imx_media_find_format(image->pix.pixelformat, CS_SEL_ANY);
 	if (!fmt)
 		return -EINVAL;
 
diff --git a/drivers/staging/media/imx/imx-media-vdic.c b/drivers/staging/media/imx/imx-media-vdic.c
index 5bb21e9..61e6017 100644
--- a/drivers/staging/media/imx/imx-media-vdic.c
+++ b/drivers/staging/media/imx/imx-media-vdic.c
@@ -520,20 +520,6 @@ static int vdic_s_stream(struct v4l2_subdev *sd, int enable)
 	return ret;
 }
 
-static int vdic_enum_mbus_code(struct v4l2_subdev *sd,
-			       struct v4l2_subdev_pad_config *cfg,
-			       struct v4l2_subdev_mbus_code_enum *code)
-{
-	if (code->pad >= VDIC_NUM_PADS)
-		return -EINVAL;
-
-	if (code->pad == VDIC_SINK_PAD_IDMAC)
-		return imx_media_enum_format(NULL, &code->code, code->index,
-					     false, false);
-
-	return imx_media_enum_ipu_format(NULL, &code->code, code->index, false);
-}
-
 static struct v4l2_mbus_framefmt *
 __vdic_get_fmt(struct vdic_priv *priv, struct v4l2_subdev_pad_config *cfg,
 	       unsigned int pad, enum v4l2_subdev_format_whence which)
@@ -544,6 +530,32 @@ __vdic_get_fmt(struct vdic_priv *priv, struct v4l2_subdev_pad_config *cfg,
 		return &priv->format_mbus[pad];
 }
 
+static int vdic_enum_mbus_code(struct v4l2_subdev *sd,
+			       struct v4l2_subdev_pad_config *cfg,
+			       struct v4l2_subdev_mbus_code_enum *code)
+{
+	int ret;
+
+	switch (code->pad) {
+	case VDIC_SINK_PAD_IDMAC:
+		ret = imx_media_enum_mbus_format(&code->code, code->index,
+						 CS_SEL_YUV, false);
+		break;
+	case VDIC_SINK_PAD_DIRECT:
+		ret = imx_media_enum_ipu_format(&code->code, code->index,
+						CS_SEL_YUV);
+		break;
+	case VDIC_SRC_PAD_DIRECT:
+		ret = imx_media_enum_ipu_format(&code->code, code->index,
+						CS_SEL_YUV);
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	return ret;
+}
+
 static int vdic_get_fmt(struct v4l2_subdev *sd,
 			struct v4l2_subdev_pad_config *cfg,
 			struct v4l2_subdev_format *sdformat)
@@ -578,19 +590,16 @@ static int vdic_set_fmt(struct v4l2_subdev *sd,
 	if (priv->stream_on)
 		return -EBUSY;
 
-	v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W_VDIC,
-			      W_ALIGN, &sdformat->format.height,
-			      MIN_H, MAX_H_VDIC, H_ALIGN, S_ALIGN);
-
 	switch (sdformat->pad) {
 	case VDIC_SRC_PAD_DIRECT:
 		infmt = __vdic_get_fmt(priv, cfg, priv->active_input_pad,
 				       sdformat->which);
 
-		cc = imx_media_find_ipu_format(0, sdformat->format.code, false);
+		cc = imx_media_find_ipu_format(sdformat->format.code,
+					       CS_SEL_YUV);
 		if (!cc) {
-			imx_media_enum_ipu_format(NULL, &code, 0, false);
-			cc = imx_media_find_ipu_format(0, code, false);
+			imx_media_enum_ipu_format(&code, 0, CS_SEL_YUV);
+			cc = imx_media_find_ipu_format(code, CS_SEL_YUV);
 			sdformat->format.code = cc->codes[0];
 		}
 
@@ -599,29 +608,36 @@ static int vdic_set_fmt(struct v4l2_subdev *sd,
 		/* output is always progressive! */
 		sdformat->format.field = V4L2_FIELD_NONE;
 		break;
-	case VDIC_SINK_PAD_IDMAC:
 	case VDIC_SINK_PAD_DIRECT:
-		if (sdformat->pad == VDIC_SINK_PAD_DIRECT) {
-			cc = imx_media_find_ipu_format(0, sdformat->format.code,
-						       false);
-			if (!cc) {
-				imx_media_enum_ipu_format(NULL, &code, 0,
-							  false);
-				cc = imx_media_find_ipu_format(0, code, false);
-				sdformat->format.code = cc->codes[0];
-			}
-		} else {
-			cc = imx_media_find_format(0, sdformat->format.code,
-						   false, false);
-			if (!cc) {
-				imx_media_enum_format(NULL, &code, 0,
-						      false, false);
-				cc = imx_media_find_format(0, code,
-							   false, false);
-				sdformat->format.code = cc->codes[0];
-			}
+		v4l_bound_align_image(&sdformat->format.width,
+				      MIN_W, MAX_W_VDIC, W_ALIGN,
+				      &sdformat->format.height,
+				      MIN_H, MAX_H_VDIC, H_ALIGN, S_ALIGN);
+
+		cc = imx_media_find_ipu_format(sdformat->format.code,
+					       CS_SEL_YUV);
+		if (!cc) {
+			imx_media_enum_ipu_format(&code, 0, CS_SEL_YUV);
+			cc = imx_media_find_ipu_format(code, CS_SEL_YUV);
+			sdformat->format.code = cc->codes[0];
 		}
+		/* input must be interlaced! Choose SEQ_TB if not */
+		if (!V4L2_FIELD_HAS_BOTH(sdformat->format.field))
+			sdformat->format.field = V4L2_FIELD_SEQ_TB;
+		break;
+	case VDIC_SINK_PAD_IDMAC:
+		v4l_bound_align_image(&sdformat->format.width,
+				      MIN_W, MAX_W_VDIC, W_ALIGN,
+				      &sdformat->format.height,
+				      MIN_H, MAX_H_VDIC, H_ALIGN, S_ALIGN);
 
+		cc = imx_media_find_mbus_format(sdformat->format.code,
+						CS_SEL_YUV, false);
+		if (!cc) {
+			imx_media_enum_mbus_format(&code, 0, CS_SEL_YUV, false);
+			cc = imx_media_find_mbus_format(code, CS_SEL_YUV, false);
+			sdformat->format.code = cc->codes[0];
+		}
 		/* input must be interlaced! Choose SEQ_TB if not */
 		if (!V4L2_FIELD_HAS_BOTH(sdformat->format.field))
 			sdformat->format.field = V4L2_FIELD_SEQ_TB;
@@ -766,7 +782,7 @@ static int vdic_registered(struct v4l2_subdev *sd)
 
 		code = 0;
 		if (i != VDIC_SINK_PAD_IDMAC)
-			imx_media_enum_ipu_format(NULL, &code, 0, true);
+			imx_media_enum_ipu_format(&code, 0, CS_SEL_YUV);
 
 		/* set a default mbus format  */
 		ret = imx_media_init_mbus_fmt(&priv->format_mbus[i],
diff --git a/drivers/staging/media/imx/imx-media.h b/drivers/staging/media/imx/imx-media.h
index 7f19739..4ac2bb6 100644
--- a/drivers/staging/media/imx/imx-media.h
+++ b/drivers/staging/media/imx/imx-media.h
@@ -171,16 +171,23 @@ struct imx_media_dev {
 	struct v4l2_async_notifier subdev_notifier;
 };
 
-const struct imx_media_pixfmt *imx_media_find_format(u32 fourcc, u32 code,
-						     bool allow_rgb,
-						     bool allow_planar);
-const struct imx_media_pixfmt *imx_media_find_ipu_format(u32 fourcc, u32 code,
-							 bool allow_rgb);
-
-int imx_media_enum_format(u32 *fourcc, u32 *code, u32 index,
-			  bool allow_rgb, bool allow_planar);
-int imx_media_enum_ipu_format(u32 *fourcc, u32 *code, u32 index,
-			      bool allow_rgb);
+enum codespace_sel {
+	CS_SEL_YUV = 0,
+	CS_SEL_RGB,
+	CS_SEL_ANY,
+};
+
+const struct imx_media_pixfmt *
+imx_media_find_format(u32 fourcc, enum codespace_sel cs_sel);
+int imx_media_enum_format(u32 *fourcc, u32 index, enum codespace_sel cs_sel);
+const struct imx_media_pixfmt *
+imx_media_find_mbus_format(u32 code, enum codespace_sel cs_sel,
+			   bool allow_bayer);
+int imx_media_enum_mbus_format(u32 *code, u32 index, enum codespace_sel cs_sel,
+			       bool allow_bayer);
+const struct imx_media_pixfmt *
+imx_media_find_ipu_format(u32 code, enum codespace_sel cs_sel);
+int imx_media_enum_ipu_format(u32 *code, u32 index, enum codespace_sel cs_sel);
 
 int imx_media_init_mbus_fmt(struct v4l2_mbus_framefmt *mbus,
 			    u32 width, u32 height, u32 code, u32 field,
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]


#1582493 — Re: [PATCH v4 33/36] media: imx: redo pixel format enumeration and negotiation

FromPhilipp Zabel <p.zabel@pengutronix.de>
Date2017-02-16 12:40 +0100
SubjectRe: [PATCH v4 33/36] media: imx: redo pixel format enumeration and negotiation
Message-ID<tbszM-5Ab-21@gated-at.bofh.it>
In reply to#1582259
On Wed, 2017-02-15 at 18:19 -0800, Steve Longerbeam wrote:
> The previous API and negotiation of mbus codes and pixel formats
> was broken, and has been completely redone.
> 
> The negotiation of media bus codes should be as follows:
> 
> CSI:
> 
> sink pad     direct src pad      IDMAC src pad
> --------     ----------------    -------------
> RGB (any)        IPU RGB           RGB (any)
> YUV (any)        IPU YUV           YUV (any)
> Bayer              N/A             must be same bayer code as sink

The IDMAC src pad should also use the internal 32-bit RGB / YUV format,
except if bayer/raw mode is selected, in which case the attached capture
video device should only allow a single mode corresponding to the output
pad media bus format.

regards
Philipp

[toc] | [prev] | [next] | [standalone]


#1586562 — Re: [PATCH v4 33/36] media: imx: redo pixel format enumeration and negotiation

FromSteve Longerbeam <slongerbeam@gmail.com>
Date2017-02-23 01:00 +0100
SubjectRe: [PATCH v4 33/36] media: imx: redo pixel format enumeration and negotiation
Message-ID<tdOZb-11s-3@gated-at.bofh.it>
In reply to#1582493
Hi Philipp,


On 02/16/2017 03:32 AM, Philipp Zabel wrote:
> On Wed, 2017-02-15 at 18:19 -0800, Steve Longerbeam wrote:
>> The previous API and negotiation of mbus codes and pixel formats
>> was broken, and has been completely redone.
>>
>> The negotiation of media bus codes should be as follows:
>>
>> CSI:
>>
>> sink pad     direct src pad      IDMAC src pad
>> --------     ----------------    -------------
>> RGB (any)        IPU RGB           RGB (any)
>> YUV (any)        IPU YUV           YUV (any)
>> Bayer              N/A             must be same bayer code as sink
>
> The IDMAC src pad should also use the internal 32-bit RGB / YUV format,
> except if bayer/raw mode is selected, in which case the attached capture
> video device should only allow a single mode corresponding to the output
> pad media bus format.

The IDMAC source pad is going to memory, so it has left the IPU.
Are you sure it should be an internal IPU format? I realize it
is linked to a capture device node, and the IPU format could then
be translated to a v4l2 fourcc by the capture device, but IMHO this
pad is external to the IPU.

Steve

[toc] | [prev] | [next] | [standalone]


#1586768 — Re: [PATCH v4 33/36] media: imx: redo pixel format enumeration and negotiation

FromPhilipp Zabel <p.zabel@pengutronix.de>
Date2017-02-23 10:20 +0100
SubjectRe: [PATCH v4 33/36] media: imx: redo pixel format enumeration and negotiation
Message-ID<tdXJ7-7Ko-15@gated-at.bofh.it>
In reply to#1586562
Hi Steve,

On Wed, 2017-02-22 at 15:52 -0800, Steve Longerbeam wrote:
> Hi Philipp,
> 
> 
> On 02/16/2017 03:32 AM, Philipp Zabel wrote:
> > On Wed, 2017-02-15 at 18:19 -0800, Steve Longerbeam wrote:
> >> The previous API and negotiation of mbus codes and pixel formats
> >> was broken, and has been completely redone.
> >>
> >> The negotiation of media bus codes should be as follows:
> >>
> >> CSI:
> >>
> >> sink pad     direct src pad      IDMAC src pad
> >> --------     ----------------    -------------
> >> RGB (any)        IPU RGB           RGB (any)
> >> YUV (any)        IPU YUV           YUV (any)
> >> Bayer              N/A             must be same bayer code as sink
> >
> > The IDMAC src pad should also use the internal 32-bit RGB / YUV format,
> > except if bayer/raw mode is selected, in which case the attached capture
> > video device should only allow a single mode corresponding to the output
> > pad media bus format.
> 
> The IDMAC source pad is going to memory, so it has left the IPU.
> Are you sure it should be an internal IPU format? I realize it
> is linked to a capture device node, and the IPU format could then
> be translated to a v4l2 fourcc by the capture device, but IMHO this
> pad is external to the IPU.

The CSI IDMAC source pad should describe the format at the connection
between the CSI and the IDMAC, just as the icprpvf and icprpenc source
pads.
The format outside of the IPU is the memory format written by the IDMAC,
but that is a memory pixel format and not a media bus format at all.

That would also make it more straightforward to enumerate the memory
pixel formats in the capture device: If the source pad media bus format
is 32-bit YUV, enumerate all YUV formats, if it is 32-bit RGB or RGB565,
enumerate all rgb formats, otherwise (bayer/raw mode) only allow the
specific memory format matching the bus format.

regards
Philipp

[toc] | [prev] | [next] | [standalone]


#1587234 — Re: [PATCH v4 33/36] media: imx: redo pixel format enumeration and negotiation

FromSteve Longerbeam <slongerbeam@gmail.com>
Date2017-02-24 02:50 +0100
SubjectRe: [PATCH v4 33/36] media: imx: redo pixel format enumeration and negotiation
Message-ID<tedbb-1cy-9@gated-at.bofh.it>
In reply to#1586768

On 02/23/2017 01:10 AM, Philipp Zabel wrote:
> Hi Steve,
>
> On Wed, 2017-02-22 at 15:52 -0800, Steve Longerbeam wrote:
>> Hi Philipp,
>>
>>
>> On 02/16/2017 03:32 AM, Philipp Zabel wrote:
>>> On Wed, 2017-02-15 at 18:19 -0800, Steve Longerbeam wrote:
>>>> The previous API and negotiation of mbus codes and pixel formats
>>>> was broken, and has been completely redone.
>>>>
>>>> The negotiation of media bus codes should be as follows:
>>>>
>>>> CSI:
>>>>
>>>> sink pad     direct src pad      IDMAC src pad
>>>> --------     ----------------    -------------
>>>> RGB (any)        IPU RGB           RGB (any)
>>>> YUV (any)        IPU YUV           YUV (any)
>>>> Bayer              N/A             must be same bayer code as sink
>>>
>>> The IDMAC src pad should also use the internal 32-bit RGB / YUV format,
>>> except if bayer/raw mode is selected, in which case the attached capture
>>> video device should only allow a single mode corresponding to the output
>>> pad media bus format.
>>
>> The IDMAC source pad is going to memory, so it has left the IPU.
>> Are you sure it should be an internal IPU format? I realize it
>> is linked to a capture device node, and the IPU format could then
>> be translated to a v4l2 fourcc by the capture device, but IMHO this
>> pad is external to the IPU.
>
> The CSI IDMAC source pad should describe the format at the connection
> between the CSI and the IDMAC, just as the icprpvf and icprpenc source
> pads.
> The format outside of the IPU is the memory format written by the IDMAC,
> but that is a memory pixel format and not a media bus format at all.
>

True, it is a memory format. I don't really mind if the CSI and PRP
ENC/VF source pads are characterized as IPU internal formats, since
they are only used to indicate the colorspace to the capture device.

And yes it did simplify the enumeration and try_fmt code a bit. So
I went ahead and made the change.

Steve


> That would also make it more straightforward to enumerate the memory
> pixel formats in the capture device: If the source pad media bus format
> is 32-bit YUV, enumerate all YUV formats, if it is 32-bit RGB or RGB565,
> enumerate all rgb formats, otherwise (bayer/raw mode) only allow the
> specific memory format matching the bus format.
>
>

[toc] | [prev] | [next] | [standalone]


#1582260 — [PATCH v4 26/36] media: imx: add support for bayer formats

FromSteve Longerbeam <slongerbeam@gmail.com>
Date2017-02-16 03:30 +0100
Subject[PATCH v4 26/36] media: imx: add support for bayer formats
Message-ID<tbjZw-8cO-43@gated-at.bofh.it>
In reply to#1582252
From: Russell King <rmk+kernel@armlinux.org.uk>

Add the bayer formats to imx-media's list of supported pixel and bus
formats.

Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>

- added a bayer boolean to struct imx_media_pixfmt.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
---
 drivers/staging/media/imx/imx-media-utils.c | 68 +++++++++++++++++++++++++++++
 drivers/staging/media/imx/imx-media.h       |  1 +
 2 files changed, 69 insertions(+)

diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
index 55603d9..6855560 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -61,6 +61,74 @@ static const struct imx_media_pixfmt imx_media_formats[] = {
 		.cs     = IPUV3_COLORSPACE_RGB,
 		.bpp    = 32,
 		.ipufmt = true,
+	}, {
+		.fourcc = V4L2_PIX_FMT_SBGGR8,
+		.codes  = {MEDIA_BUS_FMT_SBGGR8_1X8},
+		.cs     = IPUV3_COLORSPACE_RGB,
+		.bpp    = 8,
+		.bayer  = true,
+	}, {
+		.fourcc = V4L2_PIX_FMT_SGBRG8,
+		.codes  = {MEDIA_BUS_FMT_SGBRG8_1X8},
+		.cs     = IPUV3_COLORSPACE_RGB,
+		.bpp    = 8,
+		.bayer  = true,
+	}, {
+		.fourcc = V4L2_PIX_FMT_SGRBG8,
+		.codes  = {MEDIA_BUS_FMT_SGRBG8_1X8},
+		.cs     = IPUV3_COLORSPACE_RGB,
+		.bpp    = 8,
+		.bayer  = true,
+	}, {
+		.fourcc = V4L2_PIX_FMT_SRGGB8,
+		.codes  = {MEDIA_BUS_FMT_SRGGB8_1X8},
+		.cs     = IPUV3_COLORSPACE_RGB,
+		.bpp    = 8,
+		.bayer  = true,
+	}, {
+		.fourcc = V4L2_PIX_FMT_SBGGR16,
+		.codes  = {
+			MEDIA_BUS_FMT_SBGGR10_1X10,
+			MEDIA_BUS_FMT_SBGGR12_1X12,
+			MEDIA_BUS_FMT_SBGGR14_1X14,
+			MEDIA_BUS_FMT_SBGGR16_1X16
+		},
+		.cs     = IPUV3_COLORSPACE_RGB,
+		.bpp    = 16,
+		.bayer  = true,
+	}, {
+		.fourcc = V4L2_PIX_FMT_SGBRG16,
+		.codes  = {
+			MEDIA_BUS_FMT_SGBRG10_1X10,
+			MEDIA_BUS_FMT_SGBRG12_1X12,
+			MEDIA_BUS_FMT_SGBRG14_1X14,
+			MEDIA_BUS_FMT_SGBRG16_1X16,
+		},
+		.cs     = IPUV3_COLORSPACE_RGB,
+		.bpp    = 16,
+		.bayer  = true,
+	}, {
+		.fourcc = V4L2_PIX_FMT_SGRBG16,
+		.codes  = {
+			MEDIA_BUS_FMT_SGRBG10_1X10,
+			MEDIA_BUS_FMT_SGRBG12_1X12,
+			MEDIA_BUS_FMT_SGRBG14_1X14,
+			MEDIA_BUS_FMT_SGRBG16_1X16,
+		},
+		.cs     = IPUV3_COLORSPACE_RGB,
+		.bpp    = 16,
+		.bayer  = true,
+	}, {
+		.fourcc = V4L2_PIX_FMT_SRGGB16,
+		.codes  = {
+			MEDIA_BUS_FMT_SRGGB10_1X10,
+			MEDIA_BUS_FMT_SRGGB12_1X12,
+			MEDIA_BUS_FMT_SRGGB14_1X14,
+			MEDIA_BUS_FMT_SRGGB16_1X16,
+		},
+		.cs     = IPUV3_COLORSPACE_RGB,
+		.bpp    = 16,
+		.bayer  = true,
 	},
 	/*** non-mbus formats start here ***/
 	{
diff --git a/drivers/staging/media/imx/imx-media.h b/drivers/staging/media/imx/imx-media.h
index 3d4f3c7..ae3af0d 100644
--- a/drivers/staging/media/imx/imx-media.h
+++ b/drivers/staging/media/imx/imx-media.h
@@ -91,6 +91,7 @@ struct imx_media_pixfmt {
 	int     bpp;     /* total bpp */
 	enum ipu_color_space cs;
 	bool    planar;  /* is a planar format */
+	bool    bayer;   /* is a raw bayer format */
 	bool    ipufmt;  /* is one of the IPU internal formats */
 };
 
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]


#1582261 — [PATCH v4 07/36] ARM: dts: imx6-sabresd: add OV5642 and OV5640 camera sensors

FromSteve Longerbeam <slongerbeam@gmail.com>
Date2017-02-16 03:30 +0100
Subject[PATCH v4 07/36] ARM: dts: imx6-sabresd: add OV5642 and OV5640 camera sensors
Message-ID<tbjZx-8cO-49@gated-at.bofh.it>
In reply to#1582252
Enables the OV5642 parallel-bus sensor, and the OV5640 MIPI CSI-2 sensor.

The OV5642 connects to the parallel-bus mux input port on ipu1_csi0_mux.

The OV5640 connects to the input port on the MIPI CSI-2 receiver on
mipi_csi.

Until the OV5652 sensor module compatible with the SabreSD becomes
available for testing, the ov5642 node is currently disabled.
---
 arch/arm/boot/dts/imx6dl-sabresd.dts   |   5 ++
 arch/arm/boot/dts/imx6q-sabresd.dts    |   5 ++
 arch/arm/boot/dts/imx6qdl-sabresd.dtsi | 114 ++++++++++++++++++++++++++++++++-
 3 files changed, 123 insertions(+), 1 deletion(-)

diff --git a/arch/arm/boot/dts/imx6dl-sabresd.dts b/arch/arm/boot/dts/imx6dl-sabresd.dts
index 1e45f2f..9607afe 100644
--- a/arch/arm/boot/dts/imx6dl-sabresd.dts
+++ b/arch/arm/boot/dts/imx6dl-sabresd.dts
@@ -15,3 +15,8 @@
 	model = "Freescale i.MX6 DualLite SABRE Smart Device Board";
 	compatible = "fsl,imx6dl-sabresd", "fsl,imx6dl";
 };
+
+&ipu1_csi1_from_ipu1_csi1_mux {
+	clock-lanes = <0>;
+	data-lanes = <1 2>;
+};
diff --git a/arch/arm/boot/dts/imx6q-sabresd.dts b/arch/arm/boot/dts/imx6q-sabresd.dts
index 9cbdfe7..527772b 100644
--- a/arch/arm/boot/dts/imx6q-sabresd.dts
+++ b/arch/arm/boot/dts/imx6q-sabresd.dts
@@ -23,3 +23,8 @@
 &sata {
 	status = "okay";
 };
+
+&ipu1_csi1_from_mipi_vc1 {
+	clock-lanes = <0>;
+	data-lanes = <1 2>;
+};
diff --git a/arch/arm/boot/dts/imx6qdl-sabresd.dtsi b/arch/arm/boot/dts/imx6qdl-sabresd.dtsi
index 55ef535..f4e13c6 100644
--- a/arch/arm/boot/dts/imx6qdl-sabresd.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabresd.dtsi
@@ -10,6 +10,7 @@
  * http://www.gnu.org/copyleft/gpl.html
  */
 
+#include <dt-bindings/clock/imx6qdl-clock.h>
 #include <dt-bindings/gpio/gpio.h>
 #include <dt-bindings/input/input.h>
 
@@ -146,6 +147,36 @@
 	};
 };
 
+&ipu1_csi0_from_ipu1_csi0_mux {
+	bus-width = <8>;
+	data-shift = <12>; /* Lines 19:12 used */
+	hsync-active = <1>;
+	vsync-active = <1>;
+};
+
+&ipu1_csi0_mux_from_parallel_sensor {
+	remote-endpoint = <&ov5642_to_ipu1_csi0_mux>;
+};
+
+&ipu1_csi0 {
+	pinctrl-names = "default";
+	pinctrl-0 = <&pinctrl_ipu1_csi0>;
+};
+
+&mipi_csi {
+	status = "okay";
+
+	port@0 {
+		reg = <0>;
+
+		mipi_csi2_in: endpoint {
+			remote-endpoint = <&ov5640_to_mipi_csi2>;
+			clock-lanes = <0>;
+			data-lanes = <1 2>;
+		};
+	};
+};
+
 &audmux {
 	pinctrl-names = "default";
 	pinctrl-0 = <&pinctrl_audmux>;
@@ -214,7 +245,32 @@
 			0x8014 /* 4:FN_DMICCDAT */
 			0x0000 /* 5:Default */
 		>;
-       };
+	};
+
+	ov5642: camera@3c {
+		compatible = "ovti,ov5642";
+		pinctrl-names = "default";
+		pinctrl-0 = <&pinctrl_ov5642>;
+		clocks = <&clks IMX6QDL_CLK_CKO>;
+		clock-names = "xclk";
+		reg = <0x3c>;
+		DOVDD-supply = <&vgen4_reg>; /* 1.8v */
+		AVDD-supply = <&vgen3_reg>;  /* 2.8v, rev C board is VGEN3
+						rev B board is VGEN5 */
+		DVDD-supply = <&vgen2_reg>;  /* 1.5v*/
+		pwdn-gpios = <&gpio1 16 GPIO_ACTIVE_HIGH>;
+		reset-gpios = <&gpio1 17 GPIO_ACTIVE_LOW>;
+		status = "disabled";
+
+		port {
+			ov5642_to_ipu1_csi0_mux: endpoint {
+				remote-endpoint = <&ipu1_csi0_mux_from_parallel_sensor>;
+				bus-width = <8>;
+				hsync-active = <1>;
+				vsync-active = <1>;
+			};
+		};
+	};
 };
 
 &i2c2 {
@@ -223,6 +279,32 @@
 	pinctrl-0 = <&pinctrl_i2c2>;
 	status = "okay";
 
+	ov5640: camera@3c {
+		compatible = "ovti,ov5640";
+		pinctrl-names = "default";
+		pinctrl-0 = <&pinctrl_ov5640>;
+		reg = <0x3c>;
+		clocks = <&clks IMX6QDL_CLK_CKO>;
+		clock-names = "xclk";
+		DOVDD-supply = <&vgen4_reg>; /* 1.8v */
+		AVDD-supply = <&vgen3_reg>;  /* 2.8v, rev C board is VGEN3
+						rev B board is VGEN5 */
+		DVDD-supply = <&vgen2_reg>;  /* 1.5v*/
+		pwdn-gpios = <&gpio1 19 GPIO_ACTIVE_HIGH>;
+		reset-gpios = <&gpio1 20 GPIO_ACTIVE_LOW>;
+
+		port {
+			#address-cells = <1>;
+			#size-cells = <0>;
+
+			ov5640_to_mipi_csi2: endpoint {
+				remote-endpoint = <&mipi_csi2_in>;
+				clock-lanes = <0>;
+				data-lanes = <1 2>;
+			};
+		};
+	};
+
 	pmic: pfuze100@08 {
 		compatible = "fsl,pfuze100";
 		reg = <0x08>;
@@ -426,6 +508,36 @@
 			>;
 		};
 
+		pinctrl_ipu1_csi0: ipu1csi0grp {
+			fsl,pins = <
+				MX6QDL_PAD_CSI0_DAT12__IPU1_CSI0_DATA12    0x1b0b0
+				MX6QDL_PAD_CSI0_DAT13__IPU1_CSI0_DATA13    0x1b0b0
+				MX6QDL_PAD_CSI0_DAT14__IPU1_CSI0_DATA14    0x1b0b0
+				MX6QDL_PAD_CSI0_DAT15__IPU1_CSI0_DATA15    0x1b0b0
+				MX6QDL_PAD_CSI0_DAT16__IPU1_CSI0_DATA16    0x1b0b0
+				MX6QDL_PAD_CSI0_DAT17__IPU1_CSI0_DATA17    0x1b0b0
+				MX6QDL_PAD_CSI0_DAT18__IPU1_CSI0_DATA18    0x1b0b0
+				MX6QDL_PAD_CSI0_DAT19__IPU1_CSI0_DATA19    0x1b0b0
+				MX6QDL_PAD_CSI0_PIXCLK__IPU1_CSI0_PIXCLK   0x1b0b0
+				MX6QDL_PAD_CSI0_MCLK__IPU1_CSI0_HSYNC      0x1b0b0
+				MX6QDL_PAD_CSI0_VSYNC__IPU1_CSI0_VSYNC     0x1b0b0
+			>;
+		};
+
+		pinctrl_ov5640: ov5640grp {
+			fsl,pins = <
+				MX6QDL_PAD_SD1_DAT2__GPIO1_IO19 0x1b0b0
+				MX6QDL_PAD_SD1_CLK__GPIO1_IO20  0x1b0b0
+			>;
+		};
+
+		pinctrl_ov5642: ov5642grp {
+			fsl,pins = <
+				MX6QDL_PAD_SD1_DAT0__GPIO1_IO16 0x1b0b0
+				MX6QDL_PAD_SD1_DAT1__GPIO1_IO17 0x1b0b0
+			>;
+		};
+
 		pinctrl_pcie: pciegrp {
 			fsl,pins = <
 				MX6QDL_PAD_GPIO_17__GPIO7_IO12	0x1b0b0
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]


#1583009 — Re: [PATCH v4 07/36] ARM: dts: imx6-sabresd: add OV5642 and OV5640 camera sensors

FromFabio Estevam <festevam@gmail.com>
Date2017-02-17 02:00 +0100
SubjectRe: [PATCH v4 07/36] ARM: dts: imx6-sabresd: add OV5642 and OV5640 camera sensors
Message-ID<tbF3Y-5xt-9@gated-at.bofh.it>
In reply to#1582261
Hi Steve,

On Thu, Feb 16, 2017 at 12:19 AM, Steve Longerbeam
<slongerbeam@gmail.com> wrote:
> Enables the OV5642 parallel-bus sensor, and the OV5640 MIPI CSI-2 sensor.
>
> The OV5642 connects to the parallel-bus mux input port on ipu1_csi0_mux.
>
> The OV5640 connects to the input port on the MIPI CSI-2 receiver on
> mipi_csi.
>
> Until the OV5652 sensor module compatible with the SabreSD becomes
> available for testing, the ov5642 node is currently disabled.

You missed your Signed-off-by.

[toc] | [prev] | [next] | [standalone]


#1583010 — Re: [PATCH v4 07/36] ARM: dts: imx6-sabresd: add OV5642 and OV5640 camera sensors

FromSteve Longerbeam <slongerbeam@gmail.com>
Date2017-02-17 02:00 +0100
SubjectRe: [PATCH v4 07/36] ARM: dts: imx6-sabresd: add OV5642 and OV5640 camera sensors
Message-ID<tbF3Y-5xt-15@gated-at.bofh.it>
In reply to#1583009

On 02/16/2017 04:51 PM, Fabio Estevam wrote:
> Hi Steve,
>
> On Thu, Feb 16, 2017 at 12:19 AM, Steve Longerbeam
> <slongerbeam@gmail.com> wrote:
>> Enables the OV5642 parallel-bus sensor, and the OV5640 MIPI CSI-2 sensor.
>>
>> The OV5642 connects to the parallel-bus mux input port on ipu1_csi0_mux.
>>
>> The OV5640 connects to the input port on the MIPI CSI-2 receiver on
>> mipi_csi.
>>
>> Until the OV5652 sensor module compatible with the SabreSD becomes
>> available for testing, the ov5642 node is currently disabled.
>
> You missed your Signed-off-by.

Thanks, fixed.

Steve

[toc] | [prev] | [next] | [standalone]


#1582262 — [PATCH v4 10/36] ARM: dts: imx6-sabreauto: add pinctrl for gpt input capture

FromSteve Longerbeam <slongerbeam@gmail.com>
Date2017-02-16 03:30 +0100
Subject[PATCH v4 10/36] ARM: dts: imx6-sabreauto: add pinctrl for gpt input capture
Message-ID<tbjZx-8cO-45@gated-at.bofh.it>
In reply to#1582252
Add pinctrl groups for both GPT input capture channels.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
---
 arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
index 967c3b8..495709f 100644
--- a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
@@ -457,6 +457,18 @@
 			>;
 		};
 
+		pinctrl_gpt_input_capture0: gptinputcapture0grp {
+			fsl,pins = <
+				MX6QDL_PAD_SD1_DAT0__GPT_CAPTURE1	0x1b0b0
+			>;
+		};
+
+		pinctrl_gpt_input_capture1: gptinputcapture1grp {
+			fsl,pins = <
+				MX6QDL_PAD_SD1_DAT1__GPT_CAPTURE2	0x1b0b0
+			>;
+		};
+
 		pinctrl_spdif: spdifgrp {
 			fsl,pins = <
 				MX6QDL_PAD_KEY_COL3__SPDIF_IN 0x1b0b0
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]


#1582263 — [PATCH v4 06/36] ARM: dts: imx6-sabrelite: add OV5642 and OV5640 camera sensors

FromSteve Longerbeam <slongerbeam@gmail.com>
Date2017-02-16 03:30 +0100
Subject[PATCH v4 06/36] ARM: dts: imx6-sabrelite: add OV5642 and OV5640 camera sensors
Message-ID<tbjZx-8cO-47@gated-at.bofh.it>
In reply to#1582252
Adds the OV5642 parallel-bus sensor, and the OV5640 MIPI CSI-2 sensor.
Both hang off the same i2c2 bus, so they require different (and non-
default) i2c slave addresses.

The OV5642 connects to the parallel-bus mux input port on ipu1_csi0_mux.

The OV5640 connects to the input port on the MIPI CSI-2 receiver on
mipi_csi.

The OV5642 node is disabled temporarily while the subdev driver is
cleaned up and submitted later.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
---
 arch/arm/boot/dts/imx6dl-sabrelite.dts   |   5 ++
 arch/arm/boot/dts/imx6q-sabrelite.dts    |   5 ++
 arch/arm/boot/dts/imx6qdl-sabrelite.dtsi | 148 +++++++++++++++++++++++++++++++
 3 files changed, 158 insertions(+)

diff --git a/arch/arm/boot/dts/imx6dl-sabrelite.dts b/arch/arm/boot/dts/imx6dl-sabrelite.dts
index 0f06ca5..cfd5110 100644
--- a/arch/arm/boot/dts/imx6dl-sabrelite.dts
+++ b/arch/arm/boot/dts/imx6dl-sabrelite.dts
@@ -48,3 +48,8 @@
 	model = "Freescale i.MX6 DualLite SABRE Lite Board";
 	compatible = "fsl,imx6dl-sabrelite", "fsl,imx6dl";
 };
+
+&ipu1_csi1_from_ipu1_csi1_mux {
+	clock-lanes = <0>;
+	data-lanes = <1 2>;
+};
diff --git a/arch/arm/boot/dts/imx6q-sabrelite.dts b/arch/arm/boot/dts/imx6q-sabrelite.dts
index 66d10d8..e00fc06 100644
--- a/arch/arm/boot/dts/imx6q-sabrelite.dts
+++ b/arch/arm/boot/dts/imx6q-sabrelite.dts
@@ -52,3 +52,8 @@
 &sata {
 	status = "okay";
 };
+
+&ipu1_csi1_from_mipi_vc1 {
+	clock-lanes = <0>;
+	data-lanes = <1 2>;
+};
diff --git a/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi b/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
index 795b5a5..7958a0c 100644
--- a/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
@@ -39,6 +39,8 @@
  *     FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
  *     OTHER DEALINGS IN THE SOFTWARE.
  */
+
+#include <dt-bindings/clock/imx6qdl-clock.h>
 #include <dt-bindings/gpio/gpio.h>
 #include <dt-bindings/input/input.h>
 
@@ -94,6 +96,42 @@
 			pinctrl-0 = <&pinctrl_can_xcvr>;
 			gpio = <&gpio1 2 GPIO_ACTIVE_LOW>;
 		};
+
+		reg_1p5v: regulator@4 {
+			compatible = "regulator-fixed";
+			reg = <4>;
+			regulator-name = "1P5V";
+			regulator-min-microvolt = <1500000>;
+			regulator-max-microvolt = <1500000>;
+			regulator-always-on;
+		};
+
+		reg_1p8v: regulator@5 {
+			compatible = "regulator-fixed";
+			reg = <5>;
+			regulator-name = "1P8V";
+			regulator-min-microvolt = <1800000>;
+			regulator-max-microvolt = <1800000>;
+			regulator-always-on;
+		};
+
+		reg_2p8v: regulator@6 {
+			compatible = "regulator-fixed";
+			reg = <6>;
+			regulator-name = "2P8V";
+			regulator-min-microvolt = <2800000>;
+			regulator-max-microvolt = <2800000>;
+			regulator-always-on;
+		};
+	};
+
+	mipi_xclk: mipi_xclk {
+		compatible = "pwm-clock";
+		#clock-cells = <0>;
+		clock-frequency = <22000000>;
+		clock-output-names = "mipi_pwm3";
+		pwms = <&pwm3 0 45>; /* 1 / 45 ns = 22 MHz */
+		status = "okay";
 	};
 
 	gpio-keys {
@@ -220,6 +258,22 @@
 	};
 };
 
+&ipu1_csi0_from_ipu1_csi0_mux {
+	bus-width = <8>;
+	data-shift = <12>; /* Lines 19:12 used */
+	hsync-active = <1>;
+	vync-active = <1>;
+};
+
+&ipu1_csi0_mux_from_parallel_sensor {
+	remote-endpoint = <&ov5642_to_ipu1_csi0_mux>;
+};
+
+&ipu1_csi0 {
+	pinctrl-names = "default";
+	pinctrl-0 = <&pinctrl_ipu1_csi0>;
+};
+
 &audmux {
 	pinctrl-names = "default";
 	pinctrl-0 = <&pinctrl_audmux>;
@@ -299,6 +353,53 @@
 	pinctrl-names = "default";
 	pinctrl-0 = <&pinctrl_i2c2>;
 	status = "okay";
+
+	ov5640: camera@40 {
+		compatible = "ovti,ov5640";
+		pinctrl-names = "default";
+		pinctrl-0 = <&pinctrl_ov5640>;
+		reg = <0x40>;
+		clocks = <&mipi_xclk>;
+		clock-names = "xclk";
+		DOVDD-supply = <&reg_1p8v>;
+		AVDD-supply = <&reg_2p8v>;
+		DVDD-supply = <&reg_1p5v>;
+		reset-gpios = <&gpio2 5 GPIO_ACTIVE_LOW>; /* NANDF_D5 */
+		pwdn-gpios = <&gpio6 9 GPIO_ACTIVE_HIGH>; /* NANDF_WP_B */
+
+		port {
+			#address-cells = <1>;
+			#size-cells = <0>;
+
+			ov5640_to_mipi_csi2: endpoint {
+				remote-endpoint = <&mipi_csi2_in>;
+				clock-lanes = <0>;
+				data-lanes = <1 2>;
+			};
+		};
+	};
+
+	ov5642: camera@42 {
+		compatible = "ovti,ov5642";
+		pinctrl-names = "default";
+		pinctrl-0 = <&pinctrl_ov5642>;
+		clocks = <&clks IMX6QDL_CLK_CKO2>;
+		clock-names = "xclk";
+		reg = <0x42>;
+		reset-gpios = <&gpio1 8 GPIO_ACTIVE_LOW>;
+		pwdn-gpios = <&gpio1 6 GPIO_ACTIVE_HIGH>;
+		gp-gpios = <&gpio1 16 GPIO_ACTIVE_HIGH>;
+		status = "disabled";
+
+		port {
+			ov5642_to_ipu1_csi0_mux: endpoint {
+				remote-endpoint = <&ipu1_csi0_mux_from_parallel_sensor>;
+				bus-width = <8>;
+				hsync-active = <1>;
+				vsync-active = <1>;
+			};
+		};
+	};
 };
 
 &i2c3 {
@@ -412,6 +513,23 @@
 			>;
 		};
 
+		pinctrl_ipu1_csi0: ipu1csi0grp {
+			fsl,pins = <
+				MX6QDL_PAD_CSI0_DAT12__IPU1_CSI0_DATA12    0x1b0b0
+				MX6QDL_PAD_CSI0_DAT13__IPU1_CSI0_DATA13    0x1b0b0
+				MX6QDL_PAD_CSI0_DAT14__IPU1_CSI0_DATA14    0x1b0b0
+				MX6QDL_PAD_CSI0_DAT15__IPU1_CSI0_DATA15    0x1b0b0
+				MX6QDL_PAD_CSI0_DAT16__IPU1_CSI0_DATA16    0x1b0b0
+				MX6QDL_PAD_CSI0_DAT17__IPU1_CSI0_DATA17    0x1b0b0
+				MX6QDL_PAD_CSI0_DAT18__IPU1_CSI0_DATA18    0x1b0b0
+				MX6QDL_PAD_CSI0_DAT19__IPU1_CSI0_DATA19    0x1b0b0
+				MX6QDL_PAD_CSI0_PIXCLK__IPU1_CSI0_PIXCLK   0x1b0b0
+				MX6QDL_PAD_CSI0_MCLK__IPU1_CSI0_HSYNC      0x1b0b0
+				MX6QDL_PAD_CSI0_VSYNC__IPU1_CSI0_VSYNC     0x1b0b0
+				MX6QDL_PAD_CSI0_DATA_EN__IPU1_CSI0_DATA_EN 0x1b0b0
+			>;
+		};
+
 		pinctrl_j15: j15grp {
 			fsl,pins = <
 				MX6QDL_PAD_DI0_DISP_CLK__IPU1_DI0_DISP_CLK 0x10
@@ -445,6 +563,22 @@
 			>;
 		};
 
+		pinctrl_ov5640: ov5640grp {
+			fsl,pins = <
+				MX6QDL_PAD_NANDF_D5__GPIO2_IO05   0x000b0
+				MX6QDL_PAD_NANDF_WP_B__GPIO6_IO09 0x0b0b0
+			>;
+		};
+
+		pinctrl_ov5642: ov5642grp {
+			fsl,pins = <
+				MX6QDL_PAD_SD1_DAT0__GPIO1_IO16 0x1b0b0
+				MX6QDL_PAD_GPIO_6__GPIO1_IO06   0x1b0b0
+				MX6QDL_PAD_GPIO_8__GPIO1_IO08   0x130b0
+				MX6QDL_PAD_GPIO_3__CCM_CLKO2    0x000b0
+			>;
+		};
+
 		pinctrl_pwm1: pwm1grp {
 			fsl,pins = <
 				MX6QDL_PAD_SD1_DAT3__PWM1_OUT 0x1b0b1
@@ -601,3 +735,17 @@
 	vmmc-supply = <&reg_3p3v>;
 	status = "okay";
 };
+
+&mipi_csi {
+	status = "okay";
+
+	port@0 {
+		reg = <0>;
+
+		mipi_csi2_in: endpoint {
+			remote-endpoint = <&ov5640_to_mipi_csi2>;
+			clock-lanes = <0>;
+			data-lanes = <1 2>;
+		};
+	};
+};
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]


#1582264 — [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and getting of frame rates

FromSteve Longerbeam <slongerbeam@gmail.com>
Date2017-02-16 03:30 +0100
Subject[PATCH v4 29/36] media: imx: mipi-csi2: enable setting and getting of frame rates
Message-ID<tbjZx-8cO-53@gated-at.bofh.it>
In reply to#1582252
From: Russell King <rmk+kernel@armlinux.org.uk>

Setting and getting frame rates is part of the negotiation mechanism
between subdevs.  The lack of support means that a frame rate at the
sensor can't be negotiated through the subdev path.

Add support at MIPI CSI2 level for handling this part of the
negotiation.

Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
---
 drivers/staging/media/imx/imx6-mipi-csi2.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/drivers/staging/media/imx/imx6-mipi-csi2.c b/drivers/staging/media/imx/imx6-mipi-csi2.c
index 23dca80..c62f14e 100644
--- a/drivers/staging/media/imx/imx6-mipi-csi2.c
+++ b/drivers/staging/media/imx/imx6-mipi-csi2.c
@@ -34,6 +34,7 @@ struct csi2_dev {
 	struct v4l2_subdev      sd;
 	struct media_pad       pad[CSI2_NUM_PADS];
 	struct v4l2_mbus_framefmt format_mbus;
+	struct v4l2_fract      frame_interval;
 	struct clk             *dphy_clk;
 	struct clk             *cfg_clk;
 	struct clk             *pix_clk; /* what is this? */
@@ -397,6 +398,30 @@ static int csi2_set_fmt(struct v4l2_subdev *sd,
 	return 0;
 }
 
+static int csi2_g_frame_interval(struct v4l2_subdev *sd,
+				 struct v4l2_subdev_frame_interval *fi)
+{
+	struct csi2_dev *csi2 = sd_to_dev(sd);
+
+	fi->interval = csi2->frame_interval;
+
+	return 0;
+}
+
+static int csi2_s_frame_interval(struct v4l2_subdev *sd,
+				 struct v4l2_subdev_frame_interval *fi)
+{
+	struct csi2_dev *csi2 = sd_to_dev(sd);
+
+	/* Output pads mirror active input pad, no limits on input pads */
+	if (fi->pad != CSI2_SINK_PAD)
+		fi->interval = csi2->frame_interval;
+
+	csi2->frame_interval = fi->interval;
+
+	return 0;
+}
+
 /*
  * retrieve our pads parsed from the OF graph by the media device
  */
@@ -430,6 +455,8 @@ static struct v4l2_subdev_core_ops csi2_core_ops = {
 
 static struct v4l2_subdev_video_ops csi2_video_ops = {
 	.s_stream = csi2_s_stream,
+	.g_frame_interval = csi2_g_frame_interval,
+	.s_frame_interval = csi2_s_frame_interval,
 };
 
 static struct v4l2_subdev_pad_ops csi2_pad_ops = {
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]


#1583824 — Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and getting of frame rates

FromSteve Longerbeam <slongerbeam@gmail.com>
Date2017-02-18 02:20 +0100
SubjectRe: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and getting of frame rates
Message-ID<tc1QS-3yf-3@gated-at.bofh.it>
In reply to#1582264

On 02/15/2017 06:19 PM, Steve Longerbeam wrote:
> From: Russell King <rmk+kernel@armlinux.org.uk>
>
> Setting and getting frame rates is part of the negotiation mechanism
> between subdevs.  The lack of support means that a frame rate at the
> sensor can't be negotiated through the subdev path.
>
> Add support at MIPI CSI2 level for handling this part of the
> negotiation.
>
> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
> Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>


Hi Russell,

I signed-off on this but after more review I'm not sure this is right.

The CSI-2 receiver really has no control over frame rate. It's output
frame rate is the same as the rate that is delivered to it.

So this subdev should either not implement these ops, or it should
refer them to the attached source subdev.

Steve

> ---
>  drivers/staging/media/imx/imx6-mipi-csi2.c | 27 +++++++++++++++++++++++++++
>  1 file changed, 27 insertions(+)
>
> diff --git a/drivers/staging/media/imx/imx6-mipi-csi2.c b/drivers/staging/media/imx/imx6-mipi-csi2.c
> index 23dca80..c62f14e 100644
> --- a/drivers/staging/media/imx/imx6-mipi-csi2.c
> +++ b/drivers/staging/media/imx/imx6-mipi-csi2.c
> @@ -34,6 +34,7 @@ struct csi2_dev {
>  	struct v4l2_subdev      sd;
>  	struct media_pad       pad[CSI2_NUM_PADS];
>  	struct v4l2_mbus_framefmt format_mbus;
> +	struct v4l2_fract      frame_interval;
>  	struct clk             *dphy_clk;
>  	struct clk             *cfg_clk;
>  	struct clk             *pix_clk; /* what is this? */
> @@ -397,6 +398,30 @@ static int csi2_set_fmt(struct v4l2_subdev *sd,
>  	return 0;
>  }
>
> +static int csi2_g_frame_interval(struct v4l2_subdev *sd,
> +				 struct v4l2_subdev_frame_interval *fi)
> +{
> +	struct csi2_dev *csi2 = sd_to_dev(sd);
> +
> +	fi->interval = csi2->frame_interval;
> +
> +	return 0;
> +}
> +
> +static int csi2_s_frame_interval(struct v4l2_subdev *sd,
> +				 struct v4l2_subdev_frame_interval *fi)
> +{
> +	struct csi2_dev *csi2 = sd_to_dev(sd);
> +
> +	/* Output pads mirror active input pad, no limits on input pads */
> +	if (fi->pad != CSI2_SINK_PAD)
> +		fi->interval = csi2->frame_interval;
> +
> +	csi2->frame_interval = fi->interval;
> +
> +	return 0;
> +}
> +
>  /*
>   * retrieve our pads parsed from the OF graph by the media device
>   */
> @@ -430,6 +455,8 @@ static struct v4l2_subdev_core_ops csi2_core_ops = {
>
>  static struct v4l2_subdev_video_ops csi2_video_ops = {
>  	.s_stream = csi2_s_stream,
> +	.g_frame_interval = csi2_g_frame_interval,
> +	.s_frame_interval = csi2_s_frame_interval,
>  };
>
>  static struct v4l2_subdev_pad_ops csi2_pad_ops = {
>

-- 
Steve Longerbeam | Senior Embedded Engineer, ESD Services
Mentor Embedded(tm) | 46871 Bayside Parkway, Fremont, CA 94538
P 510.354.5838 | M 408.410.2735

[toc] | [prev] | [next] | [standalone]


#1583867 — Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and getting of frame rates

FromRussell King - ARM Linux <linux@armlinux.org.uk>
Date2017-02-18 10:30 +0100
SubjectRe: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and getting of frame rates
Message-ID<tc9v4-8tW-11@gated-at.bofh.it>
In reply to#1583824
On Fri, Feb 17, 2017 at 05:12:44PM -0800, Steve Longerbeam wrote:
> Hi Russell,
> 
> I signed-off on this but after more review I'm not sure this is right.
> 
> The CSI-2 receiver really has no control over frame rate. It's output
> frame rate is the same as the rate that is delivered to it.
> 
> So this subdev should either not implement these ops, or it should
> refer them to the attached source subdev.

Where in the V4L2 documentation does it say that is permissible?

If you don't implement these, media-ctl fails to propagate _anything_
to the next sink pad if you specify a frame rate, because media-ctl
throws an error and exits immediately.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

[toc] | [prev] | [next] | [standalone]


#1583936 — Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and getting of frame rates

FromSteve Longerbeam <slongerbeam@gmail.com>
Date2017-02-18 18:30 +0100
SubjectRe: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and getting of frame rates
Message-ID<tcgZz-4H1-5@gated-at.bofh.it>
In reply to#1583867

On 02/18/2017 01:23 AM, Russell King - ARM Linux wrote:
> On Fri, Feb 17, 2017 at 05:12:44PM -0800, Steve Longerbeam wrote:
>> Hi Russell,
>>
>> I signed-off on this but after more review I'm not sure this is right.
>>
>> The CSI-2 receiver really has no control over frame rate. It's output
>> frame rate is the same as the rate that is delivered to it.
>>
>> So this subdev should either not implement these ops, or it should
>> refer them to the attached source subdev.
>
> Where in the V4L2 documentation does it say that is permissible?
>

https://www.linuxtv.org/downloads/v4l-dvb-apis-old/vidioc-subdev-g-frame-interval.html

"The frame interval only makes sense for sub-devices that can control 
the frame period on their own. This includes, for instance, image 
sensors and TV tuners. Sub-devices that don't support frame intervals 
must not implement these ioctls."


> If you don't implement these, media-ctl fails to propagate _anything_
> to the next sink pad if you specify a frame rate, because media-ctl
> throws an error and exits immediately.
>

But I agree with you here. I think our only option is to ignore that
quoted requirement above and propagate [gs]_frame_interval all the way
to the CSI (which can control the frame rate via frame skipping).

Steve

[toc] | [prev] | [next] | [standalone]


#1583943 — Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and getting of frame rates

FromRussell King - ARM Linux <linux@armlinux.org.uk>
Date2017-02-18 19:20 +0100
SubjectRe: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and getting of frame rates
Message-ID<tchLY-5cw-3@gated-at.bofh.it>
In reply to#1583936
On Sat, Feb 18, 2017 at 09:29:17AM -0800, Steve Longerbeam wrote:
> On 02/18/2017 01:23 AM, Russell King - ARM Linux wrote:
> >On Fri, Feb 17, 2017 at 05:12:44PM -0800, Steve Longerbeam wrote:
> >>Hi Russell,
> >>
> >>I signed-off on this but after more review I'm not sure this is right.
> >>
> >>The CSI-2 receiver really has no control over frame rate. It's output
> >>frame rate is the same as the rate that is delivered to it.
> >>
> >>So this subdev should either not implement these ops, or it should
> >>refer them to the attached source subdev.
> >
> >Where in the V4L2 documentation does it say that is permissible?
> >
> 
> https://www.linuxtv.org/downloads/v4l-dvb-apis-old/vidioc-subdev-g-frame-interval.html
> 
> "The frame interval only makes sense for sub-devices that can control the
> frame period on their own. This includes, for instance, image sensors and TV
> tuners. Sub-devices that don't support frame intervals must not implement
> these ioctls."

That sounds clear - but the TV tuner example seems odd - the frame rate
is determined at transmission time, not reception time.  Yes, it's
possible to skip frames (which would be scaling) but you can't
_control_ the frame rate per se.

> >If you don't implement these, media-ctl fails to propagate _anything_
> >to the next sink pad if you specify a frame rate, because media-ctl
> >throws an error and exits immediately.
> >
> 
> But I agree with you here. I think our only option is to ignore that
> quoted requirement above and propagate [gs]_frame_interval all the way
> to the CSI (which can control the frame rate via frame skipping).

Sounds like something to tackle the media maintainers over - the
documentation vs media-ctl seem to have different ideas on this
point.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

[toc] | [prev] | [next] | [standalone]


#1583825 — Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and getting of frame rates

FromSteve Longerbeam <steve_longerbeam@mentor.com>
Date2017-02-18 02:20 +0100
SubjectRe: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and getting of frame rates
Message-ID<tc1QS-3yf-7@gated-at.bofh.it>
In reply to#1582264

On 02/15/2017 06:19 PM, Steve Longerbeam wrote:
> From: Russell King <rmk+kernel@armlinux.org.uk>
>
> Setting and getting frame rates is part of the negotiation mechanism
> between subdevs.  The lack of support means that a frame rate at the
> sensor can't be negotiated through the subdev path.
>
> Add support at MIPI CSI2 level for handling this part of the
> negotiation.
>
> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
> Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>


Hi Russell,

I signed-off on this but after more review I'm not sure this is right.

The CSI-2 receiver really has no control over frame rate. It's output
frame rate is the same as the rate that is delivered to it.

So this subdev should either not implement these ops, or it should
refer them to the attached source subdev.

Steve

> ---
>  drivers/staging/media/imx/imx6-mipi-csi2.c | 27 +++++++++++++++++++++++++++
>  1 file changed, 27 insertions(+)
>
> diff --git a/drivers/staging/media/imx/imx6-mipi-csi2.c b/drivers/staging/media/imx/imx6-mipi-csi2.c
> index 23dca80..c62f14e 100644
> --- a/drivers/staging/media/imx/imx6-mipi-csi2.c
> +++ b/drivers/staging/media/imx/imx6-mipi-csi2.c
> @@ -34,6 +34,7 @@ struct csi2_dev {
>  	struct v4l2_subdev      sd;
>  	struct media_pad       pad[CSI2_NUM_PADS];
>  	struct v4l2_mbus_framefmt format_mbus;
> +	struct v4l2_fract      frame_interval;
>  	struct clk             *dphy_clk;
>  	struct clk             *cfg_clk;
>  	struct clk             *pix_clk; /* what is this? */
> @@ -397,6 +398,30 @@ static int csi2_set_fmt(struct v4l2_subdev *sd,
>  	return 0;
>  }
>
> +static int csi2_g_frame_interval(struct v4l2_subdev *sd,
> +				 struct v4l2_subdev_frame_interval *fi)
> +{
> +	struct csi2_dev *csi2 = sd_to_dev(sd);
> +
> +	fi->interval = csi2->frame_interval;
> +
> +	return 0;
> +}
> +
> +static int csi2_s_frame_interval(struct v4l2_subdev *sd,
> +				 struct v4l2_subdev_frame_interval *fi)
> +{
> +	struct csi2_dev *csi2 = sd_to_dev(sd);
> +
> +	/* Output pads mirror active input pad, no limits on input pads */
> +	if (fi->pad != CSI2_SINK_PAD)
> +		fi->interval = csi2->frame_interval;
> +
> +	csi2->frame_interval = fi->interval;
> +
> +	return 0;
> +}
> +
>  /*
>   * retrieve our pads parsed from the OF graph by the media device
>   */
> @@ -430,6 +455,8 @@ static struct v4l2_subdev_core_ops csi2_core_ops = {
>
>  static struct v4l2_subdev_video_ops csi2_video_ops = {
>  	.s_stream = csi2_s_stream,
> +	.g_frame_interval = csi2_g_frame_interval,
> +	.s_frame_interval = csi2_s_frame_interval,
>  };
>
>  static struct v4l2_subdev_pad_ops csi2_pad_ops = {
>

-- 
Steve Longerbeam | Senior Embedded Engineer, ESD Services
Mentor Embedded(tm) | 46871 Bayside Parkway, Fremont, CA 94538
P 510.354.5838 | M 408.410.2735

[toc] | [prev] | [next] | [standalone]


#1584930 — Re: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and getting of frame rates

FromSakari Ailus <sakari.ailus@iki.fi>
Date2017-02-20 23:10 +0100
SubjectRe: [PATCH v4 29/36] media: imx: mipi-csi2: enable setting and getting of frame rates
Message-ID<td4jD-1Zp-7@gated-at.bofh.it>
In reply to#1582264
Hi Steve,

On Wed, Feb 15, 2017 at 06:19:31PM -0800, Steve Longerbeam wrote:
> From: Russell King <rmk+kernel@armlinux.org.uk>
> 
> Setting and getting frame rates is part of the negotiation mechanism
> between subdevs.  The lack of support means that a frame rate at the
> sensor can't be negotiated through the subdev path.

Just wondering --- what do you need this for?

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

[toc] | [prev] | [next] | [standalone]


Page 2 of 5 — ← Prev page 1 [2] 3 4 5  Next page →

Back to top | Article view | linux.kernel


csiph-web