Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1458763 > unrolled thread

[PATCH v3 0/4] Add MT8173 MDP Driver

Started byMinghsiu Tsai <minghsiu.tsai@mediatek.com>
First post2016-08-09 16:00 +0200
Last post2016-08-19 08:10 +0200
Articles 7 — 3 participants

Back to article view | Back to linux.kernel

Contents

  [PATCH v3 0/4] Add MT8173 MDP Driver  Minghsiu Tsai <minghsiu.tsai@mediatek.com> - 2016-08-09 16:00 +0200
    [PATCH v3 4/4] arm64: dts: mediatek: Add MDP for MT8173 Minghsiu Tsai <minghsiu.tsai@mediatek.com> - 2016-08-09 16:00 +0200
    [PATCH v3 2/4] dt-bindings: Add a binding for Mediatek MDP Minghsiu Tsai <minghsiu.tsai@mediatek.com> - 2016-08-09 16:00 +0200
      Re: [PATCH v3 2/4] dt-bindings: Add a binding for Mediatek MDP Rob Herring <robh@kernel.org> - 2016-08-11 00:30 +0200
    Re: [PATCH v3 3/4] media: Add Mediatek MDP Driver Hans Verkuil <hverkuil@xs4all.nl> - 2016-08-15 15:00 +0200
      Re: [PATCH v3 3/4] media: Add Mediatek MDP Driver Hans Verkuil <hverkuil@xs4all.nl> - 2016-08-19 08:10 +0200
      Re: [PATCH v3 3/4] media: Add Mediatek MDP Driver Minghsiu Tsai <minghsiu.tsai@mediatek.com> - 2016-08-19 08:10 +0200

#1458763 — [PATCH v3 0/4] Add MT8173 MDP Driver

FromMinghsiu Tsai <minghsiu.tsai@mediatek.com>
Date2016-08-09 16:00 +0200
Subject[PATCH v3 0/4] Add MT8173 MDP Driver
Message-ID<s4fJv-8kJ-3@gated-at.bofh.it>
Changes in v3:
- Modify device ndoe as structured one.
- Fix conflict in dts on Linux 4.8-rc1

Changes in v2:
- Add section to describe blocks function in dts-bindings
- Remove the assignment of device_caps in querycap()
- Remove format's name assignment
- Copy colorspace-related parameters from OUTPUT to CAPTURE
- Use m2m helper functions
- Fix DMA allocation failure
- Initialize lazily vpu instance in streamon()

==============
 Introduction
==============

The purpose of this series is to add the driver for Media Data Path HW embedded in the Mediatek's MT8173 SoC.
MDP is used for scaling and color space conversion.

It could convert V4L2_PIX_FMT_MT21 to V4L2_PIX_FMT_NV12M or V4L2_PIX_FMT_YUV420M.

NV12M/YUV420M/MT21 -> MDP -> NV12M/YUV420M

This patch series rely on MTK VPU driver in patch series "Add MT8173 Video Encoder Driver and VPU Driver"[1] and "Add MT8173 Video Decoder Driver"[2].
MDP driver rely on VPU driver to load, communicate with VPU.

Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI both have been merged in v4.6-rc1.

[1]https://patchwork.kernel.org/patch/9002171/
[2]https://patchwork.kernel.org/patch/9141245/

==================
 Device interface
==================

In principle the driver bases on v4l2 memory-to-memory framework:
it provides a single video node and each opened file handle gets its own private context with separate buffer queues. Each context consist of 2 buffer queues: OUTPUT (for source buffers) and CAPTURE (for destination buffers).
OUTPUT and CAPTURE buffer could be MMAP or DMABUF memory type.

v4l2-compliance test output:
# v4l2-compliance -d /dev/image-proc0
v4l2-compliance SHA   : ee1ab491019f80052834d14c76bdd1c1b46f2158

Driver Info:
        Driver name   : mtk-mdp
        Card type     : soc:mdp
        Bus info      : platform:mt8173
        Driver version: 4.8.0
        Capabilities  : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format

Compliance test for device /dev/image-proc0 (not using libv4l2):

Required ioctls:
        test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
        test second video open: OK
        test VIDIOC_QUERYCAP: OK
        test VIDIOC_G/S_PRIORITY: OK
        test for unlimited opens: OK

Debug ioctls:
        test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
        test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
        test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
        test VIDIOC_ENUMAUDIO: OK (Not Supported)
        test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDIO: OK (Not Supported)
        Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
        test VIDIOC_G/S_MODULATOR: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_ENUMAUDOUT: OK (Not Supported)
        test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDOUT: OK (Not Supported)
        Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
        test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
        test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
        test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
        test VIDIOC_G/S_EDID: OK (Not Supported)

        Control ioctls:
                test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
                test VIDIOC_QUERYCTRL: OK
                test VIDIOC_G/S_CTRL: OK
                test VIDIOC_G/S/TRY_EXT_CTRLS: OK
                test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
                test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
                Standard Controls: 5 Private Controls: 0

        Format ioctls:
                test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
                test VIDIOC_G/S_PARM: OK (Not Supported)
                test VIDIOC_G_FBUF: OK (Not Supported)
                test VIDIOC_G_FMT: OK
                test VIDIOC_TRY_FMT: OK
                test VIDIOC_S_FMT: OK
                test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
                test Cropping: OK
                test Composing: OK
                test Scaling: OK (Not Supported)

        Codec ioctls:
                test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
                test VIDIOC_G_ENC_INDEX: OK (Not Supported)
                test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

        Buffer ioctls:
                test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
                test VIDIOC_EXPBUF: OK

Test input 0:


Total: 43, Succeeded: 43, Failed: 0, Warnings: 0


Minghsiu Tsai (4):
  VPU: mediatek: Add mdp support
  dt-bindings: Add a binding for Mediatek MDP
  media: Add Mediatek MDP Driver
  arm64: dts: mediatek: Add MDP for MT8173

 .../devicetree/bindings/media/mediatek-mdp.txt     |  109 ++
 arch/arm64/boot/dts/mediatek/mt8173.dtsi           |   84 ++
 drivers/media/platform/Kconfig                     |   16 +
 drivers/media/platform/Makefile                    |    2 +
 drivers/media/platform/mtk-mdp/Makefile            |    9 +
 drivers/media/platform/mtk-mdp/mtk_mdp_comp.c      |  159 +++
 drivers/media/platform/mtk-mdp/mtk_mdp_comp.h      |   72 ++
 drivers/media/platform/mtk-mdp/mtk_mdp_core.c      |  294 +++++
 drivers/media/platform/mtk-mdp/mtk_mdp_core.h      |  240 ++++
 drivers/media/platform/mtk-mdp/mtk_mdp_ipi.h       |  126 ++
 drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c       | 1263 ++++++++++++++++++++
 drivers/media/platform/mtk-mdp/mtk_mdp_m2m.h       |   22 +
 drivers/media/platform/mtk-mdp/mtk_mdp_regs.c      |  153 +++
 drivers/media/platform/mtk-mdp/mtk_mdp_regs.h      |   31 +
 drivers/media/platform/mtk-mdp/mtk_mdp_vpu.c       |  145 +++
 drivers/media/platform/mtk-mdp/mtk_mdp_vpu.h       |   41 +
 drivers/media/platform/mtk-vpu/mtk_vpu.h           |    5 +
 17 files changed, 2771 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek-mdp.txt
 create mode 100644 drivers/media/platform/mtk-mdp/Makefile
 create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_comp.c
 create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_comp.h
 create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_core.c
 create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_core.h
 create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_ipi.h
 create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c
 create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_m2m.h
 create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_regs.c
 create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_regs.h
 create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_vpu.c
 create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_vpu.h

--
1.7.9.5

[toc] | [next] | [standalone]

#1458764 — [PATCH v3 4/4] arm64: dts: mediatek: Add MDP for MT8173

FromMinghsiu Tsai <minghsiu.tsai@mediatek.com>
Date2016-08-09 16:00 +0200
Subject[PATCH v3 4/4] arm64: dts: mediatek: Add MDP for MT8173
Message-ID<s4fJw-8kJ-13@gated-at.bofh.it>
In reply to#1458763
Add MDP node for MT8173

Signed-off-by: Minghsiu Tsai <minghsiu.tsai@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8173.dtsi |   84 ++++++++++++++++++++++++++++++
 1 file changed, 84 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8173.dtsi b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
index 10f638f..cd93228 100644
--- a/arch/arm64/boot/dts/mediatek/mt8173.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
@@ -41,6 +41,14 @@
 		dpi0 = &dpi0;
 		dsi0 = &dsi0;
 		dsi1 = &dsi1;
+		mdp_rdma0 = &mdp_rdma0;
+		mdp_rdma1 = &mdp_rdma1;
+		mdp_rsz0 = &mdp_rsz0;
+		mdp_rsz1 = &mdp_rsz1;
+		mdp_rsz2 = &mdp_rsz2;
+		mdp_wdma0 = &mdp_wdma0;
+		mdp_wrot0 = &mdp_wrot0;
+		mdp_wrot1 = &mdp_wrot1;
 	};
 
 	cpus {
@@ -716,6 +724,82 @@
 			#clock-cells = <1>;
 		};
 
+		mdp {
+			compatible = "mediatek,mt8173-mdp";
+			#address-cells = <2>;
+			#size-cells = <2>;
+			ranges;
+			mediatek,vpu = <&vpu>;
+
+			mdp_rdma0: rdma@14001000 {
+				compatible = "mediatek,mt8173-mdp-rdma";
+				reg = <0 0x14001000 0 0x1000>;
+				clocks = <&mmsys CLK_MM_MDP_RDMA0>,
+					 <&mmsys CLK_MM_MUTEX_32K>;
+				power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+				iommus = <&iommu M4U_PORT_MDP_RDMA0>;
+				mediatek,larb = <&larb0>;
+			};
+
+			mdp_rdma1: rdma@14002000 {
+				compatible = "mediatek,mt8173-mdp-rdma";
+				reg = <0 0x14002000 0 0x1000>;
+				clocks = <&mmsys CLK_MM_MDP_RDMA1>,
+					 <&mmsys CLK_MM_MUTEX_32K>;
+				power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+				iommus = <&iommu M4U_PORT_MDP_RDMA1>;
+				mediatek,larb = <&larb4>;
+			};
+
+			mdp_rsz0: rsz@14003000 {
+				compatible = "mediatek,mt8173-mdp-rsz";
+				reg = <0 0x14003000 0 0x1000>;
+				clocks = <&mmsys CLK_MM_MDP_RSZ0>;
+				power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+			};
+
+			mdp_rsz1: rsz@14004000 {
+				compatible = "mediatek,mt8173-mdp-rsz";
+				reg = <0 0x14004000 0 0x1000>;
+				clocks = <&mmsys CLK_MM_MDP_RSZ1>;
+				power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+			};
+
+			mdp_rsz2: rsz@14005000 {
+				compatible = "mediatek,mt8173-mdp-rsz";
+				reg = <0 0x14005000 0 0x1000>;
+				clocks = <&mmsys CLK_MM_MDP_RSZ2>;
+				power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+			};
+
+			mdp_wdma0: wdma@14006000 {
+				compatible = "mediatek,mt8173-mdp-wdma";
+				reg = <0 0x14006000 0 0x1000>;
+				clocks = <&mmsys CLK_MM_MDP_WDMA>;
+				power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+				iommus = <&iommu M4U_PORT_MDP_WDMA>;
+				mediatek,larb = <&larb0>;
+			};
+
+			mdp_wrot0: wrot@14007000 {
+				compatible = "mediatek,mt8173-mdp-wrot";
+				reg = <0 0x14007000 0 0x1000>;
+				clocks = <&mmsys CLK_MM_MDP_WROT0>;
+				power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+				iommus = <&iommu M4U_PORT_MDP_WROT0>;
+				mediatek,larb = <&larb0>;
+			};
+
+			mdp_wrot1: wrot@14008000 {
+				compatible = "mediatek,mt8173-mdp-wrot";
+				reg = <0 0x14008000 0 0x1000>;
+				clocks = <&mmsys CLK_MM_MDP_WROT1>;
+				power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+				iommus = <&iommu M4U_PORT_MDP_WROT1>;
+				mediatek,larb = <&larb4>;
+			};
+		};
+
 		ovl0: ovl@1400c000 {
 			compatible = "mediatek,mt8173-disp-ovl";
 			reg = <0 0x1400c000 0 0x1000>;
-- 
1.7.9.5

[toc] | [prev] | [next] | [standalone]

#1458767 — [PATCH v3 2/4] dt-bindings: Add a binding for Mediatek MDP

FromMinghsiu Tsai <minghsiu.tsai@mediatek.com>
Date2016-08-09 16:00 +0200
Subject[PATCH v3 2/4] dt-bindings: Add a binding for Mediatek MDP
Message-ID<s4fJw-8kJ-21@gated-at.bofh.it>
In reply to#1458763
Add a DT binding documentation of MDP for the MT8173 SoC
from Mediatek

Signed-off-by: Minghsiu Tsai <minghsiu.tsai@mediatek.com>
---
 .../devicetree/bindings/media/mediatek-mdp.txt     |  109 ++++++++++++++++++++
 1 file changed, 109 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek-mdp.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek-mdp.txt b/Documentation/devicetree/bindings/media/mediatek-mdp.txt
new file mode 100644
index 0000000..4182063
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek-mdp.txt
@@ -0,0 +1,109 @@
+* Mediatek Media Data Path
+
+Media Data Path is used for scaling and color space conversion.
+
+Required properties (controller (parent) node):
+- compatible: "mediatek,mt8173-mdp"
+- mediatek,vpu: the node of video processor unit, see
+  Documentation/devicetree/bindings/media/mediatek-vpu.txt for details.
+
+Required properties (all function blocks, child node):
+- compatible: Should be one of
+        "mediatek,mt8173-mdp-rdma"  - read DMA
+        "mediatek,mt8173-mdp-rsz"   - resizer
+        "mediatek,mt8173-mdp-wdma"  - write DMA
+        "mediatek,mt8173-mdp-wrot"  - write DMA with rotation
+- reg: Physical base address and length of the function block register space
+- clocks: device clocks, see
+  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- power-domains: a phandle to the power domain, see
+  Documentation/devicetree/bindings/power/power_domain.txt for details.
+
+Required properties (DMA function blocks, child node):
+- compatible: Should be one of
+        "mediatek,mt8173-mdp-rdma"
+        "mediatek,mt8173-mdp-wdma"
+        "mediatek,mt8173-mdp-wrot"
+- iommus: should point to the respective IOMMU block with master port as
+  argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- mediatek,larb: must contain the local arbiters in the current Socs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+
+Example:
+mdp {
+	compatible = "mediatek,mt8173-mdp";
+	#address-cells = <2>;
+	#size-cells = <2>;
+	ranges;
+	mediatek,vpu = <&vpu>;
+
+	mdp_rdma0: rdma@14001000 {
+		compatible = "mediatek,mt8173-mdp-rdma";
+		reg = <0 0x14001000 0 0x1000>;
+		clocks = <&mmsys CLK_MM_MDP_RDMA0>,
+			 <&mmsys CLK_MM_MUTEX_32K>;
+		power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+		iommus = <&iommu M4U_PORT_MDP_RDMA0>;
+		mediatek,larb = <&larb0>;
+	};
+
+	mdp_rdma1: rdma@14002000 {
+		compatible = "mediatek,mt8173-mdp-rdma";
+		reg = <0 0x14002000 0 0x1000>;
+		clocks = <&mmsys CLK_MM_MDP_RDMA1>,
+			 <&mmsys CLK_MM_MUTEX_32K>;
+		power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+		iommus = <&iommu M4U_PORT_MDP_RDMA1>;
+		mediatek,larb = <&larb4>;
+	};
+
+	mdp_rsz0: rsz@14003000 {
+		compatible = "mediatek,mt8173-mdp-rsz";
+		reg = <0 0x14003000 0 0x1000>;
+		clocks = <&mmsys CLK_MM_MDP_RSZ0>;
+		power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+	};
+
+	mdp_rsz1: rsz@14004000 {
+		compatible = "mediatek,mt8173-mdp-rsz";
+		reg = <0 0x14004000 0 0x1000>;
+		clocks = <&mmsys CLK_MM_MDP_RSZ1>;
+		power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+	};
+
+	mdp_rsz2: rsz@14005000 {
+		compatible = "mediatek,mt8173-mdp-rsz";
+		reg = <0 0x14005000 0 0x1000>;
+		clocks = <&mmsys CLK_MM_MDP_RSZ2>;
+		power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+	};
+
+	mdp_wdma0: wdma@14006000 {
+		compatible = "mediatek,mt8173-mdp-wdma";
+		reg = <0 0x14006000 0 0x1000>;
+		clocks = <&mmsys CLK_MM_MDP_WDMA>;
+		power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+		iommus = <&iommu M4U_PORT_MDP_WDMA>;
+		mediatek,larb = <&larb0>;
+	};
+
+	mdp_wrot0: wrot@14007000 {
+		compatible = "mediatek,mt8173-mdp-wrot";
+		reg = <0 0x14007000 0 0x1000>;
+		clocks = <&mmsys CLK_MM_MDP_WROT0>;
+		power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+		iommus = <&iommu M4U_PORT_MDP_WROT0>;
+		mediatek,larb = <&larb0>;
+	};
+
+	mdp_wrot1: wrot@14008000 {
+		compatible = "mediatek,mt8173-mdp-wrot";
+		reg = <0 0x14008000 0 0x1000>;
+		clocks = <&mmsys CLK_MM_MDP_WROT1>;
+		power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+		iommus = <&iommu M4U_PORT_MDP_WROT1>;
+		mediatek,larb = <&larb4>;
+	};
+};
-- 
1.7.9.5

[toc] | [prev] | [next] | [standalone]

#1460062 — Re: [PATCH v3 2/4] dt-bindings: Add a binding for Mediatek MDP

FromRob Herring <robh@kernel.org>
Date2016-08-11 00:30 +0200
SubjectRe: [PATCH v3 2/4] dt-bindings: Add a binding for Mediatek MDP
Message-ID<s4KaB-2JP-17@gated-at.bofh.it>
In reply to#1458767
On Tue, Aug 09, 2016 at 09:58:55PM +0800, Minghsiu Tsai wrote:
> Add a DT binding documentation of MDP for the MT8173 SoC
> from Mediatek
> 
> Signed-off-by: Minghsiu Tsai <minghsiu.tsai@mediatek.com>
> ---
>  .../devicetree/bindings/media/mediatek-mdp.txt     |  109 ++++++++++++++++++++
>  1 file changed, 109 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek-mdp.txt

Acked-by: Rob Herring <robh@kernel.org>

[toc] | [prev] | [next] | [standalone]

#1462760 — Re: [PATCH v3 3/4] media: Add Mediatek MDP Driver

FromHans Verkuil <hverkuil@xs4all.nl>
Date2016-08-15 15:00 +0200
SubjectRe: [PATCH v3 3/4] media: Add Mediatek MDP Driver
Message-ID<s6pEJ-5I5-27@gated-at.bofh.it>
In reply to#1458763
On 08/09/2016 03:58 PM, Minghsiu Tsai wrote:
> Add MDP driver for MT8173
> 
> Signed-off-by: Minghsiu Tsai <minghsiu.tsai@mediatek.com>
> ---
>  drivers/media/platform/Kconfig                |   16 +
>  drivers/media/platform/Makefile               |    2 +
>  drivers/media/platform/mtk-mdp/Makefile       |    9 +
>  drivers/media/platform/mtk-mdp/mtk_mdp_comp.c |  159 ++++
>  drivers/media/platform/mtk-mdp/mtk_mdp_comp.h |   72 ++
>  drivers/media/platform/mtk-mdp/mtk_mdp_core.c |  294 ++++++
>  drivers/media/platform/mtk-mdp/mtk_mdp_core.h |  240 +++++
>  drivers/media/platform/mtk-mdp/mtk_mdp_ipi.h  |  126 +++
>  drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c  | 1263 +++++++++++++++++++++++++
>  drivers/media/platform/mtk-mdp/mtk_mdp_m2m.h  |   22 +
>  drivers/media/platform/mtk-mdp/mtk_mdp_regs.c |  153 +++
>  drivers/media/platform/mtk-mdp/mtk_mdp_regs.h |   31 +
>  drivers/media/platform/mtk-mdp/mtk_mdp_vpu.c  |  145 +++
>  drivers/media/platform/mtk-mdp/mtk_mdp_vpu.h  |   41 +
>  14 files changed, 2573 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-mdp/Makefile
>  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_comp.c
>  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_comp.h
>  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_core.c
>  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_core.h
>  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_ipi.h
>  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c
>  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_m2m.h
>  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_regs.c
>  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_regs.h
>  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_vpu.c
>  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_vpu.h
> 
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index f25344b..4bb874b 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -166,6 +166,22 @@ config VIDEO_MEDIATEK_VPU
>  	    To compile this driver as a module, choose M here: the
>  	    module will be called mtk-vpu.
>  
> +config VIDEO_MEDIATEK_MDP
> +	tristate "Mediatek MDP driver"
> +	depends on MTK_IOMMU || COMPILE_TEST
> +	depends on VIDEO_DEV && VIDEO_V4L2
> +	depends on ARCH_MEDIATEK || COMPILE_TEST

This needs a 'depends on HAS_DMA' as well.

> +	select VIDEOBUF2_DMA_CONTIG
> +	select V4L2_MEM2MEM_DEV
> +	select VIDEO_MEDIATEK_VPU
> +	default n
> +	---help---
> +	    It is a v4l2 driver and present in Mediatek MT8173 SoCs.
> +	    The driver supports for scaling and color space conversion.
> +
> +	    To compile this driver as a module, choose M here: the
> +	    module will be called mtk-mdp.
> +
>  config VIDEO_MEDIATEK_VCODEC
>  	tristate "Mediatek Video Codec driver"
>  	depends on MTK_IOMMU || COMPILE_TEST

<snip>

> +
> +/*
> + * Return true if rectangle a is enclosed in rectangle b, or false otherwise.
> + */
> +static bool mtk_mdp_m2m_is_rectangle_enclosed(struct v4l2_rect *a,
> +					     struct v4l2_rect *b)
> +{
> +	if (a->left < b->left || a->top < b->top)
> +		return false;
> +
> +	if (a->left + a->width > b->left + b->width)
> +		return false;
> +
> +	if (a->top + a->height > b->top + b->height)
> +		return false;
> +
> +	return true;
> +}
> +
> +static int mtk_mdp_m2m_g_selection(struct file *file, void *fh,
> +				   struct v4l2_selection *s)
> +{
> +	struct mtk_mdp_frame *frame;
> +	struct mtk_mdp_ctx *ctx = fh_to_ctx(fh);
> +
> +	if ((s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE) &&
> +	    (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)) {
> +		mtk_mdp_dbg(1, "[%d] invalid type:%d", ctx->id, s->type);
> +		return -EINVAL;
> +	}
> +
> +	frame = mtk_mdp_ctx_get_frame(ctx, s->type);
> +
> +	switch (s->target) {
> +	case V4L2_SEL_TGT_COMPOSE_DEFAULT:
> +	case V4L2_SEL_TGT_COMPOSE_BOUNDS:
> +	case V4L2_SEL_TGT_CROP_BOUNDS:
> +	case V4L2_SEL_TGT_CROP_DEFAULT:
> +		s->r.left = 0;
> +		s->r.top = 0;
> +		s->r.width = frame->width;
> +		s->r.height = frame->height;
> +		return 0;
> +
> +	case V4L2_SEL_TGT_COMPOSE:
> +	case V4L2_SEL_TGT_CROP:
> +		s->r.left = frame->crop.left;
> +		s->r.top = frame->crop.top;
> +		s->r.width = frame->crop.width;
> +		s->r.height = frame->crop.height;
> +		return 0;
> +	}

This isn't right. For VIDEO_CAPTURE you support the COMPOSE targets, and for
the VIDEO_OUTPUT you support the CROP targets. Right now I can use e.g. TGT_CROP
with VIDEO_CAPTURE, which isn't correct.

s_selection has the same problem.

> +
> +	return -EINVAL;
> +}
> +
> +static int mtk_mdp_check_scaler_ratio(struct mtk_mdp_variant *var, int src_w,
> +				      int src_h, int dst_w, int dst_h, int rot)
> +{
> +	int tmp_w, tmp_h;
> +
> +	if (rot == 90 || rot == 270) {
> +		tmp_w = dst_h;
> +		tmp_h = dst_w;
> +	} else {
> +		tmp_w = dst_w;
> +		tmp_h = dst_h;
> +	}
> +
> +	if ((src_w / tmp_w) > var->h_scale_down_max ||
> +	    (src_h / tmp_h) > var->v_scale_down_max ||
> +	    (tmp_w / src_w) > var->h_scale_up_max ||
> +	    (tmp_h / src_h) > var->v_scale_up_max)
> +		return -EINVAL;
> +
> +	return 0;
> +}
> +
> +static int mtk_mdp_m2m_s_selection(struct file *file, void *fh,
> +				   struct v4l2_selection *s)
> +{
> +	struct mtk_mdp_frame *frame;
> +	struct mtk_mdp_ctx *ctx = fh_to_ctx(fh);
> +	struct v4l2_crop cr;
> +	struct mtk_mdp_variant *variant = ctx->mdp_dev->variant;
> +	int ret;
> +
> +	cr.type = s->type;
> +	cr.c = s->r;
> +
> +	if ((s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE) &&
> +	    (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)) {
> +		mtk_mdp_dbg(1, "[%d] invalid type:%d", ctx->id, s->type);
> +		return -EINVAL;
> +	}
> +
> +	ret = mtk_mdp_try_crop(ctx, &cr);

Please don't copy to a v4l2_crop. Just stick to v4l2_selection.
I would prefer not to see struct v4l2_crop in drivers.

> +	if (ret)
> +		return ret;
> +
> +	if (s->flags & V4L2_SEL_FLAG_LE &&
> +	    !mtk_mdp_m2m_is_rectangle_enclosed(&cr.c, &s->r))
> +		return -ERANGE;
> +
> +	if (s->flags & V4L2_SEL_FLAG_GE &&
> +	    !mtk_mdp_m2m_is_rectangle_enclosed(&s->r, &cr.c))
> +		return -ERANGE;

As has been discussed for the encoder driver: just adjust the rectangle and
don't return ERANGE. Consider the flags as hints.

The V4L2 documentation is confusing, this will be addressed soon.

> +
> +	s->r = cr.c;
> +
> +	switch (s->target) {
> +	case V4L2_SEL_TGT_COMPOSE_BOUNDS:
> +	case V4L2_SEL_TGT_COMPOSE_DEFAULT:
> +	case V4L2_SEL_TGT_COMPOSE:
> +		frame = &ctx->s_frame;
> +		break;
> +
> +	case V4L2_SEL_TGT_CROP_BOUNDS:
> +	case V4L2_SEL_TGT_CROP:
> +	case V4L2_SEL_TGT_CROP_DEFAULT:
> +		frame = &ctx->d_frame;
> +		break;
> +
> +	default:
> +		return -EINVAL;
> +	}
> +
> +	/* Check to see if scaling ratio is within supported range */
> +	if (mtk_mdp_ctx_state_is_set(ctx, MTK_MDP_DST_FMT | MTK_MDP_SRC_FMT)) {
> +		if (V4L2_TYPE_IS_OUTPUT(s->type)) {
> +			ret = mtk_mdp_check_scaler_ratio(variant, cr.c.width,
> +				cr.c.height, ctx->d_frame.crop.width,
> +				ctx->d_frame.crop.height,
> +				ctx->ctrls.rotate->val);
> +		} else {
> +			ret = mtk_mdp_check_scaler_ratio(variant,
> +				ctx->s_frame.crop.width,
> +				ctx->s_frame.crop.height, cr.c.width,
> +				cr.c.height, ctx->ctrls.rotate->val);
> +		}
> +
> +		if (ret) {
> +			dev_info(&ctx->mdp_dev->pdev->dev,
> +				"Out of scaler range");
> +			return -EINVAL;
> +		}
> +	}
> +
> +	frame->crop = cr.c;
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_ioctl_ops mtk_mdp_m2m_ioctl_ops = {
> +	.vidioc_querycap		= mtk_mdp_m2m_querycap,
> +	.vidioc_enum_fmt_vid_cap_mplane	= mtk_mdp_m2m_enum_fmt_mplane_vid_cap,
> +	.vidioc_enum_fmt_vid_out_mplane	= mtk_mdp_m2m_enum_fmt_mplane_vid_out,
> +	.vidioc_g_fmt_vid_cap_mplane	= mtk_mdp_m2m_g_fmt_mplane,
> +	.vidioc_g_fmt_vid_out_mplane	= mtk_mdp_m2m_g_fmt_mplane,
> +	.vidioc_try_fmt_vid_cap_mplane	= mtk_mdp_m2m_try_fmt_mplane,
> +	.vidioc_try_fmt_vid_out_mplane	= mtk_mdp_m2m_try_fmt_mplane,
> +	.vidioc_s_fmt_vid_cap_mplane	= mtk_mdp_m2m_s_fmt_mplane,
> +	.vidioc_s_fmt_vid_out_mplane	= mtk_mdp_m2m_s_fmt_mplane,
> +	.vidioc_reqbufs			= mtk_mdp_m2m_reqbufs,
> +	.vidioc_create_bufs		= v4l2_m2m_ioctl_create_bufs,
> +	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
> +	.vidioc_subscribe_event		= v4l2_ctrl_subscribe_event,
> +	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
> +	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
> +	.vidioc_qbuf			= v4l2_m2m_ioctl_qbuf,
> +	.vidioc_dqbuf			= v4l2_m2m_ioctl_dqbuf,
> +	.vidioc_streamon		= mtk_mdp_m2m_streamon,
> +	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
> +	.vidioc_g_selection		= mtk_mdp_m2m_g_selection,
> +	.vidioc_s_selection		= mtk_mdp_m2m_s_selection
> +};
> +
> +static int mtk_mdp_m2m_queue_init(void *priv, struct vb2_queue *src_vq,
> +				  struct vb2_queue *dst_vq)
> +{
> +	struct mtk_mdp_ctx *ctx = priv;
> +	int ret;
> +
> +	memset(src_vq, 0, sizeof(*src_vq));
> +	src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> +	src_vq->io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF;
> +	src_vq->drv_priv = ctx;
> +	src_vq->ops = &mtk_mdp_m2m_qops;
> +	src_vq->mem_ops = &vb2_dma_contig_memops;
> +	src_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
> +	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> +	src_vq->dev = &ctx->mdp_dev->pdev->dev;
> +
> +	ret = vb2_queue_init(src_vq);
> +	if (ret)
> +		return ret;
> +
> +	memset(dst_vq, 0, sizeof(*dst_vq));
> +	dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> +	dst_vq->io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF;

I wouldn't recommend setting VB2_USERPTR in combination with dma_contig.

That only works if the userptr points to physically contiguous memory,
unless there is an iommu that can fix things up.

> +	dst_vq->drv_priv = ctx;
> +	dst_vq->ops = &mtk_mdp_m2m_qops;
> +	dst_vq->mem_ops = &vb2_dma_contig_memops;
> +	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
> +	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> +	dst_vq->dev = &ctx->mdp_dev->pdev->dev;
> +
> +	return vb2_queue_init(dst_vq);
> +}
> +
> +static int mtk_mdp_s_ctrl(struct v4l2_ctrl *ctrl)
> +{
> +	struct mtk_mdp_ctx *ctx = ctrl_to_ctx(ctrl);
> +	struct mtk_mdp_dev *mdp = ctx->mdp_dev;
> +	struct mtk_mdp_variant *variant = mdp->variant;
> +	u32 state = MTK_MDP_DST_FMT | MTK_MDP_SRC_FMT;
> +	int ret = 0;
> +
> +	if (ctrl->flags & V4L2_CTRL_FLAG_INACTIVE)
> +		return 0;
> +
> +	switch (ctrl->id) {
> +	case V4L2_CID_HFLIP:
> +		ctx->hflip = ctrl->val;
> +		break;
> +	case V4L2_CID_VFLIP:
> +		ctx->vflip = ctrl->val;
> +		break;
> +	case V4L2_CID_ROTATE:
> +		if (mtk_mdp_ctx_state_is_set(ctx, state)) {
> +			ret = mtk_mdp_check_scaler_ratio(variant,
> +					ctx->s_frame.crop.width,
> +					ctx->s_frame.crop.height,
> +					ctx->d_frame.crop.width,
> +					ctx->d_frame.crop.height,
> +					ctx->ctrls.rotate->val);
> +
> +			if (ret)
> +				return -EINVAL;
> +		}
> +
> +		ctx->rotation = ctrl->val;
> +		break;
> +	case V4L2_CID_ALPHA_COMPONENT:
> +		ctx->d_frame.alpha = ctrl->val;
> +		break;
> +	}
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_ctrl_ops mtk_mdp_ctrl_ops = {
> +	.s_ctrl = mtk_mdp_s_ctrl,
> +};
> +
> +static int mtk_mdp_ctrls_create(struct mtk_mdp_ctx *ctx)
> +{
> +	v4l2_ctrl_handler_init(&ctx->ctrl_handler, MTK_MDP_MAX_CTRL_NUM);
> +
> +	ctx->ctrls.rotate = v4l2_ctrl_new_std(&ctx->ctrl_handler,
> +			&mtk_mdp_ctrl_ops, V4L2_CID_ROTATE, 0, 270, 90, 0);
> +	ctx->ctrls.hflip = v4l2_ctrl_new_std(&ctx->ctrl_handler,
> +					     &mtk_mdp_ctrl_ops,
> +					     V4L2_CID_HFLIP,
> +					     0, 1, 1, 0);
> +	ctx->ctrls.vflip = v4l2_ctrl_new_std(&ctx->ctrl_handler,
> +					     &mtk_mdp_ctrl_ops,
> +					     V4L2_CID_VFLIP,
> +					     0, 1, 1, 0);
> +	ctx->ctrls.global_alpha = v4l2_ctrl_new_std(&ctx->ctrl_handler,
> +						    &mtk_mdp_ctrl_ops,
> +						    V4L2_CID_ALPHA_COMPONENT,
> +						    0, 255, 1, 0);
> +	ctx->ctrls_rdy = ctx->ctrl_handler.error == 0;
> +
> +	if (ctx->ctrl_handler.error) {
> +		int err = ctx->ctrl_handler.error;
> +
> +		v4l2_ctrl_handler_free(&ctx->ctrl_handler);
> +		dev_err(&ctx->mdp_dev->pdev->dev,
> +			"Failed to create control handlers\n");
> +		return err;
> +	}
> +
> +	return 0;
> +}
> +
> +static void mtk_mdp_set_default_params(struct mtk_mdp_ctx *ctx)
> +{
> +	struct mtk_mdp_dev *mdp = ctx->mdp_dev;
> +	struct mtk_mdp_frame *frame;
> +
> +	frame = mtk_mdp_ctx_get_frame(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
> +	frame->fmt = mtk_mdp_find_fmt_by_index(0,
> +					V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
> +	frame->width = mdp->variant->pix_min->org_w;
> +	frame->height = mdp->variant->pix_min->org_h;
> +	frame->payload[0] = frame->width * frame->height;
> +	frame->payload[1] = frame->payload[0] / 2;
> +
> +	frame = mtk_mdp_ctx_get_frame(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
> +	frame->fmt = mtk_mdp_find_fmt_by_index(0,
> +					V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
> +	frame->width = mdp->variant->pix_min->target_rot_dis_w;
> +	frame->height = mdp->variant->pix_min->target_rot_dis_h;
> +	frame->payload[0] = frame->width * frame->height;
> +	frame->payload[1] = frame->payload[0] / 2;
> +
> +}
> +
> +static int mtk_mdp_m2m_open(struct file *file)
> +{
> +	struct mtk_mdp_dev *mdp = video_drvdata(file);
> +	struct video_device *vfd = video_devdata(file);
> +	struct mtk_mdp_ctx *ctx = NULL;
> +	int ret;
> +
> +	if (mutex_lock_interruptible(&mdp->lock))
> +		return -ERESTARTSYS;

This lock is taken very early. Move it down to where you really need the lock,
it can certainly be moved to after the kzalloc.

> +
> +	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
> +	if (!ctx) {
> +		ret = -ENOMEM;
> +		goto err_ctx_alloc;
> +	}
> +
> +	mutex_init(&ctx->slock);
> +	ctx->id = mdp->id_counter++;
> +	v4l2_fh_init(&ctx->fh, vfd);
> +	file->private_data = &ctx->fh;
> +	ret = mtk_mdp_ctrls_create(ctx);
> +	if (ret)
> +		goto error_ctrls;
> +
> +	/* Use separate control handler per file handle */
> +	ctx->fh.ctrl_handler = &ctx->ctrl_handler;
> +	v4l2_fh_add(&ctx->fh);
> +	INIT_LIST_HEAD(&ctx->list);
> +
> +	ctx->mdp_dev = mdp;
> +	mtk_mdp_set_default_params(ctx);
> +
> +	INIT_WORK(&ctx->work, mtk_mdp_m2m_worker);
> +	ctx->m2m_ctx = v4l2_m2m_ctx_init(mdp->m2m_dev, ctx,
> +					 mtk_mdp_m2m_queue_init);
> +	if (IS_ERR(ctx->m2m_ctx)) {
> +		dev_err(&mdp->pdev->dev, "Failed to initialize m2m context");
> +		ret = PTR_ERR(ctx->m2m_ctx);
> +		goto error_m2m_ctx;
> +	}
> +	ctx->fh.m2m_ctx = ctx->m2m_ctx;
> +	if (mdp->ctx_num++ == 0) {
> +		ret = vpu_load_firmware(mdp->vpu_dev);
> +		if (ret < 0) {
> +			dev_err(&mdp->pdev->dev,
> +				"vpu_load_firmware failed %d\n", ret);
> +			goto err_load_vpu;
> +		}
> +
> +		ret = mtk_mdp_vpu_register(mdp->pdev);
> +		if (ret < 0) {
> +			dev_err(&mdp->pdev->dev,
> +				"mdp_vpu register failed %d\n", ret);
> +			goto err_load_vpu;
> +		}
> +	}
> +
> +	list_add(&ctx->list, &mdp->ctx_list);
> +	mutex_unlock(&mdp->lock);
> +
> +	mtk_mdp_dbg(0, "%s [%d]", dev_name(&mdp->pdev->dev), ctx->id);
> +
> +	return 0;
> +
> +err_load_vpu:
> +	mdp->ctx_num--;
> +	v4l2_m2m_ctx_release(ctx->m2m_ctx);
> +error_m2m_ctx:
> +	v4l2_ctrl_handler_free(&ctx->ctrl_handler);
> +error_ctrls:
> +	v4l2_fh_del(&ctx->fh);
> +	v4l2_fh_exit(&ctx->fh);
> +err_ctx_alloc:
> +	kfree(ctx);
> +	mutex_unlock(&mdp->lock);
> +
> +	return ret;
> +}

<snip>

[toc] | [prev] | [next] | [standalone]

#1466040 — Re: [PATCH v3 3/4] media: Add Mediatek MDP Driver

FromHans Verkuil <hverkuil@xs4all.nl>
Date2016-08-19 08:10 +0200
SubjectRe: [PATCH v3 3/4] media: Add Mediatek MDP Driver
Message-ID<s7Laa-1TE-13@gated-at.bofh.it>
In reply to#1462760
On 08/19/2016 07:54 AM, Minghsiu Tsai wrote:
> On Mon, 2016-08-15 at 14:55 +0200, Hans Verkuil wrote:
>> On 08/09/2016 03:58 PM, Minghsiu Tsai wrote:
>> This isn't right. For VIDEO_CAPTURE you support the COMPOSE targets, and for
>> the VIDEO_OUTPUT you support the CROP targets. Right now I can use e.g. TGT_CROP
>> with VIDEO_CAPTURE, which isn't correct.
>>
>> s_selection has the same problem.
>>
> 
> So my understanding is
> VIDEO_OUTPUT -> only allow to use target XXX_CROP_XXX
> VIDEO_CAPTURE -> only allow to use target XXX_COMPOSE_XXX
> 
> Am I right?

Correct.

Regards,

	Hans

[toc] | [prev] | [next] | [standalone]

#1466041 — Re: [PATCH v3 3/4] media: Add Mediatek MDP Driver

FromMinghsiu Tsai <minghsiu.tsai@mediatek.com>
Date2016-08-19 08:10 +0200
SubjectRe: [PATCH v3 3/4] media: Add Mediatek MDP Driver
Message-ID<s7Laa-1TE-15@gated-at.bofh.it>
In reply to#1462760
On Mon, 2016-08-15 at 14:55 +0200, Hans Verkuil wrote:
> On 08/09/2016 03:58 PM, Minghsiu Tsai wrote:
> > Add MDP driver for MT8173
> > 
> > Signed-off-by: Minghsiu Tsai <minghsiu.tsai@mediatek.com>
> > ---
> >  drivers/media/platform/Kconfig                |   16 +
> >  drivers/media/platform/Makefile               |    2 +
> >  drivers/media/platform/mtk-mdp/Makefile       |    9 +
> >  drivers/media/platform/mtk-mdp/mtk_mdp_comp.c |  159 ++++
> >  drivers/media/platform/mtk-mdp/mtk_mdp_comp.h |   72 ++
> >  drivers/media/platform/mtk-mdp/mtk_mdp_core.c |  294 ++++++
> >  drivers/media/platform/mtk-mdp/mtk_mdp_core.h |  240 +++++
> >  drivers/media/platform/mtk-mdp/mtk_mdp_ipi.h  |  126 +++
> >  drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c  | 1263 +++++++++++++++++++++++++
> >  drivers/media/platform/mtk-mdp/mtk_mdp_m2m.h  |   22 +
> >  drivers/media/platform/mtk-mdp/mtk_mdp_regs.c |  153 +++
> >  drivers/media/platform/mtk-mdp/mtk_mdp_regs.h |   31 +
> >  drivers/media/platform/mtk-mdp/mtk_mdp_vpu.c  |  145 +++
> >  drivers/media/platform/mtk-mdp/mtk_mdp_vpu.h  |   41 +
> >  14 files changed, 2573 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-mdp/Makefile
> >  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_comp.c
> >  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_comp.h
> >  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_core.c
> >  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_core.h
> >  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_ipi.h
> >  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c
> >  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_m2m.h
> >  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_regs.c
> >  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_regs.h
> >  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_vpu.c
> >  create mode 100644 drivers/media/platform/mtk-mdp/mtk_mdp_vpu.h
> > 
> > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > index f25344b..4bb874b 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -166,6 +166,22 @@ config VIDEO_MEDIATEK_VPU
> >  	    To compile this driver as a module, choose M here: the
> >  	    module will be called mtk-vpu.
> >  
> > +config VIDEO_MEDIATEK_MDP
> > +	tristate "Mediatek MDP driver"
> > +	depends on MTK_IOMMU || COMPILE_TEST
> > +	depends on VIDEO_DEV && VIDEO_V4L2
> > +	depends on ARCH_MEDIATEK || COMPILE_TEST
> 
> This needs a 'depends on HAS_DMA' as well.
> 

Will add it.

> > +	select VIDEOBUF2_DMA_CONTIG
> > +	select V4L2_MEM2MEM_DEV
> > +	select VIDEO_MEDIATEK_VPU
> > +	default n
> > +	---help---
> > +	    It is a v4l2 driver and present in Mediatek MT8173 SoCs.
> > +	    The driver supports for scaling and color space conversion.
> > +
> > +	    To compile this driver as a module, choose M here: the
> > +	    module will be called mtk-mdp.
> > +
> >  config VIDEO_MEDIATEK_VCODEC
> >  	tristate "Mediatek Video Codec driver"
> >  	depends on MTK_IOMMU || COMPILE_TEST
> 
> <snip>
> 
> > +
> > +/*
> > + * Return true if rectangle a is enclosed in rectangle b, or false otherwise.
> > + */
> > +static bool mtk_mdp_m2m_is_rectangle_enclosed(struct v4l2_rect *a,
> > +					     struct v4l2_rect *b)
> > +{
> > +	if (a->left < b->left || a->top < b->top)
> > +		return false;
> > +
> > +	if (a->left + a->width > b->left + b->width)
> > +		return false;
> > +
> > +	if (a->top + a->height > b->top + b->height)
> > +		return false;
> > +
> > +	return true;
> > +}
> > +
> > +static int mtk_mdp_m2m_g_selection(struct file *file, void *fh,
> > +				   struct v4l2_selection *s)
> > +{
> > +	struct mtk_mdp_frame *frame;
> > +	struct mtk_mdp_ctx *ctx = fh_to_ctx(fh);
> > +
> > +	if ((s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE) &&
> > +	    (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)) {
> > +		mtk_mdp_dbg(1, "[%d] invalid type:%d", ctx->id, s->type);
> > +		return -EINVAL;
> > +	}
> > +
> > +	frame = mtk_mdp_ctx_get_frame(ctx, s->type);
> > +
> > +	switch (s->target) {
> > +	case V4L2_SEL_TGT_COMPOSE_DEFAULT:
> > +	case V4L2_SEL_TGT_COMPOSE_BOUNDS:
> > +	case V4L2_SEL_TGT_CROP_BOUNDS:
> > +	case V4L2_SEL_TGT_CROP_DEFAULT:
> > +		s->r.left = 0;
> > +		s->r.top = 0;
> > +		s->r.width = frame->width;
> > +		s->r.height = frame->height;
> > +		return 0;
> > +
> > +	case V4L2_SEL_TGT_COMPOSE:
> > +	case V4L2_SEL_TGT_CROP:
> > +		s->r.left = frame->crop.left;
> > +		s->r.top = frame->crop.top;
> > +		s->r.width = frame->crop.width;
> > +		s->r.height = frame->crop.height;
> > +		return 0;
> > +	}
> 
> This isn't right. For VIDEO_CAPTURE you support the COMPOSE targets, and for
> the VIDEO_OUTPUT you support the CROP targets. Right now I can use e.g. TGT_CROP
> with VIDEO_CAPTURE, which isn't correct.
> 
> s_selection has the same problem.
> 

So my understanding is
VIDEO_OUTPUT -> only allow to use target XXX_CROP_XXX
VIDEO_CAPTURE -> only allow to use target XXX_COMPOSE_XXX

Am I right?


> > +
> > +	return -EINVAL;
> > +}
> > +
> > +static int mtk_mdp_check_scaler_ratio(struct mtk_mdp_variant *var, int src_w,
> > +				      int src_h, int dst_w, int dst_h, int rot)
> > +{
> > +	int tmp_w, tmp_h;
> > +
> > +	if (rot == 90 || rot == 270) {
> > +		tmp_w = dst_h;
> > +		tmp_h = dst_w;
> > +	} else {
> > +		tmp_w = dst_w;
> > +		tmp_h = dst_h;
> > +	}
> > +
> > +	if ((src_w / tmp_w) > var->h_scale_down_max ||
> > +	    (src_h / tmp_h) > var->v_scale_down_max ||
> > +	    (tmp_w / src_w) > var->h_scale_up_max ||
> > +	    (tmp_h / src_h) > var->v_scale_up_max)
> > +		return -EINVAL;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_mdp_m2m_s_selection(struct file *file, void *fh,
> > +				   struct v4l2_selection *s)
> > +{
> > +	struct mtk_mdp_frame *frame;
> > +	struct mtk_mdp_ctx *ctx = fh_to_ctx(fh);
> > +	struct v4l2_crop cr;
> > +	struct mtk_mdp_variant *variant = ctx->mdp_dev->variant;
> > +	int ret;
> > +
> > +	cr.type = s->type;
> > +	cr.c = s->r;
> > +
> > +	if ((s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE) &&
> > +	    (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)) {
> > +		mtk_mdp_dbg(1, "[%d] invalid type:%d", ctx->id, s->type);
> > +		return -EINVAL;
> > +	}
> > +
> > +	ret = mtk_mdp_try_crop(ctx, &cr);
> 
> Please don't copy to a v4l2_crop. Just stick to v4l2_selection.
> I would prefer not to see struct v4l2_crop in drivers.
> 

I will replace v4l2_crop with u32 and struct v4l2_rect in next version.


> > +	if (ret)
> > +		return ret;
> > +
> > +	if (s->flags & V4L2_SEL_FLAG_LE &&
> > +	    !mtk_mdp_m2m_is_rectangle_enclosed(&cr.c, &s->r))
> > +		return -ERANGE;
> > +
> > +	if (s->flags & V4L2_SEL_FLAG_GE &&
> > +	    !mtk_mdp_m2m_is_rectangle_enclosed(&s->r, &cr.c))
> > +		return -ERANGE;
> 
> As has been discussed for the encoder driver: just adjust the rectangle and
> don't return ERANGE. Consider the flags as hints.

Already adjust rectangle in mtk_mdp_try_crop() before that, so I will
remove the two condition check in next version.



> The V4L2 documentation is confusing, this will be addressed soon.
> 
> > +
> > +	s->r = cr.c;
> > +
> > +	switch (s->target) {
> > +	case V4L2_SEL_TGT_COMPOSE_BOUNDS:
> > +	case V4L2_SEL_TGT_COMPOSE_DEFAULT:
> > +	case V4L2_SEL_TGT_COMPOSE:
> > +		frame = &ctx->s_frame;
> > +		break;
> > +
> > +	case V4L2_SEL_TGT_CROP_BOUNDS:
> > +	case V4L2_SEL_TGT_CROP:
> > +	case V4L2_SEL_TGT_CROP_DEFAULT:
> > +		frame = &ctx->d_frame;
> > +		break;
> > +
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +
> > +	/* Check to see if scaling ratio is within supported range */
> > +	if (mtk_mdp_ctx_state_is_set(ctx, MTK_MDP_DST_FMT | MTK_MDP_SRC_FMT)) {
> > +		if (V4L2_TYPE_IS_OUTPUT(s->type)) {
> > +			ret = mtk_mdp_check_scaler_ratio(variant, cr.c.width,
> > +				cr.c.height, ctx->d_frame.crop.width,
> > +				ctx->d_frame.crop.height,
> > +				ctx->ctrls.rotate->val);
> > +		} else {
> > +			ret = mtk_mdp_check_scaler_ratio(variant,
> > +				ctx->s_frame.crop.width,
> > +				ctx->s_frame.crop.height, cr.c.width,
> > +				cr.c.height, ctx->ctrls.rotate->val);
> > +		}
> > +
> > +		if (ret) {
> > +			dev_info(&ctx->mdp_dev->pdev->dev,
> > +				"Out of scaler range");
> > +			return -EINVAL;
> > +		}
> > +	}
> > +
> > +	frame->crop = cr.c;
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct v4l2_ioctl_ops mtk_mdp_m2m_ioctl_ops = {
> > +	.vidioc_querycap		= mtk_mdp_m2m_querycap,
> > +	.vidioc_enum_fmt_vid_cap_mplane	= mtk_mdp_m2m_enum_fmt_mplane_vid_cap,
> > +	.vidioc_enum_fmt_vid_out_mplane	= mtk_mdp_m2m_enum_fmt_mplane_vid_out,
> > +	.vidioc_g_fmt_vid_cap_mplane	= mtk_mdp_m2m_g_fmt_mplane,
> > +	.vidioc_g_fmt_vid_out_mplane	= mtk_mdp_m2m_g_fmt_mplane,
> > +	.vidioc_try_fmt_vid_cap_mplane	= mtk_mdp_m2m_try_fmt_mplane,
> > +	.vidioc_try_fmt_vid_out_mplane	= mtk_mdp_m2m_try_fmt_mplane,
> > +	.vidioc_s_fmt_vid_cap_mplane	= mtk_mdp_m2m_s_fmt_mplane,
> > +	.vidioc_s_fmt_vid_out_mplane	= mtk_mdp_m2m_s_fmt_mplane,
> > +	.vidioc_reqbufs			= mtk_mdp_m2m_reqbufs,
> > +	.vidioc_create_bufs		= v4l2_m2m_ioctl_create_bufs,
> > +	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
> > +	.vidioc_subscribe_event		= v4l2_ctrl_subscribe_event,
> > +	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
> > +	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
> > +	.vidioc_qbuf			= v4l2_m2m_ioctl_qbuf,
> > +	.vidioc_dqbuf			= v4l2_m2m_ioctl_dqbuf,
> > +	.vidioc_streamon		= mtk_mdp_m2m_streamon,
> > +	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
> > +	.vidioc_g_selection		= mtk_mdp_m2m_g_selection,
> > +	.vidioc_s_selection		= mtk_mdp_m2m_s_selection
> > +};
> > +
> > +static int mtk_mdp_m2m_queue_init(void *priv, struct vb2_queue *src_vq,
> > +				  struct vb2_queue *dst_vq)
> > +{
> > +	struct mtk_mdp_ctx *ctx = priv;
> > +	int ret;
> > +
> > +	memset(src_vq, 0, sizeof(*src_vq));
> > +	src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> > +	src_vq->io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF;
> > +	src_vq->drv_priv = ctx;
> > +	src_vq->ops = &mtk_mdp_m2m_qops;
> > +	src_vq->mem_ops = &vb2_dma_contig_memops;
> > +	src_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
> > +	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> > +	src_vq->dev = &ctx->mdp_dev->pdev->dev;
> > +
> > +	ret = vb2_queue_init(src_vq);
> > +	if (ret)
> > +		return ret;
> > +
> > +	memset(dst_vq, 0, sizeof(*dst_vq));
> > +	dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> > +	dst_vq->io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF;
> 
> I wouldn't recommend setting VB2_USERPTR in combination with dma_contig.
> 
> That only works if the userptr points to physically contiguous memory,
> unless there is an iommu that can fix things up.
> 

I will remove VB2_USERPTR.Thanks.

> > +	dst_vq->drv_priv = ctx;
> > +	dst_vq->ops = &mtk_mdp_m2m_qops;
> > +	dst_vq->mem_ops = &vb2_dma_contig_memops;
> > +	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
> > +	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> > +	dst_vq->dev = &ctx->mdp_dev->pdev->dev;
> > +
> > +	return vb2_queue_init(dst_vq);
> > +}
> > +
> > +static int mtk_mdp_s_ctrl(struct v4l2_ctrl *ctrl)
> > +{
> > +	struct mtk_mdp_ctx *ctx = ctrl_to_ctx(ctrl);
> > +	struct mtk_mdp_dev *mdp = ctx->mdp_dev;
> > +	struct mtk_mdp_variant *variant = mdp->variant;
> > +	u32 state = MTK_MDP_DST_FMT | MTK_MDP_SRC_FMT;
> > +	int ret = 0;
> > +
> > +	if (ctrl->flags & V4L2_CTRL_FLAG_INACTIVE)
> > +		return 0;
> > +
> > +	switch (ctrl->id) {
> > +	case V4L2_CID_HFLIP:
> > +		ctx->hflip = ctrl->val;
> > +		break;
> > +	case V4L2_CID_VFLIP:
> > +		ctx->vflip = ctrl->val;
> > +		break;
> > +	case V4L2_CID_ROTATE:
> > +		if (mtk_mdp_ctx_state_is_set(ctx, state)) {
> > +			ret = mtk_mdp_check_scaler_ratio(variant,
> > +					ctx->s_frame.crop.width,
> > +					ctx->s_frame.crop.height,
> > +					ctx->d_frame.crop.width,
> > +					ctx->d_frame.crop.height,
> > +					ctx->ctrls.rotate->val);
> > +
> > +			if (ret)
> > +				return -EINVAL;
> > +		}
> > +
> > +		ctx->rotation = ctrl->val;
> > +		break;
> > +	case V4L2_CID_ALPHA_COMPONENT:
> > +		ctx->d_frame.alpha = ctrl->val;
> > +		break;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct v4l2_ctrl_ops mtk_mdp_ctrl_ops = {
> > +	.s_ctrl = mtk_mdp_s_ctrl,
> > +};
> > +
> > +static int mtk_mdp_ctrls_create(struct mtk_mdp_ctx *ctx)
> > +{
> > +	v4l2_ctrl_handler_init(&ctx->ctrl_handler, MTK_MDP_MAX_CTRL_NUM);
> > +
> > +	ctx->ctrls.rotate = v4l2_ctrl_new_std(&ctx->ctrl_handler,
> > +			&mtk_mdp_ctrl_ops, V4L2_CID_ROTATE, 0, 270, 90, 0);
> > +	ctx->ctrls.hflip = v4l2_ctrl_new_std(&ctx->ctrl_handler,
> > +					     &mtk_mdp_ctrl_ops,
> > +					     V4L2_CID_HFLIP,
> > +					     0, 1, 1, 0);
> > +	ctx->ctrls.vflip = v4l2_ctrl_new_std(&ctx->ctrl_handler,
> > +					     &mtk_mdp_ctrl_ops,
> > +					     V4L2_CID_VFLIP,
> > +					     0, 1, 1, 0);
> > +	ctx->ctrls.global_alpha = v4l2_ctrl_new_std(&ctx->ctrl_handler,
> > +						    &mtk_mdp_ctrl_ops,
> > +						    V4L2_CID_ALPHA_COMPONENT,
> > +						    0, 255, 1, 0);
> > +	ctx->ctrls_rdy = ctx->ctrl_handler.error == 0;
> > +
> > +	if (ctx->ctrl_handler.error) {
> > +		int err = ctx->ctrl_handler.error;
> > +
> > +		v4l2_ctrl_handler_free(&ctx->ctrl_handler);
> > +		dev_err(&ctx->mdp_dev->pdev->dev,
> > +			"Failed to create control handlers\n");
> > +		return err;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_mdp_set_default_params(struct mtk_mdp_ctx *ctx)
> > +{
> > +	struct mtk_mdp_dev *mdp = ctx->mdp_dev;
> > +	struct mtk_mdp_frame *frame;
> > +
> > +	frame = mtk_mdp_ctx_get_frame(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
> > +	frame->fmt = mtk_mdp_find_fmt_by_index(0,
> > +					V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
> > +	frame->width = mdp->variant->pix_min->org_w;
> > +	frame->height = mdp->variant->pix_min->org_h;
> > +	frame->payload[0] = frame->width * frame->height;
> > +	frame->payload[1] = frame->payload[0] / 2;
> > +
> > +	frame = mtk_mdp_ctx_get_frame(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
> > +	frame->fmt = mtk_mdp_find_fmt_by_index(0,
> > +					V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
> > +	frame->width = mdp->variant->pix_min->target_rot_dis_w;
> > +	frame->height = mdp->variant->pix_min->target_rot_dis_h;
> > +	frame->payload[0] = frame->width * frame->height;
> > +	frame->payload[1] = frame->payload[0] / 2;
> > +
> > +}
> > +
> > +static int mtk_mdp_m2m_open(struct file *file)
> > +{
> > +	struct mtk_mdp_dev *mdp = video_drvdata(file);
> > +	struct video_device *vfd = video_devdata(file);
> > +	struct mtk_mdp_ctx *ctx = NULL;
> > +	int ret;
> > +
> > +	if (mutex_lock_interruptible(&mdp->lock))
> > +		return -ERESTARTSYS;
> 
> This lock is taken very early. Move it down to where you really need the lock,
> it can certainly be moved to after the kzalloc.
> 

I will move mutex_loc after kzalloc. Thanks for your suggestions.

> > +
> > +	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
> > +	if (!ctx) {
> > +		ret = -ENOMEM;
> > +		goto err_ctx_alloc;
> > +	}
> > +
> > +	mutex_init(&ctx->slock);
> > +	ctx->id = mdp->id_counter++;
> > +	v4l2_fh_init(&ctx->fh, vfd);
> > +	file->private_data = &ctx->fh;
> > +	ret = mtk_mdp_ctrls_create(ctx);
> > +	if (ret)
> > +		goto error_ctrls;
> > +
> > +	/* Use separate control handler per file handle */
> > +	ctx->fh.ctrl_handler = &ctx->ctrl_handler;
> > +	v4l2_fh_add(&ctx->fh);
> > +	INIT_LIST_HEAD(&ctx->list);
> > +
> > +	ctx->mdp_dev = mdp;
> > +	mtk_mdp_set_default_params(ctx);
> > +
> > +	INIT_WORK(&ctx->work, mtk_mdp_m2m_worker);
> > +	ctx->m2m_ctx = v4l2_m2m_ctx_init(mdp->m2m_dev, ctx,
> > +					 mtk_mdp_m2m_queue_init);
> > +	if (IS_ERR(ctx->m2m_ctx)) {
> > +		dev_err(&mdp->pdev->dev, "Failed to initialize m2m context");
> > +		ret = PTR_ERR(ctx->m2m_ctx);
> > +		goto error_m2m_ctx;
> > +	}
> > +	ctx->fh.m2m_ctx = ctx->m2m_ctx;
> > +	if (mdp->ctx_num++ == 0) {
> > +		ret = vpu_load_firmware(mdp->vpu_dev);
> > +		if (ret < 0) {
> > +			dev_err(&mdp->pdev->dev,
> > +				"vpu_load_firmware failed %d\n", ret);
> > +			goto err_load_vpu;
> > +		}
> > +
> > +		ret = mtk_mdp_vpu_register(mdp->pdev);
> > +		if (ret < 0) {
> > +			dev_err(&mdp->pdev->dev,
> > +				"mdp_vpu register failed %d\n", ret);
> > +			goto err_load_vpu;
> > +		}
> > +	}
> > +
> > +	list_add(&ctx->list, &mdp->ctx_list);
> > +	mutex_unlock(&mdp->lock);
> > +
> > +	mtk_mdp_dbg(0, "%s [%d]", dev_name(&mdp->pdev->dev), ctx->id);
> > +
> > +	return 0;
> > +
> > +err_load_vpu:
> > +	mdp->ctx_num--;
> > +	v4l2_m2m_ctx_release(ctx->m2m_ctx);
> > +error_m2m_ctx:
> > +	v4l2_ctrl_handler_free(&ctx->ctrl_handler);
> > +error_ctrls:
> > +	v4l2_fh_del(&ctx->fh);
> > +	v4l2_fh_exit(&ctx->fh);
> > +err_ctx_alloc:
> > +	kfree(ctx);
> > +	mutex_unlock(&mdp->lock);
> > +
> > +	return ret;
> > +}
> 
> <snip>

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web