Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1384682 > unrolled thread
| Started by | Tiffany Lin <tiffany.lin@mediatek.com> |
|---|---|
| First post | 2016-04-22 06:30 +0200 |
| Last post | 2016-04-25 11:30 +0200 |
| Articles | 14 — 5 participants |
Back to article view | Back to linux.kernel
[PATCH v7 0/8] Add MT8173 Video Encoder Driver and VPU Driver Tiffany Lin <tiffany.lin@mediatek.com> - 2016-04-22 06:30 +0200
[PATCH v7 8/8] arm64: dts: mediatek: Add Video Encoder for MT8173 Tiffany Lin <tiffany.lin@mediatek.com> - 2016-04-22 06:30 +0200
[PATCH v7 3/8] arm64: dts: mediatek: Add node for Mediatek Video Processor Unit Tiffany Lin <tiffany.lin@mediatek.com> - 2016-04-22 06:30 +0200
[PATCH v7 4/8] dt-bindings: Add a binding for Mediatek Video Encoder Tiffany Lin <tiffany.lin@mediatek.com> - 2016-04-22 06:30 +0200
[PATCH v7 7/8] [media] vcodec: mediatek: Add Mediatek H264 Video Encoder Driver Tiffany Lin <tiffany.lin@mediatek.com> - 2016-04-22 06:30 +0200
[PATCH v7 6/8] [media] vcodec: mediatek: Add Mediatek VP8 Video Encoder Driver Tiffany Lin <tiffany.lin@mediatek.com> - 2016-04-22 06:30 +0200
Re: [PATCH v7 5/8] [media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver Hans Verkuil <hverkuil@xs4all.nl> - 2016-04-22 15:50 +0200
Re: [PATCH v7 5/8] [media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver tiffany lin <tiffany.lin@mediatek.com> - 2016-04-25 07:20 +0200
Re: [PATCH v7 5/8] [media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver Wu-Cheng Li (李務誠) <wuchengli@chromium.org> - 2016-04-25 07:50 +0200
Re: [PATCH v7 5/8] [media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver tiffany lin <tiffany.lin@mediatek.com> - 2016-04-25 09:10 +0200
[PATCH v7 1/8] dt-bindings: Add a binding for Mediatek Video Processor Tiffany Lin <tiffany.lin@mediatek.com> - 2016-04-22 06:30 +0200
[PATCH v7 2/8] [media] VPU: mediatek: support Mediatek VPU Tiffany Lin <tiffany.lin@mediatek.com> - 2016-04-22 06:30 +0200
Re: [PATCH v7 2/8] [media] VPU: mediatek: support Mediatek VPU Hans Verkuil <hverkuil@xs4all.nl> - 2016-04-25 09:50 +0200
Re: [PATCH v7 2/8] [media] VPU: mediatek: support Mediatek VPU andrew-ct chen <andrew-ct.chen@mediatek.com> - 2016-04-25 11:30 +0200
| From | Tiffany Lin <tiffany.lin@mediatek.com> |
|---|---|
| Date | 2016-04-22 06:30 +0200 |
| Subject | [PATCH v7 0/8] Add MT8173 Video Encoder Driver and VPU Driver |
| Message-ID | <rqAT7-4e7-1@gated-at.bofh.it> |
==============
Introduction
==============
The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs.
Mediatek Video Codec is able to handle video encoding of in a range of formats.
This patch series also include VPU driver. Mediatek Video Codec driver rely on VPU driver to load,
communicate with VPU.
Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI both have been merged in v4.6-rc1.
==================
Device interface
==================
In principle the driver bases on v4l2 memory-to-memory framework:
it provides a single video node and each opened file handle gets its own private context with separate
buffer queues. Each context consist of 2 buffer queues: OUTPUT (for source buffers, i.e. raw video
frames) and CAPTURE (for destination buffers, i.e. encoded video frames).
==============================
VPU (Video Processor Unit)
==============================
The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
It is able to handle video decoding/encoding in a range of formats.
The driver provides with VPU firmware download, memory management and the communication interface between CPU and VPU.
For VPU initialization, it will create virtual memory for CPU access and physical address for VPU hw device access.
When a decode/encode instance opens a device node, vpu driver will download vpu firmware to the device.
A decode/encode instant will decode/encode a frame using VPU interface to interrupt vpu to handle decoding/encoding jobs.
Please have a look at the code and comments will be very much appreciated.
Change in v7:
1. Rebase against the master branch of git://linuxtv.org/media_tree.git
2. Add ycbcr_enc, quantization and xfer_func in try_fmt, g_fmt, s_fmt
3. Merge h264_enc and vp8_enc to one venc directory
Change in v6:
1. Add synchronization access protect between irq handler and work thread
2. Add DMA_ATTR_ALLOC_SINGLE_PAGES support
3. S_FMT will return coded_width, coded_height, so user space could allocate correct size memory that HW required
4. merge h264/vp8 enc ap and md32 ipi msg
5. separate h264/vp8 enc gop_size and intra_period handle
6. remove sizeimage relative code in work buffer function
7. Refine makefile to build as an module
8. Code clean up
VPU Part
1. export symbols for building VPU as an module
2. change function from "wait_event_interruptible_timeout" to "wait_event_timeout" since
CPU needs to wait for ACK from VPU even if it was interrupted by a signal
v4l2-compliance test output:
localhost Encode # ./v4l2-compliance -d /dev/video1
Driver Info:
Driver name : mtk-vcodec-enc
Card type : platform:mt8173
Bus info : platform:mt8173
Driver version: 4.4.0
Capabilities : 0x84204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Compliance test for device /dev/video1 (not using libv4l2):
Required ioctls:
test VIDIOC_QUERYCAP: OK
Allow for multiple opens:
test second video open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERYCTRL/MENU: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 12 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test VIDIOC_EXPBUF: OK
Total: 38, Succeeded: 38, Failed: 0, Warnings: 0
v4l2-compliance test output:
localhost ~ # /usr/bin/v4l2-compliance -d /dev/video1
Change in v5:
Vcodec Part
1. Pass checkpatch and v4l2-compliance test
2. Remove unused g/s_selection for now
3. add vidioc_g_parm support
4. add vidioc_create_bufs and vidioc_prepare_buf support
5. Remove instance check in fops_vcodec_open
6. Fix comments for data structure and code
7. Refine venc, venc_lt clock source name in devicetree and binding document
8. Fix Author information and copyright information
9. Refine code according to review comments
VPU Part
v4l2-compliance test output:
localhost ~ # /usr/bin/v4l2-compliance -d /dev/video1
Driver Info:
Driver name : mtk-vcodec-enc
Card type : platform:mt8173
Bus info : platform:mt8173
Driver version: 4.4.0
Capabilities : 0x84204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Compliance test for device /dev/video1 (not using libv4l2):
Required ioctls:
test VIDIOC_QUERYCAP: OK
Allow for multiple opens:
test second video open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERYCTRL/MENU: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 12 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test VIDIOC_EXPBUF: OK
Total: 38, Succeeded: 38, Failed: 0, Warnings: 0
Change in v4:
Vcodec Part
1. Remove MTK_ENCODE_PARAM_SKIP_FRAME support
2. Remove MTK_ENCODE_PARAM_FRAME_TYPE and change to use V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME[3]
3. Refine Encoder HW clock source
4. Refine debug log
5. Add watchdog support
6. With patch "media: v4l2-compat-ioctl32: fix missing length copy in put_v4l2_buffer32"[4],
v4l2-compliance test passed[5] in v4.4-rc5
VPU Part
1. These two patches were Acked-by: Rob Herring <robh <at> kernel.org> in v3
[PATCH v3 1/8] dt-bindings: Add a binding for Mediatek Video Processor
[PATCH v3 3/8] arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
Because we were wrong about how the hardware works, there is no connection between VPU and IOMMU HW
We remove VPU attaching to IOMMU
2. Support VPU running on 4GB DRAM system
3. Support VPU watchdog reset
4. Refine for coding style
[3]https://patchwork.linuxtv.org/patch/32670/
[4] https://patchwork.linuxtv.org/patch/32631/
[5]localhost ~ # /usr/bin/v4l2-compliance -d /dev/video1
Driver Info:
Driver name : mtk-vcodec-en
Card type : platform:mt817
Bus info : platform:mt817
Driver version: 4.4.0
Capabilities : 0x84204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Compliance test for device /dev/video1 (not using libv4l2):
Required ioctls:
test VIDIOC_QUERYCAP: OK
Allow for multiple opens:
test second video open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERYCTRL/MENU: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 12 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls:
warn: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(475):
VIDIOC_CREATE_BUFS not supported
warn: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(475):
VIDIOC_CREATE_BUFS not supported
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test VIDIOC_EXPBUF: OK
Total: 38, Succeeded: 38, Failed: 0, Warnings: 2
Change in v3:
1.Refine code to pass v4l2-compliance test, now it still has 2 issues 2.Refine code according to latest MTK
IOMMU patches[1] 3.Remove MFC51 specific CIDs and add MTK specific CIDs for for keyframe and
skip I-frame
4.Refine code according to review comments
Below is the v1.6 version v4l2-compliance report for the mt8173 encoder driver.
Now there are still 2 test fail in v1.6.
For VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF, we directly use v4l2_m2m_ioctl_* functions, but it still
fail. It pass in kernel 3.18 but fail in kernel 4.4.
We will try v1.8 in next version.
VIDIOC_EXPBUF is becuase we support all three memory types VB2_DMABUF, VB2_MMAP and VB2_USERPTR.
VIDIOC_EXPBUF only allowed when only VB2_MMAP supported.
localhost ~ # /usr/bin/v4l2-compliance -d /dev/video1 Driver Info:
Driver name : mtk-vcodec-en
Card type : platform:mt817
Bus info : platform:mt817
Driver version: 4.4.0
Capabilities : 0x84204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Compliance test for device /dev/video1 (not using libv4l2):
Required ioctls:
test VIDIOC_QUERYCAP: OK
Allow for multiple opens:
test second video open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERYCTRL/MENU: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 11 Private Controls: 2
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls:
fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(266): vp->length
== 0
fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(335):
buf.check(Unqueued, i)
fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(420):
testQueryBuf(node, i, q.g_buffers())
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
fail: ../../../v4l-utils-1.6.0/utils/v4l2-compliance/v4l2-test-buffers.cpp(500): q.has_expbuf(node)
test VIDIOC_EXPBUF: FAIL
Total: 38, Succeeded: 36, Failed: 2, Warnings: 0
Change in v2:
Vcodec Part
1.Remove common and include directory in mtk-vcodec 2.Refine vb2ops_venc_start_streaming and
vb2ops_venc_stop_streaming and state machine 3.Remove venc_if_init and venc_if_deinit 4.Refine
debug message 5.Refine lab and vpu decription in mediatek-vcodec.txt
VPU Part
1. Modify VPU Kconfig
2. Move encoder header files to other patch sets 3. Remove marcos for extended virtual/iova address 4.
Change register and variable names 5. Add a reference counter for VPU watchdog 6. Remove one busy waiting
in function vpu_ipi_send 7. Operate VPU clock in VPU driver (not called by encoder drivers) 8. Refine
memory mapping, firmware download and extended memory allocation/free functions 9. Release more
allocated resources in driver remove function
Andrew-CT Chen (3):
dt-bindings: Add a binding for Mediatek Video Processor
[media] VPU: mediatek: support Mediatek VPU
arm64: dts: mediatek: Add node for Mediatek Video Processor Unit
Tiffany Lin (5):
dt-bindings: Add a binding for Mediatek Video Encoder
[media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver
[media] vcodec: mediatek: Add Mediatek VP8 Video Encoder Driver
[media] vcodec: mediatek: Add Mediatek H264 Video Encoder Driver
arm64: dts: mediatek: Add Video Encoder for MT8173
.../devicetree/bindings/media/mediatek-vcodec.txt | 59 +
.../devicetree/bindings/media/mediatek-vpu.txt | 31 +
arch/arm64/boot/dts/mediatek/mt8173.dtsi | 62 +
drivers/media/platform/Kconfig | 29 +
drivers/media/platform/Makefile | 4 +
drivers/media/platform/mtk-vcodec/Makefile | 19 +
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h | 339 +++++
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1301 ++++++++++++++++++++
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h | 59 +
.../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 467 +++++++
.../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 137 +++
.../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 26 +
.../media/platform/mtk-vcodec/mtk_vcodec_intr.c | 56 +
.../media/platform/mtk-vcodec/mtk_vcodec_intr.h | 27 +
.../media/platform/mtk-vcodec/mtk_vcodec_util.c | 96 ++
.../media/platform/mtk-vcodec/mtk_vcodec_util.h | 87 ++
.../media/platform/mtk-vcodec/venc/venc_h264_if.c | 687 +++++++++++
.../media/platform/mtk-vcodec/venc/venc_vp8_if.c | 488 ++++++++
drivers/media/platform/mtk-vcodec/venc_drv_base.h | 62 +
drivers/media/platform/mtk-vcodec/venc_drv_if.c | 114 ++
drivers/media/platform/mtk-vcodec/venc_drv_if.h | 165 +++
drivers/media/platform/mtk-vcodec/venc_ipi_msg.h | 210 ++++
drivers/media/platform/mtk-vcodec/venc_vpu_if.c | 240 ++++
drivers/media/platform/mtk-vcodec/venc_vpu_if.h | 61 +
drivers/media/platform/mtk-vpu/Makefile | 3 +
drivers/media/platform/mtk-vpu/mtk_vpu.c | 950 ++++++++++++++
drivers/media/platform/mtk-vpu/mtk_vpu.h | 162 +++
27 files changed, 5941 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
create mode 100644 drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_base.h
create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.h
create mode 100644 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
create mode 100644 drivers/media/platform/mtk-vcodec/venc_vpu_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/venc_vpu_if.h
create mode 100644 drivers/media/platform/mtk-vpu/Makefile
create mode 100755 drivers/media/platform/mtk-vpu/mtk_vpu.c
create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
--
1.7.9.5
[toc] | [next] | [standalone]
| From | Tiffany Lin <tiffany.lin@mediatek.com> |
|---|---|
| Date | 2016-04-22 06:30 +0200 |
| Subject | [PATCH v7 8/8] arm64: dts: mediatek: Add Video Encoder for MT8173 |
| Message-ID | <rqAT7-4e7-3@gated-at.bofh.it> |
| In reply to | #1384682 |
Add video encoder node for MT8173
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
arch/arm64/boot/dts/mediatek/mt8173.dtsi | 39 ++++++++++++++++++++++++++++++
1 file changed, 39 insertions(+)
diff --git a/arch/arm64/boot/dts/mediatek/mt8173.dtsi b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
index ae147bb..348ce0e 100644
--- a/arch/arm64/boot/dts/mediatek/mt8173.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
@@ -717,6 +717,45 @@
clock-names = "apb", "smi";
};
+ vcodec_enc: vcodec@18002000 {
+ compatible = "mediatek,mt8173-vcodec-enc";
+ reg = <0 0x18002000 0 0x1000>, /* VENC_SYS */
+ <0 0x19002000 0 0x1000>; /* VENC_LT_SYS */
+ interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
+ mediatek,larb = <&larb3>,
+ <&larb5>;
+ iommus = <&iommu M4U_PORT_VENC_RCPU>,
+ <&iommu M4U_PORT_VENC_REC>,
+ <&iommu M4U_PORT_VENC_BSDMA>,
+ <&iommu M4U_PORT_VENC_SV_COMV>,
+ <&iommu M4U_PORT_VENC_RD_COMV>,
+ <&iommu M4U_PORT_VENC_CUR_LUMA>,
+ <&iommu M4U_PORT_VENC_CUR_CHROMA>,
+ <&iommu M4U_PORT_VENC_REF_LUMA>,
+ <&iommu M4U_PORT_VENC_REF_CHROMA>,
+ <&iommu M4U_PORT_VENC_NBM_RDMA>,
+ <&iommu M4U_PORT_VENC_NBM_WDMA>,
+ <&iommu M4U_PORT_VENC_RCPU_SET2>,
+ <&iommu M4U_PORT_VENC_REC_FRM_SET2>,
+ <&iommu M4U_PORT_VENC_BSDMA_SET2>,
+ <&iommu M4U_PORT_VENC_SV_COMA_SET2>,
+ <&iommu M4U_PORT_VENC_RD_COMA_SET2>,
+ <&iommu M4U_PORT_VENC_CUR_LUMA_SET2>,
+ <&iommu M4U_PORT_VENC_CUR_CHROMA_SET2>,
+ <&iommu M4U_PORT_VENC_REF_LUMA_SET2>,
+ <&iommu M4U_PORT_VENC_REC_CHROMA_SET2>;
+ mediatek,vpu = <&vpu>;
+ clocks = <&topckgen CLK_TOP_VENCPLL_D2>,
+ <&topckgen CLK_TOP_VENC_SEL>,
+ <&topckgen CLK_TOP_UNIVPLL1_D2>,
+ <&topckgen CLK_TOP_VENC_LT_SEL>;
+ clock-names = "venc_sel_src",
+ "venc_sel",
+ "venc_lt_sel_src",
+ "venc_lt_sel";
+ };
+
vencltsys: clock-controller@19000000 {
compatible = "mediatek,mt8173-vencltsys", "syscon";
reg = <0 0x19000000 0 0x1000>;
--
1.7.9.5
[toc] | [prev] | [next] | [standalone]
| From | Tiffany Lin <tiffany.lin@mediatek.com> |
|---|---|
| Date | 2016-04-22 06:30 +0200 |
| Subject | [PATCH v7 3/8] arm64: dts: mediatek: Add node for Mediatek Video Processor Unit |
| Message-ID | <rqAT7-4e7-9@gated-at.bofh.it> |
| In reply to | #1384682 |
From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
Add VPU drivers for MT8173
Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
arch/arm64/boot/dts/mediatek/mt8173.dtsi | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/arch/arm64/boot/dts/mediatek/mt8173.dtsi b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
index eab7efc..ae147bb 100644
--- a/arch/arm64/boot/dts/mediatek/mt8173.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
@@ -125,6 +125,18 @@
clock-output-names = "cpum_ck";
};
+ reserved-memory {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+ vpu_dma_reserved: vpu_dma_mem_region {
+ compatible = "shared-dma-pool";
+ reg = <0 0xb7000000 0 0x500000>;
+ alignment = <0x1000>;
+ no-map;
+ };
+ };
+
timer {
compatible = "arm,armv8-timer";
interrupt-parent = <&gic>;
@@ -269,6 +281,17 @@
clock-names = "spi", "wrap";
};
+ vpu: vpu@10020000 {
+ compatible = "mediatek,mt8173-vpu";
+ reg = <0 0x10020000 0 0x30000>,
+ <0 0x10050000 0 0x100>;
+ reg-names = "tcm", "cfg_reg";
+ interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&topckgen CLK_TOP_SCP_SEL>;
+ clock-names = "main";
+ memory-region = <&vpu_dma_reserved>;
+ };
+
sysirq: intpol-controller@10200620 {
compatible = "mediatek,mt8173-sysirq",
"mediatek,mt6577-sysirq";
--
1.7.9.5
[toc] | [prev] | [next] | [standalone]
| From | Tiffany Lin <tiffany.lin@mediatek.com> |
|---|---|
| Date | 2016-04-22 06:30 +0200 |
| Subject | [PATCH v7 4/8] dt-bindings: Add a binding for Mediatek Video Encoder |
| Message-ID | <rqAT7-4e7-11@gated-at.bofh.it> |
| In reply to | #1384684 |
Add a DT binding documentation of Video Encoder for the
MT8173 SoC from Mediatek.
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
Acked-by: Rob Herring <robh@kernel.org>
---
.../devicetree/bindings/media/mediatek-vcodec.txt | 59 ++++++++++++++++++++
1 file changed, 59 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/mediatek-vcodec.txt
diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
new file mode 100644
index 0000000..59a47a5
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
@@ -0,0 +1,59 @@
+Mediatek Video Codec
+
+Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
+supports high resolution encoding functionalities.
+
+Required properties:
+- compatible : "mediatek,mt8173-vcodec-enc" for encoder
+- reg : Physical base address of the video codec registers and length of
+ memory mapped region.
+- interrupts : interrupt number to the cpu.
+- mediatek,larb : must contain the local arbiters in the current Socs.
+- clocks : list of clock specifiers, corresponding to entries in
+ the clock-names property.
+- clock-names: encoder must contain "venc_sel_src", "venc_sel",
+- "venc_lt_sel_src", "venc_lt_sel".
+- iommus : should point to the respective IOMMU block with master port as
+ argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+ for details.
+- mediatek,vpu : the node of video processor unit
+
+Example:
+vcodec_enc: vcodec@0x18002000 {
+ compatible = "mediatek,mt8173-vcodec-enc";
+ reg = <0 0x18002000 0 0x1000>, /*VENC_SYS*/
+ <0 0x19002000 0 0x1000>; /*VENC_LT_SYS*/
+ interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 202 IRQ_TYPE_LEVEL_LOW>;
+ mediatek,larb = <&larb3>,
+ <&larb5>;
+ iommus = <&iommu M4U_PORT_VENC_RCPU>,
+ <&iommu M4U_PORT_VENC_REC>,
+ <&iommu M4U_PORT_VENC_BSDMA>,
+ <&iommu M4U_PORT_VENC_SV_COMV>,
+ <&iommu M4U_PORT_VENC_RD_COMV>,
+ <&iommu M4U_PORT_VENC_CUR_LUMA>,
+ <&iommu M4U_PORT_VENC_CUR_CHROMA>,
+ <&iommu M4U_PORT_VENC_REF_LUMA>,
+ <&iommu M4U_PORT_VENC_REF_CHROMA>,
+ <&iommu M4U_PORT_VENC_NBM_RDMA>,
+ <&iommu M4U_PORT_VENC_NBM_WDMA>,
+ <&iommu M4U_PORT_VENC_RCPU_SET2>,
+ <&iommu M4U_PORT_VENC_REC_FRM_SET2>,
+ <&iommu M4U_PORT_VENC_BSDMA_SET2>,
+ <&iommu M4U_PORT_VENC_SV_COMA_SET2>,
+ <&iommu M4U_PORT_VENC_RD_COMA_SET2>,
+ <&iommu M4U_PORT_VENC_CUR_LUMA_SET2>,
+ <&iommu M4U_PORT_VENC_CUR_CHROMA_SET2>,
+ <&iommu M4U_PORT_VENC_REF_LUMA_SET2>,
+ <&iommu M4U_PORT_VENC_REC_CHROMA_SET2>;
+ mediatek,vpu = <&vpu>;
+ clocks = <&topckgen CLK_TOP_VENCPLL_D2>,
+ <&topckgen CLK_TOP_VENC_SEL>,
+ <&topckgen CLK_TOP_UNIVPLL1_D2>,
+ <&topckgen CLK_TOP_VENC_LT_SEL>;
+ clock-names = "venc_sel_src",
+ "venc_sel",
+ "venc_lt_sel_src",
+ "venc_lt_sel";
+ };
--
1.7.9.5
[toc] | [prev] | [next] | [standalone]
| From | Tiffany Lin <tiffany.lin@mediatek.com> |
|---|---|
| Date | 2016-04-22 06:30 +0200 |
| Subject | [PATCH v7 7/8] [media] vcodec: mediatek: Add Mediatek H264 Video Encoder Driver |
| Message-ID | <rqAT7-4e7-17@gated-at.bofh.it> |
| In reply to | #1384687 |
Add h264 encoder driver for MT8173
Signed-off-by: PoChun Lin <pochun.lin@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
drivers/media/platform/mtk-vcodec/Makefile | 1 +
.../media/platform/mtk-vcodec/venc/venc_h264_if.c | 687 ++++++++++++++++++++
drivers/media/platform/mtk-vcodec/venc_drv_if.c | 4 +-
3 files changed, 691 insertions(+), 1 deletion(-)
create mode 100644 drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index bf73a45..dc5cb00 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -12,6 +12,7 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \
venc_drv_if.o \
venc_vpu_if.o \
+
mtk-vcodec-common-y := mtk_vcodec_intr.o \
mtk_vcodec_util.o\
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
new file mode 100644
index 0000000..6e90b3f
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
@@ -0,0 +1,687 @@
+/*
+ * Copyright (c) 2016 MediaTek Inc.
+ * Author: Jungchang Tsao <jungchang.tsao@mediatek.com>
+ * Daniel Hsiao <daniel.hsiao@mediatek.com>
+ * PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include "../mtk_vcodec_drv.h"
+#include "../mtk_vcodec_util.h"
+#include "../mtk_vcodec_intr.h"
+#include "../mtk_vcodec_enc.h"
+#include "../mtk_vcodec_enc_pm.h"
+#include "../venc_drv_base.h"
+#include "../venc_ipi_msg.h"
+#include "../venc_vpu_if.h"
+#include "mtk_vpu.h"
+
+static const char h264_filler_marker[] = {0x0, 0x0, 0x0, 0x1, 0xc};
+
+#define H264_FILLER_MARKER_SIZE ARRAY_SIZE(h264_filler_marker)
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+
+/**
+ * enum venc_h264_vpu_work_buf - h264 encoder buffer index
+ */
+enum venc_h264_vpu_work_buf {
+ VENC_H264_VPU_WORK_BUF_RC_INFO,
+ VENC_H264_VPU_WORK_BUF_RC_CODE,
+ VENC_H264_VPU_WORK_BUF_REC_LUMA,
+ VENC_H264_VPU_WORK_BUF_REC_CHROMA,
+ VENC_H264_VPU_WORK_BUF_REF_LUMA,
+ VENC_H264_VPU_WORK_BUF_REF_CHROMA,
+ VENC_H264_VPU_WORK_BUF_MV_INFO_1,
+ VENC_H264_VPU_WORK_BUF_MV_INFO_2,
+ VENC_H264_VPU_WORK_BUF_SKIP_FRAME,
+ VENC_H264_VPU_WORK_BUF_MAX,
+};
+
+/**
+ * enum venc_h264_bs_mode - for bs_mode argument in h264_enc_vpu_encode
+ */
+enum venc_h264_bs_mode {
+ H264_BS_MODE_SPS,
+ H264_BS_MODE_PPS,
+ H264_BS_MODE_FRAME,
+};
+
+/*
+ * struct venc_h264_vpu_config - Structure for h264 encoder configuration
+ * @input_fourcc: input fourcc
+ * @bitrate: target bitrate (in bps)
+ * @pic_w: picture width. Picture size is visible stream resolution, in pixels,
+ * to be used for display purposes; must be smaller or equal to buffer
+ * size.
+ * @pic_h: picture height
+ * @buf_w: buffer width. Buffer size is stream resolution in pixels aligned to
+ * hardware requirements.
+ * @buf_h: buffer height
+ * @gop_size: group of picture size (idr frame)
+ * @intra_period: intra frame period
+ * @framerate: frame rate in fps
+ * @profile: as specified in standard
+ * @level: as specified in standard
+ * @wfd: WFD mode 1:on, 0:off
+ */
+struct venc_h264_vpu_config {
+ u32 input_fourcc;
+ u32 bitrate;
+ u32 pic_w;
+ u32 pic_h;
+ u32 buf_w;
+ u32 buf_h;
+ u32 gop_size;
+ u32 intra_period;
+ u32 framerate;
+ u32 profile;
+ u32 level;
+ u32 wfd;
+};
+
+/*
+ * struct venc_h264_vpu_buf - Structure for buffer information
+ * @align: buffer alignment (in bytes)
+ * @iova: IO virtual address
+ * @vpua: VPU side memory addr which is used by RC_CODE
+ * @size: buffer size (in bytes)
+ */
+struct venc_h264_vpu_buf {
+ u32 align;
+ u32 iova;
+ u32 vpua;
+ u32 size;
+};
+
+/*
+ * struct venc_h264_vsi - Structure for VPU driver control and info share
+ * This structure is allocated in VPU side and shared to AP side.
+ * @config: h264 encoder configuration
+ * @work_bufs: working buffer information in VPU side
+ * The work_bufs here is for storing the 'size' info shared to AP side.
+ * The similar item in struct venc_h264_inst is for memory allocation
+ * in AP side. The AP driver will copy the 'size' from here to the one in
+ * struct mtk_vcodec_mem, then invoke mtk_vcodec_mem_alloc to allocate
+ * the buffer. After that, bypass the 'dma_addr' to the 'iova' field here for
+ * register setting in VPU side.
+ * @sizeimage: image size for each plane
+ */
+struct venc_h264_vsi {
+ struct venc_h264_vpu_config config;
+ struct venc_h264_vpu_buf work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
+ unsigned int sizeimage[MTK_VCODEC_MAX_PLANES];
+};
+
+/*
+ * struct venc_h264_inst - h264 encoder AP driver instance
+ * @hw_base: h264 encoder hardware register base
+ * @work_bufs: working buffer
+ * @pps_buf: buffer to store the pps bitstream
+ * @work_buf_allocated: working buffer allocated flag
+ * @frm_cnt: encoded frame count
+ * @prepend_hdr: when the v4l2 layer send VENC_SET_PARAM_PREPEND_HEADER cmd
+ * through h264_enc_set_param interface, it will set this flag and prepend the
+ * sps/pps in h264_enc_encode function.
+ * @vpu_inst: VPU instance to exchange information between AP and VPU
+ * @vsi: driver structure allocated by VPU side and shared to AP side for
+ * control and info share
+ * @ctx: context for v4l2 layer integration
+ */
+struct venc_h264_inst {
+ void __iomem *hw_base;
+ struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
+ struct mtk_vcodec_mem pps_buf;
+ bool work_buf_allocated;
+ unsigned int frm_cnt;
+ unsigned int prepend_hdr;
+ struct venc_vpu_inst vpu_inst;
+ struct venc_h264_vsi *vsi;
+ struct mtk_vcodec_ctx *ctx;
+};
+
+static inline void h264_write_reg(struct venc_h264_inst *inst, u32 addr,
+ u32 val)
+{
+ writel(val, inst->hw_base + addr);
+}
+
+static inline u32 h264_read_reg(struct venc_h264_inst *inst, u32 addr)
+{
+ return readl(inst->hw_base + addr);
+}
+
+static unsigned int h264_get_profile(struct venc_h264_inst *inst,
+ unsigned int profile)
+{
+ switch (profile) {
+ case V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE:
+ return 66;
+ case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN:
+ return 77;
+ case V4L2_MPEG_VIDEO_H264_PROFILE_HIGH:
+ return 100;
+ case V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE:
+ mtk_vcodec_err(inst, "unsupported CONSTRAINED_BASELINE");
+ return 0;
+ case V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED:
+ mtk_vcodec_err(inst, "unsupported EXTENDED");
+ return 0;
+ default:
+ mtk_vcodec_debug(inst, "unsupported profile %d", profile);
+ return 100;
+ }
+}
+
+static unsigned int h264_get_level(struct venc_h264_inst *inst,
+ unsigned int level)
+{
+ switch (level) {
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1B:
+ mtk_vcodec_err(inst, "unsupported 1B");
+ return 0;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_0:
+ return 10;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_1:
+ return 11;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_2:
+ return 12;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_3:
+ return 13;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_2_0:
+ return 20;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_2_1:
+ return 21;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_2_2:
+ return 22;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_3_0:
+ return 30;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_3_1:
+ return 31;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_3_2:
+ return 32;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_4_0:
+ return 40;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_4_1:
+ return 41;
+ default:
+ mtk_vcodec_debug(inst, "unsupported level %d", level);
+ return 31;
+ }
+}
+
+static void h264_enc_free_work_buf(struct venc_h264_inst *inst)
+{
+ int i;
+
+ mtk_vcodec_debug_enter(inst);
+
+ /* Except the SKIP_FRAME buffers,
+ * other buffers need to be freed by AP.
+ */
+ for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
+ if (i != VENC_H264_VPU_WORK_BUF_SKIP_FRAME)
+ mtk_vcodec_mem_free(inst->ctx, &inst->work_bufs[i]);
+ }
+
+ mtk_vcodec_mem_free(inst->ctx, &inst->pps_buf);
+
+ mtk_vcodec_debug_leave(inst);
+}
+
+static int h264_enc_alloc_work_buf(struct venc_h264_inst *inst)
+{
+ int i;
+ int ret = 0;
+ struct venc_h264_vpu_buf *wb = inst->vsi->work_bufs;
+
+ mtk_vcodec_debug_enter(inst);
+
+ for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) {
+ /*
+ * This 'wb' structure is set by VPU side and shared to AP for
+ * buffer allocation and IO virtual addr mapping. For most of
+ * the buffers, AP will allocate the buffer according to 'size'
+ * field and store the IO virtual addr in 'iova' field. There
+ * are two exceptions:
+ * (1) RC_CODE buffer, it's pre-allocated in the VPU side, and
+ * save the VPU addr in the 'vpua' field. The AP will translate
+ * the VPU addr to the corresponding IO virtual addr and store
+ * in 'iova' field for reg setting in VPU side.
+ * (2) SKIP_FRAME buffer, it's pre-allocated in the VPU side,
+ * and save the VPU addr in the 'vpua' field. The AP will
+ * translate the VPU addr to the corresponding AP side virtual
+ * address and do some memcpy access to move to bitstream buffer
+ * assigned by v4l2 layer.
+ */
+ inst->work_bufs[i].size = wb[i].size;
+ if (i == VENC_H264_VPU_WORK_BUF_SKIP_FRAME) {
+ inst->work_bufs[i].va = vpu_mapping_dm_addr(
+ inst->vpu_inst.dev, wb[i].vpua);
+ inst->work_bufs[i].dma_addr = 0;
+ } else {
+ ret = mtk_vcodec_mem_alloc(inst->ctx,
+ &inst->work_bufs[i]);
+ if (ret) {
+ mtk_vcodec_err(inst,
+ "cannot allocate buf %d", i);
+ goto err_alloc;
+ }
+ /*
+ * This RC_CODE is pre-allocated by VPU and saved in VPU
+ * addr. So we need use memcpy to copy RC_CODE from VPU
+ * addr into IO virtual addr in 'iova' field for reg
+ * setting in VPU side.
+ */
+ if (i == VENC_H264_VPU_WORK_BUF_RC_CODE) {
+ void *tmp_va;
+
+ tmp_va = vpu_mapping_dm_addr(inst->vpu_inst.dev,
+ wb[i].vpua);
+ memcpy(inst->work_bufs[i].va, tmp_va,
+ wb[i].size);
+ }
+ }
+ wb[i].iova = inst->work_bufs[i].dma_addr;
+
+ mtk_vcodec_debug(inst,
+ "work_buf[%d] va=0x%p iova=0x%p size=0x%lx",
+ i, inst->work_bufs[i].va,
+ (void *)inst->work_bufs[i].dma_addr,
+ inst->work_bufs[i].size);
+ }
+
+ /* the pps_buf is used by AP side only */
+ inst->pps_buf.size = 128;
+ ret = mtk_vcodec_mem_alloc(inst->ctx, &inst->pps_buf);
+ if (ret) {
+ mtk_vcodec_err(inst, "cannot allocate pps_buf");
+ goto err_alloc;
+ }
+
+ mtk_vcodec_debug_leave(inst);
+
+ return ret;
+
+err_alloc:
+ h264_enc_free_work_buf(inst);
+
+ return ret;
+}
+
+static unsigned int h264_enc_wait_venc_done(struct venc_h264_inst *inst)
+{
+ unsigned int irq_status = 0;
+ struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)inst->ctx;
+
+ if (!mtk_vcodec_wait_for_done_ctx(ctx, MTK_INST_IRQ_RECEIVED,
+ WAIT_INTR_TIMEOUT_MS)) {
+ irq_status = ctx->irq_status;
+ mtk_vcodec_debug(inst, "irq_status %x <-", irq_status);
+ }
+ return irq_status;
+}
+
+static int h264_encode_sps(struct venc_h264_inst *inst,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ int ret = 0;
+ unsigned int irq_status;
+
+ mtk_vcodec_debug_enter(inst);
+
+ ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_SPS, NULL,
+ bs_buf, bs_size);
+ if (ret)
+ return ret;
+
+ irq_status = h264_enc_wait_venc_done(inst);
+ if (irq_status != MTK_VENC_IRQ_STATUS_SPS) {
+ mtk_vcodec_err(inst, "expect irq status %d",
+ MTK_VENC_IRQ_STATUS_SPS);
+ return -EINVAL;
+ }
+
+ *bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
+ mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
+
+ return ret;
+}
+
+static int h264_encode_pps(struct venc_h264_inst *inst,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ int ret = 0;
+ unsigned int irq_status;
+
+ mtk_vcodec_debug_enter(inst);
+
+ ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_PPS, NULL,
+ bs_buf, bs_size);
+ if (ret)
+ return ret;
+
+ irq_status = h264_enc_wait_venc_done(inst);
+ if (irq_status != MTK_VENC_IRQ_STATUS_PPS) {
+ mtk_vcodec_err(inst, "expect irq status %d",
+ MTK_VENC_IRQ_STATUS_PPS);
+ return -EINVAL;
+ }
+
+ *bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
+ mtk_vcodec_debug(inst, "bs size %d <-", *bs_size);
+
+ return ret;
+}
+
+static int h264_encode_header(struct venc_h264_inst *inst,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ int ret = 0;
+ unsigned int bs_size_sps;
+ unsigned int bs_size_pps;
+
+ ret = h264_encode_sps(inst, bs_buf, &bs_size_sps);
+ if (ret)
+ return ret;
+
+ ret = h264_encode_pps(inst, &inst->pps_buf, &bs_size_pps);
+ if (ret)
+ return ret;
+
+ memcpy(bs_buf->va + bs_size_sps, inst->pps_buf.va, bs_size_pps);
+ *bs_size = bs_size_sps + bs_size_pps;
+
+ return ret;
+}
+
+static int h264_encode_frame(struct venc_h264_inst *inst,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ int ret = 0;
+ unsigned int irq_status;
+
+ mtk_vcodec_debug_enter(inst);
+
+ ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_FRAME, frm_buf,
+ bs_buf, bs_size);
+ if (ret)
+ return ret;
+
+ /*
+ * skip frame case: The skip frame buffer is composed by vpu side only,
+ * it does not trigger the hw, so skip the wait interrupt operation.
+ */
+ if (inst->vpu_inst.state == VEN_IPI_MSG_ENC_STATE_SKIP) {
+ *bs_size = inst->vpu_inst.bs_size;
+ memcpy(bs_buf->va,
+ inst->work_bufs[VENC_H264_VPU_WORK_BUF_SKIP_FRAME].va,
+ *bs_size);
+ ++inst->frm_cnt;
+ return ret;
+ }
+
+ irq_status = h264_enc_wait_venc_done(inst);
+ if (irq_status != MTK_VENC_IRQ_STATUS_FRM) {
+ mtk_vcodec_err(inst, "irq_status=%d failed", irq_status);
+ return -EIO;
+ }
+
+ *bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
+
+ ++inst->frm_cnt;
+ mtk_vcodec_debug(inst, "frm %d bs size %d key_frm %d <-",
+ inst->frm_cnt,
+ *bs_size, inst->vpu_inst.is_key_frm);
+
+ return ret;
+}
+
+static void h264_encode_filler(struct venc_h264_inst *inst, void *buf,
+ int size)
+{
+ unsigned char *p = buf;
+
+ if (size < H264_FILLER_MARKER_SIZE) {
+ mtk_vcodec_err(inst, "filler size too small %d", size);
+ return;
+ }
+
+ memcpy(p, h264_filler_marker, ARRAY_SIZE(h264_filler_marker));
+ size -= H264_FILLER_MARKER_SIZE;
+ p += H264_FILLER_MARKER_SIZE;
+ memset(p, 0xff, size);
+}
+
+static int h264_enc_init(struct mtk_vcodec_ctx *ctx, unsigned long *handle)
+{
+ int ret = 0;
+ struct venc_h264_inst *inst;
+
+ inst = kzalloc(sizeof(*inst), GFP_KERNEL);
+ if (!inst)
+ return -ENOMEM;
+
+ inst->ctx = ctx;
+ inst->vpu_inst.ctx = ctx;
+ inst->vpu_inst.dev = ctx->dev->vpu_plat_dev;
+ inst->vpu_inst.id = IPI_VENC_H264;
+ inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
+
+ mtk_vcodec_debug_enter(inst);
+
+ ret = vpu_enc_init(&inst->vpu_inst);
+
+ inst->vsi = (struct venc_h264_vsi *)inst->vpu_inst.vsi;
+
+ mtk_vcodec_debug_leave(inst);
+
+ if (ret)
+ kfree(inst);
+ else
+ (*handle) = (unsigned long)inst;
+
+ return ret;
+}
+
+static int h264_enc_encode(unsigned long handle,
+ enum venc_start_opt opt,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ struct venc_done_result *result)
+{
+ int ret = 0;
+ struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
+ struct mtk_vcodec_ctx *ctx = inst->ctx;
+
+ mtk_vcodec_debug(inst, "opt %d ->", opt);
+
+ enable_irq(ctx->dev->enc_irq);
+
+ switch (opt) {
+ case VENC_START_OPT_ENCODE_SEQUENCE_HEADER: {
+ unsigned int bs_size_hdr;
+
+ ret = h264_encode_header(inst, bs_buf, &bs_size_hdr);
+ if (ret)
+ goto encode_err;
+
+ result->bs_size = bs_size_hdr;
+ result->is_key_frm = false;
+ break;
+ }
+
+ case VENC_START_OPT_ENCODE_FRAME: {
+ int hdr_sz;
+ int hdr_sz_ext;
+ int filler_sz = 0;
+ const int bs_alignment = 128;
+ struct mtk_vcodec_mem tmp_bs_buf;
+ unsigned int bs_size_hdr;
+ unsigned int bs_size_frm;
+
+ if (!inst->prepend_hdr) {
+ ret = h264_encode_frame(inst, frm_buf, bs_buf,
+ &result->bs_size);
+ if (ret)
+ goto encode_err;
+ result->is_key_frm = inst->vpu_inst.is_key_frm;
+ break;
+ }
+
+ mtk_vcodec_debug(inst, "h264_encode_frame prepend SPS/PPS");
+
+ ret = h264_encode_header(inst, bs_buf, &bs_size_hdr);
+ if (ret)
+ goto encode_err;
+
+ hdr_sz = bs_size_hdr;
+ hdr_sz_ext = (hdr_sz & (bs_alignment - 1));
+ if (hdr_sz_ext) {
+ filler_sz = bs_alignment - hdr_sz_ext;
+ if (hdr_sz_ext + H264_FILLER_MARKER_SIZE > bs_alignment)
+ filler_sz += bs_alignment;
+ h264_encode_filler(inst, bs_buf->va + hdr_sz,
+ filler_sz);
+ }
+
+ tmp_bs_buf.va = bs_buf->va + hdr_sz + filler_sz;
+ tmp_bs_buf.dma_addr = bs_buf->dma_addr + hdr_sz + filler_sz;
+ tmp_bs_buf.size = bs_buf->size - (hdr_sz + filler_sz);
+
+ ret = h264_encode_frame(inst, frm_buf, &tmp_bs_buf,
+ &bs_size_frm);
+ if (ret)
+ goto encode_err;
+
+ result->bs_size = hdr_sz + filler_sz + bs_size_frm;
+
+ mtk_vcodec_debug(inst, "hdr %d filler %d frame %d bs %d",
+ hdr_sz, filler_sz, bs_size_frm,
+ result->bs_size);
+
+ inst->prepend_hdr = 0;
+ result->is_key_frm = inst->vpu_inst.is_key_frm;
+ break;
+ }
+
+ default:
+ mtk_vcodec_err(inst, "venc_start_opt %d not supported", opt);
+ ret = -EINVAL;
+ break;
+ }
+
+encode_err:
+
+ disable_irq(ctx->dev->enc_irq);
+ mtk_vcodec_debug(inst, "opt %d <-", opt);
+
+ return ret;
+}
+
+static int h264_enc_set_param(unsigned long handle,
+ enum venc_set_param_type type,
+ struct venc_enc_param *enc_prm)
+{
+ int i;
+ int ret = 0;
+ struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
+
+ mtk_vcodec_debug(inst, "->type=%d", type);
+
+ switch (type) {
+ case VENC_SET_PARAM_ENC:
+ inst->vsi->config.input_fourcc = enc_prm->input_yuv_fmt;
+ inst->vsi->config.bitrate = enc_prm->bitrate;
+ inst->vsi->config.pic_w = enc_prm->width;
+ inst->vsi->config.pic_h = enc_prm->height;
+ inst->vsi->config.buf_w = enc_prm->buf_width;
+ inst->vsi->config.buf_h = enc_prm->buf_height;
+ inst->vsi->config.gop_size = enc_prm->gop_size;
+ inst->vsi->config.framerate = enc_prm->frm_rate;
+ inst->vsi->config.intra_period = enc_prm->intra_period;
+ inst->vsi->config.profile =
+ h264_get_profile(inst, enc_prm->h264_profile);
+ inst->vsi->config.level =
+ h264_get_level(inst, enc_prm->h264_level);
+ inst->vsi->config.wfd = 0;
+ ret = vpu_enc_set_param(&inst->vpu_inst, type, enc_prm);
+ if (ret)
+ break;
+ if (inst->work_buf_allocated) {
+ h264_enc_free_work_buf(inst);
+ inst->work_buf_allocated = false;
+ }
+ ret = h264_enc_alloc_work_buf(inst);
+ if (ret)
+ break;
+ inst->work_buf_allocated = true;
+ for (i = 0; i < MTK_VCODEC_MAX_PLANES; i++) {
+ enc_prm->sizeimage[i] =
+ inst->vsi->sizeimage[i];
+ mtk_vcodec_debug(inst, "sizeimage[%d] size=0x%x", i,
+ enc_prm->sizeimage[i]);
+ }
+ break;
+
+ case VENC_SET_PARAM_PREPEND_HEADER:
+ inst->prepend_hdr = 1;
+ mtk_vcodec_debug(inst, "set prepend header mode");
+ break;
+
+ default:
+ ret = vpu_enc_set_param(&inst->vpu_inst, type, enc_prm);
+ break;
+ }
+
+ mtk_vcodec_debug_leave(inst);
+
+ return ret;
+}
+
+static int h264_enc_deinit(unsigned long handle)
+{
+ int ret = 0;
+ struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
+
+ mtk_vcodec_debug_enter(inst);
+
+ ret = vpu_enc_deinit(&inst->vpu_inst);
+
+ if (inst->work_buf_allocated)
+ h264_enc_free_work_buf(inst);
+
+ mtk_vcodec_debug_leave(inst);
+ kfree(inst);
+
+ return ret;
+}
+
+static struct venc_common_if venc_h264_if = {
+ h264_enc_init,
+ h264_enc_encode,
+ h264_enc_set_param,
+ h264_enc_deinit,
+};
+
+struct venc_common_if *get_h264_enc_comm_if(void)
+{
+ return &venc_h264_if;
+}
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
index 0ac6e49..9cd915f 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
@@ -26,7 +26,7 @@
#include "mtk_vcodec_enc_pm.h"
#include "mtk_vpu.h"
-
+struct venc_common_if *get_h264_enc_comm_if(void);
struct venc_common_if *get_vp8_enc_comm_if(void);
int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
@@ -38,6 +38,8 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
ctx->enc_if = get_vp8_enc_comm_if();
break;
case V4L2_PIX_FMT_H264:
+ ctx->enc_if = get_h264_enc_comm_if();
+ break;
default:
return -EINVAL;
}
--
1.7.9.5
[toc] | [prev] | [next] | [standalone]
| From | Tiffany Lin <tiffany.lin@mediatek.com> |
|---|---|
| Date | 2016-04-22 06:30 +0200 |
| Subject | [PATCH v7 6/8] [media] vcodec: mediatek: Add Mediatek VP8 Video Encoder Driver |
| Message-ID | <rqAT7-4e7-15@gated-at.bofh.it> |
| In reply to | #1384687 |
Add vp8 encoder driver for MT8173
Signed-off-by: PoChun Lin <pochun.lin@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
drivers/media/platform/mtk-vcodec/Makefile | 6 +-
.../media/platform/mtk-vcodec/venc/venc_vp8_if.c | 488 ++++++++++++++++++++
drivers/media/platform/mtk-vcodec/venc_drv_if.c | 7 +-
drivers/media/platform/mtk-vcodec/venc_vpu_if.c | 240 ++++++++++
drivers/media/platform/mtk-vcodec/venc_vpu_if.h | 61 +++
5 files changed, 800 insertions(+), 2 deletions(-)
create mode 100644 drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/venc_vpu_if.c
create mode 100644 drivers/media/platform/mtk-vcodec/venc_vpu_if.h
diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index d04433be..bf73a45 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -1,12 +1,16 @@
+
obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-enc.o mtk-vcodec-common.o
-mtk-vcodec-enc-y := mtk_vcodec_enc.o \
+mtk-vcodec-enc-y := venc/venc_vp8_if.o \
+ venc/venc_h264_if.o \
+ mtk_vcodec_enc.o \
mtk_vcodec_enc_drv.o \
mtk_vcodec_enc_pm.o \
venc_drv_if.o \
+ venc_vpu_if.o \
mtk-vcodec-common-y := mtk_vcodec_intr.o \
mtk_vcodec_util.o\
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
new file mode 100644
index 0000000..5389841
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
@@ -0,0 +1,488 @@
+/*
+ * Copyright (c) 2016 MediaTek Inc.
+ * Author: Daniel Hsiao <daniel.hsiao@mediatek.com>
+ * PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include "../mtk_vcodec_drv.h"
+#include "../mtk_vcodec_util.h"
+#include "../mtk_vcodec_intr.h"
+#include "../mtk_vcodec_enc.h"
+#include "../mtk_vcodec_enc_pm.h"
+#include "../venc_drv_base.h"
+#include "../venc_ipi_msg.h"
+#include "../venc_vpu_if.h"
+#include "mtk_vpu.h"
+
+#define VENC_BITSTREAM_FRAME_SIZE 0x0098
+#define VENC_BITSTREAM_HEADER_LEN 0x00e8
+
+/* This ac_tag is vp8 frame tag. */
+#define MAX_AC_TAG_SIZE 10
+
+/**
+ * enum venc_vp8_vpu_work_buf - vp8 encoder buffer index
+ */
+enum venc_vp8_vpu_work_buf {
+ VENC_VP8_VPU_WORK_BUF_LUMA,
+ VENC_VP8_VPU_WORK_BUF_LUMA2,
+ VENC_VP8_VPU_WORK_BUF_LUMA3,
+ VENC_VP8_VPU_WORK_BUF_CHROMA,
+ VENC_VP8_VPU_WORK_BUF_CHROMA2,
+ VENC_VP8_VPU_WORK_BUF_CHROMA3,
+ VENC_VP8_VPU_WORK_BUF_MV_INFO,
+ VENC_VP8_VPU_WORK_BUF_BS_HEADER,
+ VENC_VP8_VPU_WORK_BUF_PROB_BUF,
+ VENC_VP8_VPU_WORK_BUF_RC_INFO,
+ VENC_VP8_VPU_WORK_BUF_RC_CODE,
+ VENC_VP8_VPU_WORK_BUF_RC_CODE2,
+ VENC_VP8_VPU_WORK_BUF_RC_CODE3,
+ VENC_VP8_VPU_WORK_BUF_MAX,
+};
+
+/*
+ * struct venc_vp8_vpu_config - Structure for vp8 encoder configuration
+ * @input_fourcc: input fourcc
+ * @bitrate: target bitrate (in bps)
+ * @pic_w: picture width. Picture size is visible stream resolution, in pixels,
+ * to be used for display purposes; must be smaller or equal to buffer
+ * size.
+ * @pic_h: picture height
+ * @buf_w: buffer width (with 16 alignment). Buffer size is stream resolution
+ * in pixels aligned to hardware requirements.
+ * @buf_h: buffer height (with 16 alignment)
+ * @gop_size: group of picture size (key frame)
+ * @framerate: frame rate in fps
+ * @ts_mode: temporal scalability mode (0: disable, 1: enable)
+ * support three temporal layers - 0: 7.5fps 1: 7.5fps 2: 15fps.
+ */
+struct venc_vp8_vpu_config {
+ u32 input_fourcc;
+ u32 bitrate;
+ u32 pic_w;
+ u32 pic_h;
+ u32 buf_w;
+ u32 buf_h;
+ u32 gop_size;
+ u32 framerate;
+ u32 ts_mode;
+};
+
+/*
+ * struct venc_vp8_vpu_buf -Structure for buffer information
+ * @align: buffer alignment (in bytes)
+ * @iova: IO virtual address
+ * @vpua: VPU side memory addr which is used by RC_CODE
+ * @size: buffer size (in bytes)
+ */
+struct venc_vp8_vpu_buf {
+ u32 align;
+ u32 iova;
+ u32 vpua;
+ u32 size;
+};
+
+/*
+ * struct venc_vp8_vsi - Structure for VPU driver control and info share
+ * This structure is allocated in VPU side and shared to AP side.
+ * @config: vp8 encoder configuration
+ * @work_bufs: working buffer information in VPU side
+ * The work_bufs here is for storing the 'size' info shared to AP side.
+ * The similar item in struct venc_vp8_inst is for memory allocation
+ * in AP side. The AP driver will copy the 'size' from here to the one in
+ * struct mtk_vcodec_mem, then invoke mtk_vcodec_mem_alloc to allocate
+ * the buffer. After that, bypass the 'dma_addr' to the 'iova' field here for
+ * register setting in VPU side.
+ * @sizeimage: image size for each plane
+ */
+struct venc_vp8_vsi {
+ struct venc_vp8_vpu_config config;
+ struct venc_vp8_vpu_buf work_bufs[VENC_VP8_VPU_WORK_BUF_MAX];
+ unsigned int sizeimage[MTK_VCODEC_MAX_PLANES];
+};
+
+/*
+ * struct venc_vp8_inst - vp8 encoder AP driver instance
+ * @hw_base: vp8 encoder hardware register base
+ * @work_bufs: working buffer
+ * @work_buf_allocated: working buffer allocated flag
+ * @frm_cnt: encoded frame count, it's used for I-frame judgement and
+ * reset when force intra cmd received.
+ * @ts_mode: temporal scalability mode (0: disable, 1: enable)
+ * support three temporal layers - 0: 7.5fps 1: 7.5fps 2: 15fps.
+ * @vpu_inst: VPU instance to exchange information between AP and VPU
+ * @vsi: driver structure allocated by VPU side and shared to AP side for
+ * control and info share
+ * @ctx: context for v4l2 layer integration
+ */
+struct venc_vp8_inst {
+ void __iomem *hw_base;
+ struct mtk_vcodec_mem work_bufs[VENC_VP8_VPU_WORK_BUF_MAX];
+ bool work_buf_allocated;
+ unsigned int frm_cnt;
+ unsigned int ts_mode;
+ struct venc_vpu_inst vpu_inst;
+ struct venc_vp8_vsi *vsi;
+ struct mtk_vcodec_ctx *ctx;
+};
+
+static inline void vp8_enc_write_reg(struct venc_vp8_inst *inst, u32 addr,
+ u32 val)
+{
+ writel(val, inst->hw_base + addr);
+}
+
+static inline u32 vp8_enc_read_reg(struct venc_vp8_inst *inst, u32 addr)
+{
+ return readl(inst->hw_base + addr);
+}
+
+static void vp8_enc_free_work_buf(struct venc_vp8_inst *inst)
+{
+ int i;
+
+ mtk_vcodec_debug_enter(inst);
+
+ /* Buffers need to be freed by AP. */
+ for (i = 0; i < VENC_VP8_VPU_WORK_BUF_MAX; i++) {
+ if ((inst->work_bufs[i].size == 0))
+ continue;
+ mtk_vcodec_mem_free(inst->ctx, &inst->work_bufs[i]);
+ }
+
+ mtk_vcodec_debug_leave(inst);
+}
+
+static int vp8_enc_alloc_work_buf(struct venc_vp8_inst *inst)
+{
+ int i;
+ int ret = 0;
+ struct venc_vp8_vpu_buf *wb = inst->vsi->work_bufs;
+
+ mtk_vcodec_debug_enter(inst);
+
+ for (i = 0; i < VENC_VP8_VPU_WORK_BUF_MAX; i++) {
+ if ((wb[i].size == 0))
+ continue;
+ /*
+ * This 'wb' structure is set by VPU side and shared to AP for
+ * buffer allocation and IO virtual addr mapping. For most of
+ * the buffers, AP will allocate the buffer according to 'size'
+ * field and store the IO virtual addr in 'iova' field. For the
+ * RC_CODEx buffers, they are pre-allocated in the VPU side
+ * because they are inside VPU SRAM, and save the VPU addr in
+ * the 'vpua' field. The AP will translate the VPU addr to the
+ * corresponding IO virtual addr and store in 'iova' field.
+ */
+ inst->work_bufs[i].size = wb[i].size;
+ ret = mtk_vcodec_mem_alloc(inst->ctx, &inst->work_bufs[i]);
+ if (ret) {
+ mtk_vcodec_err(inst,
+ "cannot alloc work_bufs[%d]", i);
+ goto err_alloc;
+ }
+ /*
+ * This RC_CODEx is pre-allocated by VPU and saved in VPU addr.
+ * So we need use memcpy to copy RC_CODEx from VPU addr into IO
+ * virtual addr in 'iova' field for reg setting in VPU side.
+ */
+ if (i == VENC_VP8_VPU_WORK_BUF_RC_CODE ||
+ i == VENC_VP8_VPU_WORK_BUF_RC_CODE2 ||
+ i == VENC_VP8_VPU_WORK_BUF_RC_CODE3) {
+ void *tmp_va;
+
+ tmp_va = vpu_mapping_dm_addr(inst->vpu_inst.dev,
+ wb[i].vpua);
+ memcpy(inst->work_bufs[i].va, tmp_va, wb[i].size);
+ }
+ wb[i].iova = inst->work_bufs[i].dma_addr;
+
+ mtk_vcodec_debug(inst,
+ "work_bufs[%d] va=0x%p,iova=0x%p,size=0x%lx",
+ i, inst->work_bufs[i].va,
+ (void *)inst->work_bufs[i].dma_addr,
+ inst->work_bufs[i].size);
+ }
+
+ mtk_vcodec_debug_leave(inst);
+
+ return ret;
+
+err_alloc:
+ vp8_enc_free_work_buf(inst);
+
+ return ret;
+}
+
+static unsigned int vp8_enc_wait_venc_done(struct venc_vp8_inst *inst)
+{
+ unsigned int irq_status = 0;
+ struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)inst->ctx;
+
+ if (!mtk_vcodec_wait_for_done_ctx(ctx, MTK_INST_IRQ_RECEIVED,
+ WAIT_INTR_TIMEOUT_MS)) {
+ irq_status = ctx->irq_status;
+ mtk_vcodec_debug(inst, "isr return %x", irq_status);
+ }
+ return irq_status;
+}
+
+/*
+ * Compose ac_tag, bitstream header and bitstream payload into
+ * one bitstream buffer.
+ */
+static int vp8_enc_compose_one_frame(struct venc_vp8_inst *inst,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ unsigned int not_key;
+ u32 bs_frm_size;
+ u32 bs_hdr_len;
+ unsigned int ac_tag_size;
+ u8 ac_tag[MAX_AC_TAG_SIZE];
+
+ bs_frm_size = vp8_enc_read_reg(inst, VENC_BITSTREAM_FRAME_SIZE);
+ bs_hdr_len = vp8_enc_read_reg(inst, VENC_BITSTREAM_HEADER_LEN);
+
+ /* if a frame is key frame, not_key is 0 */
+ not_key = !inst->vpu_inst.is_key_frm;
+ *(u32 *)ac_tag = __cpu_to_le32((bs_hdr_len << 5) | 0x10 | not_key);
+ /* key frame */
+ if (not_key == 0) {
+ ac_tag_size = MAX_AC_TAG_SIZE;
+ ac_tag[3] = 0x9d;
+ ac_tag[4] = 0x01;
+ ac_tag[5] = 0x2a;
+ ac_tag[6] = inst->vsi->config.pic_w;
+ ac_tag[7] = inst->vsi->config.pic_w >> 8;
+ ac_tag[8] = inst->vsi->config.pic_h;
+ ac_tag[9] = inst->vsi->config.pic_h >> 8;
+ } else {
+ ac_tag_size = 3;
+ }
+
+ if (bs_buf->size < bs_hdr_len + bs_frm_size + ac_tag_size) {
+ mtk_vcodec_err(inst, "bitstream buf size is too small(%ld)",
+ bs_buf->size);
+ return -EINVAL;
+ }
+
+ /*
+ * (1) The vp8 bitstream header and body are generated by the HW vp8
+ * encoder separately at the same time. We cannot know the bitstream
+ * header length in advance.
+ * (2) From the vp8 spec, there is no stuffing byte allowed between the
+ * ac tag, bitstream header and bitstream body.
+ */
+ memmove(bs_buf->va + bs_hdr_len + ac_tag_size,
+ bs_buf->va, bs_frm_size);
+ memcpy(bs_buf->va + ac_tag_size,
+ inst->work_bufs[VENC_VP8_VPU_WORK_BUF_BS_HEADER].va,
+ bs_hdr_len);
+ memcpy(bs_buf->va, ac_tag, ac_tag_size);
+ *bs_size = bs_frm_size + bs_hdr_len + ac_tag_size;
+
+ return 0;
+}
+
+static int vp8_enc_encode_frame(struct venc_vp8_inst *inst,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ int ret = 0;
+ unsigned int irq_status;
+
+ mtk_vcodec_debug(inst, "->frm_cnt=%d", inst->frm_cnt);
+
+ ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, bs_size);
+ if (ret)
+ return ret;
+
+ irq_status = vp8_enc_wait_venc_done(inst);
+ if (irq_status != MTK_VENC_IRQ_STATUS_FRM) {
+ mtk_vcodec_err(inst, "irq_status=%d failed", irq_status);
+ return -EIO;
+ }
+
+ if (vp8_enc_compose_one_frame(inst, bs_buf, bs_size)) {
+ mtk_vcodec_err(inst, "vp8_enc_compose_one_frame failed");
+ return -EINVAL;
+ }
+
+ inst->frm_cnt++;
+ mtk_vcodec_debug(inst, "<-size=%d key_frm=%d", *bs_size,
+ inst->vpu_inst.is_key_frm);
+
+ return ret;
+}
+
+static int vp8_enc_init(struct mtk_vcodec_ctx *ctx, unsigned long *handle)
+{
+ int ret = 0;
+ struct venc_vp8_inst *inst;
+
+ inst = kzalloc(sizeof(*inst), GFP_KERNEL);
+ if (!inst)
+ return -ENOMEM;
+
+ inst->ctx = ctx;
+ inst->vpu_inst.ctx = ctx;
+ inst->vpu_inst.dev = ctx->dev->vpu_plat_dev;
+ inst->vpu_inst.id = IPI_VENC_VP8;
+ inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_LT_SYS);
+
+ mtk_vcodec_debug_enter(inst);
+
+ ret = vpu_enc_init(&inst->vpu_inst);
+
+ inst->vsi = (struct venc_vp8_vsi *)inst->vpu_inst.vsi;
+
+ mtk_vcodec_debug_leave(inst);
+
+ if (ret)
+ kfree(inst);
+ else
+ (*handle) = (unsigned long)inst;
+
+ return ret;
+}
+
+static int vp8_enc_encode(unsigned long handle,
+ enum venc_start_opt opt,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ struct venc_done_result *result)
+{
+ int ret = 0;
+ struct venc_vp8_inst *inst = (struct venc_vp8_inst *)handle;
+ struct mtk_vcodec_ctx *ctx = inst->ctx;
+
+ mtk_vcodec_debug_enter(inst);
+
+ enable_irq(ctx->dev->enc_lt_irq);
+
+ switch (opt) {
+ case VENC_START_OPT_ENCODE_FRAME:
+ ret = vp8_enc_encode_frame(inst, frm_buf, bs_buf,
+ &result->bs_size);
+ if (ret)
+ goto encode_err;
+ result->is_key_frm = inst->vpu_inst.is_key_frm;
+ break;
+
+ default:
+ mtk_vcodec_err(inst, "opt not support:%d", opt);
+ ret = -EINVAL;
+ break;
+ }
+
+encode_err:
+
+ disable_irq(ctx->dev->enc_lt_irq);
+ mtk_vcodec_debug_leave(inst);
+
+ return ret;
+}
+
+static int vp8_enc_set_param(unsigned long handle,
+ enum venc_set_param_type type,
+ struct venc_enc_param *enc_prm)
+{
+ int i;
+ int ret = 0;
+ struct venc_vp8_inst *inst = (struct venc_vp8_inst *)handle;
+
+ mtk_vcodec_debug(inst, "->type=%d", type);
+
+ switch (type) {
+ case VENC_SET_PARAM_ENC:
+ inst->vsi->config.input_fourcc = enc_prm->input_yuv_fmt;
+ inst->vsi->config.bitrate = enc_prm->bitrate;
+ inst->vsi->config.pic_w = enc_prm->width;
+ inst->vsi->config.pic_h = enc_prm->height;
+ inst->vsi->config.buf_w = enc_prm->buf_width;
+ inst->vsi->config.buf_h = enc_prm->buf_height;
+ inst->vsi->config.gop_size = enc_prm->gop_size;
+ inst->vsi->config.framerate = enc_prm->frm_rate;
+ inst->vsi->config.ts_mode = inst->ts_mode;
+ ret = vpu_enc_set_param(&inst->vpu_inst, type, enc_prm);
+ if (ret)
+ break;
+ if (inst->work_buf_allocated) {
+ vp8_enc_free_work_buf(inst);
+ inst->work_buf_allocated = false;
+ }
+ ret = vp8_enc_alloc_work_buf(inst);
+ if (ret)
+ break;
+ inst->work_buf_allocated = true;
+ for (i = 0; i < MTK_VCODEC_MAX_PLANES; i++) {
+ enc_prm->sizeimage[i] =
+ inst->vsi->sizeimage[i];
+ mtk_vcodec_debug(inst, "sizeimage[%d] size=0x%x", i,
+ enc_prm->sizeimage[i]);
+ }
+ break;
+
+ /*
+ * VENC_SET_PARAM_TS_MODE must be called before VENC_SET_PARAM_ENC
+ */
+ case VENC_SET_PARAM_TS_MODE:
+ inst->ts_mode = 1;
+ mtk_vcodec_debug(inst, "set ts_mode");
+ break;
+
+ default:
+ ret = vpu_enc_set_param(&inst->vpu_inst, type, enc_prm);
+ break;
+ }
+
+ mtk_vcodec_debug_leave(inst);
+
+ return ret;
+}
+
+static int vp8_enc_deinit(unsigned long handle)
+{
+ int ret = 0;
+ struct venc_vp8_inst *inst = (struct venc_vp8_inst *)handle;
+
+ mtk_vcodec_debug_enter(inst);
+
+ ret = vpu_enc_deinit(&inst->vpu_inst);
+
+ if (inst->work_buf_allocated)
+ vp8_enc_free_work_buf(inst);
+
+ mtk_vcodec_debug_leave(inst);
+ kfree(inst);
+
+ return ret;
+}
+
+static struct venc_common_if venc_vp8_if = {
+ vp8_enc_init,
+ vp8_enc_encode,
+ vp8_enc_set_param,
+ vp8_enc_deinit,
+};
+
+struct venc_common_if *get_vp8_enc_comm_if(void)
+{
+ return &venc_vp8_if;
+}
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
index e53b437..0ac6e49 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
@@ -19,12 +19,15 @@
#include <linux/kernel.h>
#include <linux/slab.h>
+#include "venc_drv_base.h"
#include "venc_drv_if.h"
+
#include "mtk_vcodec_enc.h"
#include "mtk_vcodec_enc_pm.h"
#include "mtk_vpu.h"
-#include "venc_drv_base.h"
+
+struct venc_common_if *get_vp8_enc_comm_if(void);
int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
{
@@ -32,6 +35,8 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
switch (fourcc) {
case V4L2_PIX_FMT_VP8:
+ ctx->enc_if = get_vp8_enc_comm_if();
+ break;
case V4L2_PIX_FMT_H264:
default:
return -EINVAL;
diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
new file mode 100644
index 0000000..02def5a
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
@@ -0,0 +1,240 @@
+/*
+ * Copyright (c) 2016 MediaTek Inc.
+ * Author: PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include "mtk_vpu.h"
+#include "venc_ipi_msg.h"
+#include "venc_vpu_if.h"
+
+static void handle_enc_init_msg(struct venc_vpu_inst *vpu, void *data)
+{
+ struct venc_vpu_ipi_msg_init *msg = data;
+
+ vpu->inst_addr = msg->vpu_inst_addr;
+ vpu->vsi = vpu_mapping_dm_addr(vpu->dev, msg->vpu_inst_addr);
+}
+
+static void handle_enc_encode_msg(struct venc_vpu_inst *vpu, void *data)
+{
+ struct venc_vpu_ipi_msg_enc *msg = data;
+
+ vpu->state = msg->state;
+ vpu->bs_size = msg->bs_size;
+ vpu->is_key_frm = msg->is_key_frm;
+}
+
+static void vpu_enc_ipi_handler(void *data, unsigned int len, void *priv)
+{
+ struct venc_vpu_ipi_msg_common *msg = data;
+ struct venc_vpu_inst *vpu = (struct venc_vpu_inst *)msg->venc_inst;
+
+ mtk_vcodec_debug(vpu, "msg_id %x inst %p status %d",
+ msg->msg_id, vpu, msg->status);
+
+ switch (msg->msg_id) {
+ case VPU_IPIMSG_ENC_INIT_DONE:
+ handle_enc_init_msg(vpu, data);
+ break;
+ case VPU_IPIMSG_ENC_SET_PARAM_DONE:
+ break;
+ case VPU_IPIMSG_ENC_ENCODE_DONE:
+ handle_enc_encode_msg(vpu, data);
+ break;
+ case VPU_IPIMSG_ENC_DEINIT_DONE:
+ break;
+ default:
+ mtk_vcodec_err(vpu, "unknown msg id %x", msg->msg_id);
+ break;
+ }
+
+ vpu->signaled = 1;
+ vpu->failure = (msg->status != VENC_IPI_MSG_STATUS_OK);
+
+ mtk_vcodec_debug_leave(vpu);
+}
+
+static int vpu_enc_send_msg(struct venc_vpu_inst *vpu, void *msg,
+ int len)
+{
+ int status;
+
+ mtk_vcodec_debug_enter(vpu);
+
+ if (!vpu->dev) {
+ mtk_vcodec_err(vpu, "inst dev is NULL");
+ return -EINVAL;
+ }
+
+ status = vpu_ipi_send(vpu->dev, vpu->id, msg, len);
+ if (status) {
+ uint32_t msg_id = *(uint32_t *)msg;
+
+ mtk_vcodec_err(vpu, "vpu_ipi_send msg_id %x len %d fail %d",
+ msg_id, len, status);
+ return -EINVAL;
+ }
+ if (vpu->failure)
+ return -EINVAL;
+
+ mtk_vcodec_debug_leave(vpu);
+
+ return 0;
+}
+
+int vpu_enc_init(struct venc_vpu_inst *vpu)
+{
+ int status;
+ struct venc_ap_ipi_msg_init out;
+
+ mtk_vcodec_debug_enter(vpu);
+
+ init_waitqueue_head(&vpu->wq_hd);
+ vpu->signaled = 0;
+ vpu->failure = 0;
+
+ status = vpu_ipi_register(vpu->dev, vpu->id, vpu_enc_ipi_handler,
+ NULL, NULL);
+ if (status) {
+ mtk_vcodec_err(vpu, "vpu_ipi_register fail %d", status);
+ return -EINVAL;
+ }
+
+ memset(&out, 0, sizeof(out));
+ out.msg_id = AP_IPIMSG_ENC_INIT;
+ out.venc_inst = (unsigned long)vpu;
+ if (vpu_enc_send_msg(vpu, &out, sizeof(out))) {
+ mtk_vcodec_err(vpu, "AP_IPIMSG_ENC_INIT fail");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(vpu);
+
+ return 0;
+}
+
+int vpu_enc_set_param(struct venc_vpu_inst *vpu,
+ enum venc_set_param_type id,
+ struct venc_enc_param *enc_param)
+{
+ struct venc_ap_ipi_msg_set_param out;
+
+ mtk_vcodec_debug(vpu, "id %d ->", id);
+
+ memset(&out, 0, sizeof(out));
+ out.msg_id = AP_IPIMSG_ENC_SET_PARAM;
+ out.vpu_inst_addr = vpu->inst_addr;
+ out.param_id = id;
+ switch (id) {
+ case VENC_SET_PARAM_ENC:
+ out.data_item = 0;
+ break;
+ case VENC_SET_PARAM_FORCE_INTRA:
+ out.data_item = 0;
+ break;
+ case VENC_SET_PARAM_ADJUST_BITRATE:
+ out.data_item = 1;
+ out.data[0] = enc_param->bitrate;
+ break;
+ case VENC_SET_PARAM_ADJUST_FRAMERATE:
+ out.data_item = 1;
+ out.data[0] = enc_param->frm_rate;
+ break;
+ case VENC_SET_PARAM_GOP_SIZE:
+ out.data_item = 1;
+ out.data[0] = enc_param->gop_size;
+ break;
+ case VENC_SET_PARAM_INTRA_PERIOD:
+ out.data_item = 1;
+ out.data[0] = enc_param->intra_period;
+ break;
+ case VENC_SET_PARAM_SKIP_FRAME:
+ out.data_item = 0;
+ break;
+ default:
+ mtk_vcodec_err(vpu, "id %d not supported", id);
+ return -EINVAL;
+ }
+ if (vpu_enc_send_msg(vpu, &out, sizeof(out))) {
+ mtk_vcodec_err(vpu,
+ "AP_IPIMSG_ENC_SET_PARAM %d fail", id);
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug(vpu, "id %d <-", id);
+
+ return 0;
+}
+
+int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size)
+{
+ struct venc_ap_ipi_msg_enc out;
+
+ mtk_vcodec_debug(vpu, "bs_mode %d ->", bs_mode);
+
+ memset(&out, 0, sizeof(out));
+ out.msg_id = AP_IPIMSG_ENC_ENCODE;
+ out.vpu_inst_addr = vpu->inst_addr;
+ out.bs_mode = bs_mode;
+ if (frm_buf) {
+ if ((frm_buf->fb_addr[0].dma_addr % 16 == 0) &&
+ (frm_buf->fb_addr[1].dma_addr % 16 == 0) &&
+ (frm_buf->fb_addr[2].dma_addr % 16 == 0)) {
+ out.input_addr[0] = frm_buf->fb_addr[0].dma_addr;
+ out.input_addr[1] = frm_buf->fb_addr[1].dma_addr;
+ out.input_addr[2] = frm_buf->fb_addr[2].dma_addr;
+ } else {
+ mtk_vcodec_err(vpu, "dma_addr not align to 16");
+ return -EINVAL;
+ }
+ }
+ if (bs_buf) {
+ out.bs_addr = bs_buf->dma_addr;
+ out.bs_size = bs_buf->size;
+ }
+ if (vpu_enc_send_msg(vpu, &out, sizeof(out))) {
+ mtk_vcodec_err(vpu, "AP_IPIMSG_ENC_ENCODE %d fail",
+ bs_mode);
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug(vpu, "state %d size %d key_frm %d",
+ vpu->state, vpu->bs_size,
+ vpu->is_key_frm);
+
+ mtk_vcodec_debug(vpu, "bs_mode %d <-", bs_mode);
+
+ return 0;
+}
+
+int vpu_enc_deinit(struct venc_vpu_inst *vpu)
+{
+ struct venc_ap_ipi_msg_deinit out;
+
+ mtk_vcodec_debug_enter(vpu);
+
+ memset(&out, 0, sizeof(out));
+ out.msg_id = AP_IPIMSG_ENC_DEINIT;
+ out.vpu_inst_addr = vpu->inst_addr;
+ if (vpu_enc_send_msg(vpu, &out, sizeof(out))) {
+ mtk_vcodec_err(vpu, "AP_IPIMSG_ENC_DEINIT fail");
+ return -EINVAL;
+ }
+
+ mtk_vcodec_debug_leave(vpu);
+
+ return 0;
+}
diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
new file mode 100644
index 0000000..215d1e0
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
@@ -0,0 +1,61 @@
+/*
+ * Copyright (c) 2016 MediaTek Inc.
+ * Author: PoChun Lin <pochun.lin@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _VENC_VPU_IF_H_
+#define _VENC_VPU_IF_H_
+
+#include "mtk_vpu.h"
+#include "venc_drv_if.h"
+
+/*
+ * struct venc_vpu_inst - encoder VPU driver instance
+ * @wq_hd: wait queue used for vpu cmd trigger then wait vpu interrupt done
+ * @signaled: flag used for checking vpu interrupt done
+ * @failure: flag to show vpu cmd succeeds or not
+ * @state: enum venc_ipi_msg_enc_state
+ * @bs_size: bitstream size for skip frame case usage
+ * @is_key_frm: key frame flag
+ * @inst_addr: VPU instance addr
+ * @vsi: driver structure allocated by VPU side and shared to AP side for
+ * control and info share
+ * @id: the id of inter-processor interrupt
+ * @ctx: context for v4l2 layer integration
+ * @dev: device for v4l2 layer integration
+ */
+struct venc_vpu_inst {
+ wait_queue_head_t wq_hd;
+ int signaled;
+ int failure;
+ int state;
+ int bs_size;
+ int is_key_frm;
+ unsigned int inst_addr;
+ void *vsi;
+ enum ipi_id id;
+ struct mtk_vcodec_ctx *ctx;
+ struct platform_device *dev;
+};
+
+int vpu_enc_init(struct venc_vpu_inst *vpu);
+int vpu_enc_set_param(struct venc_vpu_inst *vpu,
+ enum venc_set_param_type id,
+ struct venc_enc_param *param);
+int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
+ struct venc_frm_buf *frm_buf,
+ struct mtk_vcodec_mem *bs_buf,
+ unsigned int *bs_size);
+int vpu_enc_deinit(struct venc_vpu_inst *vpu);
+
+#endif
--
1.7.9.5
[toc] | [prev] | [next] | [standalone]
| From | Hans Verkuil <hverkuil@xs4all.nl> |
|---|---|
| Date | 2016-04-22 15:50 +0200 |
| Subject | Re: [PATCH v7 5/8] [media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver |
| Message-ID | <rqJD5-2KJ-15@gated-at.bofh.it> |
| In reply to | #1384687 |
On 04/22/2016 06:25 AM, Tiffany Lin wrote:
> Add v4l2 layer encoder driver for MT8173
>
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
>
> ---
> drivers/media/platform/Kconfig | 16 +
> drivers/media/platform/Makefile | 2 +
> drivers/media/platform/mtk-vcodec/Makefile | 14 +
> drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h | 339 +++++
> drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1301 ++++++++++++++++++++
> drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h | 59 +
> .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 467 +++++++
> .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 137 +++
> .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 26 +
> .../media/platform/mtk-vcodec/mtk_vcodec_intr.c | 56 +
> .../media/platform/mtk-vcodec/mtk_vcodec_intr.h | 27 +
> .../media/platform/mtk-vcodec/mtk_vcodec_util.c | 96 ++
> .../media/platform/mtk-vcodec/mtk_vcodec_util.h | 87 ++
> drivers/media/platform/mtk-vcodec/venc_drv_base.h | 62 +
> drivers/media/platform/mtk-vcodec/venc_drv_if.c | 107 ++
> drivers/media/platform/mtk-vcodec/venc_drv_if.h | 165 +++
> drivers/media/platform/mtk-vcodec/venc_ipi_msg.h | 210 ++++
> 17 files changed, 3171 insertions(+)
> create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_base.h
> create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.c
> create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.h
> create mode 100644 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
>
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> new file mode 100644
> index 0000000..b2b662c
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> @@ -0,0 +1,1301 @@
> +/* V4L2 specification suggests the driver corrects the format struct if any of
> + * the dimensions is unsupported
> + */
> +static int vidioc_try_fmt(struct v4l2_format *f, struct mtk_video_fmt *fmt)
> +{
> + struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
> + int i;
> +
> + pix_fmt_mp->field = V4L2_FIELD_NONE;
> +
> + if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> + pix_fmt_mp->num_planes = 1;
> + pix_fmt_mp->plane_fmt[0].bytesperline = 0;
> + } else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> + int tmp_w, tmp_h;
> +
> + pix_fmt_mp->height = clamp(pix_fmt_mp->height,
> + MTK_VENC_MIN_H,
> + MTK_VENC_MAX_H);
> + pix_fmt_mp->width = clamp(pix_fmt_mp->width,
> + MTK_VENC_MIN_W,
> + MTK_VENC_MAX_W);
Strange indentation. I see this in more places. Can you check this and fix
where needed?
> +
> + /* find next closer width align 16, heign align 32, size align
> + * 64 rectangle
> + */
> + tmp_w = pix_fmt_mp->width;
> + tmp_h = pix_fmt_mp->height;
> + v4l_bound_align_image(&pix_fmt_mp->width,
> + MTK_VENC_MIN_W,
> + MTK_VENC_MAX_W, 4,
> + &pix_fmt_mp->height,
> + MTK_VENC_MIN_H,
> + MTK_VENC_MAX_H, 5, 6);
> +
...
> +static int vidioc_try_fmt_vid_out_mplane(struct file *file, void *priv,
> + struct v4l2_format *f)
> +{
> + struct mtk_video_fmt *fmt;
> +
> + fmt = mtk_venc_find_format(f);
> + if (!fmt) {
> + f->fmt.pix.pixelformat = mtk_video_formats[OUT_FMT_IDX].fourcc;
> + fmt = mtk_venc_find_format(f);
> + }
> + if (!f->fmt.pix_mp.colorspace)
> + f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_REC709;
> + f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> + f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> + f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
This should all be under the 'if'. If colorspace is 0, then you can assume that
whatever colorspace information userspace passes to the driver is wrong and
that is should be set to a proper default.
> +
> + return vidioc_try_fmt(f, fmt);
> +}
> +
...
> +static int fops_vcodec_open(struct file *file)
> +{
> + struct video_device *vfd = video_devdata(file);
> + struct mtk_vcodec_dev *dev = video_drvdata(file);
> + struct mtk_vcodec_ctx *ctx = NULL;
> + int ret = 0;
> +
> + if (dev->instance_mask == ~0UL) {
> + /* ffz Undefined if no zero exists, err handling here */
> + mtk_v4l2_err("Too many open contexts");
> + ret = -EBUSY;
> + goto err_alloc;
I'm not happy seeing this here. You should always be able to open the device.
I would expect to see a check like this in e.g. start_streaming, since that's
where you start to use the hardware for real, and checking if you have enough
resources there is perfectly fine.
If this is an artificial constraint (i.e. not based on a real hardware limitation),
then it should perhaps just be dropped. Such constraints tend to be pointless.
If you want to encode 20 streams simultaneously, then why not? It will be very
slow, but that's not this driver's problem :-)
> + }
> +
> + mutex_lock(&dev->dev_mutex);
> +
> + ctx = devm_kzalloc(&dev->plat_dev->dev, sizeof(*ctx), GFP_KERNEL);
Why is this a devm_ call? It is not managed by a device, so it seems to me that
a regular kzalloc is good enough here.
> + if (!ctx) {
> + ret = -ENOMEM;
> + goto err_alloc;
> + }
> +
> + ctx->idx = ffz(dev->instance_mask);
> + v4l2_fh_init(&ctx->fh, video_devdata(file));
> + file->private_data = &ctx->fh;
> + v4l2_fh_add(&ctx->fh);
> + INIT_LIST_HEAD(&ctx->list);
> + ctx->dev = dev;
> + init_waitqueue_head(&ctx->queue);
> +
> + if (vfd == dev->vfd_enc) {
> + ctx->type = MTK_INST_ENCODER;
> + ret = mtk_vcodec_enc_ctrls_setup(ctx);
> + if (ret) {
> + mtk_v4l2_err("Failed to setup controls() (%d)",
> + ret);
> + goto err_ctrls_setup;
> + }
> + ctx->m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev_enc, ctx,
> + &mtk_vcodec_enc_queue_init);
> + if (IS_ERR(ctx->m2m_ctx)) {
> + ret = PTR_ERR(ctx->m2m_ctx);
> + mtk_v4l2_err("Failed to v4l2_m2m_ctx_init() (%d)",
> + ret);
> + goto err_m2m_ctx_init;
> + }
> + mtk_vcodec_enc_set_default_params(ctx);
> + } else {
> + mtk_v4l2_err("Invalid vfd !");
This shouldn't be possible at all. I would just drop the 'if (vfd == dev->vfd_enc)' check.
> + ret = -ENOENT;
> + goto err_m2m_ctx_init;
> + }
> +
> + if (v4l2_fh_is_singular(&ctx->fh)) {
> + ret = vpu_load_firmware(dev->vpu_plat_dev);
> + if (ret < 0) {
> + /*
> + * Return 0 if downloading firmware successfully,
> + * otherwise it is failed
> + */
> + mtk_v4l2_err("vpu_load_firmware failed!");
> + goto err_load_fw;
> + }
The fw load seems to be a one-time thing, but here it is done every time
someone opens the device and nobody else had it open.
If this is a one time thing, then using a bool 'loaded_fw' makes more sense.
> +
> + dev->enc_capability =
> + vpu_get_venc_hw_capa(dev->vpu_plat_dev);
> + mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
> + }
> +
> + mtk_v4l2_debug(2, "Create instance [%d]@%p m2m_ctx=%p ",
> + ctx->idx, ctx, ctx->m2m_ctx);
> + set_bit(ctx->idx, &dev->instance_mask);
> + dev->num_instances++;
> + list_add(&ctx->list, &dev->ctx_list);
> +
> + mutex_unlock(&dev->dev_mutex);
> + mtk_v4l2_debug(0, "%s encoder [%d]", dev_name(&dev->plat_dev->dev),
> + ctx->idx);
> + return ret;
> +
> + /* Deinit when failure occurred */
> +err_load_fw:
> + v4l2_m2m_ctx_release(ctx->m2m_ctx);
> +err_m2m_ctx_init:
> + v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> +err_ctrls_setup:
> + v4l2_fh_del(&ctx->fh);
> + v4l2_fh_exit(&ctx->fh);
> + devm_kfree(&dev->plat_dev->dev, ctx);
> +err_alloc:
> + mutex_unlock(&dev->dev_mutex);
> +
> + return ret;
> +}
> +
> +static int fops_vcodec_release(struct file *file)
> +{
> + struct mtk_vcodec_dev *dev = video_drvdata(file);
> + struct mtk_vcodec_ctx *ctx = fh_to_ctx(file->private_data);
> +
> + mtk_v4l2_debug(1, "[%d] encoder", ctx->idx);
> + mutex_lock(&dev->dev_mutex);
> +
> + mtk_vcodec_enc_release(ctx);
> + v4l2_fh_del(&ctx->fh);
> + v4l2_fh_exit(&ctx->fh);
> + v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> + v4l2_m2m_ctx_release(ctx->m2m_ctx);
> +
> + list_del_init(&ctx->list);
> + dev->num_instances--;
> + clear_bit(ctx->idx, &dev->instance_mask);
> + devm_kfree(&dev->plat_dev->dev, ctx);
> + mutex_unlock(&dev->dev_mutex);
> + return 0;
> +}
> +
> +static const struct v4l2_file_operations mtk_vcodec_fops = {
> + .owner = THIS_MODULE,
> + .open = fops_vcodec_open,
> + .release = fops_vcodec_release,
> + .poll = v4l2_m2m_fop_poll,
> + .unlocked_ioctl = video_ioctl2,
> + .mmap = v4l2_m2m_fop_mmap,
> +};
> +
> +static int mtk_vcodec_probe(struct platform_device *pdev)
> +{
> + struct mtk_vcodec_dev *dev;
> + struct video_device *vfd_enc;
> + struct resource *res;
> + int i, j, ret;
> + DEFINE_DMA_ATTRS(attrs);
> +
> + dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> + if (!dev)
> + return -ENOMEM;
> +
> + INIT_LIST_HEAD(&dev->ctx_list);
> + dev->plat_dev = pdev;
> +
> + dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> + if (dev->vpu_plat_dev == NULL) {
> + mtk_v4l2_err("[VPU] vpu device in not ready");
> + return -EPROBE_DEFER;
> + }
> +
> + vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_enc_reset_handler,
> + dev, VPU_RST_ENC);
> +
> + ret = mtk_vcodec_init_enc_pm(dev);
> + if (ret < 0) {
> + dev_err(&pdev->dev, "Failed to get mt vcodec clock source!");
> + return ret;
> + }
> +
> + for (i = VENC_SYS, j = 0; i < NUM_MAX_VCODEC_REG_BASE; i++, j++) {
> + res = platform_get_resource(pdev, IORESOURCE_MEM, j);
> + if (res == NULL) {
> + dev_err(&pdev->dev, "get memory resource failed.");
> + ret = -ENXIO;
> + goto err_res;
> + }
> + dev->reg_base[i] = devm_ioremap_resource(&pdev->dev, res);
> + if (IS_ERR(dev->reg_base[i])) {
> + dev_err(&pdev->dev,
> + "devm_ioremap_resource %d failed.", i);
> + ret = PTR_ERR(dev->reg_base[i]);
> + goto err_res;
> + }
> + mtk_v4l2_debug(2, "reg[%d] base=0x%p", i, dev->reg_base[i]);
> + }
> +
> + res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> + if (res == NULL) {
> + dev_err(&pdev->dev, "failed to get irq resource");
> + ret = -ENOENT;
> + goto err_res;
> + }
> +
> + dev->enc_irq = platform_get_irq(pdev, 0);
> + ret = devm_request_irq(&pdev->dev, dev->enc_irq,
> + mtk_vcodec_enc_irq_handler,
> + 0, pdev->name, dev);
> + if (ret) {
> + dev_err(&pdev->dev, "Failed to install dev->enc_irq %d (%d)",
> + dev->enc_irq,
> + ret);
> + ret = -EINVAL;
> + goto err_res;
> + }
> +
> + dev->enc_lt_irq = platform_get_irq(pdev, 1);
> + ret = devm_request_irq(&pdev->dev,
> + dev->enc_lt_irq, mtk_vcodec_enc_lt_irq_handler,
> + 0, pdev->name, dev);
> + if (ret) {
> + dev_err(&pdev->dev,
> + "Failed to install dev->enc_lt_irq %d (%d)",
> + dev->enc_lt_irq, ret);
> + ret = -EINVAL;
> + goto err_res;
> + }
> +
> + disable_irq(dev->enc_irq);
> + disable_irq(dev->enc_lt_irq); /* VENC_LT */
> + mutex_init(&dev->enc_mutex);
> + mutex_init(&dev->dev_mutex);
> + spin_lock_init(&dev->irqlock);
> +
> + snprintf(dev->v4l2_dev.name, sizeof(dev->v4l2_dev.name), "%s",
> + "[MTK_V4L2_VENC]");
> +
> + ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> + if (ret) {
> + mtk_v4l2_err("v4l2_device_register err=%d", ret);
> + goto err_res;
> + }
> +
> + init_waitqueue_head(&dev->queue);
> +
> + /* allocate video device for encoder and register it */
> + vfd_enc = video_device_alloc();
> + if (!vfd_enc) {
> + mtk_v4l2_err("Failed to allocate video device");
> + ret = -ENOMEM;
> + goto err_enc_alloc;
> + }
> + vfd_enc->fops = &mtk_vcodec_fops;
> + vfd_enc->ioctl_ops = &mtk_venc_ioctl_ops;
> + vfd_enc->release = video_device_release;
> + vfd_enc->lock = &dev->dev_mutex;
> + vfd_enc->v4l2_dev = &dev->v4l2_dev;
> + vfd_enc->vfl_dir = VFL_DIR_M2M;
> + vfd_enc->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE |
> + V4L2_CAP_STREAMING;
> +
> + snprintf(vfd_enc->name, sizeof(vfd_enc->name), "%s",
> + MTK_VCODEC_ENC_NAME);
> + video_set_drvdata(vfd_enc, dev);
> + dev->vfd_enc = vfd_enc;
> + platform_set_drvdata(pdev, dev);
> +
> + dev->alloc_ctx = vb2_dma_contig_init_ctx(&pdev->dev);
> + if (IS_ERR(dev->alloc_ctx)) {
> + dev->alloc_ctx = NULL;
> + mtk_v4l2_err("Failed to alloc vb2 dma context 0");
> + ret = PTR_ERR(dev->alloc_ctx);
> + goto err_vb2_ctx_init;
> + }
> +
> + dev->m2m_dev_enc = v4l2_m2m_init(&mtk_venc_m2m_ops);
> + if (IS_ERR(dev->m2m_dev_enc)) {
> + mtk_v4l2_err("Failed to init mem2mem enc device");
> + ret = PTR_ERR(dev->m2m_dev_enc);
> + goto err_enc_mem_init;
> + }
> +
> + dev->encode_workqueue =
> + alloc_ordered_workqueue(MTK_VCODEC_ENC_NAME,
> + WQ_MEM_RECLAIM |
> + WQ_FREEZABLE);
> + if (!dev->encode_workqueue) {
> + mtk_v4l2_err("Failed to create encode workqueue");
> + ret = -EINVAL;
> + goto err_event_workq;
> + }
> +
> + ret = video_register_device(vfd_enc, VFL_TYPE_GRABBER, 1);
> + if (ret) {
> + mtk_v4l2_err("Failed to register video device");
> + goto err_enc_reg;
> + }
> +
> + /* Avoid the iommu eat big hunks */
> + dma_set_attr(DMA_ATTR_ALLOC_SINGLE_PAGES, &attrs);
> +
> + mtk_v4l2_debug(0, "encoder registered as /dev/video%d",
> + vfd_enc->num);
> +
> + return 0;
> +
> +err_enc_reg:
> + destroy_workqueue(dev->encode_workqueue);
> +err_event_workq:
> + v4l2_m2m_release(dev->m2m_dev_enc);
> +err_enc_mem_init:
> + vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> +err_vb2_ctx_init:
> + video_unregister_device(vfd_enc);
> +err_enc_alloc:
> + v4l2_device_unregister(&dev->v4l2_dev);
> +err_res:
> + mtk_vcodec_release_enc_pm(dev);
> + return ret;
> +}
> +
> +static const struct of_device_id mtk_vcodec_enc_match[] = {
> + {.compatible = "mediatek,mt8173-vcodec-enc",},
> + {},
> +};
> +MODULE_DEVICE_TABLE(of, mtk_vcodec_enc_match);
> +
> +static int mtk_vcodec_enc_remove(struct platform_device *pdev)
> +{
> + struct mtk_vcodec_dev *dev = platform_get_drvdata(pdev);
> +
> + mtk_v4l2_debug_enter();
> + flush_workqueue(dev->encode_workqueue);
> + destroy_workqueue(dev->encode_workqueue);
> + if (dev->m2m_dev_enc)
> + v4l2_m2m_release(dev->m2m_dev_enc);
> + if (dev->alloc_ctx)
> + vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> +
> + if (dev->vfd_enc)
> + video_unregister_device(dev->vfd_enc);
> +
> + v4l2_device_unregister(&dev->v4l2_dev);
> + mtk_vcodec_release_enc_pm(dev);
> + return 0;
> +}
> +
> +static struct platform_driver mtk_vcodec_enc_driver = {
> + .probe = mtk_vcodec_probe,
> + .remove = mtk_vcodec_enc_remove,
> + .driver = {
> + .name = MTK_VCODEC_ENC_NAME,
> + .owner = THIS_MODULE,
> + .of_match_table = mtk_vcodec_enc_match,
> + },
> +};
> +
> +module_platform_driver(mtk_vcodec_enc_driver);
> +
> +
> +MODULE_LICENSE("GPL v2");
> +MODULE_DESCRIPTION("Mediatek video codec V4L2 encoder driver");
Regards,
Hans
[toc] | [prev] | [next] | [standalone]
| From | tiffany lin <tiffany.lin@mediatek.com> |
|---|---|
| Date | 2016-04-25 07:20 +0200 |
| Subject | Re: [PATCH v7 5/8] [media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver |
| Message-ID | <rrH6a-kh-13@gated-at.bofh.it> |
| In reply to | #1385191 |
Hi Hans,
On Fri, 2016-04-22 at 15:47 +0200, Hans Verkuil wrote:
> On 04/22/2016 06:25 AM, Tiffany Lin wrote:
> > Add v4l2 layer encoder driver for MT8173
> >
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> >
> > ---
> > drivers/media/platform/Kconfig | 16 +
> > drivers/media/platform/Makefile | 2 +
> > drivers/media/platform/mtk-vcodec/Makefile | 14 +
> > drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h | 339 +++++
> > drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1301 ++++++++++++++++++++
> > drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h | 59 +
> > .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 467 +++++++
> > .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 137 +++
> > .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 26 +
> > .../media/platform/mtk-vcodec/mtk_vcodec_intr.c | 56 +
> > .../media/platform/mtk-vcodec/mtk_vcodec_intr.h | 27 +
> > .../media/platform/mtk-vcodec/mtk_vcodec_util.c | 96 ++
> > .../media/platform/mtk-vcodec/mtk_vcodec_util.h | 87 ++
> > drivers/media/platform/mtk-vcodec/venc_drv_base.h | 62 +
> > drivers/media/platform/mtk-vcodec/venc_drv_if.c | 107 ++
> > drivers/media/platform/mtk-vcodec/venc_drv_if.h | 165 +++
> > drivers/media/platform/mtk-vcodec/venc_ipi_msg.h | 210 ++++
> > 17 files changed, 3171 insertions(+)
> > create mode 100644 drivers/media/platform/mtk-vcodec/Makefile
> > create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> > create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
> > create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h
> > create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.c
> > create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_intr.h
> > create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> > create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
> > create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_base.h
> > create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.c
> > create mode 100644 drivers/media/platform/mtk-vcodec/venc_drv_if.h
> > create mode 100644 drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> >
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> > new file mode 100644
> > index 0000000..b2b662c
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> > @@ -0,0 +1,1301 @@
> > +/* V4L2 specification suggests the driver corrects the format struct if any of
> > + * the dimensions is unsupported
> > + */
> > +static int vidioc_try_fmt(struct v4l2_format *f, struct mtk_video_fmt *fmt)
> > +{
> > + struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
> > + int i;
> > +
> > + pix_fmt_mp->field = V4L2_FIELD_NONE;
> > +
> > + if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> > + pix_fmt_mp->num_planes = 1;
> > + pix_fmt_mp->plane_fmt[0].bytesperline = 0;
> > + } else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> > + int tmp_w, tmp_h;
> > +
> > + pix_fmt_mp->height = clamp(pix_fmt_mp->height,
> > + MTK_VENC_MIN_H,
> > + MTK_VENC_MAX_H);
> > + pix_fmt_mp->width = clamp(pix_fmt_mp->width,
> > + MTK_VENC_MIN_W,
> > + MTK_VENC_MAX_W);
>
> Strange indentation. I see this in more places. Can you check this and fix
> where needed?
>
I will fix all this strange indentation in next version.
> > +
> > + /* find next closer width align 16, heign align 32, size align
> > + * 64 rectangle
> > + */
> > + tmp_w = pix_fmt_mp->width;
> > + tmp_h = pix_fmt_mp->height;
> > + v4l_bound_align_image(&pix_fmt_mp->width,
> > + MTK_VENC_MIN_W,
> > + MTK_VENC_MAX_W, 4,
> > + &pix_fmt_mp->height,
> > + MTK_VENC_MIN_H,
> > + MTK_VENC_MAX_H, 5, 6);
> > +
>
> ...
>
> > +static int vidioc_try_fmt_vid_out_mplane(struct file *file, void *priv,
> > + struct v4l2_format *f)
> > +{
> > + struct mtk_video_fmt *fmt;
> > +
> > + fmt = mtk_venc_find_format(f);
> > + if (!fmt) {
> > + f->fmt.pix.pixelformat = mtk_video_formats[OUT_FMT_IDX].fourcc;
> > + fmt = mtk_venc_find_format(f);
> > + }
> > + if (!f->fmt.pix_mp.colorspace)
> > + f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_REC709;
> > + f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> > + f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> > + f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
>
> This should all be under the 'if'. If colorspace is 0, then you can assume that
> whatever colorspace information userspace passes to the driver is wrong and
> that is should be set to a proper default.
>
Got it, will fix this in next version.
> > +
> > + return vidioc_try_fmt(f, fmt);
> > +}
> > +
>
> ...
>
> > +static int fops_vcodec_open(struct file *file)
> > +{
> > + struct video_device *vfd = video_devdata(file);
> > + struct mtk_vcodec_dev *dev = video_drvdata(file);
> > + struct mtk_vcodec_ctx *ctx = NULL;
> > + int ret = 0;
> > +
> > + if (dev->instance_mask == ~0UL) {
> > + /* ffz Undefined if no zero exists, err handling here */
> > + mtk_v4l2_err("Too many open contexts");
> > + ret = -EBUSY;
> > + goto err_alloc;
>
> I'm not happy seeing this here. You should always be able to open the device.
> I would expect to see a check like this in e.g. start_streaming, since that's
> where you start to use the hardware for real, and checking if you have enough
> resources there is perfectly fine.
>
> If this is an artificial constraint (i.e. not based on a real hardware limitation),
> then it should perhaps just be dropped. Such constraints tend to be pointless.
> If you want to encode 20 streams simultaneously, then why not? It will be very
> slow, but that's not this driver's problem :-)
>
We use ffz to get instance index.
This only make sure that instance id is correct since in ffz
description,
"Undefined if no zero exists, so code should check against ~0UL first."
In this case, it may not be able to move to start_streaming.
Any suggestion that how we do this?
> > + }
> > +
> > + mutex_lock(&dev->dev_mutex);
> > +
> > + ctx = devm_kzalloc(&dev->plat_dev->dev, sizeof(*ctx), GFP_KERNEL);
>
> Why is this a devm_ call? It is not managed by a device, so it seems to me that
> a regular kzalloc is good enough here.
>
> > + if (!ctx) {
> > + ret = -ENOMEM;
> > + goto err_alloc;
> > + }
> > +
> > + ctx->idx = ffz(dev->instance_mask);
> > + v4l2_fh_init(&ctx->fh, video_devdata(file));
> > + file->private_data = &ctx->fh;
> > + v4l2_fh_add(&ctx->fh);
> > + INIT_LIST_HEAD(&ctx->list);
> > + ctx->dev = dev;
> > + init_waitqueue_head(&ctx->queue);
> > +
> > + if (vfd == dev->vfd_enc) {
> > + ctx->type = MTK_INST_ENCODER;
> > + ret = mtk_vcodec_enc_ctrls_setup(ctx);
> > + if (ret) {
> > + mtk_v4l2_err("Failed to setup controls() (%d)",
> > + ret);
> > + goto err_ctrls_setup;
> > + }
> > + ctx->m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev_enc, ctx,
> > + &mtk_vcodec_enc_queue_init);
> > + if (IS_ERR(ctx->m2m_ctx)) {
> > + ret = PTR_ERR(ctx->m2m_ctx);
> > + mtk_v4l2_err("Failed to v4l2_m2m_ctx_init() (%d)",
> > + ret);
> > + goto err_m2m_ctx_init;
> > + }
> > + mtk_vcodec_enc_set_default_params(ctx);
> > + } else {
> > + mtk_v4l2_err("Invalid vfd !");
>
> This shouldn't be possible at all. I would just drop the 'if (vfd == dev->vfd_enc)' check.
Got it, will remove in next version.
>
> > + ret = -ENOENT;
> > + goto err_m2m_ctx_init;
> > + }
> > +
> > + if (v4l2_fh_is_singular(&ctx->fh)) {
> > + ret = vpu_load_firmware(dev->vpu_plat_dev);
> > + if (ret < 0) {
> > + /*
> > + * Return 0 if downloading firmware successfully,
> > + * otherwise it is failed
> > + */
> > + mtk_v4l2_err("vpu_load_firmware failed!");
> > + goto err_load_fw;
> > + }
>
> The fw load seems to be a one-time thing, but here it is done every time
> someone opens the device and nobody else had it open.
>
> If this is a one time thing, then using a bool 'loaded_fw' makes more sense.
>
More than one module use vpu firmware, encoder/decoder/mdp...etc.
If this is first encode instance, we need to check and load vpu
firmware.
vpu_load_firmware will check and load firmware when necessary.
best regards,
Tiffany
> > +
> > + dev->enc_capability =
> > + vpu_get_venc_hw_capa(dev->vpu_plat_dev);
> > + mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
> > + }
> > +
> > + mtk_v4l2_debug(2, "Create instance [%d]@%p m2m_ctx=%p ",
> > + ctx->idx, ctx, ctx->m2m_ctx);
> > + set_bit(ctx->idx, &dev->instance_mask);
> > + dev->num_instances++;
> > + list_add(&ctx->list, &dev->ctx_list);
> > +
> > + mutex_unlock(&dev->dev_mutex);
> > + mtk_v4l2_debug(0, "%s encoder [%d]", dev_name(&dev->plat_dev->dev),
> > + ctx->idx);
> > + return ret;
> > +
> > + /* Deinit when failure occurred */
> > +err_load_fw:
> > + v4l2_m2m_ctx_release(ctx->m2m_ctx);
> > +err_m2m_ctx_init:
> > + v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> > +err_ctrls_setup:
> > + v4l2_fh_del(&ctx->fh);
> > + v4l2_fh_exit(&ctx->fh);
> > + devm_kfree(&dev->plat_dev->dev, ctx);
> > +err_alloc:
> > + mutex_unlock(&dev->dev_mutex);
> > +
> > + return ret;
> > +}
> > +
> > +static int fops_vcodec_release(struct file *file)
> > +{
> > + struct mtk_vcodec_dev *dev = video_drvdata(file);
> > + struct mtk_vcodec_ctx *ctx = fh_to_ctx(file->private_data);
> > +
> > + mtk_v4l2_debug(1, "[%d] encoder", ctx->idx);
> > + mutex_lock(&dev->dev_mutex);
> > +
> > + mtk_vcodec_enc_release(ctx);
> > + v4l2_fh_del(&ctx->fh);
> > + v4l2_fh_exit(&ctx->fh);
> > + v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> > + v4l2_m2m_ctx_release(ctx->m2m_ctx);
> > +
> > + list_del_init(&ctx->list);
> > + dev->num_instances--;
> > + clear_bit(ctx->idx, &dev->instance_mask);
> > + devm_kfree(&dev->plat_dev->dev, ctx);
> > + mutex_unlock(&dev->dev_mutex);
> > + return 0;
> > +}
> > +
> > +static const struct v4l2_file_operations mtk_vcodec_fops = {
> > + .owner = THIS_MODULE,
> > + .open = fops_vcodec_open,
> > + .release = fops_vcodec_release,
> > + .poll = v4l2_m2m_fop_poll,
> > + .unlocked_ioctl = video_ioctl2,
> > + .mmap = v4l2_m2m_fop_mmap,
> > +};
> > +
> > +static int mtk_vcodec_probe(struct platform_device *pdev)
> > +{
> > + struct mtk_vcodec_dev *dev;
> > + struct video_device *vfd_enc;
> > + struct resource *res;
> > + int i, j, ret;
> > + DEFINE_DMA_ATTRS(attrs);
> > +
> > + dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> > + if (!dev)
> > + return -ENOMEM;
> > +
> > + INIT_LIST_HEAD(&dev->ctx_list);
> > + dev->plat_dev = pdev;
> > +
> > + dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> > + if (dev->vpu_plat_dev == NULL) {
> > + mtk_v4l2_err("[VPU] vpu device in not ready");
> > + return -EPROBE_DEFER;
> > + }
> > +
> > + vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_enc_reset_handler,
> > + dev, VPU_RST_ENC);
> > +
> > + ret = mtk_vcodec_init_enc_pm(dev);
> > + if (ret < 0) {
> > + dev_err(&pdev->dev, "Failed to get mt vcodec clock source!");
> > + return ret;
> > + }
> > +
> > + for (i = VENC_SYS, j = 0; i < NUM_MAX_VCODEC_REG_BASE; i++, j++) {
> > + res = platform_get_resource(pdev, IORESOURCE_MEM, j);
> > + if (res == NULL) {
> > + dev_err(&pdev->dev, "get memory resource failed.");
> > + ret = -ENXIO;
> > + goto err_res;
> > + }
> > + dev->reg_base[i] = devm_ioremap_resource(&pdev->dev, res);
> > + if (IS_ERR(dev->reg_base[i])) {
> > + dev_err(&pdev->dev,
> > + "devm_ioremap_resource %d failed.", i);
> > + ret = PTR_ERR(dev->reg_base[i]);
> > + goto err_res;
> > + }
> > + mtk_v4l2_debug(2, "reg[%d] base=0x%p", i, dev->reg_base[i]);
> > + }
> > +
> > + res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> > + if (res == NULL) {
> > + dev_err(&pdev->dev, "failed to get irq resource");
> > + ret = -ENOENT;
> > + goto err_res;
> > + }
> > +
> > + dev->enc_irq = platform_get_irq(pdev, 0);
> > + ret = devm_request_irq(&pdev->dev, dev->enc_irq,
> > + mtk_vcodec_enc_irq_handler,
> > + 0, pdev->name, dev);
> > + if (ret) {
> > + dev_err(&pdev->dev, "Failed to install dev->enc_irq %d (%d)",
> > + dev->enc_irq,
> > + ret);
> > + ret = -EINVAL;
> > + goto err_res;
> > + }
> > +
> > + dev->enc_lt_irq = platform_get_irq(pdev, 1);
> > + ret = devm_request_irq(&pdev->dev,
> > + dev->enc_lt_irq, mtk_vcodec_enc_lt_irq_handler,
> > + 0, pdev->name, dev);
> > + if (ret) {
> > + dev_err(&pdev->dev,
> > + "Failed to install dev->enc_lt_irq %d (%d)",
> > + dev->enc_lt_irq, ret);
> > + ret = -EINVAL;
> > + goto err_res;
> > + }
> > +
> > + disable_irq(dev->enc_irq);
> > + disable_irq(dev->enc_lt_irq); /* VENC_LT */
> > + mutex_init(&dev->enc_mutex);
> > + mutex_init(&dev->dev_mutex);
> > + spin_lock_init(&dev->irqlock);
> > +
> > + snprintf(dev->v4l2_dev.name, sizeof(dev->v4l2_dev.name), "%s",
> > + "[MTK_V4L2_VENC]");
> > +
> > + ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> > + if (ret) {
> > + mtk_v4l2_err("v4l2_device_register err=%d", ret);
> > + goto err_res;
> > + }
> > +
> > + init_waitqueue_head(&dev->queue);
> > +
> > + /* allocate video device for encoder and register it */
> > + vfd_enc = video_device_alloc();
> > + if (!vfd_enc) {
> > + mtk_v4l2_err("Failed to allocate video device");
> > + ret = -ENOMEM;
> > + goto err_enc_alloc;
> > + }
> > + vfd_enc->fops = &mtk_vcodec_fops;
> > + vfd_enc->ioctl_ops = &mtk_venc_ioctl_ops;
> > + vfd_enc->release = video_device_release;
> > + vfd_enc->lock = &dev->dev_mutex;
> > + vfd_enc->v4l2_dev = &dev->v4l2_dev;
> > + vfd_enc->vfl_dir = VFL_DIR_M2M;
> > + vfd_enc->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE |
> > + V4L2_CAP_STREAMING;
> > +
> > + snprintf(vfd_enc->name, sizeof(vfd_enc->name), "%s",
> > + MTK_VCODEC_ENC_NAME);
> > + video_set_drvdata(vfd_enc, dev);
> > + dev->vfd_enc = vfd_enc;
> > + platform_set_drvdata(pdev, dev);
> > +
> > + dev->alloc_ctx = vb2_dma_contig_init_ctx(&pdev->dev);
> > + if (IS_ERR(dev->alloc_ctx)) {
> > + dev->alloc_ctx = NULL;
> > + mtk_v4l2_err("Failed to alloc vb2 dma context 0");
> > + ret = PTR_ERR(dev->alloc_ctx);
> > + goto err_vb2_ctx_init;
> > + }
> > +
> > + dev->m2m_dev_enc = v4l2_m2m_init(&mtk_venc_m2m_ops);
> > + if (IS_ERR(dev->m2m_dev_enc)) {
> > + mtk_v4l2_err("Failed to init mem2mem enc device");
> > + ret = PTR_ERR(dev->m2m_dev_enc);
> > + goto err_enc_mem_init;
> > + }
> > +
> > + dev->encode_workqueue =
> > + alloc_ordered_workqueue(MTK_VCODEC_ENC_NAME,
> > + WQ_MEM_RECLAIM |
> > + WQ_FREEZABLE);
> > + if (!dev->encode_workqueue) {
> > + mtk_v4l2_err("Failed to create encode workqueue");
> > + ret = -EINVAL;
> > + goto err_event_workq;
> > + }
> > +
> > + ret = video_register_device(vfd_enc, VFL_TYPE_GRABBER, 1);
> > + if (ret) {
> > + mtk_v4l2_err("Failed to register video device");
> > + goto err_enc_reg;
> > + }
> > +
> > + /* Avoid the iommu eat big hunks */
> > + dma_set_attr(DMA_ATTR_ALLOC_SINGLE_PAGES, &attrs);
> > +
> > + mtk_v4l2_debug(0, "encoder registered as /dev/video%d",
> > + vfd_enc->num);
> > +
> > + return 0;
> > +
> > +err_enc_reg:
> > + destroy_workqueue(dev->encode_workqueue);
> > +err_event_workq:
> > + v4l2_m2m_release(dev->m2m_dev_enc);
> > +err_enc_mem_init:
> > + vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> > +err_vb2_ctx_init:
> > + video_unregister_device(vfd_enc);
> > +err_enc_alloc:
> > + v4l2_device_unregister(&dev->v4l2_dev);
> > +err_res:
> > + mtk_vcodec_release_enc_pm(dev);
> > + return ret;
> > +}
> > +
> > +static const struct of_device_id mtk_vcodec_enc_match[] = {
> > + {.compatible = "mediatek,mt8173-vcodec-enc",},
> > + {},
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_vcodec_enc_match);
> > +
> > +static int mtk_vcodec_enc_remove(struct platform_device *pdev)
> > +{
> > + struct mtk_vcodec_dev *dev = platform_get_drvdata(pdev);
> > +
> > + mtk_v4l2_debug_enter();
> > + flush_workqueue(dev->encode_workqueue);
> > + destroy_workqueue(dev->encode_workqueue);
> > + if (dev->m2m_dev_enc)
> > + v4l2_m2m_release(dev->m2m_dev_enc);
> > + if (dev->alloc_ctx)
> > + vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> > +
> > + if (dev->vfd_enc)
> > + video_unregister_device(dev->vfd_enc);
> > +
> > + v4l2_device_unregister(&dev->v4l2_dev);
> > + mtk_vcodec_release_enc_pm(dev);
> > + return 0;
> > +}
> > +
> > +static struct platform_driver mtk_vcodec_enc_driver = {
> > + .probe = mtk_vcodec_probe,
> > + .remove = mtk_vcodec_enc_remove,
> > + .driver = {
> > + .name = MTK_VCODEC_ENC_NAME,
> > + .owner = THIS_MODULE,
> > + .of_match_table = mtk_vcodec_enc_match,
> > + },
> > +};
> > +
> > +module_platform_driver(mtk_vcodec_enc_driver);
> > +
> > +
> > +MODULE_LICENSE("GPL v2");
> > +MODULE_DESCRIPTION("Mediatek video codec V4L2 encoder driver");
>
> Regards,
>
> Hans
[toc] | [prev] | [next] | [standalone]
| From | Wu-Cheng Li (李務誠) <wuchengli@chromium.org> |
|---|---|
| Date | 2016-04-25 07:50 +0200 |
| Subject | Re: [PATCH v7 5/8] [media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver |
| Message-ID | <rrHzc-yx-7@gated-at.bofh.it> |
| In reply to | #1386010 |
> >
> > ...
> >
> > > +static int fops_vcodec_open(struct file *file)
> > > +{
> > > + struct video_device *vfd = video_devdata(file);
> > > + struct mtk_vcodec_dev *dev = video_drvdata(file);
> > > + struct mtk_vcodec_ctx *ctx = NULL;
> > > + int ret = 0;
> > > +
> > > + if (dev->instance_mask == ~0UL) {
> > > + /* ffz Undefined if no zero exists, err handling here */
> > > + mtk_v4l2_err("Too many open contexts");
> > > + ret = -EBUSY;
> > > + goto err_alloc;
> >
> > I'm not happy seeing this here. You should always be able to open the device.
> > I would expect to see a check like this in e.g. start_streaming, since that's
> > where you start to use the hardware for real, and checking if you have enough
> > resources there is perfectly fine.
> >
> > If this is an artificial constraint (i.e. not based on a real hardware limitation),
> > then it should perhaps just be dropped. Such constraints tend to be pointless.
> > If you want to encode 20 streams simultaneously, then why not? It will be very
> > slow, but that's not this driver's problem :-)
> >
> We use ffz to get instance index.
> This only make sure that instance id is correct since in ffz
> description,
> "Undefined if no zero exists, so code should check against ~0UL first."
> In this case, it may not be able to move to start_streaming.
> Any suggestion that how we do this?
The instance index is only used for printing the debug information.
No? If that's the case, you can remove instance index and print
mtk_vcodec_ctx address for debugging.
>
>
> > > + }
> > > +
> > > + mutex_lock(&dev->dev_mutex);
> > > +
> > > + ctx = devm_kzalloc(&dev->plat_dev->dev, sizeof(*ctx), GFP_KERNEL);
> >
> > Why is this a devm_ call? It is not managed by a device, so it seems to me that
> > a regular kzalloc is good enough here.
> >
> > > + if (!ctx) {
> > > + ret = -ENOMEM;
> > > + goto err_alloc;
> > > + }
> > > +
> > > + ctx->idx = ffz(dev->instance_mask);
> > > + v4l2_fh_init(&ctx->fh, video_devdata(file));
> > > + file->private_data = &ctx->fh;
> > > + v4l2_fh_add(&ctx->fh);
> > > + INIT_LIST_HEAD(&ctx->list);
> > > + ctx->dev = dev;
> > > + init_waitqueue_head(&ctx->queue);
> > > +
> > > + if (vfd == dev->vfd_enc) {
> > > + ctx->type = MTK_INST_ENCODER;
> > > + ret = mtk_vcodec_enc_ctrls_setup(ctx);
> > > + if (ret) {
> > > + mtk_v4l2_err("Failed to setup controls() (%d)",
> > > + ret);
> > > + goto err_ctrls_setup;
> > > + }
> > > + ctx->m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev_enc, ctx,
> > > + &mtk_vcodec_enc_queue_init);
> > > + if (IS_ERR(ctx->m2m_ctx)) {
> > > + ret = PTR_ERR(ctx->m2m_ctx);
> > > + mtk_v4l2_err("Failed to v4l2_m2m_ctx_init() (%d)",
> > > + ret);
> > > + goto err_m2m_ctx_init;
> > > + }
> > > + mtk_vcodec_enc_set_default_params(ctx);
> > > + } else {
> > > + mtk_v4l2_err("Invalid vfd !");
> >
> > This shouldn't be possible at all. I would just drop the 'if (vfd == dev->vfd_enc)' check.
> Got it, will remove in next version.
>
> >
> > > + ret = -ENOENT;
> > > + goto err_m2m_ctx_init;
> > > + }
> > > +
> > > + if (v4l2_fh_is_singular(&ctx->fh)) {
> > > + ret = vpu_load_firmware(dev->vpu_plat_dev);
> > > + if (ret < 0) {
> > > + /*
> > > + * Return 0 if downloading firmware successfully,
> > > + * otherwise it is failed
> > > + */
> > > + mtk_v4l2_err("vpu_load_firmware failed!");
> > > + goto err_load_fw;
> > > + }
> >
> > The fw load seems to be a one-time thing, but here it is done every time
> > someone opens the device and nobody else had it open.
> >
> > If this is a one time thing, then using a bool 'loaded_fw' makes more sense.
> >
> More than one module use vpu firmware, encoder/decoder/mdp...etc.
> If this is first encode instance, we need to check and load vpu
> firmware.
> vpu_load_firmware will check and load firmware when necessary.
>
> best regards,
> Tiffany
> > > +
> > > + dev->enc_capability =
> > > + vpu_get_venc_hw_capa(dev->vpu_plat_dev);
> > > + mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
> > > + }
> > > +
> > > + mtk_v4l2_debug(2, "Create instance [%d]@%p m2m_ctx=%p ",
> > > + ctx->idx, ctx, ctx->m2m_ctx);
> > > + set_bit(ctx->idx, &dev->instance_mask);
> > > + dev->num_instances++;
> > > + list_add(&ctx->list, &dev->ctx_list);
> > > +
> > > + mutex_unlock(&dev->dev_mutex);
> > > + mtk_v4l2_debug(0, "%s encoder [%d]", dev_name(&dev->plat_dev->dev),
> > > + ctx->idx);
> > > + return ret;
> > > +
> > > + /* Deinit when failure occurred */
> > > +err_load_fw:
> > > + v4l2_m2m_ctx_release(ctx->m2m_ctx);
> > > +err_m2m_ctx_init:
> > > + v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> > > +err_ctrls_setup:
> > > + v4l2_fh_del(&ctx->fh);
> > > + v4l2_fh_exit(&ctx->fh);
> > > + devm_kfree(&dev->plat_dev->dev, ctx);
> > > +err_alloc:
> > > + mutex_unlock(&dev->dev_mutex);
> > > +
> > > + return ret;
> > > +}
> > > +
> > > +static int fops_vcodec_release(struct file *file)
> > > +{
> > > + struct mtk_vcodec_dev *dev = video_drvdata(file);
> > > + struct mtk_vcodec_ctx *ctx = fh_to_ctx(file->private_data);
> > > +
> > > + mtk_v4l2_debug(1, "[%d] encoder", ctx->idx);
> > > + mutex_lock(&dev->dev_mutex);
> > > +
> > > + mtk_vcodec_enc_release(ctx);
> > > + v4l2_fh_del(&ctx->fh);
> > > + v4l2_fh_exit(&ctx->fh);
> > > + v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> > > + v4l2_m2m_ctx_release(ctx->m2m_ctx);
> > > +
> > > + list_del_init(&ctx->list);
> > > + dev->num_instances--;
> > > + clear_bit(ctx->idx, &dev->instance_mask);
> > > + devm_kfree(&dev->plat_dev->dev, ctx);
> > > + mutex_unlock(&dev->dev_mutex);
> > > + return 0;
> > > +}
> > > +
> > > +static const struct v4l2_file_operations mtk_vcodec_fops = {
> > > + .owner = THIS_MODULE,
> > > + .open = fops_vcodec_open,
> > > + .release = fops_vcodec_release,
> > > + .poll = v4l2_m2m_fop_poll,
> > > + .unlocked_ioctl = video_ioctl2,
> > > + .mmap = v4l2_m2m_fop_mmap,
> > > +};
> > > +
> > > +static int mtk_vcodec_probe(struct platform_device *pdev)
> > > +{
> > > + struct mtk_vcodec_dev *dev;
> > > + struct video_device *vfd_enc;
> > > + struct resource *res;
> > > + int i, j, ret;
> > > + DEFINE_DMA_ATTRS(attrs);
> > > +
> > > + dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> > > + if (!dev)
> > > + return -ENOMEM;
> > > +
> > > + INIT_LIST_HEAD(&dev->ctx_list);
> > > + dev->plat_dev = pdev;
> > > +
> > > + dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> > > + if (dev->vpu_plat_dev == NULL) {
> > > + mtk_v4l2_err("[VPU] vpu device in not ready");
> > > + return -EPROBE_DEFER;
> > > + }
> > > +
> > > + vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_enc_reset_handler,
> > > + dev, VPU_RST_ENC);
> > > +
> > > + ret = mtk_vcodec_init_enc_pm(dev);
> > > + if (ret < 0) {
> > > + dev_err(&pdev->dev, "Failed to get mt vcodec clock source!");
> > > + return ret;
> > > + }
> > > +
> > > + for (i = VENC_SYS, j = 0; i < NUM_MAX_VCODEC_REG_BASE; i++, j++) {
> > > + res = platform_get_resource(pdev, IORESOURCE_MEM, j);
> > > + if (res == NULL) {
> > > + dev_err(&pdev->dev, "get memory resource failed.");
> > > + ret = -ENXIO;
> > > + goto err_res;
> > > + }
> > > + dev->reg_base[i] = devm_ioremap_resource(&pdev->dev, res);
> > > + if (IS_ERR(dev->reg_base[i])) {
> > > + dev_err(&pdev->dev,
> > > + "devm_ioremap_resource %d failed.", i);
> > > + ret = PTR_ERR(dev->reg_base[i]);
> > > + goto err_res;
> > > + }
> > > + mtk_v4l2_debug(2, "reg[%d] base=0x%p", i, dev->reg_base[i]);
> > > + }
> > > +
> > > + res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> > > + if (res == NULL) {
> > > + dev_err(&pdev->dev, "failed to get irq resource");
> > > + ret = -ENOENT;
> > > + goto err_res;
> > > + }
> > > +
> > > + dev->enc_irq = platform_get_irq(pdev, 0);
> > > + ret = devm_request_irq(&pdev->dev, dev->enc_irq,
> > > + mtk_vcodec_enc_irq_handler,
> > > + 0, pdev->name, dev);
> > > + if (ret) {
> > > + dev_err(&pdev->dev, "Failed to install dev->enc_irq %d (%d)",
> > > + dev->enc_irq,
> > > + ret);
> > > + ret = -EINVAL;
> > > + goto err_res;
> > > + }
> > > +
> > > + dev->enc_lt_irq = platform_get_irq(pdev, 1);
> > > + ret = devm_request_irq(&pdev->dev,
> > > + dev->enc_lt_irq, mtk_vcodec_enc_lt_irq_handler,
> > > + 0, pdev->name, dev);
> > > + if (ret) {
> > > + dev_err(&pdev->dev,
> > > + "Failed to install dev->enc_lt_irq %d (%d)",
> > > + dev->enc_lt_irq, ret);
> > > + ret = -EINVAL;
> > > + goto err_res;
> > > + }
> > > +
> > > + disable_irq(dev->enc_irq);
> > > + disable_irq(dev->enc_lt_irq); /* VENC_LT */
> > > + mutex_init(&dev->enc_mutex);
> > > + mutex_init(&dev->dev_mutex);
> > > + spin_lock_init(&dev->irqlock);
> > > +
> > > + snprintf(dev->v4l2_dev.name, sizeof(dev->v4l2_dev.name), "%s",
> > > + "[MTK_V4L2_VENC]");
> > > +
> > > + ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> > > + if (ret) {
> > > + mtk_v4l2_err("v4l2_device_register err=%d", ret);
> > > + goto err_res;
> > > + }
> > > +
> > > + init_waitqueue_head(&dev->queue);
> > > +
> > > + /* allocate video device for encoder and register it */
> > > + vfd_enc = video_device_alloc();
> > > + if (!vfd_enc) {
> > > + mtk_v4l2_err("Failed to allocate video device");
> > > + ret = -ENOMEM;
> > > + goto err_enc_alloc;
> > > + }
> > > + vfd_enc->fops = &mtk_vcodec_fops;
> > > + vfd_enc->ioctl_ops = &mtk_venc_ioctl_ops;
> > > + vfd_enc->release = video_device_release;
> > > + vfd_enc->lock = &dev->dev_mutex;
> > > + vfd_enc->v4l2_dev = &dev->v4l2_dev;
> > > + vfd_enc->vfl_dir = VFL_DIR_M2M;
> > > + vfd_enc->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE |
> > > + V4L2_CAP_STREAMING;
> > > +
> > > + snprintf(vfd_enc->name, sizeof(vfd_enc->name), "%s",
> > > + MTK_VCODEC_ENC_NAME);
> > > + video_set_drvdata(vfd_enc, dev);
> > > + dev->vfd_enc = vfd_enc;
> > > + platform_set_drvdata(pdev, dev);
> > > +
> > > + dev->alloc_ctx = vb2_dma_contig_init_ctx(&pdev->dev);
> > > + if (IS_ERR(dev->alloc_ctx)) {
> > > + dev->alloc_ctx = NULL;
> > > + mtk_v4l2_err("Failed to alloc vb2 dma context 0");
> > > + ret = PTR_ERR(dev->alloc_ctx);
> > > + goto err_vb2_ctx_init;
> > > + }
> > > +
> > > + dev->m2m_dev_enc = v4l2_m2m_init(&mtk_venc_m2m_ops);
> > > + if (IS_ERR(dev->m2m_dev_enc)) {
> > > + mtk_v4l2_err("Failed to init mem2mem enc device");
> > > + ret = PTR_ERR(dev->m2m_dev_enc);
> > > + goto err_enc_mem_init;
> > > + }
> > > +
> > > + dev->encode_workqueue =
> > > + alloc_ordered_workqueue(MTK_VCODEC_ENC_NAME,
> > > + WQ_MEM_RECLAIM |
> > > + WQ_FREEZABLE);
> > > + if (!dev->encode_workqueue) {
> > > + mtk_v4l2_err("Failed to create encode workqueue");
> > > + ret = -EINVAL;
> > > + goto err_event_workq;
> > > + }
> > > +
> > > + ret = video_register_device(vfd_enc, VFL_TYPE_GRABBER, 1);
> > > + if (ret) {
> > > + mtk_v4l2_err("Failed to register video device");
> > > + goto err_enc_reg;
> > > + }
> > > +
> > > + /* Avoid the iommu eat big hunks */
> > > + dma_set_attr(DMA_ATTR_ALLOC_SINGLE_PAGES, &attrs);
> > > +
> > > + mtk_v4l2_debug(0, "encoder registered as /dev/video%d",
> > > + vfd_enc->num);
> > > +
> > > + return 0;
> > > +
> > > +err_enc_reg:
> > > + destroy_workqueue(dev->encode_workqueue);
> > > +err_event_workq:
> > > + v4l2_m2m_release(dev->m2m_dev_enc);
> > > +err_enc_mem_init:
> > > + vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> > > +err_vb2_ctx_init:
> > > + video_unregister_device(vfd_enc);
> > > +err_enc_alloc:
> > > + v4l2_device_unregister(&dev->v4l2_dev);
> > > +err_res:
> > > + mtk_vcodec_release_enc_pm(dev);
> > > + return ret;
> > > +}
> > > +
> > > +static const struct of_device_id mtk_vcodec_enc_match[] = {
> > > + {.compatible = "mediatek,mt8173-vcodec-enc",},
> > > + {},
> > > +};
> > > +MODULE_DEVICE_TABLE(of, mtk_vcodec_enc_match);
> > > +
> > > +static int mtk_vcodec_enc_remove(struct platform_device *pdev)
> > > +{
> > > + struct mtk_vcodec_dev *dev = platform_get_drvdata(pdev);
> > > +
> > > + mtk_v4l2_debug_enter();
> > > + flush_workqueue(dev->encode_workqueue);
> > > + destroy_workqueue(dev->encode_workqueue);
> > > + if (dev->m2m_dev_enc)
> > > + v4l2_m2m_release(dev->m2m_dev_enc);
> > > + if (dev->alloc_ctx)
> > > + vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> > > +
> > > + if (dev->vfd_enc)
> > > + video_unregister_device(dev->vfd_enc);
> > > +
> > > + v4l2_device_unregister(&dev->v4l2_dev);
> > > + mtk_vcodec_release_enc_pm(dev);
> > > + return 0;
> > > +}
> > > +
> > > +static struct platform_driver mtk_vcodec_enc_driver = {
> > > + .probe = mtk_vcodec_probe,
> > > + .remove = mtk_vcodec_enc_remove,
> > > + .driver = {
> > > + .name = MTK_VCODEC_ENC_NAME,
> > > + .owner = THIS_MODULE,
> > > + .of_match_table = mtk_vcodec_enc_match,
> > > + },
> > > +};
> > > +
> > > +module_platform_driver(mtk_vcodec_enc_driver);
> > > +
> > > +
> > > +MODULE_LICENSE("GPL v2");
> > > +MODULE_DESCRIPTION("Mediatek video codec V4L2 encoder driver");
> >
> > Regards,
> >
> > Hans
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[toc] | [prev] | [next] | [standalone]
| From | tiffany lin <tiffany.lin@mediatek.com> |
|---|---|
| Date | 2016-04-25 09:10 +0200 |
| Subject | Re: [PATCH v7 5/8] [media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver |
| Message-ID | <rrIOC-1Du-3@gated-at.bofh.it> |
| In reply to | #1386024 |
Hi Wucheng,
On Mon, 2016-04-25 at 13:42 +0800, Wu-Cheng Li (李務誠) wrote:
> > >
> > > ...
> > >
> > > > +static int fops_vcodec_open(struct file *file)
> > > > +{
> > > > + struct video_device *vfd = video_devdata(file);
> > > > + struct mtk_vcodec_dev *dev = video_drvdata(file);
> > > > + struct mtk_vcodec_ctx *ctx = NULL;
> > > > + int ret = 0;
> > > > +
> > > > + if (dev->instance_mask == ~0UL) {
> > > > + /* ffz Undefined if no zero exists, err handling here */
> > > > + mtk_v4l2_err("Too many open contexts");
> > > > + ret = -EBUSY;
> > > > + goto err_alloc;
> > >
> > > I'm not happy seeing this here. You should always be able to open the device.
> > > I would expect to see a check like this in e.g. start_streaming, since that's
> > > where you start to use the hardware for real, and checking if you have enough
> > > resources there is perfectly fine.
> > >
> > > If this is an artificial constraint (i.e. not based on a real hardware limitation),
> > > then it should perhaps just be dropped. Such constraints tend to be pointless.
> > > If you want to encode 20 streams simultaneously, then why not? It will be very
> > > slow, but that's not this driver's problem :-)
> > >
> > We use ffz to get instance index.
> > This only make sure that instance id is correct since in ffz
> > description,
> > "Undefined if no zero exists, so code should check against ~0UL first."
> > In this case, it may not be able to move to start_streaming.
> > Any suggestion that how we do this?
> The instance index is only used for printing the debug information.
> No? If that's the case, you can remove instance index and print
> mtk_vcodec_ctx address for debugging.
> >
Got it, will remove it in next version.
best regards,
Tiffany
> >
> > > > + }
> > > > +
> > > > + mutex_lock(&dev->dev_mutex);
> > > > +
> > > > + ctx = devm_kzalloc(&dev->plat_dev->dev, sizeof(*ctx), GFP_KERNEL);
> > >
> > > Why is this a devm_ call? It is not managed by a device, so it seems to me that
> > > a regular kzalloc is good enough here.
> > >
> > > > + if (!ctx) {
> > > > + ret = -ENOMEM;
> > > > + goto err_alloc;
> > > > + }
> > > > +
> > > > + ctx->idx = ffz(dev->instance_mask);
> > > > + v4l2_fh_init(&ctx->fh, video_devdata(file));
> > > > + file->private_data = &ctx->fh;
> > > > + v4l2_fh_add(&ctx->fh);
> > > > + INIT_LIST_HEAD(&ctx->list);
> > > > + ctx->dev = dev;
> > > > + init_waitqueue_head(&ctx->queue);
> > > > +
> > > > + if (vfd == dev->vfd_enc) {
> > > > + ctx->type = MTK_INST_ENCODER;
> > > > + ret = mtk_vcodec_enc_ctrls_setup(ctx);
> > > > + if (ret) {
> > > > + mtk_v4l2_err("Failed to setup controls() (%d)",
> > > > + ret);
> > > > + goto err_ctrls_setup;
> > > > + }
> > > > + ctx->m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev_enc, ctx,
> > > > + &mtk_vcodec_enc_queue_init);
> > > > + if (IS_ERR(ctx->m2m_ctx)) {
> > > > + ret = PTR_ERR(ctx->m2m_ctx);
> > > > + mtk_v4l2_err("Failed to v4l2_m2m_ctx_init() (%d)",
> > > > + ret);
> > > > + goto err_m2m_ctx_init;
> > > > + }
> > > > + mtk_vcodec_enc_set_default_params(ctx);
> > > > + } else {
> > > > + mtk_v4l2_err("Invalid vfd !");
> > >
> > > This shouldn't be possible at all. I would just drop the 'if (vfd == dev->vfd_enc)' check.
> > Got it, will remove in next version.
> >
> > >
> > > > + ret = -ENOENT;
> > > > + goto err_m2m_ctx_init;
> > > > + }
> > > > +
> > > > + if (v4l2_fh_is_singular(&ctx->fh)) {
> > > > + ret = vpu_load_firmware(dev->vpu_plat_dev);
> > > > + if (ret < 0) {
> > > > + /*
> > > > + * Return 0 if downloading firmware successfully,
> > > > + * otherwise it is failed
> > > > + */
> > > > + mtk_v4l2_err("vpu_load_firmware failed!");
> > > > + goto err_load_fw;
> > > > + }
> > >
> > > The fw load seems to be a one-time thing, but here it is done every time
> > > someone opens the device and nobody else had it open.
> > >
> > > If this is a one time thing, then using a bool 'loaded_fw' makes more sense.
> > >
> > More than one module use vpu firmware, encoder/decoder/mdp...etc.
> > If this is first encode instance, we need to check and load vpu
> > firmware.
> > vpu_load_firmware will check and load firmware when necessary.
> >
> > best regards,
> > Tiffany
> > > > +
> > > > + dev->enc_capability =
> > > > + vpu_get_venc_hw_capa(dev->vpu_plat_dev);
> > > > + mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
> > > > + }
> > > > +
> > > > + mtk_v4l2_debug(2, "Create instance [%d]@%p m2m_ctx=%p ",
> > > > + ctx->idx, ctx, ctx->m2m_ctx);
> > > > + set_bit(ctx->idx, &dev->instance_mask);
> > > > + dev->num_instances++;
> > > > + list_add(&ctx->list, &dev->ctx_list);
> > > > +
> > > > + mutex_unlock(&dev->dev_mutex);
> > > > + mtk_v4l2_debug(0, "%s encoder [%d]", dev_name(&dev->plat_dev->dev),
> > > > + ctx->idx);
> > > > + return ret;
> > > > +
> > > > + /* Deinit when failure occurred */
> > > > +err_load_fw:
> > > > + v4l2_m2m_ctx_release(ctx->m2m_ctx);
> > > > +err_m2m_ctx_init:
> > > > + v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> > > > +err_ctrls_setup:
> > > > + v4l2_fh_del(&ctx->fh);
> > > > + v4l2_fh_exit(&ctx->fh);
> > > > + devm_kfree(&dev->plat_dev->dev, ctx);
> > > > +err_alloc:
> > > > + mutex_unlock(&dev->dev_mutex);
> > > > +
> > > > + return ret;
> > > > +}
> > > > +
> > > > +static int fops_vcodec_release(struct file *file)
> > > > +{
> > > > + struct mtk_vcodec_dev *dev = video_drvdata(file);
> > > > + struct mtk_vcodec_ctx *ctx = fh_to_ctx(file->private_data);
> > > > +
> > > > + mtk_v4l2_debug(1, "[%d] encoder", ctx->idx);
> > > > + mutex_lock(&dev->dev_mutex);
> > > > +
> > > > + mtk_vcodec_enc_release(ctx);
> > > > + v4l2_fh_del(&ctx->fh);
> > > > + v4l2_fh_exit(&ctx->fh);
> > > > + v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
> > > > + v4l2_m2m_ctx_release(ctx->m2m_ctx);
> > > > +
> > > > + list_del_init(&ctx->list);
> > > > + dev->num_instances--;
> > > > + clear_bit(ctx->idx, &dev->instance_mask);
> > > > + devm_kfree(&dev->plat_dev->dev, ctx);
> > > > + mutex_unlock(&dev->dev_mutex);
> > > > + return 0;
> > > > +}
> > > > +
> > > > +static const struct v4l2_file_operations mtk_vcodec_fops = {
> > > > + .owner = THIS_MODULE,
> > > > + .open = fops_vcodec_open,
> > > > + .release = fops_vcodec_release,
> > > > + .poll = v4l2_m2m_fop_poll,
> > > > + .unlocked_ioctl = video_ioctl2,
> > > > + .mmap = v4l2_m2m_fop_mmap,
> > > > +};
> > > > +
> > > > +static int mtk_vcodec_probe(struct platform_device *pdev)
> > > > +{
> > > > + struct mtk_vcodec_dev *dev;
> > > > + struct video_device *vfd_enc;
> > > > + struct resource *res;
> > > > + int i, j, ret;
> > > > + DEFINE_DMA_ATTRS(attrs);
> > > > +
> > > > + dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> > > > + if (!dev)
> > > > + return -ENOMEM;
> > > > +
> > > > + INIT_LIST_HEAD(&dev->ctx_list);
> > > > + dev->plat_dev = pdev;
> > > > +
> > > > + dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> > > > + if (dev->vpu_plat_dev == NULL) {
> > > > + mtk_v4l2_err("[VPU] vpu device in not ready");
> > > > + return -EPROBE_DEFER;
> > > > + }
> > > > +
> > > > + vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_enc_reset_handler,
> > > > + dev, VPU_RST_ENC);
> > > > +
> > > > + ret = mtk_vcodec_init_enc_pm(dev);
> > > > + if (ret < 0) {
> > > > + dev_err(&pdev->dev, "Failed to get mt vcodec clock source!");
> > > > + return ret;
> > > > + }
> > > > +
> > > > + for (i = VENC_SYS, j = 0; i < NUM_MAX_VCODEC_REG_BASE; i++, j++) {
> > > > + res = platform_get_resource(pdev, IORESOURCE_MEM, j);
> > > > + if (res == NULL) {
> > > > + dev_err(&pdev->dev, "get memory resource failed.");
> > > > + ret = -ENXIO;
> > > > + goto err_res;
> > > > + }
> > > > + dev->reg_base[i] = devm_ioremap_resource(&pdev->dev, res);
> > > > + if (IS_ERR(dev->reg_base[i])) {
> > > > + dev_err(&pdev->dev,
> > > > + "devm_ioremap_resource %d failed.", i);
> > > > + ret = PTR_ERR(dev->reg_base[i]);
> > > > + goto err_res;
> > > > + }
> > > > + mtk_v4l2_debug(2, "reg[%d] base=0x%p", i, dev->reg_base[i]);
> > > > + }
> > > > +
> > > > + res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
> > > > + if (res == NULL) {
> > > > + dev_err(&pdev->dev, "failed to get irq resource");
> > > > + ret = -ENOENT;
> > > > + goto err_res;
> > > > + }
> > > > +
> > > > + dev->enc_irq = platform_get_irq(pdev, 0);
> > > > + ret = devm_request_irq(&pdev->dev, dev->enc_irq,
> > > > + mtk_vcodec_enc_irq_handler,
> > > > + 0, pdev->name, dev);
> > > > + if (ret) {
> > > > + dev_err(&pdev->dev, "Failed to install dev->enc_irq %d (%d)",
> > > > + dev->enc_irq,
> > > > + ret);
> > > > + ret = -EINVAL;
> > > > + goto err_res;
> > > > + }
> > > > +
> > > > + dev->enc_lt_irq = platform_get_irq(pdev, 1);
> > > > + ret = devm_request_irq(&pdev->dev,
> > > > + dev->enc_lt_irq, mtk_vcodec_enc_lt_irq_handler,
> > > > + 0, pdev->name, dev);
> > > > + if (ret) {
> > > > + dev_err(&pdev->dev,
> > > > + "Failed to install dev->enc_lt_irq %d (%d)",
> > > > + dev->enc_lt_irq, ret);
> > > > + ret = -EINVAL;
> > > > + goto err_res;
> > > > + }
> > > > +
> > > > + disable_irq(dev->enc_irq);
> > > > + disable_irq(dev->enc_lt_irq); /* VENC_LT */
> > > > + mutex_init(&dev->enc_mutex);
> > > > + mutex_init(&dev->dev_mutex);
> > > > + spin_lock_init(&dev->irqlock);
> > > > +
> > > > + snprintf(dev->v4l2_dev.name, sizeof(dev->v4l2_dev.name), "%s",
> > > > + "[MTK_V4L2_VENC]");
> > > > +
> > > > + ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> > > > + if (ret) {
> > > > + mtk_v4l2_err("v4l2_device_register err=%d", ret);
> > > > + goto err_res;
> > > > + }
> > > > +
> > > > + init_waitqueue_head(&dev->queue);
> > > > +
> > > > + /* allocate video device for encoder and register it */
> > > > + vfd_enc = video_device_alloc();
> > > > + if (!vfd_enc) {
> > > > + mtk_v4l2_err("Failed to allocate video device");
> > > > + ret = -ENOMEM;
> > > > + goto err_enc_alloc;
> > > > + }
> > > > + vfd_enc->fops = &mtk_vcodec_fops;
> > > > + vfd_enc->ioctl_ops = &mtk_venc_ioctl_ops;
> > > > + vfd_enc->release = video_device_release;
> > > > + vfd_enc->lock = &dev->dev_mutex;
> > > > + vfd_enc->v4l2_dev = &dev->v4l2_dev;
> > > > + vfd_enc->vfl_dir = VFL_DIR_M2M;
> > > > + vfd_enc->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE |
> > > > + V4L2_CAP_STREAMING;
> > > > +
> > > > + snprintf(vfd_enc->name, sizeof(vfd_enc->name), "%s",
> > > > + MTK_VCODEC_ENC_NAME);
> > > > + video_set_drvdata(vfd_enc, dev);
> > > > + dev->vfd_enc = vfd_enc;
> > > > + platform_set_drvdata(pdev, dev);
> > > > +
> > > > + dev->alloc_ctx = vb2_dma_contig_init_ctx(&pdev->dev);
> > > > + if (IS_ERR(dev->alloc_ctx)) {
> > > > + dev->alloc_ctx = NULL;
> > > > + mtk_v4l2_err("Failed to alloc vb2 dma context 0");
> > > > + ret = PTR_ERR(dev->alloc_ctx);
> > > > + goto err_vb2_ctx_init;
> > > > + }
> > > > +
> > > > + dev->m2m_dev_enc = v4l2_m2m_init(&mtk_venc_m2m_ops);
> > > > + if (IS_ERR(dev->m2m_dev_enc)) {
> > > > + mtk_v4l2_err("Failed to init mem2mem enc device");
> > > > + ret = PTR_ERR(dev->m2m_dev_enc);
> > > > + goto err_enc_mem_init;
> > > > + }
> > > > +
> > > > + dev->encode_workqueue =
> > > > + alloc_ordered_workqueue(MTK_VCODEC_ENC_NAME,
> > > > + WQ_MEM_RECLAIM |
> > > > + WQ_FREEZABLE);
> > > > + if (!dev->encode_workqueue) {
> > > > + mtk_v4l2_err("Failed to create encode workqueue");
> > > > + ret = -EINVAL;
> > > > + goto err_event_workq;
> > > > + }
> > > > +
> > > > + ret = video_register_device(vfd_enc, VFL_TYPE_GRABBER, 1);
> > > > + if (ret) {
> > > > + mtk_v4l2_err("Failed to register video device");
> > > > + goto err_enc_reg;
> > > > + }
> > > > +
> > > > + /* Avoid the iommu eat big hunks */
> > > > + dma_set_attr(DMA_ATTR_ALLOC_SINGLE_PAGES, &attrs);
> > > > +
> > > > + mtk_v4l2_debug(0, "encoder registered as /dev/video%d",
> > > > + vfd_enc->num);
> > > > +
> > > > + return 0;
> > > > +
> > > > +err_enc_reg:
> > > > + destroy_workqueue(dev->encode_workqueue);
> > > > +err_event_workq:
> > > > + v4l2_m2m_release(dev->m2m_dev_enc);
> > > > +err_enc_mem_init:
> > > > + vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> > > > +err_vb2_ctx_init:
> > > > + video_unregister_device(vfd_enc);
> > > > +err_enc_alloc:
> > > > + v4l2_device_unregister(&dev->v4l2_dev);
> > > > +err_res:
> > > > + mtk_vcodec_release_enc_pm(dev);
> > > > + return ret;
> > > > +}
> > > > +
> > > > +static const struct of_device_id mtk_vcodec_enc_match[] = {
> > > > + {.compatible = "mediatek,mt8173-vcodec-enc",},
> > > > + {},
> > > > +};
> > > > +MODULE_DEVICE_TABLE(of, mtk_vcodec_enc_match);
> > > > +
> > > > +static int mtk_vcodec_enc_remove(struct platform_device *pdev)
> > > > +{
> > > > + struct mtk_vcodec_dev *dev = platform_get_drvdata(pdev);
> > > > +
> > > > + mtk_v4l2_debug_enter();
> > > > + flush_workqueue(dev->encode_workqueue);
> > > > + destroy_workqueue(dev->encode_workqueue);
> > > > + if (dev->m2m_dev_enc)
> > > > + v4l2_m2m_release(dev->m2m_dev_enc);
> > > > + if (dev->alloc_ctx)
> > > > + vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
> > > > +
> > > > + if (dev->vfd_enc)
> > > > + video_unregister_device(dev->vfd_enc);
> > > > +
> > > > + v4l2_device_unregister(&dev->v4l2_dev);
> > > > + mtk_vcodec_release_enc_pm(dev);
> > > > + return 0;
> > > > +}
> > > > +
> > > > +static struct platform_driver mtk_vcodec_enc_driver = {
> > > > + .probe = mtk_vcodec_probe,
> > > > + .remove = mtk_vcodec_enc_remove,
> > > > + .driver = {
> > > > + .name = MTK_VCODEC_ENC_NAME,
> > > > + .owner = THIS_MODULE,
> > > > + .of_match_table = mtk_vcodec_enc_match,
> > > > + },
> > > > +};
> > > > +
> > > > +module_platform_driver(mtk_vcodec_enc_driver);
> > > > +
> > > > +
> > > > +MODULE_LICENSE("GPL v2");
> > > > +MODULE_DESCRIPTION("Mediatek video codec V4L2 encoder driver");
> > >
> > > Regards,
> > >
> > > Hans
> >
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-media" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
[toc] | [prev] | [next] | [standalone]
| From | Tiffany Lin <tiffany.lin@mediatek.com> |
|---|---|
| Date | 2016-04-22 06:30 +0200 |
| Subject | [PATCH v7 1/8] dt-bindings: Add a binding for Mediatek Video Processor |
| Message-ID | <rqAT7-4e7-5@gated-at.bofh.it> |
| In reply to | #1384682 |
From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
Add a DT binding documentation of Video Processor Unit for the
MT8173 SoC from Mediatek.
Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
Acked-by: Rob Herring <robh@kernel.org>
---
.../devicetree/bindings/media/mediatek-vpu.txt | 31 ++++++++++++++++++++
1 file changed, 31 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/mediatek-vpu.txt
diff --git a/Documentation/devicetree/bindings/media/mediatek-vpu.txt b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
new file mode 100644
index 0000000..2a5bac3
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek-vpu.txt
@@ -0,0 +1,31 @@
+* Mediatek Video Processor Unit
+
+Video Processor Unit is a HW video controller. It controls HW Codec including
+H.264/VP8/VP9 Decode, H.264/VP8 Encode and Image Processor (scale/rotate/color convert).
+
+Required properties:
+ - compatible: "mediatek,mt8173-vpu"
+ - reg: Must contain an entry for each entry in reg-names.
+ - reg-names: Must include the following entries:
+ "tcm": tcm base
+ "cfg_reg": Main configuration registers base
+ - interrupts: interrupt number to the cpu.
+ - clocks : clock name from clock manager
+ - clock-names: must be main. It is the main clock of VPU
+
+Optional properties:
+ - memory-region: phandle to a node describing memory (see
+ Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt)
+ to be used for VPU extended memory; if not present, VPU may be located
+ anywhere in the memory
+
+Example:
+ vpu: vpu@10020000 {
+ compatible = "mediatek,mt8173-vpu";
+ reg = <0 0x10020000 0 0x30000>,
+ <0 0x10050000 0 0x100>;
+ reg-names = "tcm", "cfg_reg";
+ interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&topckgen TOP_SCP_SEL>;
+ clock-names = "main";
+ };
--
1.7.9.5
[toc] | [prev] | [next] | [standalone]
| From | Tiffany Lin <tiffany.lin@mediatek.com> |
|---|---|
| Date | 2016-04-22 06:30 +0200 |
| Subject | [PATCH v7 2/8] [media] VPU: mediatek: support Mediatek VPU |
| Message-ID | <rqAT7-4e7-7@gated-at.bofh.it> |
| In reply to | #1384685 |
From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
It is able to handle video decoding/encoding of in a range of formats.
The driver provides with VPU firmware download, memory management and
the communication interface between CPU and VPU.
For VPU initialization, it will create virtual memory for CPU access and
IOMMU address for vcodec hw device access. When a decode/encode instance
opens a device node, vpu driver will download vpu firmware to the device.
A decode/encode instant will decode/encode a frame using VPU
interface to interrupt vpu to handle decoding/encoding jobs.
Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
drivers/media/platform/Kconfig | 13 +
drivers/media/platform/Makefile | 2 +
drivers/media/platform/mtk-vpu/Makefile | 3 +
drivers/media/platform/mtk-vpu/mtk_vpu.c | 950 ++++++++++++++++++++++++++++++
drivers/media/platform/mtk-vpu/mtk_vpu.h | 162 +++++
5 files changed, 1130 insertions(+)
create mode 100644 drivers/media/platform/mtk-vpu/Makefile
create mode 100755 drivers/media/platform/mtk-vpu/mtk_vpu.c
create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 84e041c..74c3575 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -152,6 +152,19 @@ config VIDEO_CODA
Coda is a range of video codec IPs that supports
H.264, MPEG-4, and other video formats.
+config VIDEO_MEDIATEK_VPU
+ tristate "Mediatek Video Processor Unit"
+ depends on VIDEO_DEV && VIDEO_V4L2
+ depends on ARCH_MEDIATEK || COMPILE_TEST
+ ---help---
+ This driver provides downloading VPU firmware and
+ communicating with VPU. This driver for hw video
+ codec embedded in Mediatek's MT8173 SOCs. It is able
+ to handle video decoding/encoding in a range of formats.
+
+ To compile this driver as a module, choose M here: the
+ module will be called mtk-vpu.
+
config VIDEO_MEM2MEM_DEINTERLACE
tristate "Deinterlace support"
depends on VIDEO_DEV && VIDEO_V4L2 && DMA_ENGINE
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index bbb7bd1..2efb7b1 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -56,3 +56,5 @@ obj-$(CONFIG_VIDEO_AM437X_VPFE) += am437x/
obj-$(CONFIG_VIDEO_XILINX) += xilinx/
ccflags-y += -I$(srctree)/drivers/media/i2c
+
+obj-$(CONFIG_VIDEO_MEDIATEK_VPU) += mtk-vpu/
diff --git a/drivers/media/platform/mtk-vpu/Makefile b/drivers/media/platform/mtk-vpu/Makefile
new file mode 100644
index 0000000..58cc1b4
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/Makefile
@@ -0,0 +1,3 @@
+mtk-vpu-y += mtk_vpu.o
+
+obj-$(CONFIG_VIDEO_MEDIATEK_VPU) += mtk-vpu.o
diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
new file mode 100755
index 0000000..55c081e
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
@@ -0,0 +1,950 @@
+/*
+* Copyright (c) 2016 MediaTek Inc.
+* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+#include <linux/bootmem.h>
+#include <linux/clk.h>
+#include <linux/debugfs.h>
+#include <linux/firmware.h>
+#include <linux/interrupt.h>
+#include <linux/iommu.h>
+#include <linux/module.h>
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
+#include <linux/of_platform.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/sched.h>
+#include <linux/sizes.h>
+
+#include "mtk_vpu.h"
+
+/**
+ * VPU (video processor unit) is a tiny processor controlling video hardware
+ * related to video codec, scaling and color format converting.
+ * VPU interfaces with other blocks by share memory and interrupt.
+ **/
+
+#define INIT_TIMEOUT_MS 2000U
+#define IPI_TIMEOUT_MS 2000U
+#define VPU_FW_VER_LEN 16
+
+/* maximum program/data TCM (Tightly-Coupled Memory) size */
+#define VPU_PTCM_SIZE (96 * SZ_1K)
+#define VPU_DTCM_SIZE (32 * SZ_1K)
+/* the offset to get data tcm address */
+#define VPU_DTCM_OFFSET 0x18000UL
+/* daynamic allocated maximum extended memory size */
+#define VPU_EXT_P_SIZE SZ_1M
+#define VPU_EXT_D_SIZE SZ_4M
+/* maximum binary firmware size */
+#define VPU_P_FW_SIZE (VPU_PTCM_SIZE + VPU_EXT_P_SIZE)
+#define VPU_D_FW_SIZE (VPU_DTCM_SIZE + VPU_EXT_D_SIZE)
+/* the size of share buffer between Host and VPU */
+#define SHARE_BUF_SIZE 48
+
+/* binary firmware name */
+#define VPU_P_FW "vpu_p.bin"
+#define VPU_D_FW "vpu_d.bin"
+
+#define VPU_RESET 0x0
+#define VPU_TCM_CFG 0x0008
+#define VPU_PMEM_EXT0_ADDR 0x000C
+#define VPU_PMEM_EXT1_ADDR 0x0010
+#define VPU_TO_HOST 0x001C
+#define VPU_DMEM_EXT0_ADDR 0x0014
+#define VPU_DMEM_EXT1_ADDR 0x0018
+#define HOST_TO_VPU 0x0024
+#define VPU_PC_REG 0x0060
+#define VPU_WDT_REG 0x0084
+
+/* vpu inter-processor communication interrupt */
+#define VPU_IPC_INT BIT(8)
+
+/**
+ * enum vpu_fw_type - VPU firmware type
+ *
+ * @P_FW: program firmware
+ * @D_FW: data firmware
+ *
+ */
+enum vpu_fw_type {
+ P_FW,
+ D_FW,
+};
+
+/**
+ * struct vpu_mem - VPU extended program/data memory information
+ *
+ * @va: the kernel virtual memory address of VPU extended memory
+ * @pa: the physical memory address of VPU extended memory
+ *
+ */
+struct vpu_mem {
+ void *va;
+ phys_addr_t pa;
+};
+
+/**
+ * struct vpu_regs - VPU TCM and configuration registers
+ *
+ * @tcm: the register for VPU Tightly-Coupled Memory
+ * @cfg: the register for VPU configuration
+ * @irq: the irq number for VPU interrupt
+ */
+struct vpu_regs {
+ void __iomem *tcm;
+ void __iomem *cfg;
+ int irq;
+};
+
+/**
+ * struct vpu_wdt_handler - VPU watchdog reset handler
+ *
+ * @reset_func: reset handler
+ * @priv: private data
+ */
+struct vpu_wdt_handler {
+ void (*reset_func)(void *);
+ void *priv;
+};
+
+/**
+ * struct vpu_wdt - VPU watchdog workqueue
+ *
+ * @handler: VPU watchdog reset handler
+ * @ws: workstruct for VPU watchdog
+ * @wq: workqueue for VPU watchdog
+ */
+struct vpu_wdt {
+ struct vpu_wdt_handler handler[VPU_RST_MAX];
+ struct work_struct ws;
+ struct workqueue_struct *wq;
+};
+
+/**
+ * struct vpu_run - VPU initialization status
+ *
+ * @signaled: the signal of vpu initialization completed
+ * @fw_ver: VPU firmware version
+ * @enc_capability: encoder capability which is not used for now and
+ * the value is reserved for future use
+ * @wq: wait queue for VPU initialization status
+ */
+struct vpu_run {
+ u32 signaled;
+ char fw_ver[VPU_FW_VER_LEN];
+ unsigned int enc_capability;
+ wait_queue_head_t wq;
+};
+
+/**
+ * struct vpu_ipi_desc - VPU IPI descriptor
+ *
+ * @handler: IPI handler
+ * @name: the name of IPI handler
+ * @priv: the private data of IPI handler
+ */
+struct vpu_ipi_desc {
+ ipi_handler_t handler;
+ const char *name;
+ void *priv;
+};
+
+/**
+ * struct share_obj - DTCM (Data Tightly-Coupled Memory) buffer shared with
+ * AP and VPU
+ *
+ * @id: IPI id
+ * @len: share buffer length
+ * @share_buf: share buffer data
+ */
+struct share_obj {
+ s32 id;
+ u32 len;
+ unsigned char share_buf[SHARE_BUF_SIZE];
+};
+
+/**
+ * struct mtk_vpu - vpu driver data
+ * @extmem: VPU extended memory information
+ * @reg: VPU TCM and configuration registers
+ * @run: VPU initialization status
+ * @ipi_desc: VPU IPI descriptor
+ * @recv_buf: VPU DTCM share buffer for receiving. The
+ * receive buffer is only accessed in interrupt context.
+ * @send_buf: VPU DTCM share buffer for sending
+ * @dev: VPU struct device
+ * @clk: VPU clock on/off
+ * @enable_4GB: VPU 4GB mode on/off
+ * @vpu_mutex: protect mtk_vpu (except recv_buf) and ensure only
+ * one client to use VPU service at a time. For example,
+ * suppose a client is using VPU to decode VP8.
+ * If the other client wants to encode VP8,
+ * it has to wait until VP8 decode completes.
+ * @wdt_refcnt WDT reference count to make sure the watchdog can be
+ * disabled if no other client is using VPU service
+ * @ack_wq: The wait queue for each codec and mdp. When sleeping
+ * processes wake up, they will check the condition
+ * "ipi_id_ack" to run the corresponding action or
+ * go back to sleep.
+ * @ipi_id_ack: The ACKs for registered IPI function sending
+ * interrupt to VPU
+ *
+ */
+struct mtk_vpu {
+ struct vpu_mem extmem[2];
+ struct vpu_regs reg;
+ struct vpu_run run;
+ struct vpu_wdt wdt;
+ struct vpu_ipi_desc ipi_desc[IPI_MAX];
+ struct share_obj *recv_buf;
+ struct share_obj *send_buf;
+ struct device *dev;
+ struct clk *clk;
+ bool enable_4GB;
+ struct mutex vpu_mutex; /* for protecting vpu data data structure */
+ u32 wdt_refcnt;
+ wait_queue_head_t ack_wq;
+ bool ipi_id_ack[IPI_MAX];
+};
+
+static inline void vpu_cfg_writel(struct mtk_vpu *vpu, u32 val, u32 offset)
+{
+ writel(val, vpu->reg.cfg + offset);
+}
+
+static inline u32 vpu_cfg_readl(struct mtk_vpu *vpu, u32 offset)
+{
+ return readl(vpu->reg.cfg + offset);
+}
+
+static inline bool vpu_running(struct mtk_vpu *vpu)
+{
+ return vpu_cfg_readl(vpu, VPU_RESET) & BIT(0);
+}
+
+void vpu_clock_disable(struct mtk_vpu *vpu)
+{
+ /* Disable VPU watchdog */
+ mutex_lock(&vpu->vpu_mutex);
+ if (!--vpu->wdt_refcnt)
+ vpu_cfg_writel(vpu,
+ vpu_cfg_readl(vpu, VPU_WDT_REG) & ~(1L << 31),
+ VPU_WDT_REG);
+ mutex_unlock(&vpu->vpu_mutex);
+
+ clk_disable(vpu->clk);
+}
+
+int vpu_clock_enable(struct mtk_vpu *vpu)
+{
+ int ret;
+
+ ret = clk_enable(vpu->clk);
+ if (ret)
+ return ret;
+ /* Enable VPU watchdog */
+ mutex_lock(&vpu->vpu_mutex);
+ if (!vpu->wdt_refcnt++)
+ vpu_cfg_writel(vpu,
+ vpu_cfg_readl(vpu, VPU_WDT_REG) | (1L << 31),
+ VPU_WDT_REG);
+ mutex_unlock(&vpu->vpu_mutex);
+
+ return ret;
+}
+
+int vpu_ipi_register(struct platform_device *pdev,
+ enum ipi_id id, ipi_handler_t handler,
+ const char *name, void *priv)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+ struct vpu_ipi_desc *ipi_desc;
+
+ if (!vpu) {
+ dev_err(&pdev->dev, "vpu device in not ready\n");
+ return -EPROBE_DEFER;
+ }
+
+ if (id >= 0 && id < IPI_MAX && handler) {
+ ipi_desc = vpu->ipi_desc;
+ ipi_desc[id].name = name;
+ ipi_desc[id].handler = handler;
+ ipi_desc[id].priv = priv;
+ return 0;
+ }
+
+ dev_err(&pdev->dev, "register vpu ipi id %d with invalid arguments\n",
+ id);
+ return -EINVAL;
+}
+EXPORT_SYMBOL_GPL(vpu_ipi_register);
+
+int vpu_ipi_send(struct platform_device *pdev,
+ enum ipi_id id, void *buf,
+ unsigned int len)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+ struct share_obj *send_obj = vpu->send_buf;
+ unsigned long timeout;
+ int ret = 0;
+
+ if (id <= IPI_VPU_INIT || id >= IPI_MAX ||
+ len > sizeof(send_obj->share_buf) || !buf) {
+ dev_err(vpu->dev, "failed to send ipi message\n");
+ return -EINVAL;
+ }
+
+ ret = vpu_clock_enable(vpu);
+ if (ret) {
+ dev_err(vpu->dev, "failed to enable vpu clock\n");
+ return ret;
+ }
+ if (!vpu_running(vpu)) {
+ dev_err(vpu->dev, "vpu_ipi_send: VPU is not running\n");
+ ret = -EINVAL;
+ goto clock_disable;
+ }
+
+ mutex_lock(&vpu->vpu_mutex);
+
+ /* Wait until VPU receives the last command */
+ timeout = jiffies + msecs_to_jiffies(IPI_TIMEOUT_MS);
+ do {
+ if (time_after(jiffies, timeout)) {
+ dev_err(vpu->dev, "vpu_ipi_send: IPI timeout!\n");
+ ret = -EIO;
+ goto mut_unlock;
+ }
+ } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
+
+ memcpy((void *)send_obj->share_buf, buf, len);
+ send_obj->len = len;
+ send_obj->id = id;
+
+ vpu->ipi_id_ack[id] = false;
+ /* send the command to VPU */
+ vpu_cfg_writel(vpu, 0x1, HOST_TO_VPU);
+
+ mutex_unlock(&vpu->vpu_mutex);
+
+ /* wait for VPU's ACK */
+ timeout = msecs_to_jiffies(IPI_TIMEOUT_MS);
+ ret = wait_event_timeout(vpu->ack_wq, vpu->ipi_id_ack[id], timeout);
+ vpu->ipi_id_ack[id] = false;
+ if (ret == 0) {
+ dev_err(vpu->dev, "vpu ipi %d ack time out !", id);
+ ret = -EIO;
+ goto clock_disable;
+ }
+ vpu_clock_disable(vpu);
+
+ return 0;
+
+mut_unlock:
+ mutex_unlock(&vpu->vpu_mutex);
+clock_disable:
+ vpu_clock_disable(vpu);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(vpu_ipi_send);
+
+static void vpu_wdt_reset_func(struct work_struct *ws)
+{
+ struct vpu_wdt *wdt = container_of(ws, struct vpu_wdt, ws);
+ struct mtk_vpu *vpu = container_of(wdt, struct mtk_vpu, wdt);
+ struct vpu_wdt_handler *handler = wdt->handler;
+ int index, ret;
+
+ dev_info(vpu->dev, "vpu reset\n");
+ ret = vpu_clock_enable(vpu);
+ if (ret) {
+ dev_err(vpu->dev, "[VPU] wdt enables clock failed %d\n", ret);
+ return;
+ }
+ mutex_lock(&vpu->vpu_mutex);
+ vpu_cfg_writel(vpu, 0x0, VPU_RESET);
+ mutex_unlock(&vpu->vpu_mutex);
+ vpu_clock_disable(vpu);
+
+ for (index = 0; index < VPU_RST_MAX; index++) {
+ if (handler[index].reset_func) {
+ handler[index].reset_func(handler[index].priv);
+ dev_dbg(vpu->dev, "wdt handler func %d\n", index);
+ }
+ }
+}
+
+int vpu_wdt_reg_handler(struct platform_device *pdev,
+ void wdt_reset(void *),
+ void *priv, enum rst_id id)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+ struct vpu_wdt_handler *handler = vpu->wdt.handler;
+
+ if (!vpu) {
+ dev_err(vpu->dev, "vpu device in not ready\n");
+ return -EPROBE_DEFER;
+ }
+
+ if (id >= 0 && id < VPU_RST_MAX && wdt_reset) {
+ dev_dbg(vpu->dev, "wdt register id %d\n", id);
+ mutex_lock(&vpu->vpu_mutex);
+ handler[id].reset_func = wdt_reset;
+ handler[id].priv = priv;
+ mutex_unlock(&vpu->vpu_mutex);
+ return 0;
+ }
+
+ dev_err(vpu->dev, "register vpu wdt handler failed\n");
+ return -EINVAL;
+}
+EXPORT_SYMBOL_GPL(vpu_wdt_reg_handler);
+
+unsigned int vpu_get_venc_hw_capa(struct platform_device *pdev)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+ return vpu->run.enc_capability;
+}
+EXPORT_SYMBOL_GPL(vpu_get_venc_hw_capa);
+
+void *vpu_mapping_dm_addr(struct platform_device *pdev,
+ u32 dtcm_dmem_addr)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+ if (!dtcm_dmem_addr ||
+ (dtcm_dmem_addr > (VPU_DTCM_SIZE + VPU_EXT_D_SIZE))) {
+ dev_err(vpu->dev, "invalid virtual data memory address\n");
+ return ERR_PTR(-EINVAL);
+ }
+
+ if (dtcm_dmem_addr < VPU_DTCM_SIZE)
+ return dtcm_dmem_addr + vpu->reg.tcm + VPU_DTCM_OFFSET;
+
+ return vpu->extmem[D_FW].va + (dtcm_dmem_addr - VPU_DTCM_SIZE);
+}
+EXPORT_SYMBOL_GPL(vpu_mapping_dm_addr);
+
+struct platform_device *vpu_get_plat_device(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *vpu_node;
+ struct platform_device *vpu_pdev;
+
+ vpu_node = of_parse_phandle(dev->of_node, "mediatek,vpu", 0);
+ if (!vpu_node) {
+ dev_err(dev, "can't get vpu node\n");
+ return NULL;
+ }
+
+ vpu_pdev = of_find_device_by_node(vpu_node);
+ if (WARN_ON(!vpu_pdev)) {
+ dev_err(dev, "vpu pdev failed\n");
+ of_node_put(vpu_node);
+ return NULL;
+ }
+
+ return vpu_pdev;
+}
+EXPORT_SYMBOL_GPL(vpu_get_plat_device);
+
+/* load vpu program/data memory */
+static int load_requested_vpu(struct mtk_vpu *vpu,
+ const struct firmware *vpu_fw,
+ u8 fw_type)
+{
+ size_t tcm_size = fw_type ? VPU_DTCM_SIZE : VPU_PTCM_SIZE;
+ size_t fw_size = fw_type ? VPU_D_FW_SIZE : VPU_P_FW_SIZE;
+ char *fw_name = fw_type ? VPU_D_FW : VPU_P_FW;
+ size_t dl_size = 0;
+ size_t extra_fw_size = 0;
+ void *dest;
+ int ret;
+
+ ret = request_firmware(&vpu_fw, fw_name, vpu->dev);
+ if (ret < 0) {
+ dev_err(vpu->dev, "Failed to load %s, %d\n", fw_name, ret);
+ return ret;
+ }
+ dl_size = vpu_fw->size;
+ if (dl_size > fw_size) {
+ dev_err(vpu->dev, "fw %s size %zu is abnormal\n", fw_name,
+ dl_size);
+ release_firmware(vpu_fw);
+ return -EFBIG;
+ }
+ dev_dbg(vpu->dev, "Downloaded fw %s size: %zu.\n",
+ fw_name,
+ dl_size);
+ /* reset VPU */
+ vpu_cfg_writel(vpu, 0x0, VPU_RESET);
+
+ /* handle extended firmware size */
+ if (dl_size > tcm_size) {
+ dev_dbg(vpu->dev, "fw size %lx > limited fw size %lx\n",
+ dl_size, tcm_size);
+ extra_fw_size = dl_size - tcm_size;
+ dev_dbg(vpu->dev, "extra_fw_size %lx\n", extra_fw_size);
+ dl_size = tcm_size;
+ }
+ dest = vpu->reg.tcm;
+ if (fw_type == D_FW)
+ dest += VPU_DTCM_OFFSET;
+ memcpy(dest, vpu_fw->data, dl_size);
+ /* download to extended memory if need */
+ if (extra_fw_size > 0) {
+ dest = vpu->extmem[fw_type].va;
+ dev_dbg(vpu->dev, "download extended memory type %x\n",
+ fw_type);
+ memcpy(dest, vpu_fw->data + tcm_size, extra_fw_size);
+ }
+
+ release_firmware(vpu_fw);
+
+ return 0;
+}
+
+int vpu_load_firmware(struct platform_device *pdev)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+ struct device *dev = &pdev->dev;
+ struct vpu_run *run = &vpu->run;
+ const struct firmware *vpu_fw;
+ int ret;
+
+ if (!pdev) {
+ dev_err(dev, "VPU platform device is invalid\n");
+ return -EINVAL;
+ }
+
+ ret = vpu_clock_enable(vpu);
+ if (ret) {
+ dev_err(dev, "enable clock failed %d\n", ret);
+ return ret;
+ }
+
+ mutex_lock(&vpu->vpu_mutex);
+
+ if (vpu_running(vpu)) {
+ mutex_unlock(&vpu->vpu_mutex);
+ vpu_clock_disable(vpu);
+ dev_warn(dev, "vpu is running already\n");
+ return 0;
+ }
+
+ run->signaled = false;
+ dev_dbg(vpu->dev, "firmware request\n");
+ /* Downloading program firmware to device*/
+ ret = load_requested_vpu(vpu, vpu_fw, P_FW);
+ if (ret < 0) {
+ dev_err(dev, "Failed to request %s, %d\n", VPU_P_FW, ret);
+ goto OUT_LOAD_FW;
+ }
+
+ /* Downloading data firmware to device */
+ ret = load_requested_vpu(vpu, vpu_fw, D_FW);
+ if (ret < 0) {
+ dev_err(dev, "Failed to request %s, %d\n", VPU_D_FW, ret);
+ goto OUT_LOAD_FW;
+ }
+
+ /* boot up vpu */
+ vpu_cfg_writel(vpu, 0x1, VPU_RESET);
+
+ ret = wait_event_interruptible_timeout(run->wq,
+ run->signaled,
+ msecs_to_jiffies(INIT_TIMEOUT_MS)
+ );
+ if (ret == 0) {
+ ret = -ETIME;
+ dev_err(dev, "wait vpu initialization timout!\n");
+ goto OUT_LOAD_FW;
+ } else if (-ERESTARTSYS == ret) {
+ dev_err(dev, "wait vpu interrupted by a signal!\n");
+ goto OUT_LOAD_FW;
+ }
+
+ ret = 0;
+ dev_info(dev, "vpu is ready. Fw version %s\n", run->fw_ver);
+
+OUT_LOAD_FW:
+ mutex_unlock(&vpu->vpu_mutex);
+ vpu_clock_disable(vpu);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(vpu_load_firmware);
+
+static void vpu_init_ipi_handler(void *data, unsigned int len, void *priv)
+{
+ struct mtk_vpu *vpu = (struct mtk_vpu *)priv;
+ struct vpu_run *run = (struct vpu_run *)data;
+
+ vpu->run.signaled = run->signaled;
+ strncpy(vpu->run.fw_ver, run->fw_ver, VPU_FW_VER_LEN);
+ vpu->run.enc_capability = run->enc_capability;
+ wake_up_interruptible(&vpu->run.wq);
+}
+
+#ifdef CONFIG_DEBUG_FS
+static int vpu_debug_open(struct inode *inode, struct file *file)
+{
+ file->private_data = inode->i_private;
+ return 0;
+}
+
+static ssize_t vpu_debug_read(struct file *file, char __user *user_buf,
+ size_t count, loff_t *ppos)
+{
+ char buf[256];
+ unsigned int len;
+ unsigned int running, pc, vpu_to_host, host_to_vpu, wdt;
+ int ret;
+ struct device *dev = file->private_data;
+ struct mtk_vpu *vpu = dev_get_drvdata(dev);
+
+ ret = vpu_clock_enable(vpu);
+ if (ret) {
+ dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
+ return 0;
+ }
+
+ /* vpu register status */
+ running = vpu_running(vpu);
+ pc = vpu_cfg_readl(vpu, VPU_PC_REG);
+ wdt = vpu_cfg_readl(vpu, VPU_WDT_REG);
+ host_to_vpu = vpu_cfg_readl(vpu, HOST_TO_VPU);
+ vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
+ vpu_clock_disable(vpu);
+
+ if (running) {
+ len = snprintf(buf, sizeof(buf), "VPU is running\n\n"
+ "FW Version: %s\n"
+ "PC: 0x%x\n"
+ "WDT: 0x%x\n"
+ "Host to VPU: 0x%x\n"
+ "VPU to Host: 0x%x\n",
+ vpu->run.fw_ver, pc, wdt,
+ host_to_vpu, vpu_to_host);
+ } else {
+ len = snprintf(buf, sizeof(buf), "VPU not running\n");
+ }
+
+ return simple_read_from_buffer(user_buf, count, ppos, buf, len);
+}
+
+static const struct file_operations vpu_debug_fops = {
+ .open = vpu_debug_open,
+ .read = vpu_debug_read,
+};
+#endif /* CONFIG_DEBUG_FS */
+
+static void vpu_free_ext_mem(struct mtk_vpu *vpu, u8 fw_type)
+{
+ struct device *dev = vpu->dev;
+ size_t fw_ext_size = fw_type ? VPU_EXT_D_SIZE : VPU_EXT_P_SIZE;
+
+ dma_free_coherent(dev, fw_ext_size, vpu->extmem[fw_type].va,
+ vpu->extmem[fw_type].pa);
+}
+
+static int vpu_alloc_ext_mem(struct mtk_vpu *vpu, u32 fw_type)
+{
+ struct device *dev = vpu->dev;
+ size_t fw_ext_size = fw_type ? VPU_EXT_D_SIZE : VPU_EXT_P_SIZE;
+ u32 vpu_ext_mem0 = fw_type ? VPU_DMEM_EXT0_ADDR : VPU_PMEM_EXT0_ADDR;
+ u32 vpu_ext_mem1 = fw_type ? VPU_DMEM_EXT1_ADDR : VPU_PMEM_EXT1_ADDR;
+ u32 offset_4gb = vpu->enable_4GB ? 0x40000000 : 0;
+
+ vpu->extmem[fw_type].va = dma_alloc_coherent(dev,
+ fw_ext_size,
+ &vpu->extmem[fw_type].pa,
+ GFP_KERNEL);
+ if (!vpu->extmem[fw_type].va) {
+ dev_err(dev, "Failed to allocate the extended program memory\n");
+ return PTR_ERR(vpu->extmem[fw_type].va);
+ }
+
+ /* Disable extend0. Enable extend1 */
+ vpu_cfg_writel(vpu, 0x1, vpu_ext_mem0);
+ vpu_cfg_writel(vpu, (vpu->extmem[fw_type].pa & 0xFFFFF000) + offset_4gb,
+ vpu_ext_mem1);
+
+ dev_info(dev, "%s extend memory phy=0x%llx virt=0x%p\n",
+ fw_type ? "Data" : "Program",
+ (unsigned long long)vpu->extmem[fw_type].pa,
+ vpu->extmem[fw_type].va);
+
+ return 0;
+}
+
+static void vpu_ipi_handler(struct mtk_vpu *vpu)
+{
+ struct share_obj *rcv_obj = vpu->recv_buf;
+ struct vpu_ipi_desc *ipi_desc = vpu->ipi_desc;
+
+ if (rcv_obj->id < IPI_MAX && ipi_desc[rcv_obj->id].handler) {
+ ipi_desc[rcv_obj->id].handler(rcv_obj->share_buf,
+ rcv_obj->len,
+ ipi_desc[rcv_obj->id].priv);
+ if (rcv_obj->id > IPI_VPU_INIT) {
+ vpu->ipi_id_ack[rcv_obj->id] = true;
+ wake_up(&vpu->ack_wq);
+ }
+ } else {
+ dev_err(vpu->dev, "No such ipi id = %d\n", rcv_obj->id);
+ }
+}
+
+static int vpu_ipi_init(struct mtk_vpu *vpu)
+{
+ /* Disable VPU to host interrupt */
+ vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
+
+ /* shared buffer initialization */
+ vpu->recv_buf = (struct share_obj *)(vpu->reg.tcm + VPU_DTCM_OFFSET);
+ vpu->send_buf = vpu->recv_buf + 1;
+ memset(vpu->recv_buf, 0, sizeof(struct share_obj));
+ memset(vpu->send_buf, 0, sizeof(struct share_obj));
+
+ return 0;
+}
+
+static irqreturn_t vpu_irq_handler(int irq, void *priv)
+{
+ struct mtk_vpu *vpu = priv;
+ u32 vpu_to_host;
+ int ret;
+
+ /*
+ * Clock should have been enabled already.
+ * Enable again in case vpu_ipi_send times out
+ * and has disabled the clock.
+ */
+ ret = clk_enable(vpu->clk);
+ if (ret) {
+ dev_err(vpu->dev, "[VPU] enable clock failed %d\n", ret);
+ return IRQ_NONE;
+ }
+ vpu_to_host = vpu_cfg_readl(vpu, VPU_TO_HOST);
+ if (vpu_to_host & VPU_IPC_INT) {
+ vpu_ipi_handler(vpu);
+ } else {
+ dev_err(vpu->dev, "vpu watchdog timeout! 0x%x", vpu_to_host);
+ queue_work(vpu->wdt.wq, &vpu->wdt.ws);
+ }
+
+ /* VPU won't send another interrupt until we set VPU_TO_HOST to 0. */
+ vpu_cfg_writel(vpu, 0x0, VPU_TO_HOST);
+ clk_disable(vpu->clk);
+
+ return IRQ_HANDLED;
+}
+
+#ifdef CONFIG_DEBUG_FS
+static struct dentry *vpu_debugfs;
+#endif
+static int mtk_vpu_probe(struct platform_device *pdev)
+{
+ struct mtk_vpu *vpu;
+ struct device *dev;
+ struct resource *res;
+ int ret = 0;
+
+ dev_dbg(&pdev->dev, "initialization\n");
+
+ dev = &pdev->dev;
+ vpu = devm_kzalloc(dev, sizeof(*vpu), GFP_KERNEL);
+ if (!vpu)
+ return -ENOMEM;
+
+ vpu->dev = &pdev->dev;
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "tcm");
+ vpu->reg.tcm = devm_ioremap_resource(dev, res);
+ if (IS_ERR(vpu->reg.tcm)) {
+ dev_err(dev, "devm_ioremap_resource vpu tcm failed.\n");
+ return PTR_ERR(vpu->reg.tcm);
+ }
+
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "cfg_reg");
+ vpu->reg.cfg = devm_ioremap_resource(dev, res);
+ if (IS_ERR(vpu->reg.cfg)) {
+ dev_err(dev, "devm_ioremap_resource vpu cfg failed.\n");
+ return PTR_ERR(vpu->reg.cfg);
+ }
+
+ /* Get VPU clock */
+ vpu->clk = devm_clk_get(dev, "main");
+ if (!vpu->clk) {
+ dev_err(dev, "get vpu clock failed\n");
+ return -EINVAL;
+ }
+
+ platform_set_drvdata(pdev, vpu);
+
+ ret = clk_prepare(vpu->clk);
+ if (ret) {
+ dev_err(dev, "prepare vpu clock failed\n");
+ return ret;
+ }
+
+ /* VPU watchdog */
+ vpu->wdt.wq = create_singlethread_workqueue("vpu_wdt");
+ if (!vpu->wdt.wq) {
+ dev_err(dev, "initialize wdt workqueue failed\n");
+ return -ENOMEM;
+ }
+ INIT_WORK(&vpu->wdt.ws, vpu_wdt_reset_func);
+ mutex_init(&vpu->vpu_mutex);
+
+ ret = vpu_clock_enable(vpu);
+ if (ret) {
+ dev_err(dev, "enable vpu clock failed\n");
+ goto workqueue_destroy;
+ }
+
+ dev_dbg(dev, "vpu ipi init\n");
+ ret = vpu_ipi_init(vpu);
+ if (ret) {
+ dev_err(dev, "Failed to init ipi\n");
+ goto disable_vpu_clk;
+ }
+
+ /* register vpu initialization IPI */
+ ret = vpu_ipi_register(pdev, IPI_VPU_INIT, vpu_init_ipi_handler,
+ "vpu_init", vpu);
+ if (ret) {
+ dev_err(dev, "Failed to register IPI_VPU_INIT\n");
+ goto vpu_mutex_destroy;
+ }
+
+#ifdef CONFIG_DEBUG_FS
+ vpu_debugfs = debugfs_create_file("mtk_vpu", S_IRUGO, NULL, (void *)dev,
+ &vpu_debug_fops);
+ if (!vpu_debugfs) {
+ ret = -ENOMEM;
+ goto cleanup_ipi;
+ }
+#endif
+
+ /* Set PTCM to 96K and DTCM to 32K */
+ vpu_cfg_writel(vpu, 0x2, VPU_TCM_CFG);
+
+ vpu->enable_4GB = !!(max_pfn > (0xffffffffUL >> PAGE_SHIFT));
+ dev_info(dev, "4GB mode %u\n", vpu->enable_4GB);
+
+ if (vpu->enable_4GB) {
+ ret = of_reserved_mem_device_init(dev);
+ if (ret)
+ dev_info(dev, "init reserved memory failed\n");
+ /* continue to use dynamic allocation if failed */
+ }
+
+ ret = vpu_alloc_ext_mem(vpu, D_FW);
+ if (ret) {
+ dev_err(dev, "Allocate DM failed\n");
+ goto remove_debugfs;
+ }
+
+ ret = vpu_alloc_ext_mem(vpu, P_FW);
+ if (ret) {
+ dev_err(dev, "Allocate PM failed\n");
+ goto free_d_mem;
+ }
+
+ init_waitqueue_head(&vpu->run.wq);
+ init_waitqueue_head(&vpu->ack_wq);
+
+ res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
+ if (!res) {
+ dev_err(dev, "get IRQ resource failed.\n");
+ ret = -ENXIO;
+ goto free_p_mem;
+ }
+ vpu->reg.irq = platform_get_irq(pdev, 0);
+ ret = devm_request_irq(dev, vpu->reg.irq, vpu_irq_handler, 0,
+ pdev->name, vpu);
+ if (ret) {
+ dev_err(dev, "failed to request irq\n");
+ goto free_p_mem;
+ }
+
+ vpu_clock_disable(vpu);
+ dev_dbg(dev, "initialization completed\n");
+
+ return 0;
+
+free_p_mem:
+ vpu_free_ext_mem(vpu, P_FW);
+free_d_mem:
+ vpu_free_ext_mem(vpu, D_FW);
+remove_debugfs:
+ of_reserved_mem_device_release(dev);
+#ifdef CONFIG_DEBUG_FS
+ debugfs_remove(vpu_debugfs);
+cleanup_ipi:
+#endif
+ memset(vpu->ipi_desc, 0, sizeof(struct vpu_ipi_desc) * IPI_MAX);
+vpu_mutex_destroy:
+ mutex_destroy(&vpu->vpu_mutex);
+disable_vpu_clk:
+ vpu_clock_disable(vpu);
+workqueue_destroy:
+ destroy_workqueue(vpu->wdt.wq);
+
+ return ret;
+}
+
+static const struct of_device_id mtk_vpu_match[] = {
+ {
+ .compatible = "mediatek,mt8173-vpu",
+ },
+ {},
+};
+MODULE_DEVICE_TABLE(of, mtk_vpu_match);
+
+static int mtk_vpu_remove(struct platform_device *pdev)
+{
+ struct mtk_vpu *vpu = platform_get_drvdata(pdev);
+
+#ifdef CONFIG_DEBUG_FS
+ debugfs_remove(vpu_debugfs);
+#endif
+ if (vpu->wdt.wq) {
+ flush_workqueue(vpu->wdt.wq);
+ destroy_workqueue(vpu->wdt.wq);
+ }
+ vpu_free_ext_mem(vpu, P_FW);
+ vpu_free_ext_mem(vpu, D_FW);
+ mutex_destroy(&vpu->vpu_mutex);
+ clk_unprepare(vpu->clk);
+
+ return 0;
+}
+
+static struct platform_driver mtk_vpu_driver = {
+ .probe = mtk_vpu_probe,
+ .remove = mtk_vpu_remove,
+ .driver = {
+ .name = "mtk_vpu",
+ .owner = THIS_MODULE,
+ .of_match_table = mtk_vpu_match,
+ },
+};
+
+module_platform_driver(mtk_vpu_driver);
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Mediatek Video Prosessor Unit driver");
diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.h b/drivers/media/platform/mtk-vpu/mtk_vpu.h
new file mode 100644
index 0000000..5ab37f0
--- /dev/null
+++ b/drivers/media/platform/mtk-vpu/mtk_vpu.h
@@ -0,0 +1,162 @@
+/*
+* Copyright (c) 2016 MediaTek Inc.
+* Author: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License version 2 as
+* published by the Free Software Foundation.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*/
+
+#ifndef _MTK_VPU_H
+#define _MTK_VPU_H
+
+#include <linux/platform_device.h>
+
+/**
+ * VPU (video processor unit) is a tiny processor controlling video hardware
+ * related to video codec, scaling and color format converting.
+ * VPU interfaces with other blocks by share memory and interrupt.
+ **/
+
+typedef void (*ipi_handler_t) (void *data,
+ unsigned int len,
+ void *priv);
+
+/**
+ * enum ipi_id - the id of inter-processor interrupt
+ *
+ * @IPI_VPU_INIT: The interrupt from vpu is to notfiy kernel
+ VPU initialization completed.
+ IPI_VPU_INIT is sent from VPU when firmware is
+ loaded. AP doesn't need to send IPI_VPU_INIT
+ command to VPU.
+ For other IPI below, AP should send the request
+ to VPU to trigger the interrupt.
+ * @IPI_VENC_H264: The interrupt from vpu is to notify kernel to
+ handle H264 video encoder job, and vice versa.
+ * @IPI_VENC_VP8: The interrupt fro vpu is to notify kernel to
+ handle VP8 video encoder job,, and vice versa.
+ * @IPI_MAX: The maximum IPI number
+ */
+
+enum ipi_id {
+ IPI_VPU_INIT = 0,
+ IPI_VENC_H264,
+ IPI_VENC_VP8,
+ IPI_MAX,
+};
+
+/**
+ * enum rst_id - reset id to register reset function for VPU watchdog timeout
+ *
+ * @VPU_RST_ENC: encoder reset id
+ * @VPU_RST_MAX: maximum reset id
+ */
+enum rst_id {
+ VPU_RST_ENC,
+ VPU_RST_MAX,
+};
+
+/**
+ * vpu_ipi_register - register an ipi function
+ *
+ * @pdev: VPU platform device
+ * @id: IPI ID
+ * @handler: IPI handler
+ * @name: IPI name
+ * @priv: private data for IPI handler
+ *
+ * Register an ipi function to receive ipi interrupt from VPU.
+ *
+ * Return: Return 0 if ipi registers successfully, otherwise it is failed.
+ */
+int vpu_ipi_register(struct platform_device *pdev, enum ipi_id id,
+ ipi_handler_t handler, const char *name, void *priv);
+
+/**
+ * vpu_ipi_send - send data from AP to vpu.
+ *
+ * @pdev: VPU platform device
+ * @id: IPI ID
+ * @buf: the data buffer
+ * @len: the data buffer length
+ *
+ * This function is thread-safe. When this function returns,
+ * VPU has received the data and starts the processing.
+ * When the processing completes, IPI handler registered
+ * by vpu_ipi_register will be called in interrupt context.
+ *
+ * Return: Return 0 if sending data successfully, otherwise it is failed.
+ **/
+int vpu_ipi_send(struct platform_device *pdev,
+ enum ipi_id id, void *buf,
+ unsigned int len);
+
+/**
+ * vpu_get_plat_device - get VPU's platform device
+ *
+ * @pdev: the platform device of the module requesting VPU platform
+ * device for using VPU API.
+ *
+ * Return: Return NULL if it is failed.
+ * otherwise it is VPU's platform device
+ **/
+struct platform_device *vpu_get_plat_device(struct platform_device *pdev);
+
+/**
+ * vpu_wdt_reg_handler - register a VPU watchdog handler
+ *
+ * @pdev: VPU platform device
+ * @vpu_wdt_reset_func: the callback reset function
+ * @private_data: the private data for reset function
+ * @rst_id: reset id
+ *
+ * Register a handler performing own tasks when vpu reset by watchdog
+ *
+ * Return: Return 0 if the handler is added successfully,
+ * otherwise it is failed.
+ *
+ **/
+int vpu_wdt_reg_handler(struct platform_device *pdev,
+ void vpu_wdt_reset_func(void *),
+ void *priv, enum rst_id id);
+/**
+ * vpu_get_venc_hw_capa - get video encoder hardware capability
+ *
+ * @pdev: VPU platform device
+ *
+ * Return: video encoder hardware capability
+ **/
+unsigned int vpu_get_venc_hw_capa(struct platform_device *pdev);
+
+/**
+ * vpu_load_firmware - download VPU firmware and boot it
+ *
+ * @pdev: VPU platform device
+ *
+ * Return: Return 0 if downloading firmware successfully,
+ * otherwise it is failed
+ **/
+int vpu_load_firmware(struct platform_device *pdev);
+
+/**
+ * vpu_mapping_dm_addr - Mapping DTCM/DMEM to kernel virtual address
+ *
+ * @pdev: VPU platform device
+ * @dmem_addr: VPU's data memory address
+ *
+ * Mapping the VPU's DTCM (Data Tightly-Coupled Memory) /
+ * DMEM (Data Extended Memory) memory address to
+ * kernel virtual address.
+ *
+ * Return: Return ERR_PTR(-EINVAL) if mapping failed,
+ * otherwise the mapped kernel virtual address
+ **/
+void *vpu_mapping_dm_addr(struct platform_device *pdev,
+ u32 dtcm_dmem_addr);
+#endif /* _MTK_VPU_H */
--
1.7.9.5
[toc] | [prev] | [next] | [standalone]
| From | Hans Verkuil <hverkuil@xs4all.nl> |
|---|---|
| Date | 2016-04-25 09:50 +0200 |
| Subject | Re: [PATCH v7 2/8] [media] VPU: mediatek: support Mediatek VPU |
| Message-ID | <rrJrk-1Vb-11@gated-at.bofh.it> |
| In reply to | #1384686 |
On 04/22/2016 06:25 AM, Tiffany Lin wrote:
> From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
>
> The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> It is able to handle video decoding/encoding of in a range of formats.
> The driver provides with VPU firmware download, memory management and
> the communication interface between CPU and VPU.
> For VPU initialization, it will create virtual memory for CPU access and
> IOMMU address for vcodec hw device access. When a decode/encode instance
> opens a device node, vpu driver will download vpu firmware to the device.
> A decode/encode instant will decode/encode a frame using VPU
> interface to interrupt vpu to handle decoding/encoding jobs.
>
> Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
>
> ---
> drivers/media/platform/Kconfig | 13 +
> drivers/media/platform/Makefile | 2 +
> drivers/media/platform/mtk-vpu/Makefile | 3 +
> drivers/media/platform/mtk-vpu/mtk_vpu.c | 950 ++++++++++++++++++++++++++++++
> drivers/media/platform/mtk-vpu/mtk_vpu.h | 162 +++++
> 5 files changed, 1130 insertions(+)
> create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> create mode 100755 drivers/media/platform/mtk-vpu/mtk_vpu.c
> create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
>
> +int vpu_load_firmware(struct platform_device *pdev)
> +{
> + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> + struct device *dev = &pdev->dev;
> + struct vpu_run *run = &vpu->run;
> + const struct firmware *vpu_fw;
> + int ret;
> +
> + if (!pdev) {
> + dev_err(dev, "VPU platform device is invalid\n");
> + return -EINVAL;
> + }
> +
> + ret = vpu_clock_enable(vpu);
> + if (ret) {
> + dev_err(dev, "enable clock failed %d\n", ret);
> + return ret;
> + }
> +
> + mutex_lock(&vpu->vpu_mutex);
> +
> + if (vpu_running(vpu)) {
> + mutex_unlock(&vpu->vpu_mutex);
> + vpu_clock_disable(vpu);
> + dev_warn(dev, "vpu is running already\n");
This warning should be dropped. Currently vpu_load_firmware is called
every time the video device is opened and no one else has the video device
open. So calling this multiple times is perfectly normal and the log shouldn't
be spammed with warnings.
I would recommend adding a fw_loaded bool to struct mtk_vpu and just
check that at the beginning of this function and just return 0 if it is true.
Then you don't need to enable the vpu clock either.
I hope I understand the hw correctly, though.
Assuming you can do this, then this code from the v4l driver needs an
additional comment:
>>> + if (v4l2_fh_is_singular(&ctx->fh)) {
Add a comment here that says that vpu_load_firmware checks if it was
loaded already and does nothing in that case.
>>> + ret = vpu_load_firmware(dev->vpu_plat_dev);
>>> + if (ret < 0) {
>>> + /*
>>> + * Return 0 if downloading firmware successfully,
>>> + * otherwise it is failed
>>> + */
>>> + mtk_v4l2_err("vpu_load_firmware failed!");
>>> + goto err_load_fw;
>>> + }
That makes it clear to the reader (i.e. me :-) ) that you can safely call
vpu_load_firmware multiple times.
Regards,
Hans
> + return 0;
> + }
> +
> + run->signaled = false;
> + dev_dbg(vpu->dev, "firmware request\n");
> + /* Downloading program firmware to device*/
> + ret = load_requested_vpu(vpu, vpu_fw, P_FW);
> + if (ret < 0) {
> + dev_err(dev, "Failed to request %s, %d\n", VPU_P_FW, ret);
> + goto OUT_LOAD_FW;
> + }
> +
> + /* Downloading data firmware to device */
> + ret = load_requested_vpu(vpu, vpu_fw, D_FW);
> + if (ret < 0) {
> + dev_err(dev, "Failed to request %s, %d\n", VPU_D_FW, ret);
> + goto OUT_LOAD_FW;
> + }
> +
> + /* boot up vpu */
> + vpu_cfg_writel(vpu, 0x1, VPU_RESET);
> +
> + ret = wait_event_interruptible_timeout(run->wq,
> + run->signaled,
> + msecs_to_jiffies(INIT_TIMEOUT_MS)
> + );
> + if (ret == 0) {
> + ret = -ETIME;
> + dev_err(dev, "wait vpu initialization timout!\n");
> + goto OUT_LOAD_FW;
> + } else if (-ERESTARTSYS == ret) {
> + dev_err(dev, "wait vpu interrupted by a signal!\n");
> + goto OUT_LOAD_FW;
> + }
> +
> + ret = 0;
> + dev_info(dev, "vpu is ready. Fw version %s\n", run->fw_ver);
> +
> +OUT_LOAD_FW:
> + mutex_unlock(&vpu->vpu_mutex);
> + vpu_clock_disable(vpu);
> +
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(vpu_load_firmware);
[toc] | [prev] | [next] | [standalone]
| From | andrew-ct chen <andrew-ct.chen@mediatek.com> |
|---|---|
| Date | 2016-04-25 11:30 +0200 |
| Subject | Re: [PATCH v7 2/8] [media] VPU: mediatek: support Mediatek VPU |
| Message-ID | <rrL06-3oi-19@gated-at.bofh.it> |
| In reply to | #1386071 |
On Mon, 2016-04-25 at 09:40 +0200, Hans Verkuil wrote:
> On 04/22/2016 06:25 AM, Tiffany Lin wrote:
> > From: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> >
> > The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> > It is able to handle video decoding/encoding of in a range of formats.
> > The driver provides with VPU firmware download, memory management and
> > the communication interface between CPU and VPU.
> > For VPU initialization, it will create virtual memory for CPU access and
> > IOMMU address for vcodec hw device access. When a decode/encode instance
> > opens a device node, vpu driver will download vpu firmware to the device.
> > A decode/encode instant will decode/encode a frame using VPU
> > interface to interrupt vpu to handle decoding/encoding jobs.
> >
> > Signed-off-by: Andrew-CT Chen <andrew-ct.chen@mediatek.com>
> > Signed-off-by: Tiffany Lin <tiffany.lin@mediatek.com>
> >
> > ---
> > drivers/media/platform/Kconfig | 13 +
> > drivers/media/platform/Makefile | 2 +
> > drivers/media/platform/mtk-vpu/Makefile | 3 +
> > drivers/media/platform/mtk-vpu/mtk_vpu.c | 950 ++++++++++++++++++++++++++++++
> > drivers/media/platform/mtk-vpu/mtk_vpu.h | 162 +++++
> > 5 files changed, 1130 insertions(+)
> > create mode 100644 drivers/media/platform/mtk-vpu/Makefile
> > create mode 100755 drivers/media/platform/mtk-vpu/mtk_vpu.c
> > create mode 100644 drivers/media/platform/mtk-vpu/mtk_vpu.h
> >
>
>
> > +int vpu_load_firmware(struct platform_device *pdev)
> > +{
> > + struct mtk_vpu *vpu = platform_get_drvdata(pdev);
> > + struct device *dev = &pdev->dev;
> > + struct vpu_run *run = &vpu->run;
> > + const struct firmware *vpu_fw;
> > + int ret;
> > +
> > + if (!pdev) {
> > + dev_err(dev, "VPU platform device is invalid\n");
> > + return -EINVAL;
> > + }
> > +
> > + ret = vpu_clock_enable(vpu);
> > + if (ret) {
> > + dev_err(dev, "enable clock failed %d\n", ret);
> > + return ret;
> > + }
> > +
> > + mutex_lock(&vpu->vpu_mutex);
> > +
> > + if (vpu_running(vpu)) {
> > + mutex_unlock(&vpu->vpu_mutex);
> > + vpu_clock_disable(vpu);
> > + dev_warn(dev, "vpu is running already\n");
>
> This warning should be dropped. Currently vpu_load_firmware is called
> every time the video device is opened and no one else has the video device
> open. So calling this multiple times is perfectly normal and the log shouldn't
> be spammed with warnings.
>
> I would recommend adding a fw_loaded bool to struct mtk_vpu and just
> check that at the beginning of this function and just return 0 if it is true.
>
> Then you don't need to enable the vpu clock either.
>
> I hope I understand the hw correctly, though.
>
> Assuming you can do this, then this code from the v4l driver needs an
> additional comment:
We will change this in next version.
>
> >>> + if (v4l2_fh_is_singular(&ctx->fh)) {
>
> Add a comment here that says that vpu_load_firmware checks if it was
> loaded already and does nothing in that case.
We will change this in next version. Thanks
>
> >>> + ret = vpu_load_firmware(dev->vpu_plat_dev);
> >>> + if (ret < 0) {
> >>> + /*
> >>> + * Return 0 if downloading firmware successfully,
> >>> + * otherwise it is failed
> >>> + */
> >>> + mtk_v4l2_err("vpu_load_firmware failed!");
> >>> + goto err_load_fw;
> >>> + }
>
> That makes it clear to the reader (i.e. me :-) ) that you can safely call
> vpu_load_firmware multiple times.
>
> Regards,
>
> Hans
>
> > + return 0;
> > + }
> > +
> > + run->signaled = false;
> > + dev_dbg(vpu->dev, "firmware request\n");
> > + /* Downloading program firmware to device*/
> > + ret = load_requested_vpu(vpu, vpu_fw, P_FW);
> > + if (ret < 0) {
> > + dev_err(dev, "Failed to request %s, %d\n", VPU_P_FW, ret);
> > + goto OUT_LOAD_FW;
> > + }
> > +
> > + /* Downloading data firmware to device */
> > + ret = load_requested_vpu(vpu, vpu_fw, D_FW);
> > + if (ret < 0) {
> > + dev_err(dev, "Failed to request %s, %d\n", VPU_D_FW, ret);
> > + goto OUT_LOAD_FW;
> > + }
> > +
> > + /* boot up vpu */
> > + vpu_cfg_writel(vpu, 0x1, VPU_RESET);
> > +
> > + ret = wait_event_interruptible_timeout(run->wq,
> > + run->signaled,
> > + msecs_to_jiffies(INIT_TIMEOUT_MS)
> > + );
> > + if (ret == 0) {
> > + ret = -ETIME;
> > + dev_err(dev, "wait vpu initialization timout!\n");
> > + goto OUT_LOAD_FW;
> > + } else if (-ERESTARTSYS == ret) {
> > + dev_err(dev, "wait vpu interrupted by a signal!\n");
> > + goto OUT_LOAD_FW;
> > + }
> > +
> > + ret = 0;
> > + dev_info(dev, "vpu is ready. Fw version %s\n", run->fw_ver);
> > +
> > +OUT_LOAD_FW:
> > + mutex_unlock(&vpu->vpu_mutex);
> > + vpu_clock_disable(vpu);
> > +
> > + return ret;
> > +}
> > +EXPORT_SYMBOL_GPL(vpu_load_firmware);
>
[toc] | [prev] | [standalone]
Back to top | Article view | linux.kernel
csiph-web