Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1613003 > unrolled thread
| Started by | Wu Hao <hao.wu@intel.com> |
|---|---|
| First post | 2017-03-30 14:20 +0200 |
| Last post | 2017-04-06 22:30 +0200 |
| Articles | 20 on this page of 61 — 8 participants |
Back to article view | Back to linux.kernel
[PATCH 00/16] Intel FPGA Device Drivers Wu Hao <hao.wu@intel.com> - 2017-03-30 14:20 +0200
[PATCH 02/16] fpga: add FPGA device framework Wu Hao <hao.wu@intel.com> - 2017-03-30 14:20 +0200
Re: [PATCH 02/16] fpga: add FPGA device framework Greg KH <greg@kroah.com> - 2017-03-31 08:10 +0200
Re: [PATCH 02/16] fpga: add FPGA device framework Wu Hao <hao.wu@intel.com> - 2017-03-31 10:00 +0200
Re: [PATCH 02/16] fpga: add FPGA device framework Greg KH <greg@kroah.com> - 2017-03-31 11:10 +0200
Re: [PATCH 02/16] fpga: add FPGA device framework Wu Hao <hao.wu@intel.com> - 2017-03-31 14:30 +0200
Re: [PATCH 02/16] fpga: add FPGA device framework matthew.gerlach@linux.intel.com - 2017-03-31 21:10 +0200
Re: [PATCH 02/16] fpga: add FPGA device framework Wu Hao <hao.wu@intel.com> - 2017-04-01 14:30 +0200
Re: [PATCH 02/16] fpga: add FPGA device framework Greg KH <greg@kroah.com> - 2017-03-31 08:20 +0200
Re: [PATCH 02/16] fpga: add FPGA device framework Wu Hao <hao.wu@intel.com> - 2017-03-31 15:40 +0200
Re: [PATCH 02/16] fpga: add FPGA device framework Greg KH <greg@kroah.com> - 2017-03-31 16:20 +0200
Re: [PATCH 02/16] fpga: add FPGA device framework Wu Hao <hao.wu@intel.com> - 2017-04-01 13:50 +0200
[PATCH 03/16] fpga: intel: add FPGA PCIe device driver Wu Hao <hao.wu@intel.com> - 2017-03-30 14:20 +0200
Re: [PATCH 03/16] fpga: intel: add FPGA PCIe device driver Moritz Fischer <mdf@kernel.org> - 2017-04-04 04:20 +0200
RE: [PATCH 03/16] fpga: intel: add FPGA PCIe device driver "Wu, Hao" <hao.wu@intel.com> - 2017-04-05 15:20 +0200
[PATCH 15/16] fpga: intel: afu: add user afu sub feature support Wu Hao <hao.wu@intel.com> - 2017-03-30 14:20 +0200
[PATCH 12/16] fpga: intel: add FPGA Accelerated Function Unit driver basic framework Wu Hao <hao.wu@intel.com> - 2017-03-30 14:20 +0200
[PATCH 09/16] fpga: intel: fme: add header sub feature support Wu Hao <hao.wu@intel.com> - 2017-03-30 14:20 +0200
[PATCH 16/16] fpga: intel: afu: add FPGA_PORT_DMA_MAP/UNMAP ioctls support Wu Hao <hao.wu@intel.com> - 2017-03-30 14:20 +0200
[PATCH 07/16] fpga: intel: add feature device infrastructure Wu Hao <hao.wu@intel.com> - 2017-03-30 14:20 +0200
[PATCH 05/16] fpga: intel: pcie: add chardev support for feature devices Wu Hao <hao.wu@intel.com> - 2017-03-30 14:20 +0200
[PATCH 04/16] fpga: intel: pcie: parse feature list and create platform device for features. Wu Hao <hao.wu@intel.com> - 2017-03-30 14:20 +0200
Re: [PATCH 04/16] fpga: intel: pcie: parse feature list and create platform device for features. Alan Tull <atull@kernel.org> - 2017-04-03 23:50 +0200
Re: [PATCH 04/16] fpga: intel: pcie: parse feature list and create platform device for features. Wu Hao <hao.wu@intel.com> - 2017-04-05 14:10 +0200
Re: [PATCH 04/16] fpga: intel: pcie: parse feature list and create platform device for features. Alan Tull <atull@kernel.org> - 2017-04-05 00:20 +0200
Re: [PATCH 04/16] fpga: intel: pcie: parse feature list and create platform device for features. Wu Hao <hao.wu@intel.com> - 2017-04-05 16:20 +0200
[PATCH 10/16] fpga: intel: fme: add FPGA_GET_API_VERSION/CHECK_EXTENSION ioctls support Wu Hao <hao.wu@intel.com> - 2017-03-30 14:20 +0200
[PATCH 14/16] fpga: intel: afu add FPGA_GET_API_VERSION/CHECK_EXTENSION ioctls support Wu Hao <hao.wu@intel.com> - 2017-03-30 14:20 +0200
[PATCH 01/16] docs: fpga: add a document for Intel FPGA driver overview Wu Hao <hao.wu@intel.com> - 2017-03-30 14:20 +0200
Re: [PATCH 01/16] docs: fpga: add a document for Intel FPGA driver overview matthew.gerlach@linux.intel.com - 2017-03-31 20:30 +0200
Re: [PATCH 01/16] docs: fpga: add a document for Intel FPGA driver overview Alan Tull <atull@kernel.org> - 2017-03-31 20:40 +0200
Re: [PATCH 01/16] docs: fpga: add a document for Intel FPGA driver overview Wu Hao <hao.wu@intel.com> - 2017-04-01 13:30 +0200
Re: [PATCH 01/16] docs: fpga: add a document for Intel FPGA driver overview Moritz Fischer <mdf@kernel.org> - 2017-04-02 16:50 +0200
Re: [PATCH 01/16] docs: fpga: add a document for Intel FPGA driver overview Alan Tull <atull@kernel.org> - 2017-04-03 22:50 +0200
Re: [PATCH 01/16] docs: fpga: add a document for Intel FPGA driver overview Wu Hao <hao.wu@intel.com> - 2017-04-04 07:30 +0200
Re: [PATCH 01/16] docs: fpga: add a document for Intel FPGA driver overview Wu Hao <hao.wu@intel.com> - 2017-04-04 07:20 +0200
[PATCH 06/16] fpga: intel: pcie: adds fpga_for_each_port callback for fme device Wu Hao <hao.wu@intel.com> - 2017-03-30 14:20 +0200
[PATCH 13/16] fpga: intel: afu: add header sub feature support Wu Hao <hao.wu@intel.com> - 2017-03-30 14:20 +0200
[PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support Wu Hao <hao.wu@intel.com> - 2017-03-30 14:20 +0200
Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support Alan Tull <atull@kernel.org> - 2017-03-30 22:40 +0200
Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support Xiao Guangrong <xiaoguangrong.eric@gmail.com> - 2017-03-31 06:20 +0200
Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support Wu Hao <hao.wu@intel.com> - 2017-03-31 11:00 +0200
Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support Alan Tull <atull@kernel.org> - 2017-04-03 22:30 +0200
Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support Wu Hao <hao.wu@intel.com> - 2017-04-04 07:40 +0200
Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support Alan Tull <atull@kernel.org> - 2017-03-31 21:20 +0200
Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support Wu Hao <hao.wu@intel.com> - 2017-04-01 13:20 +0200
Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support Alan Tull <atull@kernel.org> - 2017-04-03 18:40 +0200
Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support Wu Hao <hao.wu@intel.com> - 2017-04-04 08:20 +0200
Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support Alan Tull <atull@kernel.org> - 2017-04-05 00:40 +0200
RE: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support "Wu, Hao" <hao.wu@intel.com> - 2017-04-05 13:50 +0200
Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support Alan Tull <atull@kernel.org> - 2017-04-05 17:30 +0200
Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support Alan Tull <atull@kernel.org> - 2017-04-05 17:50 +0200
Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support Wu Hao <hao.wu@intel.com> - 2017-04-06 13:10 +0200
Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support Alan Tull <atull@kernel.org> - 2017-04-06 21:30 +0200
Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support Wu Hao <hao.wu@intel.com> - 2017-04-07 08:10 +0200
Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support Alan Tull <atull@kernel.org> - 2017-04-03 23:30 +0200
Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support matthew.gerlach@linux.intel.com - 2017-04-04 00:50 +0200
Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support Wu Hao <hao.wu@intel.com> - 2017-04-04 09:00 +0200
Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support Wu Hao <hao.wu@intel.com> - 2017-04-04 08:40 +0200
Re: [PATCH 00/16] Intel FPGA Device Drivers Moritz Fischer <mdf@kernel.org> - 2017-03-30 19:20 +0200
Re: [PATCH 00/16] Intel FPGA Device Drivers Jerome Glisse <jglisse@redhat.com> - 2017-04-06 22:30 +0200
Page 3 of 4 — ← Prev page 1 2 [3] 4 Next page →
| From | Xiao Guangrong <xiaoguangrong.eric@gmail.com> |
|---|---|
| Date | 2017-03-31 06:20 +0200 |
| Subject | Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support |
| Message-ID | <tqWcy-7Zp-1@gated-at.bofh.it> |
| In reply to | #1613462 |
On 31/03/2017 4:30 AM, Alan Tull wrote: > On Thu, Mar 30, 2017 at 7:08 AM, Wu Hao <hao.wu@intel.com> wrote: >> From: Kang Luwei <luwei.kang@intel.com> >> >> Partial Reconfiguration (PR) is the most important function for FME. It >> allows reconfiguration for given Port/Accelerated Function Unit (AFU). >> >> This patch adds support for PR sub feature. In this patch, it registers >> a fpga_mgr and implements fpga_manager_ops, and invoke fpga_mgr_buf_load >> for PR operation once PR request received via ioctl. Below user space >> interfaces are exposed by this sub feature. >> >> Sysfs interface: >> * /sys/class/fpga/<fpga.x>/<intel-fpga-fme.x>/interface_id >> Read-only. Indicate the hardware interface information. Userspace >> applications need to check this interface to select correct green >> bitstream format before PR. >> >> Ioctl interface: >> * FPGA_FME_PORT_PR >> Do partial reconfiguration per information from userspace, including >> target port(AFU), buffer size and address info. It returns the PR status >> (PR error code if failed) to userspace. >> >> Signed-off-by: Tim Whisonant <tim.whisonant@intel.com> >> Signed-off-by: Enno Luebbers <enno.luebbers@intel.com> >> Signed-off-by: Shiva Rao <shiva.rao@intel.com> >> Signed-off-by: Christopher Rauer <christopher.rauer@intel.com> >> Signed-off-by: Alan Tull <alan.tull@intel.com> > > Hi Wu Hao, > > Thanks for submitting your patches. > > I think there's been a misunderstanding of the meaning of > 'Signed-off-by' [1]. I have not signed off on this code or had a hand > in its development. But I'm happy to get to review it now. It will > take a bit of time; I expect to be replying next week. Hi Alan, Sorry to confuse you, i think it's because you helped Chris a lot to implement this interface and we'd like to include your credit as this way. If you dislike, it will be dropped. :) Thanks for your review in advance.
[toc] | [prev] | [next] | [standalone]
| From | Wu Hao <hao.wu@intel.com> |
|---|---|
| Date | 2017-03-31 11:00 +0200 |
| Subject | Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support |
| Message-ID | <tr0zw-2ht-31@gated-at.bofh.it> |
| In reply to | #1613625 |
On Fri, Mar 31, 2017 at 12:11:12PM +0800, Xiao Guangrong wrote: > On 31/03/2017 4:30 AM, Alan Tull wrote: > >On Thu, Mar 30, 2017 at 7:08 AM, Wu Hao <hao.wu@intel.com> wrote: > >>From: Kang Luwei <luwei.kang@intel.com> > >> > >>Partial Reconfiguration (PR) is the most important function for FME. It > >>allows reconfiguration for given Port/Accelerated Function Unit (AFU). > >> > >>This patch adds support for PR sub feature. In this patch, it registers > >>a fpga_mgr and implements fpga_manager_ops, and invoke fpga_mgr_buf_load > >>for PR operation once PR request received via ioctl. Below user space > >>interfaces are exposed by this sub feature. > >> > >>Sysfs interface: > >>* /sys/class/fpga/<fpga.x>/<intel-fpga-fme.x>/interface_id > >> Read-only. Indicate the hardware interface information. Userspace > >> applications need to check this interface to select correct green > >> bitstream format before PR. > >> > >>Ioctl interface: > >>* FPGA_FME_PORT_PR > >> Do partial reconfiguration per information from userspace, including > >> target port(AFU), buffer size and address info. It returns the PR status > >> (PR error code if failed) to userspace. > >> > >>Signed-off-by: Tim Whisonant <tim.whisonant@intel.com> > >>Signed-off-by: Enno Luebbers <enno.luebbers@intel.com> > >>Signed-off-by: Shiva Rao <shiva.rao@intel.com> > >>Signed-off-by: Christopher Rauer <christopher.rauer@intel.com> > >>Signed-off-by: Alan Tull <alan.tull@intel.com> > > > >Hi Wu Hao, > > > >Thanks for submitting your patches. > > > >I think there's been a misunderstanding of the meaning of > >'Signed-off-by' [1]. I have not signed off on this code or had a hand > >in its development. But I'm happy to get to review it now. It will > >take a bit of time; I expect to be replying next week. > > Hi Alan, > > Sorry to confuse you, i think it's because you helped Chris a lot to > implement this interface and we'd like to include your credit as this > way. If you dislike, it will be dropped. :) > > Thanks for your review in advance. > Hi Alan, Sorry about this, we should ask you firstly before doing it this way. Let me know if you don't like it, I will drop it in the next version. Many thanks for your time and review on these patches. Look forward for your feedback and comments. :) Thanks Hao
[toc] | [prev] | [next] | [standalone]
| From | Alan Tull <atull@kernel.org> |
|---|---|
| Date | 2017-04-03 22:30 +0200 |
| Subject | Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support |
| Message-ID | <tsgLT-3gI-17@gated-at.bofh.it> |
| In reply to | #1613756 |
On Fri, Mar 31, 2017 at 3:50 AM, Wu Hao <hao.wu@intel.com> wrote: > On Fri, Mar 31, 2017 at 12:11:12PM +0800, Xiao Guangrong wrote: >> On 31/03/2017 4:30 AM, Alan Tull wrote: >> >On Thu, Mar 30, 2017 at 7:08 AM, Wu Hao <hao.wu@intel.com> wrote: >> >>From: Kang Luwei <luwei.kang@intel.com> >> >> >> >>Partial Reconfiguration (PR) is the most important function for FME. It >> >>allows reconfiguration for given Port/Accelerated Function Unit (AFU). >> >> >> >>This patch adds support for PR sub feature. In this patch, it registers >> >>a fpga_mgr and implements fpga_manager_ops, and invoke fpga_mgr_buf_load >> >>for PR operation once PR request received via ioctl. Below user space >> >>interfaces are exposed by this sub feature. >> >> >> >>Sysfs interface: >> >>* /sys/class/fpga/<fpga.x>/<intel-fpga-fme.x>/interface_id >> >> Read-only. Indicate the hardware interface information. Userspace >> >> applications need to check this interface to select correct green >> >> bitstream format before PR. >> >> >> >>Ioctl interface: >> >>* FPGA_FME_PORT_PR >> >> Do partial reconfiguration per information from userspace, including >> >> target port(AFU), buffer size and address info. It returns the PR status >> >> (PR error code if failed) to userspace. >> >> >> >>Signed-off-by: Tim Whisonant <tim.whisonant@intel.com> >> >>Signed-off-by: Enno Luebbers <enno.luebbers@intel.com> >> >>Signed-off-by: Shiva Rao <shiva.rao@intel.com> >> >>Signed-off-by: Christopher Rauer <christopher.rauer@intel.com> >> >>Signed-off-by: Alan Tull <alan.tull@intel.com> >> > >> >Hi Wu Hao, >> > >> >Thanks for submitting your patches. >> > >> >I think there's been a misunderstanding of the meaning of >> >'Signed-off-by' [1]. I have not signed off on this code or had a hand >> >in its development. But I'm happy to get to review it now. It will >> >take a bit of time; I expect to be replying next week. >> >> Hi Alan, >> >> Sorry to confuse you, i think it's because you helped Chris a lot to >> implement this interface and we'd like to include your credit as this >> way. If you dislike, it will be dropped. :) >> >> Thanks for your review in advance. >> > > Hi Alan, > > Sorry about this, we should ask you firstly before doing it this way. > Let me know if you don't like it, I will drop it in the next version. Yes please drop the signed-off-by: me in the next version. Also, you don't need to cc my Intel email address. Alan > > Many thanks for your time and review on these patches. Look forward > for your feedback and comments. :) > > Thanks > Hao
[toc] | [prev] | [next] | [standalone]
| From | Wu Hao <hao.wu@intel.com> |
|---|---|
| Date | 2017-04-04 07:40 +0200 |
| Subject | Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support |
| Message-ID | <tspm9-xi-11@gated-at.bofh.it> |
| In reply to | #1615540 |
On Mon, Apr 03, 2017 at 03:26:14PM -0500, Alan Tull wrote: > On Fri, Mar 31, 2017 at 3:50 AM, Wu Hao <hao.wu@intel.com> wrote: > > On Fri, Mar 31, 2017 at 12:11:12PM +0800, Xiao Guangrong wrote: > >> On 31/03/2017 4:30 AM, Alan Tull wrote: > >> >On Thu, Mar 30, 2017 at 7:08 AM, Wu Hao <hao.wu@intel.com> wrote: > >> >>From: Kang Luwei <luwei.kang@intel.com> > >> >> > >> >>Partial Reconfiguration (PR) is the most important function for FME. It > >> >>allows reconfiguration for given Port/Accelerated Function Unit (AFU). > >> >> > >> >>This patch adds support for PR sub feature. In this patch, it registers > >> >>a fpga_mgr and implements fpga_manager_ops, and invoke fpga_mgr_buf_load > >> >>for PR operation once PR request received via ioctl. Below user space > >> >>interfaces are exposed by this sub feature. > >> >> > >> >>Sysfs interface: > >> >>* /sys/class/fpga/<fpga.x>/<intel-fpga-fme.x>/interface_id > >> >> Read-only. Indicate the hardware interface information. Userspace > >> >> applications need to check this interface to select correct green > >> >> bitstream format before PR. > >> >> > >> >>Ioctl interface: > >> >>* FPGA_FME_PORT_PR > >> >> Do partial reconfiguration per information from userspace, including > >> >> target port(AFU), buffer size and address info. It returns the PR status > >> >> (PR error code if failed) to userspace. > >> >> > >> >>Signed-off-by: Tim Whisonant <tim.whisonant@intel.com> > >> >>Signed-off-by: Enno Luebbers <enno.luebbers@intel.com> > >> >>Signed-off-by: Shiva Rao <shiva.rao@intel.com> > >> >>Signed-off-by: Christopher Rauer <christopher.rauer@intel.com> > >> >>Signed-off-by: Alan Tull <alan.tull@intel.com> > >> > > >> >Hi Wu Hao, > >> > > >> >Thanks for submitting your patches. > >> > > >> >I think there's been a misunderstanding of the meaning of > >> >'Signed-off-by' [1]. I have not signed off on this code or had a hand > >> >in its development. But I'm happy to get to review it now. It will > >> >take a bit of time; I expect to be replying next week. > >> > >> Hi Alan, > >> > >> Sorry to confuse you, i think it's because you helped Chris a lot to > >> implement this interface and we'd like to include your credit as this > >> way. If you dislike, it will be dropped. :) > >> > >> Thanks for your review in advance. > >> > > > > Hi Alan, > > > > Sorry about this, we should ask you firstly before doing it this way. > > Let me know if you don't like it, I will drop it in the next version. > > Yes please drop the signed-off-by: me in the next version. Also, you > don't need to cc my Intel email address. > Sure, will do. Hao
[toc] | [prev] | [next] | [standalone]
| From | Alan Tull <atull@kernel.org> |
|---|---|
| Date | 2017-03-31 21:20 +0200 |
| Subject | Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support |
| Message-ID | <trafw-iO-17@gated-at.bofh.it> |
| In reply to | #1613019 |
On Thu, Mar 30, 2017 at 7:08 AM, Wu Hao <hao.wu@intel.com> wrote:
> From: Kang Luwei <luwei.kang@intel.com>
>
> Partial Reconfiguration (PR) is the most important function for FME. It
> allows reconfiguration for given Port/Accelerated Function Unit (AFU).
>
> This patch adds support for PR sub feature. In this patch, it registers
> a fpga_mgr and implements fpga_manager_ops, and invoke fpga_mgr_buf_load
> for PR operation once PR request received via ioctl. Below user space
> interfaces are exposed by this sub feature.
>
> Sysfs interface:
> * /sys/class/fpga/<fpga.x>/<intel-fpga-fme.x>/interface_id
> Read-only. Indicate the hardware interface information. Userspace
> applications need to check this interface to select correct green
> bitstream format before PR.
>
> Ioctl interface:
> * FPGA_FME_PORT_PR
> Do partial reconfiguration per information from userspace, including
> target port(AFU), buffer size and address info. It returns the PR status
> (PR error code if failed) to userspace.
>
> Signed-off-by: Tim Whisonant <tim.whisonant@intel.com>
> Signed-off-by: Enno Luebbers <enno.luebbers@intel.com>
> Signed-off-by: Shiva Rao <shiva.rao@intel.com>
> Signed-off-by: Christopher Rauer <christopher.rauer@intel.com>
> Signed-off-by: Alan Tull <alan.tull@intel.com>
> Signed-off-by: Kang Luwei <luwei.kang@intel.com>
> Signed-off-by: Xiao Guangrong <guangrong.xiao@linux.intel.com>
> Signed-off-by: Wu Hao <hao.wu@intel.com>
> ---
> drivers/fpga/intel/Makefile | 2 +-
> drivers/fpga/intel/feature-dev.h | 58 ++++++
> drivers/fpga/intel/fme-main.c | 44 ++++-
> drivers/fpga/intel/fme-pr.c | 400 +++++++++++++++++++++++++++++++++++++++
> drivers/fpga/intel/fme.h | 32 ++++
> include/uapi/linux/intel-fpga.h | 44 +++++
> 6 files changed, 578 insertions(+), 2 deletions(-)
> create mode 100644 drivers/fpga/intel/fme-pr.c
> create mode 100644 drivers/fpga/intel/fme.h
>
> diff --git a/drivers/fpga/intel/Makefile b/drivers/fpga/intel/Makefile
> index 546861d..0452cb6 100644
> --- a/drivers/fpga/intel/Makefile
> +++ b/drivers/fpga/intel/Makefile
> @@ -2,4 +2,4 @@ obj-$(CONFIG_INTEL_FPGA_PCI) += intel-fpga-pci.o
> obj-$(CONFIG_INTEL_FPGA_FME) += intel-fpga-fme.o
>
> intel-fpga-pci-objs := pcie.o feature-dev.o
> -intel-fpga-fme-objs := fme-main.o
> +intel-fpga-fme-objs := fme-main.o fme-pr.o
> diff --git a/drivers/fpga/intel/feature-dev.h b/drivers/fpga/intel/feature-dev.h
> index dccc283..5a25c915 100644
> --- a/drivers/fpga/intel/feature-dev.h
> +++ b/drivers/fpga/intel/feature-dev.h
> @@ -150,8 +150,66 @@ struct feature_fme_err {
> };
>
> /* FME Partial Reconfiguration Sub Feature Register Set */
> +/* FME PR Control Register */
> +struct feature_fme_pr_ctl {
> + union {
> + u64 csr;
> + struct {
> + u8 pr_reset:1; /* Reset PR Engine */
> + u8 rsvdz1:3;
> + u8 pr_reset_ack:1; /* Reset PR Engine Ack */
> + u8 rsvdz2:3;
> + u8 pr_regionid:2; /* PR Region ID */
> + u8 rsvdz3:2;
> + u8 pr_start_req:1; /* PR Start Request */
> + u8 pr_push_complete:1; /* PR Data push complete */
> + u8 pr_kind:1; /* Load Customer or Intel GBS */
> + u32 rsvdz4:17;
> + u32 config_data;
> + };
> + };
> +};
> +
> +/* FME PR Status Register */
> +struct feature_fme_pr_status {
> + union {
> + u64 csr;
> + struct {
> + u16 pr_credit:9; /* Number of PR Credits */
> + u8 rsvdz1:7;
> + u8 pr_status:1; /* PR Operation status */
> + u8 rsvdz2:3;
> + u8 pr_ctrlr_status:3; /* Controller status */
> + u8 rsvdz3:1;
> + u8 pr_host_status:4; /* PR Host status */
> + u64 rsvdz4:36;
> + };
> + };
> +};
> +
> +/* FME PR Data Register */
> +struct feature_fme_pr_data {
> + union {
> + u64 csr;
> + struct {
> + /* PR data from the raw-binary file */
> + u32 pr_data_raw;
> + u32 rsvd;
> + };
> + };
> +};
> +
> struct feature_fme_pr {
> struct feature_header header;
> + struct feature_fme_pr_ctl control;
> + struct feature_fme_pr_status status;
> + struct feature_fme_pr_data data;
> + u64 error;
> +
> + u64 rsvd[16];
> +
> + u64 intfc_id_l; /* PR interface Id Low */
> + u64 intfc_id_h; /* PR interface Id High */
> };
>
> /* PORT Header Register Set */
> diff --git a/drivers/fpga/intel/fme-main.c b/drivers/fpga/intel/fme-main.c
> index 36d0c4c..0d9a7a6 100644
> --- a/drivers/fpga/intel/fme-main.c
> +++ b/drivers/fpga/intel/fme-main.c
> @@ -23,6 +23,7 @@
> #include <linux/intel-fpga.h>
>
> #include "feature-dev.h"
> +#include "fme.h"
>
> static ssize_t ports_num_show(struct device *dev,
> struct device_attribute *attr, char *buf)
> @@ -101,6 +102,10 @@ static struct feature_driver fme_feature_drvs[] = {
> .ops = &fme_hdr_ops,
> },
> {
> + .name = FME_FEATURE_PR_MGMT,
> + .ops = &pr_mgmt_ops,
> + },
> + {
> .ops = NULL,
> },
> };
> @@ -182,14 +187,48 @@ static const struct file_operations fme_fops = {
> .unlocked_ioctl = fme_ioctl,
> };
>
> +static int fme_dev_init(struct platform_device *pdev)
> +{
> + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> + struct fpga_fme *fme;
> +
> + fme = devm_kzalloc(&pdev->dev, sizeof(*fme), GFP_KERNEL);
> + if (!fme)
> + return -ENOMEM;
> +
> + fme->pdata = pdata;
> +
> + mutex_lock(&pdata->lock);
> + fpga_pdata_set_private(pdata, fme);
> + mutex_unlock(&pdata->lock);
> + return 0;
> +}
> +
> +static void fme_dev_destroy(struct platform_device *pdev)
> +{
> + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> + struct fpga_fme *fme;
> +
> + mutex_lock(&pdata->lock);
> + fme = fpga_pdata_get_private(pdata);
> + fpga_pdata_set_private(pdata, NULL);
> + mutex_unlock(&pdata->lock);
> +
> + devm_kfree(&pdev->dev, fme);
> +}
> +
> static int fme_probe(struct platform_device *pdev)
> {
> int ret;
>
> - ret = fpga_dev_feature_init(pdev, fme_feature_drvs);
> + ret = fme_dev_init(pdev);
> if (ret)
> goto exit;
>
> + ret = fpga_dev_feature_init(pdev, fme_feature_drvs);
> + if (ret)
> + goto dev_destroy;
> +
> ret = fpga_register_dev_ops(pdev, &fme_fops, THIS_MODULE);
> if (ret)
> goto feature_uinit;
> @@ -198,6 +237,8 @@ static int fme_probe(struct platform_device *pdev)
>
> feature_uinit:
> fpga_dev_feature_uinit(pdev);
> +dev_destroy:
> + fme_dev_destroy(pdev);
> exit:
> return ret;
> }
> @@ -206,6 +247,7 @@ static int fme_remove(struct platform_device *pdev)
> {
> fpga_dev_feature_uinit(pdev);
> fpga_unregister_dev_ops(pdev);
> + fme_dev_destroy(pdev);
> return 0;
> }
>
> diff --git a/drivers/fpga/intel/fme-pr.c b/drivers/fpga/intel/fme-pr.c
> new file mode 100644
> index 0000000..3b44a3e
> --- /dev/null
> +++ b/drivers/fpga/intel/fme-pr.c
> @@ -0,0 +1,400 @@
> +/*
> + * Driver for Intel FPGA Management Engine (FME) Partial Reconfiguration
> + *
> + * Copyright (C) 2017 Intel Corporation, Inc.
> + *
> + * Authors:
> + * Kang Luwei <luwei.kang@intel.com>
> + * Xiao Guangrong <guangrong.xiao@linux.intel.com>
> + * Joseph Grecco <joe.grecco@intel.com>
> + * Enno Luebbers <enno.luebbers@intel.com>
> + * Tim Whisonant <tim.whisonant@intel.com>
> + * Ananda Ravuri <ananda.ravuri@intel.com>
> + * Christopher Rauer <christopher.rauer@intel.com>
> + * Henry Mitchel <henry.mitchel@intel.com>
> + *
> + * This work is licensed under a dual BSD/GPLv2 license. When using or
> + * redistributing this file, you may do so under either license. See the
> + * LICENSE.BSD file under this directory for the BSD license and see
> + * the COPYING file in the top-level directory for the GPLv2 license.
> + */
> +
> +#include <linux/types.h>
> +#include <linux/device.h>
> +#include <linux/vmalloc.h>
> +#include <linux/uaccess.h>
> +#include <linux/fpga/fpga-mgr.h>
> +#include <linux/intel-fpga.h>
> +
> +#include "feature-dev.h"
> +#include "fme.h"
> +
> +#define PR_WAIT_TIMEOUT 8000000
> +
> +#define PR_HOST_STATUS_IDLE 0
> +
> +DEFINE_FPGA_PR_ERR_MSG(pr_err_msg);
> +
> +static ssize_t interface_id_show(struct device *dev,
> + struct device_attribute *attr, char *buf)
> +{
> + u64 intfc_id_l, intfc_id_h;
> + struct feature_fme_pr *fme_pr
> + = get_feature_ioaddr_by_index(dev, FME_FEATURE_ID_PR_MGMT);
> +
> + intfc_id_l = readq(&fme_pr->intfc_id_l);
> + intfc_id_h = readq(&fme_pr->intfc_id_h);
> +
> + return scnprintf(buf, PAGE_SIZE, "%016llx%016llx\n",
> + (unsigned long long)intfc_id_h,
> + (unsigned long long)intfc_id_l);
> +}
> +static DEVICE_ATTR_RO(interface_id);
> +
> +static struct attribute *pr_mgmt_attrs[] = {
> + &dev_attr_interface_id.attr,
> + NULL,
> +};
> +
> +struct attribute_group pr_mgmt_attr_group = {
> + .attrs = pr_mgmt_attrs,
> + .name = "pr",
> +};
> +
> +static u64
> +pr_err_handle(struct platform_device *pdev, struct feature_fme_pr *fme_pr)
> +{
> + struct feature_fme_pr_status fme_pr_status;
> + unsigned long err_code;
> + u64 fme_pr_error;
> + int i = 0;
> +
> + fme_pr_status.csr = readq(&fme_pr->status);
> + if (!fme_pr_status.pr_status)
> + return 0;
> +
> + err_code = fme_pr_error = readq(&fme_pr->error);
> + for_each_set_bit(i, &err_code, PR_MAX_ERR_NUM)
> + dev_dbg(&pdev->dev, "%s\n", pr_err_msg[i]);
> + writeq(fme_pr_error, &fme_pr->error);
> + return fme_pr_error;
> +}
> +
> +static int fme_pr_write_init(struct fpga_manager *mgr,
> + struct fpga_image_info *info, const char *buf, size_t count)
> +{
> + struct fpga_fme *fme = mgr->priv;
> + struct platform_device *pdev;
> + struct feature_fme_pr *fme_pr;
> + struct feature_fme_pr_ctl fme_pr_ctl;
> + struct feature_fme_pr_status fme_pr_status;
> +
> + pdev = fme->pdata->dev;
> + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
> + FME_FEATURE_ID_PR_MGMT);
> + if (!fme_pr)
> + return -EINVAL;
> +
> + if (WARN_ON(info->flags != FPGA_MGR_PARTIAL_RECONFIG))
> + return -EINVAL;
> +
> + dev_dbg(&pdev->dev, "resetting PR before initiated PR\n");
> +
> + fme_pr_ctl.csr = readq(&fme_pr->control);
> + fme_pr_ctl.pr_reset = 1;
> + writeq(fme_pr_ctl.csr, &fme_pr->control);
> +
> + fme_pr_ctl.pr_reset_ack = 1;
> +
> + if (fpga_wait_register_field(pr_reset_ack, fme_pr_ctl,
> + &fme_pr->control, PR_WAIT_TIMEOUT, 1)) {
> + dev_err(&pdev->dev, "maximum PR timeout\n");
> + return -ETIMEDOUT;
> + }
> +
> + fme_pr_ctl.csr = readq(&fme_pr->control);
> + fme_pr_ctl.pr_reset = 0;
> + writeq(fme_pr_ctl.csr, &fme_pr->control);
> +
> + dev_dbg(&pdev->dev,
> + "waiting for PR resource in HW to be initialized and ready\n");
> +
> + fme_pr_status.pr_host_status = PR_HOST_STATUS_IDLE;
> +
> + if (fpga_wait_register_field(pr_host_status, fme_pr_status,
> + &fme_pr->status, PR_WAIT_TIMEOUT, 1)) {
> + dev_err(&pdev->dev, "maximum PR timeout\n");
> + return -ETIMEDOUT;
> + }
> +
> + dev_dbg(&pdev->dev, "check if have any previous PR error\n");
> + pr_err_handle(pdev, fme_pr);
> + return 0;
> +}
> +
> +static int fme_pr_write(struct fpga_manager *mgr,
> + const char *buf, size_t count)
> +{
> + struct fpga_fme *fme = mgr->priv;
> + struct platform_device *pdev;
> + struct feature_fme_pr *fme_pr;
> + struct feature_fme_pr_ctl fme_pr_ctl;
> + struct feature_fme_pr_status fme_pr_status;
> + struct feature_fme_pr_data fme_pr_data;
> + int delay, pr_credit, i = 0;
> +
> + pdev = fme->pdata->dev;
> + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
> + FME_FEATURE_ID_PR_MGMT);
> +
> + dev_dbg(&pdev->dev, "set PR port ID and start request\n");
> +
> + fme_pr_ctl.csr = readq(&fme_pr->control);
> + fme_pr_ctl.pr_regionid = fme->port_id;
> + fme_pr_ctl.pr_start_req = 1;
> + writeq(fme_pr_ctl.csr, &fme_pr->control);
> +
> + dev_dbg(&pdev->dev, "pushing data from bitstream to HW\n");
> +
> + fme_pr_status.csr = readq(&fme_pr->status);
> + pr_credit = fme_pr_status.pr_credit;
> +
> + while (count > 0) {
> + delay = 0;
> + while (pr_credit <= 1) {
> + if (delay++ > PR_WAIT_TIMEOUT) {
> + dev_err(&pdev->dev, "maximum try\n");
> + return -ETIMEDOUT;
> + }
> + udelay(1);
> +
> + fme_pr_status.csr = readq(&fme_pr->status);
> + pr_credit = fme_pr_status.pr_credit;
> + };
> +
> + if (count >= 4) {
> + fme_pr_data.rsvd = 0;
> + fme_pr_data.pr_data_raw = *(((u32 *)buf) + i);
> + writeq(fme_pr_data.csr, &fme_pr->data);
> + count -= 4;
> + pr_credit--;
> + i++;
> + } else {
> + WARN_ON(1);
> + return -EINVAL;
> + }
> + }
> +
> + return 0;
> +}
> +
> +static int fme_pr_write_complete(struct fpga_manager *mgr,
> + struct fpga_image_info *info)
> +{
> + struct fpga_fme *fme = mgr->priv;
> + struct platform_device *pdev;
> + struct feature_fme_pr *fme_pr;
> + struct feature_fme_pr_ctl fme_pr_ctl;
> +
> + pdev = fme->pdata->dev;
> + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
> + FME_FEATURE_ID_PR_MGMT);
> +
> + fme_pr_ctl.csr = readq(&fme_pr->control);
> + fme_pr_ctl.pr_push_complete = 1;
> + writeq(fme_pr_ctl.csr, &fme_pr->control);
> +
> + dev_dbg(&pdev->dev, "green bitstream push complete\n");
> + dev_dbg(&pdev->dev, "waiting for HW to release PR resource\n");
> +
> + fme_pr_ctl.pr_start_req = 0;
> +
> + if (fpga_wait_register_field(pr_start_req, fme_pr_ctl,
> + &fme_pr->control, PR_WAIT_TIMEOUT, 1)) {
> + dev_err(&pdev->dev, "maximum try.\n");
> + return -ETIMEDOUT;
> + }
> +
> + dev_dbg(&pdev->dev, "PR operation complete, checking status\n");
> + fme->pr_err = pr_err_handle(pdev, fme_pr);
> + if (fme->pr_err)
> + return -EIO;
> +
> + dev_dbg(&pdev->dev, "PR done successfully\n");
> + return 0;
> +}
> +
> +static enum fpga_mgr_states fme_pr_state(struct fpga_manager *mgr)
> +{
> + return FPGA_MGR_STATE_UNKNOWN;
> +}
> +
> +static const struct fpga_manager_ops fme_pr_ops = {
> + .write_init = fme_pr_write_init,
> + .write = fme_pr_write,
> + .write_complete = fme_pr_write_complete,
> + .state = fme_pr_state,
> +};
> +
> +static int fme_pr(struct platform_device *pdev, unsigned long arg)
> +{
> + void __user *argp = (void __user *)arg;
> + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> + struct fpga_fme *fme;
> + struct fpga_manager *mgr;
> + struct feature_fme_header *fme_hdr;
> + struct feature_fme_capability fme_capability;
> + struct fpga_image_info info;
> + struct fpga_fme_port_pr port_pr;
> + struct platform_device *port;
> + unsigned long minsz;
> + void *buf = NULL;
> + int ret = 0;
> +
> + minsz = offsetofend(struct fpga_fme_port_pr, status);
> +
> + if (copy_from_user(&port_pr, argp, minsz))
> + return -EFAULT;
> +
> + if (port_pr.argsz < minsz || port_pr.flags)
> + return -EINVAL;
> +
> + if (!IS_ALIGNED(port_pr.buffer_size, 4))
> + return -EINVAL;
> +
> + /* get fme header region */
> + fme_hdr = get_feature_ioaddr_by_index(&pdev->dev,
> + FME_FEATURE_ID_HEADER);
> + if (WARN_ON(!fme_hdr))
> + return -EINVAL;
> +
> + /* check port id */
> + fme_capability.csr = readq(&fme_hdr->capability);
> + if (port_pr.port_id >= fme_capability.num_ports) {
> + dev_dbg(&pdev->dev, "port number more than maximum\n");
> + return -EINVAL;
> + }
> +
> + if (!access_ok(VERIFY_READ, port_pr.buffer_address,
> + port_pr.buffer_size))
> + return -EFAULT;
> +
> + buf = vmalloc(port_pr.buffer_size);
> + if (!buf)
> + return -ENOMEM;
> +
> + if (copy_from_user(buf, (void __user *)port_pr.buffer_address,
> + port_pr.buffer_size)) {
> + ret = -EFAULT;
> + goto free_exit;
> + }
> +
> + memset(&info, 0, sizeof(struct fpga_image_info));
> + info.flags = FPGA_MGR_PARTIAL_RECONFIG;
> +
> + mgr = fpga_mgr_get(&pdev->dev);
> + if (IS_ERR(mgr)) {
> + ret = PTR_ERR(mgr);
> + goto free_exit;
> + }
> +
> + mutex_lock(&pdata->lock);
> + fme = fpga_pdata_get_private(pdata);
> + /* fme device has been unregistered. */
> + if (!fme) {
> + ret = -EINVAL;
> + goto unlock_exit;
> + }
> +
> + fme->pr_err = 0;
> + fme->port_id = port_pr.port_id;
It looks like you're using private data to communicate with the
driver, i.e. there is something you want to do with the fpga manager
framework and it doesn't have that feature. The better way would be
for us to expand the framework so you don't need to do that.
port_id is the kind of thing that should be communicated to the driver
through fpga_image_info, so we could add that to the struct. Should
we call it port_id? Or is there something more generic that may be
useful in the future for other architectures?.
pr_err appears to be machine specific error codes that are
communicated outside your low level driver. (Calling it pr_err is
extra confusing since Linux already has a commonly name function by
the same name). The framework has state, but that's not doing what
you want here. Maybe we could add a framework ops called status so
that status could be communicated from the low level driver. It would
be useful to abstract the machine specific state to a defined enum
that would be part of the fpga mgr framework. I remember we had
something like that in the earliest version of fpga manager but it got
changed to state rather than status for some reason.
mgr->dev won't work for you for some of the things you are using fme->pdev for?
Alan
> +
> + /* Find and get port device by index */
> + port = pdata->fpga_for_each_port(pdev, &fme->port_id,
> + fpga_port_check_id);
> + WARN_ON(!port);
> +
> + /* Disable Port before PR */
> + fpga_port_disable(port);
> +
> + ret = fpga_mgr_buf_load(mgr, &info, buf, port_pr.buffer_size);
> + port_pr.status = fme->pr_err;
> +
> + /* Re-enable Port after PR finished */
> + fpga_port_enable(port);
> +
> + put_device(&port->dev);
> +
> +unlock_exit:
> + mutex_unlock(&pdata->lock);
> + fpga_mgr_put(mgr);
> +free_exit:
> + vfree(buf);
> + if (copy_to_user((void __user *)arg, &port_pr, minsz))
> + return -EFAULT;
> + return ret;
> +}
> +
> +static int fpga_fme_pr_probe(struct platform_device *pdev)
> +{
> + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> + struct fpga_fme *priv;
> + int ret;
> +
> + mutex_lock(&pdata->lock);
> + priv = fpga_pdata_get_private(pdata);
> + ret = fpga_mgr_register(&pdata->dev->dev,
> + "Intel FPGA Manager", &fme_pr_ops, priv);
> + mutex_unlock(&pdata->lock);
> +
> + return ret;
> +}
> +
> +static int fpga_fme_pr_remove(struct platform_device *pdev)
> +{
> + fpga_mgr_unregister(&pdev->dev);
> + return 0;
> +}
> +
> +static int pr_mgmt_init(struct platform_device *pdev, struct feature *feature)
> +{
> + int ret;
> +
> + ret = fpga_fme_pr_probe(pdev);
> + if (ret)
> + return ret;
> +
> + ret = sysfs_create_group(&pdev->dev.kobj, &pr_mgmt_attr_group);
> + if (ret)
> + fpga_fme_pr_remove(pdev);
> +
> + return ret;
> +}
> +
> +static void pr_mgmt_uinit(struct platform_device *pdev, struct feature *feature)
> +{
> + sysfs_remove_group(&pdev->dev.kobj, &pr_mgmt_attr_group);
> + fpga_fme_pr_remove(pdev);
> +}
> +
> +static long fme_pr_ioctl(struct platform_device *pdev, struct feature *feature,
> + unsigned int cmd, unsigned long arg)
> +{
> + long ret;
> +
> + switch (cmd) {
> + case FPGA_FME_PORT_PR:
> + ret = fme_pr(pdev, arg);
> + break;
> + default:
> + ret = -ENODEV;
> + }
> +
> + return ret;
> +}
> +
> +struct feature_ops pr_mgmt_ops = {
> + .init = pr_mgmt_init,
> + .uinit = pr_mgmt_uinit,
> + .ioctl = fme_pr_ioctl,
> +};
> diff --git a/drivers/fpga/intel/fme.h b/drivers/fpga/intel/fme.h
> new file mode 100644
> index 0000000..d6cb7ce
> --- /dev/null
> +++ b/drivers/fpga/intel/fme.h
> @@ -0,0 +1,32 @@
> +/*
> + * Header file for Intel FPGA Management Engine (FME) Driver
> + *
> + * Copyright (C) 2017 Intel Corporation, Inc.
> + *
> + * Authors:
> + * Kang Luwei <luwei.kang@intel.com>
> + * Xiao Guangrong <guangrong.xiao@linux.intel.com>
> + * Joseph Grecco <joe.grecco@intel.com>
> + * Enno Luebbers <enno.luebbers@intel.com>
> + * Tim Whisonant <tim.whisonant@intel.com>
> + * Ananda Ravuri <ananda.ravuri@intel.com>
> + * Henry Mitchel <henry.mitchel@intel.com>
> + *
> + * This work is licensed under a dual BSD/GPLv2 license. When using or
> + * redistributing this file, you may do so under either license. See the
> + * LICENSE.BSD file under this directory for the BSD license and see
> + * the COPYING file in the top-level directory for the GPLv2 license.
> + */
> +
> +#ifndef __INTEL_FME_H
> +#define __INTEL_FME_H
> +
> +struct fpga_fme {
> + u8 port_id;
> + u64 pr_err;
> + struct feature_platform_data *pdata;
> +};
> +
> +extern struct feature_ops pr_mgmt_ops;
> +
> +#endif
> diff --git a/include/uapi/linux/intel-fpga.h b/include/uapi/linux/intel-fpga.h
> index 992e556..77658316 100644
> --- a/include/uapi/linux/intel-fpga.h
> +++ b/include/uapi/linux/intel-fpga.h
> @@ -18,6 +18,8 @@
> #ifndef _UAPI_LINUX_INTEL_FPGA_H
> #define _UAPI_LINUX_INTEL_FPGA_H
>
> +#include <linux/types.h>
> +
> #define FPGA_API_VERSION 0
>
> /*
> @@ -30,6 +32,7 @@
> #define FPGA_MAGIC 0xB6
>
> #define FPGA_BASE 0
> +#define FME_BASE 0x80
>
> /**
> * FPGA_GET_API_VERSION - _IO(FPGA_MAGIC, FPGA_BASE + 0)
> @@ -49,4 +52,45 @@
>
> #define FPGA_CHECK_EXTENSION _IO(FPGA_MAGIC, FPGA_BASE + 1)
>
> +/* IOCTLs for FME file descriptor */
> +
> +/**
> + * FPGA_FME_PORT_PR - _IOWR(FPGA_MAGIC, FME_BASE + 0, struct fpga_fme_port_pr)
> + *
> + * Driver does Partial Reconfiguration based on Port ID and Buffer (Image)
> + * provided by caller.
> + * Return: 0 on success, -errno on failure.
> + * If FPGA_FME_PORT_PR returns -EIO, that indicates the HW has detected
> + * some errors during PR, under this case, the user can fetch HW error code
> + * from fpga_fme_port_pr.status. Each bit on the error code is used as the
> + * index for the array created by DEFINE_FPGA_PR_ERR_MSG().
> + * Otherwise, it is always zero.
> + */
> +
> +#define DEFINE_FPGA_PR_ERR_MSG(_name_) \
> +static const char * const _name_[] = { \
> + "PR operation error detected", \
> + "PR CRC error detected", \
> + "PR incompatiable bitstream error detected", \
> + "PR IP protocol error detected", \
> + "PR FIFO overflow error detected", \
> + "Reserved", \
> + "PR secure load error detected", \
> +}
> +
> +#define PR_MAX_ERR_NUM 7
> +
> +struct fpga_fme_port_pr {
> + /* Input */
> + __u32 argsz; /* Structure length */
> + __u32 flags; /* Zero for now */
> + __u32 port_id;
> + __u32 buffer_size;
> + __u64 buffer_address; /* Userspace address to the buffer for PR */
> + /* Output */
> + __u64 status; /* HW error code if ioctl returns -EIO */
> +};
> +
> +#define FPGA_FME_PORT_PR _IO(FPGA_MAGIC, FME_BASE + 0)
> +
> #endif /* _UAPI_INTEL_FPGA_H */
> --
> 2.7.4
>
[toc] | [prev] | [next] | [standalone]
| From | Wu Hao <hao.wu@intel.com> |
|---|---|
| Date | 2017-04-01 13:20 +0200 |
| Subject | Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support |
| Message-ID | <trpex-1Qq-3@gated-at.bofh.it> |
| In reply to | #1614268 |
On Fri, Mar 31, 2017 at 02:10:12PM -0500, Alan Tull wrote:
> On Thu, Mar 30, 2017 at 7:08 AM, Wu Hao <hao.wu@intel.com> wrote:
> > From: Kang Luwei <luwei.kang@intel.com>
> >
> > Partial Reconfiguration (PR) is the most important function for FME. It
> > allows reconfiguration for given Port/Accelerated Function Unit (AFU).
> >
> > This patch adds support for PR sub feature. In this patch, it registers
> > a fpga_mgr and implements fpga_manager_ops, and invoke fpga_mgr_buf_load
> > for PR operation once PR request received via ioctl. Below user space
> > interfaces are exposed by this sub feature.
> >
> > Sysfs interface:
> > * /sys/class/fpga/<fpga.x>/<intel-fpga-fme.x>/interface_id
> > Read-only. Indicate the hardware interface information. Userspace
> > applications need to check this interface to select correct green
> > bitstream format before PR.
> >
> > Ioctl interface:
> > * FPGA_FME_PORT_PR
> > Do partial reconfiguration per information from userspace, including
> > target port(AFU), buffer size and address info. It returns the PR status
> > (PR error code if failed) to userspace.
> >
> > Signed-off-by: Tim Whisonant <tim.whisonant@intel.com>
> > Signed-off-by: Enno Luebbers <enno.luebbers@intel.com>
> > Signed-off-by: Shiva Rao <shiva.rao@intel.com>
> > Signed-off-by: Christopher Rauer <christopher.rauer@intel.com>
> > Signed-off-by: Alan Tull <alan.tull@intel.com>
> > Signed-off-by: Kang Luwei <luwei.kang@intel.com>
> > Signed-off-by: Xiao Guangrong <guangrong.xiao@linux.intel.com>
> > Signed-off-by: Wu Hao <hao.wu@intel.com>
> > ---
> > drivers/fpga/intel/Makefile | 2 +-
> > drivers/fpga/intel/feature-dev.h | 58 ++++++
> > drivers/fpga/intel/fme-main.c | 44 ++++-
> > drivers/fpga/intel/fme-pr.c | 400 +++++++++++++++++++++++++++++++++++++++
> > drivers/fpga/intel/fme.h | 32 ++++
> > include/uapi/linux/intel-fpga.h | 44 +++++
> > 6 files changed, 578 insertions(+), 2 deletions(-)
> > create mode 100644 drivers/fpga/intel/fme-pr.c
> > create mode 100644 drivers/fpga/intel/fme.h
> >
> > diff --git a/drivers/fpga/intel/Makefile b/drivers/fpga/intel/Makefile
> > index 546861d..0452cb6 100644
> > --- a/drivers/fpga/intel/Makefile
> > +++ b/drivers/fpga/intel/Makefile
> > @@ -2,4 +2,4 @@ obj-$(CONFIG_INTEL_FPGA_PCI) += intel-fpga-pci.o
> > obj-$(CONFIG_INTEL_FPGA_FME) += intel-fpga-fme.o
> >
> > intel-fpga-pci-objs := pcie.o feature-dev.o
> > -intel-fpga-fme-objs := fme-main.o
> > +intel-fpga-fme-objs := fme-main.o fme-pr.o
> > diff --git a/drivers/fpga/intel/feature-dev.h b/drivers/fpga/intel/feature-dev.h
> > index dccc283..5a25c915 100644
> > --- a/drivers/fpga/intel/feature-dev.h
> > +++ b/drivers/fpga/intel/feature-dev.h
> > @@ -150,8 +150,66 @@ struct feature_fme_err {
> > };
> >
> > /* FME Partial Reconfiguration Sub Feature Register Set */
> > +/* FME PR Control Register */
> > +struct feature_fme_pr_ctl {
> > + union {
> > + u64 csr;
> > + struct {
> > + u8 pr_reset:1; /* Reset PR Engine */
> > + u8 rsvdz1:3;
> > + u8 pr_reset_ack:1; /* Reset PR Engine Ack */
> > + u8 rsvdz2:3;
> > + u8 pr_regionid:2; /* PR Region ID */
> > + u8 rsvdz3:2;
> > + u8 pr_start_req:1; /* PR Start Request */
> > + u8 pr_push_complete:1; /* PR Data push complete */
> > + u8 pr_kind:1; /* Load Customer or Intel GBS */
> > + u32 rsvdz4:17;
> > + u32 config_data;
> > + };
> > + };
> > +};
> > +
> > +/* FME PR Status Register */
> > +struct feature_fme_pr_status {
> > + union {
> > + u64 csr;
> > + struct {
> > + u16 pr_credit:9; /* Number of PR Credits */
> > + u8 rsvdz1:7;
> > + u8 pr_status:1; /* PR Operation status */
> > + u8 rsvdz2:3;
> > + u8 pr_ctrlr_status:3; /* Controller status */
> > + u8 rsvdz3:1;
> > + u8 pr_host_status:4; /* PR Host status */
> > + u64 rsvdz4:36;
> > + };
> > + };
> > +};
> > +
> > +/* FME PR Data Register */
> > +struct feature_fme_pr_data {
> > + union {
> > + u64 csr;
> > + struct {
> > + /* PR data from the raw-binary file */
> > + u32 pr_data_raw;
> > + u32 rsvd;
> > + };
> > + };
> > +};
> > +
> > struct feature_fme_pr {
> > struct feature_header header;
> > + struct feature_fme_pr_ctl control;
> > + struct feature_fme_pr_status status;
> > + struct feature_fme_pr_data data;
> > + u64 error;
> > +
> > + u64 rsvd[16];
> > +
> > + u64 intfc_id_l; /* PR interface Id Low */
> > + u64 intfc_id_h; /* PR interface Id High */
> > };
> >
> > /* PORT Header Register Set */
> > diff --git a/drivers/fpga/intel/fme-main.c b/drivers/fpga/intel/fme-main.c
> > index 36d0c4c..0d9a7a6 100644
> > --- a/drivers/fpga/intel/fme-main.c
> > +++ b/drivers/fpga/intel/fme-main.c
> > @@ -23,6 +23,7 @@
> > #include <linux/intel-fpga.h>
> >
> > #include "feature-dev.h"
> > +#include "fme.h"
> >
> > static ssize_t ports_num_show(struct device *dev,
> > struct device_attribute *attr, char *buf)
> > @@ -101,6 +102,10 @@ static struct feature_driver fme_feature_drvs[] = {
> > .ops = &fme_hdr_ops,
> > },
> > {
> > + .name = FME_FEATURE_PR_MGMT,
> > + .ops = &pr_mgmt_ops,
> > + },
> > + {
> > .ops = NULL,
> > },
> > };
> > @@ -182,14 +187,48 @@ static const struct file_operations fme_fops = {
> > .unlocked_ioctl = fme_ioctl,
> > };
> >
> > +static int fme_dev_init(struct platform_device *pdev)
> > +{
> > + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> > + struct fpga_fme *fme;
> > +
> > + fme = devm_kzalloc(&pdev->dev, sizeof(*fme), GFP_KERNEL);
> > + if (!fme)
> > + return -ENOMEM;
> > +
> > + fme->pdata = pdata;
> > +
> > + mutex_lock(&pdata->lock);
> > + fpga_pdata_set_private(pdata, fme);
> > + mutex_unlock(&pdata->lock);
> > + return 0;
> > +}
> > +
> > +static void fme_dev_destroy(struct platform_device *pdev)
> > +{
> > + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> > + struct fpga_fme *fme;
> > +
> > + mutex_lock(&pdata->lock);
> > + fme = fpga_pdata_get_private(pdata);
> > + fpga_pdata_set_private(pdata, NULL);
> > + mutex_unlock(&pdata->lock);
> > +
> > + devm_kfree(&pdev->dev, fme);
> > +}
> > +
> > static int fme_probe(struct platform_device *pdev)
> > {
> > int ret;
> >
> > - ret = fpga_dev_feature_init(pdev, fme_feature_drvs);
> > + ret = fme_dev_init(pdev);
> > if (ret)
> > goto exit;
> >
> > + ret = fpga_dev_feature_init(pdev, fme_feature_drvs);
> > + if (ret)
> > + goto dev_destroy;
> > +
> > ret = fpga_register_dev_ops(pdev, &fme_fops, THIS_MODULE);
> > if (ret)
> > goto feature_uinit;
> > @@ -198,6 +237,8 @@ static int fme_probe(struct platform_device *pdev)
> >
> > feature_uinit:
> > fpga_dev_feature_uinit(pdev);
> > +dev_destroy:
> > + fme_dev_destroy(pdev);
> > exit:
> > return ret;
> > }
> > @@ -206,6 +247,7 @@ static int fme_remove(struct platform_device *pdev)
> > {
> > fpga_dev_feature_uinit(pdev);
> > fpga_unregister_dev_ops(pdev);
> > + fme_dev_destroy(pdev);
> > return 0;
> > }
> >
> > diff --git a/drivers/fpga/intel/fme-pr.c b/drivers/fpga/intel/fme-pr.c
> > new file mode 100644
> > index 0000000..3b44a3e
> > --- /dev/null
> > +++ b/drivers/fpga/intel/fme-pr.c
> > @@ -0,0 +1,400 @@
> > +/*
> > + * Driver for Intel FPGA Management Engine (FME) Partial Reconfiguration
> > + *
> > + * Copyright (C) 2017 Intel Corporation, Inc.
> > + *
> > + * Authors:
> > + * Kang Luwei <luwei.kang@intel.com>
> > + * Xiao Guangrong <guangrong.xiao@linux.intel.com>
> > + * Joseph Grecco <joe.grecco@intel.com>
> > + * Enno Luebbers <enno.luebbers@intel.com>
> > + * Tim Whisonant <tim.whisonant@intel.com>
> > + * Ananda Ravuri <ananda.ravuri@intel.com>
> > + * Christopher Rauer <christopher.rauer@intel.com>
> > + * Henry Mitchel <henry.mitchel@intel.com>
> > + *
> > + * This work is licensed under a dual BSD/GPLv2 license. When using or
> > + * redistributing this file, you may do so under either license. See the
> > + * LICENSE.BSD file under this directory for the BSD license and see
> > + * the COPYING file in the top-level directory for the GPLv2 license.
> > + */
> > +
> > +#include <linux/types.h>
> > +#include <linux/device.h>
> > +#include <linux/vmalloc.h>
> > +#include <linux/uaccess.h>
> > +#include <linux/fpga/fpga-mgr.h>
> > +#include <linux/intel-fpga.h>
> > +
> > +#include "feature-dev.h"
> > +#include "fme.h"
> > +
> > +#define PR_WAIT_TIMEOUT 8000000
> > +
> > +#define PR_HOST_STATUS_IDLE 0
> > +
> > +DEFINE_FPGA_PR_ERR_MSG(pr_err_msg);
> > +
> > +static ssize_t interface_id_show(struct device *dev,
> > + struct device_attribute *attr, char *buf)
> > +{
> > + u64 intfc_id_l, intfc_id_h;
> > + struct feature_fme_pr *fme_pr
> > + = get_feature_ioaddr_by_index(dev, FME_FEATURE_ID_PR_MGMT);
> > +
> > + intfc_id_l = readq(&fme_pr->intfc_id_l);
> > + intfc_id_h = readq(&fme_pr->intfc_id_h);
> > +
> > + return scnprintf(buf, PAGE_SIZE, "%016llx%016llx\n",
> > + (unsigned long long)intfc_id_h,
> > + (unsigned long long)intfc_id_l);
> > +}
> > +static DEVICE_ATTR_RO(interface_id);
> > +
> > +static struct attribute *pr_mgmt_attrs[] = {
> > + &dev_attr_interface_id.attr,
> > + NULL,
> > +};
> > +
> > +struct attribute_group pr_mgmt_attr_group = {
> > + .attrs = pr_mgmt_attrs,
> > + .name = "pr",
> > +};
> > +
> > +static u64
> > +pr_err_handle(struct platform_device *pdev, struct feature_fme_pr *fme_pr)
> > +{
> > + struct feature_fme_pr_status fme_pr_status;
> > + unsigned long err_code;
> > + u64 fme_pr_error;
> > + int i = 0;
> > +
> > + fme_pr_status.csr = readq(&fme_pr->status);
> > + if (!fme_pr_status.pr_status)
> > + return 0;
> > +
> > + err_code = fme_pr_error = readq(&fme_pr->error);
> > + for_each_set_bit(i, &err_code, PR_MAX_ERR_NUM)
> > + dev_dbg(&pdev->dev, "%s\n", pr_err_msg[i]);
> > + writeq(fme_pr_error, &fme_pr->error);
> > + return fme_pr_error;
> > +}
> > +
> > +static int fme_pr_write_init(struct fpga_manager *mgr,
> > + struct fpga_image_info *info, const char *buf, size_t count)
> > +{
> > + struct fpga_fme *fme = mgr->priv;
> > + struct platform_device *pdev;
> > + struct feature_fme_pr *fme_pr;
> > + struct feature_fme_pr_ctl fme_pr_ctl;
> > + struct feature_fme_pr_status fme_pr_status;
> > +
> > + pdev = fme->pdata->dev;
> > + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
> > + FME_FEATURE_ID_PR_MGMT);
> > + if (!fme_pr)
> > + return -EINVAL;
> > +
> > + if (WARN_ON(info->flags != FPGA_MGR_PARTIAL_RECONFIG))
> > + return -EINVAL;
> > +
> > + dev_dbg(&pdev->dev, "resetting PR before initiated PR\n");
> > +
> > + fme_pr_ctl.csr = readq(&fme_pr->control);
> > + fme_pr_ctl.pr_reset = 1;
> > + writeq(fme_pr_ctl.csr, &fme_pr->control);
> > +
> > + fme_pr_ctl.pr_reset_ack = 1;
> > +
> > + if (fpga_wait_register_field(pr_reset_ack, fme_pr_ctl,
> > + &fme_pr->control, PR_WAIT_TIMEOUT, 1)) {
> > + dev_err(&pdev->dev, "maximum PR timeout\n");
> > + return -ETIMEDOUT;
> > + }
> > +
> > + fme_pr_ctl.csr = readq(&fme_pr->control);
> > + fme_pr_ctl.pr_reset = 0;
> > + writeq(fme_pr_ctl.csr, &fme_pr->control);
> > +
> > + dev_dbg(&pdev->dev,
> > + "waiting for PR resource in HW to be initialized and ready\n");
> > +
> > + fme_pr_status.pr_host_status = PR_HOST_STATUS_IDLE;
> > +
> > + if (fpga_wait_register_field(pr_host_status, fme_pr_status,
> > + &fme_pr->status, PR_WAIT_TIMEOUT, 1)) {
> > + dev_err(&pdev->dev, "maximum PR timeout\n");
> > + return -ETIMEDOUT;
> > + }
> > +
> > + dev_dbg(&pdev->dev, "check if have any previous PR error\n");
> > + pr_err_handle(pdev, fme_pr);
> > + return 0;
> > +}
> > +
> > +static int fme_pr_write(struct fpga_manager *mgr,
> > + const char *buf, size_t count)
> > +{
> > + struct fpga_fme *fme = mgr->priv;
> > + struct platform_device *pdev;
> > + struct feature_fme_pr *fme_pr;
> > + struct feature_fme_pr_ctl fme_pr_ctl;
> > + struct feature_fme_pr_status fme_pr_status;
> > + struct feature_fme_pr_data fme_pr_data;
> > + int delay, pr_credit, i = 0;
> > +
> > + pdev = fme->pdata->dev;
> > + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
> > + FME_FEATURE_ID_PR_MGMT);
> > +
> > + dev_dbg(&pdev->dev, "set PR port ID and start request\n");
> > +
> > + fme_pr_ctl.csr = readq(&fme_pr->control);
> > + fme_pr_ctl.pr_regionid = fme->port_id;
> > + fme_pr_ctl.pr_start_req = 1;
> > + writeq(fme_pr_ctl.csr, &fme_pr->control);
> > +
> > + dev_dbg(&pdev->dev, "pushing data from bitstream to HW\n");
> > +
> > + fme_pr_status.csr = readq(&fme_pr->status);
> > + pr_credit = fme_pr_status.pr_credit;
> > +
> > + while (count > 0) {
> > + delay = 0;
> > + while (pr_credit <= 1) {
> > + if (delay++ > PR_WAIT_TIMEOUT) {
> > + dev_err(&pdev->dev, "maximum try\n");
> > + return -ETIMEDOUT;
> > + }
> > + udelay(1);
> > +
> > + fme_pr_status.csr = readq(&fme_pr->status);
> > + pr_credit = fme_pr_status.pr_credit;
> > + };
> > +
> > + if (count >= 4) {
> > + fme_pr_data.rsvd = 0;
> > + fme_pr_data.pr_data_raw = *(((u32 *)buf) + i);
> > + writeq(fme_pr_data.csr, &fme_pr->data);
> > + count -= 4;
> > + pr_credit--;
> > + i++;
> > + } else {
> > + WARN_ON(1);
> > + return -EINVAL;
> > + }
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static int fme_pr_write_complete(struct fpga_manager *mgr,
> > + struct fpga_image_info *info)
> > +{
> > + struct fpga_fme *fme = mgr->priv;
> > + struct platform_device *pdev;
> > + struct feature_fme_pr *fme_pr;
> > + struct feature_fme_pr_ctl fme_pr_ctl;
> > +
> > + pdev = fme->pdata->dev;
> > + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
> > + FME_FEATURE_ID_PR_MGMT);
> > +
> > + fme_pr_ctl.csr = readq(&fme_pr->control);
> > + fme_pr_ctl.pr_push_complete = 1;
> > + writeq(fme_pr_ctl.csr, &fme_pr->control);
> > +
> > + dev_dbg(&pdev->dev, "green bitstream push complete\n");
> > + dev_dbg(&pdev->dev, "waiting for HW to release PR resource\n");
> > +
> > + fme_pr_ctl.pr_start_req = 0;
> > +
> > + if (fpga_wait_register_field(pr_start_req, fme_pr_ctl,
> > + &fme_pr->control, PR_WAIT_TIMEOUT, 1)) {
> > + dev_err(&pdev->dev, "maximum try.\n");
> > + return -ETIMEDOUT;
> > + }
> > +
> > + dev_dbg(&pdev->dev, "PR operation complete, checking status\n");
> > + fme->pr_err = pr_err_handle(pdev, fme_pr);
> > + if (fme->pr_err)
> > + return -EIO;
> > +
> > + dev_dbg(&pdev->dev, "PR done successfully\n");
> > + return 0;
> > +}
> > +
> > +static enum fpga_mgr_states fme_pr_state(struct fpga_manager *mgr)
> > +{
> > + return FPGA_MGR_STATE_UNKNOWN;
> > +}
> > +
> > +static const struct fpga_manager_ops fme_pr_ops = {
> > + .write_init = fme_pr_write_init,
> > + .write = fme_pr_write,
> > + .write_complete = fme_pr_write_complete,
> > + .state = fme_pr_state,
> > +};
> > +
> > +static int fme_pr(struct platform_device *pdev, unsigned long arg)
> > +{
> > + void __user *argp = (void __user *)arg;
> > + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> > + struct fpga_fme *fme;
> > + struct fpga_manager *mgr;
> > + struct feature_fme_header *fme_hdr;
> > + struct feature_fme_capability fme_capability;
> > + struct fpga_image_info info;
> > + struct fpga_fme_port_pr port_pr;
> > + struct platform_device *port;
> > + unsigned long minsz;
> > + void *buf = NULL;
> > + int ret = 0;
> > +
> > + minsz = offsetofend(struct fpga_fme_port_pr, status);
> > +
> > + if (copy_from_user(&port_pr, argp, minsz))
> > + return -EFAULT;
> > +
> > + if (port_pr.argsz < minsz || port_pr.flags)
> > + return -EINVAL;
> > +
> > + if (!IS_ALIGNED(port_pr.buffer_size, 4))
> > + return -EINVAL;
> > +
> > + /* get fme header region */
> > + fme_hdr = get_feature_ioaddr_by_index(&pdev->dev,
> > + FME_FEATURE_ID_HEADER);
> > + if (WARN_ON(!fme_hdr))
> > + return -EINVAL;
> > +
> > + /* check port id */
> > + fme_capability.csr = readq(&fme_hdr->capability);
> > + if (port_pr.port_id >= fme_capability.num_ports) {
> > + dev_dbg(&pdev->dev, "port number more than maximum\n");
> > + return -EINVAL;
> > + }
> > +
> > + if (!access_ok(VERIFY_READ, port_pr.buffer_address,
> > + port_pr.buffer_size))
> > + return -EFAULT;
> > +
> > + buf = vmalloc(port_pr.buffer_size);
> > + if (!buf)
> > + return -ENOMEM;
> > +
> > + if (copy_from_user(buf, (void __user *)port_pr.buffer_address,
> > + port_pr.buffer_size)) {
> > + ret = -EFAULT;
> > + goto free_exit;
> > + }
> > +
> > + memset(&info, 0, sizeof(struct fpga_image_info));
> > + info.flags = FPGA_MGR_PARTIAL_RECONFIG;
> > +
> > + mgr = fpga_mgr_get(&pdev->dev);
> > + if (IS_ERR(mgr)) {
> > + ret = PTR_ERR(mgr);
> > + goto free_exit;
> > + }
> > +
> > + mutex_lock(&pdata->lock);
> > + fme = fpga_pdata_get_private(pdata);
> > + /* fme device has been unregistered. */
> > + if (!fme) {
> > + ret = -EINVAL;
> > + goto unlock_exit;
> > + }
> > +
> > + fme->pr_err = 0;
> > + fme->port_id = port_pr.port_id;
>
> It looks like you're using private data to communicate with the
> driver, i.e. there is something you want to do with the fpga manager
> framework and it doesn't have that feature. The better way would be
> for us to expand the framework so you don't need to do that.
>
> port_id is the kind of thing that should be communicated to the driver
> through fpga_image_info, so we could add that to the struct. Should
> we call it port_id? Or is there something more generic that may be
> useful in the future for other architectures?.
Hi Alan
Thanks for your feedback. :)
As you know, each Intel FPGA device may have more than one accelerator,
and all accelerators share the same fpga_manager (only one FME module).
port_id = 0 means the first accelerator on this fpga devices. So it's
needed by the fpga_manager to distinguish one accelerator from others
for partial reconfiguration.
Adding a member like a 'id' to fpga_image_info definitely works for us
in this case. We can add it this way, but I feel 'id' here seems not
related to fpga image, but characteristic of fpga region. It may be a
little confusing. One rough idea is that keep this info under fpga region
(maybe its private data), and pass the fpga-region to fpga_mgr_buf_load,
then fpga_manager knows the target region for partial reconfiguration.
If consider pr sysfs interface support under fpga-region in the future,
then we don't need to create a new 'id' sysfs file, as fpga-region itself
could provide this kind of info. But I'm not sure if this is the right
direction.
>
> pr_err appears to be machine specific error codes that are
> communicated outside your low level driver. (Calling it pr_err is
> extra confusing since Linux already has a commonly name function by
> the same name). The framework has state, but that's not doing what
> you want here. Maybe we could add a framework ops called status so
> that status could be communicated from the low level driver. It would
> be useful to abstract the machine specific state to a defined enum
> that would be part of the fpga mgr framework. I remember we had
> something like that in the earliest version of fpga manager but it got
> changed to state rather than status for some reason.
>
Adding a status function and a 'status' member to fpga manager sounds good.
But I'm not sure how to abstract these error codes, currently what we have
are all PR error codes for Intel FPGA device, e.g "PR FIFO overflow error",
"PR secure load error" and etc (see include/uapi/linux/intel-fpga.h). Is it
fine to let each driver to define how to use that 'status' for machine
specific status?
> mgr->dev won't work for you for some of the things you are using fme->pdev for?
>
Does 'fme->pdev' mean the platform device for fme? I think we use platform
device to represent the FME module, and FME module may have multiple sub
features, driver discovers these sub features via the 'Device Feature List'
in the PCIe Device Bar. PR is only one of the sub features under FME module,
if any FME module doesn't have PR sub feature, fpga-manager will not be
created at all. But this will not impact other sub features, e.g thermal
management, error reporting and etc, to create their own interfaces under
the platform device.
Thanks
Hao
> Alan
>
> > +
> > + /* Find and get port device by index */
> > + port = pdata->fpga_for_each_port(pdev, &fme->port_id,
> > + fpga_port_check_id);
> > + WARN_ON(!port);
> > +
> > + /* Disable Port before PR */
> > + fpga_port_disable(port);
> > +
> > + ret = fpga_mgr_buf_load(mgr, &info, buf, port_pr.buffer_size);
> > + port_pr.status = fme->pr_err;
> > +
> > + /* Re-enable Port after PR finished */
> > + fpga_port_enable(port);
> > +
> > + put_device(&port->dev);
> > +
> > +unlock_exit:
> > + mutex_unlock(&pdata->lock);
> > + fpga_mgr_put(mgr);
> > +free_exit:
> > + vfree(buf);
> > + if (copy_to_user((void __user *)arg, &port_pr, minsz))
> > + return -EFAULT;
> > + return ret;
> > +}
> > +
> > +static int fpga_fme_pr_probe(struct platform_device *pdev)
> > +{
> > + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> > + struct fpga_fme *priv;
> > + int ret;
> > +
> > + mutex_lock(&pdata->lock);
> > + priv = fpga_pdata_get_private(pdata);
> > + ret = fpga_mgr_register(&pdata->dev->dev,
> > + "Intel FPGA Manager", &fme_pr_ops, priv);
> > + mutex_unlock(&pdata->lock);
> > +
> > + return ret;
> > +}
> > +
> > +static int fpga_fme_pr_remove(struct platform_device *pdev)
> > +{
> > + fpga_mgr_unregister(&pdev->dev);
> > + return 0;
> > +}
> > +
> > +static int pr_mgmt_init(struct platform_device *pdev, struct feature *feature)
> > +{
> > + int ret;
> > +
> > + ret = fpga_fme_pr_probe(pdev);
> > + if (ret)
> > + return ret;
> > +
> > + ret = sysfs_create_group(&pdev->dev.kobj, &pr_mgmt_attr_group);
> > + if (ret)
> > + fpga_fme_pr_remove(pdev);
> > +
> > + return ret;
> > +}
> > +
> > +static void pr_mgmt_uinit(struct platform_device *pdev, struct feature *feature)
> > +{
> > + sysfs_remove_group(&pdev->dev.kobj, &pr_mgmt_attr_group);
> > + fpga_fme_pr_remove(pdev);
> > +}
> > +
> > +static long fme_pr_ioctl(struct platform_device *pdev, struct feature *feature,
> > + unsigned int cmd, unsigned long arg)
> > +{
> > + long ret;
> > +
> > + switch (cmd) {
> > + case FPGA_FME_PORT_PR:
> > + ret = fme_pr(pdev, arg);
> > + break;
> > + default:
> > + ret = -ENODEV;
> > + }
> > +
> > + return ret;
> > +}
> > +
> > +struct feature_ops pr_mgmt_ops = {
> > + .init = pr_mgmt_init,
> > + .uinit = pr_mgmt_uinit,
> > + .ioctl = fme_pr_ioctl,
> > +};
> > diff --git a/drivers/fpga/intel/fme.h b/drivers/fpga/intel/fme.h
> > new file mode 100644
> > index 0000000..d6cb7ce
> > --- /dev/null
> > +++ b/drivers/fpga/intel/fme.h
> > @@ -0,0 +1,32 @@
> > +/*
> > + * Header file for Intel FPGA Management Engine (FME) Driver
> > + *
> > + * Copyright (C) 2017 Intel Corporation, Inc.
> > + *
> > + * Authors:
> > + * Kang Luwei <luwei.kang@intel.com>
> > + * Xiao Guangrong <guangrong.xiao@linux.intel.com>
> > + * Joseph Grecco <joe.grecco@intel.com>
> > + * Enno Luebbers <enno.luebbers@intel.com>
> > + * Tim Whisonant <tim.whisonant@intel.com>
> > + * Ananda Ravuri <ananda.ravuri@intel.com>
> > + * Henry Mitchel <henry.mitchel@intel.com>
> > + *
> > + * This work is licensed under a dual BSD/GPLv2 license. When using or
> > + * redistributing this file, you may do so under either license. See the
> > + * LICENSE.BSD file under this directory for the BSD license and see
> > + * the COPYING file in the top-level directory for the GPLv2 license.
> > + */
> > +
> > +#ifndef __INTEL_FME_H
> > +#define __INTEL_FME_H
> > +
> > +struct fpga_fme {
> > + u8 port_id;
> > + u64 pr_err;
> > + struct feature_platform_data *pdata;
> > +};
> > +
> > +extern struct feature_ops pr_mgmt_ops;
> > +
> > +#endif
> > diff --git a/include/uapi/linux/intel-fpga.h b/include/uapi/linux/intel-fpga.h
> > index 992e556..77658316 100644
> > --- a/include/uapi/linux/intel-fpga.h
> > +++ b/include/uapi/linux/intel-fpga.h
> > @@ -18,6 +18,8 @@
> > #ifndef _UAPI_LINUX_INTEL_FPGA_H
> > #define _UAPI_LINUX_INTEL_FPGA_H
> >
> > +#include <linux/types.h>
> > +
> > #define FPGA_API_VERSION 0
> >
> > /*
> > @@ -30,6 +32,7 @@
> > #define FPGA_MAGIC 0xB6
> >
> > #define FPGA_BASE 0
> > +#define FME_BASE 0x80
> >
> > /**
> > * FPGA_GET_API_VERSION - _IO(FPGA_MAGIC, FPGA_BASE + 0)
> > @@ -49,4 +52,45 @@
> >
> > #define FPGA_CHECK_EXTENSION _IO(FPGA_MAGIC, FPGA_BASE + 1)
> >
> > +/* IOCTLs for FME file descriptor */
> > +
> > +/**
> > + * FPGA_FME_PORT_PR - _IOWR(FPGA_MAGIC, FME_BASE + 0, struct fpga_fme_port_pr)
> > + *
> > + * Driver does Partial Reconfiguration based on Port ID and Buffer (Image)
> > + * provided by caller.
> > + * Return: 0 on success, -errno on failure.
> > + * If FPGA_FME_PORT_PR returns -EIO, that indicates the HW has detected
> > + * some errors during PR, under this case, the user can fetch HW error code
> > + * from fpga_fme_port_pr.status. Each bit on the error code is used as the
> > + * index for the array created by DEFINE_FPGA_PR_ERR_MSG().
> > + * Otherwise, it is always zero.
> > + */
> > +
> > +#define DEFINE_FPGA_PR_ERR_MSG(_name_) \
> > +static const char * const _name_[] = { \
> > + "PR operation error detected", \
> > + "PR CRC error detected", \
> > + "PR incompatiable bitstream error detected", \
> > + "PR IP protocol error detected", \
> > + "PR FIFO overflow error detected", \
> > + "Reserved", \
> > + "PR secure load error detected", \
> > +}
> > +
> > +#define PR_MAX_ERR_NUM 7
> > +
> > +struct fpga_fme_port_pr {
> > + /* Input */
> > + __u32 argsz; /* Structure length */
> > + __u32 flags; /* Zero for now */
> > + __u32 port_id;
> > + __u32 buffer_size;
> > + __u64 buffer_address; /* Userspace address to the buffer for PR */
> > + /* Output */
> > + __u64 status; /* HW error code if ioctl returns -EIO */
> > +};
> > +
> > +#define FPGA_FME_PORT_PR _IO(FPGA_MAGIC, FME_BASE + 0)
> > +
> > #endif /* _UAPI_INTEL_FPGA_H */
> > --
> > 2.7.4
> >
[toc] | [prev] | [next] | [standalone]
| From | Alan Tull <atull@kernel.org> |
|---|---|
| Date | 2017-04-03 18:40 +0200 |
| Subject | Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support |
| Message-ID | <tsdbj-RK-15@gated-at.bofh.it> |
| In reply to | #1614479 |
On Sat, Apr 1, 2017 at 6:08 AM, Wu Hao <hao.wu@intel.com> wrote:
> On Fri, Mar 31, 2017 at 02:10:12PM -0500, Alan Tull wrote:
>> On Thu, Mar 30, 2017 at 7:08 AM, Wu Hao <hao.wu@intel.com> wrote:
>> > From: Kang Luwei <luwei.kang@intel.com>
>> >
>> > Partial Reconfiguration (PR) is the most important function for FME. It
>> > allows reconfiguration for given Port/Accelerated Function Unit (AFU).
>> >
>> > This patch adds support for PR sub feature. In this patch, it registers
>> > a fpga_mgr and implements fpga_manager_ops, and invoke fpga_mgr_buf_load
>> > for PR operation once PR request received via ioctl. Below user space
>> > interfaces are exposed by this sub feature.
>> >
>> > Sysfs interface:
>> > * /sys/class/fpga/<fpga.x>/<intel-fpga-fme.x>/interface_id
>> > Read-only. Indicate the hardware interface information. Userspace
>> > applications need to check this interface to select correct green
>> > bitstream format before PR.
>> >
>> > Ioctl interface:
>> > * FPGA_FME_PORT_PR
>> > Do partial reconfiguration per information from userspace, including
>> > target port(AFU), buffer size and address info. It returns the PR status
>> > (PR error code if failed) to userspace.
>> >
>> > Signed-off-by: Tim Whisonant <tim.whisonant@intel.com>
>> > Signed-off-by: Enno Luebbers <enno.luebbers@intel.com>
>> > Signed-off-by: Shiva Rao <shiva.rao@intel.com>
>> > Signed-off-by: Christopher Rauer <christopher.rauer@intel.com>
>> > Signed-off-by: Alan Tull <alan.tull@intel.com>
>> > Signed-off-by: Kang Luwei <luwei.kang@intel.com>
>> > Signed-off-by: Xiao Guangrong <guangrong.xiao@linux.intel.com>
>> > Signed-off-by: Wu Hao <hao.wu@intel.com>
>> > ---
>> > drivers/fpga/intel/Makefile | 2 +-
>> > drivers/fpga/intel/feature-dev.h | 58 ++++++
>> > drivers/fpga/intel/fme-main.c | 44 ++++-
>> > drivers/fpga/intel/fme-pr.c | 400 +++++++++++++++++++++++++++++++++++++++
>> > drivers/fpga/intel/fme.h | 32 ++++
>> > include/uapi/linux/intel-fpga.h | 44 +++++
>> > 6 files changed, 578 insertions(+), 2 deletions(-)
>> > create mode 100644 drivers/fpga/intel/fme-pr.c
>> > create mode 100644 drivers/fpga/intel/fme.h
>> >
>> > diff --git a/drivers/fpga/intel/Makefile b/drivers/fpga/intel/Makefile
>> > index 546861d..0452cb6 100644
>> > --- a/drivers/fpga/intel/Makefile
>> > +++ b/drivers/fpga/intel/Makefile
>> > @@ -2,4 +2,4 @@ obj-$(CONFIG_INTEL_FPGA_PCI) += intel-fpga-pci.o
>> > obj-$(CONFIG_INTEL_FPGA_FME) += intel-fpga-fme.o
>> >
>> > intel-fpga-pci-objs := pcie.o feature-dev.o
>> > -intel-fpga-fme-objs := fme-main.o
>> > +intel-fpga-fme-objs := fme-main.o fme-pr.o
>> > diff --git a/drivers/fpga/intel/feature-dev.h b/drivers/fpga/intel/feature-dev.h
>> > index dccc283..5a25c915 100644
>> > --- a/drivers/fpga/intel/feature-dev.h
>> > +++ b/drivers/fpga/intel/feature-dev.h
>> > @@ -150,8 +150,66 @@ struct feature_fme_err {
>> > };
>> >
>> > /* FME Partial Reconfiguration Sub Feature Register Set */
>> > +/* FME PR Control Register */
>> > +struct feature_fme_pr_ctl {
>> > + union {
>> > + u64 csr;
>> > + struct {
>> > + u8 pr_reset:1; /* Reset PR Engine */
>> > + u8 rsvdz1:3;
>> > + u8 pr_reset_ack:1; /* Reset PR Engine Ack */
>> > + u8 rsvdz2:3;
>> > + u8 pr_regionid:2; /* PR Region ID */
>> > + u8 rsvdz3:2;
>> > + u8 pr_start_req:1; /* PR Start Request */
>> > + u8 pr_push_complete:1; /* PR Data push complete */
>> > + u8 pr_kind:1; /* Load Customer or Intel GBS */
>> > + u32 rsvdz4:17;
>> > + u32 config_data;
>> > + };
>> > + };
>> > +};
>> > +
>> > +/* FME PR Status Register */
>> > +struct feature_fme_pr_status {
>> > + union {
>> > + u64 csr;
>> > + struct {
>> > + u16 pr_credit:9; /* Number of PR Credits */
>> > + u8 rsvdz1:7;
>> > + u8 pr_status:1; /* PR Operation status */
>> > + u8 rsvdz2:3;
>> > + u8 pr_ctrlr_status:3; /* Controller status */
>> > + u8 rsvdz3:1;
>> > + u8 pr_host_status:4; /* PR Host status */
>> > + u64 rsvdz4:36;
>> > + };
>> > + };
>> > +};
>> > +
>> > +/* FME PR Data Register */
>> > +struct feature_fme_pr_data {
>> > + union {
>> > + u64 csr;
>> > + struct {
>> > + /* PR data from the raw-binary file */
>> > + u32 pr_data_raw;
>> > + u32 rsvd;
>> > + };
>> > + };
>> > +};
>> > +
>> > struct feature_fme_pr {
>> > struct feature_header header;
>> > + struct feature_fme_pr_ctl control;
>> > + struct feature_fme_pr_status status;
>> > + struct feature_fme_pr_data data;
>> > + u64 error;
>> > +
>> > + u64 rsvd[16];
>> > +
>> > + u64 intfc_id_l; /* PR interface Id Low */
>> > + u64 intfc_id_h; /* PR interface Id High */
>> > };
>> >
>> > /* PORT Header Register Set */
>> > diff --git a/drivers/fpga/intel/fme-main.c b/drivers/fpga/intel/fme-main.c
>> > index 36d0c4c..0d9a7a6 100644
>> > --- a/drivers/fpga/intel/fme-main.c
>> > +++ b/drivers/fpga/intel/fme-main.c
>> > @@ -23,6 +23,7 @@
>> > #include <linux/intel-fpga.h>
>> >
>> > #include "feature-dev.h"
>> > +#include "fme.h"
>> >
>> > static ssize_t ports_num_show(struct device *dev,
>> > struct device_attribute *attr, char *buf)
>> > @@ -101,6 +102,10 @@ static struct feature_driver fme_feature_drvs[] = {
>> > .ops = &fme_hdr_ops,
>> > },
>> > {
>> > + .name = FME_FEATURE_PR_MGMT,
>> > + .ops = &pr_mgmt_ops,
>> > + },
>> > + {
>> > .ops = NULL,
>> > },
>> > };
>> > @@ -182,14 +187,48 @@ static const struct file_operations fme_fops = {
>> > .unlocked_ioctl = fme_ioctl,
>> > };
>> >
>> > +static int fme_dev_init(struct platform_device *pdev)
>> > +{
>> > + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
>> > + struct fpga_fme *fme;
>> > +
>> > + fme = devm_kzalloc(&pdev->dev, sizeof(*fme), GFP_KERNEL);
>> > + if (!fme)
>> > + return -ENOMEM;
>> > +
>> > + fme->pdata = pdata;
>> > +
>> > + mutex_lock(&pdata->lock);
>> > + fpga_pdata_set_private(pdata, fme);
>> > + mutex_unlock(&pdata->lock);
>> > + return 0;
>> > +}
>> > +
>> > +static void fme_dev_destroy(struct platform_device *pdev)
>> > +{
>> > + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
>> > + struct fpga_fme *fme;
>> > +
>> > + mutex_lock(&pdata->lock);
>> > + fme = fpga_pdata_get_private(pdata);
>> > + fpga_pdata_set_private(pdata, NULL);
>> > + mutex_unlock(&pdata->lock);
>> > +
>> > + devm_kfree(&pdev->dev, fme);
>> > +}
>> > +
>> > static int fme_probe(struct platform_device *pdev)
>> > {
>> > int ret;
>> >
>> > - ret = fpga_dev_feature_init(pdev, fme_feature_drvs);
>> > + ret = fme_dev_init(pdev);
>> > if (ret)
>> > goto exit;
>> >
>> > + ret = fpga_dev_feature_init(pdev, fme_feature_drvs);
>> > + if (ret)
>> > + goto dev_destroy;
>> > +
>> > ret = fpga_register_dev_ops(pdev, &fme_fops, THIS_MODULE);
>> > if (ret)
>> > goto feature_uinit;
>> > @@ -198,6 +237,8 @@ static int fme_probe(struct platform_device *pdev)
>> >
>> > feature_uinit:
>> > fpga_dev_feature_uinit(pdev);
>> > +dev_destroy:
>> > + fme_dev_destroy(pdev);
>> > exit:
>> > return ret;
>> > }
>> > @@ -206,6 +247,7 @@ static int fme_remove(struct platform_device *pdev)
>> > {
>> > fpga_dev_feature_uinit(pdev);
>> > fpga_unregister_dev_ops(pdev);
>> > + fme_dev_destroy(pdev);
>> > return 0;
>> > }
>> >
>> > diff --git a/drivers/fpga/intel/fme-pr.c b/drivers/fpga/intel/fme-pr.c
>> > new file mode 100644
>> > index 0000000..3b44a3e
>> > --- /dev/null
>> > +++ b/drivers/fpga/intel/fme-pr.c
>> > @@ -0,0 +1,400 @@
>> > +/*
>> > + * Driver for Intel FPGA Management Engine (FME) Partial Reconfiguration
>> > + *
>> > + * Copyright (C) 2017 Intel Corporation, Inc.
>> > + *
>> > + * Authors:
>> > + * Kang Luwei <luwei.kang@intel.com>
>> > + * Xiao Guangrong <guangrong.xiao@linux.intel.com>
>> > + * Joseph Grecco <joe.grecco@intel.com>
>> > + * Enno Luebbers <enno.luebbers@intel.com>
>> > + * Tim Whisonant <tim.whisonant@intel.com>
>> > + * Ananda Ravuri <ananda.ravuri@intel.com>
>> > + * Christopher Rauer <christopher.rauer@intel.com>
>> > + * Henry Mitchel <henry.mitchel@intel.com>
>> > + *
>> > + * This work is licensed under a dual BSD/GPLv2 license. When using or
>> > + * redistributing this file, you may do so under either license. See the
>> > + * LICENSE.BSD file under this directory for the BSD license and see
>> > + * the COPYING file in the top-level directory for the GPLv2 license.
>> > + */
>> > +
>> > +#include <linux/types.h>
>> > +#include <linux/device.h>
>> > +#include <linux/vmalloc.h>
>> > +#include <linux/uaccess.h>
>> > +#include <linux/fpga/fpga-mgr.h>
>> > +#include <linux/intel-fpga.h>
>> > +
>> > +#include "feature-dev.h"
>> > +#include "fme.h"
>> > +
>> > +#define PR_WAIT_TIMEOUT 8000000
>> > +
>> > +#define PR_HOST_STATUS_IDLE 0
>> > +
>> > +DEFINE_FPGA_PR_ERR_MSG(pr_err_msg);
>> > +
>> > +static ssize_t interface_id_show(struct device *dev,
>> > + struct device_attribute *attr, char *buf)
>> > +{
>> > + u64 intfc_id_l, intfc_id_h;
>> > + struct feature_fme_pr *fme_pr
>> > + = get_feature_ioaddr_by_index(dev, FME_FEATURE_ID_PR_MGMT);
>> > +
>> > + intfc_id_l = readq(&fme_pr->intfc_id_l);
>> > + intfc_id_h = readq(&fme_pr->intfc_id_h);
>> > +
>> > + return scnprintf(buf, PAGE_SIZE, "%016llx%016llx\n",
>> > + (unsigned long long)intfc_id_h,
>> > + (unsigned long long)intfc_id_l);
>> > +}
>> > +static DEVICE_ATTR_RO(interface_id);
>> > +
>> > +static struct attribute *pr_mgmt_attrs[] = {
>> > + &dev_attr_interface_id.attr,
>> > + NULL,
>> > +};
>> > +
>> > +struct attribute_group pr_mgmt_attr_group = {
>> > + .attrs = pr_mgmt_attrs,
>> > + .name = "pr",
>> > +};
>> > +
>> > +static u64
>> > +pr_err_handle(struct platform_device *pdev, struct feature_fme_pr *fme_pr)
>> > +{
>> > + struct feature_fme_pr_status fme_pr_status;
>> > + unsigned long err_code;
>> > + u64 fme_pr_error;
>> > + int i = 0;
>> > +
>> > + fme_pr_status.csr = readq(&fme_pr->status);
>> > + if (!fme_pr_status.pr_status)
>> > + return 0;
>> > +
>> > + err_code = fme_pr_error = readq(&fme_pr->error);
>> > + for_each_set_bit(i, &err_code, PR_MAX_ERR_NUM)
>> > + dev_dbg(&pdev->dev, "%s\n", pr_err_msg[i]);
>> > + writeq(fme_pr_error, &fme_pr->error);
>> > + return fme_pr_error;
>> > +}
>> > +
>> > +static int fme_pr_write_init(struct fpga_manager *mgr,
>> > + struct fpga_image_info *info, const char *buf, size_t count)
>> > +{
>> > + struct fpga_fme *fme = mgr->priv;
>> > + struct platform_device *pdev;
>> > + struct feature_fme_pr *fme_pr;
>> > + struct feature_fme_pr_ctl fme_pr_ctl;
>> > + struct feature_fme_pr_status fme_pr_status;
>> > +
>> > + pdev = fme->pdata->dev;
>> > + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
>> > + FME_FEATURE_ID_PR_MGMT);
>> > + if (!fme_pr)
>> > + return -EINVAL;
>> > +
>> > + if (WARN_ON(info->flags != FPGA_MGR_PARTIAL_RECONFIG))
>> > + return -EINVAL;
>> > +
>> > + dev_dbg(&pdev->dev, "resetting PR before initiated PR\n");
>> > +
>> > + fme_pr_ctl.csr = readq(&fme_pr->control);
>> > + fme_pr_ctl.pr_reset = 1;
>> > + writeq(fme_pr_ctl.csr, &fme_pr->control);
>> > +
>> > + fme_pr_ctl.pr_reset_ack = 1;
>> > +
>> > + if (fpga_wait_register_field(pr_reset_ack, fme_pr_ctl,
>> > + &fme_pr->control, PR_WAIT_TIMEOUT, 1)) {
>> > + dev_err(&pdev->dev, "maximum PR timeout\n");
>> > + return -ETIMEDOUT;
>> > + }
>> > +
>> > + fme_pr_ctl.csr = readq(&fme_pr->control);
>> > + fme_pr_ctl.pr_reset = 0;
>> > + writeq(fme_pr_ctl.csr, &fme_pr->control);
>> > +
>> > + dev_dbg(&pdev->dev,
>> > + "waiting for PR resource in HW to be initialized and ready\n");
>> > +
>> > + fme_pr_status.pr_host_status = PR_HOST_STATUS_IDLE;
>> > +
>> > + if (fpga_wait_register_field(pr_host_status, fme_pr_status,
>> > + &fme_pr->status, PR_WAIT_TIMEOUT, 1)) {
>> > + dev_err(&pdev->dev, "maximum PR timeout\n");
>> > + return -ETIMEDOUT;
>> > + }
>> > +
>> > + dev_dbg(&pdev->dev, "check if have any previous PR error\n");
>> > + pr_err_handle(pdev, fme_pr);
>> > + return 0;
>> > +}
>> > +
>> > +static int fme_pr_write(struct fpga_manager *mgr,
>> > + const char *buf, size_t count)
>> > +{
>> > + struct fpga_fme *fme = mgr->priv;
>> > + struct platform_device *pdev;
>> > + struct feature_fme_pr *fme_pr;
>> > + struct feature_fme_pr_ctl fme_pr_ctl;
>> > + struct feature_fme_pr_status fme_pr_status;
>> > + struct feature_fme_pr_data fme_pr_data;
>> > + int delay, pr_credit, i = 0;
>> > +
>> > + pdev = fme->pdata->dev;
>> > + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
>> > + FME_FEATURE_ID_PR_MGMT);
>> > +
>> > + dev_dbg(&pdev->dev, "set PR port ID and start request\n");
>> > +
>> > + fme_pr_ctl.csr = readq(&fme_pr->control);
>> > + fme_pr_ctl.pr_regionid = fme->port_id;
>> > + fme_pr_ctl.pr_start_req = 1;
>> > + writeq(fme_pr_ctl.csr, &fme_pr->control);
>> > +
>> > + dev_dbg(&pdev->dev, "pushing data from bitstream to HW\n");
>> > +
>> > + fme_pr_status.csr = readq(&fme_pr->status);
>> > + pr_credit = fme_pr_status.pr_credit;
>> > +
>> > + while (count > 0) {
>> > + delay = 0;
>> > + while (pr_credit <= 1) {
>> > + if (delay++ > PR_WAIT_TIMEOUT) {
>> > + dev_err(&pdev->dev, "maximum try\n");
>> > + return -ETIMEDOUT;
>> > + }
>> > + udelay(1);
>> > +
>> > + fme_pr_status.csr = readq(&fme_pr->status);
>> > + pr_credit = fme_pr_status.pr_credit;
>> > + };
>> > +
>> > + if (count >= 4) {
>> > + fme_pr_data.rsvd = 0;
>> > + fme_pr_data.pr_data_raw = *(((u32 *)buf) + i);
>> > + writeq(fme_pr_data.csr, &fme_pr->data);
>> > + count -= 4;
>> > + pr_credit--;
>> > + i++;
>> > + } else {
>> > + WARN_ON(1);
>> > + return -EINVAL;
>> > + }
>> > + }
>> > +
>> > + return 0;
>> > +}
>> > +
>> > +static int fme_pr_write_complete(struct fpga_manager *mgr,
>> > + struct fpga_image_info *info)
>> > +{
>> > + struct fpga_fme *fme = mgr->priv;
>> > + struct platform_device *pdev;
>> > + struct feature_fme_pr *fme_pr;
>> > + struct feature_fme_pr_ctl fme_pr_ctl;
>> > +
>> > + pdev = fme->pdata->dev;
>> > + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
>> > + FME_FEATURE_ID_PR_MGMT);
>> > +
>> > + fme_pr_ctl.csr = readq(&fme_pr->control);
>> > + fme_pr_ctl.pr_push_complete = 1;
>> > + writeq(fme_pr_ctl.csr, &fme_pr->control);
>> > +
>> > + dev_dbg(&pdev->dev, "green bitstream push complete\n");
>> > + dev_dbg(&pdev->dev, "waiting for HW to release PR resource\n");
>> > +
>> > + fme_pr_ctl.pr_start_req = 0;
>> > +
>> > + if (fpga_wait_register_field(pr_start_req, fme_pr_ctl,
>> > + &fme_pr->control, PR_WAIT_TIMEOUT, 1)) {
>> > + dev_err(&pdev->dev, "maximum try.\n");
>> > + return -ETIMEDOUT;
>> > + }
>> > +
>> > + dev_dbg(&pdev->dev, "PR operation complete, checking status\n");
>> > + fme->pr_err = pr_err_handle(pdev, fme_pr);
>> > + if (fme->pr_err)
>> > + return -EIO;
>> > +
>> > + dev_dbg(&pdev->dev, "PR done successfully\n");
>> > + return 0;
>> > +}
>> > +
>> > +static enum fpga_mgr_states fme_pr_state(struct fpga_manager *mgr)
>> > +{
>> > + return FPGA_MGR_STATE_UNKNOWN;
>> > +}
>> > +
>> > +static const struct fpga_manager_ops fme_pr_ops = {
>> > + .write_init = fme_pr_write_init,
>> > + .write = fme_pr_write,
>> > + .write_complete = fme_pr_write_complete,
>> > + .state = fme_pr_state,
>> > +};
>> > +
>> > +static int fme_pr(struct platform_device *pdev, unsigned long arg)
>> > +{
>> > + void __user *argp = (void __user *)arg;
>> > + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
>> > + struct fpga_fme *fme;
>> > + struct fpga_manager *mgr;
>> > + struct feature_fme_header *fme_hdr;
>> > + struct feature_fme_capability fme_capability;
>> > + struct fpga_image_info info;
>> > + struct fpga_fme_port_pr port_pr;
>> > + struct platform_device *port;
>> > + unsigned long minsz;
>> > + void *buf = NULL;
>> > + int ret = 0;
>> > +
>> > + minsz = offsetofend(struct fpga_fme_port_pr, status);
>> > +
>> > + if (copy_from_user(&port_pr, argp, minsz))
>> > + return -EFAULT;
>> > +
>> > + if (port_pr.argsz < minsz || port_pr.flags)
>> > + return -EINVAL;
>> > +
>> > + if (!IS_ALIGNED(port_pr.buffer_size, 4))
>> > + return -EINVAL;
>> > +
>> > + /* get fme header region */
>> > + fme_hdr = get_feature_ioaddr_by_index(&pdev->dev,
>> > + FME_FEATURE_ID_HEADER);
>> > + if (WARN_ON(!fme_hdr))
>> > + return -EINVAL;
>> > +
>> > + /* check port id */
>> > + fme_capability.csr = readq(&fme_hdr->capability);
>> > + if (port_pr.port_id >= fme_capability.num_ports) {
>> > + dev_dbg(&pdev->dev, "port number more than maximum\n");
>> > + return -EINVAL;
>> > + }
>> > +
>> > + if (!access_ok(VERIFY_READ, port_pr.buffer_address,
>> > + port_pr.buffer_size))
>> > + return -EFAULT;
>> > +
>> > + buf = vmalloc(port_pr.buffer_size);
>> > + if (!buf)
>> > + return -ENOMEM;
>> > +
>> > + if (copy_from_user(buf, (void __user *)port_pr.buffer_address,
>> > + port_pr.buffer_size)) {
>> > + ret = -EFAULT;
>> > + goto free_exit;
>> > + }
>> > +
>> > + memset(&info, 0, sizeof(struct fpga_image_info));
>> > + info.flags = FPGA_MGR_PARTIAL_RECONFIG;
>> > +
>> > + mgr = fpga_mgr_get(&pdev->dev);
>> > + if (IS_ERR(mgr)) {
>> > + ret = PTR_ERR(mgr);
>> > + goto free_exit;
>> > + }
>> > +
>> > + mutex_lock(&pdata->lock);
>> > + fme = fpga_pdata_get_private(pdata);
>> > + /* fme device has been unregistered. */
>> > + if (!fme) {
>> > + ret = -EINVAL;
>> > + goto unlock_exit;
>> > + }
>> > +
>> > + fme->pr_err = 0;
>> > + fme->port_id = port_pr.port_id;
>>
>> It looks like you're using private data to communicate with the
>> driver, i.e. there is something you want to do with the fpga manager
>> framework and it doesn't have that feature. The better way would be
>> for us to expand the framework so you don't need to do that.
>>
>> port_id is the kind of thing that should be communicated to the driver
>> through fpga_image_info, so we could add that to the struct. Should
>> we call it port_id?
Let's call it region_id.
>> Or is there something more generic that may be
>> useful in the future for other architectures?.
>
> Hi Alan
>
> Thanks for your feedback. :)
>
> As you know, each Intel FPGA device may have more than one accelerator,
> and all accelerators share the same fpga_manager (only one FME module).
> port_id = 0 means the first accelerator on this fpga devices. So it's
> needed by the fpga_manager to distinguish one accelerator from others
> for partial reconfiguration.
>
> Adding a member like a 'id' to fpga_image_info definitely works for us
> in this case. We can add it this way, but I feel 'id' here seems not
> related to fpga image, but characteristic of fpga region.
The fpga_image_info struct started life as just image specific info,
but I want it to go in the direction of including parameters needed to
program it this specific time. Otherwise we are stuck having to keep
adding parameters as our use of FPGA develops. It probably could be
documented better as 'information needed to program a FPGA image'
rather than strictly 'information about this particular FPGA image'.
My patch "fpga-mgr: pass parameters for loading fpga in image info"
goes in this direction by having the buf, firmware name, or sg list
passed in the info for the added fpga_mgr_load() function. Actually I
should probably simplify the API and get rid of fpga_mgr_buf_load,
fpga_mgr_buf_load_sg, and fpga_mgr_firmware_load and require people to
use fpga_mgr_load (passing all parameters in fpga_image_info).
> It may be a
> little confusing. One rough idea is that keep this info under fpga region
> (maybe its private data), and pass the fpga-region to fpga_mgr_buf_load,
Yes, keep this info in fpga-region. When the region wants to program
using fpga-mgr, add the region id to fpga_image_info. I propose
calling it region_id.
> then fpga_manager knows the target region for partial reconfiguration.
> If consider pr sysfs interface support under fpga-region in the future,
> then we don't need to create a new 'id' sysfs file, as fpga-region itself
> could provide this kind of info. But I'm not sure if this is the right
> direction.
>
>>
>> pr_err appears to be machine specific error codes that are
>> communicated outside your low level driver. (Calling it pr_err is
>> extra confusing since Linux already has a commonly name function by
>> the same name). The framework has state, but that's not doing what
>> you want here. Maybe we could add a framework ops called status so
>> that status could be communicated from the low level driver. It would
>> be useful to abstract the machine specific state to a defined enum
>> that would be part of the fpga mgr framework. I remember we had
>> something like that in the earliest version of fpga manager but it got
>> changed to state rather than status for some reason.
>>
>
> Adding a status function and a 'status' member to fpga manager sounds good.
> But I'm not sure how to abstract these error codes, currently what we have
> are all PR error codes for Intel FPGA device, e.g "PR FIFO overflow error",
> "PR secure load error" and etc (see include/uapi/linux/intel-fpga.h). Is it
> fine to let each driver to define how to use that 'status' for machine
> specific status?
I looked at the list of errors in include/uapi/linux/intel-fpga.h.
They all seem pretty generic to me except I am not clear what "secure
load error" or "IP protocol error" mean and whether other
architectures would have them. But certainly things like crc error,
incompatible bitstream, fifo overflow are generic. So let's see if we
can define all these in a way that's generic and just pass up a error
number. That way upper layers can know how to deal with them
possibly. I would take the word "PR" off these since the error set
applies whether someone is doing full reconfig or partial reconfig.
>
>> mgr->dev won't work for you for some of the things you are using fme->pdev for?
>>
>
> Does 'fme->pdev' mean the platform device for fme? I think we use platform
> device to represent the FME module, and FME module may have multiple sub
> features, driver discovers these sub features via the 'Device Feature List'
> in the PCIe Device Bar. PR is only one of the sub features under FME module,
> if any FME module doesn't have PR sub feature, fpga-manager will not be
> created at all. But this will not impact other sub features, e.g thermal
> management, error reporting and etc, to create their own interfaces under
> the platform device.
If we need it for private data, that's may be actually OK. Just
wondering whether the priv was needed. I think that's the one
element in the private data struct that was used privately instead of
being used to pass info outside the framework.
>
> Thanks
> Hao
>
>> Alan
>>
>> > +
>> > + /* Find and get port device by index */
>> > + port = pdata->fpga_for_each_port(pdev, &fme->port_id,
>> > + fpga_port_check_id);
>> > + WARN_ON(!port);
>> > +
>> > + /* Disable Port before PR */
>> > + fpga_port_disable(port);
>> > +
>> > + ret = fpga_mgr_buf_load(mgr, &info, buf, port_pr.buffer_size);
>> > + port_pr.status = fme->pr_err;
>> > +
>> > + /* Re-enable Port after PR finished */
>> > + fpga_port_enable(port);
>> > +
>> > + put_device(&port->dev);
>> > +
>> > +unlock_exit:
>> > + mutex_unlock(&pdata->lock);
>> > + fpga_mgr_put(mgr);
>> > +free_exit:
>> > + vfree(buf);
>> > + if (copy_to_user((void __user *)arg, &port_pr, minsz))
>> > + return -EFAULT;
>> > + return ret;
>> > +}
>> > +
>> > +static int fpga_fme_pr_probe(struct platform_device *pdev)
>> > +{
>> > + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
>> > + struct fpga_fme *priv;
>> > + int ret;
>> > +
>> > + mutex_lock(&pdata->lock);
>> > + priv = fpga_pdata_get_private(pdata);
>> > + ret = fpga_mgr_register(&pdata->dev->dev,
>> > + "Intel FPGA Manager", &fme_pr_ops, priv);
>> > + mutex_unlock(&pdata->lock);
>> > +
>> > + return ret;
>> > +}
>> > +
>> > +static int fpga_fme_pr_remove(struct platform_device *pdev)
>> > +{
>> > + fpga_mgr_unregister(&pdev->dev);
>> > + return 0;
>> > +}
>> > +
>> > +static int pr_mgmt_init(struct platform_device *pdev, struct feature *feature)
>> > +{
>> > + int ret;
>> > +
>> > + ret = fpga_fme_pr_probe(pdev);
>> > + if (ret)
>> > + return ret;
>> > +
>> > + ret = sysfs_create_group(&pdev->dev.kobj, &pr_mgmt_attr_group);
>> > + if (ret)
>> > + fpga_fme_pr_remove(pdev);
>> > +
>> > + return ret;
>> > +}
>> > +
>> > +static void pr_mgmt_uinit(struct platform_device *pdev, struct feature *feature)
>> > +{
>> > + sysfs_remove_group(&pdev->dev.kobj, &pr_mgmt_attr_group);
>> > + fpga_fme_pr_remove(pdev);
>> > +}
>> > +
>> > +static long fme_pr_ioctl(struct platform_device *pdev, struct feature *feature,
>> > + unsigned int cmd, unsigned long arg)
>> > +{
>> > + long ret;
>> > +
>> > + switch (cmd) {
>> > + case FPGA_FME_PORT_PR:
>> > + ret = fme_pr(pdev, arg);
>> > + break;
>> > + default:
>> > + ret = -ENODEV;
>> > + }
>> > +
>> > + return ret;
>> > +}
>> > +
>> > +struct feature_ops pr_mgmt_ops = {
>> > + .init = pr_mgmt_init,
>> > + .uinit = pr_mgmt_uinit,
>> > + .ioctl = fme_pr_ioctl,
>> > +};
>> > diff --git a/drivers/fpga/intel/fme.h b/drivers/fpga/intel/fme.h
>> > new file mode 100644
>> > index 0000000..d6cb7ce
>> > --- /dev/null
>> > +++ b/drivers/fpga/intel/fme.h
>> > @@ -0,0 +1,32 @@
>> > +/*
>> > + * Header file for Intel FPGA Management Engine (FME) Driver
>> > + *
>> > + * Copyright (C) 2017 Intel Corporation, Inc.
>> > + *
>> > + * Authors:
>> > + * Kang Luwei <luwei.kang@intel.com>
>> > + * Xiao Guangrong <guangrong.xiao@linux.intel.com>
>> > + * Joseph Grecco <joe.grecco@intel.com>
>> > + * Enno Luebbers <enno.luebbers@intel.com>
>> > + * Tim Whisonant <tim.whisonant@intel.com>
>> > + * Ananda Ravuri <ananda.ravuri@intel.com>
>> > + * Henry Mitchel <henry.mitchel@intel.com>
>> > + *
>> > + * This work is licensed under a dual BSD/GPLv2 license. When using or
>> > + * redistributing this file, you may do so under either license. See the
>> > + * LICENSE.BSD file under this directory for the BSD license and see
>> > + * the COPYING file in the top-level directory for the GPLv2 license.
>> > + */
>> > +
>> > +#ifndef __INTEL_FME_H
>> > +#define __INTEL_FME_H
>> > +
>> > +struct fpga_fme {
>> > + u8 port_id;
>> > + u64 pr_err;
>> > + struct feature_platform_data *pdata;
>> > +};
>> > +
>> > +extern struct feature_ops pr_mgmt_ops;
>> > +
>> > +#endif
>> > diff --git a/include/uapi/linux/intel-fpga.h b/include/uapi/linux/intel-fpga.h
>> > index 992e556..77658316 100644
>> > --- a/include/uapi/linux/intel-fpga.h
>> > +++ b/include/uapi/linux/intel-fpga.h
>> > @@ -18,6 +18,8 @@
>> > #ifndef _UAPI_LINUX_INTEL_FPGA_H
>> > #define _UAPI_LINUX_INTEL_FPGA_H
>> >
>> > +#include <linux/types.h>
>> > +
>> > #define FPGA_API_VERSION 0
>> >
>> > /*
>> > @@ -30,6 +32,7 @@
>> > #define FPGA_MAGIC 0xB6
>> >
>> > #define FPGA_BASE 0
>> > +#define FME_BASE 0x80
>> >
>> > /**
>> > * FPGA_GET_API_VERSION - _IO(FPGA_MAGIC, FPGA_BASE + 0)
>> > @@ -49,4 +52,45 @@
>> >
>> > #define FPGA_CHECK_EXTENSION _IO(FPGA_MAGIC, FPGA_BASE + 1)
>> >
>> > +/* IOCTLs for FME file descriptor */
>> > +
>> > +/**
>> > + * FPGA_FME_PORT_PR - _IOWR(FPGA_MAGIC, FME_BASE + 0, struct fpga_fme_port_pr)
>> > + *
>> > + * Driver does Partial Reconfiguration based on Port ID and Buffer (Image)
>> > + * provided by caller.
>> > + * Return: 0 on success, -errno on failure.
>> > + * If FPGA_FME_PORT_PR returns -EIO, that indicates the HW has detected
>> > + * some errors during PR, under this case, the user can fetch HW error code
>> > + * from fpga_fme_port_pr.status. Each bit on the error code is used as the
>> > + * index for the array created by DEFINE_FPGA_PR_ERR_MSG().
>> > + * Otherwise, it is always zero.
>> > + */
>> > +
>> > +#define DEFINE_FPGA_PR_ERR_MSG(_name_) \
>> > +static const char * const _name_[] = { \
>> > + "PR operation error detected", \
>> > + "PR CRC error detected", \
>> > + "PR incompatiable bitstream error detected", \
>> > + "PR IP protocol error detected", \
>> > + "PR FIFO overflow error detected", \
>> > + "Reserved", \
>> > + "PR secure load error detected", \
>> > +}
>> > +
>> > +#define PR_MAX_ERR_NUM 7
>> > +
>> > +struct fpga_fme_port_pr {
>> > + /* Input */
>> > + __u32 argsz; /* Structure length */
>> > + __u32 flags; /* Zero for now */
>> > + __u32 port_id;
>> > + __u32 buffer_size;
>> > + __u64 buffer_address; /* Userspace address to the buffer for PR */
>> > + /* Output */
>> > + __u64 status; /* HW error code if ioctl returns -EIO */
>> > +};
>> > +
>> > +#define FPGA_FME_PORT_PR _IO(FPGA_MAGIC, FME_BASE + 0)
>> > +
>> > #endif /* _UAPI_INTEL_FPGA_H */
>> > --
>> > 2.7.4
>> >
[toc] | [prev] | [next] | [standalone]
| From | Wu Hao <hao.wu@intel.com> |
|---|---|
| Date | 2017-04-04 08:20 +0200 |
| Subject | Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support |
| Message-ID | <tspYS-130-23@gated-at.bofh.it> |
| In reply to | #1615400 |
On Mon, Apr 03, 2017 at 11:30:55AM -0500, Alan Tull wrote:
> On Sat, Apr 1, 2017 at 6:08 AM, Wu Hao <hao.wu@intel.com> wrote:
> > On Fri, Mar 31, 2017 at 02:10:12PM -0500, Alan Tull wrote:
> >> On Thu, Mar 30, 2017 at 7:08 AM, Wu Hao <hao.wu@intel.com> wrote:
> >> > From: Kang Luwei <luwei.kang@intel.com>
> >> >
> >> > Partial Reconfiguration (PR) is the most important function for FME. It
> >> > allows reconfiguration for given Port/Accelerated Function Unit (AFU).
> >> >
> >> > This patch adds support for PR sub feature. In this patch, it registers
> >> > a fpga_mgr and implements fpga_manager_ops, and invoke fpga_mgr_buf_load
> >> > for PR operation once PR request received via ioctl. Below user space
> >> > interfaces are exposed by this sub feature.
> >> >
> >> > Sysfs interface:
> >> > * /sys/class/fpga/<fpga.x>/<intel-fpga-fme.x>/interface_id
> >> > Read-only. Indicate the hardware interface information. Userspace
> >> > applications need to check this interface to select correct green
> >> > bitstream format before PR.
> >> >
> >> > Ioctl interface:
> >> > * FPGA_FME_PORT_PR
> >> > Do partial reconfiguration per information from userspace, including
> >> > target port(AFU), buffer size and address info. It returns the PR status
> >> > (PR error code if failed) to userspace.
> >> >
> >> > Signed-off-by: Tim Whisonant <tim.whisonant@intel.com>
> >> > Signed-off-by: Enno Luebbers <enno.luebbers@intel.com>
> >> > Signed-off-by: Shiva Rao <shiva.rao@intel.com>
> >> > Signed-off-by: Christopher Rauer <christopher.rauer@intel.com>
> >> > Signed-off-by: Alan Tull <alan.tull@intel.com>
> >> > Signed-off-by: Kang Luwei <luwei.kang@intel.com>
> >> > Signed-off-by: Xiao Guangrong <guangrong.xiao@linux.intel.com>
> >> > Signed-off-by: Wu Hao <hao.wu@intel.com>
> >> > ---
> >> > drivers/fpga/intel/Makefile | 2 +-
> >> > drivers/fpga/intel/feature-dev.h | 58 ++++++
> >> > drivers/fpga/intel/fme-main.c | 44 ++++-
> >> > drivers/fpga/intel/fme-pr.c | 400 +++++++++++++++++++++++++++++++++++++++
> >> > drivers/fpga/intel/fme.h | 32 ++++
> >> > include/uapi/linux/intel-fpga.h | 44 +++++
> >> > 6 files changed, 578 insertions(+), 2 deletions(-)
> >> > create mode 100644 drivers/fpga/intel/fme-pr.c
> >> > create mode 100644 drivers/fpga/intel/fme.h
> >> >
> >> > diff --git a/drivers/fpga/intel/Makefile b/drivers/fpga/intel/Makefile
> >> > index 546861d..0452cb6 100644
> >> > --- a/drivers/fpga/intel/Makefile
> >> > +++ b/drivers/fpga/intel/Makefile
> >> > @@ -2,4 +2,4 @@ obj-$(CONFIG_INTEL_FPGA_PCI) += intel-fpga-pci.o
> >> > obj-$(CONFIG_INTEL_FPGA_FME) += intel-fpga-fme.o
> >> >
> >> > intel-fpga-pci-objs := pcie.o feature-dev.o
> >> > -intel-fpga-fme-objs := fme-main.o
> >> > +intel-fpga-fme-objs := fme-main.o fme-pr.o
> >> > diff --git a/drivers/fpga/intel/feature-dev.h b/drivers/fpga/intel/feature-dev.h
> >> > index dccc283..5a25c915 100644
> >> > --- a/drivers/fpga/intel/feature-dev.h
> >> > +++ b/drivers/fpga/intel/feature-dev.h
> >> > @@ -150,8 +150,66 @@ struct feature_fme_err {
> >> > };
> >> >
> >> > /* FME Partial Reconfiguration Sub Feature Register Set */
> >> > +/* FME PR Control Register */
> >> > +struct feature_fme_pr_ctl {
> >> > + union {
> >> > + u64 csr;
> >> > + struct {
> >> > + u8 pr_reset:1; /* Reset PR Engine */
> >> > + u8 rsvdz1:3;
> >> > + u8 pr_reset_ack:1; /* Reset PR Engine Ack */
> >> > + u8 rsvdz2:3;
> >> > + u8 pr_regionid:2; /* PR Region ID */
> >> > + u8 rsvdz3:2;
> >> > + u8 pr_start_req:1; /* PR Start Request */
> >> > + u8 pr_push_complete:1; /* PR Data push complete */
> >> > + u8 pr_kind:1; /* Load Customer or Intel GBS */
> >> > + u32 rsvdz4:17;
> >> > + u32 config_data;
> >> > + };
> >> > + };
> >> > +};
> >> > +
> >> > +/* FME PR Status Register */
> >> > +struct feature_fme_pr_status {
> >> > + union {
> >> > + u64 csr;
> >> > + struct {
> >> > + u16 pr_credit:9; /* Number of PR Credits */
> >> > + u8 rsvdz1:7;
> >> > + u8 pr_status:1; /* PR Operation status */
> >> > + u8 rsvdz2:3;
> >> > + u8 pr_ctrlr_status:3; /* Controller status */
> >> > + u8 rsvdz3:1;
> >> > + u8 pr_host_status:4; /* PR Host status */
> >> > + u64 rsvdz4:36;
> >> > + };
> >> > + };
> >> > +};
> >> > +
> >> > +/* FME PR Data Register */
> >> > +struct feature_fme_pr_data {
> >> > + union {
> >> > + u64 csr;
> >> > + struct {
> >> > + /* PR data from the raw-binary file */
> >> > + u32 pr_data_raw;
> >> > + u32 rsvd;
> >> > + };
> >> > + };
> >> > +};
> >> > +
> >> > struct feature_fme_pr {
> >> > struct feature_header header;
> >> > + struct feature_fme_pr_ctl control;
> >> > + struct feature_fme_pr_status status;
> >> > + struct feature_fme_pr_data data;
> >> > + u64 error;
> >> > +
> >> > + u64 rsvd[16];
> >> > +
> >> > + u64 intfc_id_l; /* PR interface Id Low */
> >> > + u64 intfc_id_h; /* PR interface Id High */
> >> > };
> >> >
> >> > /* PORT Header Register Set */
> >> > diff --git a/drivers/fpga/intel/fme-main.c b/drivers/fpga/intel/fme-main.c
> >> > index 36d0c4c..0d9a7a6 100644
> >> > --- a/drivers/fpga/intel/fme-main.c
> >> > +++ b/drivers/fpga/intel/fme-main.c
> >> > @@ -23,6 +23,7 @@
> >> > #include <linux/intel-fpga.h>
> >> >
> >> > #include "feature-dev.h"
> >> > +#include "fme.h"
> >> >
> >> > static ssize_t ports_num_show(struct device *dev,
> >> > struct device_attribute *attr, char *buf)
> >> > @@ -101,6 +102,10 @@ static struct feature_driver fme_feature_drvs[] = {
> >> > .ops = &fme_hdr_ops,
> >> > },
> >> > {
> >> > + .name = FME_FEATURE_PR_MGMT,
> >> > + .ops = &pr_mgmt_ops,
> >> > + },
> >> > + {
> >> > .ops = NULL,
> >> > },
> >> > };
> >> > @@ -182,14 +187,48 @@ static const struct file_operations fme_fops = {
> >> > .unlocked_ioctl = fme_ioctl,
> >> > };
> >> >
> >> > +static int fme_dev_init(struct platform_device *pdev)
> >> > +{
> >> > + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> >> > + struct fpga_fme *fme;
> >> > +
> >> > + fme = devm_kzalloc(&pdev->dev, sizeof(*fme), GFP_KERNEL);
> >> > + if (!fme)
> >> > + return -ENOMEM;
> >> > +
> >> > + fme->pdata = pdata;
> >> > +
> >> > + mutex_lock(&pdata->lock);
> >> > + fpga_pdata_set_private(pdata, fme);
> >> > + mutex_unlock(&pdata->lock);
> >> > + return 0;
> >> > +}
> >> > +
> >> > +static void fme_dev_destroy(struct platform_device *pdev)
> >> > +{
> >> > + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> >> > + struct fpga_fme *fme;
> >> > +
> >> > + mutex_lock(&pdata->lock);
> >> > + fme = fpga_pdata_get_private(pdata);
> >> > + fpga_pdata_set_private(pdata, NULL);
> >> > + mutex_unlock(&pdata->lock);
> >> > +
> >> > + devm_kfree(&pdev->dev, fme);
> >> > +}
> >> > +
> >> > static int fme_probe(struct platform_device *pdev)
> >> > {
> >> > int ret;
> >> >
> >> > - ret = fpga_dev_feature_init(pdev, fme_feature_drvs);
> >> > + ret = fme_dev_init(pdev);
> >> > if (ret)
> >> > goto exit;
> >> >
> >> > + ret = fpga_dev_feature_init(pdev, fme_feature_drvs);
> >> > + if (ret)
> >> > + goto dev_destroy;
> >> > +
> >> > ret = fpga_register_dev_ops(pdev, &fme_fops, THIS_MODULE);
> >> > if (ret)
> >> > goto feature_uinit;
> >> > @@ -198,6 +237,8 @@ static int fme_probe(struct platform_device *pdev)
> >> >
> >> > feature_uinit:
> >> > fpga_dev_feature_uinit(pdev);
> >> > +dev_destroy:
> >> > + fme_dev_destroy(pdev);
> >> > exit:
> >> > return ret;
> >> > }
> >> > @@ -206,6 +247,7 @@ static int fme_remove(struct platform_device *pdev)
> >> > {
> >> > fpga_dev_feature_uinit(pdev);
> >> > fpga_unregister_dev_ops(pdev);
> >> > + fme_dev_destroy(pdev);
> >> > return 0;
> >> > }
> >> >
> >> > diff --git a/drivers/fpga/intel/fme-pr.c b/drivers/fpga/intel/fme-pr.c
> >> > new file mode 100644
> >> > index 0000000..3b44a3e
> >> > --- /dev/null
> >> > +++ b/drivers/fpga/intel/fme-pr.c
> >> > @@ -0,0 +1,400 @@
> >> > +/*
> >> > + * Driver for Intel FPGA Management Engine (FME) Partial Reconfiguration
> >> > + *
> >> > + * Copyright (C) 2017 Intel Corporation, Inc.
> >> > + *
> >> > + * Authors:
> >> > + * Kang Luwei <luwei.kang@intel.com>
> >> > + * Xiao Guangrong <guangrong.xiao@linux.intel.com>
> >> > + * Joseph Grecco <joe.grecco@intel.com>
> >> > + * Enno Luebbers <enno.luebbers@intel.com>
> >> > + * Tim Whisonant <tim.whisonant@intel.com>
> >> > + * Ananda Ravuri <ananda.ravuri@intel.com>
> >> > + * Christopher Rauer <christopher.rauer@intel.com>
> >> > + * Henry Mitchel <henry.mitchel@intel.com>
> >> > + *
> >> > + * This work is licensed under a dual BSD/GPLv2 license. When using or
> >> > + * redistributing this file, you may do so under either license. See the
> >> > + * LICENSE.BSD file under this directory for the BSD license and see
> >> > + * the COPYING file in the top-level directory for the GPLv2 license.
> >> > + */
> >> > +
> >> > +#include <linux/types.h>
> >> > +#include <linux/device.h>
> >> > +#include <linux/vmalloc.h>
> >> > +#include <linux/uaccess.h>
> >> > +#include <linux/fpga/fpga-mgr.h>
> >> > +#include <linux/intel-fpga.h>
> >> > +
> >> > +#include "feature-dev.h"
> >> > +#include "fme.h"
> >> > +
> >> > +#define PR_WAIT_TIMEOUT 8000000
> >> > +
> >> > +#define PR_HOST_STATUS_IDLE 0
> >> > +
> >> > +DEFINE_FPGA_PR_ERR_MSG(pr_err_msg);
> >> > +
> >> > +static ssize_t interface_id_show(struct device *dev,
> >> > + struct device_attribute *attr, char *buf)
> >> > +{
> >> > + u64 intfc_id_l, intfc_id_h;
> >> > + struct feature_fme_pr *fme_pr
> >> > + = get_feature_ioaddr_by_index(dev, FME_FEATURE_ID_PR_MGMT);
> >> > +
> >> > + intfc_id_l = readq(&fme_pr->intfc_id_l);
> >> > + intfc_id_h = readq(&fme_pr->intfc_id_h);
> >> > +
> >> > + return scnprintf(buf, PAGE_SIZE, "%016llx%016llx\n",
> >> > + (unsigned long long)intfc_id_h,
> >> > + (unsigned long long)intfc_id_l);
> >> > +}
> >> > +static DEVICE_ATTR_RO(interface_id);
> >> > +
> >> > +static struct attribute *pr_mgmt_attrs[] = {
> >> > + &dev_attr_interface_id.attr,
> >> > + NULL,
> >> > +};
> >> > +
> >> > +struct attribute_group pr_mgmt_attr_group = {
> >> > + .attrs = pr_mgmt_attrs,
> >> > + .name = "pr",
> >> > +};
> >> > +
> >> > +static u64
> >> > +pr_err_handle(struct platform_device *pdev, struct feature_fme_pr *fme_pr)
> >> > +{
> >> > + struct feature_fme_pr_status fme_pr_status;
> >> > + unsigned long err_code;
> >> > + u64 fme_pr_error;
> >> > + int i = 0;
> >> > +
> >> > + fme_pr_status.csr = readq(&fme_pr->status);
> >> > + if (!fme_pr_status.pr_status)
> >> > + return 0;
> >> > +
> >> > + err_code = fme_pr_error = readq(&fme_pr->error);
> >> > + for_each_set_bit(i, &err_code, PR_MAX_ERR_NUM)
> >> > + dev_dbg(&pdev->dev, "%s\n", pr_err_msg[i]);
> >> > + writeq(fme_pr_error, &fme_pr->error);
> >> > + return fme_pr_error;
> >> > +}
> >> > +
> >> > +static int fme_pr_write_init(struct fpga_manager *mgr,
> >> > + struct fpga_image_info *info, const char *buf, size_t count)
> >> > +{
> >> > + struct fpga_fme *fme = mgr->priv;
> >> > + struct platform_device *pdev;
> >> > + struct feature_fme_pr *fme_pr;
> >> > + struct feature_fme_pr_ctl fme_pr_ctl;
> >> > + struct feature_fme_pr_status fme_pr_status;
> >> > +
> >> > + pdev = fme->pdata->dev;
> >> > + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
> >> > + FME_FEATURE_ID_PR_MGMT);
> >> > + if (!fme_pr)
> >> > + return -EINVAL;
> >> > +
> >> > + if (WARN_ON(info->flags != FPGA_MGR_PARTIAL_RECONFIG))
> >> > + return -EINVAL;
> >> > +
> >> > + dev_dbg(&pdev->dev, "resetting PR before initiated PR\n");
> >> > +
> >> > + fme_pr_ctl.csr = readq(&fme_pr->control);
> >> > + fme_pr_ctl.pr_reset = 1;
> >> > + writeq(fme_pr_ctl.csr, &fme_pr->control);
> >> > +
> >> > + fme_pr_ctl.pr_reset_ack = 1;
> >> > +
> >> > + if (fpga_wait_register_field(pr_reset_ack, fme_pr_ctl,
> >> > + &fme_pr->control, PR_WAIT_TIMEOUT, 1)) {
> >> > + dev_err(&pdev->dev, "maximum PR timeout\n");
> >> > + return -ETIMEDOUT;
> >> > + }
> >> > +
> >> > + fme_pr_ctl.csr = readq(&fme_pr->control);
> >> > + fme_pr_ctl.pr_reset = 0;
> >> > + writeq(fme_pr_ctl.csr, &fme_pr->control);
> >> > +
> >> > + dev_dbg(&pdev->dev,
> >> > + "waiting for PR resource in HW to be initialized and ready\n");
> >> > +
> >> > + fme_pr_status.pr_host_status = PR_HOST_STATUS_IDLE;
> >> > +
> >> > + if (fpga_wait_register_field(pr_host_status, fme_pr_status,
> >> > + &fme_pr->status, PR_WAIT_TIMEOUT, 1)) {
> >> > + dev_err(&pdev->dev, "maximum PR timeout\n");
> >> > + return -ETIMEDOUT;
> >> > + }
> >> > +
> >> > + dev_dbg(&pdev->dev, "check if have any previous PR error\n");
> >> > + pr_err_handle(pdev, fme_pr);
> >> > + return 0;
> >> > +}
> >> > +
> >> > +static int fme_pr_write(struct fpga_manager *mgr,
> >> > + const char *buf, size_t count)
> >> > +{
> >> > + struct fpga_fme *fme = mgr->priv;
> >> > + struct platform_device *pdev;
> >> > + struct feature_fme_pr *fme_pr;
> >> > + struct feature_fme_pr_ctl fme_pr_ctl;
> >> > + struct feature_fme_pr_status fme_pr_status;
> >> > + struct feature_fme_pr_data fme_pr_data;
> >> > + int delay, pr_credit, i = 0;
> >> > +
> >> > + pdev = fme->pdata->dev;
> >> > + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
> >> > + FME_FEATURE_ID_PR_MGMT);
> >> > +
> >> > + dev_dbg(&pdev->dev, "set PR port ID and start request\n");
> >> > +
> >> > + fme_pr_ctl.csr = readq(&fme_pr->control);
> >> > + fme_pr_ctl.pr_regionid = fme->port_id;
> >> > + fme_pr_ctl.pr_start_req = 1;
> >> > + writeq(fme_pr_ctl.csr, &fme_pr->control);
> >> > +
> >> > + dev_dbg(&pdev->dev, "pushing data from bitstream to HW\n");
> >> > +
> >> > + fme_pr_status.csr = readq(&fme_pr->status);
> >> > + pr_credit = fme_pr_status.pr_credit;
> >> > +
> >> > + while (count > 0) {
> >> > + delay = 0;
> >> > + while (pr_credit <= 1) {
> >> > + if (delay++ > PR_WAIT_TIMEOUT) {
> >> > + dev_err(&pdev->dev, "maximum try\n");
> >> > + return -ETIMEDOUT;
> >> > + }
> >> > + udelay(1);
> >> > +
> >> > + fme_pr_status.csr = readq(&fme_pr->status);
> >> > + pr_credit = fme_pr_status.pr_credit;
> >> > + };
> >> > +
> >> > + if (count >= 4) {
> >> > + fme_pr_data.rsvd = 0;
> >> > + fme_pr_data.pr_data_raw = *(((u32 *)buf) + i);
> >> > + writeq(fme_pr_data.csr, &fme_pr->data);
> >> > + count -= 4;
> >> > + pr_credit--;
> >> > + i++;
> >> > + } else {
> >> > + WARN_ON(1);
> >> > + return -EINVAL;
> >> > + }
> >> > + }
> >> > +
> >> > + return 0;
> >> > +}
> >> > +
> >> > +static int fme_pr_write_complete(struct fpga_manager *mgr,
> >> > + struct fpga_image_info *info)
> >> > +{
> >> > + struct fpga_fme *fme = mgr->priv;
> >> > + struct platform_device *pdev;
> >> > + struct feature_fme_pr *fme_pr;
> >> > + struct feature_fme_pr_ctl fme_pr_ctl;
> >> > +
> >> > + pdev = fme->pdata->dev;
> >> > + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
> >> > + FME_FEATURE_ID_PR_MGMT);
> >> > +
> >> > + fme_pr_ctl.csr = readq(&fme_pr->control);
> >> > + fme_pr_ctl.pr_push_complete = 1;
> >> > + writeq(fme_pr_ctl.csr, &fme_pr->control);
> >> > +
> >> > + dev_dbg(&pdev->dev, "green bitstream push complete\n");
> >> > + dev_dbg(&pdev->dev, "waiting for HW to release PR resource\n");
> >> > +
> >> > + fme_pr_ctl.pr_start_req = 0;
> >> > +
> >> > + if (fpga_wait_register_field(pr_start_req, fme_pr_ctl,
> >> > + &fme_pr->control, PR_WAIT_TIMEOUT, 1)) {
> >> > + dev_err(&pdev->dev, "maximum try.\n");
> >> > + return -ETIMEDOUT;
> >> > + }
> >> > +
> >> > + dev_dbg(&pdev->dev, "PR operation complete, checking status\n");
> >> > + fme->pr_err = pr_err_handle(pdev, fme_pr);
> >> > + if (fme->pr_err)
> >> > + return -EIO;
> >> > +
> >> > + dev_dbg(&pdev->dev, "PR done successfully\n");
> >> > + return 0;
> >> > +}
> >> > +
> >> > +static enum fpga_mgr_states fme_pr_state(struct fpga_manager *mgr)
> >> > +{
> >> > + return FPGA_MGR_STATE_UNKNOWN;
> >> > +}
> >> > +
> >> > +static const struct fpga_manager_ops fme_pr_ops = {
> >> > + .write_init = fme_pr_write_init,
> >> > + .write = fme_pr_write,
> >> > + .write_complete = fme_pr_write_complete,
> >> > + .state = fme_pr_state,
> >> > +};
> >> > +
> >> > +static int fme_pr(struct platform_device *pdev, unsigned long arg)
> >> > +{
> >> > + void __user *argp = (void __user *)arg;
> >> > + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> >> > + struct fpga_fme *fme;
> >> > + struct fpga_manager *mgr;
> >> > + struct feature_fme_header *fme_hdr;
> >> > + struct feature_fme_capability fme_capability;
> >> > + struct fpga_image_info info;
> >> > + struct fpga_fme_port_pr port_pr;
> >> > + struct platform_device *port;
> >> > + unsigned long minsz;
> >> > + void *buf = NULL;
> >> > + int ret = 0;
> >> > +
> >> > + minsz = offsetofend(struct fpga_fme_port_pr, status);
> >> > +
> >> > + if (copy_from_user(&port_pr, argp, minsz))
> >> > + return -EFAULT;
> >> > +
> >> > + if (port_pr.argsz < minsz || port_pr.flags)
> >> > + return -EINVAL;
> >> > +
> >> > + if (!IS_ALIGNED(port_pr.buffer_size, 4))
> >> > + return -EINVAL;
> >> > +
> >> > + /* get fme header region */
> >> > + fme_hdr = get_feature_ioaddr_by_index(&pdev->dev,
> >> > + FME_FEATURE_ID_HEADER);
> >> > + if (WARN_ON(!fme_hdr))
> >> > + return -EINVAL;
> >> > +
> >> > + /* check port id */
> >> > + fme_capability.csr = readq(&fme_hdr->capability);
> >> > + if (port_pr.port_id >= fme_capability.num_ports) {
> >> > + dev_dbg(&pdev->dev, "port number more than maximum\n");
> >> > + return -EINVAL;
> >> > + }
> >> > +
> >> > + if (!access_ok(VERIFY_READ, port_pr.buffer_address,
> >> > + port_pr.buffer_size))
> >> > + return -EFAULT;
> >> > +
> >> > + buf = vmalloc(port_pr.buffer_size);
> >> > + if (!buf)
> >> > + return -ENOMEM;
> >> > +
> >> > + if (copy_from_user(buf, (void __user *)port_pr.buffer_address,
> >> > + port_pr.buffer_size)) {
> >> > + ret = -EFAULT;
> >> > + goto free_exit;
> >> > + }
> >> > +
> >> > + memset(&info, 0, sizeof(struct fpga_image_info));
> >> > + info.flags = FPGA_MGR_PARTIAL_RECONFIG;
> >> > +
> >> > + mgr = fpga_mgr_get(&pdev->dev);
> >> > + if (IS_ERR(mgr)) {
> >> > + ret = PTR_ERR(mgr);
> >> > + goto free_exit;
> >> > + }
> >> > +
> >> > + mutex_lock(&pdata->lock);
> >> > + fme = fpga_pdata_get_private(pdata);
> >> > + /* fme device has been unregistered. */
> >> > + if (!fme) {
> >> > + ret = -EINVAL;
> >> > + goto unlock_exit;
> >> > + }
> >> > +
> >> > + fme->pr_err = 0;
> >> > + fme->port_id = port_pr.port_id;
> >>
> >> It looks like you're using private data to communicate with the
> >> driver, i.e. there is something you want to do with the fpga manager
> >> framework and it doesn't have that feature. The better way would be
> >> for us to expand the framework so you don't need to do that.
> >>
> >> port_id is the kind of thing that should be communicated to the driver
> >> through fpga_image_info, so we could add that to the struct. Should
> >> we call it port_id?
>
> Let's call it region_id.
>
> >> Or is there something more generic that may be
> >> useful in the future for other architectures?.
> >
> > Hi Alan
> >
> > Thanks for your feedback. :)
> >
> > As you know, each Intel FPGA device may have more than one accelerator,
> > and all accelerators share the same fpga_manager (only one FME module).
> > port_id = 0 means the first accelerator on this fpga devices. So it's
> > needed by the fpga_manager to distinguish one accelerator from others
> > for partial reconfiguration.
> >
> > Adding a member like a 'id' to fpga_image_info definitely works for us
> > in this case. We can add it this way, but I feel 'id' here seems not
> > related to fpga image, but characteristic of fpga region.
>
> The fpga_image_info struct started life as just image specific info,
> but I want it to go in the direction of including parameters needed to
> program it this specific time. Otherwise we are stuck having to keep
> adding parameters as our use of FPGA develops. It probably could be
> documented better as 'information needed to program a FPGA image'
> rather than strictly 'information about this particular FPGA image'.
> My patch "fpga-mgr: pass parameters for loading fpga in image info"
> goes in this direction by having the buf, firmware name, or sg list
> passed in the info for the added fpga_mgr_load() function. Actually I
> should probably simplify the API and get rid of fpga_mgr_buf_load,
> fpga_mgr_buf_load_sg, and fpga_mgr_firmware_load and require people to
> use fpga_mgr_load (passing all parameters in fpga_image_info).
>
Make sense.
> > It may be a
> > little confusing. One rough idea is that keep this info under fpga region
> > (maybe its private data), and pass the fpga-region to fpga_mgr_buf_load,
>
> Yes, keep this info in fpga-region. When the region wants to program
> using fpga-mgr, add the region id to fpga_image_info. I propose
> calling it region_id.
Hm.. Do we need a function which moves info from region to image info?
Another idea is, add a priv to fpga_image_info, and use a common function
to pass the fpga_region's priv to fpga_image_info's priv before PR.
fpga-mgr then knows fpga_region priv info from the fpga_image_info.
>
> > then fpga_manager knows the target region for partial reconfiguration.
> > If consider pr sysfs interface support under fpga-region in the future,
> > then we don't need to create a new 'id' sysfs file, as fpga-region itself
> > could provide this kind of info. But I'm not sure if this is the right
> > direction.
> >
> >>
> >> pr_err appears to be machine specific error codes that are
> >> communicated outside your low level driver. (Calling it pr_err is
> >> extra confusing since Linux already has a commonly name function by
> >> the same name). The framework has state, but that's not doing what
> >> you want here. Maybe we could add a framework ops called status so
> >> that status could be communicated from the low level driver. It would
> >> be useful to abstract the machine specific state to a defined enum
> >> that would be part of the fpga mgr framework. I remember we had
> >> something like that in the earliest version of fpga manager but it got
> >> changed to state rather than status for some reason.
> >>
> >
> > Adding a status function and a 'status' member to fpga manager sounds good.
> > But I'm not sure how to abstract these error codes, currently what we have
> > are all PR error codes for Intel FPGA device, e.g "PR FIFO overflow error",
> > "PR secure load error" and etc (see include/uapi/linux/intel-fpga.h). Is it
> > fine to let each driver to define how to use that 'status' for machine
> > specific status?
>
> I looked at the list of errors in include/uapi/linux/intel-fpga.h.
> They all seem pretty generic to me except I am not clear what "secure
> load error" or "IP protocol error" mean and whether other
> architectures would have them. But certainly things like crc error,
> incompatible bitstream, fifo overflow are generic. So let's see if we
> can define all these in a way that's generic and just pass up a error
> number. That way upper layers can know how to deal with them
> possibly. I would take the word "PR" off these since the error set
> applies whether someone is doing full reconfig or partial reconfig.
Sure, good to me, we can make it this way and see.
Thanks for the suggestions. :)
Hao
>
> >
> >> mgr->dev won't work for you for some of the things you are using fme->pdev for?
> >>
> >
> > Does 'fme->pdev' mean the platform device for fme? I think we use platform
> > device to represent the FME module, and FME module may have multiple sub
> > features, driver discovers these sub features via the 'Device Feature List'
> > in the PCIe Device Bar. PR is only one of the sub features under FME module,
> > if any FME module doesn't have PR sub feature, fpga-manager will not be
> > created at all. But this will not impact other sub features, e.g thermal
> > management, error reporting and etc, to create their own interfaces under
> > the platform device.
>
> If we need it for private data, that's may be actually OK. Just
> wondering whether the priv was needed. I think that's the one
> element in the private data struct that was used privately instead of
> being used to pass info outside the framework.
>
> >
> > Thanks
> > Hao
> >
> >> Alan
> >>
> >> > +
> >> > + /* Find and get port device by index */
> >> > + port = pdata->fpga_for_each_port(pdev, &fme->port_id,
> >> > + fpga_port_check_id);
> >> > + WARN_ON(!port);
> >> > +
> >> > + /* Disable Port before PR */
> >> > + fpga_port_disable(port);
> >> > +
> >> > + ret = fpga_mgr_buf_load(mgr, &info, buf, port_pr.buffer_size);
> >> > + port_pr.status = fme->pr_err;
> >> > +
> >> > + /* Re-enable Port after PR finished */
> >> > + fpga_port_enable(port);
> >> > +
> >> > + put_device(&port->dev);
> >> > +
> >> > +unlock_exit:
> >> > + mutex_unlock(&pdata->lock);
> >> > + fpga_mgr_put(mgr);
> >> > +free_exit:
> >> > + vfree(buf);
> >> > + if (copy_to_user((void __user *)arg, &port_pr, minsz))
> >> > + return -EFAULT;
> >> > + return ret;
> >> > +}
> >> > +
> >> > +static int fpga_fme_pr_probe(struct platform_device *pdev)
> >> > +{
> >> > + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> >> > + struct fpga_fme *priv;
> >> > + int ret;
> >> > +
> >> > + mutex_lock(&pdata->lock);
> >> > + priv = fpga_pdata_get_private(pdata);
> >> > + ret = fpga_mgr_register(&pdata->dev->dev,
> >> > + "Intel FPGA Manager", &fme_pr_ops, priv);
> >> > + mutex_unlock(&pdata->lock);
> >> > +
> >> > + return ret;
> >> > +}
> >> > +
> >> > +static int fpga_fme_pr_remove(struct platform_device *pdev)
> >> > +{
> >> > + fpga_mgr_unregister(&pdev->dev);
> >> > + return 0;
> >> > +}
> >> > +
> >> > +static int pr_mgmt_init(struct platform_device *pdev, struct feature *feature)
> >> > +{
> >> > + int ret;
> >> > +
> >> > + ret = fpga_fme_pr_probe(pdev);
> >> > + if (ret)
> >> > + return ret;
> >> > +
> >> > + ret = sysfs_create_group(&pdev->dev.kobj, &pr_mgmt_attr_group);
> >> > + if (ret)
> >> > + fpga_fme_pr_remove(pdev);
> >> > +
> >> > + return ret;
> >> > +}
> >> > +
> >> > +static void pr_mgmt_uinit(struct platform_device *pdev, struct feature *feature)
> >> > +{
> >> > + sysfs_remove_group(&pdev->dev.kobj, &pr_mgmt_attr_group);
> >> > + fpga_fme_pr_remove(pdev);
> >> > +}
> >> > +
> >> > +static long fme_pr_ioctl(struct platform_device *pdev, struct feature *feature,
> >> > + unsigned int cmd, unsigned long arg)
> >> > +{
> >> > + long ret;
> >> > +
> >> > + switch (cmd) {
> >> > + case FPGA_FME_PORT_PR:
> >> > + ret = fme_pr(pdev, arg);
> >> > + break;
> >> > + default:
> >> > + ret = -ENODEV;
> >> > + }
> >> > +
> >> > + return ret;
> >> > +}
> >> > +
> >> > +struct feature_ops pr_mgmt_ops = {
> >> > + .init = pr_mgmt_init,
> >> > + .uinit = pr_mgmt_uinit,
> >> > + .ioctl = fme_pr_ioctl,
> >> > +};
> >> > diff --git a/drivers/fpga/intel/fme.h b/drivers/fpga/intel/fme.h
> >> > new file mode 100644
> >> > index 0000000..d6cb7ce
> >> > --- /dev/null
> >> > +++ b/drivers/fpga/intel/fme.h
> >> > @@ -0,0 +1,32 @@
> >> > +/*
> >> > + * Header file for Intel FPGA Management Engine (FME) Driver
> >> > + *
> >> > + * Copyright (C) 2017 Intel Corporation, Inc.
> >> > + *
> >> > + * Authors:
> >> > + * Kang Luwei <luwei.kang@intel.com>
> >> > + * Xiao Guangrong <guangrong.xiao@linux.intel.com>
> >> > + * Joseph Grecco <joe.grecco@intel.com>
> >> > + * Enno Luebbers <enno.luebbers@intel.com>
> >> > + * Tim Whisonant <tim.whisonant@intel.com>
> >> > + * Ananda Ravuri <ananda.ravuri@intel.com>
> >> > + * Henry Mitchel <henry.mitchel@intel.com>
> >> > + *
> >> > + * This work is licensed under a dual BSD/GPLv2 license. When using or
> >> > + * redistributing this file, you may do so under either license. See the
> >> > + * LICENSE.BSD file under this directory for the BSD license and see
> >> > + * the COPYING file in the top-level directory for the GPLv2 license.
> >> > + */
> >> > +
> >> > +#ifndef __INTEL_FME_H
> >> > +#define __INTEL_FME_H
> >> > +
> >> > +struct fpga_fme {
> >> > + u8 port_id;
> >> > + u64 pr_err;
> >> > + struct feature_platform_data *pdata;
> >> > +};
> >> > +
> >> > +extern struct feature_ops pr_mgmt_ops;
> >> > +
> >> > +#endif
> >> > diff --git a/include/uapi/linux/intel-fpga.h b/include/uapi/linux/intel-fpga.h
> >> > index 992e556..77658316 100644
> >> > --- a/include/uapi/linux/intel-fpga.h
> >> > +++ b/include/uapi/linux/intel-fpga.h
> >> > @@ -18,6 +18,8 @@
> >> > #ifndef _UAPI_LINUX_INTEL_FPGA_H
> >> > #define _UAPI_LINUX_INTEL_FPGA_H
> >> >
> >> > +#include <linux/types.h>
> >> > +
> >> > #define FPGA_API_VERSION 0
> >> >
> >> > /*
> >> > @@ -30,6 +32,7 @@
> >> > #define FPGA_MAGIC 0xB6
> >> >
> >> > #define FPGA_BASE 0
> >> > +#define FME_BASE 0x80
> >> >
> >> > /**
> >> > * FPGA_GET_API_VERSION - _IO(FPGA_MAGIC, FPGA_BASE + 0)
> >> > @@ -49,4 +52,45 @@
> >> >
> >> > #define FPGA_CHECK_EXTENSION _IO(FPGA_MAGIC, FPGA_BASE + 1)
> >> >
> >> > +/* IOCTLs for FME file descriptor */
> >> > +
> >> > +/**
> >> > + * FPGA_FME_PORT_PR - _IOWR(FPGA_MAGIC, FME_BASE + 0, struct fpga_fme_port_pr)
> >> > + *
> >> > + * Driver does Partial Reconfiguration based on Port ID and Buffer (Image)
> >> > + * provided by caller.
> >> > + * Return: 0 on success, -errno on failure.
> >> > + * If FPGA_FME_PORT_PR returns -EIO, that indicates the HW has detected
> >> > + * some errors during PR, under this case, the user can fetch HW error code
> >> > + * from fpga_fme_port_pr.status. Each bit on the error code is used as the
> >> > + * index for the array created by DEFINE_FPGA_PR_ERR_MSG().
> >> > + * Otherwise, it is always zero.
> >> > + */
> >> > +
> >> > +#define DEFINE_FPGA_PR_ERR_MSG(_name_) \
> >> > +static const char * const _name_[] = { \
> >> > + "PR operation error detected", \
> >> > + "PR CRC error detected", \
> >> > + "PR incompatiable bitstream error detected", \
> >> > + "PR IP protocol error detected", \
> >> > + "PR FIFO overflow error detected", \
> >> > + "Reserved", \
> >> > + "PR secure load error detected", \
> >> > +}
> >> > +
> >> > +#define PR_MAX_ERR_NUM 7
> >> > +
> >> > +struct fpga_fme_port_pr {
> >> > + /* Input */
> >> > + __u32 argsz; /* Structure length */
> >> > + __u32 flags; /* Zero for now */
> >> > + __u32 port_id;
> >> > + __u32 buffer_size;
> >> > + __u64 buffer_address; /* Userspace address to the buffer for PR */
> >> > + /* Output */
> >> > + __u64 status; /* HW error code if ioctl returns -EIO */
> >> > +};
> >> > +
> >> > +#define FPGA_FME_PORT_PR _IO(FPGA_MAGIC, FME_BASE + 0)
> >> > +
> >> > #endif /* _UAPI_INTEL_FPGA_H */
> >> > --
> >> > 2.7.4
> >> >
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fpga" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[toc] | [prev] | [next] | [standalone]
| From | Alan Tull <atull@kernel.org> |
|---|---|
| Date | 2017-04-05 00:40 +0200 |
| Subject | Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support |
| Message-ID | <tsFhf-2yy-11@gated-at.bofh.it> |
| In reply to | #1615722 |
On Tue, Apr 4, 2017 at 1:05 AM, Wu Hao <hao.wu@intel.com> wrote:
> On Mon, Apr 03, 2017 at 11:30:55AM -0500, Alan Tull wrote:
>> On Sat, Apr 1, 2017 at 6:08 AM, Wu Hao <hao.wu@intel.com> wrote:
>> > On Fri, Mar 31, 2017 at 02:10:12PM -0500, Alan Tull wrote:
>> >> On Thu, Mar 30, 2017 at 7:08 AM, Wu Hao <hao.wu@intel.com> wrote:
>> >> > From: Kang Luwei <luwei.kang@intel.com>
>> >> >
>> >> > Partial Reconfiguration (PR) is the most important function for FME. It
>> >> > allows reconfiguration for given Port/Accelerated Function Unit (AFU).
>> >> >
>> >> > This patch adds support for PR sub feature. In this patch, it registers
>> >> > a fpga_mgr and implements fpga_manager_ops, and invoke fpga_mgr_buf_load
>> >> > for PR operation once PR request received via ioctl. Below user space
>> >> > interfaces are exposed by this sub feature.
>> >> >
>> >> > Sysfs interface:
>> >> > * /sys/class/fpga/<fpga.x>/<intel-fpga-fme.x>/interface_id
>> >> > Read-only. Indicate the hardware interface information. Userspace
>> >> > applications need to check this interface to select correct green
>> >> > bitstream format before PR.
>> >> >
>> >> > Ioctl interface:
>> >> > * FPGA_FME_PORT_PR
>> >> > Do partial reconfiguration per information from userspace, including
>> >> > target port(AFU), buffer size and address info. It returns the PR status
>> >> > (PR error code if failed) to userspace.
>> >> >
>> >> > Signed-off-by: Tim Whisonant <tim.whisonant@intel.com>
>> >> > Signed-off-by: Enno Luebbers <enno.luebbers@intel.com>
>> >> > Signed-off-by: Shiva Rao <shiva.rao@intel.com>
>> >> > Signed-off-by: Christopher Rauer <christopher.rauer@intel.com>
>> >> > Signed-off-by: Alan Tull <alan.tull@intel.com>
>> >> > Signed-off-by: Kang Luwei <luwei.kang@intel.com>
>> >> > Signed-off-by: Xiao Guangrong <guangrong.xiao@linux.intel.com>
>> >> > Signed-off-by: Wu Hao <hao.wu@intel.com>
>> >> > ---
>> >> > drivers/fpga/intel/Makefile | 2 +-
>> >> > drivers/fpga/intel/feature-dev.h | 58 ++++++
>> >> > drivers/fpga/intel/fme-main.c | 44 ++++-
>> >> > drivers/fpga/intel/fme-pr.c | 400 +++++++++++++++++++++++++++++++++++++++
>> >> > drivers/fpga/intel/fme.h | 32 ++++
>> >> > include/uapi/linux/intel-fpga.h | 44 +++++
>> >> > 6 files changed, 578 insertions(+), 2 deletions(-)
>> >> > create mode 100644 drivers/fpga/intel/fme-pr.c
>> >> > create mode 100644 drivers/fpga/intel/fme.h
>> >> >
>> >> > diff --git a/drivers/fpga/intel/Makefile b/drivers/fpga/intel/Makefile
>> >> > index 546861d..0452cb6 100644
>> >> > --- a/drivers/fpga/intel/Makefile
>> >> > +++ b/drivers/fpga/intel/Makefile
>> >> > @@ -2,4 +2,4 @@ obj-$(CONFIG_INTEL_FPGA_PCI) += intel-fpga-pci.o
>> >> > obj-$(CONFIG_INTEL_FPGA_FME) += intel-fpga-fme.o
>> >> >
>> >> > intel-fpga-pci-objs := pcie.o feature-dev.o
>> >> > -intel-fpga-fme-objs := fme-main.o
>> >> > +intel-fpga-fme-objs := fme-main.o fme-pr.o
>> >> > diff --git a/drivers/fpga/intel/feature-dev.h b/drivers/fpga/intel/feature-dev.h
>> >> > index dccc283..5a25c915 100644
>> >> > --- a/drivers/fpga/intel/feature-dev.h
>> >> > +++ b/drivers/fpga/intel/feature-dev.h
>> >> > @@ -150,8 +150,66 @@ struct feature_fme_err {
>> >> > };
>> >> >
>> >> > /* FME Partial Reconfiguration Sub Feature Register Set */
>> >> > +/* FME PR Control Register */
>> >> > +struct feature_fme_pr_ctl {
>> >> > + union {
>> >> > + u64 csr;
>> >> > + struct {
>> >> > + u8 pr_reset:1; /* Reset PR Engine */
>> >> > + u8 rsvdz1:3;
>> >> > + u8 pr_reset_ack:1; /* Reset PR Engine Ack */
>> >> > + u8 rsvdz2:3;
>> >> > + u8 pr_regionid:2; /* PR Region ID */
>> >> > + u8 rsvdz3:2;
>> >> > + u8 pr_start_req:1; /* PR Start Request */
>> >> > + u8 pr_push_complete:1; /* PR Data push complete */
>> >> > + u8 pr_kind:1; /* Load Customer or Intel GBS */
>> >> > + u32 rsvdz4:17;
>> >> > + u32 config_data;
>> >> > + };
>> >> > + };
>> >> > +};
>> >> > +
>> >> > +/* FME PR Status Register */
>> >> > +struct feature_fme_pr_status {
>> >> > + union {
>> >> > + u64 csr;
>> >> > + struct {
>> >> > + u16 pr_credit:9; /* Number of PR Credits */
>> >> > + u8 rsvdz1:7;
>> >> > + u8 pr_status:1; /* PR Operation status */
>> >> > + u8 rsvdz2:3;
>> >> > + u8 pr_ctrlr_status:3; /* Controller status */
>> >> > + u8 rsvdz3:1;
>> >> > + u8 pr_host_status:4; /* PR Host status */
>> >> > + u64 rsvdz4:36;
>> >> > + };
>> >> > + };
>> >> > +};
>> >> > +
>> >> > +/* FME PR Data Register */
>> >> > +struct feature_fme_pr_data {
>> >> > + union {
>> >> > + u64 csr;
>> >> > + struct {
>> >> > + /* PR data from the raw-binary file */
>> >> > + u32 pr_data_raw;
>> >> > + u32 rsvd;
>> >> > + };
>> >> > + };
>> >> > +};
>> >> > +
>> >> > struct feature_fme_pr {
>> >> > struct feature_header header;
>> >> > + struct feature_fme_pr_ctl control;
>> >> > + struct feature_fme_pr_status status;
>> >> > + struct feature_fme_pr_data data;
>> >> > + u64 error;
>> >> > +
>> >> > + u64 rsvd[16];
>> >> > +
>> >> > + u64 intfc_id_l; /* PR interface Id Low */
>> >> > + u64 intfc_id_h; /* PR interface Id High */
>> >> > };
>> >> >
>> >> > /* PORT Header Register Set */
>> >> > diff --git a/drivers/fpga/intel/fme-main.c b/drivers/fpga/intel/fme-main.c
>> >> > index 36d0c4c..0d9a7a6 100644
>> >> > --- a/drivers/fpga/intel/fme-main.c
>> >> > +++ b/drivers/fpga/intel/fme-main.c
>> >> > @@ -23,6 +23,7 @@
>> >> > #include <linux/intel-fpga.h>
>> >> >
>> >> > #include "feature-dev.h"
>> >> > +#include "fme.h"
>> >> >
>> >> > static ssize_t ports_num_show(struct device *dev,
>> >> > struct device_attribute *attr, char *buf)
>> >> > @@ -101,6 +102,10 @@ static struct feature_driver fme_feature_drvs[] = {
>> >> > .ops = &fme_hdr_ops,
>> >> > },
>> >> > {
>> >> > + .name = FME_FEATURE_PR_MGMT,
>> >> > + .ops = &pr_mgmt_ops,
>> >> > + },
>> >> > + {
>> >> > .ops = NULL,
>> >> > },
>> >> > };
>> >> > @@ -182,14 +187,48 @@ static const struct file_operations fme_fops = {
>> >> > .unlocked_ioctl = fme_ioctl,
>> >> > };
>> >> >
>> >> > +static int fme_dev_init(struct platform_device *pdev)
>> >> > +{
>> >> > + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
>> >> > + struct fpga_fme *fme;
>> >> > +
>> >> > + fme = devm_kzalloc(&pdev->dev, sizeof(*fme), GFP_KERNEL);
>> >> > + if (!fme)
>> >> > + return -ENOMEM;
>> >> > +
>> >> > + fme->pdata = pdata;
>> >> > +
>> >> > + mutex_lock(&pdata->lock);
>> >> > + fpga_pdata_set_private(pdata, fme);
>> >> > + mutex_unlock(&pdata->lock);
>> >> > + return 0;
>> >> > +}
>> >> > +
>> >> > +static void fme_dev_destroy(struct platform_device *pdev)
>> >> > +{
>> >> > + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
>> >> > + struct fpga_fme *fme;
>> >> > +
>> >> > + mutex_lock(&pdata->lock);
>> >> > + fme = fpga_pdata_get_private(pdata);
>> >> > + fpga_pdata_set_private(pdata, NULL);
>> >> > + mutex_unlock(&pdata->lock);
>> >> > +
>> >> > + devm_kfree(&pdev->dev, fme);
>> >> > +}
>> >> > +
>> >> > static int fme_probe(struct platform_device *pdev)
>> >> > {
>> >> > int ret;
>> >> >
>> >> > - ret = fpga_dev_feature_init(pdev, fme_feature_drvs);
>> >> > + ret = fme_dev_init(pdev);
>> >> > if (ret)
>> >> > goto exit;
>> >> >
>> >> > + ret = fpga_dev_feature_init(pdev, fme_feature_drvs);
>> >> > + if (ret)
>> >> > + goto dev_destroy;
>> >> > +
>> >> > ret = fpga_register_dev_ops(pdev, &fme_fops, THIS_MODULE);
>> >> > if (ret)
>> >> > goto feature_uinit;
>> >> > @@ -198,6 +237,8 @@ static int fme_probe(struct platform_device *pdev)
>> >> >
>> >> > feature_uinit:
>> >> > fpga_dev_feature_uinit(pdev);
>> >> > +dev_destroy:
>> >> > + fme_dev_destroy(pdev);
>> >> > exit:
>> >> > return ret;
>> >> > }
>> >> > @@ -206,6 +247,7 @@ static int fme_remove(struct platform_device *pdev)
>> >> > {
>> >> > fpga_dev_feature_uinit(pdev);
>> >> > fpga_unregister_dev_ops(pdev);
>> >> > + fme_dev_destroy(pdev);
>> >> > return 0;
>> >> > }
>> >> >
>> >> > diff --git a/drivers/fpga/intel/fme-pr.c b/drivers/fpga/intel/fme-pr.c
>> >> > new file mode 100644
>> >> > index 0000000..3b44a3e
>> >> > --- /dev/null
>> >> > +++ b/drivers/fpga/intel/fme-pr.c
>> >> > @@ -0,0 +1,400 @@
>> >> > +/*
>> >> > + * Driver for Intel FPGA Management Engine (FME) Partial Reconfiguration
>> >> > + *
>> >> > + * Copyright (C) 2017 Intel Corporation, Inc.
>> >> > + *
>> >> > + * Authors:
>> >> > + * Kang Luwei <luwei.kang@intel.com>
>> >> > + * Xiao Guangrong <guangrong.xiao@linux.intel.com>
>> >> > + * Joseph Grecco <joe.grecco@intel.com>
>> >> > + * Enno Luebbers <enno.luebbers@intel.com>
>> >> > + * Tim Whisonant <tim.whisonant@intel.com>
>> >> > + * Ananda Ravuri <ananda.ravuri@intel.com>
>> >> > + * Christopher Rauer <christopher.rauer@intel.com>
>> >> > + * Henry Mitchel <henry.mitchel@intel.com>
>> >> > + *
>> >> > + * This work is licensed under a dual BSD/GPLv2 license. When using or
>> >> > + * redistributing this file, you may do so under either license. See the
>> >> > + * LICENSE.BSD file under this directory for the BSD license and see
>> >> > + * the COPYING file in the top-level directory for the GPLv2 license.
>> >> > + */
>> >> > +
>> >> > +#include <linux/types.h>
>> >> > +#include <linux/device.h>
>> >> > +#include <linux/vmalloc.h>
>> >> > +#include <linux/uaccess.h>
>> >> > +#include <linux/fpga/fpga-mgr.h>
>> >> > +#include <linux/intel-fpga.h>
>> >> > +
>> >> > +#include "feature-dev.h"
>> >> > +#include "fme.h"
>> >> > +
>> >> > +#define PR_WAIT_TIMEOUT 8000000
>> >> > +
>> >> > +#define PR_HOST_STATUS_IDLE 0
>> >> > +
>> >> > +DEFINE_FPGA_PR_ERR_MSG(pr_err_msg);
>> >> > +
>> >> > +static ssize_t interface_id_show(struct device *dev,
>> >> > + struct device_attribute *attr, char *buf)
>> >> > +{
>> >> > + u64 intfc_id_l, intfc_id_h;
>> >> > + struct feature_fme_pr *fme_pr
>> >> > + = get_feature_ioaddr_by_index(dev, FME_FEATURE_ID_PR_MGMT);
>> >> > +
>> >> > + intfc_id_l = readq(&fme_pr->intfc_id_l);
>> >> > + intfc_id_h = readq(&fme_pr->intfc_id_h);
>> >> > +
>> >> > + return scnprintf(buf, PAGE_SIZE, "%016llx%016llx\n",
>> >> > + (unsigned long long)intfc_id_h,
>> >> > + (unsigned long long)intfc_id_l);
>> >> > +}
>> >> > +static DEVICE_ATTR_RO(interface_id);
>> >> > +
>> >> > +static struct attribute *pr_mgmt_attrs[] = {
>> >> > + &dev_attr_interface_id.attr,
>> >> > + NULL,
>> >> > +};
>> >> > +
>> >> > +struct attribute_group pr_mgmt_attr_group = {
>> >> > + .attrs = pr_mgmt_attrs,
>> >> > + .name = "pr",
>> >> > +};
>> >> > +
>> >> > +static u64
>> >> > +pr_err_handle(struct platform_device *pdev, struct feature_fme_pr *fme_pr)
>> >> > +{
>> >> > + struct feature_fme_pr_status fme_pr_status;
>> >> > + unsigned long err_code;
>> >> > + u64 fme_pr_error;
>> >> > + int i = 0;
>> >> > +
>> >> > + fme_pr_status.csr = readq(&fme_pr->status);
>> >> > + if (!fme_pr_status.pr_status)
>> >> > + return 0;
>> >> > +
>> >> > + err_code = fme_pr_error = readq(&fme_pr->error);
>> >> > + for_each_set_bit(i, &err_code, PR_MAX_ERR_NUM)
>> >> > + dev_dbg(&pdev->dev, "%s\n", pr_err_msg[i]);
>> >> > + writeq(fme_pr_error, &fme_pr->error);
>> >> > + return fme_pr_error;
>> >> > +}
>> >> > +
>> >> > +static int fme_pr_write_init(struct fpga_manager *mgr,
>> >> > + struct fpga_image_info *info, const char *buf, size_t count)
>> >> > +{
>> >> > + struct fpga_fme *fme = mgr->priv;
>> >> > + struct platform_device *pdev;
>> >> > + struct feature_fme_pr *fme_pr;
>> >> > + struct feature_fme_pr_ctl fme_pr_ctl;
>> >> > + struct feature_fme_pr_status fme_pr_status;
>> >> > +
>> >> > + pdev = fme->pdata->dev;
>> >> > + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
>> >> > + FME_FEATURE_ID_PR_MGMT);
>> >> > + if (!fme_pr)
>> >> > + return -EINVAL;
>> >> > +
>> >> > + if (WARN_ON(info->flags != FPGA_MGR_PARTIAL_RECONFIG))
>> >> > + return -EINVAL;
>> >> > +
>> >> > + dev_dbg(&pdev->dev, "resetting PR before initiated PR\n");
>> >> > +
>> >> > + fme_pr_ctl.csr = readq(&fme_pr->control);
>> >> > + fme_pr_ctl.pr_reset = 1;
>> >> > + writeq(fme_pr_ctl.csr, &fme_pr->control);
>> >> > +
>> >> > + fme_pr_ctl.pr_reset_ack = 1;
>> >> > +
>> >> > + if (fpga_wait_register_field(pr_reset_ack, fme_pr_ctl,
>> >> > + &fme_pr->control, PR_WAIT_TIMEOUT, 1)) {
>> >> > + dev_err(&pdev->dev, "maximum PR timeout\n");
>> >> > + return -ETIMEDOUT;
>> >> > + }
>> >> > +
>> >> > + fme_pr_ctl.csr = readq(&fme_pr->control);
>> >> > + fme_pr_ctl.pr_reset = 0;
>> >> > + writeq(fme_pr_ctl.csr, &fme_pr->control);
>> >> > +
>> >> > + dev_dbg(&pdev->dev,
>> >> > + "waiting for PR resource in HW to be initialized and ready\n");
>> >> > +
>> >> > + fme_pr_status.pr_host_status = PR_HOST_STATUS_IDLE;
>> >> > +
>> >> > + if (fpga_wait_register_field(pr_host_status, fme_pr_status,
>> >> > + &fme_pr->status, PR_WAIT_TIMEOUT, 1)) {
>> >> > + dev_err(&pdev->dev, "maximum PR timeout\n");
>> >> > + return -ETIMEDOUT;
>> >> > + }
>> >> > +
>> >> > + dev_dbg(&pdev->dev, "check if have any previous PR error\n");
>> >> > + pr_err_handle(pdev, fme_pr);
>> >> > + return 0;
>> >> > +}
>> >> > +
>> >> > +static int fme_pr_write(struct fpga_manager *mgr,
>> >> > + const char *buf, size_t count)
>> >> > +{
>> >> > + struct fpga_fme *fme = mgr->priv;
>> >> > + struct platform_device *pdev;
>> >> > + struct feature_fme_pr *fme_pr;
>> >> > + struct feature_fme_pr_ctl fme_pr_ctl;
>> >> > + struct feature_fme_pr_status fme_pr_status;
>> >> > + struct feature_fme_pr_data fme_pr_data;
>> >> > + int delay, pr_credit, i = 0;
>> >> > +
>> >> > + pdev = fme->pdata->dev;
>> >> > + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
>> >> > + FME_FEATURE_ID_PR_MGMT);
>> >> > +
>> >> > + dev_dbg(&pdev->dev, "set PR port ID and start request\n");
>> >> > +
>> >> > + fme_pr_ctl.csr = readq(&fme_pr->control);
>> >> > + fme_pr_ctl.pr_regionid = fme->port_id;
>> >> > + fme_pr_ctl.pr_start_req = 1;
>> >> > + writeq(fme_pr_ctl.csr, &fme_pr->control);
>> >> > +
>> >> > + dev_dbg(&pdev->dev, "pushing data from bitstream to HW\n");
>> >> > +
>> >> > + fme_pr_status.csr = readq(&fme_pr->status);
>> >> > + pr_credit = fme_pr_status.pr_credit;
>> >> > +
>> >> > + while (count > 0) {
>> >> > + delay = 0;
>> >> > + while (pr_credit <= 1) {
>> >> > + if (delay++ > PR_WAIT_TIMEOUT) {
>> >> > + dev_err(&pdev->dev, "maximum try\n");
>> >> > + return -ETIMEDOUT;
>> >> > + }
>> >> > + udelay(1);
>> >> > +
>> >> > + fme_pr_status.csr = readq(&fme_pr->status);
>> >> > + pr_credit = fme_pr_status.pr_credit;
>> >> > + };
>> >> > +
>> >> > + if (count >= 4) {
>> >> > + fme_pr_data.rsvd = 0;
>> >> > + fme_pr_data.pr_data_raw = *(((u32 *)buf) + i);
>> >> > + writeq(fme_pr_data.csr, &fme_pr->data);
>> >> > + count -= 4;
>> >> > + pr_credit--;
>> >> > + i++;
>> >> > + } else {
>> >> > + WARN_ON(1);
>> >> > + return -EINVAL;
>> >> > + }
>> >> > + }
>> >> > +
>> >> > + return 0;
>> >> > +}
>> >> > +
>> >> > +static int fme_pr_write_complete(struct fpga_manager *mgr,
>> >> > + struct fpga_image_info *info)
>> >> > +{
>> >> > + struct fpga_fme *fme = mgr->priv;
>> >> > + struct platform_device *pdev;
>> >> > + struct feature_fme_pr *fme_pr;
>> >> > + struct feature_fme_pr_ctl fme_pr_ctl;
>> >> > +
>> >> > + pdev = fme->pdata->dev;
>> >> > + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
>> >> > + FME_FEATURE_ID_PR_MGMT);
>> >> > +
>> >> > + fme_pr_ctl.csr = readq(&fme_pr->control);
>> >> > + fme_pr_ctl.pr_push_complete = 1;
>> >> > + writeq(fme_pr_ctl.csr, &fme_pr->control);
>> >> > +
>> >> > + dev_dbg(&pdev->dev, "green bitstream push complete\n");
>> >> > + dev_dbg(&pdev->dev, "waiting for HW to release PR resource\n");
>> >> > +
>> >> > + fme_pr_ctl.pr_start_req = 0;
>> >> > +
>> >> > + if (fpga_wait_register_field(pr_start_req, fme_pr_ctl,
>> >> > + &fme_pr->control, PR_WAIT_TIMEOUT, 1)) {
>> >> > + dev_err(&pdev->dev, "maximum try.\n");
>> >> > + return -ETIMEDOUT;
>> >> > + }
>> >> > +
>> >> > + dev_dbg(&pdev->dev, "PR operation complete, checking status\n");
>> >> > + fme->pr_err = pr_err_handle(pdev, fme_pr);
>> >> > + if (fme->pr_err)
>> >> > + return -EIO;
>> >> > +
>> >> > + dev_dbg(&pdev->dev, "PR done successfully\n");
>> >> > + return 0;
>> >> > +}
>> >> > +
>> >> > +static enum fpga_mgr_states fme_pr_state(struct fpga_manager *mgr)
>> >> > +{
>> >> > + return FPGA_MGR_STATE_UNKNOWN;
>> >> > +}
>> >> > +
>> >> > +static const struct fpga_manager_ops fme_pr_ops = {
>> >> > + .write_init = fme_pr_write_init,
>> >> > + .write = fme_pr_write,
>> >> > + .write_complete = fme_pr_write_complete,
>> >> > + .state = fme_pr_state,
>> >> > +};
>> >> > +
>> >> > +static int fme_pr(struct platform_device *pdev, unsigned long arg)
>> >> > +{
>> >> > + void __user *argp = (void __user *)arg;
>> >> > + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
>> >> > + struct fpga_fme *fme;
>> >> > + struct fpga_manager *mgr;
>> >> > + struct feature_fme_header *fme_hdr;
>> >> > + struct feature_fme_capability fme_capability;
>> >> > + struct fpga_image_info info;
>> >> > + struct fpga_fme_port_pr port_pr;
>> >> > + struct platform_device *port;
>> >> > + unsigned long minsz;
>> >> > + void *buf = NULL;
>> >> > + int ret = 0;
>> >> > +
>> >> > + minsz = offsetofend(struct fpga_fme_port_pr, status);
>> >> > +
>> >> > + if (copy_from_user(&port_pr, argp, minsz))
>> >> > + return -EFAULT;
>> >> > +
>> >> > + if (port_pr.argsz < minsz || port_pr.flags)
>> >> > + return -EINVAL;
>> >> > +
>> >> > + if (!IS_ALIGNED(port_pr.buffer_size, 4))
>> >> > + return -EINVAL;
>> >> > +
>> >> > + /* get fme header region */
>> >> > + fme_hdr = get_feature_ioaddr_by_index(&pdev->dev,
>> >> > + FME_FEATURE_ID_HEADER);
>> >> > + if (WARN_ON(!fme_hdr))
>> >> > + return -EINVAL;
>> >> > +
>> >> > + /* check port id */
>> >> > + fme_capability.csr = readq(&fme_hdr->capability);
>> >> > + if (port_pr.port_id >= fme_capability.num_ports) {
>> >> > + dev_dbg(&pdev->dev, "port number more than maximum\n");
>> >> > + return -EINVAL;
>> >> > + }
>> >> > +
>> >> > + if (!access_ok(VERIFY_READ, port_pr.buffer_address,
>> >> > + port_pr.buffer_size))
>> >> > + return -EFAULT;
>> >> > +
>> >> > + buf = vmalloc(port_pr.buffer_size);
>> >> > + if (!buf)
>> >> > + return -ENOMEM;
>> >> > +
>> >> > + if (copy_from_user(buf, (void __user *)port_pr.buffer_address,
>> >> > + port_pr.buffer_size)) {
>> >> > + ret = -EFAULT;
>> >> > + goto free_exit;
>> >> > + }
>> >> > +
>> >> > + memset(&info, 0, sizeof(struct fpga_image_info));
>> >> > + info.flags = FPGA_MGR_PARTIAL_RECONFIG;
>> >> > +
>> >> > + mgr = fpga_mgr_get(&pdev->dev);
>> >> > + if (IS_ERR(mgr)) {
>> >> > + ret = PTR_ERR(mgr);
>> >> > + goto free_exit;
>> >> > + }
>> >> > +
>> >> > + mutex_lock(&pdata->lock);
>> >> > + fme = fpga_pdata_get_private(pdata);
>> >> > + /* fme device has been unregistered. */
>> >> > + if (!fme) {
>> >> > + ret = -EINVAL;
>> >> > + goto unlock_exit;
>> >> > + }
>> >> > +
>> >> > + fme->pr_err = 0;
>> >> > + fme->port_id = port_pr.port_id;
>> >>
>> >> It looks like you're using private data to communicate with the
>> >> driver, i.e. there is something you want to do with the fpga manager
>> >> framework and it doesn't have that feature. The better way would be
>> >> for us to expand the framework so you don't need to do that.
>> >>
>> >> port_id is the kind of thing that should be communicated to the driver
>> >> through fpga_image_info, so we could add that to the struct. Should
>> >> we call it port_id?
>>
>> Let's call it region_id.
>>
>> >> Or is there something more generic that may be
>> >> useful in the future for other architectures?.
>> >
>> > Hi Alan
>> >
>> > Thanks for your feedback. :)
>> >
>> > As you know, each Intel FPGA device may have more than one accelerator,
>> > and all accelerators share the same fpga_manager (only one FME module).
>> > port_id = 0 means the first accelerator on this fpga devices. So it's
>> > needed by the fpga_manager to distinguish one accelerator from others
>> > for partial reconfiguration.
>> >
>> > Adding a member like a 'id' to fpga_image_info definitely works for us
>> > in this case. We can add it this way, but I feel 'id' here seems not
>> > related to fpga image, but characteristic of fpga region.
>>
>> The fpga_image_info struct started life as just image specific info,
>> but I want it to go in the direction of including parameters needed to
>> program it this specific time. Otherwise we are stuck having to keep
>> adding parameters as our use of FPGA develops. It probably could be
>> documented better as 'information needed to program a FPGA image'
>> rather than strictly 'information about this particular FPGA image'.
>> My patch "fpga-mgr: pass parameters for loading fpga in image info"
>> goes in this direction by having the buf, firmware name, or sg list
>> passed in the info for the added fpga_mgr_load() function. Actually I
>> should probably simplify the API and get rid of fpga_mgr_buf_load,
>> fpga_mgr_buf_load_sg, and fpga_mgr_firmware_load and require people to
>> use fpga_mgr_load (passing all parameters in fpga_image_info).
>>
>
> Make sense.
>
>> > It may be a
>> > little confusing. One rough idea is that keep this info under fpga region
>> > (maybe its private data), and pass the fpga-region to fpga_mgr_buf_load,
>>
>> Yes, keep this info in fpga-region. When the region wants to program
>> using fpga-mgr, add the region id to fpga_image_info. I propose
>> calling it region_id.
>
> Hm.. Do we need a function which moves info from region to image info?
No, just code that sets that variable in the struct before calling the
fpga_region_program_fpga function.
>
> Another idea is, add a priv to fpga_image_info, and use a common function
> to pass the fpga_region's priv to fpga_image_info's priv before PR.
> fpga-mgr then knows fpga_region priv info from the fpga_image_info.
>
Adding priv would make the interface for fpga-mgr non-uniform. The point
of having a fpga-mgr framework is that there
is the potential of the upper layers working for different FPGA devices.
If the interface for each FPGA device were different, that would then
be broken.
>>
>> > then fpga_manager knows the target region for partial reconfiguration.
>> > If consider pr sysfs interface support under fpga-region in the future,
>> > then we don't need to create a new 'id' sysfs file, as fpga-region itself
>> > could provide this kind of info. But I'm not sure if this is the right
>> > direction.
>> >
>> >>
>> >> pr_err appears to be machine specific error codes that are
>> >> communicated outside your low level driver. (Calling it pr_err is
>> >> extra confusing since Linux already has a commonly name function by
>> >> the same name). The framework has state, but that's not doing what
>> >> you want here. Maybe we could add a framework ops called status so
>> >> that status could be communicated from the low level driver. It would
>> >> be useful to abstract the machine specific state to a defined enum
>> >> that would be part of the fpga mgr framework. I remember we had
>> >> something like that in the earliest version of fpga manager but it got
>> >> changed to state rather than status for some reason.
>> >>
>> >
>> > Adding a status function and a 'status' member to fpga manager sounds good.
>> > But I'm not sure how to abstract these error codes, currently what we have
>> > are all PR error codes for Intel FPGA device, e.g "PR FIFO overflow error",
>> > "PR secure load error" and etc (see include/uapi/linux/intel-fpga.h). Is it
>> > fine to let each driver to define how to use that 'status' for machine
>> > specific status?
>>
>> I looked at the list of errors in include/uapi/linux/intel-fpga.h.
>> They all seem pretty generic to me except I am not clear what "secure
>> load error" or "IP protocol error" mean and whether other
>> architectures would have them. But certainly things like crc error,
>> incompatible bitstream, fifo overflow are generic. So let's see if we
>> can define all these in a way that's generic and just pass up a error
>> number. That way upper layers can know how to deal with them
>> possibly. I would take the word "PR" off these since the error set
>> applies whether someone is doing full reconfig or partial reconfig.
>
> Sure, good to me, we can make it this way and see.
> Thanks for the suggestions. :)
>
> Hao
[toc] | [prev] | [next] | [standalone]
| From | "Wu, Hao" <hao.wu@intel.com> |
|---|---|
| Date | 2017-04-05 13:50 +0200 |
| Subject | RE: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support |
| Message-ID | <tsRBL-25z-23@gated-at.bofh.it> |
| In reply to | #1616431 |
> >> The fpga_image_info struct started life as just image specific info, > >> but I want it to go in the direction of including parameters needed to > >> program it this specific time. Otherwise we are stuck having to keep > >> adding parameters as our use of FPGA develops. It probably could be > >> documented better as 'information needed to program a FPGA image' > >> rather than strictly 'information about this particular FPGA image'. > >> My patch "fpga-mgr: pass parameters for loading fpga in image info" > >> goes in this direction by having the buf, firmware name, or sg list > >> passed in the info for the added fpga_mgr_load() function. Actually I > >> should probably simplify the API and get rid of fpga_mgr_buf_load, > >> fpga_mgr_buf_load_sg, and fpga_mgr_firmware_load and require people to > >> use fpga_mgr_load (passing all parameters in fpga_image_info). > >> > > > > Make sense. > > > >> > It may be a > >> > little confusing. One rough idea is that keep this info under fpga region > >> > (maybe its private data), and pass the fpga-region to fpga_mgr_buf_load, > >> > >> Yes, keep this info in fpga-region. When the region wants to program > >> using fpga-mgr, add the region id to fpga_image_info. I propose > >> calling it region_id. > > > > Hm.. Do we need a function which moves info from region to image info? > > No, just code that sets that variable in the struct before calling the > fpga_region_program_fpga function. > > > > > Another idea is, add a priv to fpga_image_info, and use a common function > > to pass the fpga_region's priv to fpga_image_info's priv before PR. > > fpga-mgr then knows fpga_region priv info from the fpga_image_info. > > > > Adding priv would make the interface for fpga-mgr non-uniform. The point > of having a fpga-mgr framework is that there > is the potential of the upper layers working for different FPGA devices. > If the interface for each FPGA device were different, that would then > be broken. > I mean drivers can register their own fpga-mgr ops, and handle priv of fpga_image_info in driver specific way for pr (e.g write_init function). We don't need to change the any upper layer interfaces. If you prefer the region_id for fpga_image_info, we can go with region_id for sure. : ) Thanks Hao
[toc] | [prev] | [next] | [standalone]
| From | Alan Tull <atull@kernel.org> |
|---|---|
| Date | 2017-04-05 17:30 +0200 |
| Subject | Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support |
| Message-ID | <tsV2G-4nq-21@gated-at.bofh.it> |
| In reply to | #1616869 |
On Wed, Apr 5, 2017 at 6:40 AM, Wu, Hao <hao.wu@intel.com> wrote: >> >> The fpga_image_info struct started life as just image specific info, >> >> but I want it to go in the direction of including parameters needed to >> >> program it this specific time. Otherwise we are stuck having to keep >> >> adding parameters as our use of FPGA develops. It probably could be >> >> documented better as 'information needed to program a FPGA image' >> >> rather than strictly 'information about this particular FPGA image'. >> >> My patch "fpga-mgr: pass parameters for loading fpga in image info" >> >> goes in this direction by having the buf, firmware name, or sg list >> >> passed in the info for the added fpga_mgr_load() function. Actually I >> >> should probably simplify the API and get rid of fpga_mgr_buf_load, >> >> fpga_mgr_buf_load_sg, and fpga_mgr_firmware_load and require people to >> >> use fpga_mgr_load (passing all parameters in fpga_image_info). >> >> >> > >> > Make sense. >> > >> >> > It may be a >> >> > little confusing. One rough idea is that keep this info under fpga region >> >> > (maybe its private data), and pass the fpga-region to fpga_mgr_buf_load, >> >> >> >> Yes, keep this info in fpga-region. When the region wants to program >> >> using fpga-mgr, add the region id to fpga_image_info. I propose >> >> calling it region_id. >> > >> > Hm.. Do we need a function which moves info from region to image info? >> >> No, just code that sets that variable in the struct before calling the >> fpga_region_program_fpga function. >> >> > >> > Another idea is, add a priv to fpga_image_info, and use a common function >> > to pass the fpga_region's priv to fpga_image_info's priv before PR. >> > fpga-mgr then knows fpga_region priv info from the fpga_image_info. >> > >> >> Adding priv would make the interface for fpga-mgr non-uniform. The point >> of having a fpga-mgr framework is that there >> is the potential of the upper layers working for different FPGA devices. >> If the interface for each FPGA device were different, that would then >> be broken. >> > > I mean drivers can register their own fpga-mgr ops, and handle priv of > fpga_image_info in driver specific way for pr (e.g write_init function). > We don't need to change the any upper layer interfaces. I'm trying to avoid driver specific ways of doing things. Think of this all as a set of blocks and we want to be able to switch out individual blocks in the future. It's future-proofing and also making code more generally usable. fpga_mgr_info is part of the interface for calls to fpga-mgr to do reprogramming. My patchset will push it further in that direction as pointers to the image are added to fpga_mgr_info. Adding 'priv' to fpga_mgr_info makes that interface specific to a this driver. It's better to add a region_id variable to fpga_mgr_info that may not be used by all fpga-mgr drivers. The current model is that a fpga-mgr driver checks fpga_mgr_info flags to see if its correct. The fpga-mgr driver can check any other needed fpga_mgr_info variables and return error if the params look invalid. And ignore any it doesn't need. I don't think priv belongs in fpga_image_info. priv tends to be information for a specific instance of a driver that can have several instances. priv usually stores a driver's memory mappings, interrupts, etc for that instance. It's called private info as it is info that other blocks don't need to know and don't get to look at. It's private. So priv as in interface strikes me as not private any more and is sort of a red flag. Alan > > If you prefer the region_id for fpga_image_info, we can go with region_id > for sure. : ) > > Thanks > Hao
[toc] | [prev] | [next] | [standalone]
| From | Alan Tull <atull@kernel.org> |
|---|---|
| Date | 2017-04-05 17:50 +0200 |
| Subject | Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support |
| Message-ID | <tsVm2-4tM-5@gated-at.bofh.it> |
| In reply to | #1617050 |
On Wed, Apr 5, 2017 at 10:26 AM, Alan Tull <atull@kernel.org> wrote: > On Wed, Apr 5, 2017 at 6:40 AM, Wu, Hao <hao.wu@intel.com> wrote: >>> >> The fpga_image_info struct started life as just image specific info, >>> >> but I want it to go in the direction of including parameters needed to >>> >> program it this specific time. Otherwise we are stuck having to keep >>> >> adding parameters as our use of FPGA develops. It probably could be >>> >> documented better as 'information needed to program a FPGA image' >>> >> rather than strictly 'information about this particular FPGA image'. >>> >> My patch "fpga-mgr: pass parameters for loading fpga in image info" >>> >> goes in this direction by having the buf, firmware name, or sg list >>> >> passed in the info for the added fpga_mgr_load() function. Actually I >>> >> should probably simplify the API and get rid of fpga_mgr_buf_load, >>> >> fpga_mgr_buf_load_sg, and fpga_mgr_firmware_load and require people to >>> >> use fpga_mgr_load (passing all parameters in fpga_image_info). >>> >> >>> > >>> > Make sense. >>> > >>> >> > It may be a >>> >> > little confusing. One rough idea is that keep this info under fpga region >>> >> > (maybe its private data), and pass the fpga-region to fpga_mgr_buf_load, >>> >> >>> >> Yes, keep this info in fpga-region. When the region wants to program >>> >> using fpga-mgr, add the region id to fpga_image_info. I propose >>> >> calling it region_id. >>> > >>> > Hm.. Do we need a function which moves info from region to image info? >>> >>> No, just code that sets that variable in the struct before calling the >>> fpga_region_program_fpga function. >>> >>> > >>> > Another idea is, add a priv to fpga_image_info, and use a common function >>> > to pass the fpga_region's priv to fpga_image_info's priv before PR. >>> > fpga-mgr then knows fpga_region priv info from the fpga_image_info. >>> > >>> >>> Adding priv would make the interface for fpga-mgr non-uniform. The point >>> of having a fpga-mgr framework is that there >>> is the potential of the upper layers working for different FPGA devices. >>> If the interface for each FPGA device were different, that would then >>> be broken. >>> >> >> I mean drivers can register their own fpga-mgr ops, and handle priv of >> fpga_image_info in driver specific way for pr (e.g write_init function). >> We don't need to change the any upper layer interfaces. > > I'm trying to avoid driver specific ways of doing things. Think of this > all as a set of blocks and we want to be able to switch out individual > blocks in the future. It's future-proofing and also making code more > generally usable. > > fpga_mgr_info is part of the interface for calls to fpga-mgr to do > reprogramming. My patchset will push it further in that direction > as pointers to the image are added to fpga_mgr_info. > > Adding 'priv' to fpga_mgr_info makes that interface specific to a this driver. > It's better to add a region_id variable to fpga_mgr_info that may not be used by > all fpga-mgr drivers. The current model is that a fpga-mgr driver checks > fpga_mgr_info flags to see if its correct. The fpga-mgr driver can check > any other needed fpga_mgr_info variables and return error if the params > look invalid. And ignore any it doesn't need. > > I don't think priv belongs in fpga_image_info. priv tends to be information > for a specific instance of a driver that can have several instances. priv > usually stores a driver's memory mappings, interrupts, etc for that instance. > It's called private info as it is info that other blocks don't need to know > and don't get to look at. It's private. So priv as in interface strikes me as > not private any more and is sort of a red flag. > > Alan Besides that, I see that region_id is needed both for the fme_pr and for determining which port to disable/enable. So adding it to the fpga_image_info would allow the fpga-region to figure out which bridge to control as well as pass it to the mgr. > >> >> If you prefer the region_id for fpga_image_info, we can go with region_id >> for sure. : ) >> >> Thanks >> Hao
[toc] | [prev] | [next] | [standalone]
| From | Wu Hao <hao.wu@intel.com> |
|---|---|
| Date | 2017-04-06 13:10 +0200 |
| Subject | Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support |
| Message-ID | <ttdsB-7HU-1@gated-at.bofh.it> |
| In reply to | #1617074 |
On Wed, Apr 05, 2017 at 10:39:05AM -0500, Alan Tull wrote: > On Wed, Apr 5, 2017 at 10:26 AM, Alan Tull <atull@kernel.org> wrote: > > On Wed, Apr 5, 2017 at 6:40 AM, Wu, Hao <hao.wu@intel.com> wrote: > >>> >> The fpga_image_info struct started life as just image specific info, > >>> >> but I want it to go in the direction of including parameters needed to > >>> >> program it this specific time. Otherwise we are stuck having to keep > >>> >> adding parameters as our use of FPGA develops. It probably could be > >>> >> documented better as 'information needed to program a FPGA image' > >>> >> rather than strictly 'information about this particular FPGA image'. > >>> >> My patch "fpga-mgr: pass parameters for loading fpga in image info" > >>> >> goes in this direction by having the buf, firmware name, or sg list > >>> >> passed in the info for the added fpga_mgr_load() function. Actually I > >>> >> should probably simplify the API and get rid of fpga_mgr_buf_load, > >>> >> fpga_mgr_buf_load_sg, and fpga_mgr_firmware_load and require people to > >>> >> use fpga_mgr_load (passing all parameters in fpga_image_info). > >>> >> > >>> > > >>> > Make sense. > >>> > > >>> >> > It may be a > >>> >> > little confusing. One rough idea is that keep this info under fpga region > >>> >> > (maybe its private data), and pass the fpga-region to fpga_mgr_buf_load, > >>> >> > >>> >> Yes, keep this info in fpga-region. When the region wants to program > >>> >> using fpga-mgr, add the region id to fpga_image_info. I propose > >>> >> calling it region_id. > >>> > > >>> > Hm.. Do we need a function which moves info from region to image info? > >>> > >>> No, just code that sets that variable in the struct before calling the > >>> fpga_region_program_fpga function. > >>> > >>> > > >>> > Another idea is, add a priv to fpga_image_info, and use a common function > >>> > to pass the fpga_region's priv to fpga_image_info's priv before PR. > >>> > fpga-mgr then knows fpga_region priv info from the fpga_image_info. > >>> > > >>> > >>> Adding priv would make the interface for fpga-mgr non-uniform. The point > >>> of having a fpga-mgr framework is that there > >>> is the potential of the upper layers working for different FPGA devices. > >>> If the interface for each FPGA device were different, that would then > >>> be broken. > >>> > >> > >> I mean drivers can register their own fpga-mgr ops, and handle priv of > >> fpga_image_info in driver specific way for pr (e.g write_init function). > >> We don't need to change the any upper layer interfaces. > > > > I'm trying to avoid driver specific ways of doing things. Think of this > > all as a set of blocks and we want to be able to switch out individual > > blocks in the future. It's future-proofing and also making code more > > generally usable. > > > > fpga_mgr_info is part of the interface for calls to fpga-mgr to do > > reprogramming. My patchset will push it further in that direction > > as pointers to the image are added to fpga_mgr_info. > > > > Adding 'priv' to fpga_mgr_info makes that interface specific to a this driver. > > It's better to add a region_id variable to fpga_mgr_info that may not be used by > > all fpga-mgr drivers. The current model is that a fpga-mgr driver checks > > fpga_mgr_info flags to see if its correct. The fpga-mgr driver can check > > any other needed fpga_mgr_info variables and return error if the params > > look invalid. And ignore any it doesn't need. > > > > I don't think priv belongs in fpga_image_info. priv tends to be information > > for a specific instance of a driver that can have several instances. priv > > usually stores a driver's memory mappings, interrupts, etc for that instance. > > It's called private info as it is info that other blocks don't need to know > > and don't get to look at. It's private. So priv as in interface strikes me as > > not private any more and is sort of a red flag. Thanks a lot for the details. It's more clear to me now. :) > > > > Alan > > Besides that, I see that region_id is needed both for the fme_pr and > for determining which port to disable/enable. So adding it to the > fpga_image_info would allow the fpga-region to figure out which > bridge to control as well as pass it to the mgr. > Do we need to add an 'id' to fpga-bridge too? Per my understanding this FME driver should create fpga-region for each accelerator, and make sure each fpga-region has correct region_id. When this fpga-region wants to be programmed by fpga-mgr, then it add this info to fpga_image_info, and fpga-mgr knows which region to program via this fpga_image_info. For each fpga-region, FME driver needs to create one fpga-bridge and link it to region's bridge_list. When fpga_region_program_fpga is invoked for PR. This fpga-bridge could be disabled before PR and re-enabled after PR automatically. If fpga-bridge contains this 'id' information, then driver knows which port to enable/disable to implement of the enable_set function under this fpga-bridge. Thanks Hao > > > >> > >> If you prefer the region_id for fpga_image_info, we can go with region_id > >> for sure. : ) > >> > >> Thanks > >> Hao
[toc] | [prev] | [next] | [standalone]
| From | Alan Tull <atull@kernel.org> |
|---|---|
| Date | 2017-04-06 21:30 +0200 |
| Subject | Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support |
| Message-ID | <ttlgu-54H-23@gated-at.bofh.it> |
| In reply to | #1617876 |
On Thu, Apr 6, 2017 at 5:57 AM, Wu Hao <hao.wu@intel.com> wrote: > On Wed, Apr 05, 2017 at 10:39:05AM -0500, Alan Tull wrote: >> On Wed, Apr 5, 2017 at 10:26 AM, Alan Tull <atull@kernel.org> wrote: >> > On Wed, Apr 5, 2017 at 6:40 AM, Wu, Hao <hao.wu@intel.com> wrote: >> >>> >> The fpga_image_info struct started life as just image specific info, >> >>> >> but I want it to go in the direction of including parameters needed to >> >>> >> program it this specific time. Otherwise we are stuck having to keep >> >>> >> adding parameters as our use of FPGA develops. It probably could be >> >>> >> documented better as 'information needed to program a FPGA image' >> >>> >> rather than strictly 'information about this particular FPGA image'. >> >>> >> My patch "fpga-mgr: pass parameters for loading fpga in image info" >> >>> >> goes in this direction by having the buf, firmware name, or sg list >> >>> >> passed in the info for the added fpga_mgr_load() function. Actually I >> >>> >> should probably simplify the API and get rid of fpga_mgr_buf_load, >> >>> >> fpga_mgr_buf_load_sg, and fpga_mgr_firmware_load and require people to >> >>> >> use fpga_mgr_load (passing all parameters in fpga_image_info). >> >>> >> >> >>> > >> >>> > Make sense. >> >>> > >> >>> >> > It may be a >> >>> >> > little confusing. One rough idea is that keep this info under fpga region >> >>> >> > (maybe its private data), and pass the fpga-region to fpga_mgr_buf_load, >> >>> >> >> >>> >> Yes, keep this info in fpga-region. When the region wants to program >> >>> >> using fpga-mgr, add the region id to fpga_image_info. I propose >> >>> >> calling it region_id. >> >>> > >> >>> > Hm.. Do we need a function which moves info from region to image info? >> >>> >> >>> No, just code that sets that variable in the struct before calling the >> >>> fpga_region_program_fpga function. >> >>> >> >>> > >> >>> > Another idea is, add a priv to fpga_image_info, and use a common function >> >>> > to pass the fpga_region's priv to fpga_image_info's priv before PR. >> >>> > fpga-mgr then knows fpga_region priv info from the fpga_image_info. >> >>> > >> >>> >> >>> Adding priv would make the interface for fpga-mgr non-uniform. The point >> >>> of having a fpga-mgr framework is that there >> >>> is the potential of the upper layers working for different FPGA devices. >> >>> If the interface for each FPGA device were different, that would then >> >>> be broken. >> >>> >> >> >> >> I mean drivers can register their own fpga-mgr ops, and handle priv of >> >> fpga_image_info in driver specific way for pr (e.g write_init function). >> >> We don't need to change the any upper layer interfaces. >> > >> > I'm trying to avoid driver specific ways of doing things. Think of this >> > all as a set of blocks and we want to be able to switch out individual >> > blocks in the future. It's future-proofing and also making code more >> > generally usable. >> > >> > fpga_mgr_info is part of the interface for calls to fpga-mgr to do >> > reprogramming. My patchset will push it further in that direction >> > as pointers to the image are added to fpga_mgr_info. >> > >> > Adding 'priv' to fpga_mgr_info makes that interface specific to a this driver. >> > It's better to add a region_id variable to fpga_mgr_info that may not be used by >> > all fpga-mgr drivers. The current model is that a fpga-mgr driver checks >> > fpga_mgr_info flags to see if its correct. The fpga-mgr driver can check >> > any other needed fpga_mgr_info variables and return error if the params >> > look invalid. And ignore any it doesn't need. >> > >> > I don't think priv belongs in fpga_image_info. priv tends to be information >> > for a specific instance of a driver that can have several instances. priv >> > usually stores a driver's memory mappings, interrupts, etc for that instance. >> > It's called private info as it is info that other blocks don't need to know >> > and don't get to look at. It's private. So priv as in interface strikes me as >> > not private any more and is sort of a red flag. > > Thanks a lot for the details. It's more clear to me now. :) > >> > >> > Alan >> >> Besides that, I see that region_id is needed both for the fme_pr and >> for determining which port to disable/enable. So adding it to the >> fpga_image_info would allow the fpga-region to figure out which >> bridge to control as well as pass it to the mgr. >> > > Do we need to add an 'id' to fpga-bridge too? You don't have to, but you could. See below. > > Per my understanding this FME driver should create fpga-region for each > accelerator, and make sure each fpga-region has correct region_id. When > this fpga-region wants to be programmed by fpga-mgr, then it add this > info to fpga_image_info, and fpga-mgr knows which region to program via > this fpga_image_info. The fpga_image_info is passed both to fpga-mgr and fpga-bridge. So both will get the region_id through the info. > > For each fpga-region, FME driver needs to create one fpga-bridge and link > it to region's bridge_list. When fpga_region_program_fpga is invoked for > PR. This fpga-bridge could be disabled before PR and re-enabled after PR > automatically. If fpga-bridge contains this 'id' information, then driver > knows which port to enable/disable to implement of the enable_set function > under this fpga-bridge. Could implement it either way: the bridge could either use the bridge_id that has been passed to it in the info or code that creates the bridge could create private data and pass it to fpga_bridge_register(). But the bridge will have the id because it will be passed in the info, so it's not really needed. Alan > > Thanks > Hao > >> > >> >> >> >> If you prefer the region_id for fpga_image_info, we can go with region_id >> >> for sure. : ) >> >> >> >> Thanks >> >> Hao
[toc] | [prev] | [next] | [standalone]
| From | Wu Hao <hao.wu@intel.com> |
|---|---|
| Date | 2017-04-07 08:10 +0200 |
| Subject | Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support |
| Message-ID | <ttvfQ-3kY-11@gated-at.bofh.it> |
| In reply to | #1618299 |
On Thu, Apr 06, 2017 at 02:27:42PM -0500, Alan Tull wrote: > On Thu, Apr 6, 2017 at 5:57 AM, Wu Hao <hao.wu@intel.com> wrote: > > On Wed, Apr 05, 2017 at 10:39:05AM -0500, Alan Tull wrote: > >> On Wed, Apr 5, 2017 at 10:26 AM, Alan Tull <atull@kernel.org> wrote: > >> > On Wed, Apr 5, 2017 at 6:40 AM, Wu, Hao <hao.wu@intel.com> wrote: > >> >>> >> The fpga_image_info struct started life as just image specific info, > >> >>> >> but I want it to go in the direction of including parameters needed to > >> >>> >> program it this specific time. Otherwise we are stuck having to keep > >> >>> >> adding parameters as our use of FPGA develops. It probably could be > >> >>> >> documented better as 'information needed to program a FPGA image' > >> >>> >> rather than strictly 'information about this particular FPGA image'. > >> >>> >> My patch "fpga-mgr: pass parameters for loading fpga in image info" > >> >>> >> goes in this direction by having the buf, firmware name, or sg list > >> >>> >> passed in the info for the added fpga_mgr_load() function. Actually I > >> >>> >> should probably simplify the API and get rid of fpga_mgr_buf_load, > >> >>> >> fpga_mgr_buf_load_sg, and fpga_mgr_firmware_load and require people to > >> >>> >> use fpga_mgr_load (passing all parameters in fpga_image_info). > >> >>> >> > >> >>> > > >> >>> > Make sense. > >> >>> > > >> >>> >> > It may be a > >> >>> >> > little confusing. One rough idea is that keep this info under fpga region > >> >>> >> > (maybe its private data), and pass the fpga-region to fpga_mgr_buf_load, > >> >>> >> > >> >>> >> Yes, keep this info in fpga-region. When the region wants to program > >> >>> >> using fpga-mgr, add the region id to fpga_image_info. I propose > >> >>> >> calling it region_id. > >> >>> > > >> >>> > Hm.. Do we need a function which moves info from region to image info? > >> >>> > >> >>> No, just code that sets that variable in the struct before calling the > >> >>> fpga_region_program_fpga function. > >> >>> > >> >>> > > >> >>> > Another idea is, add a priv to fpga_image_info, and use a common function > >> >>> > to pass the fpga_region's priv to fpga_image_info's priv before PR. > >> >>> > fpga-mgr then knows fpga_region priv info from the fpga_image_info. > >> >>> > > >> >>> > >> >>> Adding priv would make the interface for fpga-mgr non-uniform. The point > >> >>> of having a fpga-mgr framework is that there > >> >>> is the potential of the upper layers working for different FPGA devices. > >> >>> If the interface for each FPGA device were different, that would then > >> >>> be broken. > >> >>> > >> >> > >> >> I mean drivers can register their own fpga-mgr ops, and handle priv of > >> >> fpga_image_info in driver specific way for pr (e.g write_init function). > >> >> We don't need to change the any upper layer interfaces. > >> > > >> > I'm trying to avoid driver specific ways of doing things. Think of this > >> > all as a set of blocks and we want to be able to switch out individual > >> > blocks in the future. It's future-proofing and also making code more > >> > generally usable. > >> > > >> > fpga_mgr_info is part of the interface for calls to fpga-mgr to do > >> > reprogramming. My patchset will push it further in that direction > >> > as pointers to the image are added to fpga_mgr_info. > >> > > >> > Adding 'priv' to fpga_mgr_info makes that interface specific to a this driver. > >> > It's better to add a region_id variable to fpga_mgr_info that may not be used by > >> > all fpga-mgr drivers. The current model is that a fpga-mgr driver checks > >> > fpga_mgr_info flags to see if its correct. The fpga-mgr driver can check > >> > any other needed fpga_mgr_info variables and return error if the params > >> > look invalid. And ignore any it doesn't need. > >> > > >> > I don't think priv belongs in fpga_image_info. priv tends to be information > >> > for a specific instance of a driver that can have several instances. priv > >> > usually stores a driver's memory mappings, interrupts, etc for that instance. > >> > It's called private info as it is info that other blocks don't need to know > >> > and don't get to look at. It's private. So priv as in interface strikes me as > >> > not private any more and is sort of a red flag. > > > > Thanks a lot for the details. It's more clear to me now. :) > > > >> > > >> > Alan > >> > >> Besides that, I see that region_id is needed both for the fme_pr and > >> for determining which port to disable/enable. So adding it to the > >> fpga_image_info would allow the fpga-region to figure out which > >> bridge to control as well as pass it to the mgr. > >> > > > > Do we need to add an 'id' to fpga-bridge too? > > You don't have to, but you could. See below. > > > > > Per my understanding this FME driver should create fpga-region for each > > accelerator, and make sure each fpga-region has correct region_id. When > > this fpga-region wants to be programmed by fpga-mgr, then it add this > > info to fpga_image_info, and fpga-mgr knows which region to program via > > this fpga_image_info. > > The fpga_image_info is passed both to fpga-mgr and fpga-bridge. So > both will get the region_id through the info. > > > > > For each fpga-region, FME driver needs to create one fpga-bridge and link > > it to region's bridge_list. When fpga_region_program_fpga is invoked for > > PR. This fpga-bridge could be disabled before PR and re-enabled after PR > > automatically. If fpga-bridge contains this 'id' information, then driver > > knows which port to enable/disable to implement of the enable_set function > > under this fpga-bridge. > > Could implement it either way: the bridge could either use the > bridge_id that has been passed to it in the info or code that creates > the bridge could create private data and pass it to > fpga_bridge_register(). But the bridge will have the id because it > will be passed in the info, so it's not really needed. Thanks for the suggestion. :) The reason I was considering adding id to fpga_bridge is, in the case driver creates all bridges (and regions/manager) in the initialization code, but at that time, nobody knows the actual fpga_image_info. In order to pass fpga_image_info to the bridges of give region during PR (e.g in fpga_region_program_fpga function), then driver must implement region->get_bridges function, but I feel we may not need get_bridges function for this case, as all bridges have already been linked to fpga-region during initialization. So I prefer the second method that keep this id in the private data. :) Thanks Hao > > Alan > > > > > Thanks > > Hao > > > >> > > >> >> > >> >> If you prefer the region_id for fpga_image_info, we can go with region_id > >> >> for sure. : ) > >> >> > >> >> Thanks > >> >> Hao > -- > To unsubscribe from this list: send the line "unsubscribe linux-fpga" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html
[toc] | [prev] | [next] | [standalone]
| From | Alan Tull <atull@kernel.org> |
|---|---|
| Date | 2017-04-03 23:30 +0200 |
| Subject | Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support |
| Message-ID | <tshHY-3Tw-25@gated-at.bofh.it> |
| In reply to | #1613019 |
On Thu, Mar 30, 2017 at 7:08 AM, Wu Hao <hao.wu@intel.com> wrote:
> From: Kang Luwei <luwei.kang@intel.com>
>
> Partial Reconfiguration (PR) is the most important function for FME. It
> allows reconfiguration for given Port/Accelerated Function Unit (AFU).
>
> This patch adds support for PR sub feature. In this patch, it registers
> a fpga_mgr and implements fpga_manager_ops, and invoke fpga_mgr_buf_load
> for PR operation once PR request received via ioctl.
The code that invokes fpga_mgr_buf_load should be a different layer.
> Below user space
> interfaces are exposed by this sub feature.
>
> Sysfs interface:
> * /sys/class/fpga/<fpga.x>/<intel-fpga-fme.x>/interface_id
> Read-only. Indicate the hardware interface information. Userspace
> applications need to check this interface to select correct green
> bitstream format before PR.
>
> Ioctl interface:
> * FPGA_FME_PORT_PR
> Do partial reconfiguration per information from userspace, including
> target port(AFU), buffer size and address info. It returns the PR status
> (PR error code if failed) to userspace.
>
> Signed-off-by: Tim Whisonant <tim.whisonant@intel.com>
> Signed-off-by: Enno Luebbers <enno.luebbers@intel.com>
> Signed-off-by: Shiva Rao <shiva.rao@intel.com>
> Signed-off-by: Christopher Rauer <christopher.rauer@intel.com>
> Signed-off-by: Alan Tull <alan.tull@intel.com>
> Signed-off-by: Kang Luwei <luwei.kang@intel.com>
> Signed-off-by: Xiao Guangrong <guangrong.xiao@linux.intel.com>
> Signed-off-by: Wu Hao <hao.wu@intel.com>
> ---
> drivers/fpga/intel/Makefile | 2 +-
> drivers/fpga/intel/feature-dev.h | 58 ++++++
> drivers/fpga/intel/fme-main.c | 44 ++++-
> drivers/fpga/intel/fme-pr.c | 400 +++++++++++++++++++++++++++++++++++++++
> drivers/fpga/intel/fme.h | 32 ++++
> include/uapi/linux/intel-fpga.h | 44 +++++
> 6 files changed, 578 insertions(+), 2 deletions(-)
> create mode 100644 drivers/fpga/intel/fme-pr.c
> create mode 100644 drivers/fpga/intel/fme.h
>
> diff --git a/drivers/fpga/intel/Makefile b/drivers/fpga/intel/Makefile
> index 546861d..0452cb6 100644
> --- a/drivers/fpga/intel/Makefile
> +++ b/drivers/fpga/intel/Makefile
> @@ -2,4 +2,4 @@ obj-$(CONFIG_INTEL_FPGA_PCI) += intel-fpga-pci.o
> obj-$(CONFIG_INTEL_FPGA_FME) += intel-fpga-fme.o
>
> intel-fpga-pci-objs := pcie.o feature-dev.o
> -intel-fpga-fme-objs := fme-main.o
> +intel-fpga-fme-objs := fme-main.o fme-pr.o
> diff --git a/drivers/fpga/intel/feature-dev.h b/drivers/fpga/intel/feature-dev.h
> index dccc283..5a25c915 100644
> --- a/drivers/fpga/intel/feature-dev.h
> +++ b/drivers/fpga/intel/feature-dev.h
> @@ -150,8 +150,66 @@ struct feature_fme_err {
> };
>
> /* FME Partial Reconfiguration Sub Feature Register Set */
> +/* FME PR Control Register */
> +struct feature_fme_pr_ctl {
> + union {
> + u64 csr;
> + struct {
> + u8 pr_reset:1; /* Reset PR Engine */
> + u8 rsvdz1:3;
> + u8 pr_reset_ack:1; /* Reset PR Engine Ack */
> + u8 rsvdz2:3;
> + u8 pr_regionid:2; /* PR Region ID */
> + u8 rsvdz3:2;
> + u8 pr_start_req:1; /* PR Start Request */
> + u8 pr_push_complete:1; /* PR Data push complete */
> + u8 pr_kind:1; /* Load Customer or Intel GBS */
> + u32 rsvdz4:17;
> + u32 config_data;
> + };
> + };
> +};
> +
> +/* FME PR Status Register */
> +struct feature_fme_pr_status {
> + union {
> + u64 csr;
> + struct {
> + u16 pr_credit:9; /* Number of PR Credits */
> + u8 rsvdz1:7;
> + u8 pr_status:1; /* PR Operation status */
> + u8 rsvdz2:3;
> + u8 pr_ctrlr_status:3; /* Controller status */
> + u8 rsvdz3:1;
> + u8 pr_host_status:4; /* PR Host status */
> + u64 rsvdz4:36;
> + };
> + };
> +};
> +
> +/* FME PR Data Register */
> +struct feature_fme_pr_data {
> + union {
> + u64 csr;
> + struct {
> + /* PR data from the raw-binary file */
> + u32 pr_data_raw;
> + u32 rsvd;
> + };
> + };
> +};
> +
> struct feature_fme_pr {
> struct feature_header header;
> + struct feature_fme_pr_ctl control;
> + struct feature_fme_pr_status status;
> + struct feature_fme_pr_data data;
> + u64 error;
> +
> + u64 rsvd[16];
> +
> + u64 intfc_id_l; /* PR interface Id Low */
> + u64 intfc_id_h; /* PR interface Id High */
> };
>
> /* PORT Header Register Set */
> diff --git a/drivers/fpga/intel/fme-main.c b/drivers/fpga/intel/fme-main.c
> index 36d0c4c..0d9a7a6 100644
> --- a/drivers/fpga/intel/fme-main.c
> +++ b/drivers/fpga/intel/fme-main.c
> @@ -23,6 +23,7 @@
> #include <linux/intel-fpga.h>
>
> #include "feature-dev.h"
> +#include "fme.h"
>
> static ssize_t ports_num_show(struct device *dev,
> struct device_attribute *attr, char *buf)
> @@ -101,6 +102,10 @@ static struct feature_driver fme_feature_drvs[] = {
> .ops = &fme_hdr_ops,
> },
> {
> + .name = FME_FEATURE_PR_MGMT,
> + .ops = &pr_mgmt_ops,
> + },
> + {
> .ops = NULL,
> },
> };
> @@ -182,14 +187,48 @@ static const struct file_operations fme_fops = {
> .unlocked_ioctl = fme_ioctl,
> };
>
> +static int fme_dev_init(struct platform_device *pdev)
> +{
> + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> + struct fpga_fme *fme;
> +
> + fme = devm_kzalloc(&pdev->dev, sizeof(*fme), GFP_KERNEL);
> + if (!fme)
> + return -ENOMEM;
> +
> + fme->pdata = pdata;
> +
> + mutex_lock(&pdata->lock);
> + fpga_pdata_set_private(pdata, fme);
> + mutex_unlock(&pdata->lock);
> + return 0;
> +}
> +
> +static void fme_dev_destroy(struct platform_device *pdev)
> +{
> + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> + struct fpga_fme *fme;
> +
> + mutex_lock(&pdata->lock);
> + fme = fpga_pdata_get_private(pdata);
> + fpga_pdata_set_private(pdata, NULL);
> + mutex_unlock(&pdata->lock);
> +
> + devm_kfree(&pdev->dev, fme);
> +}
> +
> static int fme_probe(struct platform_device *pdev)
> {
> int ret;
>
> - ret = fpga_dev_feature_init(pdev, fme_feature_drvs);
> + ret = fme_dev_init(pdev);
> if (ret)
> goto exit;
>
> + ret = fpga_dev_feature_init(pdev, fme_feature_drvs);
> + if (ret)
> + goto dev_destroy;
> +
> ret = fpga_register_dev_ops(pdev, &fme_fops, THIS_MODULE);
> if (ret)
> goto feature_uinit;
> @@ -198,6 +237,8 @@ static int fme_probe(struct platform_device *pdev)
>
> feature_uinit:
> fpga_dev_feature_uinit(pdev);
> +dev_destroy:
> + fme_dev_destroy(pdev);
> exit:
> return ret;
> }
> @@ -206,6 +247,7 @@ static int fme_remove(struct platform_device *pdev)
> {
> fpga_dev_feature_uinit(pdev);
> fpga_unregister_dev_ops(pdev);
> + fme_dev_destroy(pdev);
> return 0;
> }
>
> diff --git a/drivers/fpga/intel/fme-pr.c b/drivers/fpga/intel/fme-pr.c
> new file mode 100644
> index 0000000..3b44a3e
> --- /dev/null
> +++ b/drivers/fpga/intel/fme-pr.c
> @@ -0,0 +1,400 @@
> +/*
> + * Driver for Intel FPGA Management Engine (FME) Partial Reconfiguration
> + *
> + * Copyright (C) 2017 Intel Corporation, Inc.
> + *
> + * Authors:
> + * Kang Luwei <luwei.kang@intel.com>
> + * Xiao Guangrong <guangrong.xiao@linux.intel.com>
> + * Joseph Grecco <joe.grecco@intel.com>
> + * Enno Luebbers <enno.luebbers@intel.com>
> + * Tim Whisonant <tim.whisonant@intel.com>
> + * Ananda Ravuri <ananda.ravuri@intel.com>
> + * Christopher Rauer <christopher.rauer@intel.com>
> + * Henry Mitchel <henry.mitchel@intel.com>
> + *
> + * This work is licensed under a dual BSD/GPLv2 license. When using or
> + * redistributing this file, you may do so under either license. See the
> + * LICENSE.BSD file under this directory for the BSD license and see
> + * the COPYING file in the top-level directory for the GPLv2 license.
> + */
> +
> +#include <linux/types.h>
> +#include <linux/device.h>
> +#include <linux/vmalloc.h>
> +#include <linux/uaccess.h>
> +#include <linux/fpga/fpga-mgr.h>
> +#include <linux/intel-fpga.h>
> +
> +#include "feature-dev.h"
> +#include "fme.h"
> +
> +#define PR_WAIT_TIMEOUT 8000000
> +
> +#define PR_HOST_STATUS_IDLE 0
> +
> +DEFINE_FPGA_PR_ERR_MSG(pr_err_msg);
> +
> +static ssize_t interface_id_show(struct device *dev,
> + struct device_attribute *attr, char *buf)
> +{
> + u64 intfc_id_l, intfc_id_h;
> + struct feature_fme_pr *fme_pr
> + = get_feature_ioaddr_by_index(dev, FME_FEATURE_ID_PR_MGMT);
> +
> + intfc_id_l = readq(&fme_pr->intfc_id_l);
> + intfc_id_h = readq(&fme_pr->intfc_id_h);
> +
> + return scnprintf(buf, PAGE_SIZE, "%016llx%016llx\n",
> + (unsigned long long)intfc_id_h,
> + (unsigned long long)intfc_id_l);
> +}
> +static DEVICE_ATTR_RO(interface_id);
> +
> +static struct attribute *pr_mgmt_attrs[] = {
> + &dev_attr_interface_id.attr,
> + NULL,
> +};
> +
> +struct attribute_group pr_mgmt_attr_group = {
> + .attrs = pr_mgmt_attrs,
> + .name = "pr",
> +};
> +
> +static u64
> +pr_err_handle(struct platform_device *pdev, struct feature_fme_pr *fme_pr)
> +{
> + struct feature_fme_pr_status fme_pr_status;
> + unsigned long err_code;
> + u64 fme_pr_error;
> + int i = 0;
> +
> + fme_pr_status.csr = readq(&fme_pr->status);
> + if (!fme_pr_status.pr_status)
> + return 0;
> +
> + err_code = fme_pr_error = readq(&fme_pr->error);
> + for_each_set_bit(i, &err_code, PR_MAX_ERR_NUM)
> + dev_dbg(&pdev->dev, "%s\n", pr_err_msg[i]);
> + writeq(fme_pr_error, &fme_pr->error);
> + return fme_pr_error;
> +}
> +
> +static int fme_pr_write_init(struct fpga_manager *mgr,
> + struct fpga_image_info *info, const char *buf, size_t count)
> +{
> + struct fpga_fme *fme = mgr->priv;
> + struct platform_device *pdev;
> + struct feature_fme_pr *fme_pr;
> + struct feature_fme_pr_ctl fme_pr_ctl;
> + struct feature_fme_pr_status fme_pr_status;
> +
> + pdev = fme->pdata->dev;
> + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
> + FME_FEATURE_ID_PR_MGMT);
> + if (!fme_pr)
> + return -EINVAL;
> +
> + if (WARN_ON(info->flags != FPGA_MGR_PARTIAL_RECONFIG))
> + return -EINVAL;
flags is bitmapped so please do:
if (WARN_ON(info->flags & FPGA_MGR_PARTIAL_RECONFIG))
> +
> + dev_dbg(&pdev->dev, "resetting PR before initiated PR\n");
> +
> + fme_pr_ctl.csr = readq(&fme_pr->control);
> + fme_pr_ctl.pr_reset = 1;
> + writeq(fme_pr_ctl.csr, &fme_pr->control);
> +
> + fme_pr_ctl.pr_reset_ack = 1;
> +
> + if (fpga_wait_register_field(pr_reset_ack, fme_pr_ctl,
> + &fme_pr->control, PR_WAIT_TIMEOUT, 1)) {
> + dev_err(&pdev->dev, "maximum PR timeout\n");
> + return -ETIMEDOUT;
> + }
> +
> + fme_pr_ctl.csr = readq(&fme_pr->control);
> + fme_pr_ctl.pr_reset = 0;
> + writeq(fme_pr_ctl.csr, &fme_pr->control);
> +
> + dev_dbg(&pdev->dev,
> + "waiting for PR resource in HW to be initialized and ready\n");
> +
> + fme_pr_status.pr_host_status = PR_HOST_STATUS_IDLE;
> +
> + if (fpga_wait_register_field(pr_host_status, fme_pr_status,
> + &fme_pr->status, PR_WAIT_TIMEOUT, 1)) {
> + dev_err(&pdev->dev, "maximum PR timeout\n");
> + return -ETIMEDOUT;
> + }
> +
> + dev_dbg(&pdev->dev, "check if have any previous PR error\n");
> + pr_err_handle(pdev, fme_pr);
> + return 0;
> +}
> +
> +static int fme_pr_write(struct fpga_manager *mgr,
> + const char *buf, size_t count)
> +{
> + struct fpga_fme *fme = mgr->priv;
> + struct platform_device *pdev;
> + struct feature_fme_pr *fme_pr;
> + struct feature_fme_pr_ctl fme_pr_ctl;
> + struct feature_fme_pr_status fme_pr_status;
> + struct feature_fme_pr_data fme_pr_data;
> + int delay, pr_credit, i = 0;
> +
> + pdev = fme->pdata->dev;
> + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
> + FME_FEATURE_ID_PR_MGMT);
> +
> + dev_dbg(&pdev->dev, "set PR port ID and start request\n");
> +
> + fme_pr_ctl.csr = readq(&fme_pr->control);
> + fme_pr_ctl.pr_regionid = fme->port_id;
> + fme_pr_ctl.pr_start_req = 1;
> + writeq(fme_pr_ctl.csr, &fme_pr->control);
> +
> + dev_dbg(&pdev->dev, "pushing data from bitstream to HW\n");
> +
> + fme_pr_status.csr = readq(&fme_pr->status);
> + pr_credit = fme_pr_status.pr_credit;
> +
> + while (count > 0) {
> + delay = 0;
> + while (pr_credit <= 1) {
> + if (delay++ > PR_WAIT_TIMEOUT) {
> + dev_err(&pdev->dev, "maximum try\n");
> + return -ETIMEDOUT;
> + }
> + udelay(1);
> +
> + fme_pr_status.csr = readq(&fme_pr->status);
> + pr_credit = fme_pr_status.pr_credit;
> + };
> +
> + if (count >= 4) {
> + fme_pr_data.rsvd = 0;
> + fme_pr_data.pr_data_raw = *(((u32 *)buf) + i);
> + writeq(fme_pr_data.csr, &fme_pr->data);
> + count -= 4;
> + pr_credit--;
> + i++;
> + } else {
> + WARN_ON(1);
> + return -EINVAL;
> + }
> + }
> +
> + return 0;
> +}
> +
> +static int fme_pr_write_complete(struct fpga_manager *mgr,
> + struct fpga_image_info *info)
> +{
> + struct fpga_fme *fme = mgr->priv;
> + struct platform_device *pdev;
> + struct feature_fme_pr *fme_pr;
> + struct feature_fme_pr_ctl fme_pr_ctl;
> +
> + pdev = fme->pdata->dev;
> + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
> + FME_FEATURE_ID_PR_MGMT);
> +
> + fme_pr_ctl.csr = readq(&fme_pr->control);
> + fme_pr_ctl.pr_push_complete = 1;
> + writeq(fme_pr_ctl.csr, &fme_pr->control);
> +
> + dev_dbg(&pdev->dev, "green bitstream push complete\n");
> + dev_dbg(&pdev->dev, "waiting for HW to release PR resource\n");
> +
> + fme_pr_ctl.pr_start_req = 0;
> +
> + if (fpga_wait_register_field(pr_start_req, fme_pr_ctl,
> + &fme_pr->control, PR_WAIT_TIMEOUT, 1)) {
> + dev_err(&pdev->dev, "maximum try.\n");
> + return -ETIMEDOUT;
> + }
> +
> + dev_dbg(&pdev->dev, "PR operation complete, checking status\n");
> + fme->pr_err = pr_err_handle(pdev, fme_pr);
> + if (fme->pr_err)
> + return -EIO;
> +
> + dev_dbg(&pdev->dev, "PR done successfully\n");
> + return 0;
> +}
> +
> +static enum fpga_mgr_states fme_pr_state(struct fpga_manager *mgr)
> +{
> + return FPGA_MGR_STATE_UNKNOWN;
> +}
> +
> +static const struct fpga_manager_ops fme_pr_ops = {
> + .write_init = fme_pr_write_init,
> + .write = fme_pr_write,
> + .write_complete = fme_pr_write_complete,
> + .state = fme_pr_state,
> +};
> +
The following fme_pr() function shouldn't be in a fpga-mgr driver. It
is calling
the framework that this driver is registered with.
A fpga-mgr low level driver is supposed to be a very low level driver.
This function is controlling the port and calling fpga-mgr layer to do the
programming. There should be a layer above the fpga-mgr and fpga-bridge
that coordinates these things. I.e. can you use my proposed fpga-region code?
> +static int fme_pr(struct platform_device *pdev, unsigned long arg)
> +{
> + void __user *argp = (void __user *)arg;
> + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> + struct fpga_fme *fme;
> + struct fpga_manager *mgr;
> + struct feature_fme_header *fme_hdr;
> + struct feature_fme_capability fme_capability;
> + struct fpga_image_info info;
> + struct fpga_fme_port_pr port_pr;
> + struct platform_device *port;
> + unsigned long minsz;
> + void *buf = NULL;
> + int ret = 0;
> +
> + minsz = offsetofend(struct fpga_fme_port_pr, status);
> +
> + if (copy_from_user(&port_pr, argp, minsz))
> + return -EFAULT;
> +
> + if (port_pr.argsz < minsz || port_pr.flags)
> + return -EINVAL;
> +
> + if (!IS_ALIGNED(port_pr.buffer_size, 4))
> + return -EINVAL;
> +
> + /* get fme header region */
> + fme_hdr = get_feature_ioaddr_by_index(&pdev->dev,
> + FME_FEATURE_ID_HEADER);
> + if (WARN_ON(!fme_hdr))
> + return -EINVAL;
> +
> + /* check port id */
> + fme_capability.csr = readq(&fme_hdr->capability);
> + if (port_pr.port_id >= fme_capability.num_ports) {
> + dev_dbg(&pdev->dev, "port number more than maximum\n");
> + return -EINVAL;
> + }
> +
> + if (!access_ok(VERIFY_READ, port_pr.buffer_address,
> + port_pr.buffer_size))
> + return -EFAULT;
> +
> + buf = vmalloc(port_pr.buffer_size);
> + if (!buf)
> + return -ENOMEM;
> +
> + if (copy_from_user(buf, (void __user *)port_pr.buffer_address,
> + port_pr.buffer_size)) {
> + ret = -EFAULT;
> + goto free_exit;
> + }
> +
> + memset(&info, 0, sizeof(struct fpga_image_info));
> + info.flags = FPGA_MGR_PARTIAL_RECONFIG;
> +
> + mgr = fpga_mgr_get(&pdev->dev);
> + if (IS_ERR(mgr)) {
> + ret = PTR_ERR(mgr);
> + goto free_exit;
> + }
> +
> + mutex_lock(&pdata->lock);
> + fme = fpga_pdata_get_private(pdata);
> + /* fme device has been unregistered. */
> + if (!fme) {
> + ret = -EINVAL;
> + goto unlock_exit;
> + }
> +
> + fme->pr_err = 0;
> + fme->port_id = port_pr.port_id;
> +
> + /* Find and get port device by index */
> + port = pdata->fpga_for_each_port(pdev, &fme->port_id,
> + fpga_port_check_id);
> + WARN_ON(!port);
> +
> + /* Disable Port before PR */
> + fpga_port_disable(port);
> +
> + ret = fpga_mgr_buf_load(mgr, &info, buf, port_pr.buffer_size);
> + port_pr.status = fme->pr_err;
> +
> + /* Re-enable Port after PR finished */
> + fpga_port_enable(port);
> +
> + put_device(&port->dev);
> +
> +unlock_exit:
> + mutex_unlock(&pdata->lock);
> + fpga_mgr_put(mgr);
> +free_exit:
> + vfree(buf);
> + if (copy_to_user((void __user *)arg, &port_pr, minsz))
> + return -EFAULT;
> + return ret;
> +}
> +
> +static int fpga_fme_pr_probe(struct platform_device *pdev)
> +{
> + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> + struct fpga_fme *priv;
> + int ret;
> +
> + mutex_lock(&pdata->lock);
> + priv = fpga_pdata_get_private(pdata);
> + ret = fpga_mgr_register(&pdata->dev->dev,
> + "Intel FPGA Manager", &fme_pr_ops, priv);
> + mutex_unlock(&pdata->lock);
> +
> + return ret;
> +}
> +
> +static int fpga_fme_pr_remove(struct platform_device *pdev)
> +{
> + fpga_mgr_unregister(&pdev->dev);
> + return 0;
> +}
> +
> +static int pr_mgmt_init(struct platform_device *pdev, struct feature *feature)
> +{
> + int ret;
> +
> + ret = fpga_fme_pr_probe(pdev);
> + if (ret)
> + return ret;
> +
> + ret = sysfs_create_group(&pdev->dev.kobj, &pr_mgmt_attr_group);
> + if (ret)
> + fpga_fme_pr_remove(pdev);
> +
> + return ret;
> +}
> +
> +static void pr_mgmt_uinit(struct platform_device *pdev, struct feature *feature)
> +{
> + sysfs_remove_group(&pdev->dev.kobj, &pr_mgmt_attr_group);
> + fpga_fme_pr_remove(pdev);
> +}
> +
> +static long fme_pr_ioctl(struct platform_device *pdev, struct feature *feature,
> + unsigned int cmd, unsigned long arg)
> +{
> + long ret;
> +
> + switch (cmd) {
> + case FPGA_FME_PORT_PR:
> + ret = fme_pr(pdev, arg);
> + break;
> + default:
> + ret = -ENODEV;
> + }
> +
> + return ret;
> +}
> +
> +struct feature_ops pr_mgmt_ops = {
> + .init = pr_mgmt_init,
> + .uinit = pr_mgmt_uinit,
> + .ioctl = fme_pr_ioctl,
> +};
> diff --git a/drivers/fpga/intel/fme.h b/drivers/fpga/intel/fme.h
> new file mode 100644
> index 0000000..d6cb7ce
> --- /dev/null
> +++ b/drivers/fpga/intel/fme.h
> @@ -0,0 +1,32 @@
> +/*
> + * Header file for Intel FPGA Management Engine (FME) Driver
> + *
> + * Copyright (C) 2017 Intel Corporation, Inc.
> + *
> + * Authors:
> + * Kang Luwei <luwei.kang@intel.com>
> + * Xiao Guangrong <guangrong.xiao@linux.intel.com>
> + * Joseph Grecco <joe.grecco@intel.com>
> + * Enno Luebbers <enno.luebbers@intel.com>
> + * Tim Whisonant <tim.whisonant@intel.com>
> + * Ananda Ravuri <ananda.ravuri@intel.com>
> + * Henry Mitchel <henry.mitchel@intel.com>
> + *
> + * This work is licensed under a dual BSD/GPLv2 license. When using or
> + * redistributing this file, you may do so under either license. See the
> + * LICENSE.BSD file under this directory for the BSD license and see
> + * the COPYING file in the top-level directory for the GPLv2 license.
> + */
> +
> +#ifndef __INTEL_FME_H
> +#define __INTEL_FME_H
> +
> +struct fpga_fme {
> + u8 port_id;
> + u64 pr_err;
> + struct feature_platform_data *pdata;
> +};
> +
> +extern struct feature_ops pr_mgmt_ops;
> +
> +#endif
> diff --git a/include/uapi/linux/intel-fpga.h b/include/uapi/linux/intel-fpga.h
> index 992e556..77658316 100644
> --- a/include/uapi/linux/intel-fpga.h
> +++ b/include/uapi/linux/intel-fpga.h
> @@ -18,6 +18,8 @@
> #ifndef _UAPI_LINUX_INTEL_FPGA_H
> #define _UAPI_LINUX_INTEL_FPGA_H
>
> +#include <linux/types.h>
> +
> #define FPGA_API_VERSION 0
>
> /*
> @@ -30,6 +32,7 @@
> #define FPGA_MAGIC 0xB6
>
> #define FPGA_BASE 0
> +#define FME_BASE 0x80
>
> /**
> * FPGA_GET_API_VERSION - _IO(FPGA_MAGIC, FPGA_BASE + 0)
> @@ -49,4 +52,45 @@
>
> #define FPGA_CHECK_EXTENSION _IO(FPGA_MAGIC, FPGA_BASE + 1)
>
> +/* IOCTLs for FME file descriptor */
> +
> +/**
> + * FPGA_FME_PORT_PR - _IOWR(FPGA_MAGIC, FME_BASE + 0, struct fpga_fme_port_pr)
> + *
> + * Driver does Partial Reconfiguration based on Port ID and Buffer (Image)
> + * provided by caller.
> + * Return: 0 on success, -errno on failure.
> + * If FPGA_FME_PORT_PR returns -EIO, that indicates the HW has detected
> + * some errors during PR, under this case, the user can fetch HW error code
> + * from fpga_fme_port_pr.status. Each bit on the error code is used as the
> + * index for the array created by DEFINE_FPGA_PR_ERR_MSG().
> + * Otherwise, it is always zero.
> + */
> +
> +#define DEFINE_FPGA_PR_ERR_MSG(_name_) \
> +static const char * const _name_[] = { \
> + "PR operation error detected", \
> + "PR CRC error detected", \
> + "PR incompatiable bitstream error detected", \
> + "PR IP protocol error detected", \
> + "PR FIFO overflow error detected", \
> + "Reserved", \
> + "PR secure load error detected", \
> +}
> +
> +#define PR_MAX_ERR_NUM 7
> +
> +struct fpga_fme_port_pr {
> + /* Input */
> + __u32 argsz; /* Structure length */
> + __u32 flags; /* Zero for now */
> + __u32 port_id;
> + __u32 buffer_size;
> + __u64 buffer_address; /* Userspace address to the buffer for PR */
> + /* Output */
> + __u64 status; /* HW error code if ioctl returns -EIO */
> +};
> +
> +#define FPGA_FME_PORT_PR _IO(FPGA_MAGIC, FME_BASE + 0)
> +
> #endif /* _UAPI_INTEL_FPGA_H */
> --
> 2.7.4
>
[toc] | [prev] | [next] | [standalone]
| From | matthew.gerlach@linux.intel.com |
|---|---|
| Date | 2017-04-04 00:50 +0200 |
| Subject | Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support |
| Message-ID | <tsiXo-4DT-15@gated-at.bofh.it> |
| In reply to | #1615569 |
On Mon, 3 Apr 2017, Alan Tull wrote:
> On Thu, Mar 30, 2017 at 7:08 AM, Wu Hao <hao.wu@intel.com> wrote:
>> From: Kang Luwei <luwei.kang@intel.com>
>>
>> Partial Reconfiguration (PR) is the most important function for FME. It
>> allows reconfiguration for given Port/Accelerated Function Unit (AFU).
>>
>> This patch adds support for PR sub feature. In this patch, it registers
>> a fpga_mgr and implements fpga_manager_ops, and invoke fpga_mgr_buf_load
>> for PR operation once PR request received via ioctl.
>
> The code that invokes fpga_mgr_buf_load should be a different layer.
>
>> Below user space
>> interfaces are exposed by this sub feature.
>>
>> Sysfs interface:
>> * /sys/class/fpga/<fpga.x>/<intel-fpga-fme.x>/interface_id
>> Read-only. Indicate the hardware interface information. Userspace
>> applications need to check this interface to select correct green
>> bitstream format before PR.
>>
>> Ioctl interface:
>> * FPGA_FME_PORT_PR
>> Do partial reconfiguration per information from userspace, including
>> target port(AFU), buffer size and address info. It returns the PR status
>> (PR error code if failed) to userspace.
>>
>> Signed-off-by: Tim Whisonant <tim.whisonant@intel.com>
>> Signed-off-by: Enno Luebbers <enno.luebbers@intel.com>
>> Signed-off-by: Shiva Rao <shiva.rao@intel.com>
>> Signed-off-by: Christopher Rauer <christopher.rauer@intel.com>
>> Signed-off-by: Alan Tull <alan.tull@intel.com>
>> Signed-off-by: Kang Luwei <luwei.kang@intel.com>
>> Signed-off-by: Xiao Guangrong <guangrong.xiao@linux.intel.com>
>> Signed-off-by: Wu Hao <hao.wu@intel.com>
>> ---
>> drivers/fpga/intel/Makefile | 2 +-
>> drivers/fpga/intel/feature-dev.h | 58 ++++++
>> drivers/fpga/intel/fme-main.c | 44 ++++-
>> drivers/fpga/intel/fme-pr.c | 400 +++++++++++++++++++++++++++++++++++++++
>> drivers/fpga/intel/fme.h | 32 ++++
>> include/uapi/linux/intel-fpga.h | 44 +++++
>> 6 files changed, 578 insertions(+), 2 deletions(-)
>> create mode 100644 drivers/fpga/intel/fme-pr.c
>> create mode 100644 drivers/fpga/intel/fme.h
>>
>> diff --git a/drivers/fpga/intel/Makefile b/drivers/fpga/intel/Makefile
>> index 546861d..0452cb6 100644
>> --- a/drivers/fpga/intel/Makefile
>> +++ b/drivers/fpga/intel/Makefile
>> @@ -2,4 +2,4 @@ obj-$(CONFIG_INTEL_FPGA_PCI) += intel-fpga-pci.o
>> obj-$(CONFIG_INTEL_FPGA_FME) += intel-fpga-fme.o
>>
>> intel-fpga-pci-objs := pcie.o feature-dev.o
>> -intel-fpga-fme-objs := fme-main.o
>> +intel-fpga-fme-objs := fme-main.o fme-pr.o
>> diff --git a/drivers/fpga/intel/feature-dev.h b/drivers/fpga/intel/feature-dev.h
>> index dccc283..5a25c915 100644
>> --- a/drivers/fpga/intel/feature-dev.h
>> +++ b/drivers/fpga/intel/feature-dev.h
>> @@ -150,8 +150,66 @@ struct feature_fme_err {
>> };
>>
>> /* FME Partial Reconfiguration Sub Feature Register Set */
>> +/* FME PR Control Register */
>> +struct feature_fme_pr_ctl {
>> + union {
>> + u64 csr;
>> + struct {
>> + u8 pr_reset:1; /* Reset PR Engine */
>> + u8 rsvdz1:3;
>> + u8 pr_reset_ack:1; /* Reset PR Engine Ack */
>> + u8 rsvdz2:3;
>> + u8 pr_regionid:2; /* PR Region ID */
>> + u8 rsvdz3:2;
>> + u8 pr_start_req:1; /* PR Start Request */
>> + u8 pr_push_complete:1; /* PR Data push complete */
>> + u8 pr_kind:1; /* Load Customer or Intel GBS */
>> + u32 rsvdz4:17;
>> + u32 config_data;
>> + };
>> + };
>> +};
>> +
>> +/* FME PR Status Register */
>> +struct feature_fme_pr_status {
>> + union {
>> + u64 csr;
>> + struct {
>> + u16 pr_credit:9; /* Number of PR Credits */
>> + u8 rsvdz1:7;
>> + u8 pr_status:1; /* PR Operation status */
>> + u8 rsvdz2:3;
>> + u8 pr_ctrlr_status:3; /* Controller status */
>> + u8 rsvdz3:1;
>> + u8 pr_host_status:4; /* PR Host status */
>> + u64 rsvdz4:36;
>> + };
>> + };
>> +};
>> +
>> +/* FME PR Data Register */
>> +struct feature_fme_pr_data {
>> + union {
>> + u64 csr;
>> + struct {
>> + /* PR data from the raw-binary file */
>> + u32 pr_data_raw;
>> + u32 rsvd;
>> + };
>> + };
>> +};
>> +
>> struct feature_fme_pr {
>> struct feature_header header;
>> + struct feature_fme_pr_ctl control;
>> + struct feature_fme_pr_status status;
>> + struct feature_fme_pr_data data;
>> + u64 error;
>> +
>> + u64 rsvd[16];
>> +
>> + u64 intfc_id_l; /* PR interface Id Low */
>> + u64 intfc_id_h; /* PR interface Id High */
>> };
>>
>> /* PORT Header Register Set */
>> diff --git a/drivers/fpga/intel/fme-main.c b/drivers/fpga/intel/fme-main.c
>> index 36d0c4c..0d9a7a6 100644
>> --- a/drivers/fpga/intel/fme-main.c
>> +++ b/drivers/fpga/intel/fme-main.c
>> @@ -23,6 +23,7 @@
>> #include <linux/intel-fpga.h>
>>
>> #include "feature-dev.h"
>> +#include "fme.h"
>>
>> static ssize_t ports_num_show(struct device *dev,
>> struct device_attribute *attr, char *buf)
>> @@ -101,6 +102,10 @@ static struct feature_driver fme_feature_drvs[] = {
>> .ops = &fme_hdr_ops,
>> },
>> {
>> + .name = FME_FEATURE_PR_MGMT,
>> + .ops = &pr_mgmt_ops,
>> + },
>> + {
>> .ops = NULL,
>> },
>> };
>> @@ -182,14 +187,48 @@ static const struct file_operations fme_fops = {
>> .unlocked_ioctl = fme_ioctl,
>> };
>>
>> +static int fme_dev_init(struct platform_device *pdev)
>> +{
>> + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
>> + struct fpga_fme *fme;
>> +
>> + fme = devm_kzalloc(&pdev->dev, sizeof(*fme), GFP_KERNEL);
>> + if (!fme)
>> + return -ENOMEM;
>> +
>> + fme->pdata = pdata;
>> +
>> + mutex_lock(&pdata->lock);
>> + fpga_pdata_set_private(pdata, fme);
>> + mutex_unlock(&pdata->lock);
>> + return 0;
>> +}
>> +
>> +static void fme_dev_destroy(struct platform_device *pdev)
>> +{
>> + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
>> + struct fpga_fme *fme;
>> +
>> + mutex_lock(&pdata->lock);
>> + fme = fpga_pdata_get_private(pdata);
>> + fpga_pdata_set_private(pdata, NULL);
>> + mutex_unlock(&pdata->lock);
>> +
>> + devm_kfree(&pdev->dev, fme);
>> +}
>> +
>> static int fme_probe(struct platform_device *pdev)
>> {
>> int ret;
>>
>> - ret = fpga_dev_feature_init(pdev, fme_feature_drvs);
>> + ret = fme_dev_init(pdev);
>> if (ret)
>> goto exit;
>>
>> + ret = fpga_dev_feature_init(pdev, fme_feature_drvs);
>> + if (ret)
>> + goto dev_destroy;
>> +
>> ret = fpga_register_dev_ops(pdev, &fme_fops, THIS_MODULE);
>> if (ret)
>> goto feature_uinit;
>> @@ -198,6 +237,8 @@ static int fme_probe(struct platform_device *pdev)
>>
>> feature_uinit:
>> fpga_dev_feature_uinit(pdev);
>> +dev_destroy:
>> + fme_dev_destroy(pdev);
>> exit:
>> return ret;
>> }
>> @@ -206,6 +247,7 @@ static int fme_remove(struct platform_device *pdev)
>> {
>> fpga_dev_feature_uinit(pdev);
>> fpga_unregister_dev_ops(pdev);
>> + fme_dev_destroy(pdev);
>> return 0;
>> }
>>
>> diff --git a/drivers/fpga/intel/fme-pr.c b/drivers/fpga/intel/fme-pr.c
>> new file mode 100644
>> index 0000000..3b44a3e
>> --- /dev/null
>> +++ b/drivers/fpga/intel/fme-pr.c
>> @@ -0,0 +1,400 @@
>> +/*
>> + * Driver for Intel FPGA Management Engine (FME) Partial Reconfiguration
>> + *
>> + * Copyright (C) 2017 Intel Corporation, Inc.
>> + *
>> + * Authors:
>> + * Kang Luwei <luwei.kang@intel.com>
>> + * Xiao Guangrong <guangrong.xiao@linux.intel.com>
>> + * Joseph Grecco <joe.grecco@intel.com>
>> + * Enno Luebbers <enno.luebbers@intel.com>
>> + * Tim Whisonant <tim.whisonant@intel.com>
>> + * Ananda Ravuri <ananda.ravuri@intel.com>
>> + * Christopher Rauer <christopher.rauer@intel.com>
>> + * Henry Mitchel <henry.mitchel@intel.com>
>> + *
>> + * This work is licensed under a dual BSD/GPLv2 license. When using or
>> + * redistributing this file, you may do so under either license. See the
>> + * LICENSE.BSD file under this directory for the BSD license and see
>> + * the COPYING file in the top-level directory for the GPLv2 license.
>> + */
>> +
>> +#include <linux/types.h>
>> +#include <linux/device.h>
>> +#include <linux/vmalloc.h>
>> +#include <linux/uaccess.h>
>> +#include <linux/fpga/fpga-mgr.h>
>> +#include <linux/intel-fpga.h>
>> +
>> +#include "feature-dev.h"
>> +#include "fme.h"
>> +
>> +#define PR_WAIT_TIMEOUT 8000000
>> +
>> +#define PR_HOST_STATUS_IDLE 0
>> +
>> +DEFINE_FPGA_PR_ERR_MSG(pr_err_msg);
>> +
>> +static ssize_t interface_id_show(struct device *dev,
>> + struct device_attribute *attr, char *buf)
>> +{
>> + u64 intfc_id_l, intfc_id_h;
>> + struct feature_fme_pr *fme_pr
>> + = get_feature_ioaddr_by_index(dev, FME_FEATURE_ID_PR_MGMT);
>> +
>> + intfc_id_l = readq(&fme_pr->intfc_id_l);
>> + intfc_id_h = readq(&fme_pr->intfc_id_h);
>> +
>> + return scnprintf(buf, PAGE_SIZE, "%016llx%016llx\n",
>> + (unsigned long long)intfc_id_h,
>> + (unsigned long long)intfc_id_l);
>> +}
>> +static DEVICE_ATTR_RO(interface_id);
>> +
>> +static struct attribute *pr_mgmt_attrs[] = {
>> + &dev_attr_interface_id.attr,
>> + NULL,
>> +};
>> +
>> +struct attribute_group pr_mgmt_attr_group = {
>> + .attrs = pr_mgmt_attrs,
>> + .name = "pr",
>> +};
>> +
>> +static u64
>> +pr_err_handle(struct platform_device *pdev, struct feature_fme_pr *fme_pr)
>> +{
>> + struct feature_fme_pr_status fme_pr_status;
>> + unsigned long err_code;
>> + u64 fme_pr_error;
>> + int i = 0;
>> +
>> + fme_pr_status.csr = readq(&fme_pr->status);
>> + if (!fme_pr_status.pr_status)
>> + return 0;
>> +
>> + err_code = fme_pr_error = readq(&fme_pr->error);
>> + for_each_set_bit(i, &err_code, PR_MAX_ERR_NUM)
>> + dev_dbg(&pdev->dev, "%s\n", pr_err_msg[i]);
>> + writeq(fme_pr_error, &fme_pr->error);
>> + return fme_pr_error;
>> +}
>> +
>> +static int fme_pr_write_init(struct fpga_manager *mgr,
>> + struct fpga_image_info *info, const char *buf, size_t count)
>> +{
>> + struct fpga_fme *fme = mgr->priv;
>> + struct platform_device *pdev;
>> + struct feature_fme_pr *fme_pr;
>> + struct feature_fme_pr_ctl fme_pr_ctl;
>> + struct feature_fme_pr_status fme_pr_status;
>> +
>> + pdev = fme->pdata->dev;
>> + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
>> + FME_FEATURE_ID_PR_MGMT);
>> + if (!fme_pr)
>> + return -EINVAL;
>> +
>> + if (WARN_ON(info->flags != FPGA_MGR_PARTIAL_RECONFIG))
>> + return -EINVAL;
>
> flags is bitmapped so please do:
>
> if (WARN_ON(info->flags & FPGA_MGR_PARTIAL_RECONFIG))
>
>> +
>> + dev_dbg(&pdev->dev, "resetting PR before initiated PR\n");
>> +
>> + fme_pr_ctl.csr = readq(&fme_pr->control);
>> + fme_pr_ctl.pr_reset = 1;
>> + writeq(fme_pr_ctl.csr, &fme_pr->control);
>> +
>> + fme_pr_ctl.pr_reset_ack = 1;
>> +
>> + if (fpga_wait_register_field(pr_reset_ack, fme_pr_ctl,
>> + &fme_pr->control, PR_WAIT_TIMEOUT, 1)) {
>> + dev_err(&pdev->dev, "maximum PR timeout\n");
>> + return -ETIMEDOUT;
>> + }
>> +
>> + fme_pr_ctl.csr = readq(&fme_pr->control);
>> + fme_pr_ctl.pr_reset = 0;
>> + writeq(fme_pr_ctl.csr, &fme_pr->control);
>> +
>> + dev_dbg(&pdev->dev,
>> + "waiting for PR resource in HW to be initialized and ready\n");
>> +
>> + fme_pr_status.pr_host_status = PR_HOST_STATUS_IDLE;
>> +
>> + if (fpga_wait_register_field(pr_host_status, fme_pr_status,
>> + &fme_pr->status, PR_WAIT_TIMEOUT, 1)) {
>> + dev_err(&pdev->dev, "maximum PR timeout\n");
>> + return -ETIMEDOUT;
>> + }
>> +
>> + dev_dbg(&pdev->dev, "check if have any previous PR error\n");
>> + pr_err_handle(pdev, fme_pr);
>> + return 0;
>> +}
>> +
>> +static int fme_pr_write(struct fpga_manager *mgr,
>> + const char *buf, size_t count)
>> +{
>> + struct fpga_fme *fme = mgr->priv;
>> + struct platform_device *pdev;
>> + struct feature_fme_pr *fme_pr;
>> + struct feature_fme_pr_ctl fme_pr_ctl;
>> + struct feature_fme_pr_status fme_pr_status;
>> + struct feature_fme_pr_data fme_pr_data;
>> + int delay, pr_credit, i = 0;
>> +
>> + pdev = fme->pdata->dev;
>> + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
>> + FME_FEATURE_ID_PR_MGMT);
>> +
>> + dev_dbg(&pdev->dev, "set PR port ID and start request\n");
>> +
>> + fme_pr_ctl.csr = readq(&fme_pr->control);
>> + fme_pr_ctl.pr_regionid = fme->port_id;
>> + fme_pr_ctl.pr_start_req = 1;
>> + writeq(fme_pr_ctl.csr, &fme_pr->control);
>> +
>> + dev_dbg(&pdev->dev, "pushing data from bitstream to HW\n");
>> +
>> + fme_pr_status.csr = readq(&fme_pr->status);
>> + pr_credit = fme_pr_status.pr_credit;
>> +
>> + while (count > 0) {
>> + delay = 0;
>> + while (pr_credit <= 1) {
>> + if (delay++ > PR_WAIT_TIMEOUT) {
>> + dev_err(&pdev->dev, "maximum try\n");
>> + return -ETIMEDOUT;
>> + }
>> + udelay(1);
>> +
>> + fme_pr_status.csr = readq(&fme_pr->status);
>> + pr_credit = fme_pr_status.pr_credit;
>> + };
>> +
>> + if (count >= 4) {
>> + fme_pr_data.rsvd = 0;
>> + fme_pr_data.pr_data_raw = *(((u32 *)buf) + i);
>> + writeq(fme_pr_data.csr, &fme_pr->data);
>> + count -= 4;
>> + pr_credit--;
>> + i++;
>> + } else {
>> + WARN_ON(1);
>> + return -EINVAL;
>> + }
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static int fme_pr_write_complete(struct fpga_manager *mgr,
>> + struct fpga_image_info *info)
>> +{
>> + struct fpga_fme *fme = mgr->priv;
>> + struct platform_device *pdev;
>> + struct feature_fme_pr *fme_pr;
>> + struct feature_fme_pr_ctl fme_pr_ctl;
>> +
>> + pdev = fme->pdata->dev;
>> + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
>> + FME_FEATURE_ID_PR_MGMT);
>> +
>> + fme_pr_ctl.csr = readq(&fme_pr->control);
>> + fme_pr_ctl.pr_push_complete = 1;
>> + writeq(fme_pr_ctl.csr, &fme_pr->control);
>> +
>> + dev_dbg(&pdev->dev, "green bitstream push complete\n");
>> + dev_dbg(&pdev->dev, "waiting for HW to release PR resource\n");
>> +
>> + fme_pr_ctl.pr_start_req = 0;
>> +
>> + if (fpga_wait_register_field(pr_start_req, fme_pr_ctl,
>> + &fme_pr->control, PR_WAIT_TIMEOUT, 1)) {
>> + dev_err(&pdev->dev, "maximum try.\n");
>> + return -ETIMEDOUT;
>> + }
>> +
>> + dev_dbg(&pdev->dev, "PR operation complete, checking status\n");
>> + fme->pr_err = pr_err_handle(pdev, fme_pr);
>> + if (fme->pr_err)
>> + return -EIO;
>> +
>> + dev_dbg(&pdev->dev, "PR done successfully\n");
>> + return 0;
>> +}
>> +
>> +static enum fpga_mgr_states fme_pr_state(struct fpga_manager *mgr)
>> +{
>> + return FPGA_MGR_STATE_UNKNOWN;
>> +}
The functions that implement the fme_pr_ops are really a low level fpga
manager driver for the Altera PR IP component. A standalone version of
such a driver has been reviewed and Acked. See the links below.
Could this file use those functions and remove this code?
http://marc.info/?l=linux-kernel&m=149019678925564&w=2
http://marc.info/?l=linux-kernel&m=149019654225457&w=2
http://marc.info/?l=linux-kernel&m=149019598025274&w=2
http://marc.info/?l=linux-kernel&m=149013051007149&w=2
>> +
>> +static const struct fpga_manager_ops fme_pr_ops = {
>> + .write_init = fme_pr_write_init,
>> + .write = fme_pr_write,
>> + .write_complete = fme_pr_write_complete,
>> + .state = fme_pr_state,
>> +};
>> +
>
> The following fme_pr() function shouldn't be in a fpga-mgr driver. It
> is calling
> the framework that this driver is registered with.
> A fpga-mgr low level driver is supposed to be a very low level driver.
> This function is controlling the port and calling fpga-mgr layer to do the
> programming. There should be a layer above the fpga-mgr and fpga-bridge
> that coordinates these things. I.e. can you use my proposed fpga-region code?
>
>> +static int fme_pr(struct platform_device *pdev, unsigned long arg)
>> +{
>> + void __user *argp = (void __user *)arg;
>> + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
>> + struct fpga_fme *fme;
>> + struct fpga_manager *mgr;
>> + struct feature_fme_header *fme_hdr;
>> + struct feature_fme_capability fme_capability;
>> + struct fpga_image_info info;
>> + struct fpga_fme_port_pr port_pr;
>> + struct platform_device *port;
>> + unsigned long minsz;
>> + void *buf = NULL;
>> + int ret = 0;
>> +
>> + minsz = offsetofend(struct fpga_fme_port_pr, status);
>> +
>> + if (copy_from_user(&port_pr, argp, minsz))
>> + return -EFAULT;
>> +
>> + if (port_pr.argsz < minsz || port_pr.flags)
>> + return -EINVAL;
>> +
>> + if (!IS_ALIGNED(port_pr.buffer_size, 4))
>> + return -EINVAL;
>> +
>> + /* get fme header region */
>> + fme_hdr = get_feature_ioaddr_by_index(&pdev->dev,
>> + FME_FEATURE_ID_HEADER);
>> + if (WARN_ON(!fme_hdr))
>> + return -EINVAL;
>> +
>> + /* check port id */
>> + fme_capability.csr = readq(&fme_hdr->capability);
>> + if (port_pr.port_id >= fme_capability.num_ports) {
>> + dev_dbg(&pdev->dev, "port number more than maximum\n");
>> + return -EINVAL;
>> + }
>> +
>> + if (!access_ok(VERIFY_READ, port_pr.buffer_address,
>> + port_pr.buffer_size))
>> + return -EFAULT;
>> +
>> + buf = vmalloc(port_pr.buffer_size);
>> + if (!buf)
>> + return -ENOMEM;
>> +
>> + if (copy_from_user(buf, (void __user *)port_pr.buffer_address,
>> + port_pr.buffer_size)) {
>> + ret = -EFAULT;
>> + goto free_exit;
>> + }
>> +
>> + memset(&info, 0, sizeof(struct fpga_image_info));
>> + info.flags = FPGA_MGR_PARTIAL_RECONFIG;
>> +
>> + mgr = fpga_mgr_get(&pdev->dev);
>> + if (IS_ERR(mgr)) {
>> + ret = PTR_ERR(mgr);
>> + goto free_exit;
>> + }
>> +
>> + mutex_lock(&pdata->lock);
>> + fme = fpga_pdata_get_private(pdata);
>> + /* fme device has been unregistered. */
>> + if (!fme) {
>> + ret = -EINVAL;
>> + goto unlock_exit;
>> + }
>> +
>> + fme->pr_err = 0;
>> + fme->port_id = port_pr.port_id;
>> +
>> + /* Find and get port device by index */
>> + port = pdata->fpga_for_each_port(pdev, &fme->port_id,
>> + fpga_port_check_id);
>> + WARN_ON(!port);
>> +
>> + /* Disable Port before PR */
>> + fpga_port_disable(port);
>> +
>> + ret = fpga_mgr_buf_load(mgr, &info, buf, port_pr.buffer_size);
>> + port_pr.status = fme->pr_err;
>> +
>> + /* Re-enable Port after PR finished */
>> + fpga_port_enable(port);
>> +
>> + put_device(&port->dev);
>> +
>> +unlock_exit:
>> + mutex_unlock(&pdata->lock);
>> + fpga_mgr_put(mgr);
>> +free_exit:
>> + vfree(buf);
>> + if (copy_to_user((void __user *)arg, &port_pr, minsz))
>> + return -EFAULT;
>> + return ret;
>> +}
>> +
>> +static int fpga_fme_pr_probe(struct platform_device *pdev)
>> +{
>> + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
>> + struct fpga_fme *priv;
>> + int ret;
>> +
>> + mutex_lock(&pdata->lock);
>> + priv = fpga_pdata_get_private(pdata);
>> + ret = fpga_mgr_register(&pdata->dev->dev,
>> + "Intel FPGA Manager", &fme_pr_ops, priv);
The call to fpga_mgr_register() above would change to a call to
alt_pr_register() if the standalone version of the PR IP driver were used.
>> + mutex_unlock(&pdata->lock);
>> +
>> + return ret;
>> +}
>> +
>> +static int fpga_fme_pr_remove(struct platform_device *pdev)
>> +{
>> + fpga_mgr_unregister(&pdev->dev);
This call to fpga_mgr_unregister() would change to a call to
alt_pr_unregister() if the standalone version of the PR IP driver were
used.
>> + return 0;
>> +}
>> +
>> +static int pr_mgmt_init(struct platform_device *pdev, struct feature *feature)
>> +{
>> + int ret;
>> +
>> + ret = fpga_fme_pr_probe(pdev);
>> + if (ret)
>> + return ret;
>> +
>> + ret = sysfs_create_group(&pdev->dev.kobj, &pr_mgmt_attr_group);
>> + if (ret)
>> + fpga_fme_pr_remove(pdev);
>> +
>> + return ret;
>> +}
>> +
>> +static void pr_mgmt_uinit(struct platform_device *pdev, struct feature *feature)
>> +{
>> + sysfs_remove_group(&pdev->dev.kobj, &pr_mgmt_attr_group);
>> + fpga_fme_pr_remove(pdev);
>> +}
>> +
>> +static long fme_pr_ioctl(struct platform_device *pdev, struct feature *feature,
>> + unsigned int cmd, unsigned long arg)
>> +{
>> + long ret;
>> +
>> + switch (cmd) {
>> + case FPGA_FME_PORT_PR:
>> + ret = fme_pr(pdev, arg);
>> + break;
>> + default:
>> + ret = -ENODEV;
>> + }
>> +
>> + return ret;
>> +}
>> +
>> +struct feature_ops pr_mgmt_ops = {
>> + .init = pr_mgmt_init,
>> + .uinit = pr_mgmt_uinit,
>> + .ioctl = fme_pr_ioctl,
>> +};
>> diff --git a/drivers/fpga/intel/fme.h b/drivers/fpga/intel/fme.h
>> new file mode 100644
>> index 0000000..d6cb7ce
>> --- /dev/null
>> +++ b/drivers/fpga/intel/fme.h
>> @@ -0,0 +1,32 @@
>> +/*
>> + * Header file for Intel FPGA Management Engine (FME) Driver
>> + *
>> + * Copyright (C) 2017 Intel Corporation, Inc.
>> + *
>> + * Authors:
>> + * Kang Luwei <luwei.kang@intel.com>
>> + * Xiao Guangrong <guangrong.xiao@linux.intel.com>
>> + * Joseph Grecco <joe.grecco@intel.com>
>> + * Enno Luebbers <enno.luebbers@intel.com>
>> + * Tim Whisonant <tim.whisonant@intel.com>
>> + * Ananda Ravuri <ananda.ravuri@intel.com>
>> + * Henry Mitchel <henry.mitchel@intel.com>
>> + *
>> + * This work is licensed under a dual BSD/GPLv2 license. When using or
>> + * redistributing this file, you may do so under either license. See the
>> + * LICENSE.BSD file under this directory for the BSD license and see
>> + * the COPYING file in the top-level directory for the GPLv2 license.
>> + */
>> +
>> +#ifndef __INTEL_FME_H
>> +#define __INTEL_FME_H
>> +
>> +struct fpga_fme {
>> + u8 port_id;
>> + u64 pr_err;
>> + struct feature_platform_data *pdata;
>> +};
>> +
>> +extern struct feature_ops pr_mgmt_ops;
>> +
>> +#endif
>> diff --git a/include/uapi/linux/intel-fpga.h b/include/uapi/linux/intel-fpga.h
>> index 992e556..77658316 100644
>> --- a/include/uapi/linux/intel-fpga.h
>> +++ b/include/uapi/linux/intel-fpga.h
>> @@ -18,6 +18,8 @@
>> #ifndef _UAPI_LINUX_INTEL_FPGA_H
>> #define _UAPI_LINUX_INTEL_FPGA_H
>>
>> +#include <linux/types.h>
>> +
>> #define FPGA_API_VERSION 0
>>
>> /*
>> @@ -30,6 +32,7 @@
>> #define FPGA_MAGIC 0xB6
>>
>> #define FPGA_BASE 0
>> +#define FME_BASE 0x80
>>
>> /**
>> * FPGA_GET_API_VERSION - _IO(FPGA_MAGIC, FPGA_BASE + 0)
>> @@ -49,4 +52,45 @@
>>
>> #define FPGA_CHECK_EXTENSION _IO(FPGA_MAGIC, FPGA_BASE + 1)
>>
>> +/* IOCTLs for FME file descriptor */
>> +
>> +/**
>> + * FPGA_FME_PORT_PR - _IOWR(FPGA_MAGIC, FME_BASE + 0, struct fpga_fme_port_pr)
>> + *
>> + * Driver does Partial Reconfiguration based on Port ID and Buffer (Image)
>> + * provided by caller.
>> + * Return: 0 on success, -errno on failure.
>> + * If FPGA_FME_PORT_PR returns -EIO, that indicates the HW has detected
>> + * some errors during PR, under this case, the user can fetch HW error code
>> + * from fpga_fme_port_pr.status. Each bit on the error code is used as the
>> + * index for the array created by DEFINE_FPGA_PR_ERR_MSG().
>> + * Otherwise, it is always zero.
>> + */
>> +
>> +#define DEFINE_FPGA_PR_ERR_MSG(_name_) \
>> +static const char * const _name_[] = { \
>> + "PR operation error detected", \
>> + "PR CRC error detected", \
>> + "PR incompatiable bitstream error detected", \
>> + "PR IP protocol error detected", \
>> + "PR FIFO overflow error detected", \
>> + "Reserved", \
>> + "PR secure load error detected", \
>> +}
>> +
>> +#define PR_MAX_ERR_NUM 7
>> +
>> +struct fpga_fme_port_pr {
>> + /* Input */
>> + __u32 argsz; /* Structure length */
>> + __u32 flags; /* Zero for now */
>> + __u32 port_id;
>> + __u32 buffer_size;
>> + __u64 buffer_address; /* Userspace address to the buffer for PR */
>> + /* Output */
>> + __u64 status; /* HW error code if ioctl returns -EIO */
>> +};
>> +
>> +#define FPGA_FME_PORT_PR _IO(FPGA_MAGIC, FME_BASE + 0)
>> +
>> #endif /* _UAPI_INTEL_FPGA_H */
>> --
>> 2.7.4
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fpga" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
[toc] | [prev] | [next] | [standalone]
| From | Wu Hao <hao.wu@intel.com> |
|---|---|
| Date | 2017-04-04 09:00 +0200 |
| Subject | Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support |
| Message-ID | <tsqBA-1io-9@gated-at.bofh.it> |
| In reply to | #1615601 |
On Mon, Apr 03, 2017 at 03:49:41PM -0700, matthew.gerlach@linux.intel.com wrote:
>
>
> On Mon, 3 Apr 2017, Alan Tull wrote:
>
> >On Thu, Mar 30, 2017 at 7:08 AM, Wu Hao <hao.wu@intel.com> wrote:
> >>From: Kang Luwei <luwei.kang@intel.com>
> >>
> >>Partial Reconfiguration (PR) is the most important function for FME. It
> >>allows reconfiguration for given Port/Accelerated Function Unit (AFU).
> >>
> >>This patch adds support for PR sub feature. In this patch, it registers
> >>a fpga_mgr and implements fpga_manager_ops, and invoke fpga_mgr_buf_load
> >>for PR operation once PR request received via ioctl.
> >
> >The code that invokes fpga_mgr_buf_load should be a different layer.
> >
> >>Below user space
> >>interfaces are exposed by this sub feature.
> >>
> >>Sysfs interface:
> >>* /sys/class/fpga/<fpga.x>/<intel-fpga-fme.x>/interface_id
> >> Read-only. Indicate the hardware interface information. Userspace
> >> applications need to check this interface to select correct green
> >> bitstream format before PR.
> >>
> >>Ioctl interface:
> >>* FPGA_FME_PORT_PR
> >> Do partial reconfiguration per information from userspace, including
> >> target port(AFU), buffer size and address info. It returns the PR status
> >> (PR error code if failed) to userspace.
> >>
> >>Signed-off-by: Tim Whisonant <tim.whisonant@intel.com>
> >>Signed-off-by: Enno Luebbers <enno.luebbers@intel.com>
> >>Signed-off-by: Shiva Rao <shiva.rao@intel.com>
> >>Signed-off-by: Christopher Rauer <christopher.rauer@intel.com>
> >>Signed-off-by: Alan Tull <alan.tull@intel.com>
> >>Signed-off-by: Kang Luwei <luwei.kang@intel.com>
> >>Signed-off-by: Xiao Guangrong <guangrong.xiao@linux.intel.com>
> >>Signed-off-by: Wu Hao <hao.wu@intel.com>
> >>---
> >> drivers/fpga/intel/Makefile | 2 +-
> >> drivers/fpga/intel/feature-dev.h | 58 ++++++
> >> drivers/fpga/intel/fme-main.c | 44 ++++-
> >> drivers/fpga/intel/fme-pr.c | 400 +++++++++++++++++++++++++++++++++++++++
> >> drivers/fpga/intel/fme.h | 32 ++++
> >> include/uapi/linux/intel-fpga.h | 44 +++++
> >> 6 files changed, 578 insertions(+), 2 deletions(-)
> >> create mode 100644 drivers/fpga/intel/fme-pr.c
> >> create mode 100644 drivers/fpga/intel/fme.h
> >>
> >>diff --git a/drivers/fpga/intel/Makefile b/drivers/fpga/intel/Makefile
> >>index 546861d..0452cb6 100644
> >>--- a/drivers/fpga/intel/Makefile
> >>+++ b/drivers/fpga/intel/Makefile
> >>@@ -2,4 +2,4 @@ obj-$(CONFIG_INTEL_FPGA_PCI) += intel-fpga-pci.o
> >> obj-$(CONFIG_INTEL_FPGA_FME) += intel-fpga-fme.o
> >>
> >> intel-fpga-pci-objs := pcie.o feature-dev.o
> >>-intel-fpga-fme-objs := fme-main.o
> >>+intel-fpga-fme-objs := fme-main.o fme-pr.o
> >>diff --git a/drivers/fpga/intel/feature-dev.h b/drivers/fpga/intel/feature-dev.h
> >>index dccc283..5a25c915 100644
> >>--- a/drivers/fpga/intel/feature-dev.h
> >>+++ b/drivers/fpga/intel/feature-dev.h
> >>@@ -150,8 +150,66 @@ struct feature_fme_err {
> >> };
> >>
> >> /* FME Partial Reconfiguration Sub Feature Register Set */
> >>+/* FME PR Control Register */
> >>+struct feature_fme_pr_ctl {
> >>+ union {
> >>+ u64 csr;
> >>+ struct {
> >>+ u8 pr_reset:1; /* Reset PR Engine */
> >>+ u8 rsvdz1:3;
> >>+ u8 pr_reset_ack:1; /* Reset PR Engine Ack */
> >>+ u8 rsvdz2:3;
> >>+ u8 pr_regionid:2; /* PR Region ID */
> >>+ u8 rsvdz3:2;
> >>+ u8 pr_start_req:1; /* PR Start Request */
> >>+ u8 pr_push_complete:1; /* PR Data push complete */
> >>+ u8 pr_kind:1; /* Load Customer or Intel GBS */
> >>+ u32 rsvdz4:17;
> >>+ u32 config_data;
> >>+ };
> >>+ };
> >>+};
> >>+
> >>+/* FME PR Status Register */
> >>+struct feature_fme_pr_status {
> >>+ union {
> >>+ u64 csr;
> >>+ struct {
> >>+ u16 pr_credit:9; /* Number of PR Credits */
> >>+ u8 rsvdz1:7;
> >>+ u8 pr_status:1; /* PR Operation status */
> >>+ u8 rsvdz2:3;
> >>+ u8 pr_ctrlr_status:3; /* Controller status */
> >>+ u8 rsvdz3:1;
> >>+ u8 pr_host_status:4; /* PR Host status */
> >>+ u64 rsvdz4:36;
> >>+ };
> >>+ };
> >>+};
> >>+
> >>+/* FME PR Data Register */
> >>+struct feature_fme_pr_data {
> >>+ union {
> >>+ u64 csr;
> >>+ struct {
> >>+ /* PR data from the raw-binary file */
> >>+ u32 pr_data_raw;
> >>+ u32 rsvd;
> >>+ };
> >>+ };
> >>+};
> >>+
> >> struct feature_fme_pr {
> >> struct feature_header header;
> >>+ struct feature_fme_pr_ctl control;
> >>+ struct feature_fme_pr_status status;
> >>+ struct feature_fme_pr_data data;
> >>+ u64 error;
> >>+
> >>+ u64 rsvd[16];
> >>+
> >>+ u64 intfc_id_l; /* PR interface Id Low */
> >>+ u64 intfc_id_h; /* PR interface Id High */
> >> };
> >>
> >> /* PORT Header Register Set */
> >>diff --git a/drivers/fpga/intel/fme-main.c b/drivers/fpga/intel/fme-main.c
> >>index 36d0c4c..0d9a7a6 100644
> >>--- a/drivers/fpga/intel/fme-main.c
> >>+++ b/drivers/fpga/intel/fme-main.c
> >>@@ -23,6 +23,7 @@
> >> #include <linux/intel-fpga.h>
> >>
> >> #include "feature-dev.h"
> >>+#include "fme.h"
> >>
> >> static ssize_t ports_num_show(struct device *dev,
> >> struct device_attribute *attr, char *buf)
> >>@@ -101,6 +102,10 @@ static struct feature_driver fme_feature_drvs[] = {
> >> .ops = &fme_hdr_ops,
> >> },
> >> {
> >>+ .name = FME_FEATURE_PR_MGMT,
> >>+ .ops = &pr_mgmt_ops,
> >>+ },
> >>+ {
> >> .ops = NULL,
> >> },
> >> };
> >>@@ -182,14 +187,48 @@ static const struct file_operations fme_fops = {
> >> .unlocked_ioctl = fme_ioctl,
> >> };
> >>
> >>+static int fme_dev_init(struct platform_device *pdev)
> >>+{
> >>+ struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> >>+ struct fpga_fme *fme;
> >>+
> >>+ fme = devm_kzalloc(&pdev->dev, sizeof(*fme), GFP_KERNEL);
> >>+ if (!fme)
> >>+ return -ENOMEM;
> >>+
> >>+ fme->pdata = pdata;
> >>+
> >>+ mutex_lock(&pdata->lock);
> >>+ fpga_pdata_set_private(pdata, fme);
> >>+ mutex_unlock(&pdata->lock);
> >>+ return 0;
> >>+}
> >>+
> >>+static void fme_dev_destroy(struct platform_device *pdev)
> >>+{
> >>+ struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> >>+ struct fpga_fme *fme;
> >>+
> >>+ mutex_lock(&pdata->lock);
> >>+ fme = fpga_pdata_get_private(pdata);
> >>+ fpga_pdata_set_private(pdata, NULL);
> >>+ mutex_unlock(&pdata->lock);
> >>+
> >>+ devm_kfree(&pdev->dev, fme);
> >>+}
> >>+
> >> static int fme_probe(struct platform_device *pdev)
> >> {
> >> int ret;
> >>
> >>- ret = fpga_dev_feature_init(pdev, fme_feature_drvs);
> >>+ ret = fme_dev_init(pdev);
> >> if (ret)
> >> goto exit;
> >>
> >>+ ret = fpga_dev_feature_init(pdev, fme_feature_drvs);
> >>+ if (ret)
> >>+ goto dev_destroy;
> >>+
> >> ret = fpga_register_dev_ops(pdev, &fme_fops, THIS_MODULE);
> >> if (ret)
> >> goto feature_uinit;
> >>@@ -198,6 +237,8 @@ static int fme_probe(struct platform_device *pdev)
> >>
> >> feature_uinit:
> >> fpga_dev_feature_uinit(pdev);
> >>+dev_destroy:
> >>+ fme_dev_destroy(pdev);
> >> exit:
> >> return ret;
> >> }
> >>@@ -206,6 +247,7 @@ static int fme_remove(struct platform_device *pdev)
> >> {
> >> fpga_dev_feature_uinit(pdev);
> >> fpga_unregister_dev_ops(pdev);
> >>+ fme_dev_destroy(pdev);
> >> return 0;
> >> }
> >>
> >>diff --git a/drivers/fpga/intel/fme-pr.c b/drivers/fpga/intel/fme-pr.c
> >>new file mode 100644
> >>index 0000000..3b44a3e
> >>--- /dev/null
> >>+++ b/drivers/fpga/intel/fme-pr.c
> >>@@ -0,0 +1,400 @@
> >>+/*
> >>+ * Driver for Intel FPGA Management Engine (FME) Partial Reconfiguration
> >>+ *
> >>+ * Copyright (C) 2017 Intel Corporation, Inc.
> >>+ *
> >>+ * Authors:
> >>+ * Kang Luwei <luwei.kang@intel.com>
> >>+ * Xiao Guangrong <guangrong.xiao@linux.intel.com>
> >>+ * Joseph Grecco <joe.grecco@intel.com>
> >>+ * Enno Luebbers <enno.luebbers@intel.com>
> >>+ * Tim Whisonant <tim.whisonant@intel.com>
> >>+ * Ananda Ravuri <ananda.ravuri@intel.com>
> >>+ * Christopher Rauer <christopher.rauer@intel.com>
> >>+ * Henry Mitchel <henry.mitchel@intel.com>
> >>+ *
> >>+ * This work is licensed under a dual BSD/GPLv2 license. When using or
> >>+ * redistributing this file, you may do so under either license. See the
> >>+ * LICENSE.BSD file under this directory for the BSD license and see
> >>+ * the COPYING file in the top-level directory for the GPLv2 license.
> >>+ */
> >>+
> >>+#include <linux/types.h>
> >>+#include <linux/device.h>
> >>+#include <linux/vmalloc.h>
> >>+#include <linux/uaccess.h>
> >>+#include <linux/fpga/fpga-mgr.h>
> >>+#include <linux/intel-fpga.h>
> >>+
> >>+#include "feature-dev.h"
> >>+#include "fme.h"
> >>+
> >>+#define PR_WAIT_TIMEOUT 8000000
> >>+
> >>+#define PR_HOST_STATUS_IDLE 0
> >>+
> >>+DEFINE_FPGA_PR_ERR_MSG(pr_err_msg);
> >>+
> >>+static ssize_t interface_id_show(struct device *dev,
> >>+ struct device_attribute *attr, char *buf)
> >>+{
> >>+ u64 intfc_id_l, intfc_id_h;
> >>+ struct feature_fme_pr *fme_pr
> >>+ = get_feature_ioaddr_by_index(dev, FME_FEATURE_ID_PR_MGMT);
> >>+
> >>+ intfc_id_l = readq(&fme_pr->intfc_id_l);
> >>+ intfc_id_h = readq(&fme_pr->intfc_id_h);
> >>+
> >>+ return scnprintf(buf, PAGE_SIZE, "%016llx%016llx\n",
> >>+ (unsigned long long)intfc_id_h,
> >>+ (unsigned long long)intfc_id_l);
> >>+}
> >>+static DEVICE_ATTR_RO(interface_id);
> >>+
> >>+static struct attribute *pr_mgmt_attrs[] = {
> >>+ &dev_attr_interface_id.attr,
> >>+ NULL,
> >>+};
> >>+
> >>+struct attribute_group pr_mgmt_attr_group = {
> >>+ .attrs = pr_mgmt_attrs,
> >>+ .name = "pr",
> >>+};
> >>+
> >>+static u64
> >>+pr_err_handle(struct platform_device *pdev, struct feature_fme_pr *fme_pr)
> >>+{
> >>+ struct feature_fme_pr_status fme_pr_status;
> >>+ unsigned long err_code;
> >>+ u64 fme_pr_error;
> >>+ int i = 0;
> >>+
> >>+ fme_pr_status.csr = readq(&fme_pr->status);
> >>+ if (!fme_pr_status.pr_status)
> >>+ return 0;
> >>+
> >>+ err_code = fme_pr_error = readq(&fme_pr->error);
> >>+ for_each_set_bit(i, &err_code, PR_MAX_ERR_NUM)
> >>+ dev_dbg(&pdev->dev, "%s\n", pr_err_msg[i]);
> >>+ writeq(fme_pr_error, &fme_pr->error);
> >>+ return fme_pr_error;
> >>+}
> >>+
> >>+static int fme_pr_write_init(struct fpga_manager *mgr,
> >>+ struct fpga_image_info *info, const char *buf, size_t count)
> >>+{
> >>+ struct fpga_fme *fme = mgr->priv;
> >>+ struct platform_device *pdev;
> >>+ struct feature_fme_pr *fme_pr;
> >>+ struct feature_fme_pr_ctl fme_pr_ctl;
> >>+ struct feature_fme_pr_status fme_pr_status;
> >>+
> >>+ pdev = fme->pdata->dev;
> >>+ fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
> >>+ FME_FEATURE_ID_PR_MGMT);
> >>+ if (!fme_pr)
> >>+ return -EINVAL;
> >>+
> >>+ if (WARN_ON(info->flags != FPGA_MGR_PARTIAL_RECONFIG))
> >>+ return -EINVAL;
> >
> >flags is bitmapped so please do:
> >
> >if (WARN_ON(info->flags & FPGA_MGR_PARTIAL_RECONFIG))
> >
> >>+
> >>+ dev_dbg(&pdev->dev, "resetting PR before initiated PR\n");
> >>+
> >>+ fme_pr_ctl.csr = readq(&fme_pr->control);
> >>+ fme_pr_ctl.pr_reset = 1;
> >>+ writeq(fme_pr_ctl.csr, &fme_pr->control);
> >>+
> >>+ fme_pr_ctl.pr_reset_ack = 1;
> >>+
> >>+ if (fpga_wait_register_field(pr_reset_ack, fme_pr_ctl,
> >>+ &fme_pr->control, PR_WAIT_TIMEOUT, 1)) {
> >>+ dev_err(&pdev->dev, "maximum PR timeout\n");
> >>+ return -ETIMEDOUT;
> >>+ }
> >>+
> >>+ fme_pr_ctl.csr = readq(&fme_pr->control);
> >>+ fme_pr_ctl.pr_reset = 0;
> >>+ writeq(fme_pr_ctl.csr, &fme_pr->control);
> >>+
> >>+ dev_dbg(&pdev->dev,
> >>+ "waiting for PR resource in HW to be initialized and ready\n");
> >>+
> >>+ fme_pr_status.pr_host_status = PR_HOST_STATUS_IDLE;
> >>+
> >>+ if (fpga_wait_register_field(pr_host_status, fme_pr_status,
> >>+ &fme_pr->status, PR_WAIT_TIMEOUT, 1)) {
> >>+ dev_err(&pdev->dev, "maximum PR timeout\n");
> >>+ return -ETIMEDOUT;
> >>+ }
> >>+
> >>+ dev_dbg(&pdev->dev, "check if have any previous PR error\n");
> >>+ pr_err_handle(pdev, fme_pr);
> >>+ return 0;
> >>+}
> >>+
> >>+static int fme_pr_write(struct fpga_manager *mgr,
> >>+ const char *buf, size_t count)
> >>+{
> >>+ struct fpga_fme *fme = mgr->priv;
> >>+ struct platform_device *pdev;
> >>+ struct feature_fme_pr *fme_pr;
> >>+ struct feature_fme_pr_ctl fme_pr_ctl;
> >>+ struct feature_fme_pr_status fme_pr_status;
> >>+ struct feature_fme_pr_data fme_pr_data;
> >>+ int delay, pr_credit, i = 0;
> >>+
> >>+ pdev = fme->pdata->dev;
> >>+ fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
> >>+ FME_FEATURE_ID_PR_MGMT);
> >>+
> >>+ dev_dbg(&pdev->dev, "set PR port ID and start request\n");
> >>+
> >>+ fme_pr_ctl.csr = readq(&fme_pr->control);
> >>+ fme_pr_ctl.pr_regionid = fme->port_id;
> >>+ fme_pr_ctl.pr_start_req = 1;
> >>+ writeq(fme_pr_ctl.csr, &fme_pr->control);
> >>+
> >>+ dev_dbg(&pdev->dev, "pushing data from bitstream to HW\n");
> >>+
> >>+ fme_pr_status.csr = readq(&fme_pr->status);
> >>+ pr_credit = fme_pr_status.pr_credit;
> >>+
> >>+ while (count > 0) {
> >>+ delay = 0;
> >>+ while (pr_credit <= 1) {
> >>+ if (delay++ > PR_WAIT_TIMEOUT) {
> >>+ dev_err(&pdev->dev, "maximum try\n");
> >>+ return -ETIMEDOUT;
> >>+ }
> >>+ udelay(1);
> >>+
> >>+ fme_pr_status.csr = readq(&fme_pr->status);
> >>+ pr_credit = fme_pr_status.pr_credit;
> >>+ };
> >>+
> >>+ if (count >= 4) {
> >>+ fme_pr_data.rsvd = 0;
> >>+ fme_pr_data.pr_data_raw = *(((u32 *)buf) + i);
> >>+ writeq(fme_pr_data.csr, &fme_pr->data);
> >>+ count -= 4;
> >>+ pr_credit--;
> >>+ i++;
> >>+ } else {
> >>+ WARN_ON(1);
> >>+ return -EINVAL;
> >>+ }
> >>+ }
> >>+
> >>+ return 0;
> >>+}
> >>+
> >>+static int fme_pr_write_complete(struct fpga_manager *mgr,
> >>+ struct fpga_image_info *info)
> >>+{
> >>+ struct fpga_fme *fme = mgr->priv;
> >>+ struct platform_device *pdev;
> >>+ struct feature_fme_pr *fme_pr;
> >>+ struct feature_fme_pr_ctl fme_pr_ctl;
> >>+
> >>+ pdev = fme->pdata->dev;
> >>+ fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
> >>+ FME_FEATURE_ID_PR_MGMT);
> >>+
> >>+ fme_pr_ctl.csr = readq(&fme_pr->control);
> >>+ fme_pr_ctl.pr_push_complete = 1;
> >>+ writeq(fme_pr_ctl.csr, &fme_pr->control);
> >>+
> >>+ dev_dbg(&pdev->dev, "green bitstream push complete\n");
> >>+ dev_dbg(&pdev->dev, "waiting for HW to release PR resource\n");
> >>+
> >>+ fme_pr_ctl.pr_start_req = 0;
> >>+
> >>+ if (fpga_wait_register_field(pr_start_req, fme_pr_ctl,
> >>+ &fme_pr->control, PR_WAIT_TIMEOUT, 1)) {
> >>+ dev_err(&pdev->dev, "maximum try.\n");
> >>+ return -ETIMEDOUT;
> >>+ }
> >>+
> >>+ dev_dbg(&pdev->dev, "PR operation complete, checking status\n");
> >>+ fme->pr_err = pr_err_handle(pdev, fme_pr);
> >>+ if (fme->pr_err)
> >>+ return -EIO;
> >>+
> >>+ dev_dbg(&pdev->dev, "PR done successfully\n");
> >>+ return 0;
> >>+}
> >>+
> >>+static enum fpga_mgr_states fme_pr_state(struct fpga_manager *mgr)
> >>+{
> >>+ return FPGA_MGR_STATE_UNKNOWN;
> >>+}
>
> The functions that implement the fme_pr_ops are really a low level fpga
> manager driver for the Altera PR IP component. A standalone version of
> such a driver has been reviewed and Acked. See the links below.
> Could this file use those functions and remove this code?
>
> http://marc.info/?l=linux-kernel&m=149019678925564&w=2
> http://marc.info/?l=linux-kernel&m=149019654225457&w=2
> http://marc.info/?l=linux-kernel&m=149019598025274&w=2
> http://marc.info/?l=linux-kernel&m=149013051007149&w=2
>
Thanks for the info, I just checked these code quickly, but it seems the
register definitions are totally different, so I feel that we may not be
able to reuse this driver.
Thanks
Hao
[toc] | [prev] | [next] | [standalone]
| From | Wu Hao <hao.wu@intel.com> |
|---|---|
| Date | 2017-04-04 08:40 +0200 |
| Subject | Re: [PATCH 11/16] fpga: intel: fme: add partial reconfiguration sub feature support |
| Message-ID | <tsqif-1bv-35@gated-at.bofh.it> |
| In reply to | #1615569 |
On Mon, Apr 03, 2017 at 04:24:13PM -0500, Alan Tull wrote:
> On Thu, Mar 30, 2017 at 7:08 AM, Wu Hao <hao.wu@intel.com> wrote:
> > From: Kang Luwei <luwei.kang@intel.com>
> >
> > Partial Reconfiguration (PR) is the most important function for FME. It
> > allows reconfiguration for given Port/Accelerated Function Unit (AFU).
> >
> > This patch adds support for PR sub feature. In this patch, it registers
> > a fpga_mgr and implements fpga_manager_ops, and invoke fpga_mgr_buf_load
> > for PR operation once PR request received via ioctl.
>
> The code that invokes fpga_mgr_buf_load should be a different layer.
Please see my feedback below.
>
> > Below user space
> > interfaces are exposed by this sub feature.
> >
> > Sysfs interface:
> > * /sys/class/fpga/<fpga.x>/<intel-fpga-fme.x>/interface_id
> > Read-only. Indicate the hardware interface information. Userspace
> > applications need to check this interface to select correct green
> > bitstream format before PR.
> >
> > Ioctl interface:
> > * FPGA_FME_PORT_PR
> > Do partial reconfiguration per information from userspace, including
> > target port(AFU), buffer size and address info. It returns the PR status
> > (PR error code if failed) to userspace.
> >
> > Signed-off-by: Tim Whisonant <tim.whisonant@intel.com>
> > Signed-off-by: Enno Luebbers <enno.luebbers@intel.com>
> > Signed-off-by: Shiva Rao <shiva.rao@intel.com>
> > Signed-off-by: Christopher Rauer <christopher.rauer@intel.com>
> > Signed-off-by: Alan Tull <alan.tull@intel.com>
> > Signed-off-by: Kang Luwei <luwei.kang@intel.com>
> > Signed-off-by: Xiao Guangrong <guangrong.xiao@linux.intel.com>
> > Signed-off-by: Wu Hao <hao.wu@intel.com>
> > ---
> > drivers/fpga/intel/Makefile | 2 +-
> > drivers/fpga/intel/feature-dev.h | 58 ++++++
> > drivers/fpga/intel/fme-main.c | 44 ++++-
> > drivers/fpga/intel/fme-pr.c | 400 +++++++++++++++++++++++++++++++++++++++
> > drivers/fpga/intel/fme.h | 32 ++++
> > include/uapi/linux/intel-fpga.h | 44 +++++
> > 6 files changed, 578 insertions(+), 2 deletions(-)
> > create mode 100644 drivers/fpga/intel/fme-pr.c
> > create mode 100644 drivers/fpga/intel/fme.h
> >
> > diff --git a/drivers/fpga/intel/Makefile b/drivers/fpga/intel/Makefile
> > index 546861d..0452cb6 100644
> > --- a/drivers/fpga/intel/Makefile
> > +++ b/drivers/fpga/intel/Makefile
> > @@ -2,4 +2,4 @@ obj-$(CONFIG_INTEL_FPGA_PCI) += intel-fpga-pci.o
> > obj-$(CONFIG_INTEL_FPGA_FME) += intel-fpga-fme.o
> >
> > intel-fpga-pci-objs := pcie.o feature-dev.o
> > -intel-fpga-fme-objs := fme-main.o
> > +intel-fpga-fme-objs := fme-main.o fme-pr.o
> > diff --git a/drivers/fpga/intel/feature-dev.h b/drivers/fpga/intel/feature-dev.h
> > index dccc283..5a25c915 100644
> > --- a/drivers/fpga/intel/feature-dev.h
> > +++ b/drivers/fpga/intel/feature-dev.h
> > @@ -150,8 +150,66 @@ struct feature_fme_err {
> > };
> >
> > /* FME Partial Reconfiguration Sub Feature Register Set */
> > +/* FME PR Control Register */
> > +struct feature_fme_pr_ctl {
> > + union {
> > + u64 csr;
> > + struct {
> > + u8 pr_reset:1; /* Reset PR Engine */
> > + u8 rsvdz1:3;
> > + u8 pr_reset_ack:1; /* Reset PR Engine Ack */
> > + u8 rsvdz2:3;
> > + u8 pr_regionid:2; /* PR Region ID */
> > + u8 rsvdz3:2;
> > + u8 pr_start_req:1; /* PR Start Request */
> > + u8 pr_push_complete:1; /* PR Data push complete */
> > + u8 pr_kind:1; /* Load Customer or Intel GBS */
> > + u32 rsvdz4:17;
> > + u32 config_data;
> > + };
> > + };
> > +};
> > +
> > +/* FME PR Status Register */
> > +struct feature_fme_pr_status {
> > + union {
> > + u64 csr;
> > + struct {
> > + u16 pr_credit:9; /* Number of PR Credits */
> > + u8 rsvdz1:7;
> > + u8 pr_status:1; /* PR Operation status */
> > + u8 rsvdz2:3;
> > + u8 pr_ctrlr_status:3; /* Controller status */
> > + u8 rsvdz3:1;
> > + u8 pr_host_status:4; /* PR Host status */
> > + u64 rsvdz4:36;
> > + };
> > + };
> > +};
> > +
> > +/* FME PR Data Register */
> > +struct feature_fme_pr_data {
> > + union {
> > + u64 csr;
> > + struct {
> > + /* PR data from the raw-binary file */
> > + u32 pr_data_raw;
> > + u32 rsvd;
> > + };
> > + };
> > +};
> > +
> > struct feature_fme_pr {
> > struct feature_header header;
> > + struct feature_fme_pr_ctl control;
> > + struct feature_fme_pr_status status;
> > + struct feature_fme_pr_data data;
> > + u64 error;
> > +
> > + u64 rsvd[16];
> > +
> > + u64 intfc_id_l; /* PR interface Id Low */
> > + u64 intfc_id_h; /* PR interface Id High */
> > };
> >
> > /* PORT Header Register Set */
> > diff --git a/drivers/fpga/intel/fme-main.c b/drivers/fpga/intel/fme-main.c
> > index 36d0c4c..0d9a7a6 100644
> > --- a/drivers/fpga/intel/fme-main.c
> > +++ b/drivers/fpga/intel/fme-main.c
> > @@ -23,6 +23,7 @@
> > #include <linux/intel-fpga.h>
> >
> > #include "feature-dev.h"
> > +#include "fme.h"
> >
> > static ssize_t ports_num_show(struct device *dev,
> > struct device_attribute *attr, char *buf)
> > @@ -101,6 +102,10 @@ static struct feature_driver fme_feature_drvs[] = {
> > .ops = &fme_hdr_ops,
> > },
> > {
> > + .name = FME_FEATURE_PR_MGMT,
> > + .ops = &pr_mgmt_ops,
> > + },
> > + {
> > .ops = NULL,
> > },
> > };
> > @@ -182,14 +187,48 @@ static const struct file_operations fme_fops = {
> > .unlocked_ioctl = fme_ioctl,
> > };
> >
> > +static int fme_dev_init(struct platform_device *pdev)
> > +{
> > + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> > + struct fpga_fme *fme;
> > +
> > + fme = devm_kzalloc(&pdev->dev, sizeof(*fme), GFP_KERNEL);
> > + if (!fme)
> > + return -ENOMEM;
> > +
> > + fme->pdata = pdata;
> > +
> > + mutex_lock(&pdata->lock);
> > + fpga_pdata_set_private(pdata, fme);
> > + mutex_unlock(&pdata->lock);
> > + return 0;
> > +}
> > +
> > +static void fme_dev_destroy(struct platform_device *pdev)
> > +{
> > + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> > + struct fpga_fme *fme;
> > +
> > + mutex_lock(&pdata->lock);
> > + fme = fpga_pdata_get_private(pdata);
> > + fpga_pdata_set_private(pdata, NULL);
> > + mutex_unlock(&pdata->lock);
> > +
> > + devm_kfree(&pdev->dev, fme);
> > +}
> > +
> > static int fme_probe(struct platform_device *pdev)
> > {
> > int ret;
> >
> > - ret = fpga_dev_feature_init(pdev, fme_feature_drvs);
> > + ret = fme_dev_init(pdev);
> > if (ret)
> > goto exit;
> >
> > + ret = fpga_dev_feature_init(pdev, fme_feature_drvs);
> > + if (ret)
> > + goto dev_destroy;
> > +
> > ret = fpga_register_dev_ops(pdev, &fme_fops, THIS_MODULE);
> > if (ret)
> > goto feature_uinit;
> > @@ -198,6 +237,8 @@ static int fme_probe(struct platform_device *pdev)
> >
> > feature_uinit:
> > fpga_dev_feature_uinit(pdev);
> > +dev_destroy:
> > + fme_dev_destroy(pdev);
> > exit:
> > return ret;
> > }
> > @@ -206,6 +247,7 @@ static int fme_remove(struct platform_device *pdev)
> > {
> > fpga_dev_feature_uinit(pdev);
> > fpga_unregister_dev_ops(pdev);
> > + fme_dev_destroy(pdev);
> > return 0;
> > }
> >
> > diff --git a/drivers/fpga/intel/fme-pr.c b/drivers/fpga/intel/fme-pr.c
> > new file mode 100644
> > index 0000000..3b44a3e
> > --- /dev/null
> > +++ b/drivers/fpga/intel/fme-pr.c
> > @@ -0,0 +1,400 @@
> > +/*
> > + * Driver for Intel FPGA Management Engine (FME) Partial Reconfiguration
> > + *
> > + * Copyright (C) 2017 Intel Corporation, Inc.
> > + *
> > + * Authors:
> > + * Kang Luwei <luwei.kang@intel.com>
> > + * Xiao Guangrong <guangrong.xiao@linux.intel.com>
> > + * Joseph Grecco <joe.grecco@intel.com>
> > + * Enno Luebbers <enno.luebbers@intel.com>
> > + * Tim Whisonant <tim.whisonant@intel.com>
> > + * Ananda Ravuri <ananda.ravuri@intel.com>
> > + * Christopher Rauer <christopher.rauer@intel.com>
> > + * Henry Mitchel <henry.mitchel@intel.com>
> > + *
> > + * This work is licensed under a dual BSD/GPLv2 license. When using or
> > + * redistributing this file, you may do so under either license. See the
> > + * LICENSE.BSD file under this directory for the BSD license and see
> > + * the COPYING file in the top-level directory for the GPLv2 license.
> > + */
> > +
> > +#include <linux/types.h>
> > +#include <linux/device.h>
> > +#include <linux/vmalloc.h>
> > +#include <linux/uaccess.h>
> > +#include <linux/fpga/fpga-mgr.h>
> > +#include <linux/intel-fpga.h>
> > +
> > +#include "feature-dev.h"
> > +#include "fme.h"
> > +
> > +#define PR_WAIT_TIMEOUT 8000000
> > +
> > +#define PR_HOST_STATUS_IDLE 0
> > +
> > +DEFINE_FPGA_PR_ERR_MSG(pr_err_msg);
> > +
> > +static ssize_t interface_id_show(struct device *dev,
> > + struct device_attribute *attr, char *buf)
> > +{
> > + u64 intfc_id_l, intfc_id_h;
> > + struct feature_fme_pr *fme_pr
> > + = get_feature_ioaddr_by_index(dev, FME_FEATURE_ID_PR_MGMT);
> > +
> > + intfc_id_l = readq(&fme_pr->intfc_id_l);
> > + intfc_id_h = readq(&fme_pr->intfc_id_h);
> > +
> > + return scnprintf(buf, PAGE_SIZE, "%016llx%016llx\n",
> > + (unsigned long long)intfc_id_h,
> > + (unsigned long long)intfc_id_l);
> > +}
> > +static DEVICE_ATTR_RO(interface_id);
> > +
> > +static struct attribute *pr_mgmt_attrs[] = {
> > + &dev_attr_interface_id.attr,
> > + NULL,
> > +};
> > +
> > +struct attribute_group pr_mgmt_attr_group = {
> > + .attrs = pr_mgmt_attrs,
> > + .name = "pr",
> > +};
> > +
> > +static u64
> > +pr_err_handle(struct platform_device *pdev, struct feature_fme_pr *fme_pr)
> > +{
> > + struct feature_fme_pr_status fme_pr_status;
> > + unsigned long err_code;
> > + u64 fme_pr_error;
> > + int i = 0;
> > +
> > + fme_pr_status.csr = readq(&fme_pr->status);
> > + if (!fme_pr_status.pr_status)
> > + return 0;
> > +
> > + err_code = fme_pr_error = readq(&fme_pr->error);
> > + for_each_set_bit(i, &err_code, PR_MAX_ERR_NUM)
> > + dev_dbg(&pdev->dev, "%s\n", pr_err_msg[i]);
> > + writeq(fme_pr_error, &fme_pr->error);
> > + return fme_pr_error;
> > +}
> > +
> > +static int fme_pr_write_init(struct fpga_manager *mgr,
> > + struct fpga_image_info *info, const char *buf, size_t count)
> > +{
> > + struct fpga_fme *fme = mgr->priv;
> > + struct platform_device *pdev;
> > + struct feature_fme_pr *fme_pr;
> > + struct feature_fme_pr_ctl fme_pr_ctl;
> > + struct feature_fme_pr_status fme_pr_status;
> > +
> > + pdev = fme->pdata->dev;
> > + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
> > + FME_FEATURE_ID_PR_MGMT);
> > + if (!fme_pr)
> > + return -EINVAL;
> > +
> > + if (WARN_ON(info->flags != FPGA_MGR_PARTIAL_RECONFIG))
> > + return -EINVAL;
>
> flags is bitmapped so please do:
>
> if (WARN_ON(info->flags & FPGA_MGR_PARTIAL_RECONFIG))
Thanks for the review.
Will fix this in the next version.
>
> > +
> > + dev_dbg(&pdev->dev, "resetting PR before initiated PR\n");
> > +
> > + fme_pr_ctl.csr = readq(&fme_pr->control);
> > + fme_pr_ctl.pr_reset = 1;
> > + writeq(fme_pr_ctl.csr, &fme_pr->control);
> > +
> > + fme_pr_ctl.pr_reset_ack = 1;
> > +
> > + if (fpga_wait_register_field(pr_reset_ack, fme_pr_ctl,
> > + &fme_pr->control, PR_WAIT_TIMEOUT, 1)) {
> > + dev_err(&pdev->dev, "maximum PR timeout\n");
> > + return -ETIMEDOUT;
> > + }
> > +
> > + fme_pr_ctl.csr = readq(&fme_pr->control);
> > + fme_pr_ctl.pr_reset = 0;
> > + writeq(fme_pr_ctl.csr, &fme_pr->control);
> > +
> > + dev_dbg(&pdev->dev,
> > + "waiting for PR resource in HW to be initialized and ready\n");
> > +
> > + fme_pr_status.pr_host_status = PR_HOST_STATUS_IDLE;
> > +
> > + if (fpga_wait_register_field(pr_host_status, fme_pr_status,
> > + &fme_pr->status, PR_WAIT_TIMEOUT, 1)) {
> > + dev_err(&pdev->dev, "maximum PR timeout\n");
> > + return -ETIMEDOUT;
> > + }
> > +
> > + dev_dbg(&pdev->dev, "check if have any previous PR error\n");
> > + pr_err_handle(pdev, fme_pr);
> > + return 0;
> > +}
> > +
> > +static int fme_pr_write(struct fpga_manager *mgr,
> > + const char *buf, size_t count)
> > +{
> > + struct fpga_fme *fme = mgr->priv;
> > + struct platform_device *pdev;
> > + struct feature_fme_pr *fme_pr;
> > + struct feature_fme_pr_ctl fme_pr_ctl;
> > + struct feature_fme_pr_status fme_pr_status;
> > + struct feature_fme_pr_data fme_pr_data;
> > + int delay, pr_credit, i = 0;
> > +
> > + pdev = fme->pdata->dev;
> > + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
> > + FME_FEATURE_ID_PR_MGMT);
> > +
> > + dev_dbg(&pdev->dev, "set PR port ID and start request\n");
> > +
> > + fme_pr_ctl.csr = readq(&fme_pr->control);
> > + fme_pr_ctl.pr_regionid = fme->port_id;
> > + fme_pr_ctl.pr_start_req = 1;
> > + writeq(fme_pr_ctl.csr, &fme_pr->control);
> > +
> > + dev_dbg(&pdev->dev, "pushing data from bitstream to HW\n");
> > +
> > + fme_pr_status.csr = readq(&fme_pr->status);
> > + pr_credit = fme_pr_status.pr_credit;
> > +
> > + while (count > 0) {
> > + delay = 0;
> > + while (pr_credit <= 1) {
> > + if (delay++ > PR_WAIT_TIMEOUT) {
> > + dev_err(&pdev->dev, "maximum try\n");
> > + return -ETIMEDOUT;
> > + }
> > + udelay(1);
> > +
> > + fme_pr_status.csr = readq(&fme_pr->status);
> > + pr_credit = fme_pr_status.pr_credit;
> > + };
> > +
> > + if (count >= 4) {
> > + fme_pr_data.rsvd = 0;
> > + fme_pr_data.pr_data_raw = *(((u32 *)buf) + i);
> > + writeq(fme_pr_data.csr, &fme_pr->data);
> > + count -= 4;
> > + pr_credit--;
> > + i++;
> > + } else {
> > + WARN_ON(1);
> > + return -EINVAL;
> > + }
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static int fme_pr_write_complete(struct fpga_manager *mgr,
> > + struct fpga_image_info *info)
> > +{
> > + struct fpga_fme *fme = mgr->priv;
> > + struct platform_device *pdev;
> > + struct feature_fme_pr *fme_pr;
> > + struct feature_fme_pr_ctl fme_pr_ctl;
> > +
> > + pdev = fme->pdata->dev;
> > + fme_pr = get_feature_ioaddr_by_index(&pdev->dev,
> > + FME_FEATURE_ID_PR_MGMT);
> > +
> > + fme_pr_ctl.csr = readq(&fme_pr->control);
> > + fme_pr_ctl.pr_push_complete = 1;
> > + writeq(fme_pr_ctl.csr, &fme_pr->control);
> > +
> > + dev_dbg(&pdev->dev, "green bitstream push complete\n");
> > + dev_dbg(&pdev->dev, "waiting for HW to release PR resource\n");
> > +
> > + fme_pr_ctl.pr_start_req = 0;
> > +
> > + if (fpga_wait_register_field(pr_start_req, fme_pr_ctl,
> > + &fme_pr->control, PR_WAIT_TIMEOUT, 1)) {
> > + dev_err(&pdev->dev, "maximum try.\n");
> > + return -ETIMEDOUT;
> > + }
> > +
> > + dev_dbg(&pdev->dev, "PR operation complete, checking status\n");
> > + fme->pr_err = pr_err_handle(pdev, fme_pr);
> > + if (fme->pr_err)
> > + return -EIO;
> > +
> > + dev_dbg(&pdev->dev, "PR done successfully\n");
> > + return 0;
> > +}
> > +
> > +static enum fpga_mgr_states fme_pr_state(struct fpga_manager *mgr)
> > +{
> > + return FPGA_MGR_STATE_UNKNOWN;
> > +}
> > +
> > +static const struct fpga_manager_ops fme_pr_ops = {
> > + .write_init = fme_pr_write_init,
> > + .write = fme_pr_write,
> > + .write_complete = fme_pr_write_complete,
> > + .state = fme_pr_state,
> > +};
> > +
>
> The following fme_pr() function shouldn't be in a fpga-mgr driver. It
> is calling
> the framework that this driver is registered with.
> A fpga-mgr low level driver is supposed to be a very low level driver.
> This function is controlling the port and calling fpga-mgr layer to do the
> programming. There should be a layer above the fpga-mgr and fpga-bridge
> that coordinates these things. I.e. can you use my proposed fpga-region code?
>
Hm.. as mentioned in the intel-fpga.txt documentation 'open discussion'.
Current FME doesn't use fpga-region as related patches are not merged.
But we will make the related changes to let FME create all fpga-regions,
bridges and managers for PR function on given Intel FPGA device per
suggestion.
Please notice below fme_pr is still needed to provide ioctl interface to
enduser for PR, it should invoke fpga_region_program_fpga or equivalent
for PR instead of fpga_mgr_buf_load then.
And if in the future, the sysfs for PR is ready, then fme_pr may not be
needed, as user can do PR via fpga-region sysfs interface.
Thanks
Hao
> > +static int fme_pr(struct platform_device *pdev, unsigned long arg)
> > +{
> > + void __user *argp = (void __user *)arg;
> > + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> > + struct fpga_fme *fme;
> > + struct fpga_manager *mgr;
> > + struct feature_fme_header *fme_hdr;
> > + struct feature_fme_capability fme_capability;
> > + struct fpga_image_info info;
> > + struct fpga_fme_port_pr port_pr;
> > + struct platform_device *port;
> > + unsigned long minsz;
> > + void *buf = NULL;
> > + int ret = 0;
> > +
> > + minsz = offsetofend(struct fpga_fme_port_pr, status);
> > +
> > + if (copy_from_user(&port_pr, argp, minsz))
> > + return -EFAULT;
> > +
> > + if (port_pr.argsz < minsz || port_pr.flags)
> > + return -EINVAL;
> > +
> > + if (!IS_ALIGNED(port_pr.buffer_size, 4))
> > + return -EINVAL;
> > +
> > + /* get fme header region */
> > + fme_hdr = get_feature_ioaddr_by_index(&pdev->dev,
> > + FME_FEATURE_ID_HEADER);
> > + if (WARN_ON(!fme_hdr))
> > + return -EINVAL;
> > +
> > + /* check port id */
> > + fme_capability.csr = readq(&fme_hdr->capability);
> > + if (port_pr.port_id >= fme_capability.num_ports) {
> > + dev_dbg(&pdev->dev, "port number more than maximum\n");
> > + return -EINVAL;
> > + }
> > +
> > + if (!access_ok(VERIFY_READ, port_pr.buffer_address,
> > + port_pr.buffer_size))
> > + return -EFAULT;
> > +
> > + buf = vmalloc(port_pr.buffer_size);
> > + if (!buf)
> > + return -ENOMEM;
> > +
> > + if (copy_from_user(buf, (void __user *)port_pr.buffer_address,
> > + port_pr.buffer_size)) {
> > + ret = -EFAULT;
> > + goto free_exit;
> > + }
> > +
> > + memset(&info, 0, sizeof(struct fpga_image_info));
> > + info.flags = FPGA_MGR_PARTIAL_RECONFIG;
> > +
> > + mgr = fpga_mgr_get(&pdev->dev);
> > + if (IS_ERR(mgr)) {
> > + ret = PTR_ERR(mgr);
> > + goto free_exit;
> > + }
> > +
> > + mutex_lock(&pdata->lock);
> > + fme = fpga_pdata_get_private(pdata);
> > + /* fme device has been unregistered. */
> > + if (!fme) {
> > + ret = -EINVAL;
> > + goto unlock_exit;
> > + }
> > +
> > + fme->pr_err = 0;
> > + fme->port_id = port_pr.port_id;
> > +
> > + /* Find and get port device by index */
> > + port = pdata->fpga_for_each_port(pdev, &fme->port_id,
> > + fpga_port_check_id);
> > + WARN_ON(!port);
> > +
> > + /* Disable Port before PR */
> > + fpga_port_disable(port);
> > +
> > + ret = fpga_mgr_buf_load(mgr, &info, buf, port_pr.buffer_size);
> > + port_pr.status = fme->pr_err;
> > +
> > + /* Re-enable Port after PR finished */
> > + fpga_port_enable(port);
> > +
> > + put_device(&port->dev);
> > +
> > +unlock_exit:
> > + mutex_unlock(&pdata->lock);
> > + fpga_mgr_put(mgr);
> > +free_exit:
> > + vfree(buf);
> > + if (copy_to_user((void __user *)arg, &port_pr, minsz))
> > + return -EFAULT;
> > + return ret;
> > +}
> > +
> > +static int fpga_fme_pr_probe(struct platform_device *pdev)
> > +{
> > + struct feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
> > + struct fpga_fme *priv;
> > + int ret;
> > +
> > + mutex_lock(&pdata->lock);
> > + priv = fpga_pdata_get_private(pdata);
> > + ret = fpga_mgr_register(&pdata->dev->dev,
> > + "Intel FPGA Manager", &fme_pr_ops, priv);
> > + mutex_unlock(&pdata->lock);
> > +
> > + return ret;
> > +}
> > +
> > +static int fpga_fme_pr_remove(struct platform_device *pdev)
> > +{
> > + fpga_mgr_unregister(&pdev->dev);
> > + return 0;
> > +}
> > +
> > +static int pr_mgmt_init(struct platform_device *pdev, struct feature *feature)
> > +{
> > + int ret;
> > +
> > + ret = fpga_fme_pr_probe(pdev);
> > + if (ret)
> > + return ret;
> > +
> > + ret = sysfs_create_group(&pdev->dev.kobj, &pr_mgmt_attr_group);
> > + if (ret)
> > + fpga_fme_pr_remove(pdev);
> > +
> > + return ret;
> > +}
> > +
> > +static void pr_mgmt_uinit(struct platform_device *pdev, struct feature *feature)
> > +{
> > + sysfs_remove_group(&pdev->dev.kobj, &pr_mgmt_attr_group);
> > + fpga_fme_pr_remove(pdev);
> > +}
> > +
> > +static long fme_pr_ioctl(struct platform_device *pdev, struct feature *feature,
> > + unsigned int cmd, unsigned long arg)
> > +{
> > + long ret;
> > +
> > + switch (cmd) {
> > + case FPGA_FME_PORT_PR:
> > + ret = fme_pr(pdev, arg);
> > + break;
> > + default:
> > + ret = -ENODEV;
> > + }
> > +
> > + return ret;
> > +}
> > +
> > +struct feature_ops pr_mgmt_ops = {
> > + .init = pr_mgmt_init,
> > + .uinit = pr_mgmt_uinit,
> > + .ioctl = fme_pr_ioctl,
> > +};
> > diff --git a/drivers/fpga/intel/fme.h b/drivers/fpga/intel/fme.h
> > new file mode 100644
> > index 0000000..d6cb7ce
> > --- /dev/null
> > +++ b/drivers/fpga/intel/fme.h
> > @@ -0,0 +1,32 @@
> > +/*
> > + * Header file for Intel FPGA Management Engine (FME) Driver
> > + *
> > + * Copyright (C) 2017 Intel Corporation, Inc.
> > + *
> > + * Authors:
> > + * Kang Luwei <luwei.kang@intel.com>
> > + * Xiao Guangrong <guangrong.xiao@linux.intel.com>
> > + * Joseph Grecco <joe.grecco@intel.com>
> > + * Enno Luebbers <enno.luebbers@intel.com>
> > + * Tim Whisonant <tim.whisonant@intel.com>
> > + * Ananda Ravuri <ananda.ravuri@intel.com>
> > + * Henry Mitchel <henry.mitchel@intel.com>
> > + *
> > + * This work is licensed under a dual BSD/GPLv2 license. When using or
> > + * redistributing this file, you may do so under either license. See the
> > + * LICENSE.BSD file under this directory for the BSD license and see
> > + * the COPYING file in the top-level directory for the GPLv2 license.
> > + */
> > +
> > +#ifndef __INTEL_FME_H
> > +#define __INTEL_FME_H
> > +
> > +struct fpga_fme {
> > + u8 port_id;
> > + u64 pr_err;
> > + struct feature_platform_data *pdata;
> > +};
> > +
> > +extern struct feature_ops pr_mgmt_ops;
> > +
> > +#endif
> > diff --git a/include/uapi/linux/intel-fpga.h b/include/uapi/linux/intel-fpga.h
> > index 992e556..77658316 100644
> > --- a/include/uapi/linux/intel-fpga.h
> > +++ b/include/uapi/linux/intel-fpga.h
> > @@ -18,6 +18,8 @@
> > #ifndef _UAPI_LINUX_INTEL_FPGA_H
> > #define _UAPI_LINUX_INTEL_FPGA_H
> >
> > +#include <linux/types.h>
> > +
> > #define FPGA_API_VERSION 0
> >
> > /*
> > @@ -30,6 +32,7 @@
> > #define FPGA_MAGIC 0xB6
> >
> > #define FPGA_BASE 0
> > +#define FME_BASE 0x80
> >
> > /**
> > * FPGA_GET_API_VERSION - _IO(FPGA_MAGIC, FPGA_BASE + 0)
> > @@ -49,4 +52,45 @@
> >
> > #define FPGA_CHECK_EXTENSION _IO(FPGA_MAGIC, FPGA_BASE + 1)
> >
> > +/* IOCTLs for FME file descriptor */
> > +
> > +/**
> > + * FPGA_FME_PORT_PR - _IOWR(FPGA_MAGIC, FME_BASE + 0, struct fpga_fme_port_pr)
> > + *
> > + * Driver does Partial Reconfiguration based on Port ID and Buffer (Image)
> > + * provided by caller.
> > + * Return: 0 on success, -errno on failure.
> > + * If FPGA_FME_PORT_PR returns -EIO, that indicates the HW has detected
> > + * some errors during PR, under this case, the user can fetch HW error code
> > + * from fpga_fme_port_pr.status. Each bit on the error code is used as the
> > + * index for the array created by DEFINE_FPGA_PR_ERR_MSG().
> > + * Otherwise, it is always zero.
> > + */
> > +
> > +#define DEFINE_FPGA_PR_ERR_MSG(_name_) \
> > +static const char * const _name_[] = { \
> > + "PR operation error detected", \
> > + "PR CRC error detected", \
> > + "PR incompatiable bitstream error detected", \
> > + "PR IP protocol error detected", \
> > + "PR FIFO overflow error detected", \
> > + "Reserved", \
> > + "PR secure load error detected", \
> > +}
> > +
> > +#define PR_MAX_ERR_NUM 7
> > +
> > +struct fpga_fme_port_pr {
> > + /* Input */
> > + __u32 argsz; /* Structure length */
> > + __u32 flags; /* Zero for now */
> > + __u32 port_id;
> > + __u32 buffer_size;
> > + __u64 buffer_address; /* Userspace address to the buffer for PR */
> > + /* Output */
> > + __u64 status; /* HW error code if ioctl returns -EIO */
> > +};
> > +
> > +#define FPGA_FME_PORT_PR _IO(FPGA_MAGIC, FME_BASE + 0)
> > +
> > #endif /* _UAPI_INTEL_FPGA_H */
> > --
> > 2.7.4
> >
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fpga" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[toc] | [prev] | [next] | [standalone]
| From | Moritz Fischer <mdf@kernel.org> |
|---|---|
| Date | 2017-03-30 19:20 +0200 |
| Message-ID | <tqLTP-1a6-5@gated-at.bofh.it> |
| In reply to | #1613003 |
[Multipart message — attachments visible in raw view] — view raw
On Thu, Mar 30, 2017 at 08:08:00PM +0800, Wu Hao wrote: > Hi All, > > Here is a patch-series adding drivers for Intel FPGA devices. > > The Intel FPGA driver provides interfaces for userspace applications to > configure, enumerate, open, and access FPGA accelerators on platforms > equipped with Intel(R) FPGA solutions and enables system level management > functions such as FPGA partial reconfiguration, power management and > virtualization. > > This patch series only adds the basic functions for FPGA accelerators and > partial reconfiguration. Patches for more functions, e.g power management > and virtualization, will be submitted after this series gets reviewed. > > Patch 1: add a document for Intel FPGA driver overview, including the HW > architecture, driver organization, device enumeration, virtualization and > opens. > > Patch 2: introduce a fpga-dev class. It's used in below Intel FPGA PCIe > device driver, to represent a FPGA device on the system, and all actual > feature devices should be registered as child nodes of this container > fpga-dev device. > > Patch 3-7: implement Intel FPGA PCIe device driver. It walks through the > 'Device Feature List' in the PCI Bar, creates the container fpga-dev as > parent and platform devices as children for the feature devices it found. > > Patch 8-11: implement Intel FPGA Management Engine (FME) driver. It's a > platform driver matching with the FME platform device created by above > PCIe driver. Sysfs and device file ioctls are exposed as user interfaces > to allow partial reconfiguration to Accelerated Function Units (AFUs) from > user space applications. > > Patch 12-16: implement Intel FPGA Accelerated Function Unit (AFU) driver. > It's a platform driver matching with AFU platform device created by above > PCIe driver. It provides user interfaces to expose the AFU MMIO region, > map/unmap dma buffer, and control the port which AFU connects to. This is exciting stuff. It will take some time to review, though. I marked the patchset as 'In-Review' in patchwork. Cheers, Moritz
[toc] | [prev] | [next] | [standalone]
Page 3 of 4 — ← Prev page 1 2 [3] 4 Next page →
Back to top | Article view | linux.kernel
csiph-web