Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > linux.kernel > #1581425 > unrolled thread

[RFC 7/8] fpga-region: add sysfs interface

Started byAlan Tull <atull@kernel.org>
First post2017-02-15 17:20 +0100
Last post2017-02-15 22:30 +0100
Articles 20 on this page of 46 — 8 participants

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.


Contents

  [RFC 7/8] fpga-region: add sysfs interface Alan Tull <atull@kernel.org> - 2017-02-15 17:20 +0100
    Re: [RFC 7/8] fpga-region: add sysfs interface Jason Gunthorpe <jgunthorpe@obsidianresearch.com> - 2017-02-15 18:30 +0100
      Re: [RFC 7/8] fpga-region: add sysfs interface Alan Tull <delicious.quinoa@gmail.com> - 2017-02-15 18:50 +0100
        Re: [RFC 7/8] fpga-region: add sysfs interface Moritz Fischer <moritz.fischer@ettus.com> - 2017-02-15 19:00 +0100
        Re: [RFC 7/8] fpga-region: add sysfs interface Jason Gunthorpe <jgunthorpe@obsidianresearch.com> - 2017-02-15 19:10 +0100
          Re: [RFC 7/8] fpga-region: add sysfs interface Alan Tull <delicious.quinoa@gmail.com> - 2017-02-15 19:30 +0100
            Re: [RFC 7/8] fpga-region: add sysfs interface Moritz Fischer <moritz.fischer@ettus.com> - 2017-02-15 19:40 +0100
            Re: [RFC 7/8] fpga-region: add sysfs interface Jason Gunthorpe <jgunthorpe@obsidianresearch.com> - 2017-02-15 20:50 +0100
              Re: [RFC 7/8] fpga-region: add sysfs interface Alan Tull <delicious.quinoa@gmail.com> - 2017-02-16 00:00 +0100
                Re: [RFC 7/8] fpga-region: add sysfs interface Jason Gunthorpe <jgunthorpe@obsidianresearch.com> - 2017-02-16 00:10 +0100
            Re: [RFC 7/8] fpga-region: add sysfs interface matthew.gerlach@linux.intel.com - 2017-02-15 21:10 +0100
              Re: [RFC 7/8] fpga-region: add sysfs interface Jason Gunthorpe <jgunthorpe@obsidianresearch.com> - 2017-02-15 21:40 +0100
                Re: [RFC 7/8] fpga-region: add sysfs interface Moritz Fischer <moritz.fischer@ettus.com> - 2017-02-15 22:00 +0100
                  Re: [RFC 7/8] fpga-region: add sysfs interface Jason Gunthorpe <jgunthorpe@obsidianresearch.com> - 2017-02-15 22:20 +0100
                    Re: [RFC 7/8] fpga-region: add sysfs interface Moritz Fischer <moritz.fischer@ettus.com> - 2017-02-15 22:40 +0100
                      Re: [RFC 7/8] fpga-region: add sysfs interface Alan Tull <delicious.quinoa@gmail.com> - 2017-02-15 23:50 +0100
                        Re: [RFC 7/8] fpga-region: add sysfs interface Moritz Fischer <moritz.fischer@ettus.com> - 2017-02-16 01:20 +0100
                          Re: [RFC 7/8] fpga-region: add sysfs interface Alan Tull <delicious.quinoa@gmail.com> - 2017-02-16 18:50 +0100
                            Re: [RFC 7/8] fpga-region: add sysfs interface Jason Gunthorpe <jgunthorpe@obsidianresearch.com> - 2017-02-16 19:00 +0100
                              Re: [RFC 7/8] fpga-region: add sysfs interface Moritz Fischer <moritz.fischer@ettus.com> - 2017-02-16 19:20 +0100
                  Re: [RFC 7/8] fpga-region: add sysfs interface Yves Vandervennet <yves.vandervennet@linux.intel.com> - 2017-02-17 23:30 +0100
                    Re: [RFC 7/8] fpga-region: add sysfs interface Moritz Fischer <moritz.fischer@ettus.com> - 2017-02-18 03:40 +0100
                      RE: [RFC 7/8] fpga-region: add sysfs interface "Nadathur, Sundar" <sundar.nadathur@intel.com> - 2017-02-18 13:50 +0100
                        Re: [RFC 7/8] fpga-region: add sysfs interface Alan Tull <delicious.quinoa@gmail.com> - 2017-02-18 21:20 +0100
                          Re: [RFC 7/8] fpga-region: add sysfs interface Moritz Fischer <moritz.fischer@ettus.com> - 2017-02-18 22:00 +0100
                            Re: [RFC 7/8] fpga-region: add sysfs interface Alan Tull <delicious.quinoa@gmail.com> - 2017-02-19 16:10 +0100
                              Re: [RFC 7/8] fpga-region: add sysfs interface Alan Tull <delicious.quinoa@gmail.com> - 2017-02-20 00:20 +0100
                                Re: [RFC 7/8] fpga-region: add sysfs interface Moritz Fischer <moritz.fischer@ettus.com> - 2017-02-21 01:00 +0100
                                  Re: [RFC 7/8] fpga-region: add sysfs interface Alan Tull <delicious.quinoa@gmail.com> - 2017-02-21 19:40 +0100
                                    RE: [RFC 7/8] fpga-region: add sysfs interface "Nadathur, Sundar" <sundar.nadathur@intel.com> - 2017-02-22 04:20 +0100
                                      Re: [RFC 7/8] fpga-region: add sysfs interface Moritz Fischer <moritz.fischer@ettus.com> - 2017-02-22 04:50 +0100
                                        Re: [RFC 7/8] fpga-region: add sysfs interface Jason Gunthorpe <jgunthorpe@obsidianresearch.com> - 2017-02-22 06:20 +0100
                                          Re: [RFC 7/8] fpga-region: add sysfs interface Moritz Fischer <moritz.fischer@ettus.com> - 2017-02-22 06:40 +0100
                                            RE: [RFC 7/8] fpga-region: add sysfs interface "Nadathur, Sundar" <sundar.nadathur@intel.com> - 2017-02-22 06:50 +0100
                                              Re: [RFC 7/8] fpga-region: add sysfs interface Moritz Fischer <moritz.fischer@ettus.com> - 2017-02-22 07:10 +0100
                                                Re: [RFC 7/8] fpga-region: add sysfs interface Jason Gunthorpe <jgunthorpe@obsidianresearch.com> - 2017-02-22 17:50 +0100
                                                  Re: [RFC 7/8] fpga-region: add sysfs interface Moritz Fischer <moritz.fischer@ettus.com> - 2017-02-22 19:00 +0100
                                                    Re: [RFC 7/8] fpga-region: add sysfs interface Jason Gunthorpe <jgunthorpe@obsidianresearch.com> - 2017-02-22 19:00 +0100
                                                      Re: [RFC 7/8] fpga-region: add sysfs interface Moritz Fischer <moritz.fischer@ettus.com> - 2017-02-22 19:00 +0100
                                            Re: [RFC 7/8] fpga-region: add sysfs interface Alan Tull <delicious.quinoa@gmail.com> - 2017-02-22 17:40 +0100
                                              Re: [RFC 7/8] fpga-region: add sysfs interface Moritz Fischer <moritz.fischer@ettus.com> - 2017-02-22 17:50 +0100
                                                Re: [RFC 7/8] fpga-region: add sysfs interface Alan Tull <delicious.quinoa@gmail.com> - 2017-02-22 18:00 +0100
                                    Re: [RFC 7/8] fpga-region: add sysfs interface Moritz Fischer <moritz.fischer@ettus.com> - 2017-02-28 00:00 +0100
                                      Re: [RFC 7/8] fpga-region: add sysfs interface matthew.gerlach@linux.intel.com - 2017-02-28 07:30 +0100
                                    Re: [RFC 7/8] fpga-region: add sysfs interface Alan Tull <delicious.quinoa@gmail.com> - 2017-02-28 13:40 +0100
          Re: [RFC 7/8] fpga-region: add sysfs interface Anatolij Gustschin <agust@denx.de> - 2017-02-15 22:30 +0100

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →


#1583779

FromYves Vandervennet <yves.vandervennet@linux.intel.com>
Date2017-02-17 23:30 +0100
Message-ID<tbZcm-1Pb-9@gated-at.bofh.it>
In reply to#1581653
Moritz,

  whatever solution we decide to go with has to work with other OS'es. The 
last thing we want to do is to have wrappers that are Linux specific.

Yves

On Wed, 15 Feb 2017, Moritz Fischer wrote:

>>Hi Jason,
>>
>>On Wed, Feb 15, 2017 at 12:37 PM, Jason Gunthorpe
>><jgunthorpe@obsidianresearch.com> wrote:
>>> On Wed, Feb 15, 2017 at 12:07:15PM -0800, matthew.gerlach@linux.intel.com wrote:
>>>
>>>> The format of the meta data associated with a fpga bitstream is certainly a
>>>> subject on its own.  HTTP style plain text is definately easy to understand
>>>> and more importantly it is extendable.  On the other hand, it seems
>>>> dangerous to be doing a lot of string parsing in the kernel.
>>>
>>> It is fairly close to binary parsing.. The process is
>>>
>>> - Find the first occurance of \n\n, must be less than XX bytes
>>> - Memcpy that from the sg list into a linear buffer
>>> - Replace all \n with \0
>>>
>>> To access a key:
>>> - Case insensitive search for START + "Key: " or \0 + "Key: "
>>> - Return as a string the part after the match
>>>
>>> This isn't the sort of string parsing that typically gets you into
>>> trouble. If we can't code the above correctly then we will screw up
>>> safe binary parsing of strings too :)
>>
>>Well I don't know ;-) With something fdt based we already have parsers there,
>>compilers are already in tree. I'll take another look at the u-boot
>>code, I think their
>>FIT (Flattened Image Tree) would be a fairly good match for what we're
>>trying to do.
>>
>>Cheers,
>>Moritz
>>--
>>To unsubscribe from this list: send the line "unsubscribe linux-fpga" in
>>the body of a message to majordomo@vger.kernel.org
>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>

[toc] | [prev] | [next] | [standalone]


#1583836

FromMoritz Fischer <moritz.fischer@ettus.com>
Date2017-02-18 03:40 +0100
Message-ID<tc36h-4hL-3@gated-at.bofh.it>
In reply to#1583779
On Fri, Feb 17, 2017 at 04:28:37PM -0600, Yves Vandervennet wrote:
> Moritz,
> 
>   whatever solution we decide to go with has to work with other OS'es. The 
> last thing we want to do is to have wrappers that are Linux specific.

I do agree that we should make sure the format is reasonably well
documented. In my earlier email I pointed out several projects
successfully integrating libfdt.
There's nothing Linux specific about libfdt. FreeBSD uses it, U-Boot,
Qemu ...

I know nothing about how windows kernel development works, but I assume
however one goes about making FPGA programming work there, someone will
most likely have to write a kernel mode driver to take the job of the
fpga-mgr framework.
I assume this will be written in C or C++ or whatever people use these
days for kernel development so pulling in libfdt shouldn't be too hard
if we were to try that.

To be clear:
I did not suggest fdt to make it hard for other OSs, or because this is
my personal pet project. I think we're more likely to get it right by
reusing an existing format, with parsers that other people already
successfully use. It does not have to be fdt (I suggested that because
that was around), but  I do think we certainly can do better than
HTTP-eque plaintext headers.

Thanks,

Moritz

[toc] | [prev] | [next] | [standalone]


#1583892

From"Nadathur, Sundar" <sundar.nadathur@intel.com>
Date2017-02-18 13:50 +0100
Message-ID<tccCB-1Ya-5@gated-at.bofh.it>
In reply to#1583836
On February 17, 2017 6:30 PM, Moritz Fischer wrote:
<quote>

On Fri, Feb 17, 2017 at 04:28:37PM -0600, Yves Vandervennet wrote:
> Moritz,
> 
>   whatever solution we decide to go with has to work with other OS'es. 
> The last thing we want to do is to have wrappers that are Linux specific.

I do agree that we should make sure the format is reasonably well documented. In my earlier email I pointed out several projects successfully integrating libfdt.
There's nothing Linux specific about libfdt. FreeBSD uses it, U-Boot, Qemu ...

I know nothing about how windows kernel development works, but I assume however one goes about making FPGA programming work there, someone will most likely have to write a kernel mode driver to take the job of the fpga-mgr framework.
I assume this will be written in C or C++ or whatever people use these days for kernel development so pulling in libfdt shouldn't be too hard if we were to try that.

To be clear:
I did not suggest fdt to make it hard for other OSs, or because this is my personal pet project. I think we're more likely to get it right by reusing an existing format, with parsers that other people already successfully use. It does not have to be fdt (I suggested that because that was around), but  I do think we certainly can do better than HTTP-eque plaintext headers.

Thanks,

Moritz
--
To unsubscribe from this list: send the line "unsubscribe linux-fpga" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html
</endquote>

Hi all,
   Interesting discussion. The discussion so far has brought out many concerns such as OS independence. There is an existing format, well-known to developers, with widespread support, and which is quite extensible: Type-Length-Value triples.

To elaborate, a TLV-based format has many advantages:
* It is highly extensible in many ways
   -- You can express structures and arrays using TLVs. Our needs right now may seem limited but requirements grow over time.
   -- The space of Type values can be decomposed into standard pre-defined values that are in upstreamed code, and possibly experimental or feature-specific values.
   -- Forward compatibility: We can write parsers that can skip unexpected type values, thus allowing old parsers to work with new additions. With some tweaks, old parsers can also reject unexpected values in some ranges while accepting them in other ranges.
* It is OS-independent.
* It can be easily parsed, in kernel or user space.
* It can be validated, in terms of Type values, acceptable lengths, etc.

It  is not directly human-readable but that can be easily addressed with a tool that parses TLVs. 

Compared to some other proposals:
* Compared to DTs, TLVs are OS-independent.
* Compared to strings as key-value pairs, TLVs can express structures/arrays, can be validated, etc. 

So, I suggest we use TLVs to express metadata in image files.

Thank you very much,
Sundar Nadathur 

[toc] | [prev] | [next] | [standalone]


#1583966

FromAlan Tull <delicious.quinoa@gmail.com>
Date2017-02-18 21:20 +0100
Message-ID<tcjE6-6mv-3@gated-at.bofh.it>
In reply to#1583892
On Sat, Feb 18, 2017 at 6:45 AM, Nadathur, Sundar
<sundar.nadathur@intel.com> wrote:

> Hi all,
>    Interesting discussion. The discussion so far has brought out many concerns such as OS independence. There is an existing format, well-known to developers, with widespread support, and which is quite extensible: Type-Length-Value triples.
>
> To elaborate, a TLV-based format has many advantages:
> * It is highly extensible in many ways
>    -- You can express structures and arrays using TLVs. Our needs right now may seem limited but requirements grow over time.
>    -- The space of Type values can be decomposed into standard pre-defined values that are in upstreamed code, and possibly experimental or feature-specific values.
>    -- Forward compatibility: We can write parsers that can skip unexpected type values, thus allowing old parsers to work with new additions. With some tweaks, old parsers can also reject unexpected values in some ranges while accepting them in other ranges.
> * It is OS-independent.
> * It can be easily parsed, in kernel or user space.
> * It can be validated, in terms of Type values, acceptable lengths, etc.
>
> It  is not directly human-readable but that can be easily addressed with a tool that parses TLVs.
>
> Compared to some other proposals:
> * Compared to DTs, TLVs are OS-independent.
> * Compared to strings as key-value pairs, TLVs can express structures/arrays, can be validated, etc.
>
> So, I suggest we use TLVs to express metadata in image files.
>
> Thank you very much,
> Sundar Nadathur

Hi Sundar,

IIUC, each field is position dependent.  One of the strengths of key
value pairs is
that any key can be added in any order and ones that aren't applicable for
a particular architecture or use case can be left out.  If the header
is position
dependent, it becomes less flexible.  Once a field is added, we are
stuck with it
forever unless we drop it in some incremented version of the header format.

Alan

[toc] | [prev] | [next] | [standalone]


#1583974

FromMoritz Fischer <moritz.fischer@ettus.com>
Date2017-02-18 22:00 +0100
Message-ID<tckgN-6HX-7@gated-at.bofh.it>
In reply to#1583966
On Sat, Feb 18, 2017 at 02:10:43PM -0600, Alan Tull wrote:
> On Sat, Feb 18, 2017 at 6:45 AM, Nadathur, Sundar
> <sundar.nadathur@intel.com> wrote:
> 
> > Hi all,
> >    Interesting discussion. The discussion so far has brought out many concerns such as OS independence. There is an existing format, well-known to developers, with widespread support, and which is quite extensible: Type-Length-Value triples.
> >
> > To elaborate, a TLV-based format has many advantages:
> > * It is highly extensible in many ways
> >    -- You can express structures and arrays using TLVs. Our needs right now may seem limited but requirements grow over time.
> >    -- The space of Type values can be decomposed into standard pre-defined values that are in upstreamed code, and possibly experimental or feature-specific values.
> >    -- Forward compatibility: We can write parsers that can skip unexpected type values, thus allowing old parsers to work with new additions. With some tweaks, old parsers can also reject unexpected values in some ranges while accepting them in other ranges.
> > * It is OS-independent.
> > * It can be easily parsed, in kernel or user space.

Are there other users of the format? I have to admit I didn't look very
long, but couldn't find any libs / existing code at a first glance.

> > * It can be validated, in terms of Type values, acceptable lengths, etc.
> >
> > It  is not directly human-readable but that can be easily addressed with a tool that parses TLVs.
> >
> > Compared to some other proposals:
> > * Compared to DTs, TLVs are OS-independent.

That's just alternative facts here. Just because Linux uses fdt for
devicetree blobs it is *not* OS dependent. There are several (see
last email) non-Linux users of fdt / libfdt.

Thanks,

Moritz

[toc] | [prev] | [next] | [standalone]


#1584156

FromAlan Tull <delicious.quinoa@gmail.com>
Date2017-02-19 16:10 +0100
Message-ID<tcBhE-to-19@gated-at.bofh.it>
In reply to#1583974
On Sat, Feb 18, 2017 at 2:45 PM, Moritz Fischer
<moritz.fischer@ettus.com> wrote:
> On Sat, Feb 18, 2017 at 02:10:43PM -0600, Alan Tull wrote:
>> On Sat, Feb 18, 2017 at 6:45 AM, Nadathur, Sundar
>> <sundar.nadathur@intel.com> wrote:
>>
>> > Hi all,
>> >    Interesting discussion. The discussion so far has brought out many concerns such as OS independence. There is an existing format, well-known to developers, with widespread support, and which is quite extensible: Type-Length-Value triples.
>> >
>> > To elaborate, a TLV-based format has many advantages:
>> > * It is highly extensible in many ways
>> >    -- You can express structures and arrays using TLVs. Our needs right now may seem limited but requirements grow over time.

Device tree can express arrays.

>> >    -- The space of Type values can be decomposed into standard pre-defined values that are in upstreamed code, and possibly experimental or feature-specific values.
>> >    -- Forward compatibility: We can write parsers that can skip unexpected type values, thus allowing old parsers to work with new additions. With some tweaks, old parsers can also reject unexpected values in some ranges while accepting them in other ranges.
>> > * It is OS-independent.
>> > * It can be easily parsed, in kernel or user space.
>
> Are there other users of the format? I have to admit I didn't look very
> long, but couldn't find any libs / existing code at a first glance.

Is there a standard you are looking at?  Have you seen any use of TLV's
in the Linux kernel you could point to?

>
>> > * It can be validated, in terms of Type values, acceptable lengths, etc.
>> >
>> > It  is not directly human-readable but that can be easily addressed with a tool that parses TLVs.
>> >
>> > Compared to some other proposals:
>> > * Compared to DTs, TLVs are OS-independent.
>
> That's just alternative facts here. Just because Linux uses fdt for
> devicetree blobs it is *not* OS dependent. There are several (see
> last email) non-Linux users of fdt / libfdt.
>
> Thanks,
>
> Moritz

It is worth repeating that libdtc is GPL/BSD with the intent of
allowing proprietary code to use libdtc.  So license shouldn't be a barrier.

Using device tree in the header would give us a way of doing enumeration at
least for Linux, not sure if that kind of info can be used in Windows
in some way.

Alan

[toc] | [prev] | [next] | [standalone]


#1584277

FromAlan Tull <delicious.quinoa@gmail.com>
Date2017-02-20 00:20 +0100
Message-ID<tcIVP-5aM-9@gated-at.bofh.it>
In reply to#1584156
On Sun, Feb 19, 2017 at 9:00 AM, Alan Tull <delicious.quinoa@gmail.com> wrote:
> On Sat, Feb 18, 2017 at 2:45 PM, Moritz Fischer
> <moritz.fischer@ettus.com> wrote:
>> On Sat, Feb 18, 2017 at 02:10:43PM -0600, Alan Tull wrote:
>>> On Sat, Feb 18, 2017 at 6:45 AM, Nadathur, Sundar
>>> <sundar.nadathur@intel.com> wrote:
>>>
>>> > Hi all,
>>> >    Interesting discussion. The discussion so far has brought out many concerns such as OS independence. There is an existing format, well-known to developers, with widespread support, and which is quite extensible: Type-Length-Value triples.
>>> >
>>> > To elaborate, a TLV-based format has many advantages:
>>> > * It is highly extensible in many ways
>>> >    -- You can express structures and arrays using TLVs. Our needs right now may seem limited but requirements grow over time.
>
> Device tree can express arrays.
>
>>> >    -- The space of Type values can be decomposed into standard pre-defined values that are in upstreamed code, and possibly experimental or feature-specific values.
>>> >    -- Forward compatibility: We can write parsers that can skip unexpected type values, thus allowing old parsers to work with new additions. With some tweaks, old parsers can also reject unexpected values in some ranges while accepting them in other ranges.
>>> > * It is OS-independent.
>>> > * It can be easily parsed, in kernel or user space.
>>
>> Are there other users of the format? I have to admit I didn't look very
>> long, but couldn't find any libs / existing code at a first glance.
>
> Is there a standard you are looking at?  Have you seen any use of TLV's
> in the Linux kernel you could point to?
>
>>
>>> > * It can be validated, in terms of Type values, acceptable lengths, etc.
>>> >
>>> > It  is not directly human-readable but that can be easily addressed with a tool that parses TLVs.
>>> >
>>> > Compared to some other proposals:
>>> > * Compared to DTs, TLVs are OS-independent.
>>
>> That's just alternative facts here. Just because Linux uses fdt for
>> devicetree blobs it is *not* OS dependent. There are several (see
>> last email) non-Linux users of fdt / libfdt.
>>
>> Thanks,
>>
>> Moritz
>
> It is worth repeating that libdtc is GPL/BSD with the intent of
> allowing proprietary code to use libdtc.  So license shouldn't be a barrier.
>
> Using device tree in the header would give us a way of doing enumeration at
> least for Linux, not sure if that kind of info can be used in Windows
> in some way.

Actually, enumeration is the only advantage I see with DT.

Currently I like key/value pairs because they are easily implemented
and expandable without being rigid in any way.

If we use key/value pairs, we could pass in child device info
in one of the keys.  It could be either a device tree overlay or an
ACPI overlay.  Or could just be left out.  So platforms that
aren't already using DT wouldn't have to.  Platforms that
are have a smooth road to enumeration.

Alan

>
> Alan

[toc] | [prev] | [next] | [standalone]


#1584969

FromMoritz Fischer <moritz.fischer@ettus.com>
Date2017-02-21 01:00 +0100
Message-ID<td626-2RA-13@gated-at.bofh.it>
In reply to#1584277
Hi Alan,

On Sun, Feb 19, 2017 at 3:16 PM, Alan Tull <delicious.quinoa@gmail.com> wrote:
> On Sun, Feb 19, 2017 at 9:00 AM, Alan Tull <delicious.quinoa@gmail.com> wrote:
>> On Sat, Feb 18, 2017 at 2:45 PM, Moritz Fischer
>> <moritz.fischer@ettus.com> wrote:
>>> On Sat, Feb 18, 2017 at 02:10:43PM -0600, Alan Tull wrote:
>>>> On Sat, Feb 18, 2017 at 6:45 AM, Nadathur, Sundar
>>>> <sundar.nadathur@intel.com> wrote:
>>>>
>>>> > Hi all,
>>>> >    Interesting discussion. The discussion so far has brought out many concerns such as OS independence. There is an existing format, well-known to developers, with widespread support, and which is quite extensible: Type-Length-Value triples.
>>>> >
>>>> > To elaborate, a TLV-based format has many advantages:
>>>> > * It is highly extensible in many ways
>>>> >    -- You can express structures and arrays using TLVs. Our needs right now may seem limited but requirements grow over time.
>>
>> Device tree can express arrays.
>>
>>>> >    -- The space of Type values can be decomposed into standard pre-defined values that are in upstreamed code, and possibly experimental or feature-specific values.
>>>> >    -- Forward compatibility: We can write parsers that can skip unexpected type values, thus allowing old parsers to work with new additions. With some tweaks, old parsers can also reject unexpected values in some ranges while accepting them in other ranges.
>>>> > * It is OS-independent.
>>>> > * It can be easily parsed, in kernel or user space.
>>>
>>> Are there other users of the format? I have to admit I didn't look very
>>> long, but couldn't find any libs / existing code at a first glance.
>>
>> Is there a standard you are looking at?  Have you seen any use of TLV's
>> in the Linux kernel you could point to?
>>
>>>
>>>> > * It can be validated, in terms of Type values, acceptable lengths, etc.
>>>> >
>>>> > It  is not directly human-readable but that can be easily addressed with a tool that parses TLVs.
>>>> >
>>>> > Compared to some other proposals:
>>>> > * Compared to DTs, TLVs are OS-independent.
>>>
>>> That's just alternative facts here. Just because Linux uses fdt for
>>> devicetree blobs it is *not* OS dependent. There are several (see
>>> last email) non-Linux users of fdt / libfdt.
>>>
>>> Thanks,
>>>
>>> Moritz
>>
>> It is worth repeating that libdtc is GPL/BSD with the intent of
>> allowing proprietary code to use libdtc.  So license shouldn't be a barrier.
>>
>> Using device tree in the header would give us a way of doing enumeration at
>> least for Linux, not sure if that kind of info can be used in Windows
>> in some way.
>
> Actually, enumeration is the only advantage I see with DT.

Which seems to some point a separate issue to passing in image
specific info such as
encrypted or not, compressed or not or build info metadata.

So I think in general we can still separate this out into:
- Image specific values
- Reconfiguration specific values

> Currently I like key/value pairs because they are easily implemented
> and expandable without being rigid in any way.
>
> If we use key/value pairs, we could pass in child device info
> in one of the keys.  It could be either a device tree overlay or an
> ACPI overlay.  Or could just be left out.  So platforms that
> aren't already using DT wouldn't have to.  Platforms that
> are have a smooth road to enumeration.

I'm not sure if you can bundle up enumeration info *with* the image
since you might e.g.
load the same image (i.e. same header) into different FPGAs and the
required update to
the kernel state, i.e. live tree or ACPI would depend entirely on
which FPGA you loaded
the image into w.r.t busses it's connected to etc.

I do think this info cannot be image specific, but needs to be passed
in via something
external such as a dt overlay.

Cheers,

Moritz

[toc] | [prev] | [next] | [standalone]


#1585611

FromAlan Tull <delicious.quinoa@gmail.com>
Date2017-02-21 19:40 +0100
Message-ID<tdnvY-6me-25@gated-at.bofh.it>
In reply to#1584969
On Mon, Feb 20, 2017 at 5:49 PM, Moritz Fischer
<moritz.fischer@ettus.com> wrote:
> Hi Alan,
>
> On Sun, Feb 19, 2017 at 3:16 PM, Alan Tull <delicious.quinoa@gmail.com> wrote:
>> On Sun, Feb 19, 2017 at 9:00 AM, Alan Tull <delicious.quinoa@gmail.com> wrote:
>>> On Sat, Feb 18, 2017 at 2:45 PM, Moritz Fischer
>>> <moritz.fischer@ettus.com> wrote:
>>>> On Sat, Feb 18, 2017 at 02:10:43PM -0600, Alan Tull wrote:
>>>>> On Sat, Feb 18, 2017 at 6:45 AM, Nadathur, Sundar
>>>>> <sundar.nadathur@intel.com> wrote:
>>>>>
>>>>> > Hi all,
>>>>> >    Interesting discussion. The discussion so far has brought out many concerns such as OS independence. There is an existing format, well-known to developers, with widespread support, and which is quite extensible: Type-Length-Value triples.
>>>>> >
>>>>> > To elaborate, a TLV-based format has many advantages:
>>>>> > * It is highly extensible in many ways
>>>>> >    -- You can express structures and arrays using TLVs. Our needs right now may seem limited but requirements grow over time.
>>>
>>> Device tree can express arrays.
>>>
>>>>> >    -- The space of Type values can be decomposed into standard pre-defined values that are in upstreamed code, and possibly experimental or feature-specific values.
>>>>> >    -- Forward compatibility: We can write parsers that can skip unexpected type values, thus allowing old parsers to work with new additions. With some tweaks, old parsers can also reject unexpected values in some ranges while accepting them in other ranges.
>>>>> > * It is OS-independent.
>>>>> > * It can be easily parsed, in kernel or user space.
>>>>
>>>> Are there other users of the format? I have to admit I didn't look very
>>>> long, but couldn't find any libs / existing code at a first glance.
>>>
>>> Is there a standard you are looking at?  Have you seen any use of TLV's
>>> in the Linux kernel you could point to?
>>>
>>>>
>>>>> > * It can be validated, in terms of Type values, acceptable lengths, etc.
>>>>> >
>>>>> > It  is not directly human-readable but that can be easily addressed with a tool that parses TLVs.
>>>>> >
>>>>> > Compared to some other proposals:
>>>>> > * Compared to DTs, TLVs are OS-independent.
>>>>
>>>> That's just alternative facts here. Just because Linux uses fdt for
>>>> devicetree blobs it is *not* OS dependent. There are several (see
>>>> last email) non-Linux users of fdt / libfdt.
>>>>
>>>> Thanks,
>>>>
>>>> Moritz
>>>
>>> It is worth repeating that libdtc is GPL/BSD with the intent of
>>> allowing proprietary code to use libdtc.  So license shouldn't be a barrier.
>>>
>>> Using device tree in the header would give us a way of doing enumeration at
>>> least for Linux, not sure if that kind of info can be used in Windows
>>> in some way.
>>
>> Actually, enumeration is the only advantage I see with DT.
>
> Which seems to some point a separate issue to passing in image
> specific info such as
> encrypted or not, compressed or not or build info metadata.
>
> So I think in general we can still separate this out into:
> - Image specific values
> - Reconfiguration specific values
>
>> Currently I like key/value pairs because they are easily implemented
>> and expandable without being rigid in any way.
>>
>> If we use key/value pairs, we could pass in child device info
>> in one of the keys.  It could be either a device tree overlay or an
>> ACPI overlay.  Or could just be left out.  So platforms that
>> aren't already using DT wouldn't have to.  Platforms that
>> are have a smooth road to enumeration.
>
> I'm not sure if you can bundle up enumeration info *with* the image
> since you might e.g.
> load the same image (i.e. same header) into different FPGAs and the
> required update to
> the kernel state, i.e. live tree or ACPI would depend entirely on
> which FPGA you loaded
> the image into w.r.t busses it's connected to etc.
>
> I do think this info cannot be image specific, but needs to be passed
> in via something
> external such as a dt overlay.

Yes, I think you are right about that.

Thanks!
Alan

>
> Cheers,
>
> Moritz

[toc] | [prev] | [next] | [standalone]


#1585901

From"Nadathur, Sundar" <sundar.nadathur@intel.com>
Date2017-02-22 04:20 +0100
Message-ID<tdvDc-3G6-3@gated-at.bofh.it>
In reply to#1585611

> -----Original Message-----
> From: Alan Tull [mailto:delicious.quinoa@gmail.com]
> Sent: Tuesday, February 21, 2017 10:33 AM
> To: Moritz Fischer <moritz.fischer@ettus.com>
> Cc: Nadathur, Sundar <sundar.nadathur@intel.com>; Yves Vandervennet
> <yves.vandervennet@linux.intel.com>; Jason Gunthorpe
> <jgunthorpe@obsidianresearch.com>; matthew.gerlach@linux.intel.com;
> linux-kernel <linux-kernel@vger.kernel.org>; linux-fpga@vger.kernel.org;
> Marek VaĊĦut <marex@denx.de>
> Subject: Re: [RFC 7/8] fpga-region: add sysfs interface
> 
> On Mon, Feb 20, 2017 at 5:49 PM, Moritz Fischer
> <moritz.fischer@ettus.com> wrote:
> > Hi Alan,
> >
> > On Sun, Feb 19, 2017 at 3:16 PM, Alan Tull <delicious.quinoa@gmail.com>
> wrote:
> >> On Sun, Feb 19, 2017 at 9:00 AM, Alan Tull <delicious.quinoa@gmail.com>
> wrote:
> >>> On Sat, Feb 18, 2017 at 2:45 PM, Moritz Fischer
> >>> <moritz.fischer@ettus.com> wrote:
> >>>> On Sat, Feb 18, 2017 at 02:10:43PM -0600, Alan Tull wrote:
> >>>>> On Sat, Feb 18, 2017 at 6:45 AM, Nadathur, Sundar
> >>>>> <sundar.nadathur@intel.com> wrote:
> >>>>>
> >>>>> > Hi all,
> >>>>> >    Interesting discussion. The discussion so far has brought out many
> concerns such as OS independence. There is an existing format, well-known
> to developers, with widespread support, and which is quite extensible: Type-
> Length-Value triples.
> >>> [...]
> >>>> Are there other users of the format? I have to admit I didn't look
> >>>> very long, but couldn't find any libs / existing code at a first glance.
> >>>
> >>> Is there a standard you are looking at?  Have you seen any use of
> >>> TLV's in the Linux kernel you could point to?	

Here are some examples of TLVs in the Linux kernel:
http://lxr.free-electrons.com/source/net/ipv6/exthdrs.c <-- includes TLV parsing code
http://lxr.free-electrons.com/source/drivers/net/ethernet/broadcom/bnx2x/bnx2x_vfpf.c 
http://lxr.free-electrons.com/source/include/sound/tlv.h 

In addition, some protocols like LLDP are defined in terms of TLVs. E.g.
http://lxr.free-electrons.com/source/drivers/net/ethernet/intel/i40e/i40e_dcb.h?v=4.4 

> >> On Sun, Feb 19, 2017 at 9:00 AM, Alan Tull <delicious.quinoa@gmail.com>
> wrote:
> >>> It is worth repeating that libdtc is GPL/BSD with the intent of
> >>> allowing proprietary code to use libdtc.  So license shouldn't be a barrier.
It is better to check with Windows folks before concluding this. 

> >> If we use key/value pairs, we could pass in child device info in one
> >> of the keys.  It could be either a device tree overlay or an ACPI
> >> overlay.  Or could just be left out.  So platforms that aren't
> >> already using DT wouldn't have to.  Platforms that are have a smooth
> >> road to enumeration.
> >
> > I'm not sure if you can bundle up enumeration info *with* the image
> > since you might e.g.
> > load the same image (i.e. same header) into different FPGAs and the
> > required update to the kernel state, i.e. live tree or ACPI would
> > depend entirely on which FPGA you loaded the image into w.r.t busses
> > it's connected to etc.
> >
> > I do think this info cannot be image specific, but needs to be passed
> > in via something external such as a dt overlay.
> 
> Yes, I think you are right about that.
> 
> Thanks!
> Alan

I agree that device enumeration should be separated out from the metadata format considerations. 

I got some feedback that not everybody may be familiar with TLVs. To make the proposal more clear and specific, let me add more information here.
* We represent every datum of interest with its Type (which indicates what it is), a Length (how many bytes it takes) and a Value (its actual value, taking as many bytes as the Length field indicates.)  
* The exact lengths of the Type and Length fields are up to us, but let us say they are 4 bytes each, for concreteness. As an example, say we want to express the function in the FPGA (crypto, compress, etc.) as a UUID (128 bits long) compliant with RFC 4122. We could have a Type of say 0x00000050 (4 bytes in all) to indicate Function UUIDs, and a Length field of 0x00000010 (16 bytes) and a value of say 3d8814d8-4ecc-4030-8415-0dea4e5e829a . 
* A Type may indicate that its value is another TLV, thus allowing nested TLVs. The nested TLV may be an array of TLVs (all of same Type) or a structure (TLVs of different Types). 

With a Key-Value Pair, if the parser comes across an unknown key (such as when an old parser comes across newer metadata), it would not know how to skip that KVP and move onto the next one. With TLVs, that flexibility is built in. Further, we can use bits in the Type field to indicate that it is mandatory, i.e., if the parser does not understand it, it should error out rather than skip it silently. This degree of forward compatibility is difficult to achieve with other formats. 

Thanks,
Sundar





[toc] | [prev] | [next] | [standalone]


#1585909

FromMoritz Fischer <moritz.fischer@ettus.com>
Date2017-02-22 04:50 +0100
Message-ID<tdw6d-3Ry-1@gated-at.bofh.it>
In reply to#1585901
Hi Sundar,

On Tue, Feb 21, 2017 at 7:13 PM, Nadathur, Sundar
<sundar.nadathur@intel.com> wrote:

>> >>> Is there a standard you are looking at?  Have you seen any use of
>> >>> TLV's in the Linux kernel you could point to?
>
> Here are some examples of TLVs in the Linux kernel:
> http://lxr.free-electrons.com/source/net/ipv6/exthdrs.c <-- includes TLV parsing code
> http://lxr.free-electrons.com/source/drivers/net/ethernet/broadcom/bnx2x/bnx2x_vfpf.c
> http://lxr.free-electrons.com/source/include/sound/tlv.h
>
> In addition, some protocols like LLDP are defined in terms of TLVs. E.g.
> http://lxr.free-electrons.com/source/drivers/net/ethernet/intel/i40e/i40e_dcb.h?v=4.4

Thanks for the examples, looks interesting. I'll do some reading.

>> >> On Sun, Feb 19, 2017 at 9:00 AM, Alan Tull <delicious.quinoa@gmail.com>
>> wrote:
>> >>> It is worth repeating that libdtc is GPL/BSD with the intent of
>> >>> allowing proprietary code to use libdtc.  So license shouldn't be a barrier.
> It is better to check with Windows folks before concluding this.

About whether they can link C code? Or whether BSD licensed code is an issue?

> I agree that device enumeration should be separated out from the metadata format considerations.
>
> I got some feedback that not everybody may be familiar with TLVs. To make the proposal more clear and specific, let me add more information here.
> * We represent every datum of interest with its Type (which indicates what it is), a Length (how many bytes it takes) and a Value (its actual value, taking as many bytes as the Length field indicates.)
> * The exact lengths of the Type and Length fields are up to us, but let us say they are 4 bytes each, for concreteness. As an example, say we want to express the function in the FPGA (crypto, compress, etc.) as a UUID (128 bits long) compliant with RFC 4122. We could have a Type of say 0x00000050 (4 bytes in all) to indicate Function UUIDs, and a Length field of 0x00000010 (16 bytes) and a value of say 3d8814d8-4ecc-4030-8415-0dea4e5e829a .
> * A Type may indicate that its value is another TLV, thus allowing nested TLVs. The nested TLV may be an array of TLVs (all of same Type) or a structure (TLVs of different Types).
>
> With a Key-Value Pair, if the parser comes across an unknown key (such as when an old parser comes across newer metadata), it would not know how to skip that KVP and move onto the next one. With TLVs, that flexibility is built in. Further, we can use bits in the Type field to indicate that it is mandatory, i.e., if the parser does not understand it, it should error out rather than skip it silently. This degree of forward compatibility is difficult to achieve with other formats.

fdt does this out of the box, too. So far I've seen nothing fdt
couldn't do (or doesn't do let's rather say).

Thanks for clarifying the TLV stuff,

Moritz

[toc] | [prev] | [next] | [standalone]


#1585924

FromJason Gunthorpe <jgunthorpe@obsidianresearch.com>
Date2017-02-22 06:20 +0100
Message-ID<tdxvj-4XN-3@gated-at.bofh.it>
In reply to#1585909
On Tue, Feb 21, 2017 at 07:49:19PM -0800, Moritz Fischer wrote:

> fdt does this out of the box, too. So far I've seen nothing fdt
> couldn't do (or doesn't do let's rather say).

tlv/fdt/http headers are all essentially exactly the same
thing. Key/value pairs with various encoding schemes.

I don't think we don't need a tree of data, so fdt is overkill.

tlv is not substantially easier to parse correctly than the
structured plain text headers.. It is just in binary so it can
represent binary-ish things better.

So far the only thing we know we need is a 'bool' for encrypted and a
stringish guid thing for partial reconfiguration.

The other stuff I've always used is pretty much all textual.

Jason

[toc] | [prev] | [next] | [standalone]


#1585932

FromMoritz Fischer <moritz.fischer@ettus.com>
Date2017-02-22 06:40 +0100
Message-ID<tdxOF-58U-1@gated-at.bofh.it>
In reply to#1585924
Hi all,

On Tue, Feb 21, 2017 at 9:12 PM, Jason Gunthorpe
<jgunthorpe@obsidianresearch.com> wrote:
> On Tue, Feb 21, 2017 at 07:49:19PM -0800, Moritz Fischer wrote:
>
>> fdt does this out of the box, too. So far I've seen nothing fdt
>> couldn't do (or doesn't do let's rather say).
>
> tlv/fdt/http headers are all essentially exactly the same
> thing. Key/value pairs with various encoding schemes.
>
> I don't think we don't need a tree of data, so fdt is overkill.
>
> tlv is not substantially easier to parse correctly than the
> structured plain text headers.. It is just in binary so it can
> represent binary-ish things better.

TLV Seems easy enough. To give an update, I played with fdt a bit to see
how far I get in half an hour. I got bool / int / strings to work
quite fast (~30mins).
Please disregard the horrible hackyness of this ...

For simplicity I stuck the header on top of my bitfile with:

<snip>
/dts-v1/;

/{
        description = "Test";
        compressed = <0>;
        encrypted = <1>;
};
</snip>

$ dtc -o header.dtb header.dts

$ cat header.dtb mybitfile.bin > /lib/firmware/bitfile_header.bin

+ static int __fpga_mgr_blob_to_image_info(const void *blob,
+                                          struct fpga_image_info *info)
+ {
+         int root_offset;
+         const char *desc;
+         const uint32_t *_compressed, *_encrypted;
+         int compressed, encrypted;
+
+         if (fdt_check_header(blob)) {
+                 pr_err("Invalid device tree blob header\n");
+                 return -EINVAL;
+         }
+
+         root_offset = fdt_path_offset(blob, "/");
+         if (root_offset < 0) {
+                 pr_err("Invalid root offset\n");
+                 return -EINVAL;
+         }
+
+         desc = fdt_getprop(blob, root_offset, "description", NULL);
+
+         _compressed = fdt_getprop(blob, root_offset, "compressed", NULL);
+         if (_compressed)
+                 compressed = fdt32_to_cpu(*_compressed);
+
+         _encrypted = fdt_getprop(blob, root_offset, "encrypted", NULL);
+         if (_encrypted)
+                 encrypted = fdt32_to_cpu(*_encrypted);
+
+         if (desc)
+                 pr_info("%s: description=%s\n", __func__, desc);
+
+         if (_encrypted && _compressed)
+                 pr_info("%s: compressed? %s encrypted? %s\n", __func__,
+                         compressed ? "Yes" : "No", encrypted ? "Yes" : "No");
+
+         return 0;
+ }

Which gave me:

<snip>

[   19.325182] fpga_manager fpga0: writing bitfile_header.bin to
Xilinx Zynq FPGA Manager
[   20.091222] __fpga_mgr_blob_to_image_info: description=Test
[   20.096730] __fpga_mgr_blob_to_image_info: compressed? No encrypted? Yes

</snip>

So I'm fairly convinced I can make this work, TVLs seem like it could work, too.

> So far the only thing we know we need is a 'bool' for encrypted and a
> stringish guid thing for partial reconfiguration.

Yeah, shouldn't be hard to implement either way.

Cheers,

Moritz

[toc] | [prev] | [next] | [standalone]


#1585939

From"Nadathur, Sundar" <sundar.nadathur@intel.com>
Date2017-02-22 06:50 +0100
Message-ID<tdxYm-5d9-3@gated-at.bofh.it>
In reply to#1585932
On February 21, 2017 9:39 PM, Moritz Fischer wrote:

> TLV Seems easy enough. To give an update, I played with fdt a bit to see how
> far I get in half an hour. I got bool / int / strings to work quite fast (~30mins).
> Please disregard the horrible hackyness of this ...
> [...]
> So I'm fairly convinced I can make this work, TVLs seem like it could work,
> too.
> 
> > So far the only thing we know we need is a 'bool' for encrypted and a
> > stringish guid thing for partial reconfiguration.

These things have a way of growing beyond their original anticipated needs. 

> Yeah, shouldn't be hard to implement either way.
> 
> Cheers,
> 
> Moritz

Say we upstream a metadata parser. Subsequently, an FPGA image is released with an additional metadata field that the upstreamed version does not handle. How will this be handled if the metadata were in FDT or KVP format?

Thanks,
Sundar

[toc] | [prev] | [next] | [standalone]


#1585942

FromMoritz Fischer <moritz.fischer@ettus.com>
Date2017-02-22 07:10 +0100
Message-ID<tdyhH-5EM-1@gated-at.bofh.it>
In reply to#1585939
On Tue, Feb 21, 2017 at 9:46 PM, Nadathur, Sundar
<sundar.nadathur@intel.com> wrote:
> On February 21, 2017 9:39 PM, Moritz Fischer wrote:
>
>> TLV Seems easy enough. To give an update, I played with fdt a bit to see how
>> far I get in half an hour. I got bool / int / strings to work quite fast (~30mins).
>> Please disregard the horrible hackyness of this ...
>> [...]
>> So I'm fairly convinced I can make this work, TVLs seem like it could work,
>> too.
>>
>> > So far the only thing we know we need is a 'bool' for encrypted and a
>> > stringish guid thing for partial reconfiguration.
>
> These things have a way of growing beyond their original anticipated needs.

True. But yeah, not sure about the requirement for a tree, maybe it is overkill.

> Say we upstream a metadata parser. Subsequently, an FPGA image is released with an additional metadata field that the upstreamed version does not handle. How will this be handled if the metadata were in FDT or KVP format?

The code above will gently ignore it, as I said I spent about half an hour on
writing that, just to prove to myself it can be done easily.
Logically I don't see anything wrong with ignoring features from the future.
But if one insisted one could make a compatibility number part of the
required properties I suppose and error out instead. There are examples
of optional properties in the devicetree parsing code in the kernel.

That being said older drivers / fpga-mgr  will not deal with newer features.
TLV / KV or whatever doesn't change this fact, or am I missing something?

Adding new properties to devicetrees is a well known exercise to cope with
newer versions or variations of hardware and happens all the time in the kernel.
Older kernels will just ignore them.

Thanks,

Moritz

[toc] | [prev] | [next] | [standalone]


#1586305

FromJason Gunthorpe <jgunthorpe@obsidianresearch.com>
Date2017-02-22 17:50 +0100
Message-ID<tdIh4-4BG-5@gated-at.bofh.it>
In reply to#1585942
On Tue, Feb 21, 2017 at 10:05:42PM -0800, Moritz Fischer wrote:

> That being said older drivers / fpga-mgr  will not deal with newer features.
> TLV / KV or whatever doesn't change this fact, or am I missing something?

Often a scheme will have an OPTIONAL and REQUIRED flag for each
value. If a REQUIRED value is present but the parser does not
support it then the parse fails.

For instance, we could mark Encrypted as required, and a zynq driver
that does not support the Encrypted tag would not load the bitfile
rather than try to load it wrongly.

This can be forced into any of the approaches with various levels of
hackery.

Jason

[toc] | [prev] | [next] | [standalone]


#1586353

FromMoritz Fischer <moritz.fischer@ettus.com>
Date2017-02-22 19:00 +0100
Message-ID<tdJmO-5l7-3@gated-at.bofh.it>
In reply to#1586305
Jason,

On Wed, Feb 22, 2017 at 8:44 AM, Jason Gunthorpe
<jgunthorpe@obsidianresearch.com> wrote:
> On Tue, Feb 21, 2017 at 10:05:42PM -0800, Moritz Fischer wrote:
>
>> That being said older drivers / fpga-mgr  will not deal with newer features.
>> TLV / KV or whatever doesn't change this fact, or am I missing something?
>
> Often a scheme will have an OPTIONAL and REQUIRED flag for each
> value. If a REQUIRED value is present but the parser does not
> support it then the parse fails.

Oh, of course. I'm a dorque :) Obvioiusly better customer experience like that.

> For instance, we could mark Encrypted as required, and a zynq driver
> that does not support the Encrypted tag would not load the bitfile
> rather than try to load it wrongly.

Funny you'd mention that. Thanks to your patch the zynq driver won't ;-)
I ran into that while testing, hehe.

> This can be forced into any of the approaches with various levels of
> hackery.

Yeah doesn't seem to hard,

Cheers

Moritz

[toc] | [prev] | [next] | [standalone]


#1586354

FromJason Gunthorpe <jgunthorpe@obsidianresearch.com>
Date2017-02-22 19:00 +0100
Message-ID<tdJmO-5l7-13@gated-at.bofh.it>
In reply to#1586353
On Wed, Feb 22, 2017 at 09:50:54AM -0800, Moritz Fischer wrote:

> > For instance, we could mark Encrypted as required, and a zynq driver
> > that does not support the Encrypted tag would not load the bitfile
> > rather than try to load it wrongly.
> 
> Funny you'd mention that. Thanks to your patch the zynq driver won't ;-)
> I ran into that while testing, hehe.

Really? You mean there is no sync word? That really surprises me..

Jason

[toc] | [prev] | [next] | [standalone]


#1586355

FromMoritz Fischer <moritz.fischer@ettus.com>
Date2017-02-22 19:00 +0100
Message-ID<tdJmO-5l7-21@gated-at.bofh.it>
In reply to#1586354
On Wed, Feb 22, 2017 at 9:54 AM, Jason Gunthorpe
<jgunthorpe@obsidianresearch.com> wrote:
> On Wed, Feb 22, 2017 at 09:50:54AM -0800, Moritz Fischer wrote:
>
>> > For instance, we could mark Encrypted as required, and a zynq driver
>> > that does not support the Encrypted tag would not load the bitfile
>> > rather than try to load it wrongly.
>>
>> Funny you'd mention that. Thanks to your patch the zynq driver won't ;-)
>> I ran into that while testing, hehe.
>
> Really? You mean there is no sync word? That really surprises me..

Oh wait, that might have been because I didn't drop the header, now
that I think about it,
it probably still has the sync word. I thought it might have been
encrypted (which makes no sense
of course ... )

Ignore the noise :D

Moritz

[toc] | [prev] | [next] | [standalone]


#1586304

FromAlan Tull <delicious.quinoa@gmail.com>
Date2017-02-22 17:40 +0100
Message-ID<tdI7o-4xK-19@gated-at.bofh.it>
In reply to#1585932
On Tue, Feb 21, 2017 at 11:38 PM, Moritz Fischer
<moritz.fischer@ettus.com> wrote:

Hi Moritz,

> Hi all,
>
> On Tue, Feb 21, 2017 at 9:12 PM, Jason Gunthorpe
> <jgunthorpe@obsidianresearch.com> wrote:
>> On Tue, Feb 21, 2017 at 07:49:19PM -0800, Moritz Fischer wrote:
>>
>>> fdt does this out of the box, too. So far I've seen nothing fdt
>>> couldn't do (or doesn't do let's rather say).
>>
>> tlv/fdt/http headers are all essentially exactly the same
>> thing. Key/value pairs with various encoding schemes.
>>
>> I don't think we don't need a tree of data, so fdt is overkill.
>>
>> tlv is not substantially easier to parse correctly than the
>> structured plain text headers.. It is just in binary so it can
>> represent binary-ish things better.
>
> TLV Seems easy enough. To give an update, I played with fdt a bit to see
> how far I get in half an hour. I got bool / int / strings to work
> quite fast (~30mins).

Thanks for doing this fast piece of exploratory coding.  It does
confirm that for Linux, using fdt is pretty straightforward here.
However...

> Please disregard the horrible hackyness of this ...
>
> For simplicity I stuck the header on top of my bitfile with:
>
> <snip>
> /dts-v1/;
>
> /{
>         description = "Test";
>         compressed = <0>;
>         encrypted = <1>;
> };

I understand that this is a simplified example, but it looks a lot
like KVP which then gets compiled by dtc.

If we do KVP or TLV we get skip using dtc, which would be nice for non-dt
OS's using the same images.

Also, the license of libfdt allows the use by proprietary
os's, but that's not true for dtc.

Alan

> </snip>
>
> $ dtc -o header.dtb header.dts
>
> $ cat header.dtb mybitfile.bin > /lib/firmware/bitfile_header.bin
>
> + static int __fpga_mgr_blob_to_image_info(const void *blob,
> +                                          struct fpga_image_info *info)
> + {
> +         int root_offset;
> +         const char *desc;
> +         const uint32_t *_compressed, *_encrypted;
> +         int compressed, encrypted;
> +
> +         if (fdt_check_header(blob)) {
> +                 pr_err("Invalid device tree blob header\n");
> +                 return -EINVAL;
> +         }
> +
> +         root_offset = fdt_path_offset(blob, "/");
> +         if (root_offset < 0) {
> +                 pr_err("Invalid root offset\n");
> +                 return -EINVAL;
> +         }
> +
> +         desc = fdt_getprop(blob, root_offset, "description", NULL);
> +
> +         _compressed = fdt_getprop(blob, root_offset, "compressed", NULL);
> +         if (_compressed)
> +                 compressed = fdt32_to_cpu(*_compressed);
> +
> +         _encrypted = fdt_getprop(blob, root_offset, "encrypted", NULL);
> +         if (_encrypted)
> +                 encrypted = fdt32_to_cpu(*_encrypted);
> +
> +         if (desc)
> +                 pr_info("%s: description=%s\n", __func__, desc);
> +
> +         if (_encrypted && _compressed)
> +                 pr_info("%s: compressed? %s encrypted? %s\n", __func__,
> +                         compressed ? "Yes" : "No", encrypted ? "Yes" : "No");
> +
> +         return 0;
> + }
>
> Which gave me:
>
> <snip>
>
> [   19.325182] fpga_manager fpga0: writing bitfile_header.bin to
> Xilinx Zynq FPGA Manager
> [   20.091222] __fpga_mgr_blob_to_image_info: description=Test
> [   20.096730] __fpga_mgr_blob_to_image_info: compressed? No encrypted? Yes
>
> </snip>
>
> So I'm fairly convinced I can make this work, TVLs seem like it could work, too.
>
>> So far the only thing we know we need is a 'bool' for encrypted and a
>> stringish guid thing for partial reconfiguration.
>
> Yeah, shouldn't be hard to implement either way.
>
> Cheers,
>
> Moritz

[toc] | [prev] | [next] | [standalone]


Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

Back to top | Article view | linux.kernel


csiph-web