Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #25305 > unrolled thread

[OT] Simulation Results Managment

Back to article view | Back to comp.lang.python

Contents

  [OT] Simulation Results Managment moogyd@yahoo.co.uk - 2012-07-13 22:50 -0700
    Re: [OT] Simulation Results Managment Neal Becker <ndbecker2@gmail.com> - 2012-07-14 20:42 -0400
      Re: [OT] Simulation Results Managment moogyd@yahoo.co.uk - 2012-07-14 22:29 -0700
        Re: [OT] Simulation Results Managment Dieter Maurer <dieter@handshake.de> - 2012-07-15 08:35 +0200
          Re: Simulation Results Managment rusi <rustompmody@gmail.com> - 2012-07-15 09:20 -0700
            Re: Simulation Results Managment moogyd@yahoo.co.uk - 2012-07-17 01:20 -0700
        Re: [OT] Simulation Results Managment Neal Becker <ndbecker2@gmail.com> - 2012-07-15 07:24 -0400
      Re: [OT] Simulation Results Managment moogyd@yahoo.co.uk - 2012-07-14 22:29 -0700
    Re: Simulation Results Managment rusi <rustompmody@gmail.com> - 2012-07-14 20:25 -0700
      Re: Simulation Results Managment moogyd@yahoo.co.uk - 2012-07-14 22:26 -0700

#25305 — [OT] Simulation Results Managment

Hi,
This is a general question, loosely related to python since it will be the implementation language.
I would like some suggestions as to manage simulation results data from my ASIC design. 

For my design, 
- I have a number of simulations testcases (TEST_XX_YY_ZZ), and within each of these test cases we have: 
  - a number of properties (P_AA_BB_CC) 
  - For each property, the following information is given
    - Property name (P_NAME) 
    - Number of times it was checked (within the testcase) N_CHECKED
    - Number of times if failed (within the testcase) N_FAILED
- A simulation runs a testcase with a set of parameters.
  - Simple example, SLOW_CLOCK, FAST_CLOCK, etc
- For the design, I will run regression every night (at least), so I will have results from multiple timestamps
We have < 1000 TESTCASES, and < 1000 PROPERTIES.

At the moment, I have a script that extracts property information from simulation logfile, and provides single PASS/FAIL and all logfiles stored in a directory structure with timestamps/testnames and other parameters embedded in paths

I would like to be easily look at (visualize) the data and answer the questions
- When did this property last fail, and how many times was it checked
- Is this property checked in this test case.

Initial question: How to organize the data within python?
For a single testcase, I could use a dict. Key P_NAME, data in N_CHECKED, N_FAILED
I then have to store multiple instances of testcase based on date (and simulation parameters.

Any comments, suggestions?
Thanks,
Steven

[toc] | [next] | [standalone]

#25325

moogyd@yahoo.co.uk wrote:

> Hi,
> This is a general question, loosely related to python since it will be the
> implementation language. I would like some suggestions as to manage simulation
> results data from my ASIC design.
> 
> For my design,
> - I have a number of simulations testcases (TEST_XX_YY_ZZ), and within each of
> these test cases we have:
>   - a number of properties (P_AA_BB_CC)
>   - For each property, the following information is given
>     - Property name (P_NAME)
>     - Number of times it was checked (within the testcase) N_CHECKED
>     - Number of times if failed (within the testcase) N_FAILED
> - A simulation runs a testcase with a set of parameters.
>   - Simple example, SLOW_CLOCK, FAST_CLOCK, etc
> - For the design, I will run regression every night (at least), so I will have
> results from multiple timestamps We have < 1000 TESTCASES, and < 1000
> PROPERTIES.
> 
> At the moment, I have a script that extracts property information from
> simulation logfile, and provides single PASS/FAIL and all logfiles stored in a
> directory structure with timestamps/testnames and other parameters embedded in
> paths
> 
> I would like to be easily look at (visualize) the data and answer the
> questions - When did this property last fail, and how many times was it
> checked - Is this property checked in this test case.
> 
> Initial question: How to organize the data within python?
> For a single testcase, I could use a dict. Key P_NAME, data in N_CHECKED,
> N_FAILED I then have to store multiple instances of testcase based on date
> (and simulation parameters.
> 
> Any comments, suggestions?
> Thanks,
> Steven

One small suggestion,
I used to store test conditions and results in log files, and then write parsers 
to read the results.  The formats kept changing (add more conditions/results!) 
and maintenance was a pain.

Now, in addition to a text log file, I write a file in pickle format containing 
a dict of all test conditions and results.  Much more convenient.

[toc] | [prev] | [next] | [standalone]

#25335

On Sunday, July 15, 2012 2:42:39 AM UTC+2, Neal Becker wrote:
> me wrote:
> 
> > Hi,
> > This is a general question, loosely related to python since it will be the
> > implementation language. I would like some suggestions as to manage simulation
> > results data from my ASIC design.
> > 
> > For my design,
> > - I have a number of simulations testcases (TEST_XX_YY_ZZ), and within each of
> > these test cases we have:
> >   - a number of properties (P_AA_BB_CC)
> >   - For each property, the following information is given
> >     - Property name (P_NAME)
> >     - Number of times it was checked (within the testcase) N_CHECKED
> >     - Number of times if failed (within the testcase) N_FAILED
> > - A simulation runs a testcase with a set of parameters.
> >   - Simple example, SLOW_CLOCK, FAST_CLOCK, etc
> > - For the design, I will run regression every night (at least), so I will have
> > results from multiple timestamps We have < 1000 TESTCASES, and < 1000
> > PROPERTIES.
> > 
> > At the moment, I have a script that extracts property information from
> > simulation logfile, and provides single PASS/FAIL and all logfiles stored in a
> > directory structure with timestamps/testnames and other parameters embedded in
> > paths
> > 
> > I would like to be easily look at (visualize) the data and answer the
> > questions - When did this property last fail, and how many times was it
> > checked - Is this property checked in this test case.
> > 
> > Initial question: How to organize the data within python?
> > For a single testcase, I could use a dict. Key P_NAME, data in N_CHECKED,
> > N_FAILED I then have to store multiple instances of testcase based on date
> > (and simulation parameters.
> > 
> > Any comments, suggestions?
> > Thanks,
> > Steven
> 
> One small suggestion,
> I used to store test conditions and results in log files, and then write parsers 
> to read the results.  The formats kept changing (add more conditions/results!) 
> and maintenance was a pain.
> 
> Now, in addition to a text log file, I write a file in pickle format containing 
> a dict of all test conditions and results.  Much more convenient.

Hi Neal,
We already store the original log files.
Does pickle have any advantages over json/yaml?
Thanks,
Steven

[toc] | [prev] | [next] | [standalone]

#25338

moogyd@yahoo.co.uk writes:
> ...
> Does pickle have any advantages over json/yaml?

It can store and retrieve almost any Python object with almost no effort.

Up to you whether you see it as an advantage to be able to store
objects rather than (almost) pure data with a rather limited type set.


Of course, "pickle" is a proprietary Python format. Not so easy to
decode it with something else than Python. In addition, when
you store objects, the retrieving application must know the classes
of those objects -- and its knowledge should not be too different
from how those classes looked when the objects have been stored.


I like very much to work with objects (rather than with pure data).
Therefore, I use "pickle" when I know that the storing and retrieving
applications all use Python. I use pure (and restricted) data formats
when non Python applications come into play.

[toc] | [prev] | [next] | [standalone]

#25355 — Re: Simulation Results Managment

On Jul 15, 11:35 am, Dieter Maurer <die...@handshake.de> wrote:
> moo...@yahoo.co.uk writes:
> > ...
> > Does pickle have any advantages over json/yaml?
>
> It can store and retrieve almost any Python object with almost no effort.
>
> Up to you whether you see it as an advantage to be able to store
> objects rather than (almost) pure data with a rather limited type set.
>
> Of course, "pickle" is a proprietary Python format. Not so easy to
> decode it with something else than Python. In addition, when
> you store objects, the retrieving application must know the classes
> of those objects -- and its knowledge should not be too different
> from how those classes looked when the objects have been stored.
>
> I like very much to work with objects (rather than with pure data).
> Therefore, I use "pickle" when I know that the storing and retrieving
> applications all use Python. I use pure (and restricted) data formats
> when non Python applications come into play.

Pickle -> JSON -> Yaml
are roughly in increasing order of human-friendliness and decreasing
order of machine friendliness (where machine means python 'machine')

This means that
 - Pickle is most efficient, Yaml least
 - Pickle comes with python from as far back as I know
   Json started coming somewhere round 2.5 (I think)
   (py)yaml needs to be installed separately
 - reading pickled data will spoil your eyes whereas yaml is pleasant
to read (just like python)

[toc] | [prev] | [next] | [standalone]

#25469 — Re: Simulation Results Managment

On Sunday, July 15, 2012 6:20:34 PM UTC+2, rusi wrote:
> On Jul 15, 11:35 am, Dieter Maurer <die...@handshake.de> wrote:
> > moo...@yahoo.co.uk writes:
> > > ...
> > > Does pickle have any advantages over json/yaml?
> >
> > It can store and retrieve almost any Python object with almost no effort.
> >
> > Up to you whether you see it as an advantage to be able to store
> > objects rather than (almost) pure data with a rather limited type set.
> >
> > Of course, "pickle" is a proprietary Python format. Not so easy to
> > decode it with something else than Python. In addition, when
> > you store objects, the retrieving application must know the classes
> > of those objects -- and its knowledge should not be too different
> > from how those classes looked when the objects have been stored.
> >
> > I like very much to work with objects (rather than with pure data).
> > Therefore, I use "pickle" when I know that the storing and retrieving
> > applications all use Python. I use pure (and restricted) data formats
> > when non Python applications come into play.
> 
> Pickle -> JSON -> Yaml
> are roughly in increasing order of human-friendliness and decreasing
> order of machine friendliness (where machine means python 'machine')
> 
> This means that
>  - Pickle is most efficient, Yaml least
>  - Pickle comes with python from as far back as I know
>    Json started coming somewhere round 2.5 (I think)
>    (py)yaml needs to be installed separately
>  - reading pickled data will spoil your eyes whereas yaml is pleasant
> to read (just like python)

Hi Everyone,
Thanks for the feedback. For now, I store the data using Pickle.
Steven

[toc] | [prev] | [next] | [standalone]

#25347

Dieter Maurer wrote:

> moogyd@yahoo.co.uk writes:
>> ...
>> Does pickle have any advantages over json/yaml?
> 
> It can store and retrieve almost any Python object with almost no effort.
> 
> Up to you whether you see it as an advantage to be able to store
> objects rather than (almost) pure data with a rather limited type set.
> 
> 
> Of course, "pickle" is a proprietary Python format. Not so easy to
> decode it with something else than Python. In addition, when
> you store objects, the retrieving application must know the classes
> of those objects -- and its knowledge should not be too different
> from how those classes looked when the objects have been stored.
> 
> 
> I like very much to work with objects (rather than with pure data).
> Therefore, I use "pickle" when I know that the storing and retrieving
> applications all use Python. I use pure (and restricted) data formats
> when non Python applications come into play.

Typically what I want to do is post-process (e.g. plot) results using python 
scripts, so using pickle is great for that.

[toc] | [prev] | [next] | [standalone]

#25336

On Sunday, July 15, 2012 2:42:39 AM UTC+2, Neal Becker wrote:
> me wrote:
> 
> > Hi,
> > This is a general question, loosely related to python since it will be the
> > implementation language. I would like some suggestions as to manage simulation
> > results data from my ASIC design.
> > 
> > For my design,
> > - I have a number of simulations testcases (TEST_XX_YY_ZZ), and within each of
> > these test cases we have:
> >   - a number of properties (P_AA_BB_CC)
> >   - For each property, the following information is given
> >     - Property name (P_NAME)
> >     - Number of times it was checked (within the testcase) N_CHECKED
> >     - Number of times if failed (within the testcase) N_FAILED
> > - A simulation runs a testcase with a set of parameters.
> >   - Simple example, SLOW_CLOCK, FAST_CLOCK, etc
> > - For the design, I will run regression every night (at least), so I will have
> > results from multiple timestamps We have < 1000 TESTCASES, and < 1000
> > PROPERTIES.
> > 
> > At the moment, I have a script that extracts property information from
> > simulation logfile, and provides single PASS/FAIL and all logfiles stored in a
> > directory structure with timestamps/testnames and other parameters embedded in
> > paths
> > 
> > I would like to be easily look at (visualize) the data and answer the
> > questions - When did this property last fail, and how many times was it
> > checked - Is this property checked in this test case.
> > 
> > Initial question: How to organize the data within python?
> > For a single testcase, I could use a dict. Key P_NAME, data in N_CHECKED,
> > N_FAILED I then have to store multiple instances of testcase based on date
> > (and simulation parameters.
> > 
> > Any comments, suggestions?
> > Thanks,
> > Steven
> 
> One small suggestion,
> I used to store test conditions and results in log files, and then write parsers 
> to read the results.  The formats kept changing (add more conditions/results!) 
> and maintenance was a pain.
> 
> Now, in addition to a text log file, I write a file in pickle format containing 
> a dict of all test conditions and results.  Much more convenient.

Hi Neal,
We already store the original log files.
Does pickle have any advantages over json/yaml?
Thanks,
Steven

[toc] | [prev] | [next] | [standalone]

#25332 — Re: Simulation Results Managment

On Jul 14, 10:50 am, moo...@yahoo.co.uk wrote:
> Hi,
> This is a general question, loosely related to python since it will be the implementation language.
> I would like some suggestions as to manage simulation results data from my ASIC design.
>
> For my design,
> - I have a number of simulations testcases (TEST_XX_YY_ZZ), and within each of these test cases we have:
>   - a number of properties (P_AA_BB_CC)
>   - For each property, the following information is given
>     - Property name (P_NAME)
>     - Number of times it was checked (within the testcase) N_CHECKED
>     - Number of times if failed (within the testcase) N_FAILED
> - A simulation runs a testcase with a set of parameters.
>   - Simple example, SLOW_CLOCK, FAST_CLOCK, etc
> - For the design, I will run regression every night (at least), so I will have results from multiple timestamps
> We have < 1000 TESTCASES, and < 1000 PROPERTIES.
>
> At the moment, I have a script that extracts property information from simulation logfile, and provides single PASS/FAIL and all logfiles stored in a directory structure with timestamps/testnames and other parameters embedded in paths
>
> I would like to be easily look at (visualize) the data and answer the questions
> - When did this property last fail, and how many times was it checked
> - Is this property checked in this test case.
>
> Initial question: How to organize the data within python?
> For a single testcase, I could use a dict. Key P_NAME, data in N_CHECKED, N_FAILED
> I then have to store multiple instances of testcase based on date (and simulation parameters.
>
> Any comments, suggestions?
> Thanks,
> Steven

Not sure if you are asking about:
1. Python data structure organization
or
2. Organization of data outside python for conveniently getting in and
out of python

For 2. if the data is modestly sized and is naturally managed with
builtin python types -- lists and dictionaries -- yaml gives a nice
fit. I used pyyaml some years ago, today I guess json which is
similar, is the way to go.

For 1, you need to say what are your questions/issues.

[toc] | [prev] | [next] | [standalone]

#25334 — Re: Simulation Results Managment

On Sunday, July 15, 2012 5:25:14 AM UTC+2, rusi wrote:
> On Jul 14, 10:50 am, moo...@yahoo.co.uk wrote:
> > Hi,
> > This is a general question, loosely related to python since it will be the implementation language.
> > I would like some suggestions as to manage simulation results data from my ASIC design.
> >
> > For my design,
> > - I have a number of simulations testcases (TEST_XX_YY_ZZ), and within each of these test cases we have:
> >   - a number of properties (P_AA_BB_CC)
> >   - For each property, the following information is given
> >     - Property name (P_NAME)
> >     - Number of times it was checked (within the testcase) N_CHECKED
> >     - Number of times if failed (within the testcase) N_FAILED
> > - A simulation runs a testcase with a set of parameters.
> >   - Simple example, SLOW_CLOCK, FAST_CLOCK, etc
> > - For the design, I will run regression every night (at least), so I will have results from multiple timestamps
> > We have < 1000 TESTCASES, and < 1000 PROPERTIES.
> >
> > At the moment, I have a script that extracts property information from simulation logfile, and provides single PASS/FAIL and all logfiles stored in a directory structure with timestamps/testnames and other parameters embedded in paths
> >
> > I would like to be easily look at (visualize) the data and answer the questions
> > - When did this property last fail, and how many times was it checked
> > - Is this property checked in this test case.
> >
> > Initial question: How to organize the data within python?
> > For a single testcase, I could use a dict. Key P_NAME, data in N_CHECKED, N_FAILED
> > I then have to store multiple instances of testcase based on date (and simulation parameters.
> >
> > Any comments, suggestions?
> > Thanks,
> > Steven
> 
> Not sure if you are asking about:
> 1. Python data structure organization
> or
> 2. Organization of data outside python for conveniently getting in and
> out of python
> 
> For 2. if the data is modestly sized and is naturally managed with
> builtin python types -- lists and dictionaries -- yaml gives a nice
> fit. I used pyyaml some years ago, today I guess json which is
> similar, is the way to go.
> 
> For 1, you need to say what are your questions/issues.

Hi Rusi,

For (1), I guess that the only question I had was how to handle regression results. But I think that the most logical way for string this data is as a dict with key = datestamp, and entries being list of testcases/results.

For (2), I will look at both these.

Thanks for the help.

Steven

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web