Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #92869 > unrolled thread

Re: JSON Object to CSV Question

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: JSON Object to CSV Question Steve Hayes <hayesstw@telkomsa.net> - 2015-06-19 07:21 +0200
    Re: JSON Object to CSV Question Saran Ahluwalia <ahlusar.ahluwalia@gmail.com> - 2015-06-19 05:50 -0400
      Re: JSON Object to CSV Question Cecil Westerhof <Cecil@decebal.nl> - 2015-06-19 12:07 +0200
        Re: JSON Object to CSV Question Saran Ahluwalia <ahlusar.ahluwalia@gmail.com> - 2015-06-19 06:24 -0400
        Re: JSON Object to CSV Question Sahlusar <ahlusar.ahluwalia@gmail.com> - 2015-06-19 05:47 -0700
          Re: JSON Object to CSV Question Sahlusar <ahlusar.ahluwalia@gmail.com> - 2015-06-19 06:44 -0700
      Re: JSON Object to CSV Question Steven D'Aprano <steve@pearwood.info> - 2015-06-20 00:47 +1000
        Re: JSON Object to CSV Question Joonas Liik <liik.joonas@gmail.com> - 2015-06-19 23:32 +0300
        Re: JSON Object to CSV Question Ned Batchelder <ned@nedbatchelder.com> - 2015-06-20 08:04 -0700

#92869 — Re: JSON Object to CSV Question

On Wed, 17 Jun 2015 17:49:35 -0400, Saran Ahluwalia
<ahlusar.ahluwalia@gmail.com> wrote:

>Good Evening Everyone:
>
>I would like to have this JSON object written out to a CSV file so that the

You've already said that in another thread, and got several answers.
What are you? Some kind of troll?


-- 
Steve Hayes from Tshwane, South Africa
Web:  http://www.khanya.org.za/stevesig.htm
Blog: http://khanya.wordpress.com
E-mail - see web page, or parse: shayes at dunelm full stop org full stop uk

[toc] | [next] | [standalone]

#92874

 Steve, 

No I am not a troll. I am attempting to clarify an evolving problem. If you read the most recent thread that I just posted it states far more information - in my humble opinion.
Peter was of great assistance. The data becomes more convoluted; as a fairly new programmer I am looking for best practice.

Finally, although this is virtual space, it is important to maintain etiquette. I would appreciate it if you would maintain civility and neutrality in discourse.

Saran

Sent from my iPhone

> On Jun 19, 2015, at 1:21 AM, Steve Hayes <hayesstw@telkomsa.net> wrote:
> 
> On Wed, 17 Jun 2015 17:49:35 -0400, Saran Ahluwalia
> <ahlusar.ahluwalia@gmail.com> wrote:
> 
>> Good Evening Everyone:
>> 
>> I would like to have this JSON object written out to a CSV file so that the
> 
> You've already said that in another thread, and got several answers.
> What are you? Some kind of troll?
> 
> 
> -- 
> Steve Hayes from Tshwane, South Africa
> Web:  http://www.khanya.org.za/stevesig.htm
> Blog: http://khanya.wordpress.com
> E-mail - see web page, or parse: shayes at dunelm full stop org full stop uk
> -- 
> https://mail.python.org/mailman/listinfo/python-list

[toc] | [prev] | [next] | [standalone]

#92875

Top posting is frowned upon. Could you put your reply under where you
reply on next time?

On Friday 19 Jun 2015 11:50 CEST, Saran Ahluwalia wrote:

> No I am not a troll. I am attempting to clarify an evolving problem.
> If you read the most recent thread that I just posted it states far
> more information - in my humble opinion. Peter was of great

You should put more thought about how your postings will look for the
receiver. I also saw those mails and was wondering why you kept
posting the same question. You was not, but you seemed to. Besides
that it is annoying for the users of the list, you lower the change of
getting a response. When I think I saw the question already, I get
annoyed, do not read it, so I will not respond.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

[toc] | [prev] | [next] | [standalone]

#92877

Thank you for clarifying. I will be more cognizant to follow this protocol. 

Sent from my iPhone

> On Jun 19, 2015, at 6:07 AM, Cecil Westerhof <Cecil@decebal.nl> wrote:
> 
> Top posting is frowned upon. Could you put your reply under where you
> reply on next time?
> 
>> On Friday 19 Jun 2015 11:50 CEST, Saran Ahluwalia wrote:
>> 
>> No I am not a troll. I am attempting to clarify an evolving problem.
>> If you read the most recent thread that I just posted it states far
>> more information - in my humble opinion. Peter was of great
> 
> You should put more thought about how your postings will look for the
> receiver. I also saw those mails and was wondering why you kept
> posting the same question. You was not, but you seemed to. Besides
> that it is annoying for the users of the list, you lower the change of
> getting a response. When I think I saw the question already, I get
> annoyed, do not read it, so I will not respond.
> 
> -- 
> Cecil Westerhof
> Senior Software Engineer
> LinkedIn: http://www.linkedin.com/in/cecilwesterhof
> -- 
> https://mail.python.org/mailman/listinfo/python-list

[toc] | [prev] | [next] | [standalone]

#92879

On Friday, June 19, 2015 at 6:16:40 AM UTC-4, Cecil Westerhof wrote:
> Top posting is frowned upon. Could you put your reply under where you
> reply on next time?
> 
> On Friday 19 Jun 2015 11:50 CEST, Saran Ahluwalia wrote:
> 
> > No I am not a troll. I am attempting to clarify an evolving problem.
> > If you read the most recent thread that I just posted it states far
> > more information - in my humble opinion. Peter was of great
> 
> You should put more thought about how your postings will look for the
> receiver. I also saw those mails and was wondering why you kept
> posting the same question. You was not, but you seemed to. Besides
> that it is annoying for the users of the list, you lower the change of
> getting a response. When I think I saw the question already, I get
> annoyed, do not read it, so I will not respond.
> 
> -- 
> Cecil Westerhof
> Senior Software Engineer
> LinkedIn: http://www.linkedin.com/in/cecilwesterhof

On another note, are there any suggestions for how to address this problem. I should elaborate that I have recently switched careers in to development (formally an educator). I am seeking guidance for best practice. Thank you for your help everyone. 

[toc] | [prev] | [next] | [standalone]

#92882

On Friday, June 19, 2015 at 8:47:25 AM UTC-4, Sahlusar wrote:
> On Friday, June 19, 2015 at 6:16:40 AM UTC-4, Cecil Westerhof wrote:
> > Top posting is frowned upon. Could you put your reply under where you
> > reply on next time?
> > 
> > On Friday 19 Jun 2015 11:50 CEST, Saran Ahluwalia wrote:
> > 
> > > No I am not a troll. I am attempting to clarify an evolving problem.
> > > If you read the most recent thread that I just posted it states far
> > > more information - in my humble opinion. Peter was of great
> > 
> > You should put more thought about how your postings will look for the
> > receiver. I also saw those mails and was wondering why you kept
> > posting the same question. You was not, but you seemed to. Besides
> > that it is annoying for the users of the list, you lower the change of
> > getting a response. When I think I saw the question already, I get
> > annoyed, do not read it, so I will not respond.
> > 
> > -- 
> > Cecil Westerhof
> > Senior Software Engineer
> > LinkedIn: http://www.linkedin.com/in/cecilwesterhof
> 
> On another note, are there any suggestions for how to address this problem. I should elaborate that I have recently switched careers in to development (formally an educator). I am seeking guidance for best practice. Thank you for your help everyone.

Here is a more (I hope focused question):

http://stackoverflow.com/questions/30938462/using-python-to-convert-json-objects-in-file-to-csv-2-parts.

[toc] | [prev] | [next] | [standalone]

#92886

On Fri, 19 Jun 2015 07:50 pm, Saran Ahluwalia wrote:

> If you read the most recent thread that I just posted it states far more
> information

The problem is, you are posting *too much* of the *wrong* information.

The exception you are getting appears to be a simple one: you are getting
the exception 

ValueError: too many values to unpack


but since you don't show the traceback, only the error message, we have no
idea *where* that exception is happening. It could be a bug in your code,
in the JSON library, or who knows where.

Python provides lots of useful debugging information in the traceback, but
you are ignoring it. You should look at the complete traceback, everything
from the initial line:

    Traceback

to the end of the error message, not just the error message itself. That
will show you the series of function calls that lead to the error, and
usually also the actual line of code that caused the problem. That alone
might be enough to solve the problem. If not, it is one extra clue. Either
way, when asking for help, you should always post the complete traceback
(unless it is hundreds of lines long, in which case, ask first).


Unfortunately you have dumped over 100 lines of code in our lap, from two
different files. To solve this problem for you, we would have to copy your
code, install the libraries you use, somehow get a copy of your XML file:

C:\\Users\\wynsa2\\Desktop\\Python Folder\\PCSU\\Trial2_PCSU\\2-Response.xml

(don't ask me how we can possibly do that!), run your code, see what the
full traceback says, and debug the code for you. That is simply not going
to happen, not unless you pay someone.

The most critical skill a programmer can have is to learn how to focus on
what is essential when troubleshooting, and not get lost in a sea of
irrelevant code. The way to do that is to narrow the problem down to the
smallest possible code sample which demonstrates the same error, and see
whether that shows you the solution. If it does, you have your solution. If
not, you can gradually add more complexity until you have your answer.

You can read more about this here:

http://sscce.org/

Although it is written for Java, the principles apply equally to Python.

Here is the minimum code needed to give your same error:

py> x, y, z = (1, 2, 3, 4)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
ValueError: too many values to unpack


Here I am trying to assign four values into three variables. The solution is
obvious:

py> a, x, y, z = (1, 2, 3, 4)

will work. Does that help you see the solution you need? If not, we will
need a SSCCE to work with.


Some other things to consider.

You can avoid all those doubled backslashes by using forward slashes.
Windows accepts both:

C:/Users/wynsa2/Desktop/Python Folder/PCSU/Trial2_PCSU/2-Response.xml

which makes for much neater and easier file names.

Also, it doesn't help to ask multiple unrelated questions in the same post.
If they are directly related, that is okay, but you are asking (at least)
two questions:

- how to fix the ValueError exception;

- something about "I am wondering if there is a function or methodology that
would allow me to remove such nested keys and reassign the new keys to the
outer key..."

The second question is very vague, and vague questions get vague answers (or
no answers at all.) Yes, there is such a methodology. It is
called "programming".

Again, simplify what you are trying to do. Show us a *minimal* example of
the data you start with, and the result you hope to end up with. I'm afraid
I don't understand your question at all, simplifying it and showing an
example will help a lot.

You should post these in separate threads. People interested in one thread
may not be interested in the other. By posting them together, you only get
responses from people interested in both.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#92897

this.. might not throw an eror, but you have 2 keys with the same name
"F", and 1 of them will probably be disgarded..., you have data
corruption even before you try to process it.

{
 "F": "False",
"F": {
"Int32": ["0",
"0",
"0"]
},
 }

you mentioned Excel at one point.
perhaps you could mock up what you'd like your finished data to look
like in a spreadsheet (google docs for instance, since thats easy to
link to) and reference there.

just having a list of headers doesnt say much about the data format you want.

"client wants csv" hmm..they want "csv" or they want "csv that fists
this very particular description that fits our special decoder or the
like" ?

do you know how the client will use this data. could that info be used
to simplify the output to some degree?

and finally..
the client gives you malformed xml??
I'm very sorry to hear that. also does the client know they are
emitting invalid xml?
    is it rly xml?
    is it valid in another language? (html is much more lenient for
instance, an html parser might be able to gleam more meaning)
    by what definition is it malformed? is it outright structuralyl
broken, does it fail to meet some schema?
                      does it fail to meet some expectation you have
for some reason ("client said it has these properties")

you also mentioned you use JSON because it maps nicely to python
dicts.. this is true ofc.. but why not just read that in to a python
dict in the first place?

> DB1 : 0, DB2: 0, DB3: 0 etc. and F1: 0, F1: 0. DB1, DB2 would be the headers and the 0s as values in the CSV file.

DB1 etc seems ok at first glance however...
say there are 2 nested items and each of them have a DB property which
is an array, you will have name collisions.
you need more thought in to naming the headers at the very least.

if this is meant for a spreadsheet.. then you will end up with 2 very
very very long rows, it will NOT be readable by any stretch of the
imagination.. do you want this.

i'm afraid you'll essentially end up with a translation that looks sth like

{A:{b:"c",q:"w"}}
===============
"A.b", "A.q"
"c", "w"

if you just want key-value pairs there are better options out
there..besides csv..

[toc] | [prev] | [next] | [standalone]

#92916

On Saturday, June 20, 2015 at 2:10:51 AM UTC-4, Sahlusar wrote:
> @Joonas:
> 
> 
> The previous example was a typo. Please use the below example as a case study. 
> 
> 
> 
> 
> {'D_B': ['0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0'], 'F_Int32': ['0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0'], 'OTF': '0', 'PBDS_Double': ['0', '0', '0', '0', '0', '0', '0', '0'], 'SCS_String': ['1', '2']}
> 
> 
> All of the questions regarding XML I have asked. I have to work within their parameters. The CSV, for example may look like this:
> 
> 
> DLA,FC,PC,WC,CN,Description,Code,CMC 
> 0,00000,0,0,,,0,0

There isn't a single word in common between the Python dictionary you've
shown and the CSV sample output you've shown.  There's no way anyone can
tell what mapping you intend here.

--Ned.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web