Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #95759 > unrolled thread

TypeError: unhashable type: 'dict' when attempting to hash list - advice sought

Back to article view | Back to comp.lang.python

Contents

  TypeError: unhashable type: 'dict' when attempting to hash list - advice sought kbtyo <ahlusar.ahluwalia@gmail.com> - 2015-08-29 19:05 -0700
    Re: TypeError: unhashable type: 'dict' when attempting to hash list - advice sought Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-08-30 03:47 +0100
      Re: TypeError: unhashable type: 'dict' when attempting to hash list - advice sought kbtyo <ahlusar.ahluwalia@gmail.com> - 2015-08-30 09:35 -0700
        Re: TypeError: unhashable type: 'dict' when attempting to hash list - advice sought Laura Creighton <lac@openend.se> - 2015-08-30 19:24 +0200
          Re: TypeError: unhashable type: 'dict' when attempting to hash list - advice sought kbtyo <ahlusar.ahluwalia@gmail.com> - 2015-08-30 10:34 -0700
            Re: TypeError: unhashable type: 'dict' when attempting to hash list - advice sought Laura Creighton <lac@openend.se> - 2015-08-30 20:20 +0200
              Re: TypeError: unhashable type: 'dict' when attempting to hash list - advice sought kbtyo <ahlusar.ahluwalia@gmail.com> - 2015-08-30 14:47 -0700
    Re: TypeError: unhashable type: 'dict' when attempting to hash list - advice sought MRAB <python@mrabarnett.plus.com> - 2015-08-30 03:48 +0100
      Re: TypeError: unhashable type: 'dict' when attempting to hash list - advice sought kbtyo <ahlusar.ahluwalia@gmail.com> - 2015-08-30 09:31 -0700
        Re: TypeError: unhashable type: 'dict' when attempting to hash list - advice sought MRAB <python@mrabarnett.plus.com> - 2015-08-30 18:15 +0100
          Re: TypeError: unhashable type: 'dict' when attempting to hash list - advice sought kbtyo <ahlusar.ahluwalia@gmail.com> - 2015-08-30 10:21 -0700
    Re: TypeError: unhashable type: 'dict' when attempting to hash list - advice sought Ben Finney <ben+python@benfinney.id.au> - 2015-08-30 13:04 +1000
      Re: TypeError: unhashable type: 'dict' when attempting to hash list - advice sought kbtyo <ahlusar.ahluwalia@gmail.com> - 2015-08-30 10:02 -0700

#95759 — TypeError: unhashable type: 'dict' when attempting to hash list - advice sought

I am using Jupyter Notebook and Python 3.4. I have a data structure in the format, (type list):

[{'AccountNumber': N,
'Amount': '0',
 'Answer': '12:00:00 PM',
  'ID': None,
  'Type': 'WriteLetters',
  'Amount': '10',
  {'AccountNumber': Y,
      'Amount': '0',
      'Answer': ' 12:00:00 PM',       
       'ID': None,
      'Type': 'Transfer',
      'Amount': '2'}]

The end goal is to write this out to CSV.

For the above example the output would look like:

AccountNumber, Amount, Answer, ID, Type, Amount
N,0,12:00:00 PM,None,WriteLetters,10
Y,2,12:00:00 PM,None,Transfer,2

Below is the function that I am using to write out this data structure. Please excuse any indentation formatting issues. The data structure is returned through the function "construct_results(get_just_xml_data)". 

The data that is returned is in the format as above. "construct_headers(get_just_xml_data)" returns a list of headers. Writing out the row for "headers_list" works.

The list comprehension "data" is to maintain the integrity of the column headers and the values for each new instance of the data structure (where the keys in the dictionary are the headers and values - row instances). The keys in this specific data structure are meant to check if there is a value instance, and if there is not - place an ''.

def write_to_csv(results, headers):

    headers = construct_headers(get_just_xml_data)
    results = construct_results(get_just_xml_data)
    headers_list = list(headers)

    with open('real_csv_output.csv', 'wt') as f:
        writer = csv.writer(f)
        writer.writerow(headers_list)
        for row in results:
            data = [row.get(index, '') for index in results]
        writer.writerow(data)



However, when I run this, I receive this error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-747-7746797fc9a5> in <module>()
----> 1 write_to_csv(results, headers)

<ipython-input-746-c822437eeaf0> in write_to_csv(results, headers)
      9         writer.writerow(headers_list)
     10         for item in results:
---> 11             data = [item.get(index, '') for index in results]
     12         writer.writerow(data)

<ipython-input-746-c822437eeaf0> in <listcomp>(.0)
      9         writer.writerow(headers_list)
     10         for item in results:
---> 11             data = [item.get(index, '') for index in results]
     12         writer.writerow(data)

TypeError: unhashable type: 'dict'


I have done some research, namely, the following:

https://mail.python.org/pipermail//tutor/2011-November/086761.html

http://stackoverflow.com/questions/27435798/unhashable-type-dict-type-error

http://stackoverflow.com/questions/1957396/why-dict-objects-are-unhashable-in-python

However, I am still perplexed by this error. Any feedback is welcomed. Thank you.

[toc] | [next] | [standalone]

#95760

On 30/08/2015 03:05, kbtyo wrote:
> I am using Jupyter Notebook and Python 3.4. I have a data structure in the format, (type list):
>
> [{'AccountNumber': N,
> 'Amount': '0',
>   'Answer': '12:00:00 PM',
>    'ID': None,
>    'Type': 'WriteLetters',
>    'Amount': '10',
>    {'AccountNumber': Y,
>        'Amount': '0',
>        'Answer': ' 12:00:00 PM',
>         'ID': None,
>        'Type': 'Transfer',
>        'Amount': '2'}]
>
> The end goal is to write this out to CSV.
>
> For the above example the output would look like:
>
> AccountNumber, Amount, Answer, ID, Type, Amount
> N,0,12:00:00 PM,None,WriteLetters,10
> Y,2,12:00:00 PM,None,Transfer,2
>
> Below is the function that I am using to write out this data structure. Please excuse any indentation formatting issues. The data structure is returned through the function "construct_results(get_just_xml_data)".
>
> The data that is returned is in the format as above. "construct_headers(get_just_xml_data)" returns a list of headers. Writing out the row for "headers_list" works.
>
> The list comprehension "data" is to maintain the integrity of the column headers and the values for each new instance of the data structure (where the keys in the dictionary are the headers and values - row instances). The keys in this specific data structure are meant to check if there is a value instance, and if there is not - place an ''.
>
> def write_to_csv(results, headers):
>
>      headers = construct_headers(get_just_xml_data)
>      results = construct_results(get_just_xml_data)
>      headers_list = list(headers)
>
>      with open('real_csv_output.csv', 'wt') as f:
>          writer = csv.writer(f)
>          writer.writerow(headers_list)
>          for row in results:
>              data = [row.get(index, '') for index in results]
>          writer.writerow(data)
>
>
>
> However, when I run this, I receive this error:
>
> ---------------------------------------------------------------------------
> TypeError                                 Traceback (most recent call last)
> <ipython-input-747-7746797fc9a5> in <module>()
> ----> 1 write_to_csv(results, headers)
>
> <ipython-input-746-c822437eeaf0> in write_to_csv(results, headers)
>        9         writer.writerow(headers_list)
>       10         for item in results:
> ---> 11             data = [item.get(index, '') for index in results]
>       12         writer.writerow(data)
>
> <ipython-input-746-c822437eeaf0> in <listcomp>(.0)
>        9         writer.writerow(headers_list)
>       10         for item in results:
> ---> 11             data = [item.get(index, '') for index in results]
>       12         writer.writerow(data)
>
> TypeError: unhashable type: 'dict'
>
>
> I have done some research, namely, the following:
>
> https://mail.python.org/pipermail//tutor/2011-November/086761.html
>
> http://stackoverflow.com/questions/27435798/unhashable-type-dict-type-error
>
> http://stackoverflow.com/questions/1957396/why-dict-objects-are-unhashable-in-python
>
> However, I am still perplexed by this error. Any feedback is welcomed. Thank you.
>

I haven't looked too deeply into the problem as it's 03:45 and I'm just 
heading off to bed, but I suspect you'd make your life easier by using 
https://docs.python.org/3/library/csv.html#csv.DictWriter

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#95771

On Saturday, August 29, 2015 at 10:50:19 PM UTC-4, Mark Lawrence wrote:
> On 30/08/2015 03:05, kbtyo wrote:
> > I am using Jupyter Notebook and Python 3.4. I have a data structure in the format, (type list):
> >
> > [{'AccountNumber': N,
> > 'Amount': '0',
> >   'Answer': '12:00:00 PM',
> >    'ID': None,
> >    'Type': 'WriteLetters',
> >    'Amount': '10',
> >    {'AccountNumber': Y,
> >        'Amount': '0',
> >        'Answer': ' 12:00:00 PM',
> >         'ID': None,
> >        'Type': 'Transfer',
> >        'Amount': '2'}]
> >
> > The end goal is to write this out to CSV.
> >
> > For the above example the output would look like:
> >
> > AccountNumber, Amount, Answer, ID, Type, Amount
> > N,0,12:00:00 PM,None,WriteLetters,10
> > Y,2,12:00:00 PM,None,Transfer,2
> >
> > Below is the function that I am using to write out this data structure. Please excuse any indentation formatting issues. The data structure is returned through the function "construct_results(get_just_xml_data)".
> >
> > The data that is returned is in the format as above. "construct_headers(get_just_xml_data)" returns a list of headers. Writing out the row for "headers_list" works.
> >
> > The list comprehension "data" is to maintain the integrity of the column headers and the values for each new instance of the data structure (where the keys in the dictionary are the headers and values - row instances). The keys in this specific data structure are meant to check if there is a value instance, and if there is not - place an ''.
> >
> > def write_to_csv(results, headers):
> >
> >      headers = construct_headers(get_just_xml_data)
> >      results = construct_results(get_just_xml_data)
> >      headers_list = list(headers)
> >
> >      with open('real_csv_output.csv', 'wt') as f:
> >          writer = csv.writer(f)
> >          writer.writerow(headers_list)
> >          for row in results:
> >              data = [row.get(index, '') for index in results]
> >          writer.writerow(data)
> >
> >
> >
> > However, when I run this, I receive this error:
> >
> > ---------------------------------------------------------------------------
> > TypeError                                 Traceback (most recent call last)
> > <ipython-input-747-7746797fc9a5> in <module>()
> > ----> 1 write_to_csv(results, headers)
> >
> > <ipython-input-746-c822437eeaf0> in write_to_csv(results, headers)
> >        9         writer.writerow(headers_list)
> >       10         for item in results:
> > ---> 11             data = [item.get(index, '') for index in results]
> >       12         writer.writerow(data)
> >
> > <ipython-input-746-c822437eeaf0> in <listcomp>(.0)
> >        9         writer.writerow(headers_list)
> >       10         for item in results:
> > ---> 11             data = [item.get(index, '') for index in results]
> >       12         writer.writerow(data)
> >
> > TypeError: unhashable type: 'dict'
> >
> >
> > I have done some research, namely, the following:
> >
> > https://mail.python.org/pipermail//tutor/2011-November/086761.html
> >
> > http://stackoverflow.com/questions/27435798/unhashable-type-dict-type-error
> >
> > http://stackoverflow.com/questions/1957396/why-dict-objects-are-unhashable-in-python
> >
> > However, I am still perplexed by this error. Any feedback is welcomed. Thank you.
> >
> 
> I haven't looked too deeply into the problem as it's 03:45 and I'm just 
> heading off to bed, but I suspect you'd make your life easier by using 
> https://docs.python.org/3/library/csv.html#csv.DictWriter
> 
> -- 
> My fellow Pythonistas, ask not what our language can do for you, ask
> what you can do for our language.
> 
> Mark Lawrence



On Saturday, August 29, 2015 at 10:50:19 PM UTC-4, Mark Lawrence wrote:
> On 30/08/2015 03:05, kbtyo wrote:
> > I am using Jupyter Notebook and Python 3.4. I have a data structure in the format, (type list):
> >
> > [{'AccountNumber': N,
> > 'Amount': '0',
> >   'Answer': '12:00:00 PM',
> >    'ID': None,
> >    'Type': 'WriteLetters',
> >    'Amount': '10',
> >    {'AccountNumber': Y,
> >        'Amount': '0',
> >        'Answer': ' 12:00:00 PM',
> >         'ID': None,
> >        'Type': 'Transfer',
> >        'Amount': '2'}]
> >
> > The end goal is to write this out to CSV.
> >
> > For the above example the output would look like:
> >
> > AccountNumber, Amount, Answer, ID, Type, Amount
> > N,0,12:00:00 PM,None,WriteLetters,10
> > Y,2,12:00:00 PM,None,Transfer,2
> >
> > Below is the function that I am using to write out this data structure. Please excuse any indentation formatting issues. The data structure is returned through the function "construct_results(get_just_xml_data)".
> >
> > The data that is returned is in the format as above. "construct_headers(get_just_xml_data)" returns a list of headers. Writing out the row for "headers_list" works.
> >
> > The list comprehension "data" is to maintain the integrity of the column headers and the values for each new instance of the data structure (where the keys in the dictionary are the headers and values - row instances). The keys in this specific data structure are meant to check if there is a value instance, and if there is not - place an ''.
> >
> > def write_to_csv(results, headers):
> >
> >      headers = construct_headers(get_just_xml_data)
> >      results = construct_results(get_just_xml_data)
> >      headers_list = list(headers)
> >
> >      with open('real_csv_output.csv', 'wt') as f:
> >          writer = csv.writer(f)
> >          writer.writerow(headers_list)
> >          for row in results:
> >              data = [row.get(index, '') for index in results]
> >          writer.writerow(data)
> >
> >
> >
> > However, when I run this, I receive this error:
> >
> > ---------------------------------------------------------------------------
> > TypeError                                 Traceback (most recent call last)
> > <ipython-input-747-7746797fc9a5> in <module>()
> > ----> 1 write_to_csv(results, headers)
> >
> > <ipython-input-746-c822437eeaf0> in write_to_csv(results, headers)
> >        9         writer.writerow(headers_list)
> >       10         for item in results:
> > ---> 11             data = [item.get(index, '') for index in results]
> >       12         writer.writerow(data)
> >
> > <ipython-input-746-c822437eeaf0> in <listcomp>(.0)
> >        9         writer.writerow(headers_list)
> >       10         for item in results:
> > ---> 11             data = [item.get(index, '') for index in results]
> >       12         writer.writerow(data)
> >
> > TypeError: unhashable type: 'dict'
> >
> >
> > I have done some research, namely, the following:
> >
> > https://mail.python.org/pipermail//tutor/2011-November/086761.html
> >
> > http://stackoverflow.com/questions/27435798/unhashable-type-dict-type-error
> >
> > http://stackoverflow.com/questions/1957396/why-dict-objects-are-unhashable-in-python
> >
> > However, I am still perplexed by this error. Any feedback is welcomed. Thank you.
> >
> 
> I haven't looked too deeply into the problem as it's 03:45 and I'm just 
> heading off to bed, but I suspect you'd make your life easier by using 
> https://docs.python.org/3/library/csv.html#csv.DictWriter

I have already looked at this Mark. The problem is that my headers were a set (which were then converted into a list object for the csv writer module to use). I had to resort to a set in order to collect new headers as they were generated on the fly with my XML parser. This is not relevant to this question, but I thought I would provide some context. It would be easier with if each column header mapped perfectly with each value. Turns out they don't. Also, I would not recommend sleeping so late - even on weekends. That can really disrupt your circadian rhythm :)
> 
> -- 
> My fellow Pythonistas, ask not what our language can do for you, ask
> what you can do for our language.
> 
> Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#95775

When you get TypeError: unhashable type: 'dict'
it almost always means that you are trying to use a dict as
a key for a different dict.


Python 2.7.10 (default, Jul  1 2015, 10:54:53)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> d={1:'one', 2:'two', 3:'three'}
 >>> d1={1:'A', 2:'B', 3:'C'}

 This doesn't work.
 
 >>> print d1[d]
 Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
 TypeError: unhashable type: 'dict'

This does.

 >>> for index in d:
   ...     print d1[index]
   ...
   A
   B
   C

HTH,
Laura



	

[toc] | [prev] | [next] | [standalone]

#95776

On Sunday, August 30, 2015 at 1:25:13 PM UTC-4, Laura Creighton wrote:
> When you get TypeError: unhashable type: 'dict'
> it almost always means that you are trying to use a dict as
> a key for a different dict.
> 
> 
> Python 2.7.10 (default, Jul  1 2015, 10:54:53)
> [GCC 4.9.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> d={1:'one', 2:'two', 3:'three'}
>  >>> d1={1:'A', 2:'B', 3:'C'}
> 
>  This doesn't work.
>  
>  >>> print d1[d]
>  Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>  TypeError: unhashable type: 'dict'
> 
> This does.
> 
>  >>> for index in d:
>    ...     print d1[index]
>    ...
>    A
>    B
>    C
> 
> HTH,
> Laura

@Laura - thank you. I observed this earlier. Please see my reply to Ben and MRAB for the most up to date status. 

[toc] | [prev] | [next] | [standalone]

#95777

In a message of Sun, 30 Aug 2015 10:34:18 -0700, kbtyo writes:

>@Laura - thank you. I observed this earlier. Please see my reply to Ben and MRAB for the most up to date status. 

For the more general problem of 'how do I parse my XML'
download and use this package.
https://pypi.python.org/pypi/xmlutils/1.1

It is all set up for reading xml and for writing csv, leaving you
with the manipulation in between (if any) you need.

Laura

[toc] | [prev] | [next] | [standalone]

#95778

On Sunday, August 30, 2015 at 2:21:21 PM UTC-4, Laura Creighton wrote:
> In a message of Sun, 30 Aug 2015 10:34:18 -0700, kbtyo writes:
> 
> >@Laura - thank you. I observed this earlier. Please see my reply to Ben and MRAB for the most up to date status. 
> 
> For the more general problem of 'how do I parse my XML'
> download and use this package.
> https://pypi.python.org/pypi/xmlutils/1.1
> 
> It is all set up for reading xml and for writing csv, leaving you
> with the manipulation in between (if any) you need.
> 
> Laura

I have already addressed this

[toc] | [prev] | [next] | [standalone]

#95761

On 2015-08-30 03:05, kbtyo wrote:
> I am using Jupyter Notebook and Python 3.4. I have a data structure in the format, (type list):
>
> [{'AccountNumber': N,
> 'Amount': '0',
>   'Answer': '12:00:00 PM',
>    'ID': None,
>    'Type': 'WriteLetters',
>    'Amount': '10',
>    {'AccountNumber': Y,
>        'Amount': '0',
>        'Answer': ' 12:00:00 PM',
>         'ID': None,
>        'Type': 'Transfer',
>        'Amount': '2'}]
>
> The end goal is to write this out to CSV.
>
> For the above example the output would look like:
>
> AccountNumber, Amount, Answer, ID, Type, Amount
> N,0,12:00:00 PM,None,WriteLetters,10
> Y,2,12:00:00 PM,None,Transfer,2
>
> Below is the function that I am using to write out this data structure. Please excuse any indentation formatting issues. The data structure is returned through the function "construct_results(get_just_xml_data)".
>
> The data that is returned is in the format as above. "construct_headers(get_just_xml_data)" returns a list of headers. Writing out the row for "headers_list" works.
>
> The list comprehension "data" is to maintain the integrity of the column headers and the values for each new instance of the data structure (where the keys in the dictionary are the headers and values - row instances). The keys in this specific data structure are meant to check if there is a value instance, and if there is not - place an ''.
>
> def write_to_csv(results, headers):
>
>      headers = construct_headers(get_just_xml_data)
>      results = construct_results(get_just_xml_data)
>      headers_list = list(headers)
>
>      with open('real_csv_output.csv', 'wt') as f:
>          writer = csv.writer(f)
>          writer.writerow(headers_list)
>          for row in results:
>              data = [row.get(index, '') for index in results]
>          writer.writerow(data)
>
>
>
> However, when I run this, I receive this error:
>
> ---------------------------------------------------------------------------
> TypeError                                 Traceback (most recent call last)
> <ipython-input-747-7746797fc9a5> in <module>()
> ----> 1 write_to_csv(results, headers)
>
> <ipython-input-746-c822437eeaf0> in write_to_csv(results, headers)
>        9         writer.writerow(headers_list)
>       10         for item in results:
> ---> 11             data = [item.get(index, '') for index in results]
>       12         writer.writerow(data)
>
> <ipython-input-746-c822437eeaf0> in <listcomp>(.0)
>        9         writer.writerow(headers_list)
>       10         for item in results:
> ---> 11             data = [item.get(index, '') for index in results]
>       12         writer.writerow(data)
>
> TypeError: unhashable type: 'dict'
>
>
> I have done some research, namely, the following:
>
> https://mail.python.org/pipermail//tutor/2011-November/086761.html
>
> http://stackoverflow.com/questions/27435798/unhashable-type-dict-type-error
>
> http://stackoverflow.com/questions/1957396/why-dict-objects-are-unhashable-in-python
>
> However, I am still perplexed by this error. Any feedback is welcomed. Thank you.
>
You're taking the index values from 'results' instead of 'headers'.

[toc] | [prev] | [next] | [standalone]

#95770

On Saturday, August 29, 2015 at 10:50:18 PM UTC-4, MRAB wrote:
> On 2015-08-30 03:05, kbtyo wrote:
> > I am using Jupyter Notebook and Python 3.4. I have a data structure in the format, (type list):
> >
> > [{'AccountNumber': N,
> > 'Amount': '0',
> >   'Answer': '12:00:00 PM',
> >    'ID': None,
> >    'Type': 'WriteLetters',
> >    'Amount': '10',
> >    {'AccountNumber': Y,
> >        'Amount': '0',
> >        'Answer': ' 12:00:00 PM',
> >         'ID': None,
> >        'Type': 'Transfer',
> >        'Amount': '2'}]
> >
> > The end goal is to write this out to CSV.
> >
> > For the above example the output would look like:
> >
> > AccountNumber, Amount, Answer, ID, Type, Amount
> > N,0,12:00:00 PM,None,WriteLetters,10
> > Y,2,12:00:00 PM,None,Transfer,2
> >
> > Below is the function that I am using to write out this data structure. Please excuse any indentation formatting issues. The data structure is returned through the function "construct_results(get_just_xml_data)".
> >
> > The data that is returned is in the format as above. "construct_headers(get_just_xml_data)" returns a list of headers. Writing out the row for "headers_list" works.
> >
> > The list comprehension "data" is to maintain the integrity of the column headers and the values for each new instance of the data structure (where the keys in the dictionary are the headers and values - row instances). The keys in this specific data structure are meant to check if there is a value instance, and if there is not - place an ''.
> >
> > def write_to_csv(results, headers):
> >
> >      headers = construct_headers(get_just_xml_data)
> >      results = construct_results(get_just_xml_data)
> >      headers_list = list(headers)
> >
> >      with open('real_csv_output.csv', 'wt') as f:
> >          writer = csv.writer(f)
> >          writer.writerow(headers_list)
> >          for row in results:
> >              data = [row.get(index, '') for index in results]
> >          writer.writerow(data)
> >
> >
> >
> > However, when I run this, I receive this error:
> >
> > ---------------------------------------------------------------------------
> > TypeError                                 Traceback (most recent call last)
> > <ipython-input-747-7746797fc9a5> in <module>()
> > ----> 1 write_to_csv(results, headers)
> >
> > <ipython-input-746-c822437eeaf0> in write_to_csv(results, headers)
> >        9         writer.writerow(headers_list)
> >       10         for item in results:
> > ---> 11             data = [item.get(index, '') for index in results]
> >       12         writer.writerow(data)
> >
> > <ipython-input-746-c822437eeaf0> in <listcomp>(.0)
> >        9         writer.writerow(headers_list)
> >       10         for item in results:
> > ---> 11             data = [item.get(index, '') for index in results]
> >       12         writer.writerow(data)
> >
> > TypeError: unhashable type: 'dict'
> >
> >
> > I have done some research, namely, the following:
> >
> > https://mail.python.org/pipermail//tutor/2011-November/086761.html
> >
> > http://stackoverflow.com/questions/27435798/unhashable-type-dict-type-error
> >
> > http://stackoverflow.com/questions/1957396/why-dict-objects-are-unhashable-in-python
> >
> > However, I am still perplexed by this error. Any feedback is welcomed. Thank you.
> >
> You're taking the index values from 'results' instead of 'headers'.

Would you be able to elaborate on this? I partially understand what you mean. However, each dictionary (of results) has the same keys to map to (aka, headers when written out to CSV). I am wondering if you would be able to explain how the index is being used in this case?

[toc] | [prev] | [next] | [standalone]

#95773

On 2015-08-30 17:31, kbtyo wrote:
> On Saturday, August 29, 2015 at 10:50:18 PM UTC-4, MRAB wrote:
>> On 2015-08-30 03:05, kbtyo wrote:
>> > I am using Jupyter Notebook and Python 3.4. I have a data structure in the format, (type list):
>> >
>> > [{'AccountNumber': N,
>> > 'Amount': '0',
>> >   'Answer': '12:00:00 PM',
>> >    'ID': None,
>> >    'Type': 'WriteLetters',
>> >    'Amount': '10',
>> >    {'AccountNumber': Y,
>> >        'Amount': '0',
>> >        'Answer': ' 12:00:00 PM',
>> >         'ID': None,
>> >        'Type': 'Transfer',
>> >        'Amount': '2'}]
>> >
>> > The end goal is to write this out to CSV.
>> >
>> > For the above example the output would look like:
>> >
>> > AccountNumber, Amount, Answer, ID, Type, Amount
>> > N,0,12:00:00 PM,None,WriteLetters,10
>> > Y,2,12:00:00 PM,None,Transfer,2
>> >
>> > Below is the function that I am using to write out this data structure. Please excuse any indentation formatting issues. The data structure is returned through the function "construct_results(get_just_xml_data)".
>> >
>> > The data that is returned is in the format as above. "construct_headers(get_just_xml_data)" returns a list of headers. Writing out the row for "headers_list" works.
>> >
>> > The list comprehension "data" is to maintain the integrity of the column headers and the values for each new instance of the data structure (where the keys in the dictionary are the headers and values - row instances). The keys in this specific data structure are meant to check if there is a value instance, and if there is not - place an ''.
>> >
>> > def write_to_csv(results, headers):
>> >
>> >      headers = construct_headers(get_just_xml_data)
>> >      results = construct_results(get_just_xml_data)
>> >      headers_list = list(headers)
>> >
>> >      with open('real_csv_output.csv', 'wt') as f:
>> >          writer = csv.writer(f)
>> >          writer.writerow(headers_list)
>> >          for row in results:
>> >              data = [row.get(index, '') for index in results]
>> >          writer.writerow(data)
>> >
>> >
>> >
>> > However, when I run this, I receive this error:
>> >
>> > ---------------------------------------------------------------------------
>> > TypeError                                 Traceback (most recent call last)
>> > <ipython-input-747-7746797fc9a5> in <module>()
>> > ----> 1 write_to_csv(results, headers)
>> >
>> > <ipython-input-746-c822437eeaf0> in write_to_csv(results, headers)
>> >        9         writer.writerow(headers_list)
>> >       10         for item in results:
>> > ---> 11             data = [item.get(index, '') for index in results]
>> >       12         writer.writerow(data)
>> >
>> > <ipython-input-746-c822437eeaf0> in <listcomp>(.0)
>> >        9         writer.writerow(headers_list)
>> >       10         for item in results:
>> > ---> 11             data = [item.get(index, '') for index in results]
>> >       12         writer.writerow(data)
>> >
>> > TypeError: unhashable type: 'dict'
>> >
>> >
>> > I have done some research, namely, the following:
>> >
>> > https://mail.python.org/pipermail//tutor/2011-November/086761.html
>> >
>> > http://stackoverflow.com/questions/27435798/unhashable-type-dict-type-error
>> >
>> > http://stackoverflow.com/questions/1957396/why-dict-objects-are-unhashable-in-python
>> >
>> > However, I am still perplexed by this error. Any feedback is welcomed. Thank you.
>> >
>> You're taking the index values from 'results' instead of 'headers'.
>
> Would you be able to elaborate on this? I partially understand what you mean. However, each dictionary (of results) has the same keys to map to (aka, headers when written out to CSV). I am wondering if you would be able to explain how the index is being used in this case?
>
In the list comprehension on line 11, you have "item.get(index, '')".

What is 'index'?

You have "for index in results" in the list comprehension, and 'results' 
is a list of dicts, therefore 'index' is a _dict_.

That means that you're trying to look up an entry in the 'item' dict
using a _dict_ as the key.

Oh, and incidentally, line 12 should be indented to the same level as
line 11.

[toc] | [prev] | [next] | [standalone]

#95774

On Sunday, August 30, 2015 at 1:16:12 PM UTC-4, MRAB wrote:
> On 2015-08-30 17:31, kbtyo wrote:
> > On Saturday, August 29, 2015 at 10:50:18 PM UTC-4, MRAB wrote:
> >> On 2015-08-30 03:05, kbtyo wrote:
> >> > I am using Jupyter Notebook and Python 3.4. I have a data structure in the format, (type list):
> >> >
> >> > [{'AccountNumber': N,
> >> > 'Amount': '0',
> >> >   'Answer': '12:00:00 PM',
> >> >    'ID': None,
> >> >    'Type': 'WriteLetters',
> >> >    'Amount': '10',
> >> >    {'AccountNumber': Y,
> >> >        'Amount': '0',
> >> >        'Answer': ' 12:00:00 PM',
> >> >         'ID': None,
> >> >        'Type': 'Transfer',
> >> >        'Amount': '2'}]
> >> >
> >> > The end goal is to write this out to CSV.
> >> >
> >> > For the above example the output would look like:
> >> >
> >> > AccountNumber, Amount, Answer, ID, Type, Amount
> >> > N,0,12:00:00 PM,None,WriteLetters,10
> >> > Y,2,12:00:00 PM,None,Transfer,2
> >> >
> >> > Below is the function that I am using to write out this data structure. Please excuse any indentation formatting issues. The data structure is returned through the function "construct_results(get_just_xml_data)".
> >> >
> >> > The data that is returned is in the format as above. "construct_headers(get_just_xml_data)" returns a list of headers. Writing out the row for "headers_list" works.
> >> >
> >> > The list comprehension "data" is to maintain the integrity of the column headers and the values for each new instance of the data structure (where the keys in the dictionary are the headers and values - row instances). The keys in this specific data structure are meant to check if there is a value instance, and if there is not - place an ''.
> >> >
> >> > def write_to_csv(results, headers):
> >> >
> >> >      headers = construct_headers(get_just_xml_data)
> >> >      results = construct_results(get_just_xml_data)
> >> >      headers_list = list(headers)
> >> >
> >> >      with open('real_csv_output.csv', 'wt') as f:
> >> >          writer = csv.writer(f)
> >> >          writer.writerow(headers_list)
> >> >          for row in results:
> >> >              data = [row.get(index, '') for index in results]
> >> >          writer.writerow(data)
> >> >
> >> >
> >> >
> >> > However, when I run this, I receive this error:
> >> >
> >> > ---------------------------------------------------------------------------
> >> > TypeError                                 Traceback (most recent call last)
> >> > <ipython-input-747-7746797fc9a5> in <module>()
> >> > ----> 1 write_to_csv(results, headers)
> >> >
> >> > <ipython-input-746-c822437eeaf0> in write_to_csv(results, headers)
> >> >        9         writer.writerow(headers_list)
> >> >       10         for item in results:
> >> > ---> 11             data = [item.get(index, '') for index in results]
> >> >       12         writer.writerow(data)
> >> >
> >> > <ipython-input-746-c822437eeaf0> in <listcomp>(.0)
> >> >        9         writer.writerow(headers_list)
> >> >       10         for item in results:
> >> > ---> 11             data = [item.get(index, '') for index in results]
> >> >       12         writer.writerow(data)
> >> >
> >> > TypeError: unhashable type: 'dict'
> >> >
> >> >
> >> > I have done some research, namely, the following:
> >> >
> >> > https://mail.python.org/pipermail//tutor/2011-November/086761.html
> >> >
> >> > http://stackoverflow.com/questions/27435798/unhashable-type-dict-type-error
> >> >
> >> > http://stackoverflow.com/questions/1957396/why-dict-objects-are-unhashable-in-python
> >> >
> >> > However, I am still perplexed by this error. Any feedback is welcomed. Thank you.
> >> >
> >> You're taking the index values from 'results' instead of 'headers'.
> >
> > Would you be able to elaborate on this? I partially understand what you mean. However, each dictionary (of results) has the same keys to map to (aka, headers when written out to CSV). I am wondering if you would be able to explain how the index is being used in this case?
> >
> In the list comprehension on line 11, you have "item.get(index, '')".
> 
> What is 'index'?
> 
> You have "for index in results" in the list comprehension, and 'results' 
> is a list of dicts, therefore 'index' is a _dict_.
> 
> That means that you're trying to look up an entry in the 'item' dict
> using a _dict_ as the key.
> 
> Oh, and incidentally, line 12 should be indented to the same level as
> line 11.

Yes, as mentioned in my OP, please forgive formatting issues with indentation:

I feel that I need to provide some context to avoid any confusion over my motivations for choosing to do something. 

My original task was to parse an XML data structure stored in a CSV file with other data types and then add the elements back as headers and the text as row values. I went back to drawing board and creating a "results" list of dictionaries where the keys have values as lists using this. 

def convert_list_to_dict(get_just_xml_data): 
    d = {} 
    for item in get_just_xml_data(get_all_data): 
        for k, v in item.items(): 
            try: 
                d[k].append(v) 
            except KeyError: 
                d[k] = [v] 
    return d 

This creates a dictionary for each XML tag - for example: 
{ 
 'Number1': ['0'], 
 'Number2': ['0'], 
 'Number3': ['0'], 
 'Number4': ['0'], 
 'Number5': ['0'], 
 'RepgenName': [None], 
 'RTpes': ['Execution', 'Letters'], 
 'RTID': ['3', '5']} 


I then used this to create a "headers" set (to prevent duplicates to be added) and the list of dictionaries that I mentioned in my OP. 


I achieve this via: 

#just headers 
def construct_headers(convert_list_to_dict): 
    header = set() 
    with open('real.csv', 'rU') as infile: 
              reader = csv.DictReader(infile) 
              for row in reader: 
                    xml_data = convert_list_to_dict(get_just_xml_data) #get_just_xml_data(get_all_data) 
                    row.update(xml_data)                                 
                    header.update(row.keys()) 
    return header 

#get all of the results 
def construct_results(convert_list_to_dict): 
    header = set() 
    results = [] 
    with open('real.csv', 'rU') as infile: 
              reader = csv.DictReader(infile) 
              for row in reader: 
                    xml_data = convert_list_to_dict(get_just_xml_data) #get_just_xml_data(get_all_data) 
#                     print(row) 
                    row.update(xml_data) 
#                     print(row) 
                    results.append(row) 
#                     print(results) 
                    header.update(row.keys()) 
#     print(type(results)) 
    return results 


I guess I am using the headers list originally written out. My initial thought is to just write out the values corresponding with each transaction. For example, citing this data structure: 

{ 
 'Number1': ['0'], 
 'Number2': ['0'], 
 'Number3': ['0'], 
 'Number4': ['0'], 
 'Number5': ['0'], 
 'RPN': [None], 
 'RTypes': ['Execution', 'Letters'], 
 'RTID': ['3', '5']} 


I would get a CSV 

Number1, Number2, Number3, Number4, Number5, RPN, RTypes,RTID 

0, 0, 0, 0, 0, None, Execution, 3 
None, None, None,None,None, Letters, 5 

I am wondering how I would achieve this when all of the headers set is not sorted (should I do so before writing this out?). Also, since I have millions of transactions I want to make sure that the values for each of the headers is sequentially placed. Any guidance would be very helpful. Thanks.

[toc] | [prev] | [next] | [standalone]

#95762

kbtyo <ahlusar.ahluwalia@gmail.com> writes:

> I am using Jupyter Notebook and Python 3.4.

Thank you for saying so! It is not always required, but when it matters,
this information is important to state up front.

> I have a data structure in the format, (type list):
>
> [{'AccountNumber': N,
> 'Amount': '0',
>  'Answer': '12:00:00 PM',
>   'ID': None,
>   'Type': 'WriteLetters',
>   'Amount': '10',
>   {'AccountNumber': Y,
>       'Amount': '0',
>       'Answer': ' 12:00:00 PM',       
>        'ID': None,
>       'Type': 'Transfer',
>       'Amount': '2'}]
>
> The end goal is to write this out to CSV.

So that assumes that *every* item will be a mapping with all the same
keys. CSV is limited to a sequence of “records” which all have the same
fields in the same order.

> The list comprehension "data" is to maintain the integrity of the
> column headers and the values for each new instance of the data
> structure (where the keys in the dictionary are the headers and values
> - row instances). The keys in this specific data structure are meant
> to check if there is a value instance, and if there is not - place an
> ''.
>

[…]
>         for row in results:
>             data = [row.get(index, '') for index in results]

The ‘for’ statement iterates over ‘results’, getting an item each time.
The name ‘row’ is bound to each item in turn.

Then, each time through the ‘for’ loop, you iterate *again* over
‘results’. The name ‘index’ is bound to each item.

You then attempt to use the dict (each item from ‘results’ is itself a
dict) as a key into that same dict. A dict is not a valid key; it is not
a “hashable type” i.e. a type with a fixed value, that can produce a
hash of the value).

So you're getting dicts and attempting to use those dicts as keys into
dicts. That will give the error “TypeError: unhashable type: 'dict'”.

I think what you want is not items from the original sequence, but the
keys from the mapping::

    for input_record in results:
        output_record = [input_record.get(key, "") for key in input_record]

But you're then throwing away the constructed list, since you do nothing
with it before the end of the loop.

>         writer.writerow(data)

This statement occurs only *after* all the items from ‘results’ have
been iterated. You will only have the most recent constructed row.

Perhaps you want::

    for input_record in results:
        output_record = [input_record.get(key, "") for key in input_record]
        writer.writerow(output_record)

-- 
 \       “An idea isn't responsible for the people who believe in it.” |
  `\                                      —Donald Robert Perry Marquis |
_o__)                                                                  |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#95772

On Saturday, August 29, 2015 at 11:04:53 PM UTC-4, Ben Finney wrote:
> kbtyo writes:
> 
> > I am using Jupyter Notebook and Python 3.4.
> 
> Thank you for saying so! It is not always required, but when it matters,
> this information is important to state up front.
> 
> > I have a data structure in the format, (type list):
> >
> > [{'AccountNumber': N,
> > 'Amount': '0',
> >  'Answer': '12:00:00 PM',
> >   'ID': None,
> >   'Type': 'WriteLetters',
> >   'Amount': '10',
> >   {'AccountNumber': Y,
> >       'Amount': '0',
> >       'Answer': ' 12:00:00 PM',       
> >        'ID': None,
> >       'Type': 'Transfer',
> >       'Amount': '2'}]
> >
> > The end goal is to write this out to CSV.
> 
> So that assumes that *every* item will be a mapping with all the same
> keys. CSV is limited to a sequence of "records" which all have the same
> fields in the same order.

This clue tipped me off that I wasn't collecting the newly generate key value pairs from my XML parser properly. I was using the dictionary built in method update to update the keys. The terrible thing was that the returned dictionary was only updated with the last keys and values. What a couple of hours of shut eye can do for the mind and body. 

> 
> > The list comprehension "data" is to maintain the integrity of the
> > column headers and the values for each new instance of the data
> > structure (where the keys in the dictionary are the headers and values
> > - row instances). The keys in this specific data structure are meant
> > to check if there is a value instance, and if there is not - place an
> > ''.
> >
> 
> [...]
> >         for row in results:
> >             data = [row.get(index, '') for index in results]
> 
> The 'for' statement iterates over 'results', getting an item each time.
> The name 'row' is bound to each item in turn.
> 
> Then, each time through the 'for' loop, you iterate *again* over
> 'results'. The name 'index' is bound to each item.
> 
> You then attempt to use the dict (each item from 'results' is itself a
> dict) as a key into that same dict. A dict is not a valid key; it is not
> a "hashable type" i.e. a type with a fixed value, that can produce a
> hash of the value).

I discovered that. I need to iterate again to access the keys and values. 
> 
> So you're getting dicts and attempting to use those dicts as keys into
> dicts. That will give the error "TypeError: unhashable type: 'dict'".
> 
> I think what you want is not items from the original sequence, but the
> keys from the mapping::
> 
>     for input_record in results:
>         output_record = [input_record.get(key, "") for key in input_record]
> 
> But you're then throwing away the constructed list, since you do nothing
> with it before the end of the loop.
> 
> >         writer.writerow(data)
> 
> This statement occurs only *after* all the items from 'results' have
> been iterated. You will only have the most recent constructed row.
> 
> Perhaps you want::
> 
>     for input_record in results:
>         output_record = [input_record.get(key, "") for key in input_record]
>         writer.writerow(output_record)
> 

I tried this and some of the values maintained integrity and some were rewritten by a previous dictionary's values. 


> -- 
>  \       "An idea isn't responsible for the people who believe in it." |
>   `\                                      --Donald Robert Perry Marquis |
> _o__)                                                                  |
> Ben Finney

@BenFinney:

I feel that I need to provide some context to avoid any confusion over my motivations for choosing to do something. 

My original task was to parse an XML data structure stored in a CSV file with other data types and then add the elements back as headers and the text as row values. I went back to drawing board and creating a "results" list of dictionaries where the keys have values as lists using this. 

def convert_list_to_dict(get_just_xml_data):
    d = {}
    for item in get_just_xml_data(get_all_data):
        for k, v in item.items():
            try:
                d[k].append(v)
            except KeyError:
                d[k] = [v]
    return d 

This creates a dictionary for each XML tag - for example: 
{
 'Number1': ['0'],
 'Number2': ['0'],
 'Number3': ['0'],
 'Number4': ['0'],
 'Number5': ['0'],
 'RepgenName': [None],
 'RTpes': ['Execution', 'Letters'],
 'RTID': ['3', '5']}


I then used this to create a "headers" set (to prevent duplicates to be added) and the list of dictionaries that I mentioned in my OP. 


I achieve this via:

#just headers
def construct_headers(convert_list_to_dict):
    header = set()
    with open('real.csv', 'rU') as infile:
              reader = csv.DictReader(infile)
              for row in reader:
                    xml_data = convert_list_to_dict(get_just_xml_data) #get_just_xml_data(get_all_data)
                    row.update(xml_data)                                 
                    header.update(row.keys())
    return header

#get all of the results
def construct_results(convert_list_to_dict):
    header = set()
    results = []
    with open('real.csv', 'rU') as infile:
              reader = csv.DictReader(infile)
              for row in reader:
                    xml_data = convert_list_to_dict(get_just_xml_data) #get_just_xml_data(get_all_data)
#                     print(row)
                    row.update(xml_data)
#                     print(row)
                    results.append(row)
#                     print(results)
                    header.update(row.keys())
#     print(type(results))
    return results


I guess I am using the headers list originally written out. My initial thought is to just write out the values corresponding with each transaction. For example, citing this data structure:

{
 'Number1': ['0'],
 'Number2': ['0'],
 'Number3': ['0'],
 'Number4': ['0'],
 'Number5': ['0'],
 'RPN': [None],
 'RTypes': ['Execution', 'Letters'],
 'RTID': ['3', '5']}


I would get a CSV 

Number1, Number2, Number3, Number4, Number5, RPN, RTypes,RTID

0, 0, 0, 0, 0, None, Execution, 3
None, None, None,None,None, Letters, 5

I am wondering how I would achieve this when all of the headers set is not sorted (should I do so before writing this out?). Also, since I have millions of transactions I want to make sure that the values for each of the headers is sequentially placed. Any guidance would be very helpful. Thanks.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web