Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #62426 > unrolled thread

How to use the method loadtxt() of numpy neatly?

Back to article view | Back to comp.lang.python

Contents

  How to use the method loadtxt() of numpy neatly? chao dong <neutronest@gmail.com> - 2013-12-19 21:48 -0800
    Re: How to use the method loadtxt() of numpy neatly? rusi <rustompmody@gmail.com> - 2013-12-20 01:45 -0800
    Re: How to use the method loadtxt() of numpy neatly? Peter Otten <__peter__@web.de> - 2013-12-20 11:45 +0100

#62426 — How to use the method loadtxt() of numpy neatly?

    HI, everybody. When I try to use numpy to deal with my dataset in the style of csv, I face a little problem.
    
    In my dataset of the csv file, some columns are string that can not convert to float easily. Some of them can ignore, but other columns I need to change the data to a enum style.

    for example, one column just contain three kinds : S,Q,C. Each of them can declare one meaning, so I must convert them to a dict just like {1,2,3}
    
    Now the question is, when I use numpy.loadtxt, I must do all things above in just one line and one fuction. So as a new user in numpy, I don't know how to solve it.

    Thank you.

[toc] | [next] | [standalone]

#62432

On Friday, December 20, 2013 11:18:53 AM UTC+5:30, chao dong wrote:
> HI, everybody. When I try to use numpy to deal with my dataset in the style of csv, I face a little problem.

>     In my dataset of the csv file, some columns are string that can not convert to float easily. Some of them can ignore, but other columns I need to change the data to a enum style.

>     for example, one column just contain three kinds : S,Q,C. Each of them can declare one meaning, so I must convert them to a dict just like {1,2,3}

What does "dict like {1,2,3}" mean??

On recent python thats a set
On older ones its probably an error.
So you can mean one of:
1. Set([1,2,3])
2. List: [1,2,3]
3. Tuple: (1,2,3)
4. Dict: {"S":1, "Q":2, "C":3}
5. An enumeration (on very recent pythons)
6. A simulation of an enum using classes (or somesuch)
7. Something else


>     Now the question is, when I use numpy.loadtxt, I must do all things above in just one line and one fuction. So as a new user in numpy, I don't know how to solve it.

I suggest you supply a couple of rows of your input
And the corresponding python data-structures you desire
Someone should then suggest how to go about it

[toc] | [prev] | [next] | [standalone]

#62433

chao dong wrote:

>     HI, everybody. When I try to use numpy to deal with my dataset in the
>     style of csv, I face a little problem.
>     
>     In my dataset of the csv file, some columns are string that can not
>     convert to float easily. Some of them can ignore, but other columns I
>     need to change the data to a enum style.
> 
>     for example, one column just contain three kinds : S,Q,C. Each of them
>     can declare one meaning, so I must convert them to a dict just like
>     {1,2,3}
>     
>     Now the question is, when I use numpy.loadtxt, I must do all things
>     above in just one line and one fuction. So as a new user in numpy, I
>     don't know how to solve it.
> 
>     Thank you.

Here's a standalone demo:

import numpy

_lookup={"A": 1, "B": 2}
def convert(x):
    return _lookup.get(x, -1)

converters = {
    0: convert, # in column 0 convert "A" --> 1, "B" --> 2, 
                # anything else to -1
}


if __name__ == "__main__":
    # generate csv
    with open("tmp_sample.csv", "wb") as f:
        f.write("""\
A,1,this,67.8
B,2,should,56.7
C,3,be,34.5
A,4,skipped,12.3
""")

    # load csv
    a = numpy.loadtxt(
        "tmp_sample.csv",
        converters=converters,
        delimiter=",",
        usecols=(0, 1, 3) # skip third column
        )
    print a

Does that help?

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web