Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #65423

Re: Logging data from Arduino using PySerial

Show key headers only | View raw

On Mon, 3 Feb 2014 20:07:48 -0800 (PST), Thomas <t.tchorzewski@gmail.com>
declaimed the following:

>I've written a script to log data from my Arduino to a csv file. The script works well enough but it's very, very slow. I'm quite new to Python and I just wanted to put this out there to see if any Python experts could help optimise my code. Here it is:
>
>    import serial
>    import re
>    import csv
>    import numpy as np
>    import matplotlib.pyplot as plt
>    
>    portPath = "/dev/ttyACM0"
>    baud = 9600
>    sample_time = 0.5
>    sim_time = 30
>    
>    
>    # Initializing Lists
>    # Data Collection
>    data_log = []
>    line_data = []
>    
>    def map(x, in_min, in_max, out_min, out_max):
>        return (((x - in_min) * (out_max - out_min))/(in_max - in_min)) + out_min
>
	Doesn't the Arduino have a map() function internally? If you have
control over the Arduino couldn't you set it up to return the desired
mapping values directly?
        
>    # Establishing Serial Connection
>    connection = serial.Serial(portPath,baud)
>    
>    # Calculating the length of data to collect based on the
>    # sample time and simulation time (set by user)
>    max_length = sim_time/sample_time
>    
>    # Collecting the data from the serial port
>    while True:
>        data_log.append(connection.readline())
>        if len(data_log) > max_length - 1:
>            break
>
	Here you are building up a list of raw lines...
                
>    # Cleaning the data_log and storing it in data.csv
>    with open('data.csv','wb') as csvfile:
>        for line in data_log:
>            line_data = re.findall('\d*\.\d*',line) # Find all digits
>            line_data = filter(None,line_data)    # Filter out empty strings
>            line_data = [float(x) for x in line_data] # Convert Strings to float
>
>            for i in range(1,len(line_data)):
>                line_data[i]=map(line_data[i],0,1023,0,5)
>                
>            csvwrite = csv.writer(csvfile)

	You are creating a new csv writer instance on each pass!

>            csvwrite.writerow(line_data)
>
	And then you loop over all the lines looking for particular values,
just to scale them into another range, to write to a CSV file.

	Personally, I'd have opened the CSV file at the start, and done all
this filtering/transforming on each line as it was read from the Arduino.

	csvfile = open("data.csv", "wb")
	csvwriter = csv.writer(csvfile)
	line = ""
	while len(line) < max_length:
		if len(line) == 0:	#skip first line (which your range(1,...) does
			line = connection.readline()
		line = connection.readline()
		# do all your filtering here
		if line:		#not empty, so filtering didn't wipe it out
			  csvwrite.writerow(line)
	csvfile.close()
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Logging data from Arduino using PySerial Thomas <t.tchorzewski@gmail.com> - 2014-02-03 20:07 -0800
  Re: Logging data from Arduino using PySerial Chris Angelico <rosuav@gmail.com> - 2014-02-04 15:47 +1100
    Re: Logging data from Arduino using PySerial Thomas <t.tchorzewski@gmail.com> - 2014-02-03 20:57 -0800
      Re: Logging data from Arduino using PySerial Chris Angelico <rosuav@gmail.com> - 2014-02-04 16:18 +1100
  Re: Logging data from Arduino using PySerial Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-02-04 08:56 -0500
  Re: Logging data from Arduino using PySerial MRAB <python@mrabarnett.plus.com> - 2014-02-04 14:05 +0000

csiph-web