Path: csiph.com!usenet.pasdenom.info!aioe.org!news.stack.nl!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
To: python-list@python.org
From: Dennis Lee Bieber <wlfraed@ix.netcom.com>
Subject: Re: Logging data from Arduino using PySerial
Date: Tue, 04 Feb 2014 08:56:12 -0500
Organization: IISS Elusive Unicorn
References: <e3b195f8-5a17-4536-9926-2b2ab193719c@googlegroups.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.6387.1391522176.18130.python-list@python.org>
Lines: 81
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:65423

On Mon, 3 Feb 2014 20:07:48 -0800 (PST), Thomas <t.tchorzewski@gmail.com>
declaimed the following:

>I've written a script to log data from my Arduino to a csv file. The script works well enough but it's very, very slow. I'm quite new to Python and I just wanted to put this out there to see if any Python experts could help optimise my code. Here it is:
>
>    import serial
>    import re
>    import csv
>    import numpy as np
>    import matplotlib.pyplot as plt
>    
>    portPath = "/dev/ttyACM0"
>    baud = 9600
>    sample_time = 0.5
>    sim_time = 30
>    
>    
>    # Initializing Lists
>    # Data Collection
>    data_log = []
>    line_data = []
>    
>    def map(x, in_min, in_max, out_min, out_max):
>        return (((x - in_min) * (out_max - out_min))/(in_max - in_min)) + out_min
>
	Doesn't the Arduino have a map() function internally? If you have
control over the Arduino couldn't you set it up to return the desired
mapping values directly?
        
>    # Establishing Serial Connection
>    connection = serial.Serial(portPath,baud)
>    
>    # Calculating the length of data to collect based on the
>    # sample time and simulation time (set by user)
>    max_length = sim_time/sample_time
>    
>    # Collecting the data from the serial port
>    while True:
>        data_log.append(connection.readline())
>        if len(data_log) > max_length - 1:
>            break
>
	Here you are building up a list of raw lines...
                
>    # Cleaning the data_log and storing it in data.csv
>    with open('data.csv','wb') as csvfile:
>        for line in data_log:
>            line_data = re.findall('\d*\.\d*',line) # Find all digits
>            line_data = filter(None,line_data)    # Filter out empty strings
>            line_data = [float(x) for x in line_data] # Convert Strings to float
>
>            for i in range(1,len(line_data)):
>                line_data[i]=map(line_data[i],0,1023,0,5)
>                
>            csvwrite = csv.writer(csvfile)

	You are creating a new csv writer instance on each pass!

>            csvwrite.writerow(line_data)
>
	And then you loop over all the lines looking for particular values,
just to scale them into another range, to write to a CSV file.

	Personally, I'd have opened the CSV file at the start, and done all
this filtering/transforming on each line as it was read from the Arduino.

	csvfile = open("data.csv", "wb")
	csvwriter = csv.writer(csvfile)
	line = ""
	while len(line) < max_length:
		if len(line) == 0:	#skip first line (which your range(1,...) does
			line = connection.readline()
		line = connection.readline()
		# do all your filtering here
		if line:		#not empty, so filtering didn't wipe it out
			  csvwrite.writerow(line)
	csvfile.close()
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/