Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.soft-sys.math.mathematica > #14859 > unrolled thread
| Started by | howardfink@gmail.com |
|---|---|
| First post | 2013-06-14 08:54 +0000 |
| Last post | 2013-06-15 08:13 +0000 |
| Articles | 3 — 3 participants |
Back to article view | Back to comp.soft-sys.math.mathematica
Importing a file and extracting data howardfink@gmail.com - 2013-06-14 08:54 +0000
Re: Importing a file and extracting data David Bailey <dave@removedbailey.co.uk> - 2013-06-15 08:13 +0000
Re: Importing a file and extracting data Helen Read <readhpr@gmail.com> - 2013-06-15 08:13 +0000
| From | howardfink@gmail.com |
|---|---|
| Date | 2013-06-14 08:54 +0000 |
| Subject | Importing a file and extracting data |
| Message-ID | <kpeln9$htb$1@smc.vnet.net> |
I have a series of files of this form: June 7, 2013 Tc+Naphthalene vs Temperature (oC) Run 1 Tc_Naph_84_2C 3.740 ns Tc_Naph_87_1C 3.731 ns Tc_Naph_89_9C 3.720 ns Tc_Naph_92_9C 3.704 ns Tc_Naph_94_7C 3.687 ns Tc_Naph_97_6C 3.694 ns Run 2 Tc_Naph_83_2C 3.758 ns Tc_Naph_83_4C 3.750 ns Tc_Naph_86_4C 3.728 ns Tc_Naph_88_1C 3.725 ns Tc_Naph_90_2C 3.716 ns Tc_Naph_93_1C 3.704 ns Tc_Naph_94_7C 3.673 ns Tc_Naph_97_7C 3.684 ns Tc_Naph_97_9C 3.665 ns I used an Import command to read in the file, but now I am just sitting and= staring, without a clue how to get the 84_2 converted to the number 84.2,e= tc. and ending up with two lists: Run 1 and Run 2, consisting of pairs of t= emperature and time. The temperature will eventually be converted to 1/abs= olute temperature. I've read lots and lots of help, thumbed through dozens of pages of a Mathe= matica 5 manual, and don't know where to start. I'm trying to help a 90-ye= ar-old chemistry professor, who is currently using a calculator, but there= will be dozens of runs of this experiment.
[toc] | [next] | [standalone]
| From | David Bailey <dave@removedbailey.co.uk> |
|---|---|
| Date | 2013-06-15 08:13 +0000 |
| Message-ID | <kph7na$2di$1@smc.vnet.net> |
| In reply to | #14859 |
On 14/06/2013 09:54, howardfink@gmail.com wrote:
> I have a series of files of this form:
>
> June 7, 2013
> Tc+Naphthalene vs Temperature (oC)
>
> Run 1
> Tc_Naph_84_2C 3.740 ns
> Tc_Naph_87_1C 3.731 ns
> Tc_Naph_89_9C 3.720 ns
> Tc_Naph_92_9C 3.704 ns
> Tc_Naph_94_7C 3.687 ns
> Tc_Naph_97_6C 3.694 ns
>
>
> Run 2
> Tc_Naph_83_2C 3.758 ns
> Tc_Naph_83_4C 3.750 ns
> Tc_Naph_86_4C 3.728 ns
> Tc_Naph_88_1C 3.725 ns
> Tc_Naph_90_2C 3.716 ns
> Tc_Naph_93_1C 3.704 ns
> Tc_Naph_94_7C 3.673 ns
> Tc_Naph_97_7C 3.684 ns
> Tc_Naph_97_9C 3.665 ns
>
>
>
>
> I used an Import command to read in the file, but now I am just sitting and=
> staring, without a clue how to get the 84_2 converted to the number 84.2,e=
> tc. and ending up with two lists: Run 1 and Run 2, consisting of pairs of t=
> emperature and time. The temperature will eventually be converted to 1/abs=
> olute temperature.
>
> I've read lots and lots of help, thumbed through dozens of pages of a Mathe=
> matica 5 manual, and don't know where to start. I'm trying to help a 90-ye=
> ar-old chemistry professor, who is currently using a calculator, but there=
> will be dozens of runs of this experiment.
>
Import is designed to read text or binary data formatted in a standard
form - e.g. CSV. Clearly your files have an ad-hoc format, so you can't
expect Mathematica (or anything else) to read them without some effort!
The job is complicated by the fact that you (or the prof) used "_"
rather than a decimal point, and that the number is joined on to other
textual data. I am going to assume that the units (C and ns) are the
same throughout, and can be discarded, and that all the temperatures
have a decimal part, even if it is 0.
First define a couple of functions:
dataConvert[{{run_}, samples_}] := {run, Map[process, samples]};
process[line_] := Module[{tmp},
tmp = StringReplace[
"\"" <> line, {"_" ~~ a : (DigitCharacter ..) ~~ "_" ~~
b : (DigitCharacter ..) ~~ "C" :>
"\"," <> a <> "." <> b <> ",", " ns" :> ""}];
ToExpression["{" <> tmp <> "}"]
];
It is best to avoid Import if the data is not in a recognised format,
and to read it in as strings, discarding the empty lines, and extracting
the data in the first two lines:
In[4]:= data = ReadList["c:\\maths\\data.dat", String];
In[7]:= data = DeleteCases[data, {}];
In[5]:= fileDate = data[[1]]
Out[5]= "June 7, 2013 "
In[9]:= fileTitle = data[[2]]
Out[9]= "Tc+Naphthalene vs Temperature (oC)"
Break up the rest by detecting the 'Run' lines:
In[29]:= tmp =
Partition[SplitBy[data[[3 ;;]], StringMatchQ[#, "Run" ~~ ___] &], 2]
Out[29]= {{{"Run 1"}, {"Tc_Naph_84_2C 3.740 ns",
"Tc_Naph_87_1C 3.731 ns", "Tc_Naph_89_9C 3.720 ns",
"Tc_Naph_92_9C 3.704 ns", "Tc_Naph_94_7C 3.687 ns",
"Tc_Naph_97_6C 3.694 ns"}}, {{"Run 2"}, {"Tc_Naph_83_2C 3.758 \
ns", "Tc_Naph_83_4C 3.750 ns", "Tc_Naph_86_4C 3.728 ns",
"Tc_Naph_88_1C 3.725 ns", "Tc_Naph_90_2C 3.716 ns",
"Tc_Naph_93_1C 3.704 ns", "Tc_Naph_94_7C 3.673 ns",
"Tc_Naph_97_7C 3.684 ns", "Tc_Naph_97_9C 3.665 ns"}}}
Now apply the previous functions to produce a nested list structure of
strings and real numbers:
In[38]:= Map[dataConvert, tmp]
Out[38]= {{"Run 1", {{"Tc_Naph", 84.2, 3.74}, {"Tc_Naph", 87.1,
3.731}, {"Tc_Naph", 89.9, 3.72}, {"Tc_Naph", 92.9,
3.704}, {"Tc_Naph", 94.7, 3.687}, {"Tc_Naph", 97.6,
3.694}}}, {"Run 2", {{"Tc_Naph", 83.2, 3.758}, {"Tc_Naph", 83.4,
3.75}, {"Tc_Naph", 86.4, 3.728}, {"Tc_Naph", 88.1,
3.725}, {"Tc_Naph", 90.2, 3.716}, {"Tc_Naph", 93.1,
3.704}, {"Tc_Naph", 94.7, 3.673}, {"Tc_Naph", 97.7,
3.684}, {"Tc_Naph", 97.9, 3.665}}}}
Clearly it is best in future to record data in an easier format -
whatever language you use to process it!
You may want to lookup StringReplace and StringExpression to understand
the above, and help you with other problems of this type.
David Bailey
http://www.dbaileyconsultancy.co.uk
[toc] | [prev] | [next] | [standalone]
| From | Helen Read <readhpr@gmail.com> |
|---|---|
| Date | 2013-06-15 08:13 +0000 |
| Message-ID | <kph7ml$2dd$1@smc.vnet.net> |
| In reply to | #14859 |
For starters, make sure you import your file as a Table.
data = Import["filename", "Table"]
Now find where Run 1 and Run 2 begin.
p1 = Position[data, {"Run", 1}][[1, 1]]
p2 = Position[data, {"Run", 2}][[1, 1]]
For Run 1, we want the part of the table from just after p1 to right
before p2. Delete the blank rows {} while you are at it.
run1 = DeleteCases[data[[p1 + 1 ;; p2 - 1]], {}]
For Run 2, we want everything from right after p2 to the end.
run2 = DeleteCases[data[[p2 + 1 ;; Length[data]]], {}]
So now we have two tables, run1 and run2. We need to extract the
temperature and time from each row of the tables. The rows look for
example like the following.
{"Tc_Naph_83_2C", 3.758, "ns"}
The basic idea is to construct a function to operate on one row of the
table, then Map it across the table. You can do this quite compactly
using pure functions, but if you are new to this sort of thing you might
find it easier to follow if you write an explicit function that operates
on one row of the table at a time. I will use x to represent one row.
To begin, we get the first two parts of row x.
x[[1]] contains the temperature information (but needs some processing).
x[[2]] is the time.
In the example row, x[[1]] is "Tc_Naph_83_2C"
Use StringSplit on the "_" to get at the pieces.
StringSplit["Tc_Naph_83_2C", "_"]
results in {"Tc", "Naph", "83", "2C"}
Now, we get the 2nd-to-the-last part of this (the "83" in the example),
and the last part (the "2C"). We drop the last character from the "2C".
Then join the "83" with a "." and the "2". Finally, use ToExpression to
convert "83.2" from a string to a number.
Putting this all together, here is a function to extract the temperature
and time from one row of the table. I've used some local variables
{split,temperature,time} to keep track of things as we go.
f[x_] := Module[{split, temperature, time},
time = x[[2]];
split = StringSplit[x[[1]], "_"];
temperature =
ToExpression[
StringJoin[split[[-2]], ".", StringDrop[split[[-1]], -1]]];
{temperature, time}
]
The input of the function, x, represents one row of the table. The
output is the ordered pair {temperature,time}
Test it out on a single row.
f[{"Tc_Naph_84_2C", 3.74`, "ns"}]
Now simply Map the function over each run.
Map[f, run1]
Map[f,run2]
Helen Read
University of Vermont
On 6/14/2013 4:54 AM, howardfink@gmail.com wrote:
> I have a series of files of this form:
>
> June 7, 2013
> Tc+Naphthalene vs Temperature (oC)
>
> Run 1
> Tc_Naph_84_2C 3.740 ns
> Tc_Naph_87_1C 3.731 ns
> Tc_Naph_89_9C 3.720 ns
> Tc_Naph_92_9C 3.704 ns
> Tc_Naph_94_7C 3.687 ns
> Tc_Naph_97_6C 3.694 ns
>
>
> Run 2
> Tc_Naph_83_2C 3.758 ns
> Tc_Naph_83_4C 3.750 ns
> Tc_Naph_86_4C 3.728 ns
> Tc_Naph_88_1C 3.725 ns
> Tc_Naph_90_2C 3.716 ns
> Tc_Naph_93_1C 3.704 ns
> Tc_Naph_94_7C 3.673 ns
> Tc_Naph_97_7C 3.684 ns
> Tc_Naph_97_9C 3.665 ns
>
>
>
>
> I used an Import command to read in the file, but now I am just sitting and=
> staring, without a clue how to get the 84_2 converted to the number 84.2,e=
> tc. and ending up with two lists: Run 1 and Run 2, consisting of pairs of t=
> emperature and time. The temperature will eventually be converted to 1/abs=
> olute temperature.
>
> I've read lots and lots of help, thumbed through dozens of pages of a Mathe=
> matica 5 manual, and don't know where to start. I'm trying to help a 90-ye=
> ar-old chemistry professor, who is currently using a calculator, but there=
> will be dozens of runs of this experiment.
>
.
[toc] | [prev] | [standalone]
Back to top | Article view | comp.soft-sys.math.mathematica
csiph-web