Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.soft-sys.math.mathematica > #9172 > unrolled thread
| Started by | Mark Coleman <markspcoleman@gmail.com> |
|---|---|
| First post | 2012-03-20 07:26 +0000 |
| Last post | 2012-03-22 10:48 +0000 |
| Articles | 3 — 3 participants |
Back to article view | Back to comp.soft-sys.math.mathematica
Problems reading text files with embedded delimiters Mark Coleman <markspcoleman@gmail.com> - 2012-03-20 07:26 +0000
Re: Problems reading text files with embedded delimiters Rodrigo Murta <murta@rodrigomurta.com> - 2012-03-21 10:50 +0000
Re: Problems reading text files with embedded delimiters David Reiss <dbreiss@gmail.com> - 2012-03-22 10:48 +0000
| From | Mark Coleman <markspcoleman@gmail.com> |
|---|---|
| Date | 2012-03-20 07:26 +0000 |
| Subject | Problems reading text files with embedded delimiters |
| Message-ID | <jk9bfb$bo1$1@smc.vnet.net> |
Hi, I'm struggling with importing a large flat file into Mathematica. The file, which was pulled off one of my company's legacy mainframe systems, contains about 100 fields and several million rows, including a mix of dates, reals, and text. The fields are delimited by commas. One of the text fields, however, is quite large and contains commas, line feeds, carriage returns, etc -- lots of characters commonly used as delimiters. It is also a variable length field. There does not seem to be anything characters within the field that would allow the specification of a simple delimiter. I've tried the default Import[] command as well as various permutations of ReadList[] using Record, Word, and String specifications. I'm wondering if anyone could suggest an approach that would allow me to read this file. More precisely, is there a way to avoid reading characters embedded in the large text field that Mathematica thinks are field delimiters? Thanks, Mark
[toc] | [next] | [standalone]
| From | Rodrigo Murta <murta@rodrigomurta.com> |
|---|---|
| Date | 2012-03-21 10:50 +0000 |
| Message-ID | <jkcbpa$qht$1@smc.vnet.net> |
| In reply to | #9172 |
On Mar 20, 4:26 am, Mark Coleman <markspcole...@gmail.com> wrote: > Hi, > > I'm struggling with importing a large flat file into Mathematica. The file, > which was pulled off one of my company's legacy mainframe systems, > contains about 100 fields and several million rows, including a mix of > dates, reals, and text. The fields are delimited by commas. > > One of the text fields, however, is quite large and contains commas, > line feeds, carriage returns, etc -- lots of characters commonly used > as delimiters. It is also a variable length field. There does not seem > to be anything characters within the field that would allow the > specification of a simple delimiter. I've tried the default Import[] > command as well as various permutations of ReadList[] using Record, > Word, and String specifications. > > I'm wondering if anyone could suggest an approach that would allow me > to read this file. More precisely, is there a way to avoid reading > characters embedded in the large text field that Mathematica thinks are field > delimiters? > > Thanks, > > Mark Hi Mark I didn't completely get your point. What is your delimiter? Your large text is quoted? Can you sendo one example line? Or a toy code? For large text string you could use some low level programming. Maybe this tutorial could help you: http://reference.wolfram.com/mathematica/guide/LowLevelFileOperations.html Go to the stream topic. best regards Murta
[toc] | [prev] | [next] | [standalone]
| From | David Reiss <dbreiss@gmail.com> |
|---|---|
| Date | 2012-03-22 10:48 +0000 |
| Message-ID | <jkf02m$8hi$1@smc.vnet.net> |
| In reply to | #9172 |
One possibility for this is to read the file as Lines and, given that you know that there is only one column with the unstructured text field, to parse the other columns from the left and the right up to that column's data since each of those columns are delimited uniquely by commas. Best, David On Mar 20, 3:26 am, Mark Coleman <markspcole...@gmail.com> wrote: > Hi, > > I'm struggling with importing a large flat file into Mathematica. The file, > which was pulled off one of my company's legacy mainframe systems, > contains about 100 fields and several million rows, including a mix of > dates, reals, and text. The fields are delimited by commas. > > One of the text fields, however, is quite large and contains commas, > line feeds, carriage returns, etc -- lots of characters commonly used > as delimiters. It is also a variable length field. There does not seem > to be anything characters within the field that would allow the > specification of a simple delimiter. I've tried the default Import[] > command as well as various permutations of ReadList[] using Record, > Word, and String specifications. > > I'm wondering if anyone could suggest an approach that would allow me > to read this file. More precisely, is there a way to avoid reading > characters embedded in the large text field that Mathematica thinks are field > delimiters? > > Thanks, > > Mark
[toc] | [prev] | [standalone]
Back to top | Article view | comp.soft-sys.math.mathematica
csiph-web