Groups | Search | Server Info | Login | Register

Groups > comp.soft-sys.sas > #874

Re: reading big csv files

Show all headers | View raw

Patrick wrote:
> Hi Dinesh
>
> How do you try to import this csv? And what's the SAS version and OS?
>
> Below code creates a csv file with 50000 columns and 1000 rows.
>
> I've imported this file using the import wizard in EG4.3. On a rather
> new 64bit, 4 core, 8GB RAM, Win7 Laptop some of the steps took 1 to 2
> minutes - but it worked just beautifully.
>
> If you're using code with a filename statement then you would have to
> read the data as stream (recfm=n or recfm=s) and unbuffered.
>
>
> Below code creates the example data I was able to import:
>
> filename ManyCols 'C:\temp\test\manycols.csv' lrecl=1000000;
> data _null_;
>    file ManyCols;
>    /* header */
>    do i=1 to 50000;
>      put 'Varname_' i z5. ';' @;
>    end;
>    put;
>
>    /* values */
>    do j= 1 to 1000;
>      do i=1 to 50000;
>        x=j*i;
>        put x z8. ';' @;
>      end;
>      put;
>    end;
>
>    stop;
> run;
>
>
> HTH
> Patrick


Patrick,
Could you post the code you used to read this file?

Thanks,
Carl

Back to comp.soft-sys.sas | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

reading big csv files Dinesh <mtdinesh@gmail.com> - 2011-06-03 14:07 -0700
  Re: reading big csv files Carl Kaufmann <cwkaufmann@cox.net> - 2011-06-03 18:27 -0400
  Re: reading big csv files Patrick <patrick.matter@gmx.ch> - 2011-06-05 03:16 -0700
    Re: reading big csv files Carl Kaufmann <cwkaufmann@cox.net> - 2011-06-06 10:45 -0400
      Re: reading big csv files Patrick <patrick.matter@gmx.ch> - 2011-06-07 06:13 -0700
  Re: reading big csv files Bruno Müller <bruno.mueller@gmail.com> - 2011-06-09 05:29 -0700
    Re: reading big csv files Patrick <patrick.matter@gmx.ch> - 2011-06-11 08:43 -0700

csiph-web