Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.advocacy > #350560

Re: Fabian: now I'm processing 10,000 addresses in 0.33 seconds

Show all headers | View raw

On Wed, 13 Apr 2016 17:44:36 -0400
DFS <nospam@dfs.com> wrote:

> On 4/13/2016 5:37 PM, Melzzzzz wrote:
> > On Wed, 13 Apr 2016 17:12:58 -0400
> > DFS <nospam@dfs.com> wrote:  
> 
> 
> > Why spill from one to other structure? You can't place data in the
> > addrList in the first place? This looks pretty clumsy.  
> 
> 
> Do it better, or STFU.
> 
> 
> 
> 
> 
> 

Here it goes 0.028 secs to create 10k addresses,parse from that and to
create csv (Rust language):

[bmaxa@maxa-pc dfs]$ ls -l
total 1000
-rw-r--r-- 1 bmaxa users    2488 18.04.2016 03:22 addresses_raw.txt
-rw-r--r-- 1 bmaxa users    2487 18.04.2016 02:43 addresses_raw.txt~
-rwxr-xr-x 1 bmaxa users 1004712 18.04.2016 03:23 dfs*
-rw-r--r-- 1 bmaxa users    2138 18.04.2016 03:23 dfs.rs
-rw-r--r-- 1 bmaxa users    2132 18.04.2016 03:23 dfs.rs~
[bmaxa@maxa-pc dfs]$ time ./dfs

real	0m0.028s
user	0m0.020s
sys	0m0.007s
[bmaxa@maxa-pc dfs]$ ls -l
total 1980
-rw-r--r-- 1 bmaxa users    1364 18.04.2016 03:24 addresses.csv
-rw-r--r-- 1 bmaxa users    2488 18.04.2016 03:22 addresses_raw.txt
-rw-r--r-- 1 bmaxa users    2487 18.04.2016 02:43 addresses_raw.txt~
-rw-r--r-- 1 bmaxa users  997252 18.04.2016 03:24 addresses.txt
-rwxr-xr-x 1 bmaxa users 1004712 18.04.2016 03:23 dfs*
-rw-r--r-- 1 bmaxa users    2138 18.04.2016 03:23 dfs.rs
-rw-r--r-- 1 bmaxa users    2132 18.04.2016 03:23 dfs.rs~
[bmaxa@maxa-pc dfs]$ wc-l addresses.txt
bash: wc-l: command not found
[bmaxa@maxa-pc dfs]$ wc -l addresses.txt
50000 addresses.txt
[bmaxa@maxa-pc dfs]$ cat addresses.csv
After The Game Sports Bar and Grill2445 Moon Rd Ste 1, Grayson, GA 30017 
Burger King1990 Grayson Hwy, Grayson, GA 30017 
Carlitos Mexican Restaurant, Inc2445 Moon Rd, Grayson, GA 30017 
China Cafe1911 Grayson Hwy Ste 4, Grayson, GA 30017 
China Cafe1911 Grayson Hwy, Grayson, GA 30017 
Coopers Corner B&G1111 Athens Hwy, Grayson, GA 30017 
Grand China Chinese Restaurant910 Athens Hwy Ste O, Loganville, GA 30052 
Hail Mary Sports Pub1950 Grayson Hwy Ste 140, Grayson, GA 30017 
Joa Wings and Deli1911 Grayson Hwy Ste 17, Grayson, GA 30017 
Joa Wings and Deli1911 Grayson Hwy, Grayson, GA 30017 
Johnny's Pizza2023 Grayson Hwy Ste 110, Grayson, GA 30017 
KFC1855 Grayson Hwy, Grayson, GA 30017 
Little Caesars Pizza1950 Grayson Hwy, Grayson, GA 30017 
McDonald's1881 Grayson Hwy, Grayson, GA 30017 
My Pizza Buffet2944 Rosebud Rd Ste D, Loganville, GA 30052 
Nagoya Japanese Steak House2944 Rosebud Rd, Loganville, GA 30052 
Pancake House1911 Grayson Hwy Ste 5, Grayson, GA 30017 
Pancake House1911 Grayson Hwy, Grayson, GA 30017 
Papa John's Pizza1009 Athens Hwy Ste 8, Loganville, GA 30052 
Parkside Bakery & Cafe2023 Grayson Hwy, Grayson, GA 30017 
Riverside Pizza1845 Grayson Hwy Ste 1400, Grayson, GA 30017 
Taco Bell1855 Grayson Hwy, Grayson, GA 30017 
Thai Taste Restaurant1009 Athens Hwy, Loganville, GA 30052 
Zaxby's1931 Grayson Hwy, Grayson, GA 30017 
[bmaxa@maxa-pc dfs]$ cat dfs.rs
use std::fs::File;
use std::io::Read;
use std::io::{BufReader,BufWriter,BufRead,Write,LineWriter};
use std::collections::BTreeMap;

fn rand()->u8 {
    let mut f = File::open("/dev/urandom").unwrap();
    let mut buf = [0;1];
    let _ = f.read(&mut buf);
    buf[0]
}

fn main() {

    {
    let f = File::open("addresses_raw.txt").unwrap();
    let mut f = BufReader::new(f);
    let mut arr = Vec::new();
    loop {
        let mut buf = String::new();
        let n = f.read_line(&mut buf).unwrap();
        if n > 0 {
        let _ = f.read_line(&mut buf);
        let _ = f.read_line(&mut buf);
        let _ = f.read_line(&mut buf);
        let _ = f.read_line(&mut buf);
        } else {break;}
        arr.push(buf);
    }
    let f = File::create("addresses.txt").unwrap();
    let mut f = BufWriter::new(f);
    for _ in 0..10000 {
        let _ = f.write(arr[rand() as usize % arr.len()].as_bytes());
    }
    }
    let f = File::open("addresses.txt").unwrap();
    let mut f = BufReader::new(f);
    let mut map = BTreeMap::new();
    loop {
        let mut buf = String::new();
        let n = f.read_line(&mut buf).unwrap();
        if n > 0 {
            buf.clear();
            let _ = f.read_line(&mut buf);
            let name = buf[..buf.len()-1].to_string();
            let _ = f.read_line(&mut buf);
//            println!("{:?}",name);
            buf.clear();
            let _ = f.read_line(&mut buf);
            let address = buf[..buf.len()-1].to_string();
//            println!("{:?}",address);
            let (address,state) = address.split_at(address.find(",").unwrap());
//            println!("{:?} {:?}",address,state);
            let state = state[..state.find("Call Now!").unwrap()].to_string();
            let _ = f.read_line(&mut buf);
            let key = name.clone()+address+&state;
            map.insert(key,vec![name,address.to_string(),state]);
        } else {break;}
    }
    let f = File::create("addresses.csv").unwrap();
    let mut f = LineWriter::new(f);
    for (_,v) in map {
        let out = v[0].clone()+&v[1]+&v[2]+"\n";
        let _ = f.write(out.as_bytes());
    }
}

Back to comp.os.linux.advocacy | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Re: Fabian: now I'm processing 10,000 addresses in 0.33 seconds Melzzzzz <mel@zzzzz.com> - 2016-04-18 03:29 +0200
  Re: Fabian: now I'm processing 10,000 addresses in 0.33 seconds Melzzzzz <mel@zzzzz.com> - 2016-04-18 04:06 +0200
    Re: Fabian: now I'm processing 10,000 addresses in 0.33 seconds DFS <nospam@dfs.com> - 2016-04-17 22:36 -0400
      Re: Fabian: now I'm processing 10,000 addresses in 0.33 seconds Melzzzzz <mel@zzzzz.com> - 2016-04-18 04:45 +0200
  Re: Fabian: now I'm processing 10,000 addresses in 0.33 seconds owl <owl@rooftop.invalid> - 2016-04-18 02:13 +0000
    Re: Fabian: now I'm processing 10,000 addresses in 0.33 seconds Melzzzzz <mel@zzzzz.com> - 2016-04-18 04:15 +0200

csiph-web