Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.java.programmer > #13560
| Date | 2012-04-15 09:17 -0700 |
|---|---|
| From | Patricia Shanahan <pats@acm.org> |
| Newsgroups | comp.lang.java.programmer |
| Subject | Re: Pattern suggestion |
| References | <jmel0t$jrh$1@news2.carnet.hr> |
| Message-ID | <WradnUhB_qEbaRfSnZ2dnUVZ_i2dnZ2d@earthlink.com> (permalink) |
On 4/15/2012 7:11 AM, FrenKy wrote: > Hi *, > I have a huge file (~10GB) which I'm reading line by line. Each line has > to be analyzed by many number of different analyzers. The problem I have > is that to make it at least a bit performance optimized due to sometimes > time consuming processing (usually because of delays due to external > interfaces) i would need to make it heavily multithreaded. > File should be read only once to reduce IO on disks. > > So I need "1 driver to many workers" pattern where workers are > multithreaded. > > I have a solution now based on Observable/Observer that I use (and it > works) but I'm not sure if it is the best way. I suggest taking a look at java.util.concurrent.ThreadPoolExecutor and related classes. Try to minimize ordering relationships between processing on the lines, so that you can overlap work on multiple lines as much as possible. Patricia
Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Pattern suggestion FrenKy <frenky__nn@gmail.com> - 2012-04-15 16:11 +0200
Re: Pattern suggestion Rui Maciel <rui.maciel@gmail.com> - 2012-04-15 16:04 +0100
Re: Pattern suggestion Lew <noone@lewscanon.com> - 2012-04-15 08:15 -0700
Re: Pattern suggestion markspace <-@.> - 2012-04-15 08:17 -0700
Re: Pattern suggestion Arne Vajhøj <arne@vajhoej.dk> - 2012-04-15 22:01 -0400
Re: Pattern suggestion Jan Burse <janburse@fastmail.fm> - 2012-04-15 17:41 +0200
Re: Pattern suggestion Jan Burse <janburse@fastmail.fm> - 2012-04-16 00:37 +0200
Re: Pattern suggestion Patricia Shanahan <pats@acm.org> - 2012-04-15 09:17 -0700
Re: Pattern suggestion Arved Sandstrom <asandstrom3minus1@eastlink.ca> - 2012-04-15 13:57 -0300
Re: Pattern suggestion Martin Gregorie <martin@address-in-sig.invalid> - 2012-04-15 19:56 +0000
Re: Pattern suggestion Robert Klemme <shortcutter@googlemail.com> - 2012-04-16 09:55 +0200
Re: Pattern suggestion Arne Vajhøj <arne@vajhoej.dk> - 2012-04-15 21:58 -0400
csiph-web