Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.java.programmer > #16693
| Path | csiph.com!newsfeed.hal-mli.net!feeder3.hal-mli.net!newsfeed.hal-mli.net!feeder1.hal-mli.net!eternal-september.org!feeder.eternal-september.org!mx04.eternal-september.org!.POSTED!not-for-mail |
|---|---|
| From | David Lamb <dalamb@cs.queensu.ca> |
| Newsgroups | comp.lang.java.programmer |
| Subject | Re: How can you make idle processors pick up java work? |
| Date | Mon, 30 Jul 2012 19:34:49 -0400 |
| Organization | A noiseless patient Spider |
| Lines | 20 |
| Message-ID | <jv75mu$a21$1@dont-email.me> (permalink) |
| References | <jv759b$s7$1@speranza.aioe.org> |
| Mime-Version | 1.0 |
| Content-Type | text/plain; charset=ISO-8859-1; format=flowed |
| Content-Transfer-Encoding | 7bit |
| Injection-Date | Mon, 30 Jul 2012 23:34:54 +0000 (UTC) |
| Injection-Info | mx04.eternal-september.org; posting-host="c397b0271b118225329a50c7668d0388"; logging-data="10305"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX18erbH4DPgfZfCeToRbojxm" |
| User-Agent | Mozilla/5.0 (Windows NT 5.1; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 |
| In-Reply-To | <jv759b$s7$1@speranza.aioe.org> |
| Cancel-Lock | sha1:9Fdyr+5WoYTTu98R4ZGBfXkTfG0= |
| Xref | csiph.com comp.lang.java.programmer:16693 |
Show key headers only | View raw
On 30/07/2012 7:27 PM, qwertmonkey@syberiaoutpost.ru wrote: >>> Is there a way to make these processors pick up/share work also, or >>> do you have to use some sort of scheduling framework on top of java? > >> Use multiple threads? > ~ > a) I need to actually scan large text files (10+ million lines). > b) On each line there is a NL sentence. > c) That processing should be run only once, but as fast as possible. > ~ > d) If you go: > d.1) int iPrx = Runtime.getRuntime().availableProcessors(); > d.2) count all lines > d.3) split the file in (total lines)/iPrx > d.4) then run iPrx threads (or executable instances using a batch script) > the time you waste on d.2) and d.3) will make all that strat senseless How slow is the NL processing? Does it make any sense to read lines in one thread and pass each off to one of the iPrx-1 other threads that might run on separate processors?
Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
How can you make idle processors pick up java work? qwertmonkey@syberiaoutpost.ru - 2012-07-30 23:27 +0000 Re: How can you make idle processors pick up java work? David Lamb <dalamb@cs.queensu.ca> - 2012-07-30 19:34 -0400 Re: How can you make idle processors pick up java work? Patricia Shanahan <pats@acm.org> - 2012-07-30 16:40 -0700 Re: How can you make idle processors pick up java work? Joshua Cranmer <Pidgeot18@verizon.invalid> - 2012-07-30 19:51 -0400 Re: How can you make idle processors pick up java work? "John B. Matthews" <nospam@nospam.invalid> - 2012-07-30 23:45 -0400
csiph-web