Groups > comp.lang.java.programmer > #16694

Re: How can you make idle processors pick up java work?

Path	csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.glorb.com!border3.nntp.dca.giganews.com!Xl.tags.giganews.com!border1.nntp.dca.giganews.com!nntp.giganews.com!local2.nntp.dca.giganews.com!nntp.earthlink.com!news.earthlink.com.POSTED!not-for-mail
NNTP-Posting-Date	Mon, 30 Jul 2012 18:40:19 -0500
Date	Mon, 30 Jul 2012 16:40:13 -0700
From	Patricia Shanahan <pats@acm.org>
User-Agent	Mozilla/5.0 (Windows NT 5.2; WOW64; rv:13.0) Gecko/20120614 Thunderbird/13.0.1
MIME-Version	1.0
Newsgroups	comp.lang.java.programmer
Subject	Re: How can you make idle processors pick up java work?
References	<jv759b$s7$1@speranza.aioe.org>
In-Reply-To	<jv759b$s7$1@speranza.aioe.org>
Content-Type	text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding	7bit
Message-ID	<IdydnUpkR4P-horNnZ2dnUVZ_jednZ2d@earthlink.com> (permalink)
Lines	21
X-Usenet-Provider	http://www.giganews.com
NNTP-Posting-Host	70.230.192.108
X-Trace	sv3-ZSSBa/iioGuFgUar+W8pqaf1BpbM7yVmPUIzAS/wONkxhPxxFmf4tjwO0djZPxz1PRMEjC98U3AaxWf!Bh0CnUp7fZXZHrT9ms157f8cwF5G1j4RHPKQjFZ0VbSQeRKALtw2BBSezF/A/VvG5AXM1rrgi8Ky!2lX4Z++dfCX1LbVUuQO+mhOOk2tVXaUKkAua+9bN1FVAC1Q=
X-Abuse-and-DMCA-Info	Please be sure to forward a copy of ALL headers
X-Abuse-and-DMCA-Info	Otherwise we will be unable to process your complaint properly
X-Postfilter	1.3.40
X-Original-Bytes	2028
Xref	csiph.com comp.lang.java.programmer:16694

Show key headers only | View raw

On 7/30/2012 4:27 PM, qwertmonkey@syberiaoutpost.ru wrote:
>>> Is there a way to make these processors pick up/share work also, or
>>> do you have to use some sort of scheduling framework on top of java?
>
>> Use multiple threads?
> ~
>   a) I need to actually scan large text files (10+ million lines).
>   b) On each line there is a NL sentence.
>   c) That processing should be run only once, but as fast as possible.
> ~
>   d) If you go:
>   d.1) int iPrx = Runtime.getRuntime().availableProcessors();
>   d.2) count all lines
>   d.3) split the file in (total lines)/iPrx
>   d.4) then run iPrx threads (or executable instances using a batch script)
>   the time you waste on d.2) and d.3) will make all that strat senseless

Why worry about splitting by actual line count, rather than by byte
position in file?

Patricia

Thread

How can you make idle processors pick up java work? qwertmonkey@syberiaoutpost.ru - 2012-07-30 23:27 +0000
  Re: How can you make idle processors pick up java work? David Lamb <dalamb@cs.queensu.ca> - 2012-07-30 19:34 -0400
  Re: How can you make idle processors pick up java work? Patricia Shanahan <pats@acm.org> - 2012-07-30 16:40 -0700
  Re: How can you make idle processors pick up java work? Joshua Cranmer <Pidgeot18@verizon.invalid> - 2012-07-30 19:51 -0400
  Re: How can you make idle processors pick up java work? "John B. Matthews" <nospam@nospam.invalid> - 2012-07-30 23:45 -0400

csiph-web