Groups > comp.lang.java.programmer > #16692

How can you make idle processors pick up java work?

Path	csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.mixmin.net!aioe.org!.POSTED!not-for-mail
From	qwertmonkey@syberiaoutpost.ru
Newsgroups	comp.lang.java.programmer
Subject	How can you make idle processors pick up java work?
Date	Mon, 30 Jul 2012 23:27:39 +0000 (UTC)
Organization	Aioe.org NNTP Server
Lines	37
Message-ID	<jv759b$s7$1@speranza.aioe.org> (permalink)
NNTP-Posting-Host	z9Rq+Ge+SLJPEkk9TGcnaw.user.speranza.aioe.org
X-Complaints-To	abuse@aioe.org
X-Notice	Filtered by postfilter v. 0.8.2
X-Newsreader	NetComponents
Xref	csiph.com comp.lang.java.programmer:16692

Show key headers only | View raw

>> Is there a way to make these processors pick up/share work also, or
>> do you have to use some sort of scheduling framework on top of java?

> Use multiple threads?
~
 a) I need to actually scan large text files (10+ million lines).
 b) On each line there is a NL sentence.
 c) That processing should be run only once, but as fast as possible.
~
 d) If you go:
 d.1) int iPrx = Runtime.getRuntime().availableProcessors();
 d.2) count all lines
 d.3) split the file in (total lines)/iPrx
 d.4) then run iPrx threads (or executable instances using a batch script)
 the time you waste on d.2) and d.3) will make all that strat senseless
~
 I have no way to influence how those large files are generated
~
 e) because of the large sizes of the files you can't even go
~
 FIS = new FileInputStream(IFl);
 FileChannel IFlChnl = FIS.getChannel();
 int iChnlSz = (int)IFlChnl.size();
 MappedByteBuffer MptBytBfr = IFlChnl.map(FileChannel.MapMode.READ_ONLY, 0, iChnlSz);
~
 so, apparently, the only option I have is:
~
     BfR = Files.newBufferedReader(DirPth, ChrStUTF8);
     String aSx = BfR.readLine();
     while(aSx != null){ 

      aSx = BfR.readLine();
     }
~
 do you know of a faster way to go about this?
~
 lbrtchx

Back to comp.lang.java.programmer | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

How can you make idle processors pick up java work? qwertmonkey@syberiaoutpost.ru - 2012-07-30 23:27 +0000
  Re: How can you make idle processors pick up java work? David Lamb <dalamb@cs.queensu.ca> - 2012-07-30 19:34 -0400
  Re: How can you make idle processors pick up java work? Patricia Shanahan <pats@acm.org> - 2012-07-30 16:40 -0700
  Re: How can you make idle processors pick up java work? Joshua Cranmer <Pidgeot18@verizon.invalid> - 2012-07-30 19:51 -0400
  Re: How can you make idle processors pick up java work? "John B. Matthews" <nospam@nospam.invalid> - 2012-07-30 23:45 -0400

csiph-web