Groups > comp.lang.java.programmer > #16876

How can you make idle processors pick up java work?

Path	csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.glorb.com!news-out.readnews.com!transit3.readnews.com!news-out.news.tds.net!newsreading01.news.tds.net!53ab2750!not-for-mail
From	"qwertmonkey" <qwertmonkey@1:261/38.remove-x1c-this>
Subject	How can you make idle processors pick up java work?
Message-ID	<50182C81.55845.calajapr@time.synchro.net> (permalink)
X-Comment-To	All
Newsgroups	comp.lang.java.programmer
X-FTN-AREA	COMP.LANG.JAVA.PROGRAMMER
X-FTN-MSGID	1:261/38 98af8745
Content-Type	text/plain; charset=IBM437
Content-Transfer-Encoding	8bit
X-Gateway	time.synchro.net [Synchronet 3.16a-Win32 NewsLink 1.98]
Lines	52
Date	Tue, 31 Jul 2012 20:07:06 GMT
NNTP-Posting-Host	69.21.70.65
X-Complaints-To	news@tds.net
X-Trace	newsreading01.news.tds.net 1343765226 69.21.70.65 (Tue, 31 Jul 2012 15:07:06 CDT)
NNTP-Posting-Date	Tue, 31 Jul 2012 15:07:06 CDT
Organization	tds.net
Xref	csiph.com comp.lang.java.programmer:16876

Show key headers only | View raw

From: "qwertmonkey" <qwertmonkey@1:261/38.remove-dpk-this>

From: qwertmonkey@syberiaoutpost.ru

>> Is there a way to make these processors pick up/share work also, or
>> do you have to use some sort of scheduling framework on top of java?

> Use multiple threads?
~
 a) I need to actually scan large text files (10+ million lines).
 b) On each line there is a NL sentence.
 c) That processing should be run only once, but as fast as possible.
~
 d) If you go:
 d.1) int iPrx = Runtime.getRuntime().availableProcessors();
 d.2) count all lines
 d.3) split the file in (total lines)/iPrx
 d.4) then run iPrx threads (or executable instances using a batch script)
 the time you waste on d.2) and d.3) will make all that strat senseless
~
 I have no way to influence how those large files are generated
~
 e) because of the large sizes of the files you can't even go
~
 FIS = new FileInputStream(IFl);
 FileChannel IFlChnl = FIS.getChannel();
 int iChnlSz = (int)IFlChnl.size();
 MappedByteBuffer MptBytBfr = IFlChnl.map(FileChannel.MapMode.READ_ONLY, 0,
iChnlSz);
~
 so, apparently, the only option I have is:
~
     BfR = Files.newBufferedReader(DirPth, ChrStUTF8);
     String aSx = BfR.readLine();
     while(aSx != null){

      aSx = BfR.readLine();
     }
~
 do you know of a faster way to go about this?
~
 lbrtchx

-+- BBBS/Li6 v4.10 Dada-1
 + Origin: Prism bbs (1:261/38)
-+- Synchronet 3.16a-Win32 NewsLink 1.98
Time Warp of the Future BBS - telnet://time.synchro.net:24

--- BBBS/Li6 v4.10 Dada-1
 * Origin: Prism bbs (1:261/38)
--- Synchronet 3.16a-Win32 NewsLink 1.98
Time Warp of the Future BBS - telnet://time.synchro.net:24

Back to comp.lang.java.programmer | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

How can you make idle processors pick up java work? "qwertmonkey" <qwertmonkey@1:261/38.remove-x1c-this> - 2012-07-31 20:07 +0000
  Re: How can you make idle processors pick up java work? "David Lamb" <david.lamb@1:261/38.remove-x1c-this> - 2012-07-31 20:07 +0000
  Re: How can you make idle processors pick up java work? "Patricia Shanahan" <patricia.shanahan@1:261/38.remove-x1c-this> - 2012-07-31 20:07 +0000
  Re: How can you make idle processors pick up java work? "Joshua Cranmer" <joshua.cranmer@1:261/38.remove-x1c-this> - 2012-07-31 20:07 +0000
  Re: How can you make idle processors pick up java work? "John B. Matthews" <john.b..matthews@1:261/38.remove-x1c-this> - 2012-07-31 20:07 +0000

csiph-web