Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.glorb.com!news-out.readnews.com!transit4.readnews.com!news-out.news.tds.net!newsreading01.news.tds.net!53ab2750!not-for-mail
From: "David Lamb" <david.lamb@1:261/38.remove-x1c-this>
Subject: Re: How can you make idle processors pick up java work?
Message-ID: <50182C81.55846.calajapr@time.synchro.net>
X-Comment-To: qwertmonkey
Newsgroups: comp.lang.java.programmer
In-Reply-To: <50182C81.55845.calajapr@time.synchro.net>
References: <50182C81.55845.calajapr@time.synchro.net>
X-FTN-AREA: COMP.LANG.JAVA.PROGRAMMER
X-FTN-MSGID: 1:261/38 c27a2eca
X-FTN-REPLY: 1:261/38 98af8745
Content-Type: text/plain; charset=IBM437
Content-Transfer-Encoding: 8bit
X-Gateway: time.synchro.net [Synchronet 3.16a-Win32 NewsLink 1.98]
Lines: 36
Date: Tue, 31 Jul 2012 20:07:06 GMT
NNTP-Posting-Host: 69.21.70.65
X-Complaints-To: news@tds.net
X-Trace: newsreading01.news.tds.net 1343765226 69.21.70.65 (Tue, 31 Jul 2012 15:07:06 CDT)
NNTP-Posting-Date: Tue, 31 Jul 2012 15:07:06 CDT
Organization: tds.net
Xref: csiph.com comp.lang.java.programmer:16878

  To: qwertmonkey
From: "David Lamb" <david.lamb@1:261/38.remove-dpk-this>

  To: qwertmonkey
From: David Lamb <dalamb@cs.queensu.ca>

On 30/07/2012 7:27 PM, qwertmonkey@syberiaoutpost.ru wrote:
>>> Is there a way to make these processors pick up/share work also, or
>>> do you have to use some sort of scheduling framework on top of java?
>
>> Use multiple threads?
> ~
>   a) I need to actually scan large text files (10+ million lines).
>   b) On each line there is a NL sentence.
>   c) That processing should be run only once, but as fast as possible.
> ~
>   d) If you go:
>   d.1) int iPrx = Runtime.getRuntime().availableProcessors();
>   d.2) count all lines
>   d.3) split the file in (total lines)/iPrx
>   d.4) then run iPrx threads (or executable instances using a batch script)
>   the time you waste on d.2) and d.3) will make all that strat senseless

How slow is the NL processing? Does it make any sense to read lines in one 
thread and pass each off to one of the iPrx-1 other threads that might run on 
separate processors?

-+- BBBS/Li6 v4.10 Dada-1
 + Origin: Prism bbs (1:261/38)
-+- Synchronet 3.16a-Win32 NewsLink 1.98
Time Warp of the Future BBS - telnet://time.synchro.net:24

--- BBBS/Li6 v4.10 Dada-1
 * Origin: Prism bbs (1:261/38)
--- Synchronet 3.16a-Win32 NewsLink 1.98
Time Warp of the Future BBS - telnet://time.synchro.net:24