Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.ruby > #2913

Re: rubygems-update 1.6.0 Released

Show all headers | View raw

>> 2) combine all the gemspec files into one "cached" gemspec file, per gem
>> directory.
>
> I haven't been willing to sacrifice the simplicity for doing something
> better in this area, there are many other places in RubyGems that need
> to be improved.  Feel free to take a crack at it, though.

Yeah with some benchmarking this doesn't seem to be the general problem, 
either.


>> 3) also cache somehow a list of each "requireable" file within each gem
>> (for instance, within the file for 2).  Then for
>> Gem.searcher.find_in_unresolved you just iterate over that list instead
>> of actually hitting the file system.
>
> I tried this once before and found it to be significantly slower than
> Dir[].  I believe I was working on OS X at the time.
> Benchmarks would be required, of course.


Interesting.  On my box loading RB itself takes in full about 0.5s 
(which is non awful),

Here's a ruby-prof of a script that requires rubygems and a few gems 
(1.9.2 windows 7):

Total: 2.410000

 %self     total     self     wait    child    calls  name
 83.78      2.12     2.02     0.00     1.89      106 
Kernel#gem_original_require
  2.57      0.15     0.06     0.00     0.08      171  Kernel#eval
  2.32      0.06     0.06     0.00     0.00      102  <Class::IO>#read

so the bottleneck currently appears to be in the require call itself 
(though RG adds about 0.3s that might not be a big problem), and not in 
the Gem.searcher.

So the question is how to speed up those calls themselves, I suppose 
(since 2s to load like 3 gems causes us windows users some grief).


I'm not entirely sure if it's file stat's that slow it down or the 
parsing is slow because it has to look for an encoding string at the top 
(I can't imagine that's it, but...)
My guess is it's the file stats.

As a sidenote, if I include the faster_require gem (which caches the 
location of requires to load them directly the second or third time the 
script is run)
those 2.02 seconds go down to about 0.4,
which I assume is "about as fast as it can ever get" (since it's no 
longer doing any extra file stat's, and also skips looking for encoding 
strings).

faster_require by default does
eval(File.read(previously_cached_require_location))

If I have faster_require use
load(previously_cached_require_location)
instead, you'd think lightning fast, since it avoids lots of file stats, 
but it takes 1.7s--almost as long as normal gem_original_require does,
so I'm not sure if I have a handle on where the real bottleneck is.

Thoughts?
-roger

-- 
Posted via http://www.ruby-forum.com/.

Back to comp.lang.ruby | Previous | Next | Find similar | Unroll thread

Thread

Re: rubygems-update 1.6.0 Released Roger Pack <rogerpack2005@gmail.com> - 2011-04-14 23:48 -0500

csiph-web