Path: csiph.com!news.swapon.de!eternal-september.org!feeder.eternal-september.org!mx02.eternal-september.org!.POSTED!not-for-mail From: Paul Rubin Newsgroups: comp.lang.forth Subject: Re: Looking for some help with this gforth program. Thanks! Date: Tue, 17 May 2016 21:16:33 -0700 Organization: A noiseless patient Spider Lines: 34 Message-ID: <87d1okf1b2.fsf@jester.gateway.pace.com> References: <6548d185-2c31-40a4-bfa9-a3c353e9fa5d@googlegroups.com> <2016May15.123405@mips.complang.tuwien.ac.at> <5fc58bbc-d742-4b25-aa89-6bfdcfc3d619@googlegroups.com> <87shxioj4i.fsf@jester.gateway.pace.com> <73f98bf5-0ee2-4155-8393-58d36ed68ca6@googlegroups.com> <2016May16.154124@mips.complang.tuwien.ac.at> <871t51o9ji.fsf@jester.gateway.pace.com> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: mx02.eternal-september.org; posting-host="589dd00be5ee573d7046ed589be3c50c"; logging-data="5797"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX197nFOdStxqZ5PiPndXbK9Q" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) Cancel-Lock: sha1:bSP38o4otVHLpdnx8OjFqspcSYQ= sha1:bg4ZXFyO1sGWlrOO9Ti5DMnVSns= Xref: csiph.com comp.lang.forth:47270 Bernd Paysan writes: > I think this 1970 style development came from the enormous turnaround > times of coding.... Maybe, though the waterfall development model didn't help much either. > actually, Google just slurps in a Petabyte into RAM. It's using a big > farm with around 100k severs and each of them with enough gigabytes to > slurp in a Petabyte or two. They are very very careful about data layouts though, so I wouldn't consider that to be "slurping" in the sense of just gobbling all the data in the simplest and most convenient way. In particular, they use quite intricate bit-packing encodings because while they have gigabytes/petabytes of main ram, they want to get as much of the most frequently accessed index data as possible into the very scarce cpu cache space. Aside: do you know if that 100k servers in a map/reduce cluster is an accurate figure? I'd been wondering about that. I thought the individual Google clusters were much smaller, like 10k servers, because of their modular data center design (at least as of 2011 or so). Of course they have lots of such clusters. > If I would develop a distributed search engine, I would probably go that > way: make different computers responsible for different key terms. That's called horizontal sharding and it's a standard technique. > I probably would add a "good/bad" feedback button so that people can > add further hints for the ranking algorithms. I'm sure google is way beyond that by now. As the saying goes, they know more about you and your preferences than you yourself know ;-).