Re: An extremely large hash lookup mechanism

X-FeedAbuse	http://nntpfeed.proxad.net/abuse.pl feeded by 88.191.16.109
Path	csiph.com!eeepc.pasdenom.info!news.pasdenom.info!news.dougwise.org!nntpfeed.proxad.net!nospam.fr.eu.org!usenet-fr.net!de-l.enfer-du-nord.net!feeder1.enfer-du-nord.net!eternal-september.org!feeder.eternal-september.org!.POSTED!not-for-mail
From	"Peter H. Coffin" <hellsop@ninehells.com>
Newsgroups	comp.lang.php
Subject	Re: An extremely large hash lookup mechanism
Date	Mon, 7 Feb 2011 07:15:03 -0600
Organization	A noiseless patient Spider
Lines	35
Message-ID	<slrnikvs2n.2mh.hellsop@nibelheim.ninehells.com> (permalink)
References	<cb5d1ced-56a3-4936-bece-542b676a1528@a8g2000pri.googlegroups.com>
Mime-Version	1.0
Content-Type	text/plain; charset=us-ascii
Content-Transfer-Encoding	7bit
Injection-Info	mx02.eternal-september.org; posting-host="le/2G4+BNpieVhQI6uJ2Lw"; logging-data="9830"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+/N1AYe3MAxq2RGdPYQoLR"
User-Agent	slrn/0.9.9p1 (OpenBSD)
Cancel-Lock	sha1:g7lFFmFTQdrSuo2uFvYlwLz+kA8=
Xref	csiph.com comp.lang.php:11237

Show key headers only | View raw

On Mon, 7 Feb 2011 04:06:30 -0800 (PST), ram wrote:
> I have Mysql table with customerid (big int)   & customer_unique_key
> (varchar )
>
> I have a php script that needs to upload customers into groups.
> The  customer_unique_key  will be uploaded and all the customerids
> should be entered to a new table.
>
> Initially My script was doing a per-record query to extract
> customerid and print.
> But this turned out to be toooo slow , since the upload file may have
> upto a million records.
>
> Now I modified the script to read all customer_unique_key ->
> customerid as key value pairs into an array
> This works fine and fast , but hogs the memory and crashes whenever
> the number of records crosses around 3-4 million.
>
>
> What is the best way I can implement a hash lookup ? Should I use a
> CDB library ?

No, you should use a real database. Unless you plan on essentially never
adding a customer. But the way you've framed the question (including
babbling about hashes) makes it sound like you've already "solved" your
problem, and have decided you want to use cdb. So, you might as well
have at it, find out that it's not fixing your problem, then go back to
comp.databases.mysql again, and this time start with talking about your
problem instead of asking how to implement your solution.

-- 
5. The artifact which is the source of my power will not be kept on the 
   Mountain of Despair beyond the River of Fire guarded by the Dragons
   of Eternity. It will be in my safe-deposit box. The same applies to 
   the object which is my one weakness.  --Peter Anspach "Evil Overlord"

Back to comp.lang.php | Previous | Next | Find similar | Unroll thread

Thread

Re: An extremely large hash lookup mechanism "Peter H. Coffin" <hellsop@ninehells.com> - 2011-02-07 07:15 -0600

csiph-web