Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.php > #15910 > unrolled thread

Fast PHP way to find a file given the leftmost characters of the file name

Back to article view | Back to comp.lang.php

Contents

  Fast PHP way to find a file given the leftmost characters of the file name James Harris <james.harris.1@gmail.com> - 2015-12-05 17:39 +0000
    Re: Fast PHP way to find a file given the leftmost characters of the file name Arno Welzel <usenet@arnowelzel.de> - 2015-12-05 19:37 +0100
    Re: Fast PHP way to find a file given the leftmost characters of the file name Markus Heinz <markus.heinz@uni-dortmund.de> - 2015-12-05 19:45 +0100
      Re: Fast PHP way to find a file given the leftmost characters of the file name Matthew Carter <m@ahungry.com> - 2015-12-05 14:45 -0500
        Re: Fast PHP way to find a file given the leftmost characters of the file name Jerry Stuckle <jstucklex@attglobal.net> - 2015-12-05 15:53 -0500
        Re: Fast PHP way to find a file given the leftmost characters of the file name James Harris <james.harris.1@gmail.com> - 2015-12-06 00:50 +0000
          Re: Fast PHP way to find a file given the leftmost characters of the file name Matthew Carter <m@ahungry.com> - 2015-12-07 00:17 -0500
            Re: Fast PHP way to find a file given the leftmost characters of the file name James Harris <james.harris.1@gmail.com> - 2015-12-08 11:45 +0000

#15910 — Fast PHP way to find a file given the leftmost characters of the file name

Basic query: What is the best way in PHP to find a file given just the 
leftmost characters of its name? I am looking for something that will 
scale so that it can be expected to execute quickly even if there are 
thousands of files in a given directory.

In more detail: For a new site I am thinking to have long names on files 
and for a PHP script to find a specific file given just the leftmost 
characters of the file's name which it has got from the URL. This is for 
SEO and convenience reasons. The file name would be long so that it 
could reflect the title/subject/topic of the page.

The URL could include the full name (without an extension) or just a 
short prefix. For example, if the file names were

   1001-this-is-a-file.ext1
   1002-this-is-too.ext2
   1003-so-is-this.ext1

the first of those files could be accessed by either of these URLs:

   http://site.com/dir/1001-this-is-a-file
   http://site.com/dir/1001

As you can see, the second name is the short form of the first.

The requests would be directed to a PHP script by URL rewriting. Either 
of the URLs could be used to find the same file.

Note that even the first URL needs to match on the leftmost characters 
as it omits the extension.

In each case I would need to find a file which begins with the string 
specified in the URL. Hence this query.

I guess something like this approach to URL shortening is fairly common 
for SEO reasons. Maybe there is a standard solution...?

Any comments? Even if you think the idea is a bad one I would appreciate 
the feedback.

James

[toc] | [next] | [standalone]

#15911

James Harris schrieb am 2015-12-05 um 18:39:

[...]
> The URL could include the full name (without an extension) or just a 
> short prefix. For example, if the file names were
> 
>    1001-this-is-a-file.ext1
>    1002-this-is-too.ext2
>    1003-so-is-this.ext1
> 
> the first of those files could be accessed by either of these URLs:
> 
>    http://site.com/dir/1001-this-is-a-file
>    http://site.com/dir/1001
> 
> As you can see, the second name is the short form of the first.
> 
> The requests would be directed to a PHP script by URL rewriting. Either 
> of the URLs could be used to find the same file.

Use a database where you store the file names (for example using a kind
of maintance script which regularly scans the directory and updates a
database table with the current list of files). Otherwise you would have
to scan the directory with every request to get the existing file names
as a list and find out, which one matches - and this would not be very
efficient.

[...]
> I guess something like this approach to URL shortening is fairly common 
> for SEO reasons. Maybe there is a standard solution...?

Nope.

WordPress for example does this "auto completion" as well - but the URLs
in WordPress are already only in the database and don't even exist as
files. Many other CMSes handle this in a similar way.

So for example:

When you access

<http://arnowelzel.de/wp/en/tools/spoke>

you will get redirected to the real URL

<http://arnowelzel.de/wp/en/tools/spoke-length-calculator>

But this is not done to provide and kind of SEO - it's just for
convenience. The opposite is true: Using descriptive URLs is better
since the URLs are then also part of the search.

Also see:

<https://moz.com/blog/15-seo-best-practices-for-structuring-urls>

What really hits search engine results is changing old URLs without
forwarding to the new ones.

<http://www.w3.org/Provider/Style/URI.html> is still true.


-- 
Arno Welzel
http://arnowelzel.de
http://de-rec-fahrrad.de
http://fahrradzukunft.de

[toc] | [prev] | [next] | [standalone]

#15912

Hello.

On 2015-12-05 at 18:39 James Harris wrote:
> Basic query: What is the best way in PHP to find a file given just the
> leftmost characters of its name? I am looking for something that will
> scale so that it can be expected to execute quickly even if there are
> thousands of files in a given directory.
[...]
> Any comments? Even if you think the idea is a bad one I would appreciate
> the feedback.

The glob function might be helpful to accomplish your goal:
<http://de2.php.net/manual/en/function.glob.php>

Another alternative might be to store the full filenames in a database
table and then do a SQL query like the following:

SELECT filename FROM files WHERE filename LIKE 'prefix%'

In this query "prefix" is the prefix which is being searched for and the 
query will return all complete filenames matching this prefix.

Which solution scales better should be examined in a setup like the
target environment and is influenced by parameters such as number of
files, filesystem type, available RAM, CPU speed etc.

> James

Regards

Markus

[toc] | [prev] | [next] | [standalone]

#15913

Markus Heinz <markus.heinz@uni-dortmund.de> writes:

> Hello.
>
> On 2015-12-05 at 18:39 James Harris wrote:
>> Basic query: What is the best way in PHP to find a file given just the
>> leftmost characters of its name? I am looking for something that will
>> scale so that it can be expected to execute quickly even if there are
>> thousands of files in a given directory.
> [...]
>> Any comments? Even if you think the idea is a bad one I would appreciate
>> the feedback.
>
> The glob function might be helpful to accomplish your goal:
> <http://de2.php.net/manual/en/function.glob.php>
>
> Another alternative might be to store the full filenames in a database
> table and then do a SQL query like the following:
>
> SELECT filename FROM files WHERE filename LIKE 'prefix%'
>
> In this query "prefix" is the prefix which is being searched for and
> the query will return all complete filenames matching this prefix.
>
> Which solution scales better should be examined in a setup like the
> target environment and is influenced by parameters such as number of
> files, filesystem type, available RAM, CPU speed etc.
>
>> James
>
> Regards
>
> Markus
>

FWIW, I just tested in a directory with 50,000 files on a system with
1.5G RAM (and non-SD disk) and was able to get 190 matches when
specifying the first 2 letters in 0.103 seconds using
glob($letters.'*'), as well as similar results when specifying all the
way to a single unique name.

So, I would stick with glob vs attempting to over-engineer it (if you
are working with real files and not just database content), as the only
reason to micro-optimize would be if you had extremely high traffic (in
which case I think you could afford the $20 or less a month to just get
an SD Linode, where the cost of disk I/O is almost non-existent).

-- 
Matthew Carter (m@ahungry.com)
http://ahungry.com

[toc] | [prev] | [next] | [standalone]

#15914

On 12/5/2015 2:45 PM, Matthew Carter wrote:
> Markus Heinz <markus.heinz@uni-dortmund.de> writes:
> 
>> Hello.
>>
>> On 2015-12-05 at 18:39 James Harris wrote:
>>> Basic query: What is the best way in PHP to find a file given just the
>>> leftmost characters of its name? I am looking for something that will
>>> scale so that it can be expected to execute quickly even if there are
>>> thousands of files in a given directory.
>> [...]
>>> Any comments? Even if you think the idea is a bad one I would appreciate
>>> the feedback.
>>
>> The glob function might be helpful to accomplish your goal:
>> <http://de2.php.net/manual/en/function.glob.php>
>>
>> Another alternative might be to store the full filenames in a database
>> table and then do a SQL query like the following:
>>
>> SELECT filename FROM files WHERE filename LIKE 'prefix%'
>>
>> In this query "prefix" is the prefix which is being searched for and
>> the query will return all complete filenames matching this prefix.
>>
>> Which solution scales better should be examined in a setup like the
>> target environment and is influenced by parameters such as number of
>> files, filesystem type, available RAM, CPU speed etc.
>>
>>> James
>>
>> Regards
>>
>> Markus
>>
> 
> FWIW, I just tested in a directory with 50,000 files on a system with
> 1.5G RAM (and non-SD disk) and was able to get 190 matches when
> specifying the first 2 letters in 0.103 seconds using
> glob($letters.'*'), as well as similar results when specifying all the
> way to a single unique name.
> 
> So, I would stick with glob vs attempting to over-engineer it (if you
> are working with real files and not just database content), as the only
> reason to micro-optimize would be if you had extremely high traffic (in
> which case I think you could afford the $20 or less a month to just get
> an SD Linode, where the cost of disk I/O is almost non-existent).
> 

Matthew, I tend to agree with you.  Even if there are multiple levels of
directories, it should still be pretty fast if done right.  I think the
OP is falling into the premature optimization trap.

Do it with clear and easy to understand code and see if there is a
problem.  Chances are there will not be.  IFF there, solve the problem.

Using a database sounds fine at first.  But it can be difficult to
ensure the file system and the database are always in sync.  This is
especially true if the OP allows (or uses) ftp.

-- 
==================
Remove the "x" from my email address
Jerry Stuckle
jstucklex@attglobal.net
==================

[toc] | [prev] | [next] | [standalone]

#15915

On 05/12/2015 19:45, Matthew Carter wrote:
> Markus Heinz <markus.heinz@uni-dortmund.de> writes:
>
>> Hello.
>>
>> On 2015-12-05 at 18:39 James Harris wrote:
>>> Basic query: What is the best way in PHP to find a file given just the
>>> leftmost characters of its name? I am looking for something that will
>>> scale so that it can be expected to execute quickly even if there are
>>> thousands of files in a given directory.
>> [...]
>>> Any comments? Even if you think the idea is a bad one I would appreciate
>>> the feedback.
>>
>> The glob function might be helpful to accomplish your goal:
>> <http://de2.php.net/manual/en/function.glob.php>

Ideal, thanks Markus.

> FWIW, I just tested in a directory with 50,000 files on a system with
> 1.5G RAM (and non-SD disk) and was able to get 190 matches when
> specifying the first 2 letters in 0.103 seconds using
> glob($letters.'*'), as well as similar results when specifying all the
> way to a single unique name.

Thanks. Was that with GLOB_NOSORT included?

James

[toc] | [prev] | [next] | [standalone]

#15916

James Harris <james.harris.1@gmail.com> writes:

> On 05/12/2015 19:45, Matthew Carter wrote:
>> Markus Heinz <markus.heinz@uni-dortmund.de> writes:
>>
>>> Hello.
>>>
>>> On 2015-12-05 at 18:39 James Harris wrote:
>>>> Basic query: What is the best way in PHP to find a file given just the
>>>> leftmost characters of its name? I am looking for something that will
>>>> scale so that it can be expected to execute quickly even if there are
>>>> thousands of files in a given directory.
>>> [...]
>>>> Any comments? Even if you think the idea is a bad one I would appreciate
>>>> the feedback.
>>>
>>> The glob function might be helpful to accomplish your goal:
>>> <http://de2.php.net/manual/en/function.glob.php>
>
> Ideal, thanks Markus.
>
>> FWIW, I just tested in a directory with 50,000 files on a system with
>> 1.5G RAM (and non-SD disk) and was able to get 190 matches when
>> specifying the first 2 letters in 0.103 seconds using
>> glob($letters.'*'), as well as similar results when specifying all the
>> way to a single unique name.
>
> Thanks. Was that with GLOB_NOSORT included?
>
> James
>

Just the default glob call (I didn't make any optimization attempts,
just wanted to get a general idea for myself).

-- 
Matthew Carter (m@ahungry.com)
http://ahungry.com

[toc] | [prev] | [next] | [standalone]

#15917

On 07/12/2015 05:17, Matthew Carter wrote:
> James Harris <james.harris.1@gmail.com> writes:
>
>> On 05/12/2015 19:45, Matthew Carter wrote:

...

>>> FWIW, I just tested in a directory with 50,000 files on a system with
>>> 1.5G RAM (and non-SD disk) and was able to get 190 matches when
>>> specifying the first 2 letters in 0.103 seconds using
>>> glob($letters.'*'), as well as similar results when specifying all the
>>> way to a single unique name.
>>
>> Thanks. Was that with GLOB_NOSORT included?

> Just the default glob call (I didn't make any optimization attempts,
> just wanted to get a general idea for myself).

Sure. Was just thinking that, depending on implementation, it may/should 
be faster as it omits the sort step.

James

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.php

csiph-web