Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.php > #16439 > unrolled thread

Functions behaving badly: PHP str_split on an empty string

Back to article view | Back to comp.lang.php

Contents

  Functions behaving badly: PHP str_split on an empty string James Harris <james.harris.1@gmail.com> - 2016-02-17 09:57 +0000
    Re: Functions behaving badly: PHP str_split on an empty string "Christoph M. Becker" <cmbecker69@arcor.de> - 2016-02-17 11:53 +0100
      Re: Functions behaving badly: PHP str_split on an empty string James Harris <james.harris.1@gmail.com> - 2016-02-17 16:04 +0000
        Re: Functions behaving badly: PHP str_split on an empty string Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2016-02-17 23:12 +0100

#16439 — Functions behaving badly: PHP str_split on an empty string

I have been using

   foreach (str_split($text) as $char)

but found it does not work properly when $text is empty. Unfortunately, 
when the string is empty str_split returns an array with one element. 
That's nasty. :-(

I've also just realised that str_split works with bytes and not chars.

I see that [] notation also works with bytes and not chars. Is that right?

Is there a recommended way to iterate over the characters of a string 
when that string could be single-byte ASCII or multi-byte Unicode?

As someone fairly new to PHP I have to say I am finding it a bit of a 
nightmare. Possibly because it has grown up over time it seems 
cumbersome and kludgy with *lots* of functions, some of which have been 
added to make up for poor design. Apologies to you who are fans of PHP. 
That's not meant to be inflammatory. It is just an observation.

I guess that seasoned PHP programmers get used to using a subset of the 
language and learn about potential function gotchas such as the one above.

James

[toc] | [next] | [standalone]

#16442

James Harris wrote:

> I have been using
> 
>   foreach (str_split($text) as $char)
> 
> but found it does not work properly when $text is empty. Unfortunately,
> when the string is empty str_split returns an array with one element.
> That's nasty. :-(

ACK.  However, this behavior is documented[1]:

| If the split_length length [which defaults to 1] exceeds the length
| of string, the entire string is returned as the first (and only)
| array element.

> I've also just realised that str_split works with bytes and not chars.
> 
> I see that [] notation also works with bytes and not chars. Is that right?

Yes, that's right.  There have been attempts to add support for Unicode
in the PHP core, but for several reasons this has not been accomplished
(yet).

> Is there a recommended way to iterate over the characters of a string
> when that string could be single-byte ASCII or multi-byte Unicode?

You can use mb_split() or preg_split() for multibyte character
encodings.  Anyway, you should be aware in which character encoding the
string is encoded (mb_detect_encoding() might be helpful).

However, I suggest you reconsider the approach to iterate over the
characters of a string, because it's most likely rather slow in PHP (for
one, PHP doesn't have a char type as C does, so a character is stored as
string[2]), and there may be better alternatives available (PCRE comes
to mind).

> I guess that seasoned PHP programmers get used to using a subset of the
> language and learn about potential function gotchas such as the one above.

Indeed.  Besides waiting for improvements (and maybe contributing to the
effort), it's always possible to use existing libraries (such as
Patchwork-UTF8[3]) or to write your own. :)

[1] <http://php.net/manual/en/function.str-split.php>
[2] AFAIK PHP 7 has an efficient implementation of very small strings
(up to 4 bytes?), though.
[3] <https://github.com/nicolas-grekas/Patchwork-UTF8>

-- 
Christoph M. Becker

[toc] | [prev] | [next] | [standalone]

#16460

On 17/02/2016 10:53, Christoph M. Becker wrote:
> James Harris wrote:

...

>> Is there a recommended way to iterate over the characters of a string
>> when that string could be single-byte ASCII or multi-byte Unicode?
>
> You can use mb_split() or preg_split() for multibyte character
> encodings.  Anyway, you should be aware in which character encoding the
> string is encoded (mb_detect_encoding() might be helpful).

That was useful in making it possible for me to check the types of 
strings. Given:

$request_uri = $_SERVER["REQUEST_URI"];
$uri = urldecode($request_uri)

On this server (Apache) $request_uri is ASCII, even if the URL had some 
Chinese characters in it.

As for $uri, if the input is just ASCII characters then $uri (the output 
of urldecode) is plain ASCII. But if the input contains Chinese 
characters then $uri will be Unicode.

I presume my choices are essentially either to report non-ASCII 
characters as errors or to do all processing in Unicode.

> However, I suggest you reconsider the approach to iterate over the
> characters of a string, because it's most likely rather slow in PHP (for
> one, PHP doesn't have a char type as C does, so a character is stored as
> string[2]), and there may be better alternatives available (PCRE comes
> to mind).

There may be a better way to do it. Here is some of my current code

$request_uri = trim(urldecode($_SERVER["REQUEST_URI"]), "/");
foreach (str_split($request_uri) as $char) {
   if (strpos($path_chars_permitted, $char) === false) {
     $etext .= " Invalid character in URL: \"" .
        htmlspecialchars($char) . "\"<br/>\n";
   }
}

Suggestions for improvement welcome. Maybe there is a function to do the 
lot in one go that would save me iterating over the string. I think the 
best thing would be to convert the above to handle Unicode. That would 
allow better error messages.

(I now know that str_split won't work when $request_uri is empty.)

James

[toc] | [prev] | [next] | [standalone]

#16475

James Harris wrote:

> On 17/02/2016 10:53, Christoph M. Becker wrote:
>> James Harris wrote:
>>> Is there a recommended way to iterate over the characters of a string
>>> when that string could be single-byte ASCII or multi-byte Unicode?

It is a misconception to assume that “Unicode” would be equivalent to
“multi-byte” to begin with.

>> You can use mb_split() or preg_split() for multibyte character
>> encodings.  Anyway, you should be aware in which character encoding the
>> string is encoded (mb_detect_encoding() might be helpful).
> 
> That was useful in making it possible for me to check the types of
> strings. Given:
> 
> $request_uri = $_SERVER["REQUEST_URI"];
> $uri = urldecode($request_uri)
> 
> On this server (Apache) $request_uri is ASCII,

No, it is not.  It is encoded with ISO-8859-1/Windows-1252, but since it 
only contains characters that can be encoded with US-ASCII, "ASCII" is 
reported.  US-ASCII is a *7*-bit encoding.  Keep in mind that the character 
encoding of an arbitrary string can only be guessed, not ascertained.

> even if the URL had some Chinese characters in it.

URIs *always* use only ASCII characters (characters that can be encoded 
using US-ASCII.  Non-ASCII characters have to be UTF-8 percent-encoded, 
which is what *raw*urldecode() decodes.  See RFC 3986 and the PHP manual.
 
> As for $uri, if the input is just ASCII characters then $uri (the output
> of urldecode) is plain ASCII. But if the input contains Chinese
> characters

As I indicated, the input can only contain Chinese characters if they are 
percent-encoded.

> then $uri will be Unicode.

Unicode is not a character encoding, but a standard for a character set and 
several encodings for it.  The character encoding of the return value should 
be UTF-8 then, where a character with a code point below U+0080 is encoded 
with *one* code unit – *one* 8-bit byte (backwards-compatible to US-ASCII).

<http://unicode.org/faq/>

> I presume my choices are essentially either to report non-ASCII
> characters as errors or to do all processing in Unicode.

Non sequitur.
 
>> However, I suggest you reconsider the approach to iterate over the
>> characters of a string, because it's most likely rather slow in PHP (for
>> one, PHP doesn't have a char type as C does, so a character is stored as
>> string[2]), and there may be better alternatives available (PCRE comes
>> to mind).
> 
> There may be a better way to do it. Here is some of my current code
> 
> $request_uri = trim(urldecode($_SERVER["REQUEST_URI"]), "/");
> foreach (str_split($request_uri) as $char) {
>    if (strpos($path_chars_permitted, $char) === false) {
>      $etext .= " Invalid character in URL: \"" .
>         htmlspecialchars($char) . "\"<br/>\n";
>    }
> }
> 
> Suggestions for improvement welcome.

  if (preg_match_all('/[^\\x00-\\x7f]/u', $request_uri, $matches))
  {
    $etext = implode("<br>\n",
      array_map(
        function ($v) {
          return 'Invalid character in URL: "' . htmlspecialchars($v) . '"';
        },
        $matches[0]
      )
    );
  }

But AISB, if a request URI contains unescaped non-ASCII characters, the UA 
making the HTTP request is borked.  If it contains percent-encoded 
characters, and your PHP program cannot handle that, your detection 
algorithm is wrong, and your program is borked from the outset.

> Maybe there is a function to do the lot in one go that would save me
> iterating over the string.

There is.

-- 
PointedEars
Zend Certified PHP Engineer 
<http://www.zend.com/en/yellow-pages/ZEND024953> | Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.php

csiph-web