Path: csiph.com!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail From: Rainer Weikusat Newsgroups: comp.unix.programmer Subject: Re: UTF-8 overlong encodings Date: Sun, 19 Dec 2021 21:53:25 +0000 Lines: 37 Message-ID: <87pmps45yi.fsf@doppelsaurus.mobileactivedefense.com> References: <874k76zvrc.fsf@doppelsaurus.mobileactivedefense.com> <87y24ird6y.fsf@bsb.me.uk> <87mtkx6di7.fsf@doppelsaurus.mobileactivedefense.com> <87sfuppku0.fsf@bsb.me.uk> <87tuf446mh.fsf@doppelsaurus.mobileactivedefense.com> Mime-Version: 1.0 Content-Type: text/plain X-Trace: individual.net KWff+LJnDszJ/BNWjaPAKw4mJSdkhBHbcElzQEvNkTTuI7NKI= Cancel-Lock: sha1:pERDYD+8dpW+pZOFj7Js2ocAH6o= sha1:NB2+9ISgkPcPmbhQ57X4lchruXE= User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) Xref: csiph.com comp.unix.programmer:13655 Rainer Weikusat writes: > Ben Bacarisse writes: >> Rainer Weikusat writes: > > [...] > >>>>> An encoded sequences has a length of 2, 3, 4, 5 or 6 bytes. When >>>>> ignoring the initial special case, the shift value relative to the start >>>>> of the first six bit block for each encoded sequence is 8 - >>>>> its length: >>>>> >>>>> 3 -> 5 >>>>> 4 -> 4 >>>>> 5 -> 3 >>>>> 6 -> 2 >>>>> >>>>> Any corrections or other comments very much welcome. >>>> >>>> I was not sure what this part of the description was supposed to add to >>>> the initial definition. >>> >>> I want to calculate that with a general algorithm. >> >> I don't know what "that" refers to. Do you want to calculate the UTF-8 >> sequence length from the code point? It seems not. Do you want to >> determine if a sequence is overlong by looking at the sequence? It >> seems not. What is the algorithm given, and what it its result? > > I want to determine if a sequence is overlong using a generalized > algorithm for that, ie, not by special-casing start byte values. So far, > the untested (and very likely buggy) code for this looks like follows: > > u_len is the length of the sequence in bytes, p a pointer to the first > byte. Some unrelated consistency checks removed. At the moment, I'm convinced that this algorithm is complete nonsense. :-)