Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > muc.lists.netbsd.tech.toolchain > #3620 > unrolled thread

10.1 make oddity

Back to article view | Back to muc.lists.netbsd.tech.toolchain

Contents

  10.1 make oddity Mouse <mouse@Rodents-Montreal.ORG> - 2026-06-08 19:13 -0400
    Re: 10.1 make oddity Roland Illig <roland.illig@gmx.de> - 2026-06-09 03:06 +0200
      Re: 10.1 make oddity Mouse <mouse@Rodents-Montreal.ORG> - 2026-06-08 22:04 -0400
        Re: 10.1 make oddity Roland Illig <roland.illig@gmx.de> - 2026-06-09 10:31 +0200
      Re: 10.1 make oddity Edgar Fuß <ef@math.uni-bonn.de> - 2026-06-09 11:44 +0200
        Re: 10.1 make oddity Roland Illig <roland.illig@gmx.de> - 2026-06-09 12:41 +0200
          Re: 10.1 make oddity David Holland <dholland-tech@netbsd.org> - 2026-06-10 21:03 +0000

#3620 — 10.1 make oddity

I recently tried to help someone set up some of my infrastructure on a
10.1 machine.  I got reports of some bizarre messages from make.  So I
tried it on a 9.1 machine.  I got the same messages:

make: Unclosed substitution for  (; missing)
make: Unclosed substitution for  (; missing)
make: "/home/mouse/z.local/lib/make/local-prog" line 115: Need an operator

Here's a cut-down test case:

V = a.1
D = /tmp

.MAIN: dummy

.PHONY: dummy
dummy:

.for x in $V
$(x:C;^(.*)\.([0-9].*)$;$D/cat\2/\1.0;):		\
		$(x:C;^.*\.([0-9].*)$;$D/cat\1;)	\
		$(x:C;\.([0-9].*)$;.cat\1;)
	cp $(x:C;\.([0-9].*)$;.cat\1;) $(.TARGET)
.endfor

which, to be completely unambiguous, I repeat base64-encoded:

ViA9IGEuMQpEID0gL3RtcAoKLk1BSU46IGR1bW15CgouUEhPTlk6IGR1bW15CmR1
bW15OgoKLmZvciB4IGluICRWCiQoeDpDO14oLiopXC4oWzAtOV0uKikkOyREL2Nh
dFwyL1wxLjA7KToJCVwKCQkkKHg6QzteLipcLihbMC05XS4qKSQ7JEQvY2F0XDE7
KQlcCgkJJCh4OkM7XC4oWzAtOV0uKikkOy5jYXRcMTspCgljcCAkKHg6QztcLihb
MC05XS4qKSQ7LmNhdFwxOykgJCguVEFSR0VUKQouZW5kZm9yCg==

On a hunch, I tried replacing - in the original, and it works in the
above test case too - the ;s with ^Bs (as in, tr \; \\2 for the test
case) - and the symptom went away.

make(1) on 9.1 says that :C is just like :S except that it uses
regexes.  I *think* I originated this code, long ago, and my version of
it does indeed work with semicolons as delimiters.

The description of :S says that

                                                Any character may be used as a
          delimiter for the parts of the modifier string.

but it appears this is not actually true.  What are the actual
restrictions?  Or is this a bug that needs fixing?  Or am I completely
confused here somehow?

/~\ The ASCII				  Mouse
\ / Ribbon Campaign
 X  Against HTML		mouse@rodents-montreal.org
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-admin@muc.de

[toc] | [next] | [standalone]

#3621

Am 09.06.2026 um 01:13 schrieb Mouse:
> I recently tried to help someone set up some of my infrastructure on a
> 10.1 machine.  I got reports of some bizarre messages from make.  So I
> tried it on a 9.1 machine.  I got the same messages:
> 
> make: Unclosed substitution for  (; missing)
> make: Unclosed substitution for  (; missing)
> make: "/home/mouse/z.local/lib/make/local-prog" line 115: Need an operator
> 
> Here's a cut-down test case:
> 
> V = a.1
> D = /tmp
> 
> .MAIN: dummy
> 
> .PHONY: dummy
> dummy:
> 
> .for x in $V
> $(x:C;^(.*)\.([0-9].*)$;$D/cat\2/\1.0;):		\
> 		$(x:C;^.*\.([0-9].*)$;$D/cat\1;)	\
> 		$(x:C;\.([0-9].*)$;.cat\1;)
> 	cp $(x:C;\.([0-9].*)$;.cat\1;) $(.TARGET)
> .endfor
> 
> On a hunch, I tried replacing the ;s with ^Bs (as in, tr \; \\2 for the test
> case) - and the symptom went away.

> make(1) on 9.1 says that :C is just like :S except that it uses
> regexes.  I *think* I originated this code, long ago, and my version of
> it does indeed work with semicolons as delimiters.

According to my make-archive, there was indeed a time where your test
ran successfully:

> 1998.09.18.20.35.12
> | exit status 0
> 2008.12.21.10.44.10

Then, in December of 2008, the handling of .for loops changed, and since
then, your test failed:

> 2008.12.21.18.06.53
> | make: Unknown modifier '      '
> | make: "/home/rillig/proj/make-archive/mouse.mk" line 8: Missing dependency operator
> | make: Fatal errors encountered -- cannot continue
> |
> | make: stopped in /home/rillig/proj/make-archive
> | exit status 1
> 2009.10.02.07.43.15

More than a year earlier, make's parser was changed to look for ";" in
dependency lines, to support shell commands in the same line:

> target: sources; cc -o target -c sources

Finding that semicolon has always been messy, since its first appearance
in parse.c 1.127 from 2007.01.01.21.47.32, where a comment said:

> No attempt is made to avoid ';' inside substitution patterns.

It improved a bit in parse.c 1.158 from 2009.10.07.16.40.30, where the
comment changed to:

> Attempt to avoid ';' inside substitution patterns.

But still, the code for finding a semicolon didn't cover all edge cases,
as it didn't simply call Var_Parse but instead grew its own parser for
balanced parentheses and braces. Since 2009, that code counts "$(" as
opening and every ")" as closing. For your test case, this means:

> $(x:C;^(.*)\.([0-9].*)$;$D/cat\2/\1.0;):		\
> 		$(x:C;^.*\.([0-9].*)$;$D/cat\1;)	\
> 		$(x:C;\.([0-9].*)$;.cat\1;)

The "$(" is interpreted as an opening parenthesis, but the following "("
is not. The ")" after the ".*" is interpreted as closing, so we're back
at depth 0.

The following "(" is not opening, and the following ")" is not closing,
as we're already at depth 0.

The "$;" is then not interpreted as an expression with variable name
";", instead this semicolon is interpreted as terminating the dependency
line, starting the shell command.

> but it appears this is not actually true.  What are the actual
> restrictions?  Or is this a bug that needs fixing?  Or am I completely
> confused here somehow?

I consider the ad-hoc semicolon finder a bug.

I have a patch ready that fixes the code to use the existing expression
parser. With this fix, your test results in this dependency:

> /tmp/cat1/a.0:		 /tmp/cat1	 a.cat1

That looks good to me.

Roland


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-admin@muc.de

[toc] | [prev] | [next] | [standalone]

#3622

>> make: Unclosed substitution for  (; missing)
>> make: Unclosed substitution for  (; missing)
>> make: "/home/mouse/z.local/lib/make/local-prog" line 115: Need an operator

>> Here's a cut-down test case: [...]

> Then, in December of 2008, the handling of .for loops changed, and
> since then, your test failed:

While it admittedly is armchair quarterbacking, I would submit that it
is a process failure if the test started failing and neither the code
nor the test and documentation got fixed.  In this case, obviously, I
would have argued for fixing the code. :-)

> [...explanation...]

Yes, that does explain.

> I consider the ad-hoc semicolon finder a bug.

So do I!

> I have a patch ready that fixes the code to use the existing
> expression parser.  With this fix, your test results in this
> dependency:

>> /tmp/cat1/a.0:		 /tmp/cat1	 a.cat1

> That looks good to me.

Agreed.

Is this in the queue for committing, or would it help for me to
generate a PR for it, or...?

/~\ The ASCII				  Mouse
\ / Ribbon Campaign
 X  Against HTML		mouse@rodents-montreal.org
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-admin@muc.de

[toc] | [prev] | [next] | [standalone]

#3623

>>> make: Unclosed substitution for  (; missing)
>>> make: Unclosed substitution for  (; missing)
>>> make: "/home/mouse/z.local/lib/make/local-prog" line 115: Need an operator

> Is this in the queue for committing, or would it help for me to
> generate a PR for it, or...?

I fixed it in -current.

I have no immediate plans to pull this change up to netbsd-10 or
netbsd-9, as it has been broken since 2009 at least.

Roland


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-admin@muc.de

[toc] | [prev] | [next] | [standalone]

#3624

> The "$;" is then not interpreted as an expression with variable name ";"
Which, in fact, is correct. The $ is a RE anchor, the ; the RE terminator.

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-admin@muc.de

[toc] | [prev] | [next] | [standalone]

#3625

Am 09.06.2026 um 11:44 schrieb Edgar Fuß:
>> The "$;" is then not interpreted as an expression with variable name ";"
> Which, in fact, is correct. The $ is a RE anchor, the ; the RE terminator.

There are many possible interpretations of what is "correct", yours is
one of them. The manual page at
https://man.netbsd.org/make.1#FILE%20DEPENDENCY%20SPECIFICATIONS doesn't
even mention that expressions are allowed in dependency lines, which
could mean that all bets are off at this point.

POSIX says at
https://pubs.opengroup.org/onlinepubs/9799919799/utilities/make.html#tag_20_76_13_05
that "Macros can appear anywhere in the makefile" and that in target
lines, they are "evaluated when the target line is read", which still
gives plenty of room as to when exactly they are expanded and thus how
the semicolon is to be found. It could be searched for before expanding
the macros or after. The current implementation first searches for ";",
then expands the line, and finally searches for the ":" between targets
and sources. This means that a separator ";" cannot be generated by an
expression, but a separator ":" can. This asymmetry may be a corner case
and is probably not exploited in practice, but it is there.

I fixed the code for finding the semicolon so that it conforms to your
interpretation, assuming that many other people expect the same.
Splitting the line in the middle of an expression was so surprising that
I had to step through the code to find the accurate explanation.

Roland


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-admin@muc.de

[toc] | [prev] | [next] | [standalone]

#3628

On Tue, Jun 09, 2026 at 12:41:58PM +0200, Roland Illig wrote:
 > I fixed the code for finding the semicolon so that it conforms to your
 > interpretation, assuming that many other people expect the same.
 > Splitting the line in the middle of an expression was so surprising that
 > I had to step through the code to find the accurate explanation.

One of these years we really do need to fix the parser to never rescan
anything, which will eliminate most of this kind of trouble. But I am
not going to get to it anytime soon...

-- 
David A. Holland
dholland@netbsd.org

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-admin@muc.de

[toc] | [prev] | [standalone]

Back to top | Article view | muc.lists.netbsd.tech.toolchain

csiph-web