Path: csiph.com!weretis.net!feeder9.news.weretis.net!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end From: anton@mips.complang.tuwien.ac.at Newsgroups: comp.compilers,comp.lang.lisp Subject: Re: the death of optimization compiler Date: Fri, 16 Oct 2026 10:41:59 +0000 Organization: Compilers Central Sender: johnl%iecc.com Approved: comp.compilers@iecc.com Message-ID: <26-10-011@comp.compilers> References: <26-10-010@comp.compilers> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970"; logging-data="32489"; mail-complaints-to="abuse@iecc.com" Keywords: optimize, Lisp Posted-Date: 16 Oct 2026 09:54:28 EDT X-submission-address: compilers@iecc.com X-moderator-address: compilers-request@iecc.com X-FAQ-and-archives: http://compilers.iecc.com Xref: csiph.com comp.compilers:3776 comp.lang.lisp:61745 Anton Antimo writes: >In 1974, Donald Knuth had already remarked: > >--8<-------------------------------------------------------->8--- > For 15 years or so I have been trying to think of how to write a > compiler that really produces top quality code. For example, most of > the MIX programs in my books are considerably more efficient than any > of today���s most visionary compiling schemes would be able to > produce. [...] We [...] always [ran] up against the same problem: the > compiler needs to be in a dialog with the programmer; it needs to know > properties of the data, and whether certain cases can arise, etc. > > For some reason we all (especially me) had a mental block about > optimization, namely that we always regarded it as a behind-the- > scenes activity, [...] which the programmer isn���t supposed to know. > This veil was first lifted from my eyes in [...] 1973, when I ran > across a remark by Hoare [...] that, ideally, a language should be > designed so that an optimizing compiler can describe its optimizations > in the source language. Of course! Why hadn���t I ever thought of it? > > A program-manipulation system is obviously what we���ve been leading up > to, and I wonder why I never thought of it before. Of course, the > idea isn't original with me; when I told Hoare, he said, ``Exactly!'' > and referred me to a recent work by Darlington and Burstall [...] > which describes a system which removes some recursions from a > LISP-like language (curiously, without introducing any [go-to]���s) > [...]. Since LISP programs are easily manipulated as LISP [...] > objects, there has also been a rather extensive development of similar > ideas in this domain, notably by Warren Teitelman [...]. The time is > clearly ripe for program-manipulation systems, and a great deal of > further work suggests itself. > > Source: ``Structured programming with go to statements,'' Donald > E. Knuth, ACM Computing Surveys (CSUR) 6.4 (1974): 261--301. >--8<-------------------------------------------------------->8--- This reminds me the research project CIP (Computer-Aided, Intuition-Guided Programming): They designed a wide-spectrum language CIP-L (described in LNCS 183, published 1985) and a program transformation system CIP-S (described in LNCS 292, published 1987). The idea was that you write a specification in CIP-L, and then tell CIP-S to perform various transformation steps, that eventually result in an executable and efficient program. This idea looked interesting, amd I looked in the books, but did not find them inspiring apart from the basic idea. And apparently the idea has not taken the world by storm, not even in the research world (according to Springer there are 63 citations of LNCS 183 and 45 of LNCS 292). It seems that we prefer to write executable programs from the start, and either optimize them by applying transformations manually, or fully automatic (using an optimizing compiler), not semi-automatically. I guess one of the problems of the idea of semi-automatic transformations is that the transformations are no longer applicable after changes due to changes in requirements. So you either want to apply the changes to the transformed program (and when you transform it manually, and treat the transformed program as the new source, you can easily apply changes to the transformed program), or you want fully automatic transformation, even if you only get a best-effort result. >It seems to me that telling the compiler what is mutable is a clear >illustration of how a compiler is in dialog with the programmer, just >like declaring a variable's type, with the C ``register'' keyword as >another example. It's actually a monolog. You might consider that error messages from a compiler turn it into a dialog, but that's not how programming in most languages works (including Lisp, while Rust may be the exception that proves the rule). But that should not distract from the idea that one should be able to express the optimized code in the source language. And in many respects C (not the standardized subset) gives us that. E.g., we can search for the index of the smallest element in an array by writing it in the common subset of Java and C: for (i=0, best=0; i, but these extensions are limited and don't allow programmers to express all operations supported by SIMD instruction sets nor all instructions generated (when you are lucky) by auto-vectorization. So I have written loops like for (j = ncities-SIMD_SIZE; (long)j >= (long)i; j -= SIMD_SIZE) { [...] for (k=0; k