Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > sci.electronics.design > #747519

AI code and circuit designs (was Re: hneemann's "Digital" for more than just introduction-to-EE?)

Show all headers | View raw

someone
<2a59d59e3809f827ce709d3815e3950eef4a6a93af5557a93a7fdfba71460843@example.com>
writes:
> Almost all AI code is something that has been scraped from the
> literature that is mostly tutorial and demonstration level. It is not
> recommended for a working product. Testing it is a start but no
> guarantee.

It’s probably true that most AI code is tutorial and demonstration
level, but it’s probably not “scraped from the literature”.  But it
might depend on which AI model you’re using, and how you’re using it.

***

Sunday night, I asked Claude Opus 5 (no longer a frontier model now that
Fable is out) to write a bit-serial Verilog soft core for a minimal CPU
architecture I’d designed 17 years ago: four 12-bit registers, two of
which form an operand stack.  (The other two are the PC, which is
actually 11 bits, and the instruction register.)  The ALU instructions
are just subtraction and NAND, so you have to synthesize addition out of
subtraction, and XOR out of NAND, and the like.  Very inconvenient, and
definitely not an architecture you can “scrape code from the literature”
for.

After asking me about the instruction encoding, which I’d left out of
the design doc, it invented an instruction encoding and wrote the
Verilog.  But, because it didn’t have Verilator, it hand-translated it
into a simulator in RTL-level Python, wrote an idiomatic Python
simulator for the instruction set, and wrote a fuzzer to generate
thousands of random programs and ensure that they executed the identical
sequence of register values on both simulators.

For good measure, it wrote an assembler, a disassembler, and a number of
reasonable small example programs in the assembly language, and also
used those programs in the equivalence test, and tested that their
output was what was expected.  Also, some other tooling.

The Verilog specified `timestamp inconsistently, and also had a comment
that Verilator interpreted as a pragma, and once those were removed, the
Verilog also worked on the example programs — as you would expect from
the fact that Claude had tested them on the Python RTL-level simulator.

This is a level of testing common in electronics but much higher than
people normally apply to software.

***

With Yosys, the CPU synthesized to 104 Lattice 4-input-LUT logic cells,
which seems like a pretty reasonable size.  I haven’t tested it in an
actual FPGA yet.  I’m skeptical that this design is actually a good idea
for any product — SeRV would be slower, but it’s much easier to program,
and it’s less than twice as big.  This CPU design has been mostly an
exercise for learning about LLMs for me.

It was pretty astounding to me that it just went ahead and casually
wrote a bunch of assembly-language programs for a weird architecture
unlike anything that’s ever been made.  I didn’t look at the log of its
“tool calls” but I wouldn’t be surprised if it had to fix the programs
several times before they worked.  But they did work.

***

On the other hand, the assembler was passing the definitions of
numerical constants to Python’s `eval`, allowing a malicious
assembly-language source file to run the assembler out of memory and
crash it — and maybe crash your whole computer.  I suspect assembling a
malicious assembly-language source file could delete your home directory
after emailing your dick pics to your in-laws and a death threat to the
President, but the obvious things I tried to escape the assembler’s
`__builtins__:{}` sandbox didn’t work.  But Python hasn’t supported
sandboxing since Python 2.2, so there’s probably a way.

This inappropriate `eval` is an example of a serious problem in a piece
of AI-generated code.  No amount of undirected testing is likely to
uncover things like that; you really did need to review the code.  And,
of course, the AI’s rigorous testing did not uncover it — that was
focused entirely on the CPU simulator, not the assembler.  Maybe if I’d
asked the AI to do a security review of its own code, it would have
flagged it.

Kragen

Back to sci.electronics.design | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

hneemann's "Digital" for more than just introduction-to-EE? Kragen Javier Sitaker <kragen@canonical.org> - 2026-09-04 22:41 -0300
  Re: hneemann's "Digital" for more than just introduction-to-EE? Don Y <blockedofcourse@foo.invalid> - 2026-09-05 07:11 -0700
  Re: hneemann's "Digital" for more than just introduction-to-EE? john larkin <jl@glen--canyon.com> - 2026-09-05 09:45 -0700
    Re: hneemann's "Digital" for more than just introduction-to-EE? Cóilín Nioclásín Glostéir <thanks-to@Taf.com> - 2026-09-05 21:41 +0000
      Re: hneemann's "Digital" for more than just introduction-to-EE? john larkin <jl@glen--canyon.com> - 2026-09-05 17:23 -0700
        Re: hneemann's "Digital" for more than just introduction-to-EE? someone <2a59d59e3809f827ce709d3815e3950eef4a6a93af5557a93a7fdfba71460843@example.com> - 2026-09-09 17:45 +0000
          AI code and circuit designs (was Re: hneemann's "Digital" for more than just introduction-to-EE?) Kragen Javier Sitaker <kragen@canonical.org> - 2026-09-10 02:39 -0300
            Re: AI code and circuit designs (was Re: hneemann's "Digital" for more than just introduction-to-EE?) Don Y <blockedofcourse@foo.invalid> - 2026-09-09 23:19 -0700
  Re: hneemann's "Digital" for more than just introduction-to-EE? Cóilín Nioclásín Glostéir <thanks-to@Taf.com> - 2026-09-05 18:59 +0000
  Re: hneemann's "Digital" for more than just introduction-to-EE? someone <2a59d59e3809f827ce709d3815e3950eef4a6a93af5557a93a7fdfba71460843@example.com> - 2026-09-09 17:45 +0000
    Re: hneemann's "Digital" for more than just introduction-to-EE? Kragen Javier Sitaker <kragen@canonical.org> - 2026-09-10 03:19 -0300

csiph-web