Path: csiph.com!weretis.net!feeder6.news.weretis.net!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end From: Derek Newsgroups: comp.compilers Subject: Re: Figuring out grammars from examples Date: Mon, 15 Apr 2024 02:17:04 +0100 Organization: Compilers Central Sender: johnl%iecc.com Approved: comp.compilers@iecc.com Message-ID: <24-04-004@comp.compilers> References: <24-04-001@comp.compilers> <24-04-002@comp.compilers> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970"; logging-data="10024"; mail-complaints-to="abuse@iecc.com" Keywords: parse Posted-Date: 14 Apr 2024 22:16:37 EDT X-submission-address: compilers@iecc.com X-moderator-address: compilers-request@iecc.com X-FAQ-and-archives: http://compilers.iecc.com In-Reply-To: <24-04-002@comp.compilers> Xref: csiph.com comp.compilers:3563 John, > [I would like to see some actual data. In my experience, LLMs are > impressive, confident, and frequently wrong. -John] LLM's performance on fact recall is poor. It seems to be much better than other tools when dealing with grammars. I could not find the example I was looking for, but here are two others: https://arxiv.org/abs/2305.19234 https://szopa.medium.com/teaching-chatgpt-to-speak-my-sons-invented-language-9d109c0a0f05 My own experience using local (i.e., very small) models https://shape-of-code.com/2024/02/25/extracting-named-entities-from-a-change-log-using-an-llm/