Re: Hallucinated citations are polluting the scientific literature. What can be done?

From	phoenix <j63840576@gmail.com>
Newsgroups	sci.edu, comp.ai.philosophy, alt.books, alt.politics.media, rec.arts.books, misc.writing
Subject	Re: Hallucinated citations are polluting the scientific literature. What can be done?
Date	2026-05-18 18:37 -0600
Message-ID	<n71pm5F8ptmU1@mid.individual.net> (permalink)
References	<20260512.024503.fea93242@mixmin.net> <eb950l1ibvjm1povpc1s3q808nlro4bitq@4ax.com> <87fr3ozsgc.fsf@posteo.de> <adan0lp69tpupjgnmabj1ef28ltoe1qnm7@4ax.com>

Cross-posted to 6 groups.

Show all headers | View raw

Steve Hayes wrote:
> On Mon, 18 May 2026 10:46:11 -0400, Jonathan Lamothe
> <jonathan@jlamothe.net> wrote:
> 
>> Steve Hayes <hayesstw@telkomsa.net> writes:
>>
>> <snip>
>>> On Tue, 12 May 2026 02:45:03 +0100, Lawfare Review
>>> <noreply@mixmin.net> wrote:
>>> LLM bots can be useful tools but they are not AI, and are not
>>> reliable.
>>
>> I have never seen an actual definition if what AI *is*.  Infind it to be
>> a rather useless term fornthat reason.
>>
>>> They are not intelligent or sentient (though the programmers
>>> of some of them try to make them appear so). They do not "understand"
>>> what they are fed, or what they spit out. Their use in education
>>> should be limited to what they are good at, and one needs education
>>> apart form LLM bots to be able to discern what they are good at and
>>> what they are not good at. If students use them to write essays, they
>>> will not learn that discernment.
>>
>> I don't knownthat there's an argument to be made that they're
>> particularly good at *anything*.
> 
> They are useful as a kind of super search engine.
> 
> They can gather information from all over the web and present it in
> digest form. Of couyrse this information is only as accurate as the
> sources it gathers it from, and needs to be checked, but it generally
> saves a lot of time.
> 
>>
>>> My friend who submitted the journal article to Claude is a nuclear
>>> physicist, but the article he got it to write was in my field, not
>>> his, and he rather naively trusted what Claude spat out. If it had
>>> been in his field, his bullshit detectors would have been better
>>> equipped to deal with it.
>>>
>>> In the age of so-called AI, educators need to give serious thought to
>>> better ways of honing students' bullshit detectors.
>>
>> I usually tell people the following about LLMs:
>>
>> Ask it questions about a subject you know well.  See how long it takes
>> it to tell you sonething inaccurate.  Ask yourself if you'd have caught
>> the mistake if you didn't have the knowledge you do.  Finally, ask
>> yourself: do you still trust the LLM?
> 
> In addition to the journal article I referred to, I've been trying it
> out on fiction, since I've seen ads from people who claim that if you
> use their tools you can publish dozens if not hundreds of novels under
> your own name in a fraction of the time it would take to write them.
> 
> We fed it two chapters of a novel I have written but not published,
> and asked it to complete it. Chapter 3 was a quite entertaining
> extrapolation from the first two, but after that it began to
> deteriorate.
> 
> I was impressed with its ability to generate authentic names for new
> characters it introduced (better than some human authoers have done.
> It had accurate informationa bvout some aspects of the setting, but
> the plot was trying to make a mystery out of something trivial and
> improbable, and people behaved and responded to things in ways
> unlikely because of the setting.
> 
> A human author understands human emotions and can make a better guess
> at human reactions. LLms have no empathy, so can't do this. Someone
> says something that most people would see as insulting, but the bot
> writes that the hearer took comfort from it:
> 
> "I don't feel brave."
> "You don't look it either."
> 
> LLMs can be no better than the masterial they work with. They are good
> with factual material, but sometimes have difficulty in distinguishing
> between facts and factoids. Of course humans have the same difficulty:
> as you say, ask yourself if you'd avoid the same mistake if you didn't
> have the knowledge that you do.
> 
The reason I feel comfortable continuing to speak is because I have 
attended seminars on AI and I see no credentials among the others 
speaking here.

If I were to engage in an AI novel, I would talk back and forth a bit in 
order to design a framework of maybe ten chapters.

Then, I might possibly jumble it mildly just so any preconceived notions 
the AI had built about it finishing it were dashed. This is in the 
manner of someone who jumbled a Rubik's cube for me, they jumbled it 
real good and I had to drop back to techniques that I had learned at the 
beginning but not used recently.

Finally I would feed that framework into the AI and tell it to flesh it 
out into the book.

What I see you doing wrong is allowing the AI to construct the plot, and 
the climax and the finish. The more you have it do on its own the more 
generic it will get and will be just like other users engaging in 
"slopnoveling."

YMMV but if you want a really good book, do the hard work and then edit 
it, but this is probably too much for the get rich scheme being pitched.

2. Rinse.
3. Repeat.

-- 
War in the east
War in the west
War up north
War down south
War War

Thread

csiph-web