Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.project > #13704 > unrolled thread

A Different Take on AI

Back to article view | Back to linux.debian.project

Contents

  A Different Take on AI Sam Hartman <hartmans@debian.org> - 2025-02-05 16:00 +0100
    Re: A Different Take on AI "M. Zhou" <lumin@debian.org> - 2025-02-05 18:30 +0100
    Re: A Different Take on AI "M. Zhou" <lumin@debian.org> - 2025-02-10 21:50 +0100
    Re: A Different Take on AI Gard Spreemann <gspr@nonempty.org> - 2025-02-21 11:30 +0100

#13704 — A Different Take on AI

[Multipart message — attachments visible in raw view] — view raw

TL;DR: I think it is important for Debian to consider AI models free
even if those models are based on models that do not release their
training data. In the terms of the DFSG, I think that a model itself is
often a preferred form of modification for creating derived works. Put
another way, I don't think toxic candy is as toxic as  I thought it was
reading  lumin's original ML policy.
If we focus too much on availability of data, I think we will help the
large players and force individuals and small contributors out of the
free software ecosystem.
I will be drafting a GR option to support this position.


Dear lumin:

First, thanks for all your work on AI and free software.
When I started my own AI explorations, I found your ML policy
inspirational in how I thought about AI and free software.
As I have begun my own explorations, which often involve trying to
change/remove bias from models, I have come to think somewhat
differently than you did in your original ML policy.

I apologize that I did not include a lot of references in this message.
I found that I was having trouble coming up with enough time to write it
at all.  I wanted to give you some notice that I planned to draft what I
believe is a competing GR option, and doing that took the time I have.
I am not a researcher by trade, and I do not have all the references and
links I wish I did handy.
I'm just a free software person who has been working on AI as a side
project, because I hope it can make parts of the world I care about
better.

As I understand it, you believe that:

1) Looking at the original training data would be the best approach for
trying to remove bias from a model.

2) It would be difficult/impossible to do that kind of work without
access to the original training data.

I have come to believe that:

1) AI models are not very transparent even if you have the training
data. taking advantage of the training data for a base model is probably
outside the scope of most of us even if we had it.  That's definitely
true for retraining a model, but I think is also true for understanding
where bias is coming from.  That's for base models.  I think that fine
tuning data sets for things like Open Assistant are within the scope of
the masses to examine and use.

2) I think that retraining, particularly with training techniques like
ORPO, is a more effective strategy for the democratized (read
non-Google, non-Meta) community to change bias than using training data.
In other words, I am not convinced that we would use training data even
if we had it, to adjust the bias of our models.
Which is to say I think the preferred form of modification for models is
often the model itself rather than the training data.

Goals
=====


I think both of us are more concerned about democratizing AI. I think we
are more interested in preserving individuals' ability to modify and
create software than we are in promoting monopolies or advantaging
OpenAI, Meta, Google, and the like.
I think we may disagree about how to do that.

With my Debian hat on, I don't really care whether base models are
considered free or non-free. I don't think it will be important for
Debian to include base-models in our archive.
What I do care about is what we can do with software that takes base
models and adapts them for a particular use case.
If LibreOffice gained an AI assistant, our users would be served if we
are able to include a high quality AI assistant that preserves their
core freedoms.
With my Debian hat on, I care more about what we can do with things
derived from base models than the base models themselves.

Core Freedoms
=============

I think that the core freedoms we care about are:

1) Being able to use software.
2) Being able to share software.
3) Being able to modify software.
4) transparency: being able to understand how software works.

Debian has always valued transparency, but I think the DFSG and our
practices have always valued transparency less than the other freedoms.
There's nothing in the DFSG itself that requires transparency.
We've had plenty of arguments over the years about things like minimized
forms of code and whether they met the conditions of the DFSG.
One factor that has been raised is transparency, but it mostly gets
swept aside by the question of  can we modify the software.
The idea appears to be that if we have the preferred form of
modification  that  's transparent enough.
If the upstream doesn't have any advantages in transparency , well, we
decide that's free enough.

One argument that comes up over the years looking at vendoring is
whether replacement is the preferred form of modification.
I have some vendored blob that is a minimized representation of an
upstream software project.  Say minimized javascript or some form of
byte code.
Most of the time I'm going to modify that by replacing the upstream
sources entirely with a new version.
So, at least for vendored code, is that good enough?
generally we've decided that no it is not.
We want individuals to be able to make arbitrary  modifications to the
code, not just replace it.

My claim is that analysis works differently for AI than for minimized
Javascript.

AI is Big
=========

I cannot get my head around how big AI training sets for base models
are.

I was recently looking at the DeepSeek Math paper [1]:

  [1]: https://arxiv.org/abs/2402.03300

As I understand it, they took Their Deepseek Code model as a base.
So that's trained on some huge dataset--so big that they didn't even
want to repeat it.

Then they had a 1.2 billion token dataset (say 6G uncompressed text)
that they used for a training round--some sort of fine tuning round.

Then they applied 2**17 examples (so over a hundred thousand examples)
where they knew both the question and a correct answer.
But the impressive part for me was how the 1.2 billion token dataset was
produced.  I found the discussion of that process fascinating, but it
involved going over a significant chunk of the Common Crawl dataset,
which is mind bogglingly unbelievably huge, to figure out which fraction
of that dataset talks about math reasoning.

Searching the 1.2 billion token data set is clearly within our
capability.
But it's not at all clear to me that I could find what in a 6G dataset
is going to be producing bias.
I think it would be quite possible to hide bias in such a dataset
intentionally, in such a way that even given the 1.2 billion tokens we
would find it difficult to remove the bias by modifying the dataset. I
think there will certainly be unintentional bias there I colud not find.

So, to really have the training data, we need Common Crawl, and we need
the scripts and random seeds necessary to reproduce the 1.2 billion
token dataset.
I also believe there was at least one language model in that process, so
you would also need the training data for that model.

I am quite sure that finding bias in something that large, or even
examining it is outside the scope of all but well funded players.

I am absolutely sure that reducing Common Crawl to the 1.2 billion
tokens --that is actually running the data analysis, including all the
runs of any language models involved--is outside the scope of all but
well funded players. In other words, taking that original training data
and using it is a preferred form of modification is outside of the scope
of everyone we want to center in our work.

And then we're left repeating the process for the base model DeepSeek
Code.

My position is that by taking this approach we've sacrificed
modifyability for transparency, and I am not even sure we have gained
transparency at a price that is available to the members of the
community we want to center in our work.
In this focus on data, we have taken the wrong value trade off for
Debian.
Debian has always put the ability to modify software first.

Free Today, Gone Tomorrow
=========================

One significant concern I know Lumen is aware of with requiring data is
what happens when the data is available today but not tomorrow.
One of the models that tried to be as open as possible ran into problems
because they were forced to take down part of their dataset after the
model was released.
(I believe a copyright issue.)

The AI copyright landscape is very fluid.
Right now we do not know what is fair use.
We do not even have a firm ethical ground for what sharing of data for
AI training should look like in terms of social policy.

We run a significant risk that significant chunks of free software will
depend on what we believe is a free model today, only to have it get
reclassified as non-free tomorrow when some of the training data is no
longer available to the public.

We run significant risks when different jurisdictions have different
laws.

It is very likely that there will be cases where models will still be
distributable even when some of the training datasets underlying the
model can no longer be distributed.

So, you say, let's have several models and switch from one to another if
we run into problems with one model.
Hold that in the back of your mind.  We'll come back.

Debian as Second Class
======================

I am concerned that if we are not careful the quality of models we are
able to offer our users will lag significantly behind the rest of the
world.
If we are much more strict than other free-software projects, we will
limit the models our users can use.
Significant sources of training data will be available to others but not
our users.
I suspect that models that  only need to release data information rather
than training data will be  higher quality because they can have access
to things like published books, works that can be freely used, but not
freely distributed and the like.

Our social contract promises we will value our users and free software.
If we reduce the selection (and thus quality) of what we offer our
users, it should somehow serve free software.
In this instance, I believe that it probably does not serve transparency
and harms our core goal of making software easy to modify. In other
words I do not believe free software is being helped enough to justify
disadvantaging our users.

Preferred Form of Modification
==============================

I talked earlier about how if one model ended up being non-free, we
could switch to another one.
That happens all the time in the AI ecosystem.
A software system has a fine tuning dataset.
They might fine tune a version of Llama3, Mistral, one of the newer
models, all against their dataset.
They will pick the one that performs best.
As new models come out, the base model for some software might switch.

As a practical matter, for the non-monopolies in the free software
ecosystem, the preferred form of modification for base models is the
model themselves.
We switch out models and then adjust the code on top of that, using
various fine tuning and prompt engineering tasks to adapt a model.

The entire ecosystem has evolved to support this.  There are
competitions between models with similar (or the same) inputs.
There are sites that allow you to interact with more than one model at
once so you can choose which works best for you and switch out.  (Or get
around biases or restrictions, perhaps using Chat GPT to write part of a
story, and a more open model to write adult scenes that Chat GPT would
refuse to write.)

On the other hand, I did talk about fine tuning and task-specific or
program-specific datasets.
many of those are at scopes we could modify, and fine tuning (or
producing adapters) models based on those datasets is part of the
preferred form of modification for the programs involved.

What I want for Debian
======================

here's what I want to be able to do for Debian:

* First, the bits of the model--its code and parameters--need to be
  under a DFSG-free license.  So Llama3 is never going to meet Debian
  main's needs under its current license.

* We look at what the software authors actually do to modify models they
  incorporate to determine the preferred form of modification. If in
  practice they switch out base models and fine tune, that's okay.  In
  this situation we probably would need full access to the fine tuning
  data, but the training data for the base model.

*Where it is plausible that the preferred form of modification works
 this way, we effectively cut off the source code and do not look further.  If you
 are integrating model x into your software, your software is free if
 model x is under a free license and any fine tuning data/scripts you
 use are free.  I.E. if our users could actually go from upstream model
 x  to  what the software uses, that's DFSG-free enough even if the user
 could not reproduce model x itself.

I firmly believe that the ability to retrain models to change their bias
without access to the original training data will only continue to
improve.
i think that especially with techniques like ORPO, my explorations
suggest that for smaller models we may have already reached a point that
is good enough for free software.

So What about the OSI Definition
================================

I don't know.
I think it depends on how the OSI definition treats derivative works.
If what we're saying is base models need to release training data, I
think that would harm the free software community.
It would mean free models were always of lower quality than proprietary
models, at least unless the fair use cases go in a direction where all
the models are of low quality.
I  think data information is best for base models.

If instead, what we're saying is that OSI's definition is more focused
on software incorporating models, and it is okay to use a model without
fully specified data is an input so long as you give all the data for
what you do to that model in your  program, I could agree.

If we are saying that to be open source software, any model you use
needs to provide full training data up to the original training run with
random parameters, I think that would harm our community.

[toc] | [next] | [standalone]

#13705

Hi Sam,

Thank you for the input. I see your point, and those are exactly why I
wrote proposal B in my draft. Here is my quick response after going through
the text.

On Wed, 2025-02-05 at 07:45 -0700, Sam Hartman wrote:
> 
> TL;DR: I think it is important for Debian to consider AI models free
> even if those models are based on models that do not release their
> training data. In the terms of the DFSG, I think that a model itself is
> often a preferred form of modification for creating derived works. Put
> another way, I don't think toxic candy is as toxic as  I thought it was
> reading  lumin's original ML policy.
> If we focus too much on availability of data, I think we will help the
> large players and force individuals and small contributors out of the
> free software ecosystem.
> I will be drafting a GR option to support this position.

I want to point out that the "preferred form of modification for creating
DERIVED WORKS" -- the "derived works" is where your proposal (and proposal B)
differs from the proposal A.

Proposal A (toxic candy is not free software), preserves the full freedom
for derived works, but also the freedom to inspect, study, reproduce, modify
the original base model. Only covering derived work is not a integral freedom.

Proposal B (toxic candy is free software), is similar to treating those base
models as blobs (such as firmware) that no free software community can really
handle (at the current stage).

I do not see how proposal A harms the ecosystem. It just prevents huge
binary blobs from entering Debian's main section of the archive. It does not
stop people from uploading the binary blobs to non-free section.

General AI applications are not something to worry about even with proposal A.
DebGPT [https://tracker.debian.org/pkg/debgpt] itself incorporated two common
practice how the existing AI applications work:

(1) by default, DebGPT behaves as a REST API client. It supports a wide range
    of existing service end points, including commercial and self-hosted ones.
(2) the build-in backend of DebGPT can pull a binary blob from internet and
    provide the REST endpoint using that model.

I personally do not see how insisting proposal A can harm the ecosystem. While
developers cannot put binary blobs into main, but you can still trigger the
automatic download from the software in main.

I consistently believe putting a giant binary blob (base model) into main, that
nobody other than the upstream can reproduce is ridiculously funny. That said,
non-free is somewhere such model can go.


My appreciation to software freedom roots in the equal sharing of knowledge
that benefits human in the long run. When I was young, looking at the binary
blobs of Microsoft Windows, while being unable to easily learn how computer
works really disappointed me. The discovery of Debian, makes me feel happy
with open source crap even if they falls behind the closed-source Ferrari.

Proposal A preserves the integrity of knowledge when anybody wants to study
the stuff in depth. Proposal B departures from my appreciation to software
freedom. I hope free software can still help people achieve their personal
revolution in terms of knowledge and skill in the future that belongs to AI,
just like how it has done for me.


Let's leave enough time preparing the proposal. I'll focus on my proposal A
and incorporate the others' suggestions from the list.

[toc] | [prev] | [next] | [standalone]

#13735

On Mon, 2025-02-10 at 19:12 +0100, Christian Kastner wrote:
> > Preferred Form of Modification
> > ==============================
> > [...]
> > As a practical matter, for the non-monopolies in the free software
> > ecosystem, the preferred form of modification for base models is the
> > model themselves.
> 
> I would have strongly disagreed with this until a short while ago, and
> stated that unless I can run a modified training process -- which would
> require the training data -- I don't have the preferred form of
> modification.
> 
> However, recent advances point to new useful models being built from
> other models, for example what DeepSeek accomplished with Llama. They
> obviously didn't have the original training data, yet still built
> something very useful from the base model.
> 
> So I now have a slight doubt. But it is only slight; my gut says that
> even many useful derivations cannot "heal" an initial problem of
> free-ness. Because if the original base model were to disappear (as you
> put it in "Free Today, Gone Tomorrrow"), all derivations in the chain
> would lose their reproducibility, too.

And independence too, which connects to a healthy ecosystem for long run.

Think about the case where basemodel-v1 is released under MIT, and there
are some derivative works around this v1 model. Then someday, the license
of basemodel-v2 has been changed into a proprietary one. The open source
ecosystem around v2 will simply decay.

For a traditional open source or free software, if people are unsatisfied
about how software-v1 is written, or the upstream of software-v1 decides to
discontinue the effort, people can still fork the v1 work, and potentially
create a v2 independently.

Data access matters even more for academia. Without the original training
data, there will never be a fair comparison, let alone rigorous research
for making real improvements. For example, ResNet (for image classification)
is trained on ImageNet (a large scale image dataset, academic-use-only).
The original author has already stopped making improvements on this "base
model". However, people can still try to train new "base models"
such as ViT (vision transformer) on ImageNet to make real improvements.
The original training dataset being accessible, although academic-use-only,
is one key factor that maintains the line of research healthy. If anybody
is unsatisfied with ResNet / ViT / etc, they can reproduce the original
base model, and try to make improvements.

No model is endgame so far. Pre-trained models will be replaced very quickly.
An open-source ecosystem built upon a frozen toxic candy base model is not
iteratable. As long as the frozen base model is outdated, the whole ecosystem
is outdated, because the system is not independent, and cannot iterate itself.

Similarly, if we treat "sbuild" as a "frozen base model". Then the community
can create sbuild-schroot, sbuild-unshare, etc around it. When sbuild discontinues,
the derivative works will be impacted. However, if the fundamentals (dpkg-dev)
remains public, people can still independently design other "frozen base models",
like debspawn (systemd-nspawn) or even docker-based ones. In that sense, the
debian package builder ecosystem is still healthy.

My interpretation on the "toxic candy" is not only focusing on the current,
but also the future, especially the key factors that contributes to a healthy,
positive loop where the ecosystem can constructively grow.

If the software freedom is defined on top of a "toxic candy" base model and
is dependent on it -- once the base model quit the game and discontinues,
the "software freedom" will have to quit the game and discontinue, because
nobody other than the original author has the freedom to improve the original
base model itself.

"Toxic candy" models are not reproducible, and are not something people can
independently improve. By definition, I don't believe this satisfies the definition
of software freedom. If disagreed on this point, then the question turns to
whether "being able to do secondary development" covers all freedoms in definition.

Independence matters for some aspects, like trustworthiness. For example,
what if a "toxic candy" language model respond with spam advertisements instead
of really answering user's question. Nobody is able to fix this "base model" other
than the original author. Should I trust the "toxic candy" model and regard it
as "free software", while being unable to study or modify the "base model" itself?

[toc] | [prev] | [next] | [standalone]

#13746

[Multipart message — attachments visible in raw view] — view raw

Sam Hartman <hartmans@debian.org> writes:

> Dear lumin:
>
> First, thanks for all your work on AI and free software.
> When I started my own AI explorations, I found your ML policy
> inspirational in how I thought about AI and free software.

I'd like to pile on and repeat this sentiment; thank you, Mo!

> With my Debian hat on, I don't really care whether base models are
> considered free or non-free. I don't think it will be important for
> Debian to include base-models in our archive.  What I do care about is
> what we can do with software that takes base models and adapts them
> for a particular use case.

I really struggle to follow this reasoning. What about this way of
thinking does _not_ transfer to "classical" software? And why? Why isn't
what you're saying an equally good (or, I claim, bad) argument for
acceptance of classical software that is somehow derived from non-free
software? (An actual real-world example that springs to mind might be
so-called open source projects that start out with leaked source code
from e.g. a proprietary game).


 Best,
 Gard

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.project

csiph-web