Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.bugs.dist > #1042926 > unrolled thread

Bug#980302: dictionaries-common-dev: vim dictionary support

Back to article view | Back to linux.debian.bugs.dist

Contents

  Bug#980302: dictionaries-common-dev: vim dictionary support Kurt Roeckx <kurt@roeckx.be> - 2021-01-17 16:10 +0100
    Bug#980302: dictionaries-common-dev: vim dictionary support Agustin Martin <agmartin@debian.org> - 2021-01-18 13:30 +0100
      Bug#980302: dictionaries-common-dev: vim dictionary support Kurt Roeckx <kurt@roeckx.be> - 2021-01-18 14:00 +0100
        Bug#980302: dictionaries-common-dev: vim dictionary support Agustin Martin <agmartin@debian.org> - 2021-01-18 19:40 +0100
          Bug#980302: dictionaries-common-dev: vim dictionary support Kurt Roeckx <kurt@roeckx.be> - 2021-01-18 20:40 +0100
            Bug#980302: dictionaries-common-dev: vim dictionary support Agustin Martin <agmartin@debian.org> - 2021-01-19 11:00 +0100
              Bug#980302: dictionaries-common-dev: vim dictionary support Kurt Roeckx <kurt@roeckx.be> - 2021-01-19 11:10 +0100
          Bug#980302: dictionaries-common-dev: vim dictionary support Agustin Martin <agmartin@debian.org> - 2021-02-01 00:30 +0100
            Bug#980302: dictionaries-common-dev: vim dictionary support James McCoy <jamessan@debian.org> - 2021-02-02 03:50 +0100
              Bug#980302: dictionaries-common-dev: vim dictionary support Agustin Martin <agmartin@debian.org> - 2021-02-03 21:10 +0100

#1042926 — Bug#980302: dictionaries-common-dev: vim dictionary support

Package: dictionaries-common-dev

Hi,

Vim has support for spellchecking using it's own spell check
files. It can be generated based on hunspell .aff/.dic files, or
based on a wordlist. Files currently need to be placed in
/usr/share/vim/vim82/spell/

It would be good if we had a policy and helper scripts to install
the files so vim can use them, and how to name the packages.


Kurt

[toc] | [next] | [standalone]

#1043112

El dom, 17 ene 2021 a las 16:06, Kurt Roeckx (<kurt@roeckx.be>) escribió:
>
> Package: dictionaries-common-dev
>
> Hi,
>
> Vim has support for spellchecking using it's own spell check
> files. It can be generated based on hunspell .aff/.dic files, or
> based on a wordlist. Files currently need to be placed in
> /usr/share/vim/vim82/spell/
>
> It would be good if we had a policy and helper scripts to install
> the files so vim can use them, and how to name the packages.

Hi, Kurt,

This was mentioned a while ago in #490987. It was closed because at
that time building vim spell files creation seemed to be very memory
intensive and using libhunspell via a redhat patch was under
consideration. I have no idea about current status of spellchecking in
vim and if those concerns were addressed.

On the other hand most hunspell dicts come from libreoffice-dicts and
do not provide a dict-common info file. In my TODO list there is a
point to play with libreoffice-dicts build process to automatically
create a minimal info file, but it has been there for years due to
lack of both time and python skills at that time.

Regards,

-- 
Agustin

[toc] | [prev] | [next] | [standalone]

#1043114

On Mon, Jan 18, 2021 at 01:22:48PM +0100, Agustin Martin wrote:
> El dom, 17 ene 2021 a las 16:06, Kurt Roeckx (<kurt@roeckx.be>) escribió:
> >
> > Package: dictionaries-common-dev
> >
> > Hi,
> >
> > Vim has support for spellchecking using it's own spell check
> > files. It can be generated based on hunspell .aff/.dic files, or
> > based on a wordlist. Files currently need to be placed in
> > /usr/share/vim/vim82/spell/
> >
> > It would be good if we had a policy and helper scripts to install
> > the files so vim can use them, and how to name the packages.
> 
> Hi, Kurt,
> 
> This was mentioned a while ago in #490987. It was closed because at
> that time building vim spell files creation seemed to be very memory
> intensive and using libhunspell via a redhat patch was under
> consideration. I have no idea about current status of spellchecking in
> vim and if those concerns were addressed.

Converting it from .aff and .dic to .spl and .sug takes 2.3
seconds here and 99 MB of RAM.

Note that I need to patch the .aff files to actually make it work
properly because vim doesn't implement all the same things
hunspell does.

I was looking at the libhunspell patch upstream, which seems to be
going nowhere, the pull requests was closed because it wasn't
complete and nobody seems to want to fix it.

> On the other hand most hunspell dicts come from libreoffice-dicts and
> do not provide a dict-common info file. In my TODO list there is a
> point to play with libreoffice-dicts build process to automatically
> create a minimal info file, but it has been there for years due to
> lack of both time and python skills at that time.

I have no idea what those files are used for, so I can't even test
that they are correct or not. I assume that it might have something
to do with emacs, but I'm a vim user, not an emacs user.


Kurt

[toc] | [prev] | [next] | [standalone]

#1043163

El lun, 18 ene 2021 a las 13:54, Kurt Roeckx (<kurt@roeckx.be>) escribió:
>
> On Mon, Jan 18, 2021 at 01:22:48PM +0100, Agustin Martin wrote:
> > El dom, 17 ene 2021 a las 16:06, Kurt Roeckx (<kurt@roeckx.be>) escribió:
> > >
> > > Package: dictionaries-common-dev
> > >
> > > Hi,
> > >
> > > Vim has support for spellchecking using it's own spell check
> > > files. It can be generated based on hunspell .aff/.dic files, or
> > > based on a wordlist. Files currently need to be placed in
> > > /usr/share/vim/vim82/spell/
> > >
> > > It would be good if we had a policy and helper scripts to install
> > > the files so vim can use them, and how to name the packages.
> >
> > Hi, Kurt,
> >
> > This was mentioned a while ago in #490987. It was closed because at
> > that time building vim spell files creation seemed to be very memory
> > intensive and using libhunspell via a redhat patch was under
> > consideration. I have no idea about current status of spellchecking in
> > vim and if those concerns were addressed.
>
> Converting it from .aff and .dic to .spl and .sug takes 2.3
> seconds here and 99 MB of RAM.

Thanks for the info, Kurt.

Some (many) years ago, there used to be a vim-spellfiles package
containing dictionaries for use with vim. It was removed because of
maintainer request (#471285). Reason given by maintainer was "Vim
provides its own method for downloading spellfiles that are needed and
I don't actually have a computer that can build the spellfiles
packages (doing so is quite memory intensive)". Happy to know that
things (or computers) were improved.

> Note that I need to patch the .aff files to actually make it work
> properly because vim doesn't implement all the same things
> hunspell does.

Knowing that vim does not use standard hunspell files makes things
more clear. I thought you meant something like what is done for aspell
or ispell dicts, allowing hashes to be built at postinst stage.
However, seems that what you would like is to recover something like
vim-spellfiles, but adapted to current vim.

> I was looking at the libhunspell patch upstream, which seems to be
> going nowhere, the pull requests was closed because it wasn't
> complete and nobody seems to want to fix it.
>
> > On the other hand most hunspell dicts come from libreoffice-dicts and
> > do not provide a dict-common info file. In my TODO list there is a
> > point to play with libreoffice-dicts build process to automatically
> > create a minimal info file, but it has been there for years due to
> > lack of both time and python skills at that time.
>
> I have no idea what those files are used for, so I can't even test
> that they are correct or not. I assume that it might have something
> to do with emacs, but I'm a vim user, not an emacs user.

Note that I was thinking about autobuild, and for ispell and aspell
some info may be there for hash autobuild, together with for other
things like Emacs or jed.

I do not object adding a vim section to policy if there is need to
coordinate with packages outside the vim environment. However, from
what I seem to understand, everything is inside vim ecosystem.
spellchecker is part of vim, hunspell dicts need to be adapted for vim
and this spellchecking facility seems not to be used outside vim.

So we would need to have a better understanding of what is expected.
Will vim files be built from a central package (like libreoffice-dicts
does for most hunspell dicts) or there will be separate source
packages for each dict, probably with different maintainers? If the
second, a vim spellckeck policy and helper scripts are useful. Note,
however, that I do not use vim, so some help will be needed, where is
this feature documented?

Regards,

-- 
Agustin

[toc] | [prev] | [next] | [standalone]

#1043173

On Mon, Jan 18, 2021 at 07:34:55PM +0100, Agustin Martin wrote:
> > Note that I need to patch the .aff files to actually make it work
> > properly because vim doesn't implement all the same things
> > hunspell does.
> 
> Knowing that vim does not use standard hunspell files makes things
> more clear. I thought you meant something like what is done for aspell
> or ispell dicts, allowing hashes to be built at postinst stage.
> However, seems that what you would like is to recover something like
> vim-spellfiles, but adapted to current vim.

Vim has it's own file format and implementation that works totally
different than hunspell, and so needs to convert it to it's own
format.

I have no idea what vim-spellfiles was, it only seems to have been
in experimental.

Vim has various files on it's ftp site at:
https://ftp.nluug.nl/vim/runtime/spell/

I assume someone packaged up all the files from the ftp site.

As you can see at
https://github.com/vim/vim/tree/master/runtime/spell
most haven't been updated in the past 10 years, nor is the source
of them actually available, just a patch against something, and
it's unclear to me to which version the patch is. I haven't tried
to run the aap script to see if it can actually still fetch
the source.

If you have the input files, it's easy to generate the files from
it.

> > I was looking at the libhunspell patch upstream, which seems to be
> > going nowhere, the pull requests was closed because it wasn't
> > complete and nobody seems to want to fix it.
> >
> > > On the other hand most hunspell dicts come from libreoffice-dicts and
> > > do not provide a dict-common info file. In my TODO list there is a
> > > point to play with libreoffice-dicts build process to automatically
> > > create a minimal info file, but it has been there for years due to
> > > lack of both time and python skills at that time.
> >
> > I have no idea what those files are used for, so I can't even test
> > that they are correct or not. I assume that it might have something
> > to do with emacs, but I'm a vim user, not an emacs user.
> 
> Note that I was thinking about autobuild, and for ispell and aspell
> some info may be there for hash autobuild, together with for other
> things like Emacs or jed.
> 
> I do not object adding a vim section to policy if there is need to
> coordinate with packages outside the vim environment. However, from
> what I seem to understand, everything is inside vim ecosystem.
> spellchecker is part of vim, hunspell dicts need to be adapted for vim
> and this spellchecking facility seems not to be used outside vim.
> 
> So we would need to have a better understanding of what is expected.
> Will vim files be built from a central package (like libreoffice-dicts
> does for most hunspell dicts) or there will be separate source
> packages for each dict, probably with different maintainers? If the
> second, a vim spellckeck policy and helper scripts are useful. Note,
> however, that I do not use vim, so some help will be needed, where is
> this feature documented?

There is documentation about it in:
/usr/share/vim/vim82/doc/spell.txt
Or online at: https://vimhelp.org/spell.txt.html

I think we should at least ship the files in Debian somehow, the
user shouldn't need to download it.


Kurt

[toc] | [prev] | [next] | [standalone]

#1043279

El lun, 18 ene 2021 a las 20:26, Kurt Roeckx (<kurt@roeckx.be>) escribió:
>
> Vim has it's own file format and implementation that works totally
> different than hunspell, and so needs to convert it to it's own
> format.
>
> I have no idea what vim-spellfiles was, it only seems to have been
> in experimental.

It is still available from http://snapshot.debian.org/package/vim-spellfiles/

After a quick glance it seems to contain some diff and aap files,
probably like those in
https://github.com/vim/vim/tree/master/runtime/spell but even more
outdated and with a different structure. I wonder if some of the old
maintainers may still be interested in vim spellchecking, currently
there is a single vim uploader.

One thing, do you know if processed vim dictionaries are arch:all?

> There is documentation about it in:
> /usr/share/vim/vim82/doc/spell.txt
> Or online at: https://vimhelp.org/spell.txt.html

Thanks for the info. Have to look deeper, may be aff format supported
by vim is closer to an old hunspell format, hopefully the same subset
aspell supports. That could make things easier.

--
Agustin

[toc] | [prev] | [next] | [standalone]

#1043284

On Tue, Jan 19, 2021 at 10:55:28AM +0100, Agustin Martin wrote:
> One thing, do you know if processed vim dictionaries are arch:all?

I have no idea, but I assume it is.


Kurt

[toc] | [prev] | [next] | [standalone]

#1045618

Control: reassign -1 dictionaries-common-dev,vim

El lun, 18 ene 2021 a las 19:34, Agustin Martin
(<agmartin@debian.org>) escribió:
> El lun, 18 ene 2021 a las 13:54, Kurt Roeckx (<kurt@roeckx.be>) escribió:
> > > > Vim has support for spellchecking using it's own spell check
> > > > files. It can be generated based on hunspell .aff/.dic files, or
> > > > based on a wordlist. Files currently need to be placed in
> > > > /usr/share/vim/vim82/spell/
...
> > > This was mentioned a while ago in #490987. It was closed because at
> > > that time building vim spell files creation seemed to be very memory
> > > intensive and using libhunspell via a redhat patch was under
> > > consideration. I have no idea about current status of spellchecking in
> > > vim and if those concerns were addressed.
...
> I do not object adding a vim section to policy if there is need to
> coordinate with packages outside the vim environment. However, from
> what I seem to understand, everything is inside vim ecosystem.
> spellchecker is part of vim, hunspell dicts need to be adapted for vim
> and this spellchecking facility seems not to be used outside vim.

Hi, Kurt and James,

I am temporarily reassigning this bug report to both vim and
dictionaries-common-dev, so vim maintaners are aware of it and can
share their POV. I will not go anywhere without their approval.

I have been looking at
https://github.com/vim/vim/tree/master/runtime/spell and
http://snapshot.debian.org/package/vim-spellfiles/ to have an idea
about this issue. Seems vim uses myspell dictionaries (or early
hunspell dicts). Current hunspell dicts normally use a number of
things that seem unsupported by vim spellchecker, so while using
recent hunspell dicts may appear to work, it will likely work worse
than old files.

I can think of two approaches, resurrecting vim-spellfiles package
and/or creating vim dicts from packages currently using a myspell dict
(normally as a source for aspell dict). Have to ellaborate a bit more.

James, ¿what do you think about this?

Regards,

-- 
Agustin

[toc] | [prev] | [next] | [standalone]

#1045823

On Mon, Feb 01, 2021 at 12:22:03AM +0100, Agustin Martin wrote:
> Control: reassign -1 dictionaries-common-dev,vim
> 
> El lun, 18 ene 2021 a las 19:34, Agustin Martin
> (<agmartin@debian.org>) escribió:
> > El lun, 18 ene 2021 a las 13:54, Kurt Roeckx (<kurt@roeckx.be>) escribió:
> > > > > Vim has support for spellchecking using it's own spell check
> > > > > files. It can be generated based on hunspell .aff/.dic files, or
> > > > > based on a wordlist. Files currently need to be placed in
> > > > > /usr/share/vim/vim82/spell/
> ...
> > > > This was mentioned a while ago in #490987. It was closed because at
> > > > that time building vim spell files creation seemed to be very memory
> > > > intensive and using libhunspell via a redhat patch was under
> > > > consideration. I have no idea about current status of spellchecking in
> > > > vim and if those concerns were addressed.
> ...
> > I do not object adding a vim section to policy if there is need to
> > coordinate with packages outside the vim environment. However, from
> > what I seem to understand, everything is inside vim ecosystem.
> > spellchecker is part of vim, hunspell dicts need to be adapted for vim
> > and this spellchecking facility seems not to be used outside vim.
> 
> Hi, Kurt and James,
> 
> I am temporarily reassigning this bug report to both vim and
> dictionaries-common-dev, so vim maintaners are aware of it and can
> share their POV. I will not go anywhere without their approval.

Thanks.

> I have been looking at
> https://github.com/vim/vim/tree/master/runtime/spell and
> http://snapshot.debian.org/package/vim-spellfiles/ to have an idea
> about this issue. Seems vim uses myspell dictionaries (or early
> hunspell dicts). Current hunspell dicts normally use a number of
> things that seem unsupported by vim spellchecker, so while using
> recent hunspell dicts may appear to work, it will likely work worse
> than old files.
> 
> I can think of two approaches, resurrecting vim-spellfiles package
> and/or creating vim dicts from packages currently using a myspell dict
> (normally as a source for aspell dict). Have to ellaborate a bit more.

Back when I last tried to maintain vim-spellfiles, most of the languages
were reasonable to build.  There were just a couple which completely
crippled my computer when building.

Bram appears receptive to reviewing an fully fleshed out version of the
libhunspell patch, so I'd prefer to focus efforts on cleaning that up
and getting it upstreamed.  That would also avoid duplicating efforts on
keeping multiple sources of dictionaries updated.

Cheers,
-- 
James
GPG Key: 4096R/91BF BF4D 6956 BD5D F7B7  2D23 DFE6 91AE 331B A3DB

[toc] | [prev] | [next] | [standalone]

#1046158

[Multipart message — attachments visible in raw view] — view raw

El mar, 2 feb 2021 a las 3:45, James McCoy (<jamessan@debian.org>) escribió:

> Back when I last tried to maintain vim-spellfiles, most of the languages
> were reasonable to build.  There were just a couple which completely
> crippled my computer when building.

Thanks for the feedback, James,

I was having some fun with the espa~nol package, which still provides
a pure myspell-es dict (different and from different authors than
hunspell-es) to see how hard would be to generate a vim spellchecking
dict from it. For those curious I am attaching a diff with the used
changes, although I would prefer the libhunspell way. It took around
8s per dict, not a big problem. To properly build utf-8 part in a
pbuilder chroot I had to set LC_ALL instead of LANG. Another problem I
noticed is that vim expects those dicts in a versioned dir, so unless
somehing else is added to vim search path package will need to be
rebuilt everytime that version changes.

> Bram appears receptive to reviewing an fully fleshed out version of the
> libhunspell patch, so I'd prefer to focus efforts on cleaning that up
> and getting it upstreamed.  That would also avoid duplicating efforts on
> keeping multiple sources of dictionaries updated.

Agreed. Most myspell only dicts are somewhat old and most new hunspell
dicts will not work well with vim spellchecking engine. Hope someone
revitalizes the libunspell patch way.

Regards,

-- 
Agustin

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.bugs.dist

csiph-web