Groups > linux.debian.bugs.dist > #1231945

Bug#1063673: ITP: llama.cpp -- Inference of Meta's LLaMA model (and others) in pure C/C++

From	"M. Zhou" <lumin@debian.org>
Newsgroups	linux.debian.bugs.dist
Subject	Bug#1063673: ITP: llama.cpp -- Inference of Meta's LLaMA model (and others) in pure C/C++
Date	2025-02-06 02:50 +0100
Message-ID	<KcZbI-eIzF-3@gated-at.bofh.it> (permalink)
References	(17 earlier) <KcY5X-eHYo-5@gated-at.bofh.it> <KcY5X-eHYo-3@gated-at.bofh.it> <KcZbI-eIzF-5@gated-at.bofh.it> <KcZbI-eIzF-7@gated-at.bofh.it> <KcY5X-eHYo-3@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

On Thu, 2025-02-06 at 01:33 +0100, Petter Reinholdtsen wrote:
> 
> I was sad to discover the server example is missing, as it is the
> llama.cpp progam I use the most.  Without it, I will have to continue
> using my own build.

I second this. llama-server is also the service endpoint for DebGPT.

I pushed a fix for ppc64el. The hwcaps works correctly for power9, given the baseline is power 8.

(chroot:unstable-ppc64el-sbuild) root@debian-project-1 /h/d/llama.cpp.pkg [1]# ldd (which llama-cli)                                                                                               
        linux-vdso64.so.1 (0x00007fffa4810000)                                                                                                                                                     
        libeatmydata.so => /lib/powerpc64le-linux-gnu/libeatmydata.so (0x00007fffa4600000)                                                                                                         
        libllama.so => /usr/lib/powerpc64le-linux-gnu/llama.cpp/glibc-hwcaps/power9/libllama.so (0x00007fffa4450000)                                                                               
        libggml.so => /usr/lib/powerpc64le-linux-gnu/llama.cpp/glibc-hwcaps/power9/libggml.so (0x00007fffa4420000)                                                                                 
        libggml-base.so => /usr/lib/powerpc64le-linux-gnu/llama.cpp/glibc-hwcaps/power9/libggml-base.so (0x00007fffa4330000)                                                                       
        libstdc++.so.6 => /lib/powerpc64le-linux-gnu/libstdc++.so.6 (0x00007fffa3fd0000)                                                                                                           
        libm.so.6 => /lib/powerpc64le-linux-gnu/libm.so.6 (0x00007fffa3e90000)
        libgcc_s.so.1 => /lib/powerpc64le-linux-gnu/libgcc_s.so.1 (0x00007fffa3e50000)
        libc.so.6 => /lib/powerpc64le-linux-gnu/libc.so.6 (0x00007fffa3be0000)
        /lib64/ld64.so.2 (0x00007fffa4820000)
        libggml-cpu.so => /usr/lib/powerpc64le-linux-gnu/llama.cpp/glibc-hwcaps/power9/libggml-cpu.so (0x00007fffa3b20000)
        libgomp.so.1 => /lib/powerpc64le-linux-gnu/libgomp.so.1 (0x00007fffa3a90000)

Thread

Bug#1063673: ITP: llama.cpp -- Inference of Meta's LLaMA model (and others) in pure C/C++ Petter Reinholdtsen <pere@hungry.com> - 2025-02-05 22:00 +0100
  Bug#1063673: ITP: llama.cpp -- Inference of Meta's LLaMA model (and others) in pure C/C++ Petter Reinholdtsen <pere@hungry.com> - 2025-02-06 01:40 +0100
    Bug#1063673: ITP: llama.cpp -- Inference of Meta's LLaMA model (and others) in pure C/C++ "M. Zhou" <lumin@debian.org> - 2025-02-06 02:50 +0100
      Bug#1063673: ITP: llama.cpp -- Inference of Meta's LLaMA model (and others) in pure C/C++ Petter Reinholdtsen <pere@hungry.com> - 2025-02-06 09:00 +0100
      Bug#1063673: ITP: llama.cpp -- Inference of Meta's LLaMA model (and others) in pure C/C++ "M. Zhou" <lumin@debian.org> - 2025-02-06 15:20 +0100
    Bug#1063673: ITP: llama.cpp -- Inference of Meta's LLaMA model (and others) in pure C/C++ Petter Reinholdtsen <pere@hungry.com> - 2025-02-06 09:30 +0100

csiph-web