Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #80170 > unrolled thread

Python extension using a C library with one 'hello' function

Back to article view | Back to comp.lang.python

Contents

  Python extension using a C library with one 'hello' function Veek M <vek.m1234@gmail.com> - 2014-11-04 16:22 +0630
    Re: Python extension using a C library with one 'hello' function Chris Angelico <rosuav@gmail.com> - 2014-11-05 01:57 +1100
    Re: Python extension using a C library with one 'hello' function Søren <sd@syntonetic.com> - 2014-11-04 13:11 +0100
      Re: Python extension using a C library with one 'hello' function Veek M <vek.m1234@gmail.com> - 2014-11-04 21:46 +0630
    Re: Python extension using a C library with one 'hello' function Jason Swails <jason.swails@gmail.com> - 2014-11-04 10:30 -0500
      Re: Python extension using a C library with one 'hello' function Veek M <vek.m1234@gmail.com> - 2014-11-04 21:45 +0630
        Re: Python extension using a C library with one 'hello' function Jason Swails <jason.swails@gmail.com> - 2014-11-04 11:28 -0500
          Re: Python extension using a C library with one 'hello' function Veek M <vek.m1234@gmail.com> - 2014-11-04 22:13 +0630
          Re: Python extension using a C library with one 'hello' function Veek M <vek.m1234@gmail.com> - 2014-11-04 22:39 +0630
            Re: Python extension using a C library with one 'hello' function Jason Swails <jason.swails@gmail.com> - 2014-11-04 12:28 -0500
      Re: Python extension using a C library with one 'hello' function Veek M <vek.m1234@gmail.com> - 2014-11-04 23:03 +0630
        Re: Python extension using a C library with one 'hello' function Jason Swails <jason.swails@gmail.com> - 2014-11-04 13:28 -0500

#80170 — Python extension using a C library with one 'hello' function

https://github.com/Veek/Python/tree/master/junk/hello
doesn't work.
I have:
hello.c which contains: int hello(void);
hello.h

To wrap that up, i have:
hello.py -> _hello (c extension) -> pyhello.c -> method py_hello()

People using this will do:
python3.2>> import hello
python3.2>> hello.hello()

It doesn't compile/work. 

deathstar> python setup.py build_ext --inplace
running build_ext
building '_hello' extension
creating build
creating build/temp.linux-x86_64-3.2
gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -
D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -
Werror=format-security -fPIC -I/usr/include/python3.2mu -c pyhello.c -o 
build/temp.linux-x86_64-3.2/pyhello.o
pyhello.c:15:6: warning: character constant too long for its type [enabled 
by default]
pyhello.c:15:5: warning: initialization makes pointer from integer without a 
cast [enabled by default]
pyhello.c:15:5: warning: (near initialization for 'hellomethods[0].ml_name') 
[enabled by default]
pyhello.c:15:5: warning: initialization from incompatible pointer type 
[enabled by default]
pyhello.c:15:5: warning: (near initialization for 'hellomethods[0].ml_meth') 
[enabled by default]
pyhello.c:15:5: warning: initialization makes integer from pointer without a 
cast [enabled by default]
pyhello.c:15:5: warning: (near initialization for 
'hellomethods[0].ml_flags') [enabled by default]
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro 
build/temp.linux-x86_64-3.2/pyhello.o -o 
/root/github/junk/hello/_hello.cpython-32mu.so


>>> import hello
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "hello.py", line 1, in <module>
    from _hello import *
ImportError: ./_hello.cpython-32mu.so: undefined symbol: hello
>>> 

[toc] | [next] | [standalone]

#80189

On Tue, Nov 4, 2014 at 8:52 PM, Veek M <vek.m1234@gmail.com> wrote:
> https://github.com/Veek/Python/tree/master/junk/hello
> doesn't work.
> I have:
> hello.c which contains: int hello(void);
> hello.h
>
> To wrap that up, i have:
> hello.py -> _hello (c extension) -> pyhello.c -> method py_hello()
>
> People using this will do:
> python3.2>> import hello
> python3.2>> hello.hello()
>
> It doesn't compile/work.
> [details chomped]

Have you considered using Cython? It'll likely make your job a LOT easier.

ChrisA

[toc] | [prev] | [next] | [standalone]

#80194

I'm not sure if it fits your needs, but we are very happy with calling c 
libs directly from python using ctypes:

https://docs.python.org/2/library/ctypes.html
It requires a few extra lines in Python to handle the parameter and 
return types.

import ctypes
result = ctypes.windll.Hello.hello()

[toc] | [prev] | [next] | [standalone]

#80200

Søren wrote:

> import ctypes
Hi, yeah i kind of liked it - still reading the docs though, Beazley has the 
Python.h solution so I though I'd try that first.

[toc] | [prev] | [next] | [standalone]

#80195

On Tue, 2014-11-04 at 16:22 +0630, Veek M wrote:
> https://github.com/Veek/Python/tree/master/junk/hello
> doesn't work.
> I have:
> hello.c which contains: int hello(void);
> hello.h
> 
> To wrap that up, i have:
> hello.py -> _hello (c extension) -> pyhello.c -> method py_hello()
> 
> People using this will do:
> python3.2>> import hello
> python3.2>> hello.hello()
> 
> It doesn't compile/work. 
> 
> deathstar> python setup.py build_ext --inplace
> running build_ext
> building '_hello' extension
> creating build
> creating build/temp.linux-x86_64-3.2
> gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -
> D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -
> Werror=format-security -fPIC -I/usr/include/python3.2mu -c pyhello.c -o 
> build/temp.linux-x86_64-3.2/pyhello.o
> pyhello.c:15:6: warning: character constant too long for its type [enabled 
> by default]
> pyhello.c:15:5: warning: initialization makes pointer from integer without a 
> cast [enabled by default]
> pyhello.c:15:5: warning: (near initialization for 'hellomethods[0].ml_name') 
> [enabled by default]
> pyhello.c:15:5: warning: initialization from incompatible pointer type 
> [enabled by default]
> pyhello.c:15:5: warning: (near initialization for 'hellomethods[0].ml_meth') 
> [enabled by default]
> pyhello.c:15:5: warning: initialization makes integer from pointer without a 
> cast [enabled by default]
> pyhello.c:15:5: warning: (near initialization for 
> 'hellomethods[0].ml_flags') [enabled by default]
> gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro 
> build/temp.linux-x86_64-3.2/pyhello.o -o 
> /root/github/junk/hello/_hello.cpython-32mu.so
> 
> 
> >>> import hello
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "hello.py", line 1, in <module>
>     from _hello import *
> ImportError: ./_hello.cpython-32mu.so: undefined symbol: hello

When I try your code, I get the error:

ImportError: dynamic module does not define init function (PyInit__hello)

There were a couple other problems as well.  Like 'hello' in
hellomethods instead of "hello" (note the double quotes).  Also, NULL is
no longer acceptable as a METH_XXXARGS replacement, you need to set it
to METH_NOARGS (or MET_VARARGS if you plan on accepting arguments).

I've found that you also need a NULL sentinel in the hellomethods array
to avoid segfaults on my Linux box.

After fixing these problems, you still need to add "hello.c" to the list
of sources in setup.py to make sure that module is built.

I've submitted a PR to your github repo showing you the changes
necessary to get your module working on my computer.

Hope this helps,
Jason

[toc] | [prev] | [next] | [standalone]

#80199

Jason Swails wrote:

> I've submitted a PR to your github repo showing you the changes
> necessary to get your module working on my computer.

Segfaults :p which is an improvement :)

open("./_hello.cpython-32mu.so", O_RDONLY) = 5
read(5, 
"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300\7\0\0\0\0\0\0"..., 832) 
= 832
fstat(5, {st_mode=S_IFREG|0755, st_size=19217, ...}) = 0
getcwd("/root/github/junk/hello", 128)  = 24
mmap(NULL, 2101544, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 
0x7f65daea5000
mprotect(0x7f65daea6000, 2093056, PROT_NONE) = 0
mmap(0x7f65db0a5000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|
MAP_DENYWRITE, 5, 0) = 0x7f65db0a5000
close(5)                                = 0
mprotect(0x7f65db0a5000, 4096, PROT_READ) = 0
--- SIGSEGV (Segmentation fault) @ 0 (0) ---

[toc] | [prev] | [next] | [standalone]

#80201

On Tue, 2014-11-04 at 21:45 +0630, Veek M wrote:
> Jason Swails wrote:
> 
> > I've submitted a PR to your github repo showing you the changes
> > necessary to get your module working on my computer.
> 
> Segfaults :p which is an improvement :)

What operating system are you running this on?  It works fine for me on
Linux:

bash$ ls
hello.c  hello.h  hello.py  pyhello.c  setup.py
bash$ python3.4 setup.py build_ext --inplace
running build_ext
building '_hello' extension
creating build
creating build/temp.linux-x86_64-3.4
x86_64-pc-linux-gnu-gcc -pthread -fPIC -I/usr/include/python3.4 -c pyhello.c -o build/temp.linux-x86_64-3.4/pyhello.o
pyhello.c:15:5: warning: initialization from incompatible pointer type [enabled by default]
     {"hello", py_hello, METH_NOARGS, py_hello_doc},
     ^
pyhello.c:15:5: warning: (near initialization for ‘hellomethods[0].ml_meth’) [enabled by default]
x86_64-pc-linux-gnu-gcc -pthread -fPIC -I/usr/include/python3.4 -c hello.c -o build/temp.linux-x86_64-3.4/hello.o
x86_64-pc-linux-gnu-gcc -pthread -shared build/temp.linux-x86_64-3.4/pyhello.o build/temp.linux-x86_64-3.4/hello.o -L/usr/lib64 -lpython3.4 -o /home/swails/BugHunter/CAPI/Python/junk/hello/_hello.cpython-34.so
bash$ python3.4
Python 3.4.1 (default, Aug 24 2014, 10:04:41) 
[GCC 4.7.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import hello
>>> hello.hello()
hello world
0

You can get rid of the warning by casting py_hello to (PyCFunction)...
maybe that's causing your segfault?

All the best,
Jason

[toc] | [prev] | [next] | [standalone]

#80202

Jason Swails wrote:

> What operating system are you running this on?  It works fine for me on
> Linux:
Wheezy Debian, Linux deathstar 3.2.0-4-amd64 #1 SMP Debian 3.2.60-1+deb7u3 
x86_64 GNU/Linux
gcc (Debian 4.7.2-5) 4.7.2
Python 3.2.3

I ran it through gdb - not very useful:
(gdb) bt
#0  0x000000000055e1bd in PyModule_Create2 ()
#1  0x000000000043a830 in _PyImport_LoadDynamicModule ()
#2  0x0000000000476d84 in ?? ()
#3  0x00000000004aa3f9 in ?? ()
#4  0x0000000000477010 in ?? ()
#5  0x000000000048ee8d in ?? ()
#6  0x00000000004a833b in PyObject_Call ()
#7  0x00000000004b88c9 in PyEval_EvalFrameEx ()
#8  0x00000000004cdee7 in PyEval_EvalCodeEx ()

strace -f python3.2
munmap(0x7f215d4b9000, 4096)            = 0
stat("hello.py", {st_mode=S_IFREG|0644, st_size=22, ...}) = 0
stat("_hello", 0x7fff9bd38110)          = -1 ENOENT (No such file or 
directory)
stat("_hello.cpython-32mu.so", {st_mode=S_IFREG|0755, st_size=19217, ...}) = 
0
open("_hello.cpython-32mu.so", O_RDONLY) = 4
fstat(4, {st_mode=S_IFREG|0755, st_size=19217, ...}) = 0
open("./_hello.cpython-32mu.so", O_RDONLY) = 5
read(5, 
"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300\7\0\0\0\0\0\0"..., 832) 
= 832
fstat(5, {st_mode=S_IFREG|0755, st_size=19217, ...}) = 0
getcwd("/root/github/junk/hello", 128)  = 24
mmap(NULL, 2101544, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 
0x7f215b71c000
mprotect(0x7f215b71d000, 2093056, PROT_NONE) = 0
mmap(0x7f215b91c000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|
MAP_DENYWRITE, 5, 0) = 0x7f215b91c000
close(5)                                = 0
mprotect(0x7f215b91c000, 4096, PROT_READ) = 0
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
+++ killed by SIGSEGV +++
Segmentation fault

[toc] | [prev] | [next] | [standalone]

#80205

static PyMethodDef hellomethods[] = {
    {"hello", py_hello, METH_VARARGS, py_hello_doc},
    {NULL, NULL, 0, NULL},
};

It's basically the METH_VARARGS field that's giving the problem. Switching 
it to NULL gives, 
SystemError: Bad call flags in PyCFunction_Call. METH_OLDARGS is no longer 
supported!

and METH_NOARGS doesn't work in 3.2

[toc] | [prev] | [next] | [standalone]

#80209

[Multipart message — attachments visible in raw view] — view raw

On Tue, Nov 4, 2014 at 11:09 AM, Veek M <vek.m1234@gmail.com> wrote:

> static PyMethodDef hellomethods[] = {
>     {"hello", py_hello, METH_VARARGS, py_hello_doc},
>     {NULL, NULL, 0, NULL},
> };
>
> It's basically the METH_VARARGS field that's giving the problem. Switching
> it to NULL gives,
> SystemError: Bad call flags in PyCFunction_Call. METH_OLDARGS is no longer
> supported!
>

​Yes, I got that problem too, which is why I switched it to METH_NOARGS.
​

> and METH_NOARGS doesn't work in 3.2
>

I
​t does for me:

​
Python 3.2.5 (default, Aug 24 2014, 10:06:23)
[GCC 4.7.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import hello
>>> hello.hello()
hello world
0
>>>

​As you can see -- this is a Python 3.2 built with GCC 4.7 (On Gentoo
Linux).  It also works on Python 3.1 and 3.0 (but obviously doesn't work
for Python 2.X).  I can't tell why you're having so many problems...​  Try
doing a "git clean -fxd" to make sure you don't have leftover files lying
around somewhere that are causing grief.

Also, you need to add "-g" to the compiler arguments to make sure you build
with debug symbols if you want a meaningful traceback.

Good luck,
Jason

-- 
Jason M. Swails
BioMaPS,
Rutgers University
Postdoctoral Researcher

[toc] | [prev] | [next] | [standalone]

#80208

okay got it working - thanks Jason! The 3.2 docs are slightly different.

[toc] | [prev] | [next] | [standalone]

#80215

On Tue, 2014-11-04 at 23:03 +0630, Veek M wrote:
> okay got it working - thanks Jason! The 3.2 docs are slightly different.

What did you need to do to get it working?

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web