Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.debian.maint.python > #15692
| From | Dirk Eddelbuettel <edd@debian.org> |
|---|---|
| Newsgroups | linux.debian.bugs.dist, linux.debian.maint.python, linux.debian.devel |
| Subject | Bug#970021: Seeking a small group to package Apache Arrow (was: Bug#970021: RFP: apache-arrow -- cross-language development platform for in-memory analytics) |
| Date | 2024-03-31 14:00 +0200 |
| Message-ID | <Io1Rf-310B-9@gated-at.bofh.it> (permalink) |
| References | (2 earlier) <InNuV-2S5d-3@gated-at.bofh.it> <Io1xT-30TQ-3@gated-at.bofh.it> <Io1xT-30TQ-1@gated-at.bofh.it> <ANrGG-4D0-13@gated-at.bofh.it> <Io1xT-30TQ-1@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
Cross-posted to 3 groups.
Julian, Arrow is a complicated and large package. We use it at work (where there is a fair amount of Python, also to Conda etc) and do have issues with more complex builds especially because it is 'data infrastructure' and can come in from different parts. I would recommend against packaging at old one -- we also have seen issues with different (py)arrow version biting. Have you seen https://github.com/apache/arrow-nanoarrow ? It works via the C API to Arrow which interchanges data via two void* to the the two structs for arrow array and schema -- and avoids linkage issue. (In user space the pyarrow or R arrow packages can still be used also interfacing via these.) I have been using it for R package bindings for some time and we plan to expand that (again, at work) -- as do others. It is already use by duckdb, by the Arrow 'ADBC' interfaces (which are generic in the ODBC/JDBC sense but for Arrow, and also by a python interface to snowflake. Dirk -- dirk.eddelbuettel.com | @eddelbuettel | edd@debian.org
Back to linux.debian.maint.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Seeking a small group to package Apache Arrow (was: Bug#970021: RFP: apache-arrow -- cross-language development platform for in-memory analytics) Julian Gilbey <jdg@debian.org> - 2024-03-25 19:20 +0100
Bug#970021: Seeking a small group to package Apache Arrow (was: Bug#970021: RFP: apache-arrow -- cross-language development platform for in-memory analytics) Rene Engelhard <rene@debian.org> - 2024-03-29 21:00 +0100
Bug#970021: Seeking a small group to package Apache Arrow (was: Bug#970021: RFP: apache-arrow -- cross-language development platform for in-memory analytics) Julian Gilbey <julian@d-and-j.net> - 2024-03-30 21:30 +0100
Bug#970021: Seeking a small group to package Apache Arrow (was: Bug#970021: RFP: apache-arrow -- cross-language development platform for in-memory analytics) Julian Gilbey <julian@d-and-j.net> - 2024-03-31 13:30 +0200
Bug#970021: Seeking a small group to package Apache Arrow (was: Bug#970021: RFP: apache-arrow -- cross-language development platform for in-memory analytics) Dirk Eddelbuettel <edd@debian.org> - 2024-03-31 14:00 +0200
Bug#970021: Seeking a small group to package Apache Arrow (was: Bug#970021: RFP: apache-arrow -- cross-language development platform for in-memory analytics) Richard Duivenvoorde <richard@duiv.nl> - 2024-04-06 09:52 +0200
csiph-web