Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.bugs.rc > #357931 > unrolled thread

Bug#1064824: node-d3: fails to export map and other functions

Back to article view | Back to linux.debian.bugs.rc

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Bug#1064824: node-d3: fails to export map and other functions Nilesh Patra <nilesh@iki.fi> - 2024-03-04 21:40 +0100
    Bug#1064824: node-d3: fails to export map and other functions Julian Gilbey <jdg@debian.org> - 2024-03-04 22:30 +0100
      Bug#1064824: node-d3: fails to export map and other functions Julian Gilbey <jdg@debian.org> - 2024-03-05 22:00 +0100

#357931 — Bug#1064824: node-d3: fails to export map and other functions

[Multipart message — attachments visible in raw view] — view raw

> This gives lots of differences still; stripping down to just the
> differences still has many, many differences: some new exports not in
> the original d3, and some lost exports; the list begins:
> $ diff -u /tmp/d3-npm.exports.trimmed /tmp/d3-debian.exports.trimmed
>
> +exports.Adder = Adder;
> -exports.bisect = bisectRight;
> +exports.bin = bin;
> +exports.bisect = bisect;
> +exports.bisectCenter = bisectCenter;
> +exports.blur2 = blur2;
> +exports.blur = blur;
> +exports.blurImage = blurImage;
> +exports.count = count;
> -exports.csvFormatRow = csvFormatRow;
> -exports.csvFormatValue = csvFormatValue;

$ cat /tmp/d3-debian.exports.trimmed | egrep --color '(bisectRight|csvFormatRow|csvFormatValue)'
exports.bisectRight = bisectRight;
exports.csvFormatRows = csvFormatRows;

Some of the diff entries are false positive -- it is just that entries are not identical across these
files and despite sorting them, you do not get the exact picture of the diff in exports.

However I did not find csvFormatValue and I dug a little bit into it. Seems this is sourced from node-d3-dsv
and exported with index.js: https://github.com/d3/d3/blob/v5.16.0/index.js

And checking on current unstable

| [/tmp/node-d3-dsv]$ git checkout debian/1.2.0+_1.2.3-1
| HEAD is now at 76229a9 releasing package node-d3-dsv version 1.2.0+~1.2.3-1
| [76229a9][/tmp/node-d3-dsv]$ grep -irn csvFormatValue
| types-d3-dsv/index.d.ts:195:// csvFormatValue(...) ========================================================================
| types-d3-dsv/index.d.ts:202:export function csvFormatValue(value: string): string;
| src/csv.js:11:export var csvFormatValue = csv.formatValue;
| src/index.js:2:export {csvParse, csvParseRows, csvFormat, csvFormatBody, csvFormatRows, csvFormatRow, csvFormatValue} from "./csv.js";

So this should work, weird. I wanted to dig in, as per the build log: https://buildd.debian.org/status/fetch.php?pkg=node-d3&arch=all&ver=5.16.0%2B%7Ecs5.28.10-1&stamp=1693715544&raw=0

| Selecting previously unselected package node-d3-dsv.
| Preparing to unpack .../317-node-d3-dsv_1.1.1-9_all.deb ...
| Unpacking node-d3-dsv (1.1.1-9) ...
| Selecting previously unselected packa

And

| [76229a9][/tmp/node-d3-dsv]$ git checkout debian/1.1.1-9       
| Previous HEAD position was 76229a9 releasing package node-d3-dsv version 1.2.0+~1.2.3-1
| HEAD is now at 84fbfce releasing package node-d3-dsv version 1.1.1-9
| [84fbfce][/tmp/node-d3-dsv]$ grep -irn csvFormatValue | wc -l
| 0

Which is why. Seems the versions of dev dependencies have not been appropriately tightened by the upstream
so we are running into weird surprises like these. Re-compiling node-d3 again now should fixup this export however.

:-(

These minor deltas in exports are more or less due to
version differences in different d3 plugins.

> ...
> Background to this: I'm trying to package a new package which provides
> a web server to visualise some data.  The package includes a few
> precompiled JavaScript libraries obtained from npmjs.com, and the
> server works fine with them.  But following Debian policy, I need to
> replace those with the source packages.  And the server then doesn't
> work.
>
> The JavaScript libraries which the package uses are: d3 v5.16.0,
> d3-tip, apparently v0.9.1, along with jQuery and bootstrap4.  I have
> replaced all of these with the versions in the corresponding Debian
> packages (and I've just uploaded a new version of d3-tip, thinking
> that that was the cause of the bug).
>
> When visiting the served web page, the console log gives the error
> message:
>
> Uncaught (in promise) TypeError: t.map is not a function
>    n http://localhost:8080/js/d3/d3-tip.min.js:1
>    main http://localhost:8080/js/index.js:848
>    async* http://localhost:8080/js/index.js:993
>
> (This has changed from the original bug report as the minimised new
> version of d3-tip has t.map instead of h.map.)
>
> d3-tip.js requires d3-collection, from which it calls a map function.
> I tried replacing d3-tip.min.js with the pre-packaged version rather
> than the (newly built) Debian version, but that did not help.  I
> reverted that change and instead replaced d3.v5.min.js (which is a
> copy of /usr/share/nodejs/d3/dist/d3.min.js) with the version provided
> by upstream, which is a verbatim copy of the npmjs.com d3.min.js, and
> the server then worked perfectly.  So this told me that it is the
> Debian compiled d3 which is not working correctly.

Julian, I am very confused by that wording - from what I could gauge, your
target package does not work with debian libs but it does work with npmjs, yes?

In that case linking your target package and listing the exact steps to
that error can help someone debug it in more detail as to what might be missing.

Best,
Nilesh

[toc] | [next] | [standalone]

#357936

Hi Nilesh,

On Tue, Mar 05, 2024 at 02:06:08AM +0530, Nilesh Patra wrote:
> > This gives lots of differences still; stripping down to just the
> > differences still has many, many differences: some new exports not in
> > the original d3, and some lost exports; the list begins:
> > $ diff -u /tmp/d3-npm.exports.trimmed /tmp/d3-debian.exports.trimmed
> >
> > +exports.Adder = Adder;
> > -exports.bisect = bisectRight;
> > +exports.bin = bin;
> > +exports.bisect = bisect;
> > +exports.bisectCenter = bisectCenter;
> > +exports.blur2 = blur2;
> > +exports.blur = blur;
> > +exports.blurImage = blurImage;
> > +exports.count = count;
> > -exports.csvFormatRow = csvFormatRow;
> > -exports.csvFormatValue = csvFormatValue;
> 
> $ cat /tmp/d3-debian.exports.trimmed | egrep --color '(bisectRight|csvFormatRow|csvFormatValue)'
> exports.bisectRight = bisectRight;
> exports.csvFormatRows = csvFormatRows;
> 
> Some of the diff entries are false positive -- it is just that entries are not identical across these
> files and despite sorting them, you do not get the exact picture of the diff in exports.

Well spotted, thanks!  I'll snip the discussion of csvFormatValue...

> [...]
> Which is why. Seems the versions of dev dependencies have not been appropriately tightened by the upstream
> so we are running into weird surprises like these. Re-compiling node-d3 again now should fixup this export however.

Great!

> These minor deltas in exports are more or less due to
> version differences in different d3 plugins.

> > ...
> > Background to this: I'm trying to package a new package which provides
> > a web server to visualise some data.  The package includes a few
> > precompiled JavaScript libraries obtained from npmjs.com, and the
> > server works fine with them.  But following Debian policy, I need to
> > replace those with the source packages.  And the server then doesn't
> > work.
> >
> > The JavaScript libraries which the package uses are: d3 v5.16.0,
> > d3-tip, apparently v0.9.1, along with jQuery and bootstrap4.  I have
> > replaced all of these with the versions in the corresponding Debian
> > packages (and I've just uploaded a new version of d3-tip, thinking
> > that that was the cause of the bug).
> >
> > When visiting the served web page, the console log gives the error
> > message:
> >
> > Uncaught (in promise) TypeError: t.map is not a function
> >    n http://localhost:8080/js/d3/d3-tip.min.js:1
> >    main http://localhost:8080/js/index.js:848
> >    async* http://localhost:8080/js/index.js:993
> >
> > (This has changed from the original bug report as the minimised new
> > version of d3-tip has t.map instead of h.map.)
> >
> > d3-tip.js requires d3-collection, from which it calls a map function.
> > I tried replacing d3-tip.min.js with the pre-packaged version rather
> > than the (newly built) Debian version, but that did not help.  I
> > reverted that change and instead replaced d3.v5.min.js (which is a
> > copy of /usr/share/nodejs/d3/dist/d3.min.js) with the version provided
> > by upstream, which is a verbatim copy of the npmjs.com d3.min.js, and
> > the server then worked perfectly.  So this told me that it is the
> > Debian compiled d3 which is not working correctly.
> 
> Julian, I am very confused by that wording - from what I could gauge, your
> target package does not work with debian libs but it does work with npmjs, yes?

I'm sorry I wasn't clear.

Yes, that's it: I'm trying to package taskflow for Debian (it's a
dependency of something else that I'm trying to package), and it
provides a profiler visualisation tool in the form of a webserver.

The upstream package (https://github.com/taskflow/taskflow) has
d3.v5.min.js and d3-tip.min.js included (in tfprof/js/d3), and these
are bitwise identical to the npmjs versions (versions 5.16.0 and
0.9.1).  The webserver works using these versions.  To satisfy Debian
policy, I replaced these with the versions found in libjs-d3-tip and
node-d3, but then the webserver fails to produce a usuable
visualisation.

My exploration, described above, was that "map" was not exported from
d3-collection, and that led me to explore whether this was a unique
missing export or not; I discovered that it was not.

> In that case linking your target package and listing the exact steps to
> that error can help someone debug it in more detail as to what might be missing.

I've just rebuilt node-d3 locally from (Debian) source on an unstable
schroot, and I think I may have found the source of the bug: the build
reads:

-----
   dh_auto_build --buildsystem=nodejs
No build command found, searching known files
No build command found, searching known files
Found debian/nodejs/d3-sankey/build
	cd ./d3-sankey && sh -ex ../debian/nodejs/d3-sankey/build
+ rollup -c

src/index.js → dist/d3-sankey.js...
created dist/d3-sankey.js in 120ms

src/index.js → dist/d3-sankey.min.js...
created dist/d3-sankey.min.js in 561ms
Found debian/nodejs/./build
	cd ./. && sh -ex debian/nodejs/./build
+ mkdir dist
+ json2module package.json
(node:1300749) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
+ rollup -c

index.js → dist/d3.node.js...
created dist/d3.node.js in 53ms

index.js → dist/d3.js...
(!) Conflicting re-exports
"index.js" re-exports "map" from both "../../../usr/share/nodejs/d3-array/src/index.js" and "../../../usr/share/nodejs/d3-collection/src/index.js" (will be ignored).
created dist/d3.js in 1.9s

index.js → dist/d3.min.js...
(!) Conflicting re-exports
"index.js" re-exports "map" from both "../../../usr/share/nodejs/d3-array/src/index.js" and "../../../usr/share/nodejs/d3-collection/src/index.js" (will be ignored).
created dist/d3.min.js in 4.2s
-----

So it's specifically "map" that is problematic, and I just happen to
have stumbled upon it: d3 v5 depends on d3-array version 1, but the
version of node-d3-array in unstable is 3.2.0+~cs5.0.6-2, and this is
causing the conflict.

I don't know the best way to fix this.  node-d3-array version was
upgraded from 1.2.4 to 3.x about two years ago, so d3 would have had
this bug since then, but I'm the first one to stumble upon it :-/

Perhaps we could package node-d3-array-1 (version 1.2.4) and have
node-d3 build-depends on that?

Best wishes,

   Julian

[toc] | [prev] | [next] | [standalone]

#357997

On Tue, Mar 05, 2024 at 05:39:00PM +0530, Nilesh Patra wrote:
> On Mon, Mar 04, 2024 at 09:18:01PM +0000, Julian Gilbey wrote:
> [...]
> > (!) Conflicting re-exports
> > "index.js" re-exports "map" from both "../../../usr/share/nodejs/d3-array/src/index.js" and "../../../usr/share/nodejs/d3-collection/src/index.js" (will be ignored).
> > created dist/d3.min.js in 4.2s
> > -----
> 
> I have pushed a commit to salsa that hopefully fixes this - can you please try with the same and see if that
> helps you somewhat?

That's perfect, thanks!  And such a simple solution :-) (Though I
don't pretend to understand how the patch works!)  The taskflow
webserver now works exactly as intended (as far as I can tell).

> > So it's specifically "map" that is problematic, and I just happen to
> > have stumbled upon it: d3 v5 depends on d3-array version 1, but the
> > version of node-d3-array in unstable is 3.2.0+~cs5.0.6-2, and this is
> > causing the conflict.
> > 
> > I don't know the best way to fix this.  node-d3-array version was
> > upgraded from 1.2.4 to 3.x about two years ago, so d3 would have had
> > this bug since then, but I'm the first one to stumble upon it :-/
> >
> > Perhaps we could package node-d3-array-1 (version 1.2.4) and have
> > node-d3 build-depends on that?
> 
> I tried to embed it and realised it is creating an unholy mess. I got it working eventually
> but it can open a can of worms sometime later. Maybe packaging the older version
> would be the way to go if my fix above does not work.
> 
> I've added yadd to the thread for more qualified advice.

It might be a good idea to do this anyway, but given that no-one else
has noticed a problem in > 2 years suggests it's not worth the effort.

Best wishes,

   Julian

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.bugs.rc

csiph-web