Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.bugs.dist > #1260794 > unrolled thread

Bug#1085140: converting from mnesia to sqlite

Back to article view | Back to linux.debian.bugs.dist

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Bug#1085140: converting from mnesia to sqlite Johannes Schauer Marin Rodrigues <josch@debian.org> - 2025-09-10 08:20 +0200
    Bug#1085140: converting from mnesia to sqlite Johannes Schauer Marin Rodrigues <josch@debian.org> - 2025-09-10 08:30 +0200

#1260794 — Bug#1085140: converting from mnesia to sqlite

[Multipart message — attachments visible in raw view] — view raw

Hi Holger,

today I upgraded my ejabberd installation from Bookworm to Trixie and I think I
ran into the same problem. From journalctl:

Sep 10 07:31:13 02ejabberd sh[5446]: 2025-09-10 07:31:13.723030+02:00 [info] Loading configuration from /etc/ejabberd/ejabberd.yml
Sep 10 07:31:13 02ejabberd sh[5446]: 2025-09-10 07:31:13.843328+02:00 [warning] Module mod_mam is recommended for module mod_muc but is not found in the config
Sep 10 07:31:13 02ejabberd sh[5446]: 2025-09-10 07:31:13.965803+02:00 [info] Configuration loaded successfully
Sep 10 07:36:13 02ejabberd systemd[1]: ejabberd.service: start operation timed out. Terminating.
Sep 10 07:36:13 02ejabberd sh[5446]: 2025-09-10 07:36:13.609211+02:00 [notice] SIGTERM received - shutting down
Sep 10 07:36:13 02ejabberd sh[5446]: 2025-09-10 07:36:13.609490+02:00 [error] External eimp process (pid=5487) has terminated unexpectedly, restarting in a few seconds
Sep 10 07:36:13 02ejabberd sh[5446]: 2025-09-10 07:36:13.609701+02:00 [error] External eimp process (pid=5488) has terminated unexpectedly, restarting in a few seconds
Sep 10 07:36:13 02ejabberd sh[5446]: 2025-09-10 07:36:13.609876+02:00 [error] External eimp process (pid=5493) has terminated unexpectedly, restarting in a few seconds
Sep 10 07:36:13 02ejabberd sh[5446]: 2025-09-10 07:36:13.609804+02:00 [error] External eimp process (pid=5489) has terminated unexpectedly, restarting in a few seconds
Sep 10 07:36:13 02ejabberd sh[5446]: 2025-09-10 07:36:13.609922+02:00 [notice] SIGTERM received - shutting down
Sep 10 07:36:13 02ejabberd sh[5446]: 2025-09-10 07:36:13.609854+02:00 [error] External eimp process (pid=5491) has terminated unexpectedly, restarting in a few seconds
Sep 10 07:36:13 02ejabberd sh[5446]: 2025-09-10 07:36:13.609863+02:00 [error] External eimp process (pid=5492) has terminated unexpectedly, restarting in a few seconds
Sep 10 07:36:13 02ejabberd sh[5446]: 2025-09-10 07:36:13.609948+02:00 [error] External eimp process (pid=5494) has terminated unexpectedly, restarting in a few seconds
Sep 10 07:36:13 02ejabberd sh[5446]: 2025-09-10 07:36:13.609802+02:00 [error] External eimp process (pid=5490) has terminated unexpectedly, restarting in a few seconds
Sep 10 07:36:14 02ejabberd sh[5446]: {exit,{shutdown,{gen_server,call,[application_controller,{start_application,ejabberd,temporary},infinity]}},[{gen_server,call,3,[{file,"gen_server.erl"},{line,1222}]},{application_controller,call,2,[{file,"application_controller.erl"},{line,509}]},{application,enqueue_or_start_app,6,[{file,"application.erl"},{line,419}]},{application,enqueue_or_start,6,[{file,"application.erl"},{line,384}]},{application,ensure_all_started,3,[{file,"application.erl"},{line,359}]},{ejabberd,start,0,[{file,"ejabberd.erl"},{line,54}]},{init,start_it,1,[]},{init,start_em,1,[]}]}
Sep 10 07:36:15 02ejabberd systemd[1]: ejabberd.service: Failed with result 'timeout'.


All the file beam.smp is running at 100% CPU:

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
2399141 200102    20   0 6129212   2.7g  40256 S 111.6   8.6   4:09.78 beam.smp

I also started out with a very large archive_msg.DAT:

$ root@02ejabberd:~# ls -lha --sort=size /var/lib/ejabberd
total 1.5G
-rw-r--r--  1 ejabberd ejabberd 900M Sep 10 07:23 archive_msg.DAT
-rw-r--r--  1 ejabberd ejabberd 358M Sep 10 00:27 MnesiaCore.ejabberd@02ejabberd_1757_456828_906542
-rw-r--r--  1 ejabberd ejabberd 358M Sep 10 00:22 PREVIOUS.LOG
-rw-r--r--  1 ejabberd ejabberd 1.2M Sep 10 00:07 translations.cache
-rw-r-----  1 ejabberd ejabberd 713K Feb 11  2017 filename.txt
-rw-r-----  1 ejabberd ejabberd 508K Sep 10 00:22 caps_features.DAT
-rw-r-----  1 ejabberd ejabberd 428K Feb 16  2025 vcard.DAT
-rw-r--r--  1 ejabberd ejabberd 415K Sep 10 00:22 offline_msg.DAT
-rw-r--r--  1 ejabberd ejabberd 282K Nov  7  2024 MnesiaCore.ejabberd@02ejabberd_1731_13964_107817
-rw-r--r--  1 ejabberd ejabberd 282K Nov  7  2024 MnesiaCore.ejabberd@02ejabberd_1731_13510_476279
-rw-r--r--  1 ejabberd ejabberd 281K Nov  7  2024 MnesiaCore.ejabberd@02ejabberd_1731_13167_358081
-rw-r--r--  1 ejabberd ejabberd 281K Nov  7  2024 MnesiaCore.ejabberd@02ejabberd_1731_13097_172341
-rw-r--r--  1 ejabberd ejabberd 281K Nov  7  2024 MnesiaCore.ejabberd@02ejabberd_1731_12643_848046
-rw-r--r--  1 ejabberd ejabberd 281K Nov  7  2024 MnesiaCore.ejabberd@02ejabberd_1731_12189_804828
-rw-r--r--  1 ejabberd ejabberd 281K Nov  7  2024 MnesiaCore.ejabberd@02ejabberd_1731_11735_463660
-rw-r--r--  1 ejabberd ejabberd 281K Sep 10 00:22 pubsub_item.DAT

As advised, I removed (backed up) /var/lib/ejabberd/archive_msg* and
/var/lib/ejabberd/pubsub* but ejabberd still fails to start within 5 minutes
(the default timeout) with the journalctl log from above.

I don't mind loosing a bit more data (only the user database is really
important I think) and am looking for advice on how to get this service to
start after my upgrade to Trixie. If this is a different bug (I'm filing it
here because my archive_msg.DAT was large) then I'll open another issue. But at
this point I would appreciate more advice from you.

I now manually increased the timeout for ejabberd to start from 5 minutes to 2
hours and maybe this will fix itself, given a few more CPU cycles to spare?

> 2) Convert to SQL before upgrading (strongly recommended).
>
> I could help with either (e.g., in the ejabberd support channel), but 
> maybe that's out of scope here?

Once everything runs I'd greatly appreciate advice on how to convert my
installation to sqlite. I have a very small instance with only a handful of
users and thus thought that using mnesia would be fine. Looking at the database
sizes, maybe not the case. If you could point me to the guide on how to do the
conversion, I'd greatly appreciate it. I was not successful finding such
instruction in the web (with google and bing/duckduckgo).

Thanks!

cheers, josch

[toc] | [next] | [standalone]

#1260796

[Multipart message — attachments visible in raw view] — view raw

Quoting Johannes Schauer Marin Rodrigues (2025-09-10 08:10:16)
> As advised, I removed (backed up) /var/lib/ejabberd/archive_msg* and
> /var/lib/ejabberd/pubsub* but ejabberd still fails to start within 5 minutes
> (the default timeout) with the journalctl log from above.
> 
> I don't mind loosing a bit more data (only the user database is really
> important I think) and am looking for advice on how to get this service to
> start after my upgrade to Trixie. If this is a different bug (I'm filing it
> here because my archive_msg.DAT was large) then I'll open another issue. But at
> this point I would appreciate more advice from you.
> 
> I now manually increased the timeout for ejabberd to start from 5 minutes to 2
> hours and maybe this will fix itself, given a few more CPU cycles to spare?

Ah, it is the same issue. Though in my case, removing archive_msg.DAT was not
enough. Running "du -s /var/lib/ejabberd/* | sort -n" revealed that I had this:

1300288	/var/lib/ejabberd/upload

And it seems that the uploaded files are also part of the database?

Anyways, moving these out of the way now gives me a running server:

https://compliance.conversations.im/server/mister-muffin.de/

> > 2) Convert to SQL before upgrading (strongly recommended).
> >
> > I could help with either (e.g., in the ejabberd support channel), but 
> > maybe that's out of scope here?
> 
> Once everything runs I'd greatly appreciate advice on how to convert my
> installation to sqlite. I have a very small instance with only a handful of
> users and thus thought that using mnesia would be fine. Looking at the database
> sizes, maybe not the case. If you could point me to the guide on how to do the
> conversion, I'd greatly appreciate it. I was not successful finding such
> instruction in the web (with google and bing/duckduckgo).

I'd still appreciate some pointers for this so that I do not run into the same
issue the next time I upgrade.

Thank you so much for maintaining ejabberd in Debian! I have been using (and
upgrading) it for nearly a decade by now. :)

Thanks!

cheers, josch

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.bugs.dist

csiph-web