gromox 3.0

segro

hallo zusammen,

im Community Repo ist seit heute früh gromox 3.0 drin
Lange Release liste... https://github.com/grommunio/gromox/releases

War schon jemand so mutig?

auf EL9 ging danach das webmail nicht mehr, finde aber grad den Fehler nicht... Error 500
nginx error.log steht nix drin.

crpb

Auf meinem test-system (openSuSe) mit community geht es aber mehr als einloggen in admin und web hab ich nicht gemacht.

sinux

Juhu zusammen,
auch ein test-system (openSuSe) mit community repo
WebLogin User und admin geht, es läuft normal, Mail rein raus mit sync Termine aber habe es (noch) nicht im Business laufen.

ozy

Hatte vorher gromox 2.48.145.gc40fe13, bin heute morgen auf gromox 3.0 community umgestiegen.
Webmail einloggen geht, war ein bisschen langsam, aber hab auch gesehen das sqlite schema von EV-18 auf EV-23 updated wird, also kann ich mir vorstellen das erste login langsam war.
Outlook 2021 funktionniert auch gut.
Keine fehler in system logs.
Klingt alles sehr gut.

Mal sehen in den nächsten tage.

Mister2

Installed this just after 09:00 UK time today. No errors found as of yet, bar the CUI shows as nginx not running (if you don't reboot server after update - it is not enforced at end of update), but Admin GUI shows all services running OK and all services working as expected. Surprisingly even Files working normally. Suggest a server reboot is performed asap after update to clear the CUI nginx error, it goes away after the reboot.

As said above, server was slow after update because of sqlite updates from EV-18 to EV-23. This will take some time especially on large installations and server will run slower than normal until complete. Use journalctl -f to keep and eye on progress.

TGM

Nor after 3 weeks I would like to ask:
Is it usable? How much users do you have and how big is your installation?

We have an installation with round about 300 users and 700GB mail data. And we have a lot of problems with the current supported release version, for example a lot of core dumps which make the system not usable. The user see more error messages then they can use the system... And I am looking for a solution. Maybe it is a solution to update temporarily to the community version until the next supported release will be available.

DVCon

TGM And we have a lot of problems with the current supported release version, for example a lot of core dumps which make the system not usable. The user see more error messages then they can use the system... And I am looking for a solution.

Hi,
I wonder if I have the same issue with one of my installations (supported).
Is it like this with your installation? gromox-http crashes (or it does not look like it crashes). In http log I see endless entries like
sqlite3_step(/var/lib/gromox/user/xxx.xx/xxx/exmdb/exchange.sqlite3) "SELECT username, permission FROM permissions": file is not a database (26)
The user can't access his mailbox. And incoming mails go to queue/save.
Restarting http and zcore helps temporarily for one or two days. It seems that it affects mostly the same users (mailbox size > 10 GB, size of exchange.sqlite3 > 1 GB).

Mister2

@TGM I have not had any significant issues with the upgrade to Gromox 3. I think there were some users who had some issues with a particular update a week or 2 ago, but I saw the posts before I tried the update. Waited a day or so (while following the posts) and found the issue would not effect my installation so went ahead without issues.

My installations are for just a few users, and Community Edition, so my experience is not overly helpful to your somewhat larger system. If @WalterH reads you post above he might be able to give a bit more insight as he manages many large installations using the Supported Edition so may have more useful insight for you.

WalterH

DVCon file is not a database (26)

Try a SQLite recovery, https://docs.grommunio.com/kb/sqlite.html

SecureVibe

DVCon Hello,

we are currently facing the same issues. Did you find a solution to that problem?

DVCon

WalterH
The databases and mailboxes are OK.

WalterH

What I have seen with: gromox-3.0.133.x359f93b-lp156.20.1.x86_64.rpm are core dumps in midb, development stated this are Chinese spam mails. But the core dumps needs to be solved.

DVCon

@SecureVibe
Sadly no. I have this on two appliances currently.
Even if everything is OK when you restart gromox-hhtp, the databases are fine and you can re-import missed mails, you have act very carefully. I once lost 10GB of mailbox data, because the database files were inaccessible while gromox cleaner was running.

What's your environment? I'm running the appliances on HyperV.

ckd

With the latest version of Gromox you could try exporting the said mailbox and then importing it into a newly created mailbox. Perhaps the issue is gone afterwards. Just an idea. 🙂

https://community.grommunio.com/d/2459-backup-and-restore-with-gromox-exportimport/12

SecureVibe

ckd thank you for your hint. We are currently on 3.2 on the supported branch. However, after the last update from 2.48 to 3.2 broke more things than it fixed, we are a bit cautious to update. But we will do some testing on your test instance.

We also get lots of these error messages. Have you seen them before?:

sqlite3_exec(/var/lib/gromox/user/domain/user/exmdb/exchange.sqlite3) "BEGIN": cannot start a transaction within a transaction (1) at [<libgromox_common.so.0(_ZN6gromox11gx_sql_execEP7sqlite3PKcj+0xa0) [0x7f9446ce305d]><libgromox_common.so.0(_ZN6gromox13gx_sql_begin3ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEP7sqlite3NS_8txn_modeE+0x67) [0x7f9446ce3709]><libgxs_exmdb_provider.so.0(+0x7988b) [0x7f944696288b]><libgxs_exmdb_provider.so.0(+0x4f2ef) [0x7f94469382ef]><libgxs_exmdb_provider.so.0(+0x51119) [0x7f944693a119]><libgxs_exmdb_provider.so.0(+0x513b6) [0x7f944693a3b6]><libgxs_exmdb_provider.so.0(+0x52055) [0x7f944693b055]><libc.so.6(+0xa6f6c) [0x7f94456a6f6c]><libc.so.6(+0x12e388) [0x7f944572e388]>]

SecureVibe

ckd

so we tried the gromox-export on your test appliance with gromox v3.3 with a backup of our problematic mailbox.
The export file is only half the size of the actual mailbox size (without compression). And the import to another mailbox throws this error: exm: write_message RPC failed

The import works with the -D (delivery) flag. But then the folder structure is lost and only about half the emails are imported, because the export file is incomplete.

Have you experienced these issues as well?

DVCon

It doesn't always happen to the same mailbox.

SecureVibe

DVCon We are running on Proxmox. For us the issue is always the same mailbox that has a size of 12GB.
gromox-mbck doesnt show any errors. We have the same issue, that the whole mailbox is deleted when gromox-mbop -purge-datafiles is used on the mailbox. Luckly we had backups of the mailbox so no data was lost.

WalterH

I saw something like this with a customer. The gromox process couldn't access the SQLite database and kept reporting a database error, and the cleaner process then removed all files because they weren't recorded in the database, precisely because the database was inaccessible. I suspect a locking problem, but we couldn't prove it.

SecureVibe

WalterH thank you for your input. Thats what we suspected aswell. We also get these errors on the same mailbox:
sqlite_busy on /var/lib/gromox/user/domain/user/exmdb/exchange.sqlite3: held by unknown, take from exch/exmdb/store.cpp:320

Were you able to resolve this issue? Our server also has much higher CPU load and disk writes since the update to 3.2.

WalterH

SecureVibe /var/lib/gromox/user/domain/user/exmdb/exchange.sqlite3

Was sagt ein lsof /var/lib/gromox/user/domain/user/exmdb/exchange.sqlite3 ? Den Pfad natürlich richtig ausfüllen.
Dann versuchen alle Prozesse die das File offen haben zu beenden und dann den http restarten und nochmals testen.
Wenn das nicht hilft, Server rebooten und testen.

DVCon

@SecureVibe
OK, so on proxmox, too...
I see the exact same errors that you see.

I upgraded one of the problematic appliances to 3.3, because that solves an issue with Outlook calendars. But I guess it wont't help with the database locks.

SecureVibe

DVCon how did you update your appliances? Via the web-ui, the cli-ui or with the terminal? Could it be that using the web-ui for the update can cause problems?

WalterH

DVCon OK, so on proxmox, too...

The customer where we encountered this issue is also using Proxmox. However, we have almost all of our virtualizations on Proxmox and have only encountered this problem once, so I don't believe it is a Proxmox-specific issue.