• Info

  • MIDB what are these databases for and how to regenerate the MIDB

Andy Wouldn't it be advantageous to store another menu item for such a process in the CUI, which triggers the process if necessary?

IMO definitely NO!
These fixes shouldn't be done out of the blue by clicking on a button in the webgui.

This should be a one-time task if you encounter problems with IMAP and i hope after all those changes in the last few weeks/months these problems are a thing of the past.

I will maybe get to it to extend Walters Script so one could just use it instead of doing all the steps manually(mostly for personal use) but other than that i don't expect that the Grommunio-Developers would integrate such a task in the Admin-Web.

    crpb

    grommunio-admin user query maildir --filter pop3_imap=True | sort > /tmp/mail_dirs.txt
    generates a sorted list.
    I need to test this and if its OK, I will change the step 4.
    Thank you.

    • crpb replied to this.

        WalterH

        grommunio-admin user query maildir --filter pop3_imap=True --sort maildir |grep -v '^$'

            crpb |grep -v '^$'

            why you added this command, please elaborate?

              This is just getting rid of any empty lines if there are any.

              • -v - --invert-match
              • ^ - Start of a line
              • $ - End of a line

                crpb there is a filter: --filter pop3_imap=True. Any object that fits this filter, can't have an empty mail directory - it is a valide mailbox or did I overlook something?

                • crpb replied to this.

                    WalterH
                    In a perfect world i guess you are correct with that assumption.
                    I did an grommunio-admin user modify ldapgroup@domain.tld --pop3-imap 1 which worked and then the result might look like this

                    root@grom-deb:~# grommunio-admin user query maildir --filter pop3_imap=True --sort maildir

/var/lib/gromox/user/0/1
/var/lib/gromox/user/0/2
...

                    So you might run into that "empty line" after all due to misconfiguration by the admin :-).

                    I then deleted the group and did an downsync with Enabled POP3/IMAP by default and it didn't enable it for the Group so it shouldn't happen in normal cases. The switch to that backend is relatively new anyhow so i guess @jschroeder 👋 will maybe fix that?

                    And with all that in mind: just forget the |grep -v... part 🙈 🙊 🙉

                      5 days later

                      Nicht schön aber selten...

                      • midb_repair.py

                        • Take a look at what would be executed
                          python3 midb_repair.py --all --verbose
                        • execute for real!
                          python3 midb_repair.py --all --verbose --no-dry-run

                      For all intents and purposes: UNTESTED!!!

                      EDIT: not python3.6 compatible (OpenSuse 15.4 default)
                      FIX: 15.4 zypper install -y python39 / 15.5 zypper install -y python311 (if missing)

                      2023.06.07: Fixed a few typos and cleaned up a bit.
                      First test-runs on a shared mailbox look good so far.

                      root@grom-deb:~/scripts/tools# ./midb_repair.py --verbose --user sharedmbx@domain.tld --no-dry-run
Namespace(all=True, user='sharedmbx@domain.tld', backup=False, recreate=True, dry_run=False, verbose=True)
Users
/var/lib/gromox/user/4/0        sharedmbx@domain.tld
recreate midbs
['systemctl', 'stop', 'gromox-midb', 'gromox-imap', 'gromox-pop3']
['gromox-mkmidb', '-vf', 'sharedmbx@domain.tld']
dbop_sqlite: Creating table "configurations"
dbop_sqlite: Creating table "folders"
dbop_sqlite: Creating table "messages"
dbop_sqlite: Creating table "mapping"
['systemctl', 'restart', 'gromox-midb', 'gromox-imap', 'gromox-pop3']
['/usr/libexec/gromox/cleaner', '-d', '/var/lib/gromox/user/4/0']
Cleaner: sharedmbx@domain.tld
Processing /var/lib/gromox/user/4/0/cid...
Purged 0 files (0B) from /var/lib/gromox/user/4/0/cid
Processing /var/lib/gromox/user/4/0/eml...
Purged 9250 files (1956MB) from /var/lib/gromox/user/4/0/eml
Processing /var/lib/gromox/user/4/0/ext...
Purged 9250 files (13MB) from /var/lib/gromox/user/4/0/ext
Running Trigger
1 mail directories for MIDB synchronization found.
Mailbox   1: /var/lib/gromox/user/4/0
[Closing connection]
                      9 days later

                      After some testing with different cache on client and whatnot...

                      My first tests were on Debian with the Community-Repo (2.9.95.38687c9-1+43.1) which looked pretty promising. To be certain i did an downgrade to 2.9.18.538f503-1+3.1 (because the actual Server runs gromox-2.9.18.538f503-lp155.3.1.x86_64), yet again purged the mailbox-data, copied it over, set permissions and restarted and reconnected.
                      Then i ran the script again and it didn't work quite as good.
                      There were actually many changes between those releases as far as i could make out in the git log.

                      All i'm trying to say is:

                      If you experience problems which might require to do this task then i recommend waiting for the "stable" release of Gromox >= 2.9.95 to yield the best results.

                      With the Community-Release you are already set - 2023.06.08

                      PS: I'm currently not a 100% sure if those problems are now gone for good.. @jengelh ?

                      For those who do IMAP-Based Backups of individual Mailboxes:
                      I tried this task already a while ago on only my personal Mailbox and it should be noted that Mailstore did archive quite a lot of mails a second time as the Headers changed.

                      a month later
                      4 months later

                      Hey,
                      since some days I have problems with IMAP access to one adress. I already tried to regenerate midb like described above. But it not helped.
                      I get in the log of gromox-midb some of this:
                      E-2403: mail_engine: timed out obtaining a reference on /var/lib/gromox/user/0/1

                      Any ideas?

                      5 days later

                      © 2020-2024 grommunio GmbH. All rights reserved. | https://grommunio.com | Data Protection | Legal notice