• Info
  • MIDB what are these databases for and how to regenerate the MIDB

Wouldn't it be advantageous to store another menu item for such a process in the CUI, which triggers the process if necessary?

  • crpb replied to this.

    Andy Wouldn't it be advantageous to store another menu item for such a process in the CUI, which triggers the process if necessary?

    IMO definitely NO!
    These fixes shouldn't be done out of the blue by clicking on a button in the webgui.

    This should be a one-time task if you encounter problems with IMAP and i hope after all those changes in the last few weeks/months these problems are a thing of the past.

    I will maybe get to it to extend Walters Script so one could just use it instead of doing all the steps manually(mostly for personal use) but other than that i don't expect that the Grommunio-Developers would integrate such a task in the Admin-Web.

    crpb

    grommunio-admin user query maildir --filter pop3_imap=True | sort > /tmp/mail_dirs.txt
    generates a sorted list.
    I need to test this and if its OK, I will change the step 4.
    Thank you.

    • crpb replied to this.

      WalterH

      grommunio-admin user query maildir --filter pop3_imap=True --sort maildir |grep -v '^$'

        crpb |grep -v '^$'

        why you added this command, please elaborate?

        This is just getting rid of any empty lines if there are any.

        • -v - --invert-match
        • ^ - Start of a line
        • $ - End of a line

          crpb there is a filter: --filter pop3_imap=True. Any object that fits this filter, can't have an empty mail directory - it is a valide mailbox or did I overlook something?

          • crpb replied to this.

            WalterH
            In a perfect world i guess you are correct with that assumption.
            I did an grommunio-admin user modify ldapgroup@domain.tld --pop3-imap 1 which worked and then the result might look like this

            root@grom-deb:~# grommunio-admin user query maildir --filter pop3_imap=True --sort maildir
            
            /var/lib/gromox/user/0/1
            /var/lib/gromox/user/0/2
            ...

            So you might run into that "empty line" after all due to misconfiguration by the admin :-).

            I then deleted the group and did an downsync with Enabled POP3/IMAP by default and it didn't enable it for the Group so it shouldn't happen in normal cases. The switch to that backend is relatively new anyhow so i guess @jschroeder πŸ‘‹ will maybe fix that?

            And with all that in mind: just forget the |grep -v... part πŸ™ˆ πŸ™Š πŸ™‰

            5 days later

            Nicht schΓΆn aber selten...

            • midb_repair.py

              • Take a look at what would be executed
                python3 midb_repair.py --all --verbose
              • execute for real!
                python3 midb_repair.py --all --verbose --no-dry-run

            For all intents and purposes: UNTESTED!!!

            EDIT: not python3.6 compatible (OpenSuse 15.4 default)
            FIX: 15.4 zypper install -y python39 / 15.5 zypper install -y python311 (if missing)

            2023.06.07: Fixed a few typos and cleaned up a bit.
            First test-runs on a shared mailbox look good so far.

            root@grom-deb:~/scripts/tools# ./midb_repair.py --verbose --user sharedmbx@domain.tld --no-dry-run
            Namespace(all=True, user='sharedmbx@domain.tld', backup=False, recreate=True, dry_run=False, verbose=True)
            Users
            /var/lib/gromox/user/4/0        sharedmbx@domain.tld
            recreate midbs
            ['systemctl', 'stop', 'gromox-midb', 'gromox-imap', 'gromox-pop3']
            ['gromox-mkmidb', '-vf', 'sharedmbx@domain.tld']
            dbop_sqlite: Creating table "configurations"
            dbop_sqlite: Creating table "folders"
            dbop_sqlite: Creating table "messages"
            dbop_sqlite: Creating table "mapping"
            ['systemctl', 'restart', 'gromox-midb', 'gromox-imap', 'gromox-pop3']
            ['/usr/libexec/gromox/cleaner', '-d', '/var/lib/gromox/user/4/0']
            Cleaner: sharedmbx@domain.tld
            Processing /var/lib/gromox/user/4/0/cid...
            Purged 0 files (0B) from /var/lib/gromox/user/4/0/cid
            Processing /var/lib/gromox/user/4/0/eml...
            Purged 9250 files (1956MB) from /var/lib/gromox/user/4/0/eml
            Processing /var/lib/gromox/user/4/0/ext...
            Purged 9250 files (13MB) from /var/lib/gromox/user/4/0/ext
            Running Trigger
            1 mail directories for MIDB synchronization found.
            Mailbox   1: /var/lib/gromox/user/4/0
            [Closing connection]
            9 days later

            After some testing with different cache on client and whatnot...

            My first tests were on Debian with the Community-Repo (2.9.95.38687c9-1+43.1) which looked pretty promising. To be certain i did an downgrade to 2.9.18.538f503-1+3.1 (because the actual Server runs gromox-2.9.18.538f503-lp155.3.1.x86_64), yet again purged the mailbox-data, copied it over, set permissions and restarted and reconnected.
            Then i ran the script again and it didn't work quite as good.
            There were actually many changes between those releases as far as i could make out in the git log.

            All i'm trying to say is:

            If you experience problems which might require to do this task then i recommend waiting for the "stable" release of Gromox >= 2.9.95 to yield the best results.

            With the Community-Release you are already set - 2023.06.08

            PS: I'm currently not a 100% sure if those problems are now gone for good.. @jengelh ?

            For those who do IMAP-Based Backups of individual Mailboxes:
            I tried this task already a while ago on only my personal Mailbox and it should be noted that Mailstore did archive quite a lot of mails a second time as the Headers changed.

            a month later
            4 months later

            Hey,
            since some days I have problems with IMAP access to one adress. I already tried to regenerate midb like described above. But it not helped.
            I get in the log of gromox-midb some of this:
            E-2403: mail_engine: timed out obtaining a reference on /var/lib/gromox/user/0/1

            Any ideas?

            5 days later

            Β© 2020-2024 grommunio GmbH. All rights reserved. | https://grommunio.com | Data Protection | Legal notice