Quiescing exchange.sqlite3 for VMware VDP backups

grofor

We are looking for a way to stop write transactions to exchange.sqlite3 temporarily during backups made with VMware's VDP so that those backups will be fully consistent. One idea we had was to stop the grommunio services that write to that database. Which services write to the exchange.sqlite3 database and is it viable to stop and restart those services periodically?

crpb

lsfd -Q '(USER == "gromox") and (NAME =~ ".sqlite3")'

exchange.sqlite3 - http(8gx)
midb.sqlite3 - midb(8gx)

there is the command gromox-mbop -u a@b.c unload but that doesn't necessarily close it for good as described in gromox-mbop(8).

I would probably do the following

stop gromox-imap(if used at all)
stop gromox-pop(if used at all)
stop nginx (so no new user session can be established)
stop gromox-delivery-queue (which accepts mails from postfix)
run gromox-mbop foreach.here.mb unload (probably not needed at all with a stop of zcore but well..)
stop gromox-http
check if the .wal and .shm files have been removed as they are not in use anymore.
backup
start gromox services
start nginx

I used to restart the services to get the sqlite's written down, then take a lvm snapshot and backup from that but that is problematic if you take like 4+ backups a day so i stopped doing that restart stuff.
But if you only do that once a night or so i don't see any problems at all.

grofor

Thank you for the explanation. It is very helpful.

You mention stopping zcore, but there is no stop zcore command in your list. Should that command also be issued?

crpb

grofor

oh right, i meant the http...
was not sure myself while writing it and after re-reading the manpages of zcore/http/delivery/*** i must have had a brain freeze 😛.

Tho if you do all these jobs it would likely be easier to just power down, backup, power up the system as that can't take much longer i suspect.

Which will take the longest if you don't change it in the whole process is the /etc/default/grub:GRUB_TIMEOUT=10 and sticking with wickedd for the interface configuration also takes it's toll on boot time. I switched to systemd.networkd on the opensuse machines and a grub_timeout=2 and the whole bootprocess with the emulated bios are around ₁₅ seconds when i look on a couple of systemd with systemd-analyze.

IIRC you could configure that in the vcenter to do a shutdown/snapshot->backup/boot or not? This i would then even call "consistent".

grofor

We were trying to accomplish consistency using VMware Tools-managed freeze-thaw scripts local to the VM, which is quick and simple to do with MariaDB/MySQL and requires no configuration external to the VM. Now that we understand the complexity of using that method to achieve consistency with SQLite, we will have to consider just powering off the VM as you suggest.

Thank you for your suggestions.

MichaelN

grofor

I think you also could use the "VACCUUM INTO" command. As far as i know, it integrates the WAL and SHM Contents and optimize the database into a new file, ready for backup.

@crpb Do you think that will also work ?

grofor

MichaelN I believe VACUUM INTO works the way you suggest, it but it is not a solution to the problem we are trying to solve. We want to quiesce the existing database so that VMware VDP backups of the VM on which is is running will give us consistent backups of the database without requiring additional storage to hold a second copy of the database (both live and on our backup appliance). Powering off the VM, or stopping the services that write to the database, during backups still appear to be our two best alternatives.