Update issues on RHEL 9

Issue 2B - Brotil/VTS

It looks like the newly updated Brotli/VTS modules that Grommunio has released to fix Issue 2 in this thread have themselves created a problem. After installing this update, NGINX will no longer start.
The module files look to have been moved from /usr/share/nginx/modules to /usr/lib64/nginx/modules in this latest update, but the main config file /etc/nginx/nginx.conf is still attempting to load these modules from the old location.
In the old /usr/share/nginx/modules location there are now a couple of conf files that reference the new location for these modules, so I believe the entries in /etc/nginx/nginx.conf are now redundant. After commenting them out, NGINX starts up fine.

Thanks daniel. Did you experience the Python/SQLite issue (Issue 3 in the original post)? Apart from creating error logs in the "Gromox http" log it seemed to cause issues with ActiveSync sessions dropping and needing to reauthenticate every few hours, but has been fine since the workaround of adding my own libraries to LD_LIBRARY_PATH.
I suspect that one would be a bit more effort to fix properly as it would require either a code change or Grommunio to distribute their own build of Python with an updated version of SQLite.

daniel That sounds like a similar issue. Mine was with the UNIXEPOCH function that was added in SQLite 3.38. Yours seems to be with the CONCAT function that looks to have been added in SQLite 3.44. RHEL 9.5 uses SQLite 3.34 so neither of these functions are valid. Grommunio looks to have updated their code to rely on a version of SQLite not supported by RHEL and derivative distros.
Unfortunately, because SQLite is being called by Python, manually installing a later version of SQLite itself isn't sufficient; because the libraries Python uses are independent to the system version of SQLite and are compiled as part of Python itself. Messing with the system version of Python doesn't seem like a good idea given the number of things that depend on it, so the the workaround I used is as follows.

Download and build the latest version of SQLite from source:

curl https://www.sqlite.org/2024/sqlite-autoconf-3470200.tar.gz -o sqlite-autoconf-3470200.tar.gz
tar xvfz sqlite-autoconf-3470200.tar.gz
cd sqlite-autoconf-3470200
./configure
make

Copy the compiled libraries from the .libs subdirectory to one where they can be accessed by Grommunio:

mkdir /opt/libsqlite3
cp -P .libs/* /opt/libsqlite3
cp --remove-destination libsqlite3.la /opt/libsqlite3

You could also just cp .libs/* /opt/libsqlite3 but there are a few symlinks included that would result in duplicate files that way. I'm not sure if all of these files are required, but I copied them all to be on the safe side.

Create a file /opt/libsqlite3/env with content LD_LIBRARY_PATH=/opt/libsqlite3

Create an override for the gromox-http service with systemctl edit gromox-http.service and add the following between the start and end comments to have the service reference the newly created environment file.

[Service]
EnvironmentFile=/opt/libsqlite3/env

Save the override and restart gromox-http systemctl restart gromox-http. The service should now use the updated SQLite libraries created in /opt/libsqlite3.

This seems to work and I haven't yet encountered any issues within my use case (Outlook MAPI/HTTP, ActiveSync, Grommunio Web), but it does mean that any time an update is released for SQLite I'll have to go through the process to compile the new version all over again to keep it patched. I definitely put it into the category of a kludge, but it keeps my system stable for now. Hopefully Grommunio can work out a more permanent solution for the EL distros.

Issue 2B Returns - Brotil/VTS

My server installed the latest set of updates last night and NGINX died again.
It looks like the Grommunio updates have re-added the faulty /etc/nginx/nginx.conf instructions to load the Brotil/VTS modules from /usr/share/nginx/modules instead of the /usr/lib64/nginx/modules location Grommunio has moved them to.
I have removed the load_module lines again, and NGINX is working, but if Grommunio keeps adding the faulty lines back in, it will only last until the next update.

Big thanks to the team at Grommunio for resolving the majority of these issues.
The only one outstanding I believe now is the rogue load_module lines getting added to NGINX. I updated again last night and they got added back in, breaking NGINX once more. Deleting these lines (they're trying to load the modules from the original /usr/share/nginx/modules location but Grommunio moved them to /usr/lib64/nginx/modules) fixes it temporarily, but every time an update gets installs it re-adds the offending lines and breaks NGINX again.

load_module modules/ngx_http_brotli_static_module.so;
load_module modules/ngx_http_brotli_filter_module.so;
load_module modules/ngx_http_vhost_traffic_status_module.so;

FYI the NGINX config issue is still happening. I updated again last night and it re-added the bad config lines.

daniel just checked, and yes - DNF is broken on my machine after installing a version of SQLite from Grommunio. The offending package appears to be sqlite-libs. If I append --exclude=sqlite-libs then DNF commands succeed and I can install updates.