Update supported Edition 2.45.174 - Rework after the update

WalterH

The supported edition 2.45.174 was released at the weekend.
An error correction requires manual rework.

WalterH

The most important changes in 2.45.174:

The SQLite schema has been increased to EV-18.
Version 2.45.174 contains an error correction for the ICS, therefore the ICS attributes must be checked and repaired if necessary.

SQLite Schema:
After the update, wait a few minutes for the databases to raise the schema to EV-18.

ICS error correction:
Then run the script g_cgkrepair.sh to check the ICS attributes. If errors are reported, run the script with parameter -f to start the repair.

If Outlook is used in caching mode, the .OST files must be deleted for all mailboxes where the script has repaired errors, otherwise the faulty ICS attributes will be synchronized from the .OST file back to the mailbox on the server.

The Community Edition is on version 2.46. It therefore has the same changes and requires the same rework.

The script g_cgkrepair.sh:

#!/bin/bash
#
# Check or repair all mailboxes with cgkrepair.
#
# Script to check or repair ICS properties in all mailboxes. In some cases
# flags like read/unread, forwarded, answered, categories etc. are not
# processed when these errors are present in a mailbox.
#
# First, run this script without parameters, only if you see messages like:
#
#   message 6f5824h [--P]
#   message 6f5829h [--P]
#   message 6f5847h [-NP]
#   message 6f590eh [-NP]
#   message 6f59a1h [-NP]
#   folder ah [--P]
#   folder 524466h [--P]
#
# Run the script with the -f parameter to fix the problems.
#
#
# (c) 2025 by Walter@Hofstaedtler.com and CRPB
#
# 28.04.2025 WH,   V 1.0, First release
# 05.06.2025 CRPB, V 1.1, Enhancements
# 14.06.2025 CRPB, V 1.2, Enhancements
# 15.06.2025 WH,   V 1.3, Enhancements
# 20.06.2025 WH,   V 1.4, Count the mailboxes with errors, improved formatting
# 20.06.2025 WH,   V 1.5, Improve the formatting
# 21.06.2025 WH,   V 1.6, Fixed some redirect errors and duplicate messages, 
#                         silence improved, count access errors
#
#
# Installation:
# 1. Place this script in /scripts/manual/ as g_cgkrepair.sh
# 2. Convert the script to Linux LF with:
#    dos2unix /scripts/manual/g_cgkrepair.sh
# 3. Make the script executable with:
#    chmod +x /scripts/manual/g_cgkrepair.sh
#
# Run the script and specify the required parameter, just -f.
#
#
CYA="\033[0;36m"; YEL="\033[0;33m"; BLU="\033[0;34m"; RED="\033[0;31m"; NORM="\033[0m"
log='/tmp/g_cgkrepair.log'
quiet=false
repair=false
silent=false
verbose=false
dry_run='-n'
count=0
error_dir=0
error_rep=0
errors=0
#
showhelp() {
  echo -e "${CYA}Checks or repairs all grommunio mailboxes on this system with cgkrepair.${NORM}"
  echo -e "  Use the ${YEL}-f${NORM} parameter to start the repair."
  echo -e "  Without -f the mailbox will only be checked (dry run)."
  echo -e "  Use ${YEL}-v${NORM} to be verbose, show test and repair results."
  echo -e "  Use ${CYA}-s${NORM} to only show defective mailboxes."
  echo -e "  Use ${RED}-q${NORM} to be quiet (the log will still be filled)."
  echo -e "  Use -l path/to/file to use another logfile than the default /tmp/g_ckgrepair.log."
  echo -e "  Use -e user@domain.tld to only run check/fix for one user.\n"
}
#
while getopts hfsqve:l: opts; do
    case "${opts}" in
        e) email="${OPTARG:?}"
             ;;
        f) repair=true
           ;;
        h) showhelp
           exit 0
           ;;
        l) if touch -c "$OPTARG" 2>/dev/null; then
               log="${OPTARG:?}"
            else
               exit 77
           fi
           ;;
        q) quiet=true
           ;;
        s) silent=true
           ;;
        v) verbose=true
           ;;
        ?) showhelp
           exit 1
           ;;
    esac
done
shift "$((OPTIND - 1))"

if $quiet; then
    # Redirects all output (stdout and stderr)
    exec 3>&1 4>&2 1>/dev/null 2>&4
fi
# Removes ANSI color escape codes and writes to the log file
logg() { sed -e 's/\x1b\[[0-9;]*m//g' >> "$log"; }

MSGT="Check"

if $repair; then
    echo -e "${YEL}-f is set, repair all mailboxes ...${NORM}"
    MSGT="${YEL}Repair"
    dry_run=""
fi
echo
# We append to an existing log
echo "Script $0 started at: $(date +%Y%m%d-%H%M%S)" >>"$log"
#
# If -e user@domain.tld was supplied
[ -n "$email" ] && mailboxes=$email
# Read all mailboxes in a variable
#[ -z "$email" ] && mailboxes=$(grommunio-admin user query username maildir | grep "/user/" | grep "@" | awk '{print $1}')
[ -z "$email" ] && mailboxes=$(gromox-mbop foreach.mb echo-username)
#
# Now iterate through all mailboxes
for mailbox in $mailboxes; do
    entries=
    output=
    maildir=$(gromox-mbop -u "$mailbox" echo-maildir)
    count=$((count +1))
    fnum=$(printf "%4d\n" $count)
    if ! [ -d "$maildir" ]; then
        # Count mail directory not found error
        error_dir=$((error_dir +1))
        $silent && echo -e "$MSGT mailbox # ${fnum}: ${CYA}$mailbox${NORM} ... " |tee >(logg)
        #$silent && echo -e "${RED}Error: No Maildir/Access: $mailbox${NORM}" >>/dev/stderr
        echo -e "${RED}Error: No Maildir/Access: $mailbox${NORM}" |tee >(logg)
        continue
    fi

    $silent || echo -e "$MSGT mailbox # ${fnum}: ${CYA}$mailbox${NORM} ... " |tee >(logg)

    output=$(/usr/libexec/gromox/cgkrepair $dry_run -e "$mailbox" | tee -a "$log")
    ret=$?
    if [[ $ret -ne 0 ]]; then
        # Count repair errors
        error_rep=$((error_rep +1))
        $silent && echo -e "$MSGT mailbox # ${count}: ${CYA}$mailbox${NORM} ... " |tee >(logg)
        echo -e "${RED}Error occurred while repairing $mailbox!${NORM}" |tee /dev/stderr |tee >(logg)
    fi
    if [ -n "$output" ]; then
        entries=$(grep -c 'message' <<< "$output")
        if [ -n "$entries" ]; then
            entries=${#output}
            # Count the mailboxes with errors
            errors=$((errors +1))
            $silent && echo -e "$MSGT mailbox # ${fnum}: ${CYA}$mailbox${NORM} ... " |tee >(logg)
            echo -ne "${RED}*****                 Found $entries problematic objects.${NORM}\n" |tee >(logg)
        fi
        if $verbose; then
            echo -e "${BLU}Verbose mode active. Dumping result:${NORM}" |tee >(logg)
            echo "$output"
        fi
    else
        $verbose && echo -e "No problematic objects detected.\n" |tee >(logg)
    fi
    #
done
fnum=$(printf "%4d\n" $count)
echo -e "       ${CYA}${fnum} mailboxes checked.${NORM}" |tee >(logg)
# Check errors
if [[ $errors -gt 0 ]]; then
    fnum=$(printf "%4d\n" $errors)
    echo -e "${RED}*****  ${fnum} mailboxes with errors were found!${NORM}" |tee >(logg)
    #$silent && echo -e "${RED}*****  ${fnum} mailboxes with errors were found!${NORM}" >>/dev/stderr
fi
# Directory not found errors
if [[ $error_dir -gt 0 ]]; then
    fnum=$(printf "%4d\n" $error_dir)
    echo -e "${RED}*****  ${fnum} directories not found!${NORM}" |tee >(logg)
    #$silent && echo -e "${RED}*****  ${fnum} directories not found!${NORM}" >>/dev/stderr
fi
# Repair errors
if [[ $error_rep -gt 0 ]]; then
    fnum=$(printf "%4d\n" $error_rep)
    echo -e "${RED}*****  ${fnum} mailboxes are not repaired!${NORM}" |tee >(logg)
    #$silent && echo -e "${RED}*****  ${fnum} mailboxes are not repaired!${NORM}" >>/dev/stderr
fi
echo -e "The log file is: $log"
$repair && echo -e "${CYA}Please note: Run the script until no more errors occur.${NORM}" |tee >(logg)
#
echo -e "Script $0 stopped at: $(date +%Y%m%d-%H%M%S)\n" >> "$log"
#
# eof
#

Edit on 23.07.2025: Please note, cgkrepair is no longer available, now you have to use gromox-mbop for repair, like:
gromox-mbop cgkreset-u <MAILADRESS>
and delete the Outlook .OST file, otherwise the errors will be synchronized back from Outlook.

Edit on 05.06.2025, CRPB has embellished and extended the script.
Edit on 15.06.2025, we extended the script, use -h for help.
Edit on 20.06.2025, Count the mailboxes with errors and improve the formatting.
Edit on 21.06.2025, Silent mode, removed duplicate messages, silence improved, count access errors.
Edit on 24.06.2025, See post 11!

PF_JHB

Hallo,

danke für den Hinweis.
Was genau wird denn da repariert?
Kann ich das Script mehrfach laufen lassen?

Habe 286 Mailboxen - einige geben o. g. Meldung aus, darunter sind auch einige Outlook-User. Diese würde ich gerne Stück für Stück abarbeiten. Ich gehe davon aus, dass nach einem Reparaturlauf einige Mailboxen wieder zum Server synchronisiert haben - daher müsste ich das ganze mehrfach laufen lassen - dürfte ja keine Probleme geben, oder?

Danke für die Infos

WalterH

Ja man kann das Script immer wieder laufen lassen.

WalterH

In grommunio there was an improvement: https://github.com/grommunio/gromox/commit/bac7ae5349d9125ab5004c0ea69b8c7f2ffbb533
you have to run the script again.

crpb

Yep, just wanted to note that aswell. It's not as many as before but still work to be done.

fwiw,

Get-Process -Name outlook -ErrorAction ignore | Stop-Process
Get-ChildItem "$Env:USERPROFILE\AppData\Local\Microsoft\Outlook" |Remove-Item -Recurse -Confirm:$false -Verbose
Start-Process outlook

WalterH

I updated the script, the improvements are: Count the mailboxes with errors and improve the formatting.

millilenium

Seit dem Update am Script erhalte ich leider folgende Fehlermeldung:

./g_cgkrepair.sh: line 123: syntax error near unexpected token `('
./g_cgkrepair.sh: line 123: `        echo "Error: No Maildir/Access: $mailbox" |tee /dev/stderr |tee >(logg)'

Finde aber keinen direkten Fehler - gibt es hier evtl. ein Update?

WalterH

Ich habe das Script oben nochmals hineinkopiert, das Script funktioniert auf ca. 10 Systemen.

Was man versuchen könnte ist, Zeile 113 zu enablen und Zeile 114 zu disablen.

Aktuell:

# Read all mailboxes in a variable
#[ -z "$email" ] && mailboxes=$(grommunio-admin user query username maildir | grep "/user/" | grep "@" | awk '{print $1}')
[ -z "$email" ] && mailboxes=$(gromox-mbop foreach.mb echo-username)

Ändern:

# Read all mailboxes in a variable
[ -z "$email" ] && mailboxes=$(grommunio-admin user query username maildir | grep "/user/" | grep "@" | awk '{print $1}')
#[ -z "$email" ] && mailboxes=$(gromox-mbop foreach.mb echo-username)

Wenn das hilft, wird eine Mailbox nicht gefunden. Müsste man sich dann ansehen.

millilenium

WalterH

Danke, jetzt klappt es!

WalterH

If you read these 2 commits (see below), you can see that this script will soon no longer work. First in the Community Edition and later in the Supported Edition. I will provide a new script as soon as possible, but currently no ETA.

https://github.com/grommunio/gromox/commit/3fb2a18b52fafaa14e9a013e34cad7cf0fc90621
https://github.com/grommunio/gromox/commit/2adcde037fbacdc12bb5019f61dd4c99060a33d5

crpb

WalterH If you read these 2 commits (see below), you can see that this script will soon no longer work.

🤣

KnoJoh

Ich nutze aktuell noch gromox < 2.45.174. Kann ich das update mittlerweile ohne manuelle Nacharbeit durchführen oder sind hier weitere Schritte notwendig? Ich finde dazu leider nichts weiter als diesen Beitrag hier.

Danke Euch!

WalterH

Updaten kann man immer und wenn einem die möglichen ICS Fehler egal sind, dann musste man nie nacharbeiten.
Mit dem aktuellen Update gibt es das Tool cgkrepair nicht mehr und dieses Script funktioniert nicht mehr.

KnoJoh

Danke für die schnelle Rückmeldung.
Ich kann das aber nicht einordnen. Wie wirken sich ICS Fehler aus? Mir fehlt da das KnowHow um es abzuschätzen.
Und wenn das ein Problem darstellt, haben doch auch zahlende Kunden ein Problem - wird es dafür zukünftig eine Lösung geben? Ich würde mir ungern das System durch Unwissenheit "zerschießen"

Danke!

WalterH

KnoJoh ICS Fehler zeigen mehrer Effekte, z.B. in shard Mailboxen werden "Weitergeleitet" oder die "Kategorien" und andere Attribute nicht für alle User richtig angezeigt.

Das Problem können alle Mailboxen haben, deshalb gibt es diesen Artikel. Die Lösung ist die betroffene Mailbox zu reparieren und die Outlook .OST Datei zu löschen.