• General
  • grommunio-sync how to handle sync-issues?

Hi,

we experience some problems with syncing of Mail/Calendar-Items, Loops or whatever they might be and i am trying to determine the best way of resolving these issues and anything what might happen in the future.

Mobile-Device-Management

When i remove a Device in either the Web-UI or Admin-UI the Device wouldn't be listed in the Web-UI anymore but the Deviced ID is still listed in the Admin-UI without any other known facts.

  • Shouldn't those devices be removed from the Admin-UI aswell? I tried Refresh/Relog/Reboot in case of "Browser-Cache-Issues" :-).

  • Is there a way to completely remove a Device from the System so any lingering state would be removed/purged and on the next connection from the Device it would have to do an resync? Something alike z-push's z-push-admin -a remove -d DEVICEID

I also used z-push-admin just to do some Maintenance like cleaning up old Devices from the Database with something like this

#Remove all devices which lastsync-date is older 60 days
z-push-admin -a lastsync --days-old 60 |head -n -1 |tail -n +6 |awk '{print $1}' |xargs -n 1 z-push-admin -a remove -d
  • Is there any similiar command i could use to remove old Devices from the CLI? I couldn't find anything in grommunio-admin :P. I probably have to do some jq-Magic on the Redis-Cache?

Redis-Cache

As i was trying to determine where to look i found the Redis-Cache for which i added a little function to my ENV just to get a quick overview - WIP...

# REDIS-CACHE Liste Grommunio-Sync
grommunio_sync_rediscache() { 
  for key in $(redis-cli KEYS 'grommunio-sync*'); do
    type=$(redis-cli type "$key")
    printf "%s\n" "$key $type"
    if [[ "$type" = "hash" ]]; then
      hashkeys=$(redis-cli hkeys "$key")
      printf "\t%s\n" $hashkeys
    fi
  done
}
  • Is it safe to remove Devices from the cache?

  • Are their any states saved for the Devices in /var/lib/gromox/user/N/N i should be aware of?

  • How should i interpret KEYS in grommunio-sync:loopdetection? Those will be removed automagicallly after they have been dealt with?

  • Could i just delete ALL Redis-Entries with something like redis-cli $(redis-cli KEYS grommunio-sync*) to remove all states so any Device would have to do a complete Resync?

    Hi crpb,

    it would be helpful if you would describe what problems with the calendar items you are experiencing.

    crpb When i remove a Device in either the Web-UI or Admin-UI the Device wouldn't be listed in the Web-UI anymore but the Deviced ID is still listed in the Admin-UI without any other known facts.

    Shouldn't those devices be removed from the Admin-UI aswell? I tried Refresh/Relog/Reboot in case of "Browser-Cache-Issues" :-).

    Yes, they actually should be removed in the Admin-UI as well. We'll look into it. You have to be aware though, that simply removing device from a list doesn't prevent it from syncing again if the account is still configured on the device.

    Is there a way to completely remove a Device from the System so any lingering state would be removed/purged and on the next connection from the Device it would have to do an resync? Something alike z-push's z-push-admin -a remove -d DEVICEID

    That's what "Remove Device" in Mobile Devices in grommunio-web or Admin-UI actually do. They remove the state folder and force the device to resync.

    crpb As i was trying to determine where to look i found the Redis-Cache for which i added a little function to my ENV just to get a quick overview - WIP...

    First of all, grommunio-sync uses redis for interprocess communication states aka volatile states. It doesn't contain the actual sync state of the device which is saved in the user's store (sqlite).

    crpb Is it safe to remove Devices from the cache?

    What cache exactly do you mean? We do not recommend to perform any manipulation directly in redis as it might affect the performance of the system. If you remove some or even all keys, they will be rebuild again with time. There are fallbacks in the code if the data in redis is not available, so the sync itself should work, it just might be slower.

    crpb Are their any states saved for the Devices in /var/lib/gromox/user/N/N i should be aware of?

    grommunio-sync itself doesn't save any state information on the file system directly. Without going into details, technically all device sync states are under /var/lib/gromox/user/N/N as grommunio-sync saves them in the user's store which in turn is a sqlite DB.

    crpb How should i interpret KEYS in grommunio-sync:loopdetection? Those will be removed automagicallly after they have been dealt with?

    Why do you need to interpret those keys? grommunio-sync will add, update or remove them if necessary.

    The code block you linked to removes old search folders, so that they don't pollute the system unnecessarily.

    crpb Could i just delete ALL Redis-Entries with something like redis-cli $(redis-cli KEYS grommunio-sync*) to remove all states so any Device would have to do a complete Resync?

    You could delete all grommunio-sync related redis entries, but it won't force the resync of the devices as the sync states are saved in the user's store and not in redis.

    • crpb replied to this.
      5 days later

      Hi Andreas,

      andreaslang it would be helpful if you would describe what problems with the calendar items you are experiencing.

      iphone

      for instance, if a User with an iPhone creates a new Calendar-Entry, the new Appointment gets created up to 30 times with the same Information.

      android

      if i create a appointment in my Android 11 - Google Calendar (2022.22.1-452027150) and add a participant the Appointment will be created twice in my own Calendar.

      • Seems that if i add a participant, my own account is added in the list and therefore the original Appointment + The Appointment for me as a participant is created.

      i also provided some logs Ticket# 81331

      Update eingespielt

      gromi:~ # rpm -qa |grep -E ^grom |sort
      grommunio-admin-api-1.7.31.9b395f7-lp153.121.1.noarch
      grommunio-admin-api-bash-completion-1.7.31.9b395f7-lp153.121.1.noarch
      grommunio-admin-common-6.cb985db-lp153.9.1.noarch
      grommunio-admin-web-2.3.0.10.1fb197f-lp153.69.1.noarch
      grommunio-antispam-3.1-lp153.3.2.x86_64
      grommunio-chat-6.2.1-lp153.15.1.x86_64
      grommunio-common-6.2dbc6d3-lp153.13.1.x86_64
      grommunio-cui-1.0.7.edc336c-lp153.11.1.noarch
      grommunio-dav-1.0.7.9cee8ba-lp153.1.1.noarch
      grommunio-dbconf-1.0.1.6cec9d1-lp153.2.1.x86_64
      grommunio-error-pages-1.0.6.9c50afb-lp153.7.1.noarch
      grommunio-imapsync-2.200-lp153.4.1.noarch
      grommunio-index-0.1.11.8787c3c-lp153.16.1.x86_64
      grommunio-release-2022.05.1-lp153.1.1.x86_64
      grommunio-setup-1.0.61.3e0fe54-lp153.39.1.noarch
      grommunio-sync-1.0.17-lp153.128.1.noarch
      grommunio-web-3.0.12.b67bff60-lp153.90.1.noarch
      gromox-1.25.49.d54c5b996-lp153.5.1.x86_64
      gromox-debuginfo-1.25.49.d54c5b996-lp153.5.1.x86_64
      gromox-debugsource-1.25.49.d54c5b996-lp153.5.1.x86_64

      grommunio-sync-top

      FYI: Agent ist nun überall "unknown"

      Ein paar von denen hier in /var/log/grommunio-sync/grommunio-sync-error.log

      21/06/2022 07:51:10 [21769] [WARN] [gg@mail.de] MAPIProvider->getAppointment: The attendee 'info' of type ZARAFA can not be resolved. Code: 0x8004010F

      Kann aber auch sein das die noch davon waren bevor ich die config.php mit der neuen ausgewechselt habe.

      Mal beobachten... und die Eiertelefon-Besitzer testen lassen.....

      Android Kalender

      org.lineageos.etar schauts gut aus - test mit
      org.google.android.calendar immer noch doppler bei einladungen test mit v.2

      21/06/2022 08:42:20 [24330] [WARN] [cb@mail.de] SyncAppointment->Check(): Parameter 'organizername' and 'organizeremail' should be set for a meeting request
      21/06/2022 08:45:26 [24328] [WARN] [cb@mail.de] SyncAppointment->Check(): Parameter 'organizername' and 'organizeremail' should be set for a meeting request

        crpb

        Ich denke das Issue ist mit dem letzten Update gelöst, oder?

        • crpb replied to this.

          mwilliams Ich denke das Issue ist mit dem letzten Update gelöst, oder?

          Die Chaos-Einträge sollten behoben sein so wie es scheint.

          Aber.. doppelte Einträge durch Google-Apps-Kalender werden noch angelegt.

          Also mir scheint es das der Google Apps Kalender wenn man jemanden einlädt das eigene Konto auch bei Teilnehmern zu hinterlegen.
          Dann legt es beim Speichern den Termin an.
          Und dann werden an alle Teilnehmer, also das eigene Konto sowie an andere eine Einladung geschickt.

          Somit werden die Einträge dann immer gedoppelt.

          Ich habe dann auch eine Kalender-Einladung im Postfach von mir selbst.
          Mit der LineageOS Kalender-Anwendung passiert es nicht.

          Und auch noch mal in der Web-App geschaut bei "Planen". Da ist dann mein Benutzer, ein Fremdes Konto und dann noch mal meine E-Mail Adresse als Teilnehmer gelistet. Und der andere Kalender-Eintrag hat nur mein eigenes Konto drin bei Planen.

          Habe mal die Grommunio-Sync-Logs in Ticket# 81331 hinterlegt.

            Hallo,

            crpb Habe mal die Grommunio-Sync-Logs in Ticket# 81331 hinterlegt.

            Den Termin "Test mit" gibt's im Log, allerdings mit der Startzeit von 20220626T070000Z. "Test mit v.2" gibt's im Log nicht. Können Sie auch das Log mit den 2 Terminen aus dem Screenshot an das Ticket anhängen?

            Sind nur die Einträge mit anderen Teilnehmern doppelt? Oder auch die Einträge ohne Teilnehmern?

            Treten

            • crpb replied to this.

              andreaslang Können Sie auch das Log mit den 2 Terminen aus dem Screenshot an das Ticket anhängen?

              Hab vorhin mal alles frisch probiert und alle logs, ics und screenshots hoch geladen.

              Vielen Dank!

              Wir werden einige Sachen beim Verarbeiten von Terminanfragen in web und sync verändern, sodass dieses Problem hoffentlich behoben ist. Es könnte allerdings ein wenig dauern, bis diese Änderung in ein Release schafft, da sie hohe Auswirkungen haben kann, und wir es vorher noch ausführlicher als sonst testen müssen.

              Ich mach eh nie Termine mit dem Telefon :-).
              Ist mir aus zufall aufgefallen da ich das was mit iPhones bei uns Probleme gemacht hat mal mit dem Android testen :-).

              © 2020-2024 grommunio GmbH. All rights reserved. | https://grommunio.com | Data Protection | Legal notice