• General

  • grommunio-web (and Outlook) search - unsorted results (indexing?)

testing Grommunio web with some larger mailboxes

when entering some search terms (example sender name) and selecting a time frame (past two weeks) the results shown are "all over the place" and strangely sorted.

I'm also unable to "modify" the sort order by clicking on the "date" header in the results as it "complains" with:

which is understandable... but as the search conditions are not "honoured"...

used versions:
grommunio Web: 2.5.0.01abcdef-lp153.4.1
Gromox: 2.13.41.19134b4-1+27.1

i treied to recreate the grommunio-web sqlite index but that does not help.

Outlook behaves similarly. (configured in on-line mode (no cached exchange mode))
searching for sender (by typing just the name) shows first 250 "hits" in strangely sorted order.
At the end of the list is the "notice" that search is limited and other data could be found on server...

This 250 items limit can be removed by unchecking "improve search speed by limiting the number of results" in the Outlook "search tab" settings.
In that case (unlimited results) all hits are returned... eventually... but the list is populated in a random order.
For example it "immediately" shows some results dating back a year or so, then few weeks etc
as the search progresses the list is populated by other results and if you let it work for a looooong time it eventually produces the expected results...

But the users won't wait 30min+ to get the results (mails from a few days ago, for example, which are populated at the end...)

I understand the grommunio-web is using the sqlite index file for faster searching... what is used (it anything) by MAPI/Outlook access to speed up searches?
Is there anything that can be re-indexed/re-created to solve this behavior?

Thanks!

Try to remove all indexes:
rm -rf /var/lib/grommunio-web/sqlite-index/*

And enable and restart the indexer to refresh all indexes:
systemctl --now enable grommunio-index.timer
systemctl restart grommunio-index.service

Please note, refreshing all indexes needs some time.

    WalterH

    Yeah... I did something similar but manually and just for one mailbox.

    • Closed all web sessions to the mailbox
    • removed the sqlite index for the particular user
    • manually executed grommunio-index for that user

    but... did not notice any "improvement" in the behavior.

    Is there any "index" for mapi/outlook searches? (Besides local windows search indexer which does nothing if there is no local files (pst/ost))

    but ok... I can retry with the current/last gromox version (as this was done a few revisions back of gromox and at least one of grommunio-web)

      WalterH

      Great! Now I broke it!

      the grommunio-index(-run.h) is now consitently throwing errors: exmdb call failed: Dispatch error
      in logs I noticed quite some zcore entries with the same "subject":
      exmdb_client_rpc_do: dispatch error

      That mbox had a 900MB index.sqlite3 file... which I cannot recreate now... and the mailbox is a large one (migrated less than half of it - 250K messages, 60GB)

      I have log_level=4 for zcore but nothing is there... suppose I should increase the log level, right?

      Also... I don't really get the log settings...
      I have set various log_file=/var/log/gromox/xxxx.log but only a very small set of messages are logged there... and all the other are logged in syslog?! WTF?
      Shouldn't the config files be "relevant" for log settings?

      Ideas on how to fix/check this dispatch_error? What to look for??

        Enough free space?
        df -h

        Are all services running?
        systemctl --failed

        Emmm....
        grepping through logs I saw grommunio-index-run throwing some SQLITE exceptions 8
        which should be "read only database"....
        So I started playing with file permissions which seems a little "off" for some folders/indexes and, in the meanwhile, I see the index being regenerated by the grommunio-index.timer triggering reindexing...

        OK... so maybe the issue was with the file permissions? (but that does not explain the dispatch error... as the file was generated, folder counted and then the error was thrown leaving a 25KB index.sqlite3 file)

        Anyway... the index is regenerated... but it does not help.

        WEB searching returns the "right" results only for past weeek time filter, any other filter give the same (bad) results... (month, custom, anything else gives the same results)

        16 days later

        Any more "hints" for this problem of mine?
        I'm the only one having such issues?

        It's very similar behavior in Grommunio-web and in Outlook - the search results are not "sorted" and not filtered correctly, consequently the sorting of results (trying to fix the problem) is not working also.

        I have similar issues with a bigger mailbox size, here ist my story:

        gromox-2.15.4.8a35481-lp154.12.1.x86_64
        grommunio-index-0.1.37.74e396b-lp154.40.2.x86_64
        grommunio-web-3.4.15.eb56496c-lp154.16.3.noarch
        SUSE 15.4, installed via grommunio-setup (setup.sh)

        After installation via setup.sh, the web search was not functioning.

        Reason:
        grommunio-index was not installed by the script. I installed it accordingly afterward. grommunio-index-run.sh then did not want to connect to the database (wrong password).

        Reason:
        grommunio-index-run.sh does not handle special characters in the password well (hint to the developers!). In my case, it was a hash "#". After changing the password, the index build ran without errors and the SQLite databases were built. In the web client, the search now returns results.

        However:
        The search results are inaccurate. Only results older than 5 years are listed. Also, the sorting by date is messed up. It starts with an email from 2015 and then continues with 2017. I had requested the last 6 months in the filter. The search does not seem to react to this at all. The result is the same. When searching for a sender email address, the search sorting cannot be changed. A click on the "Date" column leads to the message "Sortiervorschlag: Sie können nach Datum sortieren, wenn Sie Ihre Suche verfeinern. Verwenden Sie die Filter oder einen anderen Suchbegriff." (see reply from mculibrk above)

        Maybe a dev lookup for grommunio-web and grommunio-index is needed?!

        Yes... as i mentioned, the results are "all over the place".
        And the (time) filters works only for the "two weeks" option (at least for me, on multiple domains tested, same server)

        Can anyone comment or give any hint on indexing in regard to "protocol" access (Activesync, Outlook, EWS, IMAP...)?
        I understand the Grommunio-web is using the sqlite per user indexing databases....
        As I don't get basically nothing when searching through ActiveSync (but maybe my "oversized" inbox is the culprit) and Outlook (in online mode) works veeeery sloooow with the before mentioned "all over the place" results

        17 days later

        I guess you may have an issue with file system right. Try to repair the file system rights and recreate the search index:

        Repair the file system rights:

        cd /var/lib/grommunio-web/sqlite-index
ls -ltr
chown groweb:groweb * -R
chmod 775 *
ls -ltr
find . -type f -exec chmod 664 {} \;

        Close all browsers and recreate the indexes:

        rm -rf /var/lib/grommunio-web/sqlite-index/*
systemctl restart grommunio-index.service

        I hope this helps.

        Sorry for not replying before...

        After some of the "later" updates the grommunio-web search (+filters) actually works "as expected"!
        Yay!

        I checked/rechecked the file rights before and rebuilt index a few times before but that did not help... before the updated grommunio-web.

        Results are sorted newer to older and it's also possible to sort the results by date without that "strange" error saying that you should change the search terms.

        Also filtering options seems working - at least the time period you choose in the search options is honoured.
        Sooo... great!

        It's not "blazing fast" but OK... I tested on a large mailbox with 150K+ items in the inbox...

        BUT searches through Outlook (in non-cached mode) are still "messy"...
        It first shows some "random" results and only after you leave it alone for quite some time it actually finishes the search and shows all the results sorted in a "reasonable" way...

        Is this Outlook search using any kind of "accelerator"/index? If so... which one?? Is there any "reindex/rebuild" option for that?

        AND AirSync searches (on mobile devices) "does not work"... at least on the same 150K+ items mailbox... no results for any query.
        How to debug/check/fix this??

        © 2020-2024 grommunio GmbH. All rights reserved. | https://grommunio.com | Data Protection | Legal notice