Indexing does not catch up with archived emails if a high indexing interval is configured

Versions / Builds Affected

20131111

Status

Resolved

Problem Summary

Indexing does not catch up with archived emails if a high indexing interval is configured

TT / JIRAID

2168

How to Identify

If a high interval is configured under Configuration > Archive Stores > Index Management indexes in [Normal Mode] might not catch up with the amount of emails getting archived. There are not particular errors indicate this situation. The issue might be best understood by follow the steps below which describe how the situation can be reproduced: XXXX-XX-XX,XX:XX - Start with a fresh installation of MARC XXXX-XX-XX,XX:XX - Do NOT create a datasource entry (journal mailbox) XXXX-XX-XX,XX:XX - Go through the New Archive Store Settings wizard and configure Yearly as the schedule for Archive Stores XXXX-XX-XX,XX:XX - Archive a single email from the year 2014 so that an Archive Store for 2014 is created 2014-04-16,15:34 - Wait until the single email is indexed and confirm that the index is running in Normal mode 2014-04-16,15:36 - Change the indexing schedule to a higher value (I used every 6 hours) 2014-04-16,15:39 until 16:22 - Archive a high amount of emails into the 2012 archive store (I imported ~15k them via MAIS) Now wait and observe what happens on the indexing side once the indexing is being performed 1ST INDEXING ACTIVITY 6 HOURS LATER PROCESSED 501 EMAILS, THEN SLEEPS FOR ANOTHER 6 HOURS 2014-04-16,21:36:44,671,1,"#00000550","#00000008","info ","Index","Sync [2014 Jan - Dec] Index TS: 2005 Store TS: 101604 (normal mode)" 2014-04-16,21:36:46,021,1,"#00000550","#00000008","info ","Index","New: 501 Upd: 0 Del: 0" ... 2014-04-16,21:37:13,969,1,"#00000550","#00000008","info ","Index","AddMessagesToIndex [2014 Jan - Dec]. lastDocId: 502, currentSubIndex: 0" 2014-04-16,21:37:13,969,1,"#00000550","#00000008","info ","Index","Indexing path: C:\Program Files (x86)\GFI\MailArchiver\Indexes\2014 Jan - Dec\index0, starting DocId: 503" 2ND INDEXING ACTIVITY 6 HOURS LATER PROCESSED 500 EMAILS, THEN SLEEPS FOR ANOTHER 6 HOURS 2014-04-17,03:37:35,345,1,"#00000550","#00000008","info ","Index","Sync [2014 Jan - Dec] Index TS: 8085 Store TS: 101604 (normal mode)" 2014-04-17,03:37:36,274,1,"#00000550","#00000008","info ","Index","New: 500 Upd: 1 Del: 0" ... 2014-04-17,03:37:36,324,1,"#00000550","#00000008","info ","Index","AddMessagesToIndex [2014 Jan - Dec]. lastDocId: 1002, currentSubIndex: 0" 2014-04-17,03:37:36,324,1,"#00000550","#00000008","info ","Index","Indexing path: C:\Program Files (x86)\GFI\MailArchiver\Indexes\2014 Jan - Dec\index0, starting DocId: 1003" 3RD INDEXING ACTIVITY 6 HOURS LATER PROCESSED 500 EMAILS, THEN SLEEPS FOR ANOTHER 6 HOURS 2014-04-17,09:37:58,663,1,"#00000550","#00000008","info ","Index","Sync [2014 Jan - Dec] Index TS: 11096 Store TS: 101604 (normal mode)" 2014-04-17,09:37:59,300,1,"#00000550","#00000008","info ","Index","New: 500 Upd: 1 Del: 0" ... 2014-04-17,09:37:59,349,1,"#00000550","#00000008","info ","Index","AddMessagesToIndex [2014 Jan - Dec]. lastDocId: 1502, currentSubIndex: 0" 2014-04-17,09:37:59,350,1,"#00000550","#00000008","info ","Index","Indexing path: C:\Program Files (x86)\GFI\MailArchiver\Indexes\2014 Jan - Dec\index0, starting DocId: 1503" And so it would continue. If there is a constant and high enough mailflow the system would never catch up with indexing vs. archived emails.

Workaround / Fix Details

Fixed in MARC2014 build 20140616

Required Actions

Upgrade to MARC2014 build 20140616 or newer