Versions / Builds Affected20140616, 20141117, 20150218
Problem SummaryArchive Store indexes grow too large in size
TT / JIRAID2675
How to IdentifyIndexes can become much larger than needed when building in Normal mode (as compared to Bulk mode, which is used when rebuilding an index).
~~~ Arbitrary example:
1. An Archive Store contains 100k emails and is 25GB in size
2. The index folder is also 25GB in size (this was build in Normal mode)
3. You rebuild the index (which utilizes Bulk mode) and resulting index is only 5GB in size
The important fact here is, that the index is significantly smaller in size after rebuilding it.
~~~ Concrete example from a customer:
Analyzing the index's history.ix file allows to determine a few numbers incl. how many emails were indexed within a certain time frame and how large the index grew
1. history.ix coverd a time frame of 6 hours and 20min (from 2014-10-08 from 11:33 until 17:53) ~~~ this can be read from the first and last timestamps within the file)
2. During this time 799 emails have been indexed ~~~ this is the difference between the last and first line reading: Docs in index:
3. The index grew from 94MB to 1441MB (an increase of 1347MB) ~~~ this is the difference between the last and first line reading: Total index size: (note that the file lists the size in bytes rather than megabytes)
This example shows that the index grew 1347MB within 6hours and 20min while indexing 799 emails. This increase appears way too high.
- The indexing interval is a factor here - Immediate (which translates to MARC checking every 10sec if indexing work needs to be performed)
- A factor which leads to this situation is the interval in which indexing occurs in combination with a constant flow of emails being archived.
- This is very similar to known issue article 8968 / TT1881 which covered to large indexes on MARC 2013 versions.
Workaround / Fix DetailsStep 1 - to avoid indexes growing large going forward:
Set the indexing interval (Configuration > Archive Stores > Index Management) to occur every 2 hours, 4 hours or a higher value (the higher the value the better in regard of this topic)
Step 2 - to reduce the size of existing large indexes:
Use the IndexCompactTool to compress existing large indexes (see the following article for further information on this tool: 8967 > Tool: IndexCompactTool (INTERNAL ONLY)
Rebuild the large index
Required ActionsEscalate with:
1. A set of TS files in which the customer left debug mode enabled for at least 6 hours
2. The history.ix file of the "current" index copied at the time when the troubleshooter was run