Error: '0137 Too many words retrieved in index' when searching using a date criteria and archive stores with a high number of emails (larger than 256k)

Versions / Builds Affected

20131111

Status

Resolved

Problem Summary

Error: '0137 Too many words retrieved in index' when searching using a date criteria and archive stores with a high number of emails (larger than 256k)

TT / JIRAID

1876

How to Identify

1. At least some archive stores at a minimum of 256k emails or more 2. Confirm that the logging in Search\debuglogs\QueryEngine.log contains the points below 2014-01-20,14:30:13,583,1,"#0000109C","#00000031","info ","QueryEngine","Running dtSearch query..." 2014-01-20,14:30:13,583,1,"#0000109C","#00000031","info ","QueryEngine","limit: 5000" 2014-01-20,14:30:13,583,1,"#0000109C","#00000031","info ","QueryEngine","query: [( ( ( sender __contains ( ralph ) ) ) __and ( ( to __contains ( ahearn ) ) ) __and ( ( ( (utc __contains (True)) __and sentdates::62766709200~~63525785410 ) __or ( (__not (utc __contains (True))) __and sentdated::0000007305~~0000016090 ) ) ) ) __and ( __not ( ( itemtype __contains ( ((3)__or(4)__or(2)__or(12)__or(11)) ) ) ) )]" 2014-01-20,14:30:13,583,1,"#0000109C","#00000031","info ","QueryEngine","xfilter: (xfilter(word owners::811463959f240945bfafc1131be95987) __or xfilter(word owners::81146395-9f24-0945-bfaf-c1131be95987))" 2014-01-20,14:30:24,948,1,"#0000109C","#00000031","info ","QueryEngine","Search failed" 2014-01-20,14:30:24,948,1,"#0000109C","#00000031","error ","QueryEngine","error: dtSearch error 137 at Search: $E 0137 Too many words retrieved in index D:\GFIMAIL\2007-Jul-Sep\index 62766709200~~63525785410: 255998; ahearn: 1; ralph: 1; true: 0" 2a. The error line contains the error "dtSearch error 137 at Search: $E 0137 Too many words retrieved in index" 2b. The error line contains a high amount of words for the date criteria (which is this part here: 62766709200~~63525785410: 255998) 2c. The line with the query contains a date critieria (which is this part here: sentdates::62766709200~~63525785410)

Workaround / Fix Details

Root cause fixed in MARC2014 build 20140616 Fix details: To address the root cause development needed to change the way how dates/times are stored within the indexes. Unfortunately, this change requires indexes to be rebuilt. ----- 20131111 PATCH WHICH DOES NOT ADDRESS THE ROOT CAUSE: http://ftp.gfisoftware.com/patches/MARC2013/20131111/MARC2013_R2_PATCH_20140423_1876.zip The patch makes the simple search masks in the web interface to NOT pass a date criteria as a search query automatically which bypasses the issue. This means that the root situation still exists and customers might still run into the error; e.g. when manually specifying a date criteria in the search advanced options or the Import Export Tool which hits more than 256k emails.

Required Actions

1. Upgrade to MARC2014 build 20140616 or newer 2. Rebuild indexes