Phone Lines icon GFI Support

Articles in this section

See more

Processing Orphaned Files in Core\Queue or MAIS\Queue

Author: Timothy Parker Updated

Overview

This article will cover the process to manually review orphaned files located in the Core\Queue and MAIS\Queue directories and reprocess for archival

 

Diagnosis

 

Administrators may notice that there are leftover files stuck in the Core\Queue or MAIS\Queue leaving the directories to grow continually and taking up unnecessary space on the drive. Mail should continually move through this directory, be split in to the appropriate parts and removed after it has been processed. Service failures and software exclusions can cause items to hang in the folder and no longer be processed. 

Emails that are processed have a *.envelope or *.envelopefailed extension. Each email is converted into 3 to 4 parts:

  • .body is a copy of the message body appears only if the email has characters in the body.
  • .emc - a compressed copy of the email.
  • .eml - a copy of the email, which only this can be used to reprocess the email.
  • .ical - only if it's a calendar item.
  • Envelope XML file which contains information on how to archive the email and it's components (eml-emc-body filenames, owners, etc.).
    • .envelope - to be archived.
    • .envelopefailed - failed to archive, but will be retried on the 3rd pass of the queue folder.

There may also be a .retry[number]. The .envelopefail files are processed for every two runs against successful .envelope files. Each email that fails has a retry count associated. When the retry count of failed emails reaches 2, the administrator is sent a warning. If the retry count increases to 5, then the administrator receives an email indicating an error.

 

Prerequisites

To avoid future issues such as these, make sure proper backup and AV solution exclusions are in place.

 

Solution

 

Typical Scenarios

  1. If envelope and eml file types are missing, most likely is that the email was archived, but due to 3rd parties locking files, the remnants (body, ical, etc.) could not be removed.
  2. If we have envelopefailed, but the eml file is missing, then this is a strong indication that the eml was quarantined by an AV solution and was NOT archived.
  3. If the envelope is missing, but we see the eml, then Archiver doesn't know that it should archive the email and most likely, was NOT archived and should be reprocessed.

Below we see an example of how an email is split in to different files:

GFI-1_907956cf-9ffd-45cb-9555-6c3ced9fe5bd_44822.body
GFI-1_907956cf-9ffd-45cb-9555-6c3ced9fe5bd_44822.emc
GFI-1_907956cf-9ffd-45cb-9555-6c3ced9fe5bd_44822.eml
GFI-1_907956cf-9ffd-45cb-9555-6c3ced9fe5bd_44822.envelope

Follow the steps according to the type of file present in the folders:

  • If *.eml is present:
    1. Delete the other file types, if present.
    2. Move *.eml from Core\Queue to Core\Pickup.
    3. Rename the files from *.eml to *.RAW.
  • If *.body is present:
    1. Open the files with a text editor.
    2. Locate a unique combination of words.
    3. Use Advanced Search from the Archiver Console > Archive Search to see if a match is found. If there isn't one, use the Import Export Tool to download missing email(/s).

NOTES:

  • The Pickup and Queue are active and GFI Archiver Core service works in the directory every 5-30 seconds. It may be easier to stop the Core service if you need to process the files manually.
  • The EML file contains all information and once it's processed and moved to Queue, it will be ready to be archived, including the file types we delete previously.
  • Even if the email was archived, it will not cause a duplicate due to Single Instance Storage. A hashing function is called and checked for duplicates before archiving.

 

Testing

 

If successfully processed there will no longer be files stuck in the folder. With the service running we should see mail move in to the folder, be processed and move out of the folder again successfully after being written to the Archive Store.