SummaryIf there is a loss of power, or an unexpected or forced reboot when upgrading firmware, database corruption can occur
OverviewWhen upgrading firmware on the appliance (Ref: Procedure for Firmware Upgrade) , it is very important that the appliance is not restarted or shut down prematurely for risk of causing corruption in the database.
CauseOne of the final stages of the firmware upgrade is the database update. The database update process may take anywhere from between 5 minutes to half an hour depending on several factors such as database size, the amount of users, network traffic, device model, etc. This part of the upgrade is done after the device reboots to apply the new firmware after installation. It is possible that a user might get impatient waiting to be able to log in to the UI after the install, as they are unaware the database is still upgrading in the background, and will reboot the device.
If the device reboots or loses power, during the database update process, corruption in one or more of the database tables will likely occur.
Some symptoms of database corruption are:
- Unexpected reboots
- High hardware resource usage (High CPU, RAM usage)
- Network slowness
- Unable to log into the appliance
- Graphs showing 'No Data for Selected Period' or not loading at all
WorkaroundThere are no workarounds for a corrupted database. Follow the path to resolution.
ResolutionIt is good practice to ensure the system disk has enough free space for the database to operate properly. Delete old and unused system, TCP dumps and/or snapshots.
In order to resolve a situation where there is a corrupted database, restart the database will be the first step to troubleshooting.
Log into the appliance via a CLI program of your choice and type the following:
service database restart
If the above command does not resolve the issue, it is possible that either the data contained within the database itself is corrupt or there is no corruption in the database at all. In both circumstances, contact Exinda support.
Additional InformationBest practice is to wait for half an hour before attempting to log on to the Exinda appliance in order to allow the database time to update. Also ensure that the appliance receives its power from a working and stable power source.
Internal Notesstop mysqld, clear the data in /var/data/monitor and restart mysqld
exinda(config)# pm process mysqld terminate
[admin@exinda~] # rm -rf /var/data/monitor/*
admin@exinda monitor]# cli
exinda# pm process mysqld restart