Arcserve RHA Troubleshooting › Spool Limit Exceeded

Spool Limit Exceeded

CR00404 "Spool limit exceeded. The scenario is stopped"

Reason:

This message can refer either to the spool on the Master or to the spool on the Replica. It means that the value you entered in one of the Spool properties - Max Spool Size or Min Disk Free Size - has been exceeded. As a result of reaching the threshold, the system issues an error and stops the ongoing replication. There is one exception to this rule - when the value of the Max Spool Size on the Replica is exceeded, a message is issued but the scenario is not stopped. In this case the following message appears:"ER00002" "Spool size limit exceeded. Current size: %1"

Several factors can cause the spool growth:

1. On the Master and Replica - when a real-time Antivirus scanning is active, it filters every replicated file before applying the change to the data file. This filtering process causes the replicated files to cache in the spool on both the Master and Replica, before they are transferred or applied. As a result, a bottleneck appears, and the spool limit may exceed.
2. On the Master - when there is a low bandwidth connection between the Master and the Replica, and many updates occur on the Master, the spool limit may exceed.
3. On the Replica - when synchronization is initiated during an ongoing replication, the new updates from the Master are stored on the Replica spool. Only when the synchronization is finished, the replicated files that are cached in the spool directory are applied to the Replica data. If the synchronization is long, or many changes occurred on the Master in the meantime, the spool limit may exceed.
4. On the Replica - during the Assured Recovery test, the data changes that take place on the Master are sent to the Replica, but they are not immediately applied. Instead, these changes are accumulated and stored in the spool, and only when the testing is completed they are applied to the Replica data. This may cause the spool limit to exceed.
5. On the Replica - when there is a lot of activity on the data directory of the Replica server, a Disk IO queue may be formed. This causes the replicated files to begin cache in the spool, waiting for a sequential access to the disk subsystem, in order to apply the data changes to the files on the disk.
6. There is high Disk IO on the data directory on the Replica server. Disk IO is causing disk writes to queue, forcing the replication files to begin caching in the spool, waiting for sequential access to the disk subsystem in order to apply the data changes to the files on disk.
7. To resolve consider running Perform Counters to review Disk IO. consider moving spool to a volume which has relatively low Disk IO. It is not recommended to place the Spool on the same volume where the data files are located. In the case of an application server such as Exchange, SQL, Oracle, etc) the Spool should not be placed on a volume that contains either the DB files or the Transaction logs.

Note:

• When there is a disconnection between a Master and a Replica, the spool on the Master that stores changes for the disconnected Replica will be emptied.
• When the spool or the spool drive is full, Arcserve RHA stops the scenario. Once you restart the scenario, the spool is emptied, so the scenario can start running. However, if you do not make any change in the Spool definition or the Spool drive, this problem will occur again.

Action:

You can do one or all of the following actions:

• Exclude the Arcserve RHA Spool directory from AV scanning on all hosts involved in the replication.
• Decrease the value you entered in the Spool - Min Disk Free Size property.
• Increase the value you entered in the Spool - Max Spool Size property.
• Run Perform Counters to review the Disk IO activity. If necessary, change the Spool Directory location, and select a free and relatively inactive drive.

  Note: It is not recommended to place the Arcserve RHA Spool on the same volume where the data files are located. In the case of database servers, such as Exchange, SQL, and Oracle, the Spool should not be placed on a volume that contains either the DB files or the Transaction logs.

• Delete or move files from the current spool drive, and reduce its other activities.