DPM 2012 R2 SQL Backups Failing After November 2014 OS Updates
Following the installation of the November 2014 OS updates and the updated DPM agent (to update rollup 4) onto a number of servers, I saw failures when attempting to perform a backup of SQL Server 2012 databases hosted on Windows Server 2012. These manifested as ‘replica is inconsistent’ in the DPM console: On the affected SQL Server, the following error was appearing in the server event log each time a consistency check was triggered:
The COM+ Event System failed to fire the RequestWriterInfo method on event class {FAF53CC4-BD73-4E36-83F1-2B23F46E513E} for publisher and subscriber . The display name of the subscription is "{E6057DCA-9DE3-42FC-9A6E-A057156277B4}". The HRESULT was 80042318.
I tried a number of things to resolve the issue:
- Modified the protection group. Initially, the protection group wizard did not show the ‘SQL Server’ option and I could not re-add the SQL Server databases. The agent required an update (again) and following this, I could then re-add the SQL Server databases, however when synchronised, the databases were still reporting ‘replica is inconsistent’.
- Removed the protection from the affected server, removed the DPM agent, rebooted, then reinstalled and attached the DPM agent and re-added the SQL Server databases to the protection group. This also had no effect, with the databases still reporting as ‘replica is inconsistent’.
- Updated the affected server to SQL Sever 2012 SP2 CU2 as the cumulative update mentioned VSS related fixes. Again this had no effect.
I have a Microsoft support call open at the moment in an effort to determine the root cause of the issue, but have found that removing a specific update appears to resolve the issue. The update in question is KB3000853. KB3000853 is an update rollup for Windows 8, Windows RT and Windows Server 2012 and contains a number of fixes including KB2996928 which is a backup related hotfix that references VSS. My suspicion is that this update is what has caused the issue, but will update this post when I get a final resolution from Microsoft support. Update: I've had this confirmed as an issue with KB3000853 by the Microsoft engineer I've been working with. The workaround for the moment is to either change the account that runs the SQL VSS Writer service to the domain admin account (I assume an account that has local admin permissions, but have not tested this so far), or remove KB3000853, at which point the backups start functioning correctly again. There is currently no confirmed release date for an updated version of KB3000853. Update 2: There’s now a revised hotfix to correct the issue experienced above. The steps to correct the issue are:
- Install KB3000853.
- Install hotfix KB2996928.
This combination corrects the issues on the servers on which I have installed them.