SQL Server upgrade step 'msdb110_upgrade.sql' encountered error 2714, state 6, severity 25

Upgrading SQL server to SP4 or SQL Server 2016 encountered error 'msdb110_upgrade.sql' encountered error 2714, state 6, severity 25

When doing an dummy run for a SQL instance upgrade, I encountered this error and it resulted in the SQL service not starting after upgrade and the upgrade wizard reporting errors.

After a couple of attempts I had to dig into it to find what was going on so referencing the article SQL Server Service fails to start after applying patch. Error: CREATE SCHEMA failed due to previous errors I tried deleting the DatabaseMailUserRole Schema from msdb but the server still failed to start.

This was SQL Server 2012 so I checked the SQL server logs found in at C:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\Log\ERRORLOG.txt (that path may vary between versions).

The tail of the log looked like the following, where I had already removed the DatabaseMailUserRole, so it may have reported that too previously:

019-02-15 17:04:20.53 spid4s      Setting object permissions...
2019-02-15 17:04:20.63 spid4s      Error: 2714, Severity: 16, State: 6.
2019-02-15 17:04:20.63 spid4s      There is already an object named 'TargetServersRole' in the database.
2019-02-15 17:04:20.63 spid4s      Error: 2759, Severity: 16, State: 0.
2019-02-15 17:04:20.63 spid4s      CREATE SCHEMA failed due to previous errors.
2019-02-15 17:04:20.63 spid4s      Error: 912, Severity: 21, State: 2.
2019-02-15 17:04:20.63 spid4s      Script level upgrade for database 'master' failed because upgrade step 'msdb110_upgrade.sql' encountered error 2714, state 6, severity 25. This is a serious error condition which might interfere with regular operation and the database will be taken offline. If the error happened during upgrade of the 'master' database, it will prevent the entire SQL Server instance from starting. Examine the previous errorlog entries for errors, take the appropriate corrective actions and re-start the database so that the script upgrade steps run to completion.
2019-02-15 17:04:20.63 spid4s      Error: 3417, Severity: 21, State: 3.
2019-02-15 17:04:20.63 spid4s      Cannot recover the master database. SQL Server is unable to run. Restore master from a full backup, repair it, or rebuild it. For more information about how to rebuild the master database, see SQL Server Books Online.
2019-02-15 17:04:20.63 spid4s      SQL Server shutdown has been initiated
2019-02-15 17:04:20.63 spid4s      SQL Trace was stopped due to server shutdown. Trace ID = '1'. This is an informational message only; no user action is required.
2019-02-15 17:04:20.80 spid14s     The SQL Server Network Interface library successfully deregistered the Service Principal Name (SPN) [ MSSQLSvc/***] for the SQL Server service.
2019-02-15 17:04:20.80 spid14s     The SQL Server Network Interface library successfully deregistered the Service Principal Name (SPN) [ MSSQLSvc/***:1433 ] for the SQL Server service.

So it seems I had the same problem as the referenced blog post but with another schema role.

Solution

After running the upgrade and it failing, I started SQL server with the trace flag 902, using net start mssqlserver /T902 from an elevated command prompt. This prevents the startup scripts running.

image

Then connected to the SQL server instance using SSMS and located the 'TargetServersRole'  under the msdb database and right click, deleted it.

image

I then stopped SQL server and restarted it normally, without the trace flag, so it runs the 'msdb110_upgrade.sql' at startup again.

This time it started normally.

image

 

Problem sorted!

 

If this was helpful please comment as it motivates me to blog more!

Prevent SQL job running when another job is running

 
CREATE VIEW myschema.Admin_RunningSQLJobs
AS
SELECT job.NAME
,job.job_id
,job.originating_server
,activity.run_requested_date
,DATEDIFF(SECOND, activity.run_requested_date, GETDATE()) AS Elapsed
FROM msdb.dbo.sysjobs_view job
JOIN msdb.dbo.sysjobactivity activity ON job.job_id = activity.job_id
JOIN msdb.dbo.syssessions sess ON sess.session_id = activity.session_id
JOIN (
SELECT MAX(agent_start_date) AS max_agent_start_date
FROM msdb.dbo.syssessions
) sess_max ON sess.agent_start_date = sess_max.max_agent_start_date
WHERE run_requested_date IS NOT NULL
AND stop_execution_date IS NULL


Using this view we can check for a running job before running our job…
In this example we don’t want the nightly pricing build to run if a monthly build is in progress, note, nightly is scheduled later than monthly. So wrap the call to the stored procedure in the job step like this…
 
IF NOT EXISTS(SELECT * FROM myschema.Admin_RunningSQLJobs WHERE name='Pricing - Monthly Rebuild') 
BEGIN
EXEC myschema.Nightly Pricing build
END

Note about backups:
We restore the live company into the test company of our ERP system, once a week, to keep the test company data fresh. It is worth noting that it is wise to keep the above SQL view in a database that is not involved in a restore, otherwise errors will occur trying to access the view as that database is in restore mode and cannot be accessed by the SQL job.
For this reason we keep this view in an “admin” database that is not part of the automated restore process, using the view in SQL jobs used against the test database prevents them erroring when the restore process in in progress.
 

SQL formatting and Power Query

Today I got caught out by using SSMSBoost to SQL format a query being used as the source for a Microsoft Power Query.

On updating the stored procedure the query stopped working in Power Query, with the error that the column “Temp” was missing.

Looking at the query output in SSMS, I could see that the column had changed to TEMP in upper case from Temp in the original query, easily fixed. Then I realised what might have happened to cause it and proved it. Formatting the SQL had caused the temp column to be upper cased, my suspicion is that Temp was wrongly identified as a keyword by the formatting template and thus with the setting in SSMSBoost being to uppercase keywords, it got uppercased. Power Query is case sensitive so got upset.

SSMSBoot Settings Menu showing the UPPERCASE keywords = True

Just a warning for those of you that use SQL formatting tooling, one to keep an eye out for, luckily having a column named Temp is not too common.

Recovering a deleted Reporting Services Report

Yes, I deleted the wrong report while housekeeping. This raised the question as to how to recover it once it has been deleted? There is no recycle bin to or undelete option in reporting services on the version I work with and at the time of writing (Nov 2017).

It turns out to be very simple and quick to restore, if you have backups of your “ReportServer” database, which of course every one has.

Option 1 – Restore to point in time

The simplest way is to restore the “ReportServer” database from backup, to a point in time just before the deletion, but this would loose any changes on the report server made since it was deleted. In my case I had spent a few hours after deleting the report with housekeeping all all kinds of things on the report server, so this was not desirable as I’d loose that work.

Option 2 – Restore the single report from backup

Slightly more involved as an option, but still quick and easy, follow the steps below, it only takes a few minutes to do.

To restore the “ReportServer” database

    • Right click the database in SSMS, select restore
    • Rename the Destination Database to ReportServerRestored
    • Use the check boxes to find the point in time to restore to, here I chose to not apply the logs to make for a speedy restore
    • Click Files on page selector in left hand side. If the default path is not appropriate (under Restore As, in the grid, you may need to scroll left to see), then change the path that the database files will be restored to
    • Click options on page selector in left hand side to select options. Unselect Take tail-log backup.
    • Click ok to start the restore

2017-11-02_12-07-37

Extract the XML that represents the report from the restored database

Reports are stored in a binary field in a table in the database as XML. The following article explains how to get at the XML that defines the reports,

 Extracting SSRS Report RDL (XML) from the ReportServer database

Using the information in this article we can get to the report we need by running the script in SSMS.

--The first CTE gets the content as a varbinary(max)
--as well as the other important columns for all reports,
--data sources and shared datasets.
WITH ItemContentBinaries AS
(
SELECT
ItemID,Name,[Type]
,CASE Type
WHEN 2 THEN 'Report'
WHEN 5 THEN 'Data Source'
WHEN 7 THEN 'Report Part'
WHEN 8 THEN 'Shared Dataset'
ELSE 'Other'
END AS TypeDescription
,CONVERT(varbinary(max),Content) AS Content
FROM ReportServerRestored.dbo.Catalog
WHERE Type IN (2,5,7,8)
),
--The second CTE strips off the BOM if it exists...
ItemContentNoBOM AS
(
SELECT
ItemID,Name,[Type],TypeDescription
,CASE
WHEN LEFT(Content,3) = 0xEFBBBF
THEN CONVERT(varbinary(max),SUBSTRING(Content,4,LEN(Content)))
ELSE
Content
END AS Content
FROM ItemContentBinaries
)
--The outer query gets the content in its varbinary, varchar and xml representations...
SELECT
ItemID,Name,[Type],TypeDescription
,Content --varbinary
,CONVERT(varchar(max),Content) AS ContentVarchar --varchar
,CONVERT(xml,Content) AS ContentXML --xml

FROM ItemContentNoBOM
where name like '%{enter part of report name here}%'

Note you need to edit the {enter par of report name here} to be what it says, you will then be presented with the row(s) of interest. Work out which is the report you need if multiple rows are returned (note the restored database name is embedded into the SQL, change if you restored to another name).

 2017-11-02_12-35-35

On the row, example shown above, click the hyperlink of the column “ContentXML” to open the XML in an XML editor within SSMS (cool feature). Then go to File>>Save As.. within SSMS to save to a drive for later import to the report server.

Rename the xml file

Rename the xml file as a .rdl file you can make the filename an appropriate descriptive name for the report as this will later show in the report server

Upload the report definition file to the server

Upload the .rdl file to the reports folder on the reporting services server. Navigate to the folder it should reside in and select “Upload File” as shown below

2017-11-02_12-22-29

Recreate subscriptions and schedules

You will have to manually set up subscription schedules again for the report if they existed before.

Delete the restored database

From SSMS right click the database name and select delete to remove it. Check that the database files are also deleted on the server.

 

The report is now restored and the task is completed!

If you found this useful please comment, it helps motivate me to keep adding to the blog…