28 March 2013

Modified fix orphaned users script




I modified this script to handle the 'dbo' user. So yo no longer have to run sp_changedbowner 'sa' before (Or after) running this proc.

/*************************************************************************************
This procedure should be created in the Master database. This procedure takes no
parameters. It will remap orphaned users in the current database to EXISTING logins
of the same name. This is usefull in the case a new database is created by restoring
a backup to a new database, or by attaching the datafiles to a new server.
*************************************************************************************/

IF OBJECT_ID('dbo.sp_fixusers') IS NOT NULL
BEGIN
DROP PROCEDURE dbo.sp_fixusers
IF OBJECT_ID('dbo.sp_fixusers') IS NOT NULL
PRINT '<<< FAILED DROPPING PROCEDURE dbo.sp_fixusers >>>'
ELSE
PRINT '<<< DROPPED PROCEDURE dbo.sp_fixusers >>>'
END
GO
CREATE PROCEDURE dbo.sp_fixusers
AS
BEGIN
DECLARE @username varchar(25)
DECLARE fixusers CURSOR
FOR
SELECT UserName = name FROM sysusers
WHERE issqluser = 1 and (sid is not null and sid <> 0x0)
and suser_sname(sid) is null
ORDER BY name
OPEN fixusers
FETCH NEXT FROM fixusers
INTO @username
WHILE @@FETCH_STATUS = 0
BEGIN
IF @username='dbo'
BEGIN
EXEC sp_changedbowner 'sa'
END
ELSE
BEGIN
EXEC sp_change_users_login 'update_one', @username, @username
END
FETCH NEXT FROM fixusers
INTO @username
END

CLOSE fixusers
DEALLOCATE fixusers
END
go
IF OBJECT_ID('dbo.sp_fixusers') IS NOT NULL
PRINT '<<< CREATED PROCEDURE dbo.sp_fixusers >>>'
ELSE
PRINT '<<< FAILED CREATING PROCEDURE dbo.sp_fixusers >>>'
go

Fix Orphaned users using script


/*************************************************************************************
This procedure should be created in the Master database. This procedure takes no
parameters. It will remap orphaned users in the current database to EXISTING logins
of the same name. This is usefull in the case a new database is created by restoring
a backup to a new database, or by attaching the datafiles to a new server.
*************************************************************************************/


IF OBJECT_ID('dbo.sp_fixusers') IS NOT NULL
BEGIN
DROP PROCEDURE dbo.sp_fixusers
IF OBJECT_ID('dbo.sp_fixusers') IS NOT NULL
PRINT '<<< FAILED DROPPING PROCEDURE dbo.sp_fixusers >>>'
ELSE
PRINT '<<< DROPPED PROCEDURE dbo.sp_fixusers >>>'
END

GO
CREATE PROCEDURE dbo.sp_fixusers
AS
BEGIN
DECLARE @username varchar(25)
DECLARE fixusers CURSOR
FOR

SELECT UserName = name FROM sysusers
WHERE issqluser = 1 and (sid is not null and sid <> 0x0)
and suser_sname(sid) is null
ORDER BY name

OPEN fixusers
FETCH NEXT FROM fixusers
INTO @username

WHILE @@FETCH_STATUS = 0
BEGIN
EXEC sp_change_users_login 'update_one', @username, @username
FETCH NEXT FROM fixusers
INTO @username
END


CLOSE fixusers
DEALLOCATE fixusers
END
go
IF OBJECT_ID('dbo.sp_fixusers') IS NOT NULL
PRINT '<<< CREATED PROCEDURE dbo.sp_fixusers >>>'
ELSE
PRINT '<<< FAILED CREATING PROCEDURE dbo.sp_fixusers >>>'
go

 

compatibility levels for web sychronizations


Replication Topologies

Replecation Features:




Error code 0x5. [SQLSTATE 42000] (Error 15404): Could not obtain information about Windows NT group/user


Error 0x5 means access denied.

Try to changing the owner of the jobs to the ‘sa’ account, on the properties of the job.

If the above does no help, please change the service account for the SQL Server agent service to account with proper permissions.

http://technet.microsoft.com/en-us/library/ms345578.aspx

I hope this will helps .

21 March 2013

Express edition features version wise

These are the features improved in express edition of sql server version wise
 
 
 
 

 

Backup the database using RETAINDAYS option


I have many times seen people specifying something like 3, and expect SQL Server to keep the three most recent backups in the backup file and overwrite everything which is older than that. Well, that is not what the option does.

But before we go into details, let's look at an example backup command which is using this option:

BACKUP DATABASE sqlmaint TO DISK = 'R:\sqlmaint.bak' WITH RETAINDAYS = 3

The RETAINDAYS is also exposed in the backup dialog in SSMS: "Backup set will expire: After x days".

It is also exposed in Maintenance Plans, the backup task. The option is named "Backup set will expire: After x days". It is only enabled if you select the "Back up databases across one or more files" option, which is not the default option. This makes sense.
The default option is "Create a backup files for every database", which means that every time a backup is performed, a new file is created consisting of the database name, date and time. Since we will see that this option is only relevant when we do append, it makes sense in the RETAINDAYS not being enabled for this choice.

So what does this option do? All it does is make SQL Server return an error message of you try to do a backup using the INIT option (which means overwrite) before the date and time has occurred. In other words, it tries to help you in not overwriting a backup file, using the INIT option, before it is time. You can still overwrite earlier by either using the stronger FORMAT option instead of INIT; or by simply deleting the backup file. Btw, the RETAINDAYS parameter has a cousin named EXPIREDATE, which does the same thing but you specify a datetime value instead of number of days.
Backup generations
So, we have seen that RETAINDAYS do not in any way provide any automatic backup generation handling. There is no such functionality built-in in the BACKUP command. This means that when you are looking for functionality such as "keep backup files three days, and remove older than that", you need to look outside the BACKUP command. I.e., some script or tool of some sort.

To know how the job is organizing internally

First a quick recap of how agent does things:

When a job is owned by somebody who is sysadmin, then no impersonation is performed.

  1. T-SQL jobsteps will be executed as Agent's own Windows authenticated login.
  2. Other jobsteps (which operates as the OS level) is done in Agent's service account's context.
Above is, assuming you didn't ask for impersonation for the jobstep (Run As).
One of my friend asking about job owned by Windows account and what happens when that person leaves the company. I was about to say "no problem, if the job owner is sysadmin", but I felt there was a bit more to it. The scenario is:

  • The job owner is member of a Windows/AD group. Say you have a "SQL Server DBA" group in your AD.
  • This group is added as a login to your SQL Servers. The persons in that group do not have individual logins in SQL Server - only through this group.
  • This group is sysadmin. (I'm not saying that having jobs in general owned by sysadmins is a best practice. You can for instance have vendors installing some software and you don't want to make them sysadmin. In such a case, clearly we don't want the job to keep running if that person would disappear. So, I'm targeting jobs owned by sysadmin here, and I would bet you have bunch of those.)
  • Somebody in this group creates an Agent job.
  • The person in the group leaves the company and is removed from this Windows/AD group.
 The thing now is that the job no longer works. And here's the reason:

If the owner was somebody in Windows/AD who's SID exists in SQL Server (the individual was added as a login to SQL Server), then the job will still work. This also applies for SQL Logins which might be disabled (like "sa" for instance). The reason for this is that the SID for the job owner exists in sys.server_principals and SQL Server will know that this SID is sysadmin. Since Agent won't do impersonation for sysadmins, there's no need to "go out to Windows" and see if this SID exists in Windows/AD. Remove the login, and the job will stop working, though - but you have at least don't something explicitly inside your SQL Server to cause this (removed the login).

But when the owner's SID don't exist in sysadmin we have a problem. The problem is that the recorded owner of the job is the SID for the Windows user, even though that SID doesn’t exist in sys.server_principals. As long as this still exists in Windows/AD, the job will work just fine. Agent will run this job owned by ?, ask Windows who this SID is and see that this SID is a member of a group which exists as a login in SQL Server and that is sysadmin. Now, imagine what happen if the SID doesn't exist in Windows anymore. The job is owned by ?, and that is all that SQL Server knows. The SID no longer exist in Windows/AD so you there's nobody to tell SQL Server "Yeah, this used to be a sysadmin in SQL Server - through a group membership in Windows/AD".

Even worse, the job seems to keep working until you re-start the SQL Server service, some caching going on. Makes it even more difficult to determine why the job suddenly stops working: "We deleted this Windows account 4.5 months ago." This is easy to repro, and you don't even need a domain (local Windows accounts and groups work just fine):

  • Create a Windows group for this.
  • Create a Windows account for this.
  • Add the Windows account to above group.
  • Add above group as a login to SQL Server.
  • Make this group sysadmin.
  • Create a job, with two jobsteps:
  • T-SQL which executes: PRINT ORIGINAL_LOGIN() + ' | ' + SUSER_SNAME() + ' | ' + USER_NAME()
  • CmdExec which executes: WHOAMI.EXE
  • (Above isn't really necessary, but useful for testing other impersonation aspects of Agent)
  • Specify the Windows account as owner of the job.
  • Run the job, make sure it works fine.
  • Remove the Windows account from the Windows group.
  • Run the job and see it fails.
I'm sure this has been blogged before, but I was a bit surprised by it. I have now added a check for this in my "Check best practices" stored procedure I run every day... On the flip side, this doesn't seem to happen for multi-server (MSX) jobs, the ownership seems to have special handling for these - you will see if you check the owner_sid in sysjobs.

Memory usage control using Resource Governor


You might know that Resource Governor (RG) can be used to allow you to affect resource allocation inside SQL Server. RG was introduced with SQL Server 2008 and requires Enterprise Edition or similar/higher. There are two things you can govern using RG: cpu and memory.

For introductory information on RG, see for instance these BOL topics.

A misconception I see from time to time is that the memory setting is to restrict the Buffer Pool, BP, (mainly used for caching pages from disk). This seems difficult since a page in BP has a cost, but isn't owned by whoever brought it into cache or who referenced it last time. So, it seems difficult with current implementation of cache handling and ageing to govern the BP memory for RG. What RG does govern is "working memory" for a query. Think memory used for sorting, locks, hashing and such.

We just had such a discussion in the MSDN forums, and I decided to do a test to see if we could show that RG do not govern the BP. Brief outline of the script

  • Restrict the BP to 300 MB
  • Create two databases (a and b)
  • Create a table with size of 255MB in each database
  • Create two logins with name a and b, which will be used to execute queries
  • The logins end up with two different resource pools (aPool and bPool), where aPool has max mem 80% and bPool has max mem 20%
  • A loop which generates queries that are executed using xp_cmdshell and SQLCMD
  • The loop first scan over the first approx 9% of data, first login a in database a, then in login b in database b
  • Next iteration is then done over the following 9 % of data
  • After 11 iteration, we've done most data
  • Finally check how much memory in the BP each database is using

If RG were to govern the BP, we expect to see database a using lots of more pages than database b. That not what we are seeing. You will probably see that database b is using slightly more memory than a. That is because you done exactly the same queries over exactly the same data for each database and the most recent database you queried was database b (pushing out some of the pages with same cost that database a had in cache). You might want to execute the loop a few times to possibly see things even out a bit.

Seeing is believing, they say, and to me this show that Resource Governor do not govern the Buffer Pool.

If you believe there are faults in how the script does things, feel free to discuss. If you have some alternative script you want us to try, I suggest you open a thread in the MSDN forum (for example) and there post a script which is pretty immediately executable for us(possibly with modifications of file path and server name). Unless the changes you propose are so minor that can be immediately explained in your text.

Upgrading from Express to Standard edition


Let you encounter an SQL Server which is Express Edition, and it really should have been some higher edition. Sounds familiar? It is common for me as a consultant to find plenty of SQL Servers at a customer's site. Many of the databases in these will be moved (typically using backup and restore) to a "real" SQL Server. But in some cases, this might not be desirable. You want to convert the whole instance, from Express to a "real" SQL Server edition.

Selecting "Standalone", SharePoint install will not only install all SharePoint parts on a single server, it will also install SQL Server Express Edition as the database engine. In short, SharePoint is one example where you might have an existing Express Edition which you want to make a "real" (higher) edition. There are other such scenarios, of course.

As of SQL Server 2008, you can let SQL Server's setup program change from one edition to some other edition by just selecting "Maintenance", "Edition Upgrade" in setup and specify the product key that came with the install media for the desired edition (which you have acquired first, of course). The cool thing here is that setup basically just registers the different product key and WHAM - you have the higher edition! This is what I wanted to test, and specifically going from Express to a higher edition. Below is my experience:

I have an Express edition which came with the SharePoint installation. It is SQL Server 2008 (not R2), with no tools (SSMS etc) installed. I want to make this a "real" Standard Edition. I have a Standard Edition 2008 installation media, with a product key.

The database engine 
I first ran setup from the installed SQL Server (Start, All Programs, Microsoft SQL Server 2008, Configuration Tools, SQL Server Installation Center (64-bit)). I selected "Maintenance", "Edition Upgrade" in setup and specify the product key that came with my standard Edition. This did not allow me to continue. I got an error (which is lost now) and I could not continue.
Next try was to run setup from the installation media (my standard Edition install media) instead. This did allow me to continue. The change was very quick (a couple of minutes). I then verified using SQLCMD that I now have Standard Edition. Great! The database engine done.

Tool-time
You might want to have the tools (a.k.a. Shared Components) as well on the machine. I refer to things such as SQL Server Management Studio, Integration Services etc. So, I just ran setup from the installation media and installed the desired shared components. Nothing strange here. I now have Management Studio, Books Online and whatever more I might want to have.

SQL Server Agent
SQL Server Express doesn't come with SQL Server Agent, right? Not really. It comes with agent, but the service is disabled and you cannot start it even if you try. But what about after an edition upgrade? I could still not start Agent. However, checking SQLAGENT.OUT revealed the problem. The first thing agent does at startup is to login to SQL Server and verify that it is sysadmin. The SharePoint setup installed Agent service account as Network Service, which was added as a login in SQL Server, but the login isn't sysadmin. Now, this is in general a good thing (don't run your services with higher privileges than necessary), but in this case it causes Agent to not start. Some of the SharePoint services are configured to use Network Service, and apparently the server roles securityadmin and dbcreator is enough for SharePoint. So, this needs to be taken care of. I could use some other service account for Agent, and make sure that service account is sysadmin in SQL Server. For my test, though, it is OK to make the login in SQL Server for Network Service as sysadmin. After that was done, Agent started just as expected. And just for fun, I created a Maintennance Plan for all databases, which also worked just fine.

What about the rest?
Obviously, you might want to check other things, such as database options, file size, auto grow settings, maintenance for the databases etc. But that isn't strictly related to the edition change, it is just general best practices. For my case, for instance, I might want to upgrade the SQL Server from 2008 to 2008 R2, while I'm at it. Specifically for SharePoint, it seems its setup program created the database files in the Hive (a SharePoint term for a folder in Program Files, Common Files), and you probably want to move the SharePoint database files to a more suitable location. But that is outside the scope for this post, and hopefully no sweat for your dba.

In short, my test show that is is easy to change from one edition to a different edition, even if you want to change from Express Edition.
 

Point in time recovery using Database backups:


 
SQL Server 2005 added the STOPAT option for the RESTORE DATABASE command. This sounds great - we can stop at some point in time during the database backup process was running! Or? No, we can't. Here follows some tech stuff why not, and then what the option is really meant for:

A database backup includes all used extents and also all log records that were produced while the backup process was running (possibly older as well, to handle open transactions). When you restore such a backup, SQL Server will from backup media copy the data pages to mdf and ndf file(s), log records to ldf file(s) and then do REDO (roll forward the log records) and possibly also UNDO (rollback open transactions - this is skipped if you specify NORECOVERY option). Why does it need the log records? Because you can do modifications in the database while the database backup is running. After backup process was started (and before it finishes) you can both modify pages not yet copied to backup media, but also pages already copied to backup media. So, the data pages in themselves do not present a consistent state of the database. At restore time, SQL Server uses the log records included in the database backup to "sync" the modifications that were performed while the backup process were running. So, what you end up with is what the database looked like at the end time of the backup process. I'm sure this is well documented somewhere in Books Online so I won't dive further into the tech stuff here.

So, what is the purpose of the STOPAT option for RESTORE DATABASE?

It is to give you an early warning, in case you try to restore a database backup which is too late for a subsequent STOPAT operation for a log restore. Here's an example:

1.    13:00 db backup

2.    13:10 log backup

3.    13:20 log backup

4.    13:30 db backup

5.    13:40 log backup

6.    13:50 log backup

Say you now want to restore to 13:15. Intuitively, you say that you will restore 1, 2 and for 3 you do STOPAT 13:15. Yes, that is right.

But say you are under stress, and perhaps even used some tool which confuses things for you? So, you end up restore 4, 5 and for 6 you try STOPAT 13:15. Clearly, you can't stopat 13:15 for backup number 6 (it only covers 13:40 to 13:50). I.e., restore of 6 gives you an error message. So, you have to re-start the restore process (1, 2 and 3), which could take a long time (if the backups are large). If you had specified STOPAT 13:15 when you restore backup 4, SQL Server would have given you an error immediately, so you wouldn't have wasted valuable time restoring unnecessary backups!

That is the purpose of STOPAT for RESTORE DATABASE.

18 March 2013

To improve the I/O performance


The subject of this month is IO. Here is my quick blog post on how Cover Index can Improve Performance by Reducing IO.
Let us kick off this post with disclaimers about Index. Index is a very complex subject and should be exercised with experts. Too many indexes, and in particular, too many covering indexes can hamper the performance. Again, indexes are very important aspect of performance tuning. In this post, I am demonstrating very limited capacity of Index. We will create covering index for query and see how its performance improves as IO reduces. Please note that do not run this scripts on production server as they contain DBCC commands which can create performance issues. This script is created for demo purpose only.
Let us run the following query and see the DBCC outcome along with the enabling of actual execution plan. Estimated Execution plan may work but I have used actual execution plan in this demo.
USE [AdventureWorks]
GO
SET STATISTICS IO ON
GO
-- Clean the buffers
-- Do not run these DBCC on production server
DBCC FREEPROCCACHE
DBCC DROPCLEANBUFFERS
GO
-- Run Select Statement
SELECT ProductID, ProductNumber, Color
FROM Production.Product
WHERE ProductID < 500
GO
The result of the statistics is as follows.
Table 'Product'. Scan count 1, logical reads 6, physical reads 3, read-ahead reads 16, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
The result of the execution plan mouseover displays many different information. Please pay attention to IO cost.


























Now let us create covering index on the query which we have just ran. We will include all the columns from the select statement in our newly created index.
-- Create Covering Index
IF EXISTS (SELECT * FROM sys.indexes WHERE OBJECT_ID =OBJECT_ID(N'[Production].[Product]') AND name = N'AK_Product_Cover')
DROP INDEX [AK_Product_Cover] ON [Production].[Product]
GO
CREATE UNIQUE NONCLUSTERED INDEX [AK_Product_Cover] ON [Production].[Product]
(
ProductID, ProductNumber, Color
) ON [PRIMARY]
GO
Now as cover index is created, let us run the above query once again. I will keep the actual execution plan and Statistics IO on.
-- Clean the buffers
DBCC FREEPROCCACHE
DBCC DROPCLEANBUFFERS
GO
-- Run the following Select
-- Use the hint if the query does not use the index only
SELECT ProductID, ProductNumber, Color
FROM Production.Product --WITH (INDEX(AK_Product_Cover))
WHERE ProductID < 500
GO
The above query will pick up the newly created index right away. If due to any reason, it does not use our index for demo purpose, you can use hint to use that index. Let us examine the results once the query runs.
Table 'Product'. Scan count 1, logical reads 3, physical reads 1, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
The result of the execution plan mouseover displays many different information. Please pay attention to IO cost.





























Let us compare the statistics here first.
Before Cover Index: IO Cost 0.006088
logical reads 6, physical reads 3, read-ahead reads 16,
After Cover Index: IO Cost 0.0038657
logical reads 3, physical reads 1, read-ahead reads 0
Logical read is almost half and Physical reads are much reduced after the creation of  cover index. Additionally, read-ahead reads are reduced with big numbers as well. Now when comparing execution plans, the IO cost is reduced to almost half after creating the covering index.
It is quite clear from this demo that IO is reduced due to cover index.
Let us compare the performance of the both the queries here. I have forced the indexes on them to simulate their original situation.
-- Comparing Performance
SELECT ProductID, ProductNumber, Color
FROM Production.Product WITH (INDEX(PK_Product_ProductID))
WHERE ProductID < 500
GO
SELECT ProductID, ProductNumber, Color
FROM Production.Product WITH (INDEX(AK_Product_Cover))
WHERE ProductID < 500
GO
It is quite clear from this demo that IO is reduced due to cover index, and additionally, the performance improves.













Run following clean up code to drop newly created objects.
-- Clean up
SET STATISTICS IO OFF
GO
IF EXISTS (SELECT * FROM sys.indexes WHERE OBJECT_ID =OBJECT_ID(N'[Production].[Product]') AND name = N'AK_Product_Cover')
DROP INDEX [AK_Product_Cover] ON [Production].[Product]
GO