SQL Server Database Load Balancing

We have an enterprise application using Microsoft SQL Server as a database. We come across several examples where the client turned the application into a huge db, and some of the running requests cause locks and performance problems for themselves and other users.

We tried to apply as many indexes as possible, and perforated all the queries too, but we have one application that should correspond to many different types of clients, so it is difficult to create one solution that suits everyone. We do not have the resources to apply indexing / performance for a particular client for each client.

We know that the main issues that cause problems are those that are generated for reporting and kpi.

My question is, is there a way to spread the load on the application, so everyday use does not interfere with the / kpi report generation. That is, can we somehow mirror db so that daily activities are sent to SQL Entity A and intense data queries are sent to SQL Entity B? Therefore, intensive data queries do not affect the day-to-day side of things, and we can queue the queries to SQL Entity B.

In this scenario, SQL Entity A and B should be constantly maintained in alignment, but SQL Entity B will always be read-only.

Could there be any suggestions that we could try to follow? Or is there another method that I have to look at in order to succeed in performance.

thanks

+7
database sql-server indexing sql-server-2008 enterprise
source share
6 answers

It seems you can go with any replication option and be fine. In one of my previous works, we used Log Shipping http://technet.microsoft.com/en-us/library/ms187103.aspx for this purpose.

You can also work through the types of replication: http://technet.microsoft.com/en-us/library/bb677158.aspx and see which one is better for you, because you can do more than just report on a secondary database .

If I remember my initial experience correctly, Log Shipping is very easy to set up, so you can start there.

+3
source share

Ahh ... performance tuning SQL Server et. et al. my favorite thing!

Can anyone suggest any ways we could try to follow?

From the information you provided, I would vertically split the data. The value supports one database (server A) for actual OLTP (CRUD transactions) and one for KPI (server B).

For replication, I would use transactional replication - with proper operation, latency will be no more than one second. I cannot come up with a practical scenario when this is inappropriate. Indeed, most reports are done before the end of the previous day, and “real-time” usually means the last 5 minutes

To manage the replication process, I would start with a simple console application, expecting it to meet the requirements accordingly. The console application should use the following namespaces (for other reasons, maybe they are available for SQL2012)

using Microsoft.SqlServer.Management.Common; using Microsoft.SqlServer.Management.Smo; using Microsoft.SqlServer.Replication; 

Using the console application, you can manage the publication, subscription and any trace tokens in one interface. This will be the PAIN to configure (all these permissions, passwords and paths), but after it starts, you can optimize the transaction database for the data and the report server for ... reports.

I would have a replication topology, which was actually one subscription per table for large tables and one subscription for the rest (search tables, view sp's). I would replicate primary keys, but not restrictions, table references, triggers (relying on db source for integrity). You also do not have to copy indexes - you can manually configure / optimize them for the report server.

You can also choose which articles are suitable for KPI, i.e. (no need to replicate text, varchar (max), etc.)

Listed below are some helper functions to get you going.

Or is there another method that I should pay attention to performance wins?

In my humble experience, there is ALL that can be done to increase productivity. It comes down to saving time → cost →. Sometimes a small compromise in functionality will bring you great performance benefits.

The devil is in the details, but with that caveat ...

Other random thoughts

You have identified one of your infrastructure problems - mixing OLTP and BI / Reporting. I do not understand your experience and how bad your performance problems are, so when replication is definitely the right way, if you are in the "fire fighting" mode, you can try.

  • Server-side KPI caching (5min, 1hr, 1day?) In db or RAM
  • Using the views associated with the schema, you can create indexed views (in standard and enterprise versions). Depending on the type of KPI, this may even be all you need to do! See http://msdn.microsoft.com/en-us/library/ms191432.aspx to find out more. Essentially, if your KPIs are a sum / group, you should look good.
  • One-night preliminary calculation of daily KPIs. Then you can add only current day data.
  • Sorting is expensive KPIs due to order by clauses. Make sure your cluster indexes are correct (REM: they should not exist on the primary key). Try sorting on the client when you have data.
  • The size of the clustered index. Less is better. Start here if you are using a GUID.
  • Separate the data vertically - for example, if you have a table of 200 columns, but KPI use only 10 columns - put 10 in another table - you will get more data to read the I / O page (it will work if your disk is a bottleneck).
  • Offer the functionality of "Send reports by e-mail" - take away the character in real time. You may be able to deliver% of the age of the reports overnight when the situation is calmer, and during the day will have a lower volume of reports in real time. Some customers may really prefer this feature.
  • Make payment to your customers for reports! "Just enter your credit card information here ..." is the sure way to reduce the number of reports :)

Additional information about your config would be helpful - when you say huge, how huge? How big / what type of drives, what is the RAM specification, etc. - the reason I ask is ... you can spend the next 40 man-days (with a setting of $ 500 / day?) - that would buy you quite a bit of hardware! - more RAM, more disks, faster disks - SSD for temporary or index partitions. Put another way ... you can request too much hardware (and your boss asks you too much)

Next, you describe an enterprise application; these are Enterpise SQL Server licenses. If so, you're in luck - you can create schemas for related kinds of partitions and delegate requests to the "correct" server. There are problems with this model, although, namely, it joins, but it gives you an effective alternative.

Replication code

I knew it was somewhere. Find below some of the helper functions for RMO that you might find useful when starting replication. At some point in the past, it was live code, but probably longer than I would like to think - please treat it as pseudo.

(PS I would be happy to contact you directly if you want)

  public static class RMOHelper { public static void PreparePublicationDb(MyServer Src, MyServer Dist) { ReplicationDatabase publicationDb = new ReplicationDatabase(Src.Database, Src.ServerConnection); if (publicationDb.LoadProperties()) { if (!publicationDb.EnabledTransPublishing) { publicationDb.EnabledTransPublishing = true; } // If the Log Reader Agent does not exist, create it. if (!publicationDb.LogReaderAgentExists) { // Specify the Windows account under which the agent job runs. // This account will be used for the local connection to the // Distributor and all agent connections that use Windows Authentication. publicationDb.LogReaderAgentProcessSecurity.Login = Dist.WinUId; publicationDb.LogReaderAgentProcessSecurity.Password = Dist.WinPwd; // Explicitly set authentication mode for the Publisher connection // to the default value of Windows Authentication. publicationDb.LogReaderAgentPublisherSecurity.WindowsAuthentication = true; // Create the Log Reader Agent job. publicationDb.CreateLogReaderAgent(); DeleteJobAgentSchedule(publicationDb.LogReaderAgentName); } } else { throw new ApplicationException(String.Format( "The {0} database does not exist at {1}.", publicationDb, Src.ServerName)); } } public static TransPublication PrepareTransPublication(MyServer Src, MyServer Dist, string publicationName) { // Set the required properties for the transactional publication. TransPublication publication = new TransPublication(); publication.ConnectionContext = Src.ServerConnection; publication.Name = publicationName; publication.DatabaseName = Src.Database; if (publicationName == "relation") { float d = 0; } // Specify a transactional publication (the default). publication.Type = PublicationType.Transactional; publication.ConflictRetention = 4; publication.RetentionPeriod = 72; // Activate the publication so that we can add subscriptions. publication.Status = State.Active; // Enable push and pull subscriptions and independent Distribition Agents. publication.Attributes = PublicationAttributes.AllowPull|PublicationAttributes.AllowPush|PublicationAttributes.IndependentAgent; //publication.Attributes &= PublicationAttributes.AllowSyncToAlternate; // Specify the Windows account under which the Snapshot Agent job runs. // This account will be used for the local connection to the // Distributor and all agent connections that use Windows Authentication. publication.SnapshotGenerationAgentProcessSecurity.Login = Dist.WinUId; publication.SnapshotGenerationAgentProcessSecurity.Password = Dist.WinPwd; // Explicitly set the security mode for the Publisher connection // Windows Authentication (the default). publication.SnapshotGenerationAgentPublisherSecurity.WindowsAuthentication = true; publication.SnapshotGenerationAgentProcessSecurity.Login =Dist.WinUId; publication.SnapshotGenerationAgentProcessSecurity.Password = Dist.WinPwd; publication.AltSnapshotFolder = @"\\192.168.35.4\repldata\"; if (!publication.IsExistingObject) { // Create the transactional publication. publication.Create(); // Create a Snapshot Agent job for the publication. publication.CreateSnapshotAgent(); // DeleteJobAgentSchedule(ByVal jobID As Guid) As Boolean } else { //throw new ApplicationException(String.Format( // "The {0} publication already exists.", publicationName)); } return publication; } public static TransArticle PrepareTransArticle(TransPublication TransPub, Happy.MI.Replication.Article Article) { TransArticle TransArticle = new TransArticle(); TransArticle.ConnectionContext = TransPub.ConnectionContext; TransArticle.Name = Article.Name; TransArticle.DatabaseName = TransPub.DatabaseName; TransArticle.SourceObjectName = Article.Name; TransArticle.SourceObjectOwner = "dbo"; TransArticle.PublicationName = TransPub.Name; //article.Type = ArticleOptions.LogBased; //article.FilterClause = "DiscontinuedDate IS NULL"; // Ensure that we create the schema owner at the Subscriber. if (TransArticle.IsExistingObject) { //do somethinbg?? } else { TransArticle.SchemaOption |= CreationScriptOptions.Schema; TransArticle.SchemaOption |= CreationScriptOptions.AttemptToDropNonArticleDependencies; if (!Article.ObjectType.HasValue) { throw new Exception(string.Format("unknown schema object type for trans article {0}", Article.Name)); } if (Article.ObjectType.Value== DataAccessADO.ObjectType.USER_TABLE) { TransArticle.SchemaOption |= CreationScriptOptions.ClusteredIndexes; TransArticle.SchemaOption |= CreationScriptOptions.DriChecks; TransArticle.SchemaOption |= CreationScriptOptions.DriDefaults; TransArticle.SchemaOption |= CreationScriptOptions.DriPrimaryKey; TransArticle.SchemaOption |= CreationScriptOptions.DriUniqueKeys; //TransArticle.SchemaOption |= CreationScriptOptions.ExtendedProperties; //TransArticle.SchemaOption |= CreationScriptOptions.NonClusteredIndexes; TransArticle.Type = ArticleOptions.LogBased; TransArticle.AddReplicatedColumns(Article.IncludedColumns.ToArray()); } else if (Article.ObjectType.Value == DataAccessADO.ObjectType.VIEW) { TransArticle.Type= ArticleOptions.ViewSchemaOnly; } else if (Article.ObjectType.Value == DataAccessADO.ObjectType.SQL_SCALAR_FUNCTION) { TransArticle.Type = ArticleOptions.FunctionSchemaOnly; } else if (Article.ObjectType.Value == DataAccessADO.ObjectType.SQL_STORED_PROCEDURE) { TransArticle.Type = ArticleOptions.ProcSchemaOnly; } else { throw new Exception(string.Format("unsupported schema object type {0}", Article.ObjectType.Value)); } // Create the article. TransArticle.Create(); } return TransArticle; } public static TransSubscription PrepareSubscription(TransPublication TransPub, MyServer Src, MyServer Dest, MyServer Dist) { // Define the push subscription. //TransPullSubscription subscription = new TransPullSubscription(); //subscription.ConnectionContext = Dest.ServerConnection; //subscription.PublisherName = Src.ServerName; //subscription.PublicationName = TransPub.Name; //subscription.PublicationDBName = Src.Database; //subscription.DatabaseName = Dest.Database; TransSubscription subscription = new TransSubscription(); subscription.ConnectionContext = TransPub.ConnectionContext; subscription.PublicationName = TransPub.Name; subscription.DatabaseName = TransPub.DatabaseName; subscription.SubscriptionDBName = Dest.Database; subscription.SubscriberName = Dest.ServerName; subscription.LoadProperties(); //subscription.Remove(); // Specify the Windows login credentials for the Distribution Agent job. subscription.SynchronizationAgentProcessSecurity.Login = Dist.WinUId; subscription.SynchronizationAgentProcessSecurity.Password = Dist.WinPwd; if(!subscription.IsExistingObject){ // Create the push subscription. // By default, subscriptions to transactional publications are synchronized // continuously, but in this case we only want to synchronize on demand. subscription.AgentSchedule.FrequencyType = ScheduleFrequencyType.Continuously; subscription.Create(); PrepareSnapshot(TransPub, Src, Dist); } return subscription; } public static void PrepareSnapshot(TransPublication TPub, MyServer Src, MyServer Dist) { SnapshotGenerationAgent agent = new SnapshotGenerationAgent(); agent.Distributor = Dist.ServerName; agent.DistributorSecurityMode = SecurityMode.Standard; agent.DistributorLogin = Dist.SQLUId; agent.DistributorPassword = Dist.WinPwd; agent.Publisher = TPub.SqlServerName; agent.PublisherSecurityMode = SecurityMode.Standard; agent.PublisherLogin = Src.SQLUId; agent.PublisherPassword = Src.SQLPwd; agent.Publication = TPub.Name; agent.PublisherDatabase = TPub.DatabaseName; agent.ReplicationType = ReplicationType.Transactional; // Start the agent synchronously. agent.GenerateSnapshot(); } public static void ApplySubscription(Happy.MI.Replication.Subscription _subscription) { Happy.MI.Replication.Publication p = _subscription.Publication; RMOHelper.PreparePublicationDb(_subscription.Publication.Src, _subscription.Publication.Dist); TransPublication TransPub = RMOHelper.PrepareTransPublication(p.Src, p.Dist, p.PublicationName); foreach (Happy.MI.Replication.Article a in p.Articles) { a.LoadProperties(); TransArticle ta = RMOHelper.PrepareTransArticle(TransPub, a); ta.ConnectionContext.Disconnect(); } TransSubscription TransSub = RMOHelper.PrepareSubscription(TransPub, p.Src, _subscription.Dest, p.Dist); if (TransSub.LoadProperties() && TransSub.AgentJobId == null) { // Start the Distribution Agent asynchronously. TransSub.SynchronizeWithJob(); } TransSub.ConnectionContext.Disconnect(); //foreach (Happy.MI.Replication.Subscription s in p.Subscriptions) //{ // TransSubscription TransSub = RMOHelper.PrepareSubscription(TransPub, p.Src, s.Dest, p.Dist); // if (TransSub.LoadProperties() && TransSub.AgentJobId == null) // { // // Start the Distribution Agent asynchronously. // TransSub.SynchronizeWithJob(); // } // TransSub.ConnectionContext.Disconnect(); //} //TransPub.ConnectionContext.Disconnect(); } public static void Create(Happy.MI.Replication.Publication p) { RMOHelper.PreparePublicationDb(p.Src, p.Dist); TransPublication TransPub = RMOHelper.PrepareTransPublication(p.Src, p.Dist, p.PublicationName); foreach (Happy.MI.Replication.Article a in p.Articles) { a.LoadProperties(); RMOHelper.PrepareTransArticle(TransPub, a); } foreach (Happy.MI.Replication.Subscription s in p.Subscriptions) { TransSubscription TransSub = RMOHelper.PrepareSubscription(TransPub, p.Src, s.Dest, p.Dist); if (TransSub.LoadProperties() && TransSub.AgentJobId == null) { // Start the Distribution Agent asynchronously. TransSub.SynchronizeWithJob(); } } } private static void DeleteJobAgentSchedule(string s) { // Private Function DeleteSchedule(ByVal scheduleID As Integer) As Boolean // Dim result As Boolean // If (scheduleID > 0) Then // Dim msdbConnectionString As String = Me.PublicationConnectionString.Replace(String.Format("Initial Catalog={0};", Me.PublicationDbName), "Initial Catalog=msdb;") // Dim db As New SQLDataAccessHelper.DBObject(msdbConnectionString) // '-- Delete Job Schedule // Dim parameters As New List(Of System.Data.SqlClient.SqlParameter) // parameters.Add(New System.Data.SqlClient.SqlParameter("@schedule_id", SqlDbType.Int)) // parameters.Add(New System.Data.SqlClient.SqlParameter("@force_delete", SqlDbType.Bit)) // parameters(0).Value = scheduleID // parameters(1).Value = True // Dim rowsAffected As Integer // result = (db.RunNonQueryProcedure("sp_delete_schedule", parameters, rowsAffected) = 0) // db.Connection.Close() // db.Connection.Dispose() // Else // Throw New ArgumentException("DeleteSchedule(): ScheduleID must be greater than 0") // End If // Return result //End Function } public static int PublicationEstimatedTimeBehind(Happy.MI.Replication.Subscription s) { PublicationMonitor mon = new PublicationMonitor(); mon.DistributionDBName = s.Publication.Dist.Database; mon.PublisherName = s.Publication.Src.ServerName; mon.PublicationDBName = s.Publication.Src.Database; mon.Name = s.Publication.PublicationName; mon.ConnectionContext = s.Publication.Src.ServerConnection; DataSet ds1 = mon.EnumSubscriptions2(SubscriptionResultOption.AllSubscriptions); ds1.WriteXml(@"c:\desktop\ds1.xml"); //ds.Tables[0].ToString(); if (mon.LoadProperties()) { PendingCommandInfo pci = mon.TransPendingCommandInfo(s.Dest.ServerName, s.Dest.Database, SubscriptionOption.Push); return pci.EstimatedTimeBehind; } else { throw new Exception(string.Format("Unable to load properties for subscription [{0}][{1}]",s.Dest.ServerName, s.Publication.PublicationName)); } } public static int TraceTokenPost(Happy.MI.Replication.Subscription s) { TransPublication TransPub = new TransPublication(); TransPub.ConnectionContext = s.Publication.Src.ServerConnection; TransPub.Name = s.Publication.PublicationName; TransPub.DatabaseName = s.Publication.Src.Database; if (TransPub.LoadProperties()) { return TransPub.PostTracerToken(); } return 0; } public static bool TraceTokenReceive(Happy.MI.Replication.Subscription s, int TokenId){ PublicationMonitor mon = new PublicationMonitor(); mon.DistributionDBName = s.Publication.Dist.Database; mon.PublisherName = s.Publication.Src.ServerName; mon.PublicationDBName = s.Publication.Src.Database; mon.Name = s.Publication.PublicationName; mon.ConnectionContext = s.Publication.Src.ServerConnection; if (mon.LoadProperties()) { DataSet ds= mon.EnumTracerTokenHistory(TokenId); int latency; string str = ds.Tables[0].Rows[0]["overall_latency"].ToString(); bool res = int.TryParse(str, out latency); return res; } else { throw new Exception(string.Format("Unable to load properties for subscription [{0}][{1}]", s.Dest.ServerName, s.Publication.PublicationName)); } } public static void Cmd(string cnct) { string script = System.IO.File.ReadAllText(@"C:\tfs\CA\Dev\MI\Happy.MI\PostReplicationScripts\GC1.txt"); SqlConnection connection = new SqlConnection(cnct+";Connection Timeout=5"); Server server = new Server(new ServerConnection(connection)); //server.ConnectionContext.InfoMessage += new System.Data.SqlClient.SqlInfoMessageEventHandler(ConnectionContext_InfoMessage); server.ConnectionContext.ExecuteNonQuery(script); server.ConnectionContext.Disconnect(); } } 
+2
source share

You can look in partitioned tables, dividing the data so that reporting / BI operations do not affect your daily OLTP performance. It can also save some valuable time when you need to clear old data.

+1
source share

Take a look at ScaleArc. This is the SQL Connection Manager, which balances the load by partitioning read and write across multiple instances. That means you have to sign do do replication ..

+1
source share

I would say, divide the problem into smaller parts before deciding any approach to solving it.

Database mirroring, replication, and other high availability or DR functions for SQL SERVER are there when you really need them. But these features also do not provide 100% real-time synchronization. As another experienced database administrator has already mentioned, you need to schedule "Planned downtime" and / or "Delay in a few minutes", and also configure client expectations accordingly if you go to these settings.

Again, these functions may occur as biased problems, but did not really solve it unless we first look at the root cause. The sentences below will look like general statements, but this is a question. Your problem is too wide to cover, and at first it takes a lot of areas for disclosure before someone can answer.

Now what I wanted to point out poses small questions to the problem.

You mentioned:

"some of the running queries cause blockages and performance problems for themselves and other users"

  Are these queries blocking other reads and/or write? If both lets handle separately. Ideally any read should not be blocked by other read/write. What type of ISOLATIONLVEL you have in DB? If "READ COMMITED", or any other more strict level than think about SNAPSHOT ISOLATION. Does the queries have lot of table and/or index hints in them ? Try not to optimize queries by hints as first option instead treat as last option. if issue is blocking of write/write then couple of point you can consider. Does write queries written properly to acquires appropriate locks. (due to the table hints if any) Have you look at the server configurations MAX memory/Thread/DOP/AUTO Statistics Async? Can the large insert/update be queued from APP tier ? Can the large insert/update be chunked in smaller batch ? Can the large insert/update be executed as Asynchronous operation ? Can you take advantage of Partitioning in database? 

All of the above “Possible Questions” require more input, depending on your first answer. If the database or code was not originally designed to host such a function, then the right time to start thinking about it.

  In terms of performance are you seeing write is getting slower or read ? What is causing slowness? CPU, Memory, DISK IO, DB properties, Too many Object Recompilations ? Have you extracted the Execution plan for identified main queries ? If queries are very long and complex instead look for how can we simplify/optimize the logic ? Does Tables are overly indexed ? Write operation can suffer severely if tables are over optimized for read by adding lot of indexes. See the index fragmentation and statistics. You must have the db maintenance plan in place. How soon the indexes become fragmented again. Is there lot of aggregations and calculation in the query that runs frequently? If query has lot of aggregations/UDF's/Views that runs frequently we can also find out if we can store semi aggregated data separately. Does the reporting query retrieves lot of data? If the queries serving results to report they may end up being retuning thousands of rows. Think about what does user do with this much of results on UI? Does it really necessary ? if not can we limit the result set to return certain number of row based on user settings. If yes, can we implement PAGING for this queries. (that can be controlled by setting as well) If this much of data is feeding another sub-system (SSRS) then anyways any reporting tool will have some latency depending on how much data we are dumping in front of user. 

“We tried to apply as many indexes as possible and perforated all the queries to the limit, but we have one application that should correspond to many different types of clients, so it’s difficult to create one solution that suits everyone. We don’t have the resources to apply indexing / performance to a particular customer for each customer.

  Can we find out customizations and think about how can we implement separately? This is a huge piece by itself. But I am telling from my own experience that it took us almost an year to transform our DB design that can accommodate over 300 clients w/o worrying how one customization will affect other Client custom functionality and/or our core product features. If you can manage to get a right plan laid out first, sure you can get resources to accomplish that. 

"We know that the main issues that cause problems are those that are generated for reporting and kpi."

  How many tables this queries covers ? If numbers less 30% of DB, then instead of whole DB, we should think around these tables and queries only. If you find any/some of above points you haven't visited yet do so. You will find very simple things that can save you lot. It is better to look at root of the problem instead covering/overcoming it temporarily by using alternatives. Many of the DBAs and Developer on this community will be happy to assist you for "END-TO-END Resolution" or "help as needed" . 
0
source share

You can replicate snapshots, use one database for production, and another for reporting. Move the reporting index to the reporting database and save the rest necessary for the application in the database used by the application.

0
source share

All Articles