Monday, December 10, 2012

Autofac beta and dependencies

I created a projected that used a new version of autofac installed via nuget
Install-Package Autofac -Pre
Everything was working fine until I deployed the project to a production. And then, I saw an error:
Unhandled Exception: System.IO.FileLoadException: Could not load file or assembly 'System.Core, Version=2.0.5.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e, Retargetable=Yes' or one of its dependencies. The given assembly name or code
base was invalid. (Exception from HRESULT: 0x80131047)
   at Autofac.Builder.RegistrationData..ctor(Service defaultService)
   at Autofac.Builder.RegistrationBuilder`3..ctor(Service defaultService, TActiv
atorData activatorData, TRegistrationStyle style)
   at Autofac.Builder.RegistrationBuilder.ForType[TImplementer]()
   at Autofac.RegistrationExtensions.RegisterType[TImplementer](ContainerBuilder
 builder)
   at TTC.ContentHubDataCache.ContainerSetup.BuildContainer() in C:\Development\
BrandWebsites\TrafalgarTools\TTC.ContentHubDataCache\TTC.ContentHubDataCache\Con
tainerSetup.cs:line 26
   at TTC.ContentHubDataCache.UpdateDataCacheProcess..ctor() in C:\Development\B
randWebsites\TrafalgarTools\TTC.ContentHubDataCache\TTC.ContentHubDataCache\Upda
teDataCacheProcess.cs:line 58
   at TTC.ContentHubDataCache.Program.Main(String[] args) in C:\Development\Bran
dWebsites\TrafalgarTools\TTC.ContentHubDataCache\TTC.ContentHubDataCache\Program
.cs:line 9
It looked like autofac is referencing a System.Core in a really old version. A quick look at Autofac.dll dependencies in ILDASM under MANIFEST section showed that:
.assembly extern retargetable System.Core
{
  .publickeytoken = (7C EC 85 D7 BE A7 79 8E )                         // |.....y.
  .ver 2:0:5:0
}
The beta version of autofac (Autofac 3.0.0-beta) is using an old System.Core, it is build against .NET in a version 4.0 but yet it is using System.Core in version 2.0, how bizarre. I uninstalled autofac in this version, and took an older one
uninstall-package autofac
Install-Package Autofac -Version 2.6.3.862
A quick check at dependencies
.assembly extern System.Core
{
  .publickeytoken = (B7 7A 5C 56 19 34 E0 89 )                         // .z\V.4..
  .ver 4:0:0:0
}
Looks good, and it solved my problem, but why they used a System.Core in version 2.0.5.0 I do not know, probably they haven't noticed yet.

Monday, December 3, 2012

Measuring SQL Query execution time

It is not recommended to measure SQL execution time on a DB, wise guys believe that it is much more meaningful to run performance tests from an application, so the response time will also include a network delay, and SQL provider computation time. In my scenario, I do not have an access to an application, and there are many entry points for a one SQL Query. That is why it is optimized on a DB side. A query below displays performance metrics of a query, in my example I am interested with each query that includes 'VersionedFields' string.
select top 40 * from sys.dm_exec_query_stats qs CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) AS qt where qt.text like '%VersionedFields%' order by last_execution_time desc
'Elapsed time' is a most important metrics for me. Much more low level way to measure a query response time is to use STATISTICS
DBCC DROPCLEANBUFFERS
SET STATISTICS IO ON 
GO
SET STATISTICS TIME ON
GO

-- SQL Query goes here like, SELECT * FROM VersionedFields

DBCC DROPCLEANBUFFERS
SET STATISTICS IO OFF
GO
SET STATISTICS TIME OFF
GO
Notes regarding DROPCLEANBUFFERS. I only run it on a test bed, not on a production.
Use DBCC DROPCLEANBUFFERS to test queries with a cold buffer cache without shutting down and restarting the server.
To drop clean buffers from the buffer pool, first use CHECKPOINT to produce a cold buffer cache. This forces all dirty pages for the current database to be written to disk and cleans the buffers. After you do this, you can issue DBCC DROPCLEANBUFFERS command to remove all buffers from the buffer pool.
If one really wants to clear entire cache, a CHECKPOINT should be also used.
[CHECKPOINT] Writes all dirty pages for the current database to disk. Dirty pages are data pages that have been entered into the buffer cache and modified, but not yet written to disk. Checkpoints save time during a later recovery by creating a point at which all dirty pages are guaranteed to have been written to disk.
I do not run checkpoint that often.

Shrinking Log file in SQL Server 2008

Procedure of shrinking DB log file (transaction log file). In SQL Server Management Studio
Right click on a DB, Properties -> Options-> Recovery model -> change from 'Full' to 'Simple' -> OK
Right click on a DB -> Tasks -> Shrink -> Files ->  File type -> Log -> OK
The shrinking procedure should not take more than 3 s. It is not possible to change DB Recovery mode this way if a mirroring is setup. Now, some rules of shrinking and maintaining log file.
  • By default Recovery model is set to FULL
  • If you store in a DB crucial/important information, then recovery model should be set to FULL
  • If recovery model is set to FULL, it means that there should be a backup in place, a backup that also includes a transaction log file
  • When a backup for a transaction log file runs, the transaction log file is shrink. The truncation occurs after a Checkpoint process
  • So if your transaction log is big, like 7 GB, it means that you:
    • Don't have a backup that includes transaction log fie. And it means that you don't need FULL recovery mode
    • Your backup is not working
    • You have a big and heavily used database

Tuesday, November 27, 2012

Host and Dig under Windows

Recently I had some DNS problems, I was looking for a Host and Dig command under windows. There is an excellent port that also includes whois command. I totally forgot that under windows there is nslookup (I am getting too old for this). But I figured out that I know how to do it based on system.net library, so I used PowerShell to resolve a host:
[System.Net.Dns]::GetHostAddresses("www.google.com")
I always keep in my memory a public google DNS server.
google public DNS: 8.8.8.8
But unfortunately there is no way for .NET to specify what DNS server use to resolve a host. The reason for that is because DNS.Resolve method relies on the internal Win32 APIs which in turn go through the DNS servers associated with the network connection. In order to change a DNS server one needs to change and configure a network adapter. Ech... I can tell that I am more a developer then admin :)

Thursday, November 22, 2012

A great query to identify missing indexes and print it as a create index query. They are ordered by a Total Cost
SELECT  TOP 10 
        [Total Cost]  = ROUND(avg_total_user_cost * avg_user_impact * (user_seeks + user_scans),0) 
        , avg_user_impact
        , TableName = statement
        , [EqualityUsage] = equality_columns 
        , [InequalityUsage] = inequality_columns
        , [Include Cloumns] = included_columns
FROM        sys.dm_db_missing_index_groups g 
INNER JOIN    sys.dm_db_missing_index_group_stats s 
       ON s.group_handle = g.index_group_handle 
INNER JOIN    sys.dm_db_missing_index_details d 
       ON d.index_handle = g.index_handle
ORDER BY [Total Cost] DESC;
Where Total Cost stands for:
total cost=(avg_total_user_cost *avg_user_impact *(user_seeks +user_scans))/1000'000
  • avg_total_user_cost – Average cost of the user queries that could be reduced by the index in the group
  • avg_user_impact – Average percentage benefit that user queries could experience if this missing index group was implemented. The value means that the query cost would on average drop by this percentage if this missing index group was implemented
  • user_seeks – Number of seeks caused by user queries that the recommended index in the group could have been used for
  • user_scans – Number of scans caused by user queries that the recommended index in the group could have been used for
Below is a nice query to generate an actual command for an index creation:
PRINT 'Missing Indexes: '
PRINT 'The "improvement_measure" column is an indicator of the (estimated) improvement that might '
PRINT 'be seen if the index was created. This is a unitless number, and has meaning only relative '
PRINT 'the same number for other indexes. The measure is a combination of the avg_total_user_cost, '
PRINT 'avg_user_impact, user_seeks, and user_scans columns in sys.dm_db_missing_index_group_stats.'
PRINT ''
PRINT '-- Missing Indexes --'
SELECT CONVERT (varchar, getdate(), 126) AS runtime,
  mig.index_group_handle, mid.index_handle,
  CONVERT (decimal (28,1), migs.avg_total_user_cost * migs.avg_user_impact * (migs.user_seeks + migs.user_scans)) AS improvement_measure,
  'CREATE INDEX missing_index_' + CONVERT (varchar, mig.index_group_handle) + '_' + CONVERT (varchar, mid.index_handle)
  + ' ON ' + mid.statement
  + ' (' + ISNULL (mid.equality_columns,'')
    + CASE WHEN mid.equality_columns IS NOT NULL AND mid.inequality_columns IS NOT NULL THEN ',' ELSE '' END + ISNULL (mid.inequality_columns, '')
  + ')'
  + ISNULL (' INCLUDE (' + mid.included_columns + ')', '') AS create_index_statement,
  migs.*, mid.database_id, mid.[object_id]
FROM sys.dm_db_missing_index_groups mig
INNER JOIN sys.dm_db_missing_index_group_stats migs ON migs.group_handle = mig.index_group_handle
INNER JOIN sys.dm_db_missing_index_details mid ON mig.index_handle = mid.index_handle
WHERE CONVERT (decimal (28,1), migs.avg_total_user_cost * migs.avg_user_impact * (migs.user_seeks + migs.user_scans)) > 10
ORDER BY migs.avg_total_user_cost * migs.avg_user_impact * (migs.user_seeks + migs.user_scans) DESC
PRINT ''
GO 

Tuesday, November 20, 2012

Thread, Wait and debugging Sitecore under WinDbg

When running a command to display all threads and stack assigned to a thread it is common to see a following picture:
~*e !CLRStack
...
OS Thread Id: 0x10ec (37)
Child-SP         RetAddr          Call Site
000000000c4de8e0 000007ff006e168f Sitecore.IO.FileWatcher.Worker()
000000000c4de930 000007ff0100e510 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
000000000c4de980 000007fef96ceb52 System.Threading.ThreadHelper.ThreadStart()
OS Thread Id: 0x10f8 (38)
...
OS Thread Id: 0x1268 (43)
Child-SP         RetAddr          Call Site
000000000ba5e4e0 000007ff01298285 System.Threading.WaitHandle.WaitAny(System.Threading.WaitHandle[], Int32, Boolean)
000000000ba5e540 000007ff006e168f System.Net.TimerThread.ThreadProc()
000000000ba5e610 000007ff0100e510 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
000000000ba5e660 000007fef96ceb52 System.Threading.ThreadHelper.ThreadStart()
...
OS Thread Id: 0x1368 (46)
Child-SP         RetAddr          Call Site
000000000cf7e5c0 000007ff00f6ed56 System.Threading.Thread.Sleep(System.TimeSpan)
000000000cf7e600 000007ff006e168f Sitecore.Services.Heartbeat.WorkLoop()
000000000cf7e6c0 000007ff0100e510 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
000000000cf7e710 000007fef96ceb52 System.Threading.ThreadHelper.ThreadStart()
...
the process
OS Thread Id: 0x1008 (49)
Child-SP         RetAddr          Call Site
000000000d03e5f0 000007ff0163a0b7 Sitecore.Threading.Semaphore.P()
000000000d03e640 000007ff006e168f Sitecore.Threading.ManagedThreadPool.ProcessQueuedItems()
000000000d03e6a0 000007ff0100e510 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
000000000d03e6f0 000007fef96ceb52 System.Threading.ThreadHelper.ThreadStart()
...
Each of listed above stack traces appear few times in a memory dump. When analyzing any of those stacks there is no magic or suspicious staff. This is how Sitecore was designed and how it works.

PerfMon typical counters

A list of general counters that I usually use. I set up a PerfMon to run in a background and log data.
Web Service->Current Connections->Publishing-shared
Web Service->Get Requests/sec->Publishing-shared
Web Service->Current Anonymous Users->Publishing-shared
System -> Processor Queue Length
Processor -> % Processor Time->_Total
Process -> Working Set->_Total
PhysicalDisk -> Current Disk Queue Length->_Total
PhysicalDisk -> Disk Bytes/sec->_Total
Network Interface-> Bytes Total/sec->[Select network adapter]
Network Interface-> Output Queue Length->[Select network adapter]
Network Interface-> Packets Received Errors->[Select network adapter]
Memory -> Available MBytes
Memory -> Pages Input/sec