Commit 4c1af6c3 authored by carplaptop's avatar carplaptop

removing unneeded posh code

parent 539e54c8
......@@ -3,26 +3,19 @@ PerformanceCollector/PerformanceEngineer-in-a-Box
This utility captures a series of performance metrics and logs them to various tables. For more information see my blog series at: http://www.davewentzel.com/PerformanceCollectorSeries.
**This code is on the succinct branch. This branch has an abbreviated list of features and a simpler installer. **
How to Install Performance Collector
=======================================
Initial deployment and upgrades are handled by the same Powershell script: Deploy-PerformanceCollector.ps1. There is one parameter: $servername. It assumes you are running as an NT acct with sysadmin permissions.
Installation Process
----------------------
Start|Run|cmd.exe (Run As Administrator)
cd /d <folder that contains .ps1 file>
powershell .\Deploy-PerformanceCollector.ps1 "ServerName"
Prerequisites
--------------
1. Ensure SQL Agent is running. If it isn't PerformanceCollector won't work correctly.
2. Ensure you run the installer script as sysadmin.
Signed Scripts Error
----------------------
You may see this error when running Powershell scripts:
1. create the PerformanceCollector database on your server using your preferred file layouts. If you don't do this the Installer script will do this for you using the default data file and log placement.
File Deploy-PerformanceCollector.ps1 cannot be loaded because the execution of scripts is disabled on this system.
In this case run the following and then run the .ps1 file again.
set-executionpolicy unrestricted
How Does Performance Collector Work?
=======================================
......@@ -32,7 +25,7 @@ Everything is installed in the PerformanceCollector schema within the Performanc
How do the AddIns Work?
=================================================
There are a series of procedures named PerformanceCollector.Addin*. PerformanceCollector.RunAddIns is tickled every 5 minutes from ServiceBroker. This proc builds a cursor over ANY entry in PerformanceCollector.Config that is enabled and runs it.
There are a series of procedures named PerformanceCollector.Addin*. PerformanceCollector.RunAddIns is called every 5 minutes from the RunAddIns Job. This proc builds a cursor over ANY entry in PerformanceCollector.Config that is enabled and runs it.
The AddIns procedures are responsible for determining when the given AddIn is actually supposed to run (every 8 hours, once a day, etc). It determines this from querying PerformanceCollector.Config. It then calls the actual worker proc if it needs to.
......@@ -48,6 +41,7 @@ Other Objects
==================
Abandoned Spids
---------------
**This is disabled BY DEFAULT**
This view will also show "abandoned spids." An abandoned spid is any spid with an open transaction for at least 60 seconds that is blocking something else. Here is the most important aspect...the spid is sitting on AWAITING COMMAND. That is what differentiates an abandoned spid from an ordinary blocking spid. When the spid is sitting on AWAITING COMMAND that means the SQL Server is waiting for another command from the spid to do something...most likely issue a COMMIT TRAN.
I have never seen a case where the spid will come back and issue the COMMIT. In every case this is a data access framework that thinks it is NOT in implicit_transaction mode and SQL Server thinks it is. For instance, various combinations of jdbc and websphere produce this behavior because jdbc does not issue sp_reset_connection commands like .NET does.
......@@ -151,14 +145,8 @@ The PerformanceCollector database should never get much larger than 15GB. Once
Next Steps/TO DO
================
MissingIndexesFromQueryPlansSummary and Dtls are not yet implemented. I have been running a version of this outside PerformanceCollector and it displays a lot of good missing index information out of the cached query plans instead of the DMVs. I would think this would be very similar information, but it isn't. I find this to be far more valuable because I can directly see the patterns in the query plans as well as their SQL statements and I can perhaps tweak those statements instead of just going on blind faith that the missing index DMV recommendations will help me.
We need to create an AddIn that will gather this data every x hours (maybe 13 hours).
MissingIndexesFromQueryPlansSummary and Dtls are not yet fully implemented. I have been running a version of this outside PerformanceCollector and it displays a lot of good missing index information out of the cached query plans instead of the DMVs. I would think this would be very similar information, but it isn't. I find this to be far more valuable because I can directly see the patterns in the query plans as well as their SQL statements and I can perhaps tweak those statements instead of just going on blind faith that the missing index DMV recommendations will help me.
Summary will contain the data aggregated by dbid, statement (which is really the object), and "columns". Those will be assigned an ID. We will aggregate the count of statements and the max impact.
The ID will be referenced in Dtls which we can use to see the actual sql_text and query_plan. This will be used to determine what statements the index is intending to help.
I have this stubbed out in the table scripts as well as PerformanceCollector.GatherMissingIndexesFromPlans.sql.
......
SQLPSX module is required to connect to SQL Server from PoSH and run .sql scripts.
\ No newline at end of file
Binary files a/ThirdParty/SQLPSX/Agent/Agent.psd1 and /dev/null differ
This diff is collapsed.
Binary files a/ThirdParty/SQLPSX/ISECreamBasic/Add-IseMenu.ps1 and /dev/null differ
Binary files a/ThirdParty/SQLPSX/ISECreamBasic/ISECreamBasic.psm1 and /dev/null differ
Binary files a/ThirdParty/SQLPSX/ISECreamBasic/Remove-IseMenu.ps1 and /dev/null differ
Microsoft Public License (Ms-PL)
This license governs use of the accompanying software. If you use the software, you accept this license. If you do not accept the license, do not use the software.
1. Definitions
The terms "reproduce," "reproduction," "derivative works," and "distribution" have the same meaning here as under U.S. copyright law.
A "contribution" is the original software, or any additions or changes to the software.
A "contributor" is any person that distributes its contribution under this license.
"Licensed patents" are a contributor's patent claims that read directly on its contribution.
2. Grant of Rights
(A) Copyright Grant- Subject to the terms of this license, including the license conditions and limitations in section 3, each contributor grants you a non-exclusive, worldwide, royalty-free copyright license to reproduce its contribution, prepare derivative works of its contribution, and distribute its contribution or any derivative works that you create.
(B) Patent Grant- Subject to the terms of this license, including the license conditions and limitations in section 3, each contributor grants you a non-exclusive, worldwide, royalty-free license under its licensed patents to make, have made, use, sell, offer for sale, import, and/or otherwise dispose of its contribution in the software or derivative works of the contribution in the software.
3. Conditions and Limitations
(A) No Trademark License- This license does not grant you rights to use any contributors' name, logo, or trademarks.
(B) If you bring a patent claim against any contributor over patents that you claim are infringed by the software, your patent license from such contributor to the software ends automatically.
(C) If you distribute any portion of the software, you must retain all copyright, patent, trademark, and attribution notices that are present in the software.
(D) If you distribute any portion of the software in source code form, you may do so only under this license by including a complete copy of this license with your distribution. If you distribute any portion of the software in compiled or object code form, you may only do so under a license that complies with this license.
(E) The software is licensed "as-is." You bear the risk of using it. The contributors give no express warranties, guarantees or conditions. You may have additional consumer rights under your local laws which this license cannot change. To the extent permitted under your local laws, the contributors exclude the implied warranties of merchantability, fitness for a particular purpose and non-infringement.
\ No newline at end of file
Note: IseCreamBasic is derived from "PowerShell ISE-Cream" http://psisecream.codeplex.com/
The module is redistributed according to the license terms as noted in license.txt
<#
.SYNOPSIS
Uses the .NET OracleBulkCopy class to quickly copy rows into a destination table.
.DESCRIPTION
Also, the Invoke-OracleBulkcopy function allows you to pass a command object instead of a set of records in order to "stream" the records
into the destination in cases where there are a lot of records and you don't want to allocate memory to hold the entire result set.
.PARAMETER records
Either a datatable (like one returned from invoke-query or invoke-storedprocedure) or
A sql command object (use new-sqlcommand)
.PARAMETER TNS
The destination server TNS to connect to.
.PARAMETER User
The sql user to use for the connection. If user is not passed, NT Authentication is used.
.PARAMETER Password
The password for the sql user named by the User parameter.
.PARAMETER Table
The destination table for the bulk copy operation.
.PARAMETER dbaPrivilege
"DBA Privilege" for the connection
.PARAMETER Mapping
A dictionary of column mappings of the form DestColumn=SourceColumn
.PARAMETER BatchSize
The batch size for the bulk copy operation.
.PARAMETER NotifyAfter
The number of rows to fire the notification event after transferring. 0 means don't notify.
Ex: 1000 means to fire the notify event after each 1000 rows are transferred.
.PARAMETER NotifyFunction
A scriptblock to be executed after each $notifyAfter records has been copied. The second parameter ($param[1])
is a SqlRowsCopiedEventArgs object, which has a RowsCopied property. The default value for this parameter echoes the
number of rows copied to the console
.PARAMETER Options
An object containing special options to modify the bulk copy operation.
See http://download.oracle.com/docs/html/E10927_01/OracleBulkCopyOptionsEnumeration.htm#CHDEHFFF for values.
.EXAMPLE
PS C:\> $cmd=new-sqlcommand -server MyServer -sql "Select * from MyTable"
PS C:\> invoke-Oraclebulkcopy -records $cmd -tns MyOtherServer -user myuser -password topsecret -table CopyOfMyTable
.EXAMPLE
PS C:\> $rows=invoke-query -server MyServer -sql "Select * from MyTable"
PS C:\> invoke-Oraclebulkcopy -records $rows -tns MyOtherServer -password topsecret -table CopyOfMyTable
.INPUTS
None.
You cannot pipe objects to Invoke-OracleBulkcopy
.OUTPUTS
None.
#>
function Invoke-OracleBulkcopy{
param([Parameter(Position=0, Mandatory=$true)]$records,
[Parameter(Position=1, Mandatory=$true)]$tns,
[Parameter(Position=2, Mandatory=$false)][string]$user,
[Parameter(Position=3, Mandatory=$false)][string]$password,
[Parameter(Position=4, Mandatory=$true)][string]$table,
[Parameter(Position=5, Mandatory=$false)][string]$dbaPrivilege,
[Parameter(Position=6, Mandatory=$false)]$mapping=@{},
[Parameter(Position=7, Mandatory=$false)]$batchsize=0,
[Parameter(Position=8, Mandatory=$false)]$notifyAfter=0,
[Parameter(Position=9, Mandatory=$false)][scriptblock]$notifyFunction={Write-Host "$($args[1].RowsCopied) rows copied."}
#[Parameter(Position=10, Mandatory=$false)][Oracle.DataAccess.Client.OracleBulkCopyOptions ]$options=[Oracle.DataAccess.Client.OracleBulkCopyOptions ]::Default
)
# I#m not using existing "New-Oracle_connection" function to create a connection string.
# because I do not get back the password
$ConnectionString = "Data Source=$tns;User ID=$user;Password=$password"
Write-host $ConnectionString
if ($dbaPrivilege)
{
$ConnectionString += ";DBA Privilege=$dbaPrivilege"
}
if ($options)
{
$bulkCopy = new-object "Oracle.DataAccess.Client.OracleBulkCopy" $connectionString $options
}
else
{
$bulkCopy = new-object "Oracle.DataAccess.Client.OracleBulkCopy" $connectionString
}
$bulkCopy.BatchSize = $batchSize
$bulkCopy.DestinationTableName = $table
$bulkCopy.BulkCopyTimeout = 10000000
if ($notifyAfter -gt 0){
$bulkCopy.NotifyAfter = $notifyafter
$bulkCopy.Add_OracleRowscopied($notifyFunction)
}
#Add column mappings if they were supplied
foreach ($key in $mapping.Keys){
$bulkCopy.ColumnMappings.Add($mapping[$key],$key) | out-null
}
write-debug "Bulk copy starting at $(get-date)"
if ($records -is [System.Data.Common.DBCommand]){
#if passed a command object (rather than a datatable), ask it for a datareader to stream the records
$bulkCopy.WriteToServer($records.ExecuteReader())
} elsif ($records -is [System.Data.Common.DbDataReader]){
#if passed a Datareader object use it to stream the records
$bulkCopy.WriteToServer($records)
} else {
$bulkCopy.WriteToServer($records)
}
write-debug "Bulk copy finished at $(get-date)"
}
# ---------------------------------------------------------------------------
### <Script>
### <Author>
### Bernd Kriszio
### </Author>
### <Description>
### Defines functions for executing Ado.net queries against Oracle
### </Description>
### <Usage>
### import-module OracleClient
### </Usage>
### </Script>
# ---------------------------------------------------------------------------
[System.Reflection.Assembly]::LoadWithPartialName("Oracle.DataAccess")
[System.Reflection.Assembly]::LoadWithPartialName("System.Data.OracleClient")
. $psScriptRoot\new-connection.ps1
. $psScriptRoot\invoke-query.ps1
. $psScriptRoot\get-commandresults.ps1
. $psScriptRoot\OracleBulkcopy.ps1
export-modulemember new-oracle_connection
# export-modulemember -function invoke-sql
export-modulemember -function invoke-oracle_query, ConvertTo-oracleDataSource, Invoke-OracleBulkcopy
# export-modulemember -function invoke-storedprocedure
Binary files a/ThirdParty/SQLPSX/OracleClient/get-commandresults.ps1 and /dev/null differ
Binary files a/ThirdParty/SQLPSX/OracleClient/invoke-query.ps1 and /dev/null differ
Binary files a/ThirdParty/SQLPSX/OracleClient/new-connection.ps1 and /dev/null differ
function ConvertFrom-Xml($XML) {
foreach ($Object in @($XML.Objects.Object)) {
$PSObject = New-Object PSObject
foreach ($Property in @($Object.Property)) {
$PSObject | Add-Member NoteProperty -Name $Property.Name -Value $Property.InnerText
}
$PSObject
}
}
function ConvertTo-StringData
{
Begin
{
$string = "@{`n"
function Expand-Value
{
param($value)
if ($value -ne $null) {
switch ($value.GetType().Name)
{
'String' { "`"$value`"" }
'Boolean' { "`$$value" }
default { $value }
}
}
else
{ "`$null" }
}
}
Process
{
$string += $_.GetEnumerator() | foreach {"{0} = {1}`n" -f $_.Name,(Expand-Value $_.Value)}
}
End
{
$string += "}"
Write-Output $string
}
} #ConvertTo-StringData
param([String]$text)
$returnedText = ""
foreach ( $line in $text -split [System.Environment]::NewLine ) {
if ( $line.length -gt 0) {
if ($line -match '(\$(\w+))')
#{ $line = $line -replace '(\$(\w+))',(get-variable $matches[2] -Scope Global -ValueOnly) }
{ $line = $line -replace '(\$(\w+))',(get-variable $matches[2] -ValueOnly) }
}
$returnedText += "{0}{1}" -f $line, [System.Environment]::NewLine
}
$returnedText
Binary files a/ThirdParty/SQLPSX/OracleIse/Get-ConnectionInfo.ps1 and /dev/null differ
#######################
function Test-UserStore
{
param([string]$fileName,[string]$dirName)
$userStore = [System.IO.IsolatedStorage.IsolatedStorageFile]::GetUserStoreForAssembly()
if ($userStore.GetDirectoryNames($dirName))
{
if ($userStore.GetFileNames("$dirName\$fileName"))
{ Write-Output $true }
else
{ Write-Output $false }
}
else
{ Write-Output $false }
} #Test-UserStore
#######################
function Initialize-UserStore
{
param([string]$fileName,[string]$dirName,[string]$defaultFile)
if (-not (Test-UserSTore $fileName $dirName))
{
$defaults = &"$defaultFile"
Write-UserStore $fileName $dirName $defaults
}
} #Initialize-UserStore
#######################
function Write-UserStore
{
param([string]$fileName,[string]$dirName,$object)
$userStore = [System.IO.IsolatedStorage.IsolatedStorageFile]::GetUserStoreForAssembly()
if (-not $userStore.GetDirectoryNames($dirName))
{ $userStore.CreateDirectory($dirName) }
try {
#$file = New-Object System.IO.IsolatedStorage.IsolatedStorageFileStream("$dirName\$fileName",[System.IO.FileMode]::OpenOrCreate,$userStore)
if (Test-UserSTore $fileName $dirName)
{ $file = New-Object System.IO.IsolatedStorage.IsolatedStorageFileStream("$dirName\$fileName",[System.IO.FileMode]::Truncate,$userStore) }
else
{ $file = New-Object System.IO.IsolatedStorage.IsolatedStorageFileStream("$dirName\$fileName",[System.IO.FileMode]::OpenOrCreate,$userStore) }
if ($object -is [Hashtable])
{
$string = $object | ConvertTo-StringData
$sw = New-Object System.IO.StreamWriter($file)
$sw.Write($string)
$sw.Close()
}
else
{
$xml = $object | ConvertTo-Xml -noTypeInformation
$xml.Save($file)
}
}
finally {
$file.Close()
$userStore.Close()
}
} #Write-UserStore
#######################
function Read-UserStore
{
param([string]$fileName,[string]$dirName,[string]$typeName)
$userStore = [System.IO.IsolatedStorage.IsolatedStorageFile]::GetUserStoreForAssembly()
try {
$file = New-Object System.IO.IsolatedStorage.IsolatedStorageFileStream("$dirName\$fileName",[System.IO.FileMode]::Open,$userStore)
}
catch {
Write-Error "Cannot open file $dirName\$fileName"
break
}
try {
if ($typeName -eq "Hashtable")
{
$sr = New-Object System.IO.StreamReader($file)
$object = $sr.ReadToEnd()
$sr.Close()
Write-host $object
invoke-expression "$object"
}
else
{
$xmlReader = New-Object System.Xml.XmlTextReader($file)
$xml = New-Object System.Xml.XmlDocument
$xml.Load($xmlReader)
$object = ConvertFrom-Xml -xml $xml
Write-Output $object
}
}
finally {
$file.Close()
$userStore.Close()
}
} #Read-UserStore
#######################
function Remove-UserStore
{
param([string]$fileName,[string]$dirName)
$userStore = [System.IO.IsolatedStorage.IsolatedStorageFile]::GetUserStoreForAssembly()
$userStore.DeleteFile("$dirName\$fileName")
$userStore.DeleteDirectory("$dirName")
$userStore.Close()
} #Remove-UserStore
#######################
function New-UserStore
{
param([string]$dirName)
$userStore = [System.IO.IsolatedStorage.IsolatedStorageFile]::GetUserStoreForAssembly()
$userStore.CreateDirectory("$dirName")
$userStore.Close()
} #New-UserStore
Binary files a/ThirdParty/SQLPSX/OracleIse/OracleIse.psm1 and /dev/null differ
Binary files a/ThirdParty/SQLPSX/OracleIse/Set-Options.ps1 and /dev/null differ
Binary files a/ThirdParty/SQLPSX/OracleIse/Switch-CommentOrText.ps1 and /dev/null differ
Binary files a/ThirdParty/SQLPSX/OracleIse/Switch-SelectedCommentOrText.ps1 and /dev/null differ
# ---------------------------------------------------------------------------
### <Script>
### <Author>
### Chad Miller
### </Author>
### <Description>
### This file is used to set options default options for SQLIse
### </Description>
### </Script>
# ---------------------------------------------------------------------------
@{
# MultilineWherePredicatesList = $true
# NewLineBeforeGroupByClause = $true
# NewLineBeforeHavingClause = $true
PoshMode = $true
# IndentViewBody = $false
# AlignColumnDefinitionFields = $false
# NewLineBeforeOutputClause = $true
# AlignClauseBodies = $false
# AlignSetClauseItem = $false
# AsKeywordOnOwnLine = $false
# IncludeSemicolons = $false
# NewLineBeforeOpenParenthesisInMultilineList = $true
# MultilineInsertSourcesList = $true
# MultilineSelectElementsList = $true
# QuotedIdentifierOff = $false
Results = "To Grid"
# OutputVariable = $null
# NewLineBeforeCloseParenthesisInMultilineList = $true
# NewLineBeforeWhereClause = $true
# NewLineBeforeFromClause = $true
# MultilineSetClauseItems = $true
# MultilineInsertTargetsList = $true
# NewLineBeforeJoinClause = $true
# NewLineBeforeOrderByClause = $true
# KeywordCasing = "Uppercase"
# IndentationSize = 4
# IndentSetClause = $false
# SqlVersion = "Sql100"
# MultilineViewColumnsList = $true
}
CREATE TABLE [dbo].[PolicyEval](
[PolicyEvalID] [int] IDENTITY(1,1) NOT NULL,
[ConfigurationGroup] [varchar](128) NOT NULL,
[PolicyCategoryFilter] [varchar](128) NOT NULL,
[PolicyEvalMode] [varchar](32) NOT NULL,
[PolicyName] [varchar](128) NOT NULL,
[ServerInstance] [varchar](128) NOT NULL,
[TargetQueryExpression] [varchar](500) NOT NULL,
[Result] [bit] NOT NULL,
[Exception] [varchar](500) NULL,
[PolicyEvalDate] [datetime] NOT NULL,
CONSTRAINT [PK_PolicyEvalID] PRIMARY KEY CLUSTERED
(
[PolicyEvalID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
ALTER TABLE [dbo].[PolicyEval] ADD CONSTRAINT [DF_PolicyEvalDate] DEFAULT (getdate()) FOR [PolicyEvalDate]
GO
CREATE TABLE [dbo].[PolicyEvalError](
[PolicyEvalErrorID] [int] IDENTITY(1,1) NOT NULL,
[ServerInstance] [varchar](128) NOT NULL,
[PolicyName] [varchar](128) NOT NULL,
[Exception] [varchar](500) NULL,
[PolicyEvalErrorDate] [datetime] NOT NULL,
CONSTRAINT [PK_PolicyEvalErrorID] PRIMARY KEY CLUSTERED
(
[PolicyEvalErrorID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
ALTER TABLE [dbo].[PolicyEvalError] ADD CONSTRAINT [DF_PolicyEvalErrorDate] DEFAULT (getdate()) FOR [PolicyEvalErrorDate]
GO
#if ($shellid -ne 'Microsoft.SqlServer.Management.PowerShell.sqlps' -or (!(get-command -name Invoke-PolicyEvaluation)))
#{ throw 'PBM must be run within sqlps mini-shell or SQL Server cmdlets must be loaded' }
# Note: This module is meant to run within sqlps or within a PowerShell host with sqlps cmdlets loaded.
# Sqlps does not support import-module or any post V1 cmdlets including write-eventlog.
# In sqlps the module must be dot sourced...
# . C:\scripts\pbm.psm1
# Dot sourcing a module is not recommanded.
# This module is loosely based on epmframework http://epmframework.codeplex.com but cleaned up to be more PowerShell like
# and remove multi-table/complex XML parsing
$Script:EvaluationMode = "Check"
$Script:PolicyServer = "Z003\R2"
$Script:PolicyDatabase = "MDW"
$Script:CMS = "Z003\SQLEXPRESS"
$Script:WriteEventLog = $false
$Script:LogName = "Application"
$Script:LogSource = "PBMScript"
$Script:EntryType = "Error"
$Script:EventId = 34052
#######################
function Get-PolicyStore
{
$conn = new-object Microsoft.SQlServer.Management.Sdk.Sfc.SqlStoreConnection("server=$Script:PolicyServer;Trusted_Connection=true")
$policyStore = new-object Microsoft.SqlServer.Management.DMF.PolicyStore($conn)
Write-Output $policyStore
} #Get-PolicyStore
#######################
function Get-TargetServer
{
[CmdletBinding()]
param(
[Parameter(Position=0, Mandatory=$true)] [string]$ConfigurationGroup)
$query = @"
SELECT s.server_name AS server_name
FROM msdb.dbo.sysmanagement_shared_registered_servers_internal s
INNER JOIN msdb.dbo.sysmanagement_shared_server_groups cg
ON s.server_group_id = cg.server_group_id
WHERE cg.name = '$ConfigurationGroup'
"@
Invoke-SqlCmd -ServerInstance $Script:CMS -Query $query | Select-object -ExpandProperty server_name
} #Get-TargetServer
#######################
function Write-PolicyEvalError
{
param(
[Parameter(Position=0, Mandatory=$true)] [string]$ServerInstance,
[Parameter(Position=1, Mandatory=$true)] [string]$PolicyName,
[Parameter(Position=2, Mandatory=$true)] [string]$Exception,
[Parameter(Position=3, Mandatory=$true)] [string]$PolicyEvalErrorDate)