When working on improving the performance of a PowerShell script, going from a single threaded to a multi threaded script will almost certainly give you a good performance boost. Boe Prox have an excellent module for this, called PoshRSJob which you can download from the PowerShell Gallery.
But sometimes you don’t want, or need, an external module as a prerequisite to get access to this functionality. It’s not that hard rolling your own code to run stuff in separate threads, but as a general rule I would highly recommend using something like PoshRSJob if you have to spin up a high number of threads. Typical use cases for this are if you are enumerating through an array, and want to execute code for each as a separate thread.
On the other hand, if you have a small number jobs, you might opt to do this yourself. One of (many) hurdles you might run into is how to use custom functions and types inside your newly created threads, and in this post I will show you how to do this.
Setup
Lets start by defining our own custom type:
Add-Type -MemberDefinition @" public static string SayHello() { string Hello = "Hello"; return Hello; } "@ -Name Stuff -Namespace My
Here we have created the custom class My.Stuff that have one static method called SayHello. This method will return the word ‘Hello’. Pretty nice, right? Feel free to use this in your production code by the way.
To make things interesting lets also define a couple of custom functions:
function Invoke-CustomType { [My.Stuff]::SayHello() } function Invoke-CustomFunction { Write-Output 'Not using Custom Type!' }
As you can see, one of the functions is using our newly created custom type, while the other is not.
Next, lets create the scriptblocks that we will inject into separate threads later on.
$callingFunction = { Write-Output (Invoke-CustomFunction) } $callingType = { Write-Output (Invoke-CustomType) }
Again, we have created two; one that will run the function calling our custom type, while the other is calling the function that is just using Write-Output.
What we have set up here is a pretty typical scenario in larger scripts, and we need to make sure that both of these methods will work correctly when run in separate threads.
Test 1 – Not using custom type
Let’s start with the simplest one, namely the scriptblock that calls our custom function that do not rely on any custom types.
# Setting up an initial session state object $initialSessionState = [System.Management.Automation.Runspaces.InitialSessionState]::CreateDefault() # Getting the function definition for the function I want to add $functionDefinition = Get-Content function:\Invoke-CustomFunction $functionEntry = New-Object System.Management.Automation.Runspaces.SessionStateFunctionEntry -ArgumentList "Invoke-CustomFunction", $functionDefinition # And add it to the iss object $initialSessionState.Commands.Add($functionEntry) # Create a new runspace using my iss object $runspace = [System.Management.Automation.PowerShell]::Create($initialSessionState) # Add the script I want to run in the runspace [void]$runspace.AddScript($callingFunction) # Start the job $handle = $runspace.BeginInvoke() # Sleep for a while so the job gets time to finish Start-Sleep -Milliseconds 50 # Check if job is complete, and retreive the results if it is. if ($handle.IsCompleted) { $runspace.EndInvoke($handle) } else { "Job not complete!" } # Dispose of the runspace object $runspace.Dispose()
I’m not using a runspace pool in these examples, because I strongly feel that if you need this, you are better off using PoshRSJob or something similar.
The first thing we need to do is to create an InitialSessionState (ISS) object. This object is used to set up the initial state of the new runspace we will be creating.
Then we need to read the function definition and create a SessionStateFunctionEntry. This is an object used to add a function to the ISS object, and we give it the name and the definition of the function as arguments, before adding it to the ISS.
Next we create a new runspace, add our scriptblock and call the BeginInvoke method to start executing code in the runspace. Make sure you always capture the results of BeginInvoke into a variable, or you will not be able to retrieve the results later on!
Now we just need to let the script sleep a little, so the code gets time to finish, before we fetch the results.
Remember to dispose of the runspace when you are done with it!
Test 2 – Using custom type
For this example we are using our second function, the one that is using our custom type.
Remove-Variable -Name initialSessionState,functionDefinition,functionEntry,runspace,handle -ErrorAction SilentlyContinue # Setting up an initial session state object $initialSessionState = [System.Management.Automation.Runspaces.InitialSessionState]::CreateDefault() # Getting the function definition for the function I want to add $functionDefinition = Get-Content function:\Invoke-CustomType $functionEntry = New-Object System.Management.Automation.Runspaces.SessionStateFunctionEntry -ArgumentList "Invoke-CustomType", $functionDefinition # And add it to the iss object $initialSessionState.Commands.Add($functionEntry) # Get the type data for the custom type that I want to add $typeData = New-Object System.Management.Automation.Runspaces.TypeData -ArgumentList "My.Stuff" $typeEntry = New-Object System.Management.Automation.Runspaces.SessionStateTypeEntry -ArgumentList $typeData,$false # And add it to the iss object $initialSessionState.Types.Add($typeEntry) # Create a new runspace using my iss object $runspace = [System.Management.Automation.PowerShell]::Create($initialSessionState) # Add the script I want to run in the runspace [void]$runspace.AddScript($callingType) # Start the job $handle = $runspace.BeginInvoke() # Sleep for a while so the job gets time to finish Start-Sleep -Milliseconds 100 # Check if job is complete, and retreive the results if it is. if ($handle.IsCompleted) { $runspace.EndInvoke($handle) } else { "Job not complete!" } # Dispose of the runspace object $runspace.Dispose()
We start by removing the variables used in the first example, so we are sure that we start fresh.
The approach is similar to the first example, but this time we also need to make sure that the custom type we made is also added to the ISS object.
The way to do this is to create a TypeData object, which we will use to create a SessionStateTypeEntry object. When we add this to the ISS, this type will be available to all runspaces created using this ISS.
Note that you can also add types that are defined in a types.ps1xml file. When doing this you don’t need to create a TypeData object, just give the path to the ps1xml-file as single parameter when creating the SessionStateTypeEntry object.
The rest of the code is the same as the first example, the only change is that we are injecting a different script block to the new runspace.
But what about…
The InitialSessionState object also supports adding modules, assemblies, formats and variables. The approach for adding these are similar to the approach I have used in the examples.
Though, if you feel the need to add module(s) to the ISS you are making a dependency to a third-party module, and I would suggest using PoshRSJob instead.
Here is all the code as a gist if you want to test this out yourself:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Add custom type with static method | |
Add-Type –MemberDefinition @" | |
public static string SayHello() | |
{ | |
string Hello = "Hello"; | |
return Hello; | |
} | |
"@ –Name Stuff –Namespace My | |
# Define custom function calling my custom type | |
function Invoke-CustomType { | |
[My.Stuff]::SayHello() | |
} | |
# Define custom function NOT using my custom type | |
function Invoke-CustomFunction { | |
Write-Output 'Not using Custom Type!' | |
} | |
# Define script using my Invoke-CustomFunction function | |
$callingFunction = { | |
Write-Output (Invoke-CustomFunction) | |
} | |
# Define script using my Invoke-CustomType function | |
$callingType = { | |
Write-Output (Invoke-CustomType) | |
} | |
## | |
## TEST 1 – Not using my custom type | |
## | |
# Setting up an initial session state object | |
$initialSessionState = [System.Management.Automation.Runspaces.InitialSessionState]::CreateDefault() | |
# Getting the function definition for the function I want to add | |
$functionDefinition = Get-Content function:\Invoke-CustomFunction | |
$functionEntry = New-Object System.Management.Automation.Runspaces.SessionStateFunctionEntry –ArgumentList "Invoke-CustomFunction", $functionDefinition | |
# And add it to the iss object | |
$initialSessionState.Commands.Add($functionEntry) | |
# Create a new runspace using my iss object | |
$runspace = [System.Management.Automation.PowerShell]::Create($initialSessionState) | |
# Add the script I want to run in the runspace | |
[void]$runspace.AddScript($callingFunction) | |
# Start the job | |
$handle = $runspace.BeginInvoke() | |
# Sleep for a while so the job gets time to finish | |
Start-Sleep –Milliseconds 50 | |
# Check if job is complete, and retreive the results if it is. | |
if ($handle.IsCompleted) { | |
$runspace.EndInvoke($handle) | |
} | |
else { | |
"Job not complete!" | |
} | |
# Dispose of the runspace object | |
$runspace.Dispose() | |
## CLEAN-UP | |
Remove-Variable –Name initialSessionState,functionDefinition,functionEntry,runspace,handle –ErrorAction SilentlyContinue | |
## | |
## TEST 2 – Using my custom type | |
## | |
# Setting up an initial session state object | |
$initialSessionState = [System.Management.Automation.Runspaces.InitialSessionState]::CreateDefault() | |
# Getting the function definition for the function I want to add | |
$functionDefinition = Get-Content function:\Invoke-CustomType | |
$functionEntry = New-Object System.Management.Automation.Runspaces.SessionStateFunctionEntry –ArgumentList "Invoke-CustomType", $functionDefinition | |
# And add it to the iss object | |
$initialSessionState.Commands.Add($functionEntry) | |
# Get the type data for the custom type that I want to add | |
$typeData = New-Object System.Management.Automation.Runspaces.TypeData –ArgumentList "My.Stuff" | |
$typeEntry = New-Object System.Management.Automation.Runspaces.SessionStateTypeEntry –ArgumentList $typeData,$false | |
# And add it to the iss object | |
$initialSessionState.Types.Add($typeEntry) | |
# Create a new runspace using my iss object | |
$runspace = [System.Management.Automation.PowerShell]::Create($initialSessionState) | |
# Add the script I want to run in the runspace | |
[void]$runspace.AddScript($callingType) | |
# Start the job | |
$handle = $runspace.BeginInvoke() | |
# Sleep for a while so the job gets time to finish | |
Start-Sleep –Milliseconds 100 | |
# Check if job is complete, and retreive the results if it is. | |
if ($handle.IsCompleted) { | |
$runspace.EndInvoke($handle) | |
} | |
else { | |
"Job not complete!" | |
} | |
# Dispose of the runspace object | |
$runspace.Dispose() |
I hope you have found this post helpful, and if you spot any errors or have suggestions for improvements, don’t hesitate to let me know in the comments below!
How does it compare to running multithreaded via Invoke-Command locally instead. Reason being I don’t see how in this example to limit number of threads while Invoke-Command cmdlet have -ThrottleLimit parameter which comes handy if you don’t know how big collection you will be enumerating against and wether you can affort to have dedicated thread for each of them.
LikeLike
If you need thread throttling, you need to use a runspace pool, and I would suggest you use PoshRSJob instead. Creating new threads manually, as I have done in these examples, is only really useful if you only need to use a small (fixed) set of threads.
LikeLike