Runspaces made simple

PowerShell is great at simplifying and automating your daily tasks. And it’s only natural that after you have automated or scripted most of your simple tasks, you start creating bigger scripts. If you work in a large environment, running for instance a script that uses WMI to query computers for data can take a long time to complete. The solution? Multi-threading of course. One way of doing this is using the built-in ‘Jobs’ cmdlets, but what this post is about is Runspaces.

Runspaces lets you create separate PowerShell sessions running in separate runspaces (hence the name), within the same process. This means that you won’t see any difference when looking at Task Manager (except CPU and memory will probably go up, depending on the number of runspaces you use).

Working with runspaces have for some been a daunting task, as it means you need to work with .NET directly. I have created a set of wrappers for working with runspaces that simplifies this and lets you work with functions that behave as common PowerShell cmdlets.

The following functions are included:

  • New-RunspacePool
  • New-RunspaceJob
  • Receive-RunspaceJob
  • Show-RunspaceJob
  • Clear-RunspaceJobs

New-RunspacePool

This function will create a new runspace pool. A runspace pool can be thought of as the object that is controlling the behaviour of you runspaces, as well as tying them all together. Setting up the runspace pool lets you control how many concurrent threads you want to run, as well as control snapins, modules, functions and variables that you want each runspace to have access to when spawned. The New-RunspacePool function returns a RunspacePool object that you will need to capture in a variable, as this is used together with the next function: New-RunspaceJob.

New-RunspaceJob

This is the function that takes the supplied code (in the form of a script block), creates a new runspace and injects the code together with any parameters (in the form of a hashtable) that you want the code to use. Each new runspace job is connected to the runspace pool, and the pool takes care of how many runspaces are current running at the same time, and will automatically start a new one when a new opening in the queue comes up. All the runspace jobs are saved in the global ‘runspaces’ variable. You can name your jobs as well, so if you are running several different jobs at the same time, it’s easy to differentiate between them when looking in the ‘runspaces’ array.

Receive-RunspaceJob

Receive-RunspaceJob will check if the job is finished and return the result if it is. This function have a ‘Wait’ parameter that you can use (together with a ‘Timeout’ parameter) to let it run until all jobs are finished (or the timeout value is exceeded). It also supports getting only runspaces belonging to a certain job, or even by the unique ID of each job.

Show-RunspaceJob

Show-RunspaceJob will list out all jobs currently in the global runspaces array. It only have one parameter, ‘JobName’, that lets you list only runspaces belonging to a particular job.

Clear-Runspaces

Lastly we have the Clear-Runspaces function. The only thing this one does, is remove everything from the global runspaces variable (it is in fact deleting the variable). Hopefully you would never need this, but working with WMI I know from experience that sometimes a WMI session can hang, and it’s nice to be able to completely remove any jobs left.

Creating these functions, I had a goal of creating something that I could use both in scripts, but also directly from the console, and I hope I have succeeded. To get you going I have also created a basic template that you can use when running code against an array of computers. It is by no means a complete script, and should only be used as a starting-point you can work on, but it should be enough to give you a picture of how to use the different functions.

As always, if you spot any bugs, or have any suggestions for future improvements, or just need some input on how to use these functions, don’t hesitate to contact me!

7 comments

  1. Awesome – thanks!

    Why does New-RunspaceJob require a jobname? (I get an error with your example until I give it an arbitratary name):

    # iterate through each computer and create new runspace jobs
    # also run Receive-RunspaceJob to collect any already finished jobs
    foreach ($computer in $computers) {
    New-RunspaceJob -RunspacePool $thisRunspacePool -ScriptBlock $code -Parameters @{ComputerName = $computer} -JobName LeBeuf
    $results += Receive-RunspaceJob
    }

    Like

  2. Ah.. Because I had accidentally defined the JobName parameter as mandatory. That was not my intention. I have updated the Gist now. Thanks for letting me know 🙂

    Like

  3. Great.

    You’ve just a really great job at making threading easy to include in scripts, so that all the advanced stuff is seperated from the maincode.
    If you like feedback/suggestions – here’s some..
    As I see your intention, what you want is for the caller to just worry about the following (which is a really great idea):
    1: List of computernames
    2. How many threads
    3. Timeout

    How would it sound to you, if the following, from you maincode, where part of the module?:
    ——————————————————————————————————————–
    # create new runspace pool
    $thisRunspacePool = New-RunspacePool

    # define results array
    $results = @()

    # iterate through each computer and create new runspace jobs
    # also run Receive-RunspaceJob to collect any already finished jobs
    foreach ($computer in $computers) {
    New-RunspaceJob -RunspacePool $thisRunspacePool -ScriptBlock $code -Parameters @{ComputerName = $computer}
    $results += Receive-RunspaceJob
    }

    # if any jobs left, wait until all jobs are finished, or timeout is reached
    if ([bool](Show-RunspaceJob)) {
    $results += Receive-RunspaceJob -Wait -TimeOut $timeout
    }

    Write-Output $results
    ——————————————————————————————————————–

    Anders

    Like

  4. I can see why that would be tempting to do. There is however a couple of reason I wouldn’t do that.

    First, if you take a look at the New-RunspacePool function, it takes a lot of parameters. If you want to have any third-party/custom functions, modules, snapins or variables available to you runspaces, this is all set up along the runspace pool. I have no way of knowing what kind of information the users wants to be part of the pool, and by putting it in the main module, I’m forcing the creation of a runspace pool on the user.

    The same with the New-RunspaceJob really, this is where you add on any parameters you need for your code block. Usually this includes at least ComputerName. That’s why I added it to my example. But more often than not, some other parameters would be needed as well (for instance a credential parameter).

    That’s why I have opted for the fully user-customizable approach. What you can do of course, is to add a line in your powershell profile that automatically creates a ‘default’ runspace pool, always ready for when you want to quickly run some code in multiple threads.

    And if you’d like, feel free to play with my code and customize it to your own needs 🙂

    I had originally not intended to publish anything but the functions themselves, but thought I’d add a basic template as an example of how you could use the functions, and also as a starting point for further development. But that’s all it is really, a very basic template, meant to be further customized to individual needs.

    You could very easily convert my basic template into a function btw, and use this function the way you describe: As an entry-point to all the other functions, for any scenarios where you just have a script block and an array of computer names.

    But thanks a lot for the positive and constructive feedback! If you need any help customizing this to your needs, let me know 🙂

    Like

  5. Hi Øyvind – great name btw – There’s an eagle in the danish version of Donald Duck called Øjvind Ørn (he’s one of the bad guys though) 🙂

    That was a very informative explanation – thanks for taking your time! I see your intention and understand the choices you’ve taken.
    I’ve made a module out of your code, that will be put to use some how I’m sure.

    Thanks for the generosity of sharing your great work – I’ll let you know any other questions surfaces or theres a need to enjoy your friendlyness!

    Anders (the danish Donald Duck is called Anders And 🙂

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s