When I talk to customers about System Center these days, System Center Orchestrator 2012 is a product that everyone wants to know more about. To that end, this series covers some fundamental concepts to help get you master System Center Orchestrator. This 5-part blog series (an updated and expanded from our previous series on Opalis), discusses SCOrch 2012 concepts and functionality, including real-world tips and sample automation sequences (runbooks) to jumpstart your process automation efforts.
HAND ON LABS: To reinforce the concepts covered each day, articles in the series will include a pointer to accompanying hands-on lab exercises (on Microsoft’s TechNet Virtual Labs) that reinforce the concepts discussed in that day’s installment. These labs are largely Opalis 6.3 labs, but the authoring experience (interface, runbook concepts, etc.) in Orchestrator is basically unchanged from that of Opalis (although some terminology has been updated). In fact, the runbooks used in these Opalis labs all work in Orchestrator 2012!
In the Series
The series will consist of the following articles. Since the entire series is ready, you should see a new article every 3 days or so.
- Day 1 (this article): Runbook Concepts, Components and Rules of the Databus
- Day 2: Advanced Features and Functionality
- Day 3: Bridging Gaps, Extending Capabilities, Best Practices
- Day 4: Deep Dive into the Orchestrator Web Service
- Day 5: Wrap Up, Challenge lab and Bonus Download
In This Installment
In this first installment, we will dig into:
- The Building Blocks of a Runbook
- Data Publishing and the “Rules of the Data Bus”
- Which Processes Should I Automate?
- Hands-on Lab to put concepts into practice
SCOrch provides IT Process Automation (ITPA) also known as Runbook Automation (RBA) – the ability to define, build, orchestrate, manage and report on runbooks that support system and network operational processes. This capability allows IT professionals to automate tasks with a minimal need for programming and scripting (notice I said ‘minimal need for scripting’, not ‘without scripting’). RBA using SCOrch can cross multiple IT management disciplines in a single automation sequence, integrating multiple IT management tools and interacting with all types of infrastructure elements to automate processes ranging from simple to complex, such as automating resolution of a known error or provisioning a new server and installing the necessary applications.
First, the question: “What is a runbook anyway?”
The Building Blocks of a Runbook
SCOrch automation capabilities are based on the concept of runbooks (sometimes referred to as workflows…more on this later) which are visual representations of your data center processes as they are automated in SCOrch. You create runbooks by dragging and dropping activities into a workspace in the Runbook Designer (the primary administration and authoring interface of SCOrch) and connecting them with links. Each activity performs a specific action when it is executed (the precise behavior depending on how the activity is configured by the runbook author). Once an activity has completed it will output one or more data elements and trigger any activities that are linked to it. For example, the runbook in Figure 1 contains an activity that monitors a folder. When a file enters the folder, the activity triggers a second activity to move the file to an archive directory (here’s a very basic backup approach!), which in turn links an activity to log an event.
Figure 1 – Sample Runbook
Links connect activities in a runbook directing the flow of activity and data within a runbook based on conditions encountered at runtime. Whenever you create a link in a runbook, by default it is configured to trigger the next activity in the runbook when the previous one succeeds. However, links also provide filtering; this allows you to limit the data arriving at the following activity in the runbook and control the flow of runbook execution based on the result of activity execution. Link conditions provide a set of author-configurable functions for creating complex decision logic involving text, numeric or time-related data. Links configured with conditional filtering logic as described here are called Conditional Links. Configuring runbooks with multiple branches driven by conditional links is a concept called branching. To make your runbooks visually more intuitive, you can also change the display name of activities and link labels to make them more descriptive of their purpose in the runbook. You can also change link color to highlight success and failure branches within your runbooks.
For example, look at the runbook depicted in Figure 2, where the link labeled “Service Running” is configured to trigger the next activity in the runbook only if the Get Service Status activity returns a Service Status of ‘running’ (as pictured in the link properties in Figure 3).
Figure 2- Sample runbook implementing conditional links and branching
Conditional Filtering in Links
You can configure conditional filtering in link properties, using both include and exclude logic. The Include tab specifies the conditions that will allow data to flow to the next activity in the runbook. The Exclude tab specifies the conditions that will cause data to be excluded from the next activity in the runbook.
Figure 3 – Link properties of ‘Service Running’ link from runbook in Figure 2
Note: When implementing conditional filtering, bear in mind that rules on the Exclude tab always supersede the rules on the Include tab.
Activities, Integration Packs and Runbook Types
SCOrch comes with several dozen product-agnostic activities that perform a variety of functions; these are known as Standard Activities. To expand and customize the functionality of SCOrch, you can add additional product-specific activities, contained in packages known as integration packs (IPs). There currently exist System Center IPs with product-specific activities available for each member product in the System Center suite, as well as many third party applications, including several network monitoring platforms and service desk products. You can register and deploy integration packs using the Deployment Manager interface. You can even download community-developed integration packs and runbook samples from codeplex.com or the TechNet Gallery.
SCOrch runbooks will fall into one of two categories based on the first activities in the runbook:
- An ad hoc runbook is a runbook started on demand by a runbook operator, author or from another runbook as needed. An ad hoc runbook will run once, perform the tasks it is configured to complete, and terminate.
- A monitor runbook runs automatically or on a schedule, waiting for a specific condition to trigger further action. You can usually tell the difference between the two because a monitor runbook will begin with an activities that has ‘monitor’ in the name. For example, the runbook shown in Figure 1 is a monitor runbook, which runs perpetually.
TIP: Monitor type activities must always be the first activity in the runbook. Monitor activities are triggered by the condition they are monitoring for, not by another activity.
Runbooks and Workflows…is there a difference?
You will often here the terms runbook and workflow used interchangeably. A workflow is unofficially and (very) loosely defined as an automation sequence involving multiple (nested) runbooks. More on this in “Advanced Runbook Features and Functionality,” in Part 2 of this series.
Data Publishing and the “Rules of the Data Bus”
SCOrch features a publish-and-subscribe data bus, which is the mechanism used within SCOrch to pass information from one activity in a runbook to the next activity. The data flowing along the path of the runbook is called published data, and each subsequent activity in the runbook adds its own data to the data bus. As the runbook progresses, more data becomes available. The published data capability of SCOrch is automatic and not configurable. Figure 4 illustrates runbook execution and data publishing.
Figure 4 – Runbook execution and data publishing (concept)
The data collected or created by an activity is automatically published to the SCOrch data bus. As later activities execute, they can draw information from one or more previous activities. The runbook author can subscribe to this published data and use it in the configuration of activities within the runbook. For example, in the example runbook shown in Figure 4, the Query Web Service activity retrieves a SOAP message from a .Net web service, which is published to the data. The Query XML activity is configured to subscribe to this SOAP message and perform a query to retrieve a specific piece of data within the message. Finally, the Write Results to SQL activity is configured to subscribe to the XML query result and then write it to the database.
Every runbook runs within its own windows process (PolicyModule.exe – shown in Figure 4) and the data bus exists within this process. When the runbook completes execution, the data published to the data bus is lost. The number of data elements produced by an activity, as well as the configuration of the activity properties can affect how many times an activity executes and how many data elements are in the activity output. Here are some rules of the data bus that describe activity execution behavior in SCOrch:
Single Execution: An activity will run one time for each time the previous activity runs.
Multi-Value Data: An activity will run one time for each data item generated by the previous activity. For example, if you a Read Line activity that retrieves five lines of text, the next activity in the runbook will execute five times as well. If the next activity in the runbook were a Send Event Log Message, five events would be logged to the Application Event Log (each with one line of data if the activity is configured to subscribe to the line text output of the Read Line activity).
Flatten: When you select the Flatten option, a multi-value data set will be consolidated to a single array with data items separated by the delimiter of your choice. The activity that follows will run only one time. This is handy when you want to write a multi-value data set to a single Windows event or database record. For example, if you a Read Line activity that retrieves five lines of text and you check the Flatten checkbox, the next activity in the runbook will only execute once, so only one event would be logged to the Application Event Log ( data if the activity is configured to subscribe to the line text output of the Read Line activity).
Multiple Executions: You can configure Looping on an activity and enable multiple execution attempts. The Flatten option mentioned earlier does not affect an activity configured to execute multiple times. In short, when Looping is enabled on an activity and configured to exit only after five attempts, the activity will run the requested number of times whether Flatten is selected or not. Figure 5 shows an example of Looping configured to allow multiple execution attempts.
Figure 5 – Looping configuration on a Read Line activity
Which Processes Should I Automate?
A common question often heard when talking to customers and partners is “Which processes should I automate? Where should I start?
You could start by asking yourself a few of the more obvious questions:
Which processes are the most time-consuming?
Where are service levels suffering the most?
Which problems recur most frequently? Which are most expensive for the company?
Which process failures are visible to customers?
However, processes do not have to be inherently time consuming, complicated or expensive for SCOrch to deliver benefits. The fact of the matter is that any predictable or repetitive task a human can perform SCOrch can perform just as well, with greater consistency, speed, logging, and integration with change management processes. In addition, every process automated saves money and time, freeing administrators up for other tasks.
For more on the criteria for choosing good candidates for runbook automation, see ROI for RBA: Considerations and Best Practices for Runbook Automation Planning
UPDATE MARCH 12, 2013:These labs are currently unavailable. I will update this post when suitable replacements are again online and available.
Now it’s time to build some runbooks! Microsoft has two hands-on labs that reinforce the concepts discussed in this installment perfectly.
- TechNet Virtual Lab: Introduction to Opalis
- TechNet Virtual Lab: Opalis: Creating Basic Policies with Foundation Objects
IMPORTANT: If you’re not familiar with Opalis, here are a couple of pieces of information that will help
- Policies in Opalis are what we now call Runbooks in Orchestrator….same concept, new terminology.
- Foundation Objects are now called Standard Activities….same concept, new terminology.
That’s all for this post. The next installment continues this discussion of SCOrch fundamentals, including:
- Advanced Policy Features and Functionality
- Testing and Troubleshooting Your Runbooks
- Taking Automation to the next level with SCOrch and System Center
Get Orchestrator Unleashed
Pete Zerger is an author for the upcoming System Center 2012 Orchestrator Unleashed (http://www.amazon.com/System-Center-2012-Orchestrator-Unleashed/dp/0672336103). This book is authored by an expert team of current and former Microsoft MVPs who also wrote the industry’s leading guide to Opalis 6.3, Orchestrator 2012′s predecessor product.