ROI for RBA: Considerations and Best Practices for Runbook Automation Planning (1st Draft)

I’m often asked by customers, “Where should we start with runbook automation (RBA)?” I think this question is better restated as

“How can we maximize the ROI of our RBA investment?”

In the end, IT operations is about efficiency… minimizing the cost of operations so there’s more money left over or innovation (R&D, growth), so RBA should be approached with return on investment (ROI) in mind. There are a few simple characteristics that can help identify the best candidates; candidates that are not only for high ROI, but that are also ideal starting points in terms of development cost and complexity for an organization implementing RBA for the first time.

The goals for runbook automation are pretty clear, although there are some benefits we may not consider. While Microsoft has provided a few best practices for authoring, you will not find end-to-end guidance on how identify the processes that your organization would most benefit from. In reality, there are a number of organizationally specific factors that influence the degree to which an organization will benefit from automating a given process.

1. Ask these questions to create an ‘automation wish list’

The first step involves talking to You could start by asking yourself a few of the more obvious questions:

  • Which processes are the most time-consuming?
  • Where are service levels suffering the most?
  • Which problems recur most frequently?
  • Which are most expensive for the company?
  • Which process failures are visible to customers?

However, processes do not have to be inherently time consuming, complicated or expensive for OIS to deliver benefits. The fact of the matter is that any predictable or repetitive task a human can perform OIS can perform just as well, with greater consistency, speed, logging, and integration with change management processes. In addition, every process automated saves money and time, freeing administrators up for other tasks.

This is only the initial phase of the process. We’ve identified the problems and tasks that cause us anxiety, expense, etc. Now we need to run these through some additional filters to identify the true “best candidates” for RBA.

Wish List FunnelRunning the RBA Wish List through the ‘RBA Reality Funnel’

While this first list of processes may give us a list we think will allow us to start developing runbooks immediately, this is just not the case. We need to run these candidates for RBA run additional series of filters I like to call the ‘RBA Reality Funnel’. These filters that apply some additional feasibility criteria to our RBA candidates to identify those that are best automated within our organization based on the number of additional criteria.

1. Frequency of Use: Calculating for each runbook automation includes how often the process is executed.   The more often the process is executed, the higher the impact of runbook automation.  As I review processes with customers, I work to ensure that any candidate process provides enough value to justify the cost in time and resources to develop the runbook and frequency of use is the first filter for prioritizing candidates for RBA.  You can identify high-frequency processes by reviewing data from your helpdesk ticketing and change management activity from previous months. Often we find that a significant percentage of the activity at helpdesk when a change management system can be attributed to a few recurring processes.  These tasks can range from creating new user accounts, providing access to network resources, to collecting troubleshooting data based on an alert in the monitoring system.

IMPORTANT: It’s important to go beyond identifying how often the process recurs to also identify how much effort and expense are involved. We have two processes that recur 10 times a week and one takes five minutes to complete and the other 60 minutes, it makes sense in this first step to assign higher priority to the more labor intensive and or expensive process.

2. Orchestration Effort: Once we have identified the most frequently occurring processes and the effort associated with completing them manually, we need to determine how much effort and expense are  involved to develop the runbook to automate the process. Some processes may require advanced scripting and/or programming, which is a skill set not possessed by all organizations. In the end these processes may prove to expensive to automate to make the list of best candidates in the first wave. Before approaching RBA with a ‘start simply’ mindset, processes that prove to be high effort in RBA development should probably be moved down the list and addressed at a later date.

IMPORTANT: Be pragmatic and approach the topic of cost and complexity with Murphy’s Law in mind. What can go wrong usually will and rosy estimates that don’t pan out in the end can sour your organizations view of RBA investment in the future.

3. Variance / Exceptions: The truth of the matter is some processes are just not all that consistent. There may be multiple input factors that change frequently and are not easily identified without human intervention. The more exceptions there are to a process, the more challenging it is to automate the process. Defining consistent rules for how a process should be executed enables your organization to realize a reduction in development time in the automation effort. If the process doesn’t meet the consistency test, it should probably be moved down the list for consideration a later date.

IMPORTANT: Talk to the people most familiar with the process to get the real answer to this question. If you’re a casual observer not core to the completion of this process you likely won’t fully understand the scope of variance within the process on a day-to-day basis.

The Last Mile of RBA: Closing the loop with ITSM Integration

Documentation and governance of an organizations  RBA strategy is really important to ROI and operational maturity in the long run. I’m seeing more chatter in the technical community about closed-loop change management. What is closed-loop change management? Here’s a definition:

“Closed-loop Change Management is a strategy for extending current tools and processes to cover the complete Change Management lifecycle, from request through execution to review and completion. Closed-loop Change Management tracks actual changes for comparison to approved changes. Policies for how and when changes can be made are established, based on skill sets, application, time, required approvals and standard operations procedures. The goal is to more accurately verify approved changes, detect or prevent unapproved changes, and over time standardize how changes are performed.”

Source: Change Management – The Other Half of the Story

In order to close the loop we have to instrument a runbooks with integration into the systems that support ITSM within the organization. The closing the loop and the session we automate notification, documentation and provide a ready source for ROI calculation through reporting tools attached to our CMDB (such as the SharePoint integrated ROI dashboard in service manager 2012). Just as importantly, integrating a runbooks in this way supports the ‘begin with the end-user in mind ‘strategy that I introduced in my talk at System Center Universe 2012.


By considering these factors, you can help your organization identifying which processes and associated systems are in reality the best candidates for RBA  from an ROI perspective.  Not unreasonable to assume substantial ROI (or even full payback on RBA investment) can be realized within the first year If you approach RBA with ROI in mind.

Additional Reading

Opalis Runbook Fundamentals Series (written largely by me and published on

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.