CE 2000/010

30 June 2000

To all Chief Executives

MONITORING OF MAJOR IT PROJECTS

This Circular brings you up to date with developments on IT monitoring and on major IT projects in the public sector. Last November, I informed you about Cabinet decisions on the monitoring regime for major information technology (IT) projects (CE 1999/020 refers). Those decisions included a direction to SSC and Treasury to report back to Ministers on:

• the effectiveness of the chief executive accountability regime as it applies to IT project management, and
• the budget process in relation to the estimation of costs for IT projects.

Ministers also noted that a review of IT in the public sector was being carried out [CAB (99) M 20/33 refers], that the findings of this review may have implications for the manner in which public sector IT projects are monitored in the future, and that further recommendations may be required when that review reports back.

These reviews have now been completed and considered by Ministers. As a result of the review of the chief executive accountability framework and budget process, Ministers noted that when a new IT project is approved, the chief executive of the initiating department is accountable for the quality of the business case and for delivering on the approved project in accordance with the business case. Ministers also noted that the review of chief executive accountability for major IT projects found that no fundamental changes are necessary to the current structure and approach, and agreed that for IT projects, the chief executive of the initiating department is accountable for:

• ensuring the business case is sound and that the department's risk management, project management, and monitoring structures follow best practice;
• delivery of the module and associated benefits, as outlined in the business case; and
• processes put in place to manage off-track projects, if this becomes necessary.

For the approval process for major IT business cases, Ministers agreed that, unless otherwise agreed by the department and Treasury, a two-stage approval process should be adopted for major IT business cases (details in Annex 1). This process will consist of:

• stage 1: on the basis of indicative costs and benefits, seek approval for developing a detailed scoping and finalisation of costs and benefits associated with the project;
• stage 2: final consideration of the business case, based on more fully developed costs and benefits.

In addition, Ministers agreed that:

• unless central agencies and the department agree that there are compelling reasons to retain a single large project, major IT projects should be broken down into modules, with each module considered as an independent business case within the department's overall IT strategy; and
• quantitative risk analysis should be used as the basis for appropriations, access to contingency funding, and cash draw-downs for major IT projects, unless relevant Ministers agree that the size and nature of the project do not warrant this approach (details in Annex 2).

These decisions take effect from 1 July 2000. Ministers also agreed to the immediate disestablishment of the Ad Hoc Officials IT Committee. Central agencies (SSC and Treasury) will continue to carry out project approval and monitoring functions.

The full detail of these decisions is set out in FIN (00) M 12/3. The full report, The Chief Executive Accountability Framework and the Budget Process as They Relate to Information Technology Projects, is available on the SSC website.

In 1999, Cabinet commissioned a review of the management of Public Sector IT projects, following its concerns over the management of two significant public sector IT projects - the NZ Police's INCIS project and INSLAW, the Department for Court's case management project. The review was a joint undertaking by the Simpl Group, an organisation involved in project management, management consulting and IT systems development and implementation, and the New Zealand Institute of Economic Research (NZIER).

Overall the report concludes that the New Zealand public sector appears to be meeting international and local standards in terms of managing IT projects. The report, which is available on the SSC website, contains some clear lessons on the ingredients to seek or to avoid in designing and executing IT projects, including:

• project risk increases with project size and duration, so special attention needs to be paid to large projects;
• project management skills are essential to the conduct of complex IT projects;
• risk of failure increases with the extent of development activity; and
• in almost all cases, it is sensible to avoid using leading edge but untried technology.

Other recent reports on IT projects have included the report of the Controller and Auditor-General Governance and Oversight of Large Information Technology Projects, which looked at governance structures for IT projects, how IT projects actually happen, and reasons for success of failure. The report, (available on the OAG website), was written for chief executives, Ministers and members of Select Committees, and identifies the issues and questions which each of these groups should address with respect to large IT projects.

The Hood Report, Information Technology Requirements for New Zealand Police and Related Justice Sector Agencies also provides some lessons on how to deal with major projects, (available on the SSC website), and I expect the imminent report of the Ministerial inquiry into INCIS, will be similarly valuable.

I would be grateful if you could bring this circular to the attention of those in your organisation who have responsibility for developing and managing IT projects, and also to the attention of Crown entities and other Crown organisations with which you are linked.

I would also like to stress the importance of departments keeping in regular contact with their SSC advisors and Treasury Vote analysts on IT projects, and to include central agencies early in the process of scoping projects and developing business cases.

If you have any particular queries about the IT monitoring regime, please contact Hugh McPhail (495 6688, hugh.mcphail@ssc.govt.nz) or Andrew Thompson (471 5248 Andrew.Thompson@treasury.govt.nz)

Ross Tanner

Deputy State Services Commissioner

ANNEX 1

Business Case approval process

New Approval Process

Ministers have agreed that, unless otherwise agreed by the department and Treasury, a two-stage approval process should be followed for major IT projects. This process will consist of:

• Stage 1: on the basis of indicative costs and benefits, seek approval for developing a detailed scoping and finalisation of costs and benefits associated with the project;
• Stage 2: final consideration of the business case, based on more fully developed costs and benefits.

Ministers have also agreed that major IT projects should be broken down into modules, with each module considered as an independent business case within the department's overall IT strategy, unless central agencies and the department agree there are compelling reasons to retain a single, large project.

Rationale for New Process

The changes to the approval process are intended to provide greater flexibility in Cabinet's approach to approvals. This greater flexibility will allow business cases to follow an approval process that is appropriate for the size, risk and complexity of the project.

The two-stage approval process is designed to give more certainty around the final business case that Ministers approve, by ensuring that the final business case is completed after the detailed scoping and costing of the project. Previously, detailed scoping often occurred after Cabinet approval of the business case, which meant that costs and benefits associated with the project could change significantly from those originally approved by Cabinet.

Breaking projects into modules is designed to improve project management and provide more opportunities for Ministers to monitor progress of the project at defined stages. This monitoring allows Ministers to review progress on large-scale projects at the end of each module, allowing earlier identification and management of off-track projects. Requiring each module to deliver some benefit means that if a project needs to be terminated prior to the completion of the last module, the Crown should still obtain some benefit from the earlier completed modules.

Content of Stage 1

We envisage that the content of a stage 1 business case would be similar to the current business case content, but emphasising that the costs and benefits are indicative only. The purpose of the business case is to seek approval to proceed with the detailed scoping of the project. The focus of stage 1 consideration would be on:

• Strategic fit;
• What business need is being met by the project;
• What other alternatives are there for meeting that need and why this is the preferred option;
• Considering that preferred option within the context of a range of likely costs and an indicative cost-benefit analysis based on this range;
• Indicative funding implications, and if there are any potentially robust offsetting savings as a result of the project; and
• The cost of the scoping project.

It should be noted that approval at stage 1 does not constitute approval that the project can proceed to be implemented, it is only approval for further scoping work. Cabinet's final decision on approval of the project is taken at stage 2.

Content of Stage 2

The content of the business case at stage 2 would focus on the details of the proposed solution, and seek final Cabinet approval to proceed with the project (or the first module if the project has been broken into modules). The focus of consideration at stage 2 should be on:

• Detailed costs and benefits;
• Baseline funding implications;
• Deliverables (including timeframe);
• Risk management and project management structures, including quality assessment approach; and
• Demonstrating a rigorous process of peer review.

Breaking Projects into Modules

If a project is broken into modules, each module will be considered as stand-alone within the wider IT strategy. Each module should be supported by its own business case, including a cost-benefit analysis and show some benefit to the Crown. Funding for individual modules will be considered separately by Cabinet, once the previous module has been completed. It should be noted that the department's IT strategy may need updating after the completion of each module.

As an indicator, modules should not exceed $5m and be of less than 1 year's duration, although these are only guidelines. In practice, the decision on whether to break a project into modules, and what size to make those modules, will be guided by:

• The practicality of chunking the project
• Compliance costs to the department, central agencies and Ministers; and
• Supplier relationship management considerations.

ANNEX 2

Quantitative risk analysis

New Policy

Ministers have agreed that, for major IT projects, quantitative risk analysis should be used as the basis for appropriations, access to contingency funding, and cash draw-downs, unless relevant Ministers agree that the size and nature of the project do not warrant this approach.

Rationale for New Policy

Using quantitative risk analysis makes risks, and the financial impact of those risks, more explicit to Ministers when considering the business case. Impacts, other than financial, must also be considered and documented. If the financial impact and likelihood of risks is known, then contingency amounts and access arrangements can be set to allow those risks to be managed. Incorporating risk analysis into the business case also recognises that one single cost figure for the project is not be as useful as a range of likely costs around the expected cost figure.

Using Risk Analysis

As with other elements of business case development, the initiating department should conduct the risk analysis, with central agencies providing second-opinion advice on that analysis. The results of the risk analysis can be incorporated into the final business case that is submitted for Cabinet consideration.

Quantitative risk analysis requires conducting detailed sensitivity analysis and analysing the likely effect of these scenarios on total project costs. This involves assessing each probability and impact) and modelling how the total project will turn out based on simulations of each risk. This will produce an estimated probability distribution of likely total costs. The final probability distribution describes the range of outcomes and their relative likelihood. The potential impact of project-specific risks on both cost and timeframes is then explicitly considered by the department, central agencies and Ministers as part of the project analysis. For example, if one major risk is driving the variability of total cost forecasts, the department may wish to address this before proceeding.

Using this approach, appropriation funding would be on the basis of this probability distribution, with appropriation being based on the most likely cost, but with Ministers agreeing to disburse a smaller amount of cash than the full appropriation. If project risks do materialise, the department could seek Joint Ministerial approval for additional cash disbursement up to the full appropriation. This is demonstrated below. It should be noted that the 15^th and 85^th percentiles used in the example are for illustrative purposes only and are not necessarily the recommended thresholds - these thresholds will be determined in each individual case.

The above diagram is one expected output of a quantitative risk analysis. It shows the probability of delivering the project at the specified cost, given the risk scenarios that have been considered during the analysis. It also gives a visual indication of the range of likely costs associated with the project or module within the project, and is therefore useful in presentation of the business case to Ministers. The probability curve can also be used to set the thresholds for appropriations and cash-draw down limits for the project/module, based on the likely final costs.

When departments are considering doing a quantitative risk analysis, they should contact their Treasury vote analyst for advice.