Project Delivery - Capability

Last updated: 2 December 2013

This group identifies capabilities and structures that support successful delivery of government projects, and facilitates their use by agencies.

Update 2 December 2013:

Since 2001 the Major Projects Assurance group has been located in the State Services Commission. On 2 December 2013 the group transferred to the Treasury, as part of the Portfolio Performance Management group. In the New Year this guidance will be repositioned on the Treasury website at:www.treasury.govt.nz/publications/guidance/mgmt/majorprojects

 The following three tools are supported by SSC:

  • Investment Logic Mapping [ILM]
  • Quantitative Risk Analysis (QRA)
  • Training for Programme and Project Sponsors/Senior Responsible Owners (SROs)

1 Investment Logic Mapping (ILM)

Better Business Case (BBC) guidance was introduced by New Zealand Treasury in 2010. The guidance mandates the use of the first two ILM workshops (Problem Definition and Benefits Definition) for the development of the BBC Strategic Assessment for all High Risk and/or large scale capital projects in the State Sector. For High Risk and large scale projects an accredited ILM Facilitator must be used.

ILM can be effectively used on projects or programmes of any size. Regardless of the dollar cost of the project or its High or Low risk significance to the organisation, the same structured workshop approach is used. 

More information about BBC processes and requirements can be found at www.infrastructure.govt.nz/publications/betterbusinesscases .

Background

ILM (known as Investment Management within Australia) was developed by the State Government of Victoria, Department of Treasury and Finance (DTF) in 2003, to screen budget bids and strengthen thinking and decision-making around prioritisation and resource allocation within agencies and across government. It was formally introduced to New Zealand by the State Services Commission (SSC) in July 2008, following successful pilots by the Ministry of Health. It is being used increasingly by New Zealand Government agencies and is included in the New Zealand Treasury's guidelines for Public Sector Business Cases. Although its origins are in the public sector it has proved an equally valuable technique when used in the private sector.

Overview

Investment Logic Mapping (ILM) is a technique to ensure that intelligent discussion and thinking is done up-front, before solutions are identified and before any investment decision is made. It is a technique to ensure the 'story' about any proposed investment makes sense (the 'logic' part of ILM) and to test and confirm that the rationale for a proposed investment is evidence-based and sufficiently compelling to convince decision makers to commit to invest in further investigation and planning.

The main 'artefacts' from Investment Logic Mapping are the Investment Logic Map and the Benefit Management Plan.

The Investment Logic Map is a simple single-page flowchart that tells the story of an investment and exposes its underpinning logic. It is in plain English and has been designed to answer many of the key questions required to make an investment decision.  The ILM is modified to reflect any changes to the logic of an investment throughout its lifecycle.

The Benefit Management Plan specifies the benefits an investment will be required to deliver, and the evidence that will be required to prove the benefits have been delivered. This information is captured in the form of a Benefit Management Plan (BMP). A BMP is made up of a benefit map and a statement of reporting and responsibilities.

ILM can help business case developers and Senior Responsible Owners (SROs) define and validate the business need and the benefits the solution is expected to deliver.

 Back to Major Projects Assurance page.

 

How ILM works: the workshops

Investment Logic is developed in a series of up to four, two-hour facilitated workshops.

'' 

'Intelligent discussion' sits at the centre of the workshops. The facilitator is responsible for:

  • extracting and telling the investment story in a way that will maximise its value to the organisation
  • expressing the investment story in plain language and concepts
  • obtaining the agreement of all participants to the outcome of the discussions
  • ensuring that each statement can be supported by evidence.

Accredited Facilitators are able to provide advice to Agencies about the type, number and timing of ILM workshops appropriate to their particular project.

Note: Where ILM is used in the context of developing a business case that must follow the New Zealand Treasury BBC standard, then the investment logic and the options are developed in up to five workshops. Workshops 3 to 5 are not part of the DTF Investment Management Standard and are unique to New Zealand -

1. Problem Definition
2. Benefits Definition
3. Investment Objectives
4. Identify (long list) Options
5. Assess/Shortlist the Options.

Workshop one - Problem Definition

The problem definition workshop aims to identify the problems that need to be addressed, and the high-level interventions that will address the problems. This is a departure from the way many projects are identified, as they are often solution-driven (such as a new financial management system, or a new building) and are proposed without the analysis of what the underlying problems are, and if the particular solution will address them.

The workshop can be quite challenging, as participants clarify their understanding of what the underlying problems are. This is essential to get a shared understanding before moving forward. At the end you will have produced an Investment Logic Map (ILM).

The interventions identified should be broad enough to not overly constrain different options for bringing them about.

There are three possible outcomes from this workshop:

  • The investment logic depicted will indicate a strong case to proceed further with the consideration of the investment
  • The investment logic depicted will indicate that the case to proceed further is weak
  • No coherent investment logic was able to be produced

Workshop two - Benefits Definition

If the decision is made to proceed, the second workshop is held to identify the benefits and key performance indicators (KPIs), measures and targets appropriate to them. The benefits identify the value the investment will return to the organisation. This is an important aspect of the ILM 'storytelling' as this will be what convinces decision makers that the investment should be made. This information can become an input to cost/benefit analysis, benefits realisation management, and alignment to organisational outcomes.

This workshop may have some different participants (the Investor should participate in both workshops). Much of the focus is on identifying meaningful, measurable and attributable KPIs and targets, responsibilities and timing for their measurement. These become indicators of the success of the investment if it is pursued. At the end the workshop has the Benefit Manaagement Plan is produced. This is a short document that defines the pre-requisites for the delivery of each expected Benefit, how the delivery of each Benefit will be measured, and who will be responsible for measuring and realising each Benefit.

Workshop three - Strategic Response / Options

The Strategic Response workshop aims to explore a broad range of potential strategic level options, assess them against core criteria. A preferred strategic option is selected and becomes the Strategic Response.  It should be run after completion of the Problem and Benefit workshops and is not stand alone.

The intent is to explore:

  • the strategic interventions required to deliver the identified KPIs (and respond to the problems defined in the earlier workshop one.)
  • how these interventions could be packaged into a range of sensible strategic options
  • which of these strategic options is preferred - taking into account the benefits to be delivered, cost, timeliness, risk and any disbenefits.

Workshop four - Solutions Definition

The three previous workshops established the need for an investment and the preferred strategic response. The fourth workshop aims to specify a solution consistent with the strategic response - a case of a problem driving a solution. The Solution Definition workshop will identify:

  • what business changes will be needed to implement the strategic response;
  • what assets (if any) will be required to support these business changes;
  • the defined solution (expressed as the changes and assets) and show how this investment will deliver the investment key performance indicators (KPIs) identified in the benefit management plan; and
  • what costs, risks, timeframes and disbenefits are associated with the defined solution.

This solution identification is essential for developing business cases in the public sector.

Again, apart from the Investor, some of the the workshop participants may be different to those at the earlier workshops, and people who can bring the relevant perspectives to the workshop should be invited. At the workshop itself, the draft solution is presented and discussed, and agreement on a preferred solution is added to the Investment Logic Map. The entire map is reviewed to make sure the story is understandable and compelling. In the last half hour of the workshop, the facilitator pulls together the presented information for subsequent development of an Investment Concept Brief (ICB). An ICB is a two-page document that depicts the logic underpinning an investment and identifies the likely costs, risks, dependencies and deliverables of the proposed solution. It is used to summarise the merits of an investment and allow decision makers to prioritise competing investments before proceeding to business case. The solution described at this point should be tested, validated and potentially changed in light of deeper analysis and evaluation conducted as part of the business case preparation work.

Back to Major Projects Assurance page.

 

Key Players

The Investor is a consistent participant in each workshop.

Investor / Investment decision-maker

The Senior Responsible Owner who has the business problem and is responsible for the delivery of the benefits.

Is there a need to do anything?

What benefits will be delivered?

Is the solution both strategic and innovative?

Does the existing scope miss any opportunities?

Stakeholder

A person who has strategic understanding of the problem and its impact.

What is the problem?

Is there evidence that articulates both the cause and effect of the problem.

How valuable are the benefits that will be delivered?

Analyst

Someone responsible for confirming that the assumptions underpinning an investment are valid. This should include one or two people from broader strategic policy areas of the organisation to ensure the problems identified are appropriately informed and challenged.

Is there evidence that the problem really exists?

Are the strategic interventions that have been selected the result of a broad and robust analysis?

Is the solution complete and is it consistent with the Interventions?

Benefit specialist

Someone who has experience with benefit specification and tracking within the organisation and often brings specific experience with particular business areas.

Is there evidence that the problem really exists?

Is it clear what the claimed benefits are will the solution really deliver them?

How can we track and monitor KPIs throughout the lifecycle of the investment?

Project or Programme Manager
(where one is in the post)

The person responsible for delivering all or part of the solution.

Is the solution practical?

Is it complete?

Will the solution deliver the KPIs required to prove the benefits?

Back to Major Projects Assurance page.

 

Using ILM for your project

Only accredited ILM Facilitators may provide ILM services to High Risk monitored government projects. Trained facilitators working towards accreditation will welcome the opportunity to run workshops for all other investment types.

To maintain brand quality, it is essential that facilitators running workshops have been properly trained and that their training is reasonably current. (The current version of DTF's Investment Management Standard for ILM is Edition 5.)

The methodology uses licensed templates and employs rules about their completion. DTF have identified some instances of low quality of work by a number of non-accredited facilitators and now require facilitators to note their status on the ILM template. Note: "Trained" is not the same as formally accredited by DTF.

DTF maintains a list of all people who have been accredited. If you wish to engage a particular facilitator you should contact the consultant direct. The full list of accredited facilitators is available on the DTF website:

As at October 2013, the accredited facilitators in New Zealand are:

For non-High Risk or monitored projects you may wish to engage a trained ILM facilitator who is based in New Zealand and working towards accreditation. If you are interested in using one, please contact majorprojects@ssc.govt.nz

Back to Major Projects Assurance page.

 

Becoming an ILM Facilitator

Only accredited ILM Facilitators may provide ILM services to High Risk monitored government projects.

To gain accreditation a facilitator must submit examples of their work for evaluation to DTF Melbourne. A facilitator typically submits an initial "desk" ILM followed by two to three more examples of live ILM workshops they have managed. Feedback is provided in writing and by phone. There is a fee for the accreditation process. For more details see the DTF website

DTF have awarded ILM training provision to Fankhauser & Associates in Melbourne. This company is the only company accredited to provide ILM courses. Regular courses are run in Melbourne and may be booked on line.

SSC has very successfully used Fankhauser & Associates to run both Awareness and Facilitator courses in its offices in Wellington, saving delegates air fares and hotel expenses. The courses typically take eight students and are role play based to provide practice in the preparation for and running of the investor workshops.

More recently, the Project Management Institute New Zealand (PMINZ) has been organising ILM training for PMI members and non-members. To register your interest in ILM training please contact PMINZ at central.spc@pmi.org.nz or SSC at majorprojects@ssc.govt.nz

Where there is sufficient demand, Fankhauser & Associates are also willing to run in-house ILM courses at an organisation's offices here in New Zealand. Agencies wishing to use this alternative approach should contact the company direct. http://facilitatortraining.com.au/page/courses

Back to Major Projects Assurance page.

 

ILM Communities of Practice

NZ Community of Practice

SSC runs ad hoc Community of Practice meetings for the New Zealand based accredited facilitators and those who are working towards accreditation. The aim is to provide a forum in which to learn from the more experienced practitioners. Members can bring questions about the ILM process or jobs they would like guidance on. It also provides an opportunity to share templates and good practice. 

To join the group please contact majorprojects@ssc.govt.nz

Note: Non government employees will be asked to sign a simple confidentiality agreement.

DTF Melbourne Community of Practice - for accredited facilitators only

DTF runs regular forums where all accredited facilitators for the Asia Pacific region are invited to hear how DTF is shaping and further developing the methodology. Members provide an input to the future use and direction of ILM and share experiences.

Back to Major Projects Assurance page.

 

ILM resources


Note: For examples of ILM maps and details of the training course content, please refer to the documents available for download in the Related Files section at the bottom of this page.
 


Back to
Major Projects Assurance page.

 

2 Quantitative Risk Analysis of Costs (QRA)

Quantitative Risk Analysis (QRA) of costs is mandatory as part of the development of a Detailed Business Case (DBC) for High Risk major projects or programmes monitored by the SSC and Treasury.

QRA assesses cost elements in the business case in terms of the risks that might impact on achievement of estimates. This is typically assessed through a mix of project documentation review, interviews and workshops to determine the full range of risks applicable to a particular element, plus the likelihood and potential impact if they occur. Data is then processed through a QRA calculation model, normally including a randomising routine such as a Monte Carlo simulation, to predict as accurately as possible the likely range for each element. It also predicts an estimate of the likely range for total costs.

QRA should be used to more accurately estimate the likely costs of the project. It can be used to validate requests for appropriations, assess the level of contingency funding required and inform decisions about cash draw-downs for major IT projects. It should not be used if relevant Ministers agree that the size and nature of the project does not warrant this approach.

QRA provides several important benefits for a project:

  • a better understanding of the sources of risk to an investment's costs (and hence direct cost savings through better risk management of costs)
  • a more accurate estimate of the likely cost outcome for an investment.
  • a basis for the estimation of a project's contingency budget

2012-2015 Quantitative Risk Analysis (QRA) Panel Members Selected

In August 2010, Cabinet mandated that all high risk projects must include the results of a Quantitative Risk Analysis (QRA) of its project costs. QRA analysis is required by Cabinet to be used as the basis for all project appropriations, access to contingency funding and cash draw-downs. QRA provides visibility of the range of likely cost outcomes for a project. It also reduces the costs of projects by creating a greater understanding of the risks to a project’s costs.

Rather than access these services on an as-required, ad-hoc basis, the State Services Commission (SSC) established an expert panel in March 2008 to supply QRA services to all government departments. This panel was initially for a 3-year period. Recently, the Commission reviewed the expert panel and has subsequently entered into an agreement with six QRA qualified suppliers to form an expert panel of preferred suppliers for government agencies to continue to access high-quality, QRA services at a reduced cost from one central point.

The QRA panel members for 2012-2015 were selected through a tender and interview process. They comprise:

Back to Major Projects Assurance page.

 

3 Training for Programme and Project Sponsors/Senior Responsible Owners)

Business strategy and change are enabled and delivered through projects and programmes. These require a leader and champion to sponsor and take accountability for successful delivery.

The Capability group has tailored a Programme and Project Sponsorship Masterclass CBT course to meet the needs of New Zealand Project and Programme Sponsors and has negotiated a reduced cost for New Zealand government staff.

This programme is available to New Zealand government staff through the LearningState website as part of the LearningState suite of Programme and Project Management related courses.

The NZ Gateway Unit's Lessons Learned report, based on findings from 50+ Gateway reviews and the first 10 years of SSC monitoring of major projects in NZ government, contains useful information that can help SROs drive their projects and programmes to a successful outcome.

SRO training

Programme and Project Sponsorship (PPS) Masterclass training for SROs is provided by the ILX Group (www.enquiries@lxgroup.co.nz/pps) as a series of e-learning modules www.ilxgroup.com/nz/programme-project-sponsorship-pack.asp

An 'overview' flyer for this course can be downloaded from the Related Files section at the bottom of this page.

Back to Major Projects Assurance page.