5 Methodology and Processes: Design, Costing, Procurement and Scheduling

Introduction

5.1 In this section, the methodology and processes employed by the Department to design, cost, procure and schedule the projects is reviewed. Relatively more emphasis is given to procurement. This is because the Department chose to use the CWA methodology for the procurement of Spring Hill and Otago. CWA is not a common form of procurement in New Zealand, although it is used by Transit on some road projects. Reflecting the relative newness of the CWA methodology in the New Zealand context, the Department's experiences with it provide some valuable insights as to its advantages and disadvantages compared to more traditional forms of procurement.

Design

5.2 The origins of the design for the Spring Hill and Otago facilities have linkages with designs in Australia and Canada. As noted in section three, the design for Spring Hill was broadly aligned with that for Northland and the design for Otago was broadly based around that for Auckland Women's. That said, however, there are significant differences. For example, Otago involves differences in heating, ventilation and insulation reflecting the cooler climate. Otago also has other design differences compared to Auckland Women's which is a women's prison. Similarly, the different scale of Spring Hill and Northland led to some design differences.

5.3 We have not reviewed in any detail the approach to the development of the designs used for Spring Hill and Otago given that the origins of the designs pre-date both of these projects. Notwithstanding this, several points can be noted.

• The CWA methodology has the advantage of promoting close interaction between the construction contractors, the design teams and the client in the development of detailed designs. In contrast, under traditional forms of procurement, it is not unusual for design contracts to be let, followed by construction contracts once designs are largely completed. The advantage of the more integrated approach under CWA is that there is greater scope for assessing the implications of design on construction and on operations and, hence, greater scope to adjust designs to achieve the optimal balance between design, construction and operational considerations.
• The Northland project was the first of the prisons to use the new style of facility. Several lessons were learned from the construction of Northland that assisted in refining the design to achieve better outcomes from an operational viewpoint. Reflecting the relative newness of the design, the Department developed standards through the design process for Northland. It was only following the completion of Northland that standards were finalised and documented. The Department took advantage of these lessons by incorporating them into the Spring Hill and Otago projects.
• The relative newness of the prison design also placed added pressure on design resources. These resources were, at times, stretched. In the case of Otago, pressure was incurred through the reallocation of design resources from Otago to Auckland Women's (approx June - December 2004) to address the capacity increase from 150 beds to 286 beds.
• Although outside the scope of this review, we note that there were issues around design in the case of the Northland project. As discussed later in this section, the CWA methodology provided a mechanism for addressing these issues and for ensuring those difficulties were not repeated for Spring Hill and Otago.

Costing

5.4 The approach to costing has several phases:

• various initial cost estimates, based on high level $/bed comparisons were made in 2000 for Spring Hill and Otago. These were updated at various points over the period 2001 to 2003;
• more substantive estimates were made as part of Budget 04. These were subsequently updated as part of Budget 05 to take into account construction price increases over the intervening period; and
• the latest cost estimates were reported to Cabinet in December 2005. These estimates incorporate the outcome of the detailed work that has been undertaken as part of the development of the TOC for both projects.

5.5 We focus on the second and third of these phases. The initial phase of cost estimates in 2000 - 2003 should be seen as preliminary only. They did not have any material bearing on whether or not to proceed with the Spring Hill and Otago projects (or to what specification) as the key funding decisions were taken as part of Budget 04 and Budget 05.

Budget 04 Estimates

5.6 The majority of the funding for Otago was approved as part of Budget 04 (and significant funding decisions were also taken with respect to Spring Hill). Accordingly, the estimated completion costs provided as part of Budget 04 took on considerable significance. At that time, the costs for Spring Hill and Otago were signalled to be $248 million and $161 million respectively.

5.7 Because of the importance attaching to the estimates provided at that time, it is important to understand the context within which the estimates were provided and the basis upon which they had been developed.

• The outcome of the Northland tender was taken into account in developing the estimates of cost for Spring Hill and Otago. The tender was analysed and the rates were applied to an area schedule for Spring Hill and Otago. The significance of this is that the prices tendered for Northland were approximately $12 million above the estimates for Northland pre-tender. In the case of Otago, estimates developed in connection with Auckland Women's were also used to inform the rates for Otago. Auckland Women's was, at that time, at the "developed design" stage with several detailed estimates having been completed.
• Northland was tendered as a traditional NZS 3910 form of contract (only later did the procurement method for Northland modify into a hybrid form of CWA). By implication, while the tender prices for Northland were visible to the Department, the actual costs associated with the project were not transparent (as is always the case with NZS 3910 contracts).
• The estimates were made by the Department with input from its independent quantity surveyors - Rider Hunt. At the time of Budget 04, the RFP for the Spring Hill and Otago projects had yet to be issued, so the estimates were made well in advance of the CWAs being formed.
• The estimates made as part of Budget 04 were based on concept designs. Detailed design work had yet to occur. By implication, the Budget 04 estimates could only be indicative at best.
• The approach to estimating costs at that time rested on applying $ rates per M² of gross floor area (taking into account the Northland tender prices). However, benchmarking the rates applied to Spring Hill and Otago with other projects, including Northland, was problematic because of the greenfield nature of the Spring Hill and Otago sites. With the advantage of hindsight, it is also apparent that the remoteness of the Spring Hill and Otago sites has played a significant part in explaining the differences between the latest estimates of cost and the estimates communicated in Budget 04.

5.8 In short, the basis upon which the Budget 04 estimates rested were, at best, indicative only. Reflecting the basis upon which they had been developed, the estimates provided as part of Budget 04 had quite high levels of risk and uncertainty attaching to them. However, the indicative status of the cost estimates could have been made clearer in the business cases submitted to Cabinet as part of Budget 04. Because the cost estimates were presented to three decimal places consistent with Budget process requirements, there is a risk that Ministers would have interpreted the estimates as conveying a high degree of certainty. Any such perception needed to be corrected.

5.9 The business cases submitted as part of Budget 04 noted that escalation was a significant risk for the Department (and, by implication, the Crown). However, the magnitude of the risk was not quantified. Ministers could not have gauged from the business case whether the risk amounted to a small amount or, as it turned out, tens of millions of dollars. In our view, there was a need to indicate more clearly the status of the cost estimates including, in particular, the level of certainty attaching to the estimates.

5.10 We note that contingency funding was also provided for, but the interpretation of what the contingency represented was not made clear in the business case presented to Ministers.

Budget 05 Estimates

5.11 The next major reporting milestone to Cabinet was in April 2005 as part of Budget 05. At that time, the costs associated with Spring Hill and Otago had been revised upwards to $283 million and $175 million respectively. In the case of Spring Hill, the explanation provided to Cabinet for the increase of approximately $34.5 million since Budget 04 was not particularly clear but appeared to be a function of adjustment for further escalation. With respect to Otago, the total estimated cost increased by approximately $13.8 million compared to Budget 04. Almost all of the increase was attributed to escalation (there was a further small amount relating to commissioning).

5.12 At the time of Budget 05, work on Northland and Auckland Women's was underway. However, both projects were of limited usefulness in terms of informing cost estimates for Spring Hill and Otago.

• Northland has a hybrid form of CWA. As such, the project was being completed under fixed price conditions and the Department did not have visibility over the actual costs being incurred by the project.
• Auckland Women's was a full CWA approach and progress reports indicated that costs were tracking according to plan and, accordingly, did not provide any early-warning signals of impending cost issues that were to affect the Spring Hill and Otago projects.
• The Auckland Women's would not have provided detailed insights regarding Spring Hill in any event given the different designs between the two facilities (and to some extent the same point applied with respect to Otago which is a men's prison).

December 2005 Estimates

5.13 During the period from April 2005 to October 2005, work on the TOC for Spring Hill and Otago proceeded (at the same time that construction was also proceeding).

5.14 The TOC is, in effect, the agreed budget for the project. It is intended to include all project costs except those specifically defined and agreed as being excluded and, hence, the responsibility of others. The TOC is, in theory, a pre-construction estimate of the efficient level of cost to complete the project. It has the following components:

• Direct costs. These comprise labour, materials and plant. Labour rates are "burdened" to reflect personnel-related costs such as ACC, sick leave, annual leave and, if appropriate, allowance for travel time, protective clothing and so on. In the context of the RPDP, the process of calculating burdened labour rates has been undertaken by CMS and reviewed by Sherwin, Chan & Walshe. There is also a burdened rate for plant that includes depreciation, repairs and maintenance and consumables (e.g. fuel).
• Normalised Profit Margins. Each contractor (and sub-contractor) that is part of the CWA can expect to earn their "normalised" profit margins in addition to recovering their actual costs and overheads contribution. In the case of the RPDP projects, normalised profit margins have typically been established with reference to the previous three years financial accounts of the contractor. Adjustments have been made to smooth out unusual/abnormal profit results (either good or bad), the current state of the construction market and levels of risk involved. This work has also been performed by CMS and reviewed by Sherwin, Chan & Walshe. Once margins are determined, they are "ring-fenced"; that is, the dollar value assigned to margins does not change unless there is a substantial change to the scope of work involved.
• On-site overheads (also referred to as Preliminary and General or P&G). This includes, for example, the costs associated with on-site project managers and management team, project offices, site stores, administration staff and so on.
• Off-site overheads. This recognises that an element of the off-site corporate (e.g. head office) overhead is needed in order to support the project. As with the margins calculation, a normalisation process is involved with off-site overheads and once the level of off-site overhead is determined, it is "ring-fenced".

5.15 It is important to emphasise that the TOC focuses on cost. In contrast, under a traditional NZS 3910 form of contract, the tender quoted is a price; that is, it is the amount the client pays for the project as distinct from what it actually costs to complete the project (subject to any agreed variations). Because the TOC focuses on cost as distinct from price, it implies a fundamentally different basis for the allocation of financial risk. Under a strict fixed price contract, financial risk is substantially borne by the provider (although provisions relating to contract variations can result in sharing of financial risk). In contrast, under a CWA as adopted by the Department, it is not until the project has been completed and the final costs accounted for, that the client has certainty over the total amount payable for the project.

5.16 Although not accepted by the Department at the time of preparing the report that went to Cabinet in December 2005, the TOCs were at an advanced stage. The draft TOCs indicated that costs had increased to $381 million and $218 million for Spring Hill and Otago respectively.

5.17 The December 05 report to Cabinet included a description of the factors leading to the cost increases compared to the Budget 05 estimates. In brief, the contributing factors were grouped under the headings of market influences, regulatory and consent changes and omissions from earlier estimates (items seen by the Department as not previously included rather than items inadvertently overlooked). While the increases and underlying causes are site-specific, in the case of escalation, there is considerable overlap between the two sites and this issue is considered first.

5.18 From the Crown's perspective, it has a keen interest in achieving project completion on time, within budget and to specification. In normal fixed price contracts, price is established up front as part of the tender process (although actual price outturn is often the subject of contract variations). The way in which CWA has been used by the Department means that there is no price information arising from the selection process. Development of the TOC (and related QRA and gain/pain share arrangements) is, therefore, of critical importance, particularly from a fiscal management perspective.

5.19 Building the TOC is not a straightforward process. In very general terms, the processes around its development are illustrated below.

5.20 Several points should be noted.

• Detailed designs need to be in place before any serious attempt can be made at determining the TOC.
• The time taken to develop the TOC once detailed designs are completed will depend on the size and complexity of the project. However, as a guide to good practice, a time frame of 2-4 months should be expected.
• The process of determining the TOC is heavily dependent on transparency of financial and other data and on independent scrutiny of the numbers, assumptions and calculations. In general, CWA involves an "open book" approach to the determination of costs. Transparency and independent scrutiny are the mechanisms through which assurance is gained that costs are efficient (on the assumption that there is no market testing of the cost estimates through a tender).
• The process is very iterative. The client will (should) have a budget in mind. Initial estimates of the TOC may exceed this expectation in which case a process of value engineering is used to strip out unnecessary expenditure and explore more cost effective ways of achieving the client's requirements and budget expectations.
• A lot hinges on getting the TOC in place as soon as possible. Until the TOC is agreed, the commercial CWA partners are remunerated according to the actual costs they incur. Until the process of independent scrutiny of direct costs, audit of margin, overhead and burdening calculations and value management is completed, the client does not have final assurance that the estimated costs represent an efficient level of costs. Moreover, until the TOC is in place, the gain and pain share arrangements cannot be finalised. The gain/pain share provide an important and further spur to efficiency and delivery of the project within, or in excess of, client expectations.

5.21 In relation to this last point, a key point to understand with alliancing is that while it offers benefits over traditional contracting methodology, it raises new and different risks that have to be managed. In particular, there is a balance to be struck between having faith in the collaborative nature of alliancing and the need to protect the client's financial interests. CWA does not involve a fixed price from the client's perspective.

TOC Development Timing

5.22 The TOC plays a pivotal role in CWA agreements. At the time that the Spring Hill and Otago tenders were awarded (to Mainzeal and Hawkins respectively), there was an expectation that the CWA Agreement and TOC would be finalised within a matter of months. Negotiation Framework Agreements³ (NFAs) were entered into with the CWA partners at both Spring Hill and Otago. In both cases, the NFA was agreed through to 15 January 2005 during which the parties would work together to develop an accurate TOC. It is worth noting that, in the context of the Auckland Women's project, it was expected that approximately nine weeks would be required to finalise the CWA (including TOC).⁴

5.23 In the event, the development of the TOC has taken considerably longer than the initial expectation. In respect of Spring Hill, the Steering Group accepted the TOC as presented on 21 June 2006 and the CWA Agreement is being finalised for execution. In the case of Otago, the TOC was approved by the CWA commercial members and the Department on 25 May 2006. The CWA Agreement is currently with the CWA members for execution.

5.24 The NFAs for Spring Hill and Otago have been rolled over multiple times (at least 7 times in the case of Spring Hill since October 2004). On several occasions, the roll-over of the NFA took place after the expiry of the preceding agreement. It is not clear what the legal position of the parties was on such occasions. At several points during 2005, revised timetables for completing the TOC were advised to the Steering Group. For example, in May 2005, it was expected that 30% of the Spring Hill project would be scoped into a TOC by mid-July 2005 with 80% scoped by mid-August. By implication, there was clear expectation that elements of the Spring Hill TOC would be progressively known through the middle of calendar-2005, but we note that the numbers around the TOC were not being communicated to the Steering Group during this time.

5.25 In the case of Spring Hill, the Department considered in mid-May 2005 whether to delay construction work until the TOC was in place or to proceed on a fast-track basis. The latter option was taken on the understanding that the first TOC would be available in mid-July 2005. In the event, this deadline was not met. It was not until around late-September that the TOC for Spring Hill was beginning to be disclosed and slightly earlier in the case of Otago.

5.26 The long period over which the TOC was developed created financial risk for the Department. During the period prior to completion of the TOC, the commercial CWA members are effectively reimbursed according to cost. While such costs are transparent and subject to review by the Department and independent advisers, the fact that critical TOC information did not emerge sooner meant that the Department (and Government more generally), was denied opportunities to consider options for reducing cost in light of the higher than expected cost. Moreover, prior to agreeing the TOC, it was not possible to agree the gain and pain share arrangements. As a result, the added incentives that these arrangements bring were missing. In the case of Otago and, to a somewhat lesser extent Spring Hill, the timing of the TOC meant that the projects were well underway before the TOC was agreed. This situation is not consistent with best practice CWA methodology.

5.27 The fact of the substantial delay in getting the TOC in place obviously raises the question as to why the delay occurred. In this respect, we consider that there is a combination of factors.

• The Department had no experience in CWA methodology. Experience in CWA was being gained as work on the RPDP projects progressed.
• Pressure on design teams meant detailed designs took longer to finalise than was expected (detailed designs are a necessary prerequisite for TOC development). We understand that the issue around design related more to building services than to the buildings themselves (for example, designs for underfloor heating for Otago were not complete until August 2005).
• The sheer volume of work involved in reviewing and independently verifying TOC estimates is considerable. We consider this was under-scoped by the Department and its advisers.
• The CWA methodology has been pushed down the supply chain to a very considerable extent. We understand that at the time of this review, approximately 95% by value of the cost incurred and committed on Spring Hill was captured by the CWA methodology (i.e. only 5% or work was being performed under standard NZS 3910 procurement). We question whether it is sensible to drive the level of CWA as far as this. A consequence of this approach is that it adds significantly to the workload involved in developing a TOC. Our understanding is that the successful Auckland Women's project had around 60% of cost under a CWA arrangement.
• There is a large number of partners involved in the CWA (there are 17 CWA members on the Otago project). The remoteness of the sites, particularly in Otago, has meant that the net for contractors has had to be cast widely so much so that, at the peak, approximately a third of the workforce on the Otago site is "imported" from outside of the coastal Otago area. Increasing the number of contractors increases the number of margins and overheads that have to be calculated. More fundamentally, the larger the number of CWA members, the more training in CWA that needed to be undertaken.
• There was generally insufficient experienced estimation and cost planning expertise available in the market and who were capable of undertaking the first principles approach to cost determination that is characteristics of CWA. Moreover, the expertise that did exist was more used to short-term projects rather than the major and long-term nature of the prisons project.
• Governance and project management arrangements prior to changes in mid - late 2005 have not, in our view, been as tight as they needed to be. This point is discussed further in the next section. However, issues regarding project management meant that the range of factors that were impacting on the timeliness of the TOC were not being managed as closely as they warranted.
• The commercial CWA members, who were being reimbursed for costs incurred, did not necessarily face strong incentives to expedite the development of the TOC. That said, however, we note that incentives can change. For example, we understand some of the CWA principals involved in the Otago project entered into an arrangement to underwrite risks faced by some of the sub-contractors, stemming from potential time delays. This may well have contributed to the more timely completion of the proposed TOC for the Otago project (taking into account the design issues noted above).

5.28 In summary, several resource constraints impacted on the timeliness of the TOC. These included pressures on scarce design resources and a relative lack of experienced estimation and cost planning expertise. The timeliness of the TOC was also influenced by the heavy demands being placed on management of the CWA in part reflecting the extent to which the CWA has been driven down the supply chain. The resource constraints and demand pressures are, however, matters that are capable of being anticipated and managed.

5.29 Notwithstanding these points, the Department faced a difficult set of circumstances. The need to maintain progress with the projects so as to ensure their completion in time to meet expected prison population growth meant that the option of delaying the project pending completion of the TOC was not a realistic option.

5.30 In short, from a best practices perspective, it is not desirable to have the TOC being developed so far into the construction phase. A key lesson arising from the Spring Hill and Otago projects is the need to ensure that the TOC is developed much sooner in the process so as to minimise the amount of construction undertaken without there being an agreed TOC.

TOC Development - Independent Assurance

5.31 One of the guiding principles governing CWA is the need to ensure that costs are efficient and represent value for money. The Department is part of the TOC development process and, as part of this, it reviews and assesses the TOC with the objective of ensuring that it is receiving value for money and, consistent with this, that the TOC is based on an efficient level of prices.

5.32 To assist in achieving these objectives, the Department obtains advice from a wide range of third parties as shown below.

5.33 While use of outside expertise is desirable, several points can be made.

• There is a very heavy dependence on CMS. From a risk management perspective, having such a high proportion of the overall knowledge of the projects in one (small) firm needs careful management. The Department has sought to manage this through the roles performed by Sherwin, Chan & Walshe (a chartered accountancy firm). Their roles include review of margin calculations performed by, and the methodologies used by, CMS.
• There is a need for high degree of independence because of the need to challenge the draft TOC numbers being prepared by the commercial CWA partners. The Department engaged CMS in respect of this. CMS has a key role in facilitating the CWAs and, as part of its brief, takes responsibility for the quality of work going into the development of the TOC and plays a key role in determining margins and overheads which are key inputs to the overall determination of the TOC. In this regard we view CMS' role as being very much on behalf of the CWA; not the Department. However, it is clear that the Department has placed considerable reliance on the advice of CMS. Notwithstanding the role performed by Sherwin, Chan & Walshe, we consider that the Department should seek to supplement CMS' role with another adviser expert in CWA methodology in order to ensure the level of independence needed in order to effectively counter-balance and challenge the TOC estimates being submitted by the commercial CWA members.
• Rider Hunt has responsibility for reviewing the direct costs (labour, materials and plant). To undertake this role, Rider Hunt compares the quantities and values in the TOC against a database of information built up from many other projects. A difficulty that Rider Hunt faces is that it is being asked to review labour rates that do not include any element of profit or overhead. In traditional forms of contracting, labour rates do include profit and/or overhead (i.e. a builder's hourly rate will normally include profit and a contribution to overhead). Rider Hunt does not, however, have access to margin or overhead data. While noting that there are commercial sensitivities around profit/overhead information, we consider that it would be better to provide Rider Hunt with access to this information as it would provide a better basis for cost comparisons.
• More generally, the Department, as well as Rider Hunt, did not have full transparency over margins and, in this respect, the way in which CWA was implemented did not fully accord with the "open book" requirements of the CWA methodology. Only CMS had full transparency over margins.

5.34 The process for determining margins and overheads involves extensive detailed unbundling of contractors' accounts. There is potential for costs to be double counted. Considerable judgement needs to be applied by CMS in the process of trying to normalise the accounts and ensure that costs are counted only once. Sherwin, Chan & Walshe are retained by the Department as an independent check on the margin and overhead calculations performed by CMS. However, because the Department does not of itself have visibility over margins, it is totally reliant on the advice of third parties. Consistent with the open book principle upon which CWA is based, and recognising that the margin calculations are an important part of overall cost determination, we consider that the Department should have greater direct visibility over margins and their calculation.

Development of the TOC - Lessons Learned

5.35 Given the pivotal role played by the TOC, its development needs to be concluded much sooner in the overall process than has been the case for the regional prisons. Several factors need to combine in order for this to occur.

• First and foremost, there needs to be more flexibility over the lead time for constructing the prison. A key issue affecting the regional prisons was the need to adhere to specified completion dates in order to accommodate the growing prison population. Realistically, the Department did not have the choice of deferring construction of Spring Hill and Otago to allow for the development of the TOC prior to starting construction.
• Related to this last point, the Department needs to take steps to ensure that appropriate levels of design capability are available to take designs from concept status through to detailed plans status so as to provide the basis for detailed TOC development work. Design was a bottleneck in the prisons' development.
• Timely completion of the TOC depends on there being completed and agreed designs. In part, delays in completing designs were a function of scarce design resources but, in part, there were design changes late in the process (e.g. in relation to site services at Otago).
• Pressure has to be maintained on the commercial CWA partners to ensure early development of the TOC. Under cost plus margins arrangements there is an inherent risk that the commercial partners do not necessarily have strong incentives to conclude early agreement on the TOC. The TOC is an iterative process and requires the client to be in a position to challenge elements of the TOC from an informed position. This requires appropriate levels of resource and flexibility and nimbleness in decision making. Structures and processes need to support this.

5.36 The corollary of ensuring the early completion of the TOC is that the amount of work that is undertaken on fast-track basis should be minimised. Getting the TOC determined before work starts maximises the opportunity for Government to make changes if costs turn out higher than expected.

Procurement and Scheduling

5.37 The method under which the prisons have been constructed is a major part of this review. As will be discussed below, a hybrid form of CWA was adopted for the Northland project. A key reason behind this was the need to address some issues with that project that had the potential to adversely affect the timing of that project.

5.38 Prior to the Northland project, prison construction in New Zealand had been procured by a fixed price form of contracting (more formally, procurement had been undertaken pursuant to the NZ standard NZS 3910). This form of procurement is commonplace across a wide range of construction activity. In a sense, it is the traditional form of procurement.

5.39 In contrast, Spring Hill and Otago have been procured using the CWA, or alliancing, form of procurement. There are major differences between the two forms of procurement. It has not been within the brief of this review to assess the merits or otherwise of the CWA versus more traditional forms of contracting. However, it is useful to have an appreciation of the differences involved and the key features of CWA methodology.

Key Features of CWA Methodology

5.40 CWA is one form of a more general form of procurement referred to as alliancing. Alliancing can take many forms, examples of which include:

• Traditional joint ventures - until recently, the joint venture, in which two or more parent organisations join forces to create a separate legal entity, has been one of the most common type of alliance. In the traditional joint venture, organisations unite to obtain economies of scale and scope.
• Outsourcing arrangements - these are partnerships in which an organisation outsources a business process or function (e.g., human resources or information technology) that is not central to its business mission. This allows the organisation to concentrate on its primary business objectives.
• Marketing alliances - marketing alliances bring together organisations that market different products or services to the same group of consumers. They can take many forms, including co-packing alliances and co-operative advertising. Under a co-packing alliance, an organisation licenses the use of its brand name and product formula to organisations that have production, packaging and distribution capabilities.
• Supplier/vendor alliances - closely related to outsourcing arrangements are supplier (or vendor) alliances in which organisations partner with suppliers to enhance the service, each bringing its specific area of expertise to bear on the project or service.

5.41 The CWA is closely related to the last of these forms.

5.42 A key feature of CWA (and alliancing more generally) is that all parties in the alliance have a shared interest in overall project outcomes. Alliancing approaches share a number of distinct features that set them apart from more traditional forms of procurement. Included among these are:

• alignment of the business goals of all parties;
• selection of parties in the CWA agreement is based on capability, approaches and systems (as in traditional evaluation processes) and more subjective assessment around the level of project commitment, the composition and nature of the sponsor team, the likelihood of the combined team delivering outstanding results;
• a selection process that may not include price;
• joint risk allocation and management;
• joint and integrated project governance and management;
• fully open book among the parties implying very high levels of financial and other disclosure;
• a principles-based collaborative form of contract that seeks to avoid adversarial conduct. This means, for examples, that there are no disputes resolution processes in CWA agreements. Where problems arise, the parties work collaboratively to identify solutions rather than seeking to "point the finger of blame";
• a cost reimbursement process that is open and pays actual costs and agreed margins (profits and overheads);
• an equitable and appropriate risk and reward scheme (referred to as pain share and gain share arrangements);
• an active relationship development and maintenance regime; and
• high levels of support from parent companies in support of the contracting arrangement (as reflected by senior representation on what is referred to as the Principals Group).

5.43 The CWA approach is aimed at fostering a best-for-project approach through a "no-blame-no-claim" culture where all parties have a unity of purpose to achieve a successful outcome. Decision making in regard to the project is conducted through a Principals Group. Decisions by the Principals Group are made on a unanimous basis; that is, all parties have to agree.

5.44 In traditional fixed price contracts, the principal point of focus is the price that is paid for the services. Under the CWA approach, the focus is on the efficient level of costs because this is the amount that the client, in theory, pays for the services.

Background To CWA for Prison Construction

5.45 The CWA methodology has been used in the context of the Northland (albeit in hybrid form) and the recently opened Auckland Women's prison. The Northland prison commenced as a traditional NZS 3910 contract. However, by August 2003, several factors pointed toward the need to modify the contractual approach to construction:

• financial pressure on one critical consultant was such that it was doubtful if they would continue;
• there was evidence of on-site "man-marking" where significant effort was being directed toward allocating blame rather than resolving issues; and
• the construction programme was not universally agreed among the parties involved.

5.46 In considering the options open to the Department, CWA was raised as a potential way forward. Several points are worth noting in this regard.

• We understand that Mainzeal had for some time been considering alternative forms of procurement including alliancing.
• Stewart Rix (of CMS) had considerable experience of CWA from previous roles in the United Kingdom (where CWA and alliancing generally is more commonplace). A meeting between CMS and the RPDP Project Director (John Hamilton) to present on CWA was arranged at the instigation of the Chair of the Inmate Employment Advisory Committee. CMS also presented to Hawkins Construction Ltd (the main contractor on the Auckland Women's and Otago projects).
• When difficulties with Northland emerged, the RPDP Project Director approached CMS out of which came a suggestion to convert the contractual framework for Northland to a CWA agreement (the Northland project had been running on-site for approximately six months). Mainzeal were the prime contractors for the Northland project.

5.47 At the August 2003 meeting of the RPDP Steering Group, the Project Director provided an oral briefing on the proposal to enter into a hybrid form of CWA (the hybrid arrangements retained an element of the former NZS 3910 contract including, in particular, a cap on construction costs). The meeting approved in principle the change to the hybrid CWA subject to the condition that a formal paper for approval be tabled with the Steering Group.

5.48 Following the decisions to adopt the hybrid methodology, the Department decided to retain CMS to assist with training and to provide advice on managing the transition to the new arrangements. Both aspects of CMS' role were vital: the Department had no prior experience of CWA.

Choice of Procurement Methodology

5.49 Subsequent to the decisions taken with respect to Northland, the Department has moved to adopt CWA for the Spring Hill and Otago projects (as well as Auckland Women's).

5.50 CWA represents a fundamental shift in procurement method. Accordingly, we would have expected a first-principles analysis of this, and other, procurement options to have been considered and debated at length before moving toward its adoption across the other projects in the RPDP. As part of this review, we have not, however, sighted papers provided to, or minutes of, the Steering Group that explicitly affirm the choice of CWA ahead of NZS 3910 (or any other form of contracting method) for either Spring Hill or Otago.

5.51 In the context of the Northland project, the lack of in-depth analysis and consideration could perhaps be explained by the immediate and significant set of problems that needed to be addressed. CWA provided a framework to make progress with the project and avoid the time delay and cost implications that could have eventuated had the project either remained as a NZS 3910 or been re-tendered.

5.52 Notwithstanding these circumstances, however, the decision to adopt CWA for the Spring Hill and Otago projects should have been justified on a project by project basis. The analysis should have extended not just to the traditional NZS 3910 contract but to the various forms of alliance arrangements and other options beyond alliancing such as:

• reimbursable contracts under which the client pays for all of the work plus a profit margin to the supplier;
• a construction management approach under which the client employs a construction manager who works for a fee which is applied to the value of the work;
• design and build, and design, build and operate contracts; and
• design, build, operate and finance contracts.⁵

5.53 It is not the purpose of this review to advocate these, or any other, forms of procurement. The point is that the Department chose to move away from NZS 3910 to a fundamentally different form of procurement. This decision should have been based on detailed first principles analysis of CWA and other forms of procurement.

5.54 This issue is not just a matter of good process for two main reasons. The first of these is that CWA potentially exposes the client to financial risk unless the CWA approach is well managed. To the extent that CWA was discussed with the Steering Group in the context of Northland, it is clear that the potential benefits of CWA were described. What is less clear is whether the risks and costs around CWA were explained along with the critical factors that must be achieved in order for a CWA to work well and in the client's interests. The lack of paper trail makes this difficult to evaluate but our impression is that the benefits of CWA were emphasised to a much greater extent than the potential risks and costs involved.

5.55 The second reason why the relative lack of consideration of procurement options is an issue is because prior to Northland, the Department had no knowledge or experience of alliancing. The need for rigorous consideration of this and other procurement options is even greater, the lesser is the knowledge and understanding of something new.

5.56 These comments should not be interpreted as a criticism of the CWA methodology. Indeed, we note advice from the UK National Audit Office (NAO) recommending that departments not use traditional forms of procurement for construction projects unless it can be demonstrated that this form of procurement will provide better value for money ahead of design and build, or alliancing (the closest form of which is described as "prime contracting" by the NAO) or PFI.⁶ Moreover, the comments should not be interpreted as saying that CWA was the wrong choice. In this regard, we note that the Auckland Women's project appears to have been delivered on time and within budget. In the case of the Spring Hill and Otago projects, independent advice provided to the Department in October 2005 advised that the Department should not depart from the CWA path to another form of procurement unless as an absolute last resort (although this advice was tendered at the time when construction of Spring Hill and Otago was well underway).⁷

5.57 Notwithstanding these points, we consider that the choice of procurement approach for any given project should be made on a case-by-case basis. In this regard, there are several factors that need to be considered before choices regarding procurement method are made. Below, we set out some high-level characteristics that would tend to favour use of CWA in preference to traditional NZS 3910 forms of procurement. CWA is more likely to be preferred when the following conditions are characteristics are present.

• The best way of achieving desired outcomes, in terms of the choice of technologies and processes, is uncertain. In the context of the regional prisons, their design was markedly different to that which had gone before and reflected a desire for a different approach to the management of inmates. By implication, however, there was uncertainty around the details of design and how this would interface with operational requirements.
• There is a high degree of complexity in design, construction, technology and/or development which cannot be satisfactorily scoped at project commencement. Complexity per se does not favour CWA over other forms of procurement, but where complexity gives rise to risk and uncertainty CWA may provide a more flexible contracting methodology. In the context of the Spring Hill and Otago projects, it is generally the case that the construction techniques are not new or overly complex. While there is a considerable amount of technology that goes into prison buildings (security related), this is separable from overall construction (as indeed was the case with a separate CWA for security systems). Of themselves, the building designs do not appear to be overly complex although, as noted above, there was uncertainty around aspects of design.
• Technology is rapidly evolving. This does not appear to be a characteristic relevant to prison buildings.
• Timeframes for completion of the project are tight and, accordingly, the project demands innovative and flexible approaches to ensure project completion. This clearly is a feature of the Spring Hill and Otago projects and, arguably, is the strongest reason for preferring a CWA methodology for these facilities.
• There is considerable scope for value engineering into the early stages of the project (at development and design stages). This is another way of saying that there is uncertainty over the optimal design approach and, further, that the client wants to explore rather than getting locked into a particular approach under traditional NZS 3910 procurement.
• There is a necessity for innovation and step-change developments in design, technology and construction methodology. CWAs create a better framework than traditional forms of contracting for such developments to take place. Arguably, elements of this requirement existed for the regional prisons in respect of design, but less so with respect to technology and construction methodology. As experience with the regional prisons design grows, this characteristic may become less relevant.
• The projects are large. CWA has a significant management overlay because many checks and balances have to be built into the procurement process to compensate for the lack of market testing of prices through competitive tender. Various people interviewed during the review have suggested that the minimum size of project that can support CWA is probably in the order of at least $100 million. Clearly, the Spring Hill and Otago projects meet this condition.

5.58 In addition to these points, comments by various Department and contractor personnel have suggested that CWA might be better suited to projects being undertaken in a heated construction market, but less appropriate when market conditions are benign or declining. Various reasons support this view:

• The CWA methodology provides transparency over margins and seeks to normalise margins. The process of agreeing margins under CWA therefore enables adjustments to be made to margins such that abnormally high margins experienced at the peak of the market are adjusted downwards.
• Because the CWA methodology allows for the pass through of direct costs, suppliers are not as exposed as they would be to unexpected cost increases. This can also be advantageous from a client perspective in that in a heated market, fixed price contracts are likely to seek to guard against unexpected price increases by quoting higher prices to start with.
• From a practical perspective, and in the specific context of Spring Hill and Otago, the comment has been made to us by Department and contractor personnel that as the construction market grew tighter, there was less willingness on the part of the construction industry to enter into traditional fixed price contracts. The market was becoming a sellers market. Those in the market were increasingly being able to dictate their own terms and prices and this included, as noted earlier, a trend toward preferring CWA.

5.59 In contrast to CWA, traditional NZS 3910 approaches to procurement are suitable for projects where the scope and requirements of the project are well defined and where the risks are known, understood and capable of being appropriately factored into tender prices. In our view, these characteristics are present across elements of the prisons construction programme and, accordingly, we do not consider that traditional forms of procurement should be automatically ruled out. Each project should be considered in its own right and the business case made for preferring one form of procurement over others.

5.60 On a related point, we consider that there are lessons arising from the Spring Hill and Otago projects in terms of how far down the supply chain to carry the CWA approach. As noted earlier, 95% of cost at Spring Hill is being incurred by CWA members. This is a very high ratio.

5.61 CWA is management intensive. For those CWA members who are new to CWA, there is also a considerable learning curve and associated training requirements. The deeper CWA penetrates through the supply chain, the greater are the administrative and management overheads and at some point, the additional costs involved exceed the benefits that CWA might bring. In the case of Otago, we note that there are 17 CWA members. This is a large number of firms to manage. Where CWA is implemented, it makes sense to include the primary contractors as CWA members. However, application of CWA becomes increasingly questionable the further below this level the CWA seeks to probe. In our view, there may not be sufficient benefits over cost to drive down as far as the small and even medium sized businesses that supply trade services to the project.

Selection Process

5.62 Although not part of the original brief for this review, the Commission has requested that comment be made on the process for the selection and appointment of the main contractors involved with Spring Hill and Otago and the appointment of the Project Director and CMS (reflecting the key role that CMS has played in facilitating the CWA).

5.63 In the case of Spring Hill, a Request for Proposal (RFP) for provider of building works was issued in September 2004 and Mainzeal was the preferred respondent. The Department and Mainzeal entered into a CWA Negotiation Framework Agreement (NFA) on 26 October 2004. Also in September 2004, the Department issued a RFP for building works in respect of Otago and Hawkins Construction was selected. A NFA with Hawkins was entered into also on 26 October 2004. The evaluation of the tenders was undertaken with particular focus on performance, culture and capability. Cost was not part of the evaluation criteria.

5.64 The Department has documented procurement processes and tender procedures. The latest version of these is based on an original document prepared in 2002 and which, in turn, is based on the framework contained in the Statement of Good Practice issued by the Office of the Auditor General (that framework remains at the core of the current processes).

5.65 A probity audit of the selection processes for the construction contractors for Spring Hill and Otago was undertaken by Audit NZ in October 2004. Those reports concluded that:

• generally, the RFP documentation and the processes surrounding its issue, the receipt, opening and evaluation of responses conform to good practice;
• the evaluations were conducted fairly and impartially and in accordance with the documented process plan; and
• Audit New Zealand was unaware of any probity issues outstanding.

5.66 We also note that Sherwin, Chan & Walshe were engaged to undertake a financial due diligence of the main contractors.

5.67 Notwithstanding the positive outcome of the review undertaken by Audit New Zealand, we note that there is an issue as to whether or not the evaluation criteria should include cost. We are aware that some CWA tenders do not include cost (as was the case for the RFPs issued by the Department), but equally it is possible to include cost as part of the evaluation process. However, in order to do this, a number of steps have to be completed. In particular, the client has to have undertaken some work to take design to a stage that can be used for costing purposes.

5.68 As part of the tender process, the client can invite contractors to submit bids based on a reimbursable target-cost contracting basis. The benefits to the client are that contractors submit indicative target costs in a competitive environment. It is important to note, however, that where the scope of work is imprecisely defined, or there are particular features of the operating environment that create significant uncertainty, then assumptions need to be developed by the client regarding scope, specification and schedule. This enables bidders to submit indicative target costs based on shared and common understandings. We are unaware as to whether or not the Department considered approaching the tender on this basis. We note, however, that by late September 2004 when the RFP's were issued, the construction market was already heated. Based on discussions with industry and Department personnel, there is a view that the market would not have reacted favourably to such a tender approach given the amount of additional work involved in preparing a bid. Moreover, the time available to the Department to issue the RFP and then make a selection was very tight (a point noted by Audit New Zealand in its probity reviews). We doubt that the timeframes available would have allowed for indicative costs to be added into the tender process.

5.69 The re-appointment of the Project Director position occurred in 2004. Prior to the re-appointment, the Project Director had been fulfilling project management and other roles within the Department and, accordingly, had acquired considerable institutional knowledge. A tender process for the Project Director position was initiated and managed by the then Chief Financial Officer. The CFO, together with two members of the Steering Group, formed the interview panel. We understand that four or five possible candidates were considered and this was refined to a short-list of two.

5.70 In July 2004, Audit New Zealand provided some informal advice on an early draft of the Request for Proposal for the Project Director position. The appointment process had commenced in May 2004 and the appointment was made in October 2004. The review by Audit New Zealand raised a concern that the Project Director and the Manager, Administration and Contracting were involved in the renewal process for each other's contracts. The Manager, Administration and Contracting had been involved in preparing the RFP for the Project Director position and the evaluation plan. We understand from Audit New Zealand that there were also some minor issues around documentation. We agree with the comment by Audit New Zealand that the engagement of key personnel for RPDP should be managed by the Department alone as far as possible, or by using a completely independent recruitment specialist.

Contracting Expertise

5.71 At the time of entering into CWA for the construction of the regional prisons, the Department had very little knowledge and understanding of CWA. It was heavily dependent on a small number of key individuals including, in particular, Stewart Rix of CMS and John Hamilton (the Project Director). Neither are employees of the Department. As a general point, it is preferable, particularly on projects of the size, complexity and risk (financial, operational and reputation) of the RPDP projects, to avoid being too reliant on a small number of external advisers. That said, however, it is acknowledged that expertise, particularly in relation to the CWA methodology, was relatively scarce (in part, reflecting its limited use in New Zealand).

5.72 Through the projects completed to date (and nearing completion), the Department has built significant organisational capability within the RPDP team. For future projects, the Department needs to ensure that it retains people who are able to advise, from a first principles perspective, on the choice of optimal procurement methodology and then, once the decision is taken, lead and advise on the implementation of the chosen procurement method. This includes the role of being a member on the Principal's Group.

5.73 A blend of competencies and experiences are required to undertake these tasks; commercial, construction as well as understanding of Government machinery and decision making processes. The range of competencies is unlikely to be found in one individual. We note, and support, the decision by the Department to add a department employee to the Principal's Group at both Spring Hill and Otago. This initiative usefully supplemented the commercial and construction background and expertise of the Project Director notwithstanding his previous involvement with the Department since 1997 (as employee and as contractor).

5.74 We note that the issue of having the right mix and level of expertise might apply more widely than just the Department of Corrections. For example, a unit has been established within the State Services Commission to assist with and advise on aspects of major IT investments. Although well beyond the scope of this review, it is worth considering whether there is a need for an equivalent group in respect of major building construction or even procurement more generally for the public sector. Although not appropriate for New Zealand given the much smaller size of our economy, we note the example from the United Kingdom which has established the Office of Government Commerce (OGC) which is an independent Office of the Treasury. The OGC is responsible for a wide ranging programme which focuses on improving the efficiency and effectiveness of procurement within central government.

Market Testing of Costs

5.75 The Department and Government more generally, needs assurance that the amounts being paid for the regional prisons are based on efficient levels of cost. The way in which CWA has been implemented by the Department means that such assurance is sought through a process of subjecting the development of the TOC to extensive independent review and audit. Under traditional NZS 3910 contracts, prices are market tested through the competitive tendering process. As applied to Spring Hill and Otago, the CWA methodology has not provided this same market testing. Accordingly, under CWA, there needs to be exceptionally robust and transparent processes around determination of TOC/QRA and the inputs to these - direct costs, margins, off-site and on-site overheads.

5.76 Notwithstanding these processes, however, we consider that the option of providing further assurance by requesting potential CWA members to submit bids based on reimbursable target-cost contracts should be considered for future projects. Such a process would involve submitting indicative target costs in a competitive environment before the selection of CWA members is made (but would also require more time and resource to implement than was available in the context of Spring Hill and Otago).

5.77 In the context of the regional prisons completed, or under construction, we doubt that there was the time to undertake this process. However, for future prison builds, we see no reason why consideration should not at least be given to modifying the tendering process in this manner. The key advantages are that this approach comes closer to providing a market test of likely cost, creates the opportunity to consider options if indicative costs exceed available funding and provides an improved ability to give Ministers a realistic view of potential costs.

Summary

5.78 A key feature of the methodology and processes employed in relation to Spring Hill and Otago is the adoption of the CWA approach to procurement. A hybrid form of this methodology was successfully utilised on the Northland project. In that particular case, use of CWA assisted in overcoming some issues that were threatening to disrupt progress with the facility.

5.79 The CWA methodology has many potential advantages compared to traditional fixed price forms of contracting. The Department has captured many of these including:

• close integration of design, construction and operational perspectives;
• cost savings and other benefits through undertaking several value management reviews;
• efficient use of resources at site-level;
• collaborative approaches to resolving issues as they arise at a day-to-day level; and
• high levels of transparency around project costs at the project level (although as discussed in the next section, this transparency did not always percolate up to the Steering Group to the extent needed).

5.80 Overarching these benefits, there are sufficient indications to suggest that had the Department elected to follow a more traditional procurement method, timeframes for completing the projects may not have been met. Had that been the case, this review would have had a fundamentally different focus and set of drivers.

5.81 Notwithstanding the benefits that have arisen through the use of CWA, there are some lessons that can be learned in terms of design, costing, procurement and scheduling. Specifically:

• CWA is not necessarily suited to all projects. There is a need to identify the conditions under which CWA is likely to be preferred ahead of other forms of procurement. These considerations need to applied on a project-by-project basis and decisions documented.
• The CWA methodology requires careful management and sound contracting experience. The Department entered the RPDP without recent major construction project experience. It is building that capability. This needs to continue if further new prisons are proposed. At a wider level, consideration may need to be given as to whether the public sector generally needs to build its contracting expertise for major projects (although this would depend on there being a sufficiently large scope of future projects).
• CWA, along with any form of procurement, needs to be based on sound facility designs. In turn, this requires agreed standards and adequate design capacity.
• As part of the CWA member selection process, it may be possible in some circumstances to seek indicative target costs. This would help to provide a degree of market testing of costs. However, to do this requires that the following conditions be met which, in the situation of Spring Hill and Otago, was not the case:
• • there be sufficient contractors prepared to engage in this process;
  • designs be developed to a stage sufficient to support estimation of costs
  • the Department have the resources to support the process; and
  • sufficient time be available for the process to work through.
• The development of the TOC is a fundamental part of the CWA methodology and needs to be completed sooner than has been the case in the Spring Hill and Otago projects. This emphasises further the need for timely completion of designs and appropriate levels of project management resource.
• Until the TOC is underway, cost estimates are indicative only. This needs to be made clear to decision makers. Once TOC estimates begin to emerge, they should be reported regularly so as to provide maximum opportunity to modify requirements if budgets are under pressure.

³ NFAs are interim arrangement pending formalisation of the CWA Agreement. They do not represent the award of any contract, but signal the intention to enter into a CWA Agreement. NFAs set out various matters governing the negotiations leading toward a CWA Agreement.

⁴ RPDP Steering Group meeting of 30 March 2004

⁵ This form of procurement is increasingly common in Australia under the banner of pubic-private partnerships and in the United Kingdom under the banner of private finance initiatives (PFI). In New Zealand, such forms of contract would require, under the Public Finance Act, approval from the Minister of Finance.

⁶ National Audit Office (2005) "Improving Public Services Through Better Construction" , London, p23.

⁷ Peter Waterhouse 31 October 2005 "Report to RPDP on Construction Costs and Alternative Procurement Strategies"