2002 AACE International Transactions

CSC.06

Best Owner Practices for Project Control

John K. Hollmann, PE CCE
or the past several decades, owner companies have been reorganizing, reengineering, downsizing, and, at times, eliminating their in-house cost engineering function in a drive to improve cost competitiveness by reducing what company management perceives as overhead. Capital program managershoping that these actions would improve cost outcomeshave largely been disappointed. In this paper, a better approach than cutting and blind outsourcing is described; an approach that uses best owner cost engineering practices to ensure competitive outcomes. The approach has been demonstrated to work through quantitative research of empirical industry data. Independent Project Analysis (IPA) research, as reported in industry literature in 1994, had shown that average cost growth (the ratio of actual/estimated costs) for projects was declining. However, more recent research has shown that since 1994, neither average cost growth nor variability (the plus or minus range) has improved [4, 2]. Past improvements in average cost growth were largely driven by improved up-front project definition prior to full authorization of project funds (i.e., front-end loading or FEL). However, good FEL can improve the cost outcome only so much before weaknesses in other practices become a constraint. Owner cost engineering/project control practices used during the FEL and execution phases of a project are one such constraint and an opportunity for improvement.

BENCHMARKING IDENTIFIES TRENDS AND BEST PRACTICES FOR COST ENGINEERING/ PROJECT CONTROL Project system benchmarking is used to identify industry trends and best practices, including those for cost engineering. The benchmarking process compares a companys project system performance with that of its competitors. To support benchmarking, data about actual project practices (inputs) and results (outcomes) are collected. Using the empirical data in an industry database, we perform statistical analysis to identify those inputs that drive the best outcomes. We use an empirical industry database to research cost engineering practices and determine which are best at improving cost and schedule outcomes. Many of our research findings about cost engineering are common sense, but it is too easy for management to reject common sense when other cost reduction options (often counter-productive) look so much more attractive on paper. The

practices discussed in this article have been empirically demonstrated to work. Before describing which practices are best, we need to explain the scope of the cost engineering practices studied by IPA and the typical practices used in Industry. The practices studied include cost estimating, planning and scheduling, and cost/schedule control; these practices collectively are often referred to as project control, which is a core subset of the cost engineering field. Figure 1 illustrates the primary steps in the project control process as documented by AACE Internationals Total Cost Management Framework [3]. Starting at the top-center of the process map in figure 1, the cost engineer estimates the project cost and establishes the cost budget for the project. The cost engineer uses the planning and scheduling step to translate the projects scope into a logical sequence of work activities. The project activities are then executed while the cost engineer measures and assesses the projects performance against the plan (i.e., cost/schedule control). On the left side of the process map, the cost engineer identifies corrective actions when deviations from the plans occur. Finally, the control cycle loop is closed when the cost engineer updates project plans to address project trends and changes. Through benchmarking, we have identified typical industry project control practices, both good and bad. Typically, engineering and construction contractors perform most day-to-day project control functions during project execution. Owners generally set project control requirements, evaluate contractor estimates and proposals, review contractor progress reports, and negotiate contract changes. If the project scope is well defined at the time of project funds authorization (i.e., during FEL), then an engineering contractor prepares the cost estimate and critical-path schedule to support both authorization and control. However, in many projects, definition is lacking in some way and owners will prepare a conceptual or preliminary estimate and milestone schedule for funds authorization, and an engineering contractor then prepares a detailed estimate and schedule later on (often too late) that will serve as a basis for control. For small projects (i.e., <$5 million), all of the owners project control tasks are typically performed on a part-time basis by a project manager or lead engineer from the plant who has experience managing projects. As projects increase in size, an owner project control specialist, who is often based at a central office, may be assigned to the project. However, many owners have downsized their centralized functions; without some way of shar-

CSC.06.1

2002 AACE International Transactions ing resources (and without empirical evidence of their value), plants and smaller business units find it difficult to justify keeping project control specialists in-house. As owners downsize their project control capabilities, there is a corresponding increase in the use of fixed-price contracting or alliances with contractors to reduce the owners project control responsibilities (and presumably risk). As a result, owners typically have very limited estimating and project control experience and capability in-house. Basic owner responsibilities such as setting contractor requirements for project control are given minimal attention. This shrinkage in owner project control capability has not led to improved capital competitiveness; owners cannot divorce themselves from core project control responsibilities and practices and expect to get improved project outcomes. One of the most important findings of IPAs research is that the outcome of fixed-price projects is improved by good owner project control practices. IRR = [(present value of revenues) - (present value of costs)] /(value of capital investment). (equation 1) Project cost is a critical driver of IRR; it comprises most of the denominator in the IRR equation. The project schedule affects the IRR equation by determining when the revenue stream begins: the longer the schedule, the more heavily discounted the revenue and the greater the chance that the market window of opportunity will close. Figure 1 illustrates how changes in project cost and schedule affect the IRR of a typical commodity project in the CPI. As shown in figure 2, for each 10% increase in capital project cost, there is about a 2% decrease in IRR. Controlling capital cost is therefore vital to achieving competitive rates of return. Schedule control is more critical for specialty markets that are sensitive to market windows and higher margins for those who get to market fastest. For commodities, it is not a good practice to try to time price swings or cycles. As we stated earlier, FEL is the key driver of improved project cost. FEL is the process by which projects are selected, defined, and prepared for execution (detailed design and construction). FEL is important because all other excellent practices require good FEL to be practicable and effective. To rate the quality of FEL, we use a numeric scale to evaluate the completeness and quality of front-end work in alignment and clarity in business objectives, site definition, project execution planning, and engineering definition. To make the numeric scale more intuitive, we group the scale into five categories ranging from best to screening. Figure 3 illustrates the FEL categories and shows how FEL drives project cost outcomes. Figure 4 illustrates that the average

COST AND SCHEDULE ARE THE OBJECTIVES (FEL DRIVES, PROJECT CONTROL STEERS) Cost and schedule are key outcomes that generally determine whether a capital project meets its business objectives. Financial objectives are often expressed by measures such as internal rate of return (IRR). IRR and similar measures are based on the evaluation of the life cycle cost and revenue streams in a way that considers the time value of money. In simplistic terms, IRR can be represented by the following equation:

Strategic Asset Management

Strategic Asset Scope Description, Project System Requirements, & Budget

Project Requirements Assessment

Asset, Project History & Learnings

Planning & Scheduling

Schedule Baseline (Activities)

Scope Development Forecasting, Corrective Action, & Change Management

Analysis Basis & Feedback Improvement Opportunities (variance from baseline plans)

WBS

Cost Estimating & Budgeting

Cost Baseline (Budget)

Communciate, Execute Plans

Project Accounting

Resource Planning
Iterative, concurrent processes Analysis Basis & Feedback

Resource Baseline (Quantities)

Performance Measurement

Project Performance Assessment

Value Engineering

Risk Analysis & Contingency

Baseline Plans

Expenditure and Progress Measures

Figure 1Project Control Is a Success CSC.06.2

2002 AACE International Transactions

7% 5% 3%

Cost Schedule
-10% Change in IRR -20%

Change in Performance Metric 20% 10%

1% -1% -3% -5% -7%

Figure 2Cost Drives IRR for Commodity Projects

20%

Percentage Above or Below Industry Average Cost

15% 10% 5% 0% -5% -10%

3 .5

BEST 4 4 .5

GOOD

FAIR

POOR

SCREENING

5 .5

6 .5

7 .5

8 .5

FEL Index

Figure 3FEL Drives Down Project Cost quality of project definition has improved-from the low end of the fair category to the good categorysince 1990. Although FEL is a powerful driver of cost outcomes, benchmarking has shown that not all outcomes have been improving with FEL. One disappointment has been the lack of improvement in the reliability of cost estimates. Figure 5 shows that estimate reliability has remained the same since 1994 with a standard deviation of plus or minus 12% (i.e., approximately 66% of project authorization estimates have reliability or accuracy within that range). Industry estimates typically quote an accuracy range that covers closer to two standard deviations (i.e., about 95% of observations). Using that criterion, the IPA statistics indicate that industry authorization estimates are accurate only to about plus or minus 20%not the 5 or 10% range that many consider typical. Variability in actual versus estimated cost is a cause of concern because overestimates may lead to excess capital being committed to a project (our data show that excess capital tends to be spent), underestimates may lead to the wrong project being selected because of overstated IRR, or underestimates may contribute to poor cost control because they result in unrealistic plans such that the project is not sure what its true status is. The failure of project systems to produce more reliable estimates is largely caused by a lack of improvement in industry project control practices (i.e., a lack of improvement of estimate quality during FEL, a lack of improvement in discipline and control during execution, or both). Status quo project control practices are not working well.

CSC.06.3

2002 AACE International Transactions

3 3.5

4.5

FEL Index

5.5

6.5

7.5

8 8.5

1990

SCREENING

POOR

FAIR
7

GOOD

BEST
4 5

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

Project Authorization Year

Figure 4FEL Has Been Improving

1.3
Mean +1 standard deviation

Actual / Estimate Ratio

1.2

MEDIAN Mean -1 standard deviation

1.1

1.0

0.9

0.8 1994 1995 1996 1997 1998 1999

Y ear of C onstruction C om pletion (3 year m oving average)

Figure 5Estimate Reliability (Accuracy) Has Been Stagnant

BEST PRACTICES FOR COST ENGINEERING/ PROJECT CONTROL Our research has identified a set of owner project control practices that correlate with improved cost outcomes. In this section we describe these practices and explain the degree to which they can improve your project results. The list does not represent a complete project control process or system by any means. For example, we assume that every project team will plan and control cost and schedule using a process similar to Figure 1. The practices described below are those practices that reflect real owner commitment to project control.

Have an In-House Cost Estimator Have an in-house cost estimator (or person with equivalent skills and knowledge) quantitatively validate the project cost estimate prior to authorization. IPA has found that whether the owner or contractor prepares the authorization estimate does not matter as long as the estimate is validated quantitatively by an owner cost specialist. While project managers can sometimes be good cost estimators, most become overextended trying to do all project control tasks themselves. The most common means of quantitative validation include a comparison with actual absolute cost of past projects, comparison with absolute cost of check estimates (e.g., the owner estimate), or comparison with relative cost metrics or ratios [1].

CSC.06.4

2002 AACE International Transactions The purpose of quantitative validation is not to repeat history, assign tend to have a project control process and an overall but to identify exceptions from the norm. You must be able to stronger project system. explain the exceptions. If the estimate is more aggressive (i.e., better productivity or lower cost) than past projects, specific project practices that will support the improved outcomes must be either Report Project Status and Progress Frequently and in Detail This practice is likely a proxy measure for project control in place or plannedwishful thinking is not enough. process discipline. To produce frequent, detailed reports, all pieces of effective control will likely be in place: strong FEL and project planning, a well-structured definitive estimate, physical Ensure that the Estimators are Independent of the Project Independent means free of undue bias. The estimator can progressing for all cost categories, and an assigned project control work for the owner or the engineering contractor, but the estima- specialist. A contractor can prepare the report, but the owner must tors personnel performance evaluation must not be too heavily at least establish the reporting requirements and then evaluate influenced by the project or business manager responsible for the and react appropriately to the report contents. project or asset. Managers can be biased toward their favorite projects, or they can have a predetermined cost in mind. Project costs are often underestimated because the estimator was subject to Collect Actual Cost Information and Use it to Plan and subtle (or not so subtle) coercion. Other projects are overestimat- Validate the Estimate The historical cost database is where the owner closes ed because project team members are punished for even minor loop on the project control process [1]. To quantitatively validate cost overruns. an estimate, the cost specialist must have something against which the estimate can be compared. A historical database provides Use Physical Progressing to Measure Work Progress for all Cost absolute comparison project data (estimated and actual), cost metrics, and ratios. External benchmarks should be obtained as Categories Unfortunately, too many project estimates are prepared with well. Another benefit of developing an historical database is that the primary or sole objective of obtaining the number for the practice of asking for detail cost proposals and detail final authorization with little regard for the estimates purpose and closeout data shows the contractor that the owner understands structure to support that purpose. The best practice is to develop project costs and is not likely to be fooled. In fact, our research the authorization estimate as the basis for project control for all shows that owners who quantitatively validate estimates and colcost categories; the bottom-line number is a byproduct. Even for lect detail cost data do obtain better fixed-price proposals. fixed-price projects, an owner estimate designed for control will be the best basis for evaluating contractor proposals. PUTTING THE PRACTICES TOGETHER The estimate structure needs to be able to support physical progressing. During engineering and construction, many project IPA has developed a numeric scale with which to rate project managers rely on subjective measures of progress and then fall prey to over-optimism. Even worse are project managers who use control practices on a project. To make the numeric scale more the percentage of budget hours or dollars consumed as an indica- intuitive, we group the scale into four categories ranging from tion of progress. One all too common scenario of poor progressing good to deficient. Figure 6 shows how cost growth can be is the project manager who thinks the project is 95%t complete reduced by 20% using the best project control practices (the figand discovers while developing the punchlist that only 75% of the ure shows the percentage of contingency used on a $5 to 10 milwork has been completed. The project managers immediate reac- lion project with good FEL; the percentage reduction is greater tion is to pile more people on the job to accelerate the progress, for smaller projects). Strong project control practices reduce conbut the crowding then causes a complete breakdown of discipline tingency use for both reimbursable and fixed-price contracts. and control. Another scenario is the project manager who ramps Fixed-price is a misnomer because the price is fixed only if there up construction crews too early; the crews hang around waiting are no changes; strong FEL and project control both reduce the for drawings or materials that have been promised but not deliv- amount of change. ered. STRATEGY FOR SUCCESS DURING EXECUTION: PLAN AGGRESSIVELY, THEN CONTROL Will best FEL backed by good project control practices result in the most competitive project cost outcomes? Not necessarily. Project managers who start with conservative (i.e., fat) plans at authorization and then control against that conservative base will succeed only in achieving the noncompetitive outcome that they planned for. Figure 7 shows how project control reduces contingency usage for aggressively scheduled projects to 0%, but project control does little to reduce contingency usage for conservatively scheduled projects because conservatively planned projects have

Assign an Owner Project Control Specialist to the Project During Execution Project managers have plenty to handle in leading the project without performing all the tasks of project control. Even if the heroic project manager can handle it all, the project is at risk of disruption (particularly schedule slip) if the project manager is pulled off or otherwise leaves the project (heros are in great demand). The project control specialist does not need to be assigned full time; in fact, it is common for one project control specialist to handle multiple small projects (just as the project manager might). Owners who have project control specialists to

CSC.06.5

2002 AACE International Transactions

25%

20%

Percentage Contingency Used

Reimbursable Contracts
15%

10%

5%

0%

-5 %

Fixed-Price Contracts (excludes contractors contingency)

-1 0 %

0 Deficient

2 Poor

Fair 4

Good 6

Project Control Index

Figure 6Less Contingency Is Used With Good Project Control

30%

Projects with Conservative Plans

Percentage Contingency Used

20%

10%

0%

Projects with Aggressive Plans

-1 0 %

0 Deficient

2 Poor

Fair 4

Good 6

Project Control Index

Figure 7Aggressive Projects with Strong Project Control Achieve the Best Results little trouble avoiding growth; these projects tend to be loaded with fat to cover ineffective practices. In other words, project control cannot make an ineffective project effective. It can only preserve the value that is built into a plan during FEL. In summary, lets review the practices that successful owner project systems use to be competitive. To ensure project success, the project team defines the project well during the FEL phase of the project. The plans that the team develops during FEL are aggressive (they know that the estimate is aggressive because the owners cost experts have quantitatively validated it) and the estimator structures the cost estimate at authorization to support physical progressing of all cost categories. During the execution phase of the project, the team measures their progress using physical progressing techniques with close oversight from an assigned owner project control specialist. The team reports their progress frequently in detail, and finally, they capture their actual project results in a database that they will use for future planning. As we said, this set of practices is common sense, but these practices are not used on many projects. IPA has found that 70% of small projects (i.e., < $5 million) had poor or deficient project control practices. This is unfortunate because small projects, in relative terms, benefit the most from project control because small projects tend to have huge percentage cost variations. As might be expected, large projects tend to use better project control practices, but still only 28% of large projects use good project control practices. Please note that the practices listed do not proscribe specific estimating and control tools, detail procedures, and minutia of how the practices are done day-to-day. For example, we have found some owners who achieve good results using estimating

CSC.06.6

2002 AACE International Transactions tools developed in-house while others are equally successful using commercial estimating software packages.

RESURRECTING THE OWNER PROJECT CONTROL FUNCTION If your company has been deficient in project control practices, start now on building (or rebuilding) your in-house functional knowledge and capabilities. As discussed, most day-to-day project control activity is the responsibility of project contractors, but some tasks should not be outsourced. Your company needs inhouse personnel to quantitatively validate cost estimates and bids based on cogent, up-to-date knowledge of what asset and project costs should be. In-house project control expertise is needed for establishing project control procedures and requirements for contracts. During project execution, in-house cost and project control expertise supports the project managers efforts to effectively spot and respond to project trends, address potential changes, negotiate change orders, and otherwise control the project.

uilding an in-house cost engineering/project control capability takes time and patience. Estimate at least three years to get up the learning curve. In year one, core people are identified and put in place, a project control process is established as part of your project system (along with plans to roll the process out and obtain buy-in from everyone in your companys project community), and basic tools acquired or built (e.g., software and databases). In year two, practices are piloted on selected projects and early learnings are fed back to improve the process, tools, and training. Benchmarking with peers in your industry and profession will help with the improvement process. In year three, some early successes should be noted that will help to overcome resistance from project teams and others. Your company is on its way to more competitive capital projects!

REFERENCES 1. Hollmann, John K. Project HistoryClosing the Loop. 1995 AACE International Transactions. Morgantown, WV: AACE International, 1995. Hollmann, John K., and Edward W. Merrow. Controlling Project Costs. Chemical Engineering (November 2001). Hollmann, John K., and Wesley R. Querns. The Total Cost Management (TCM) Framework. Cost Engineering (March 2001). Merrow, Edward W., and M.E. Yarrosi. Managing Capital Projects. Chemical Engineering (October 1994).

2. 3.

4.

John K. Hollmann, PE CCE Independent Project Analysis 44426 Atwater Drive, Suite 100 Ashburn, VA 20147 E-mail: jhollmann@ipaglobal.com

CSC.06.7