{"id":8604,"date":"2013-08-23T06:00:25","date_gmt":"2013-08-22T20:00:25","guid":{"rendered":"http:\/\/www.aspistrategist.ru\/?p=8604"},"modified":"2013-08-26T09:27:49","modified_gmt":"2013-08-25T23:27:49","slug":"pick-a-number-now-add-five-and-double-it","status":"publish","type":"post","link":"https:\/\/www.aspistrategist.ru\/pick-a-number-now-add-five-and-double-it\/","title":{"rendered":"Pick a number. Now add five and double it\u2026"},"content":{"rendered":"

\"Space<\/a>In one of my earlier pieces<\/a> I mentioned the cost growth experienced by the Apollo program. It\u2019s worth digging a little deeper into that story. In a wonderful paper on cost estimation (yes, I need to get out more) for a NASA symposium*, this story is told:<\/p>\n

\n

Apollo is often painted as a successful program that came close to its initial 1961 [schedule] estimate for a 1967 moon landing. [The] 1958 Air Force estimate\u2026 was 1.5 billion with completion targeted in 1965. The \u2018actual\u2019 historical events went something like this. The NASA cost estimating gurus in 1961 projected an amount close to $7 Billion to do the entire program. This figure was apparently padded to $10-$12 Billion by management prior to giving that estimate to James Webb, the NASA Administrator. Mr. Webb (within hours of receiving the $10-$12 Billion figure) placed an \u2018administrator’s discount\u2019 on NASA\u2019s ability to predict costs with due precision and by the stroke of his own pen, changed the estimate to $20 billion and submitted it to Vice President Linden B. Johnson. In the words of Robert Seamans Jr., (the Associate Administrator at the time) \u2018We were aghast\u2019! This cavalier beginning describes how Apollo’s original fiscal requirements arrived at the steps of the Capitol and was subsequently blessed by Congress.<\/p>\n<\/blockquote>\n

Imagine trying to justify that process to Senate Estimates! It would be a brave official to take the line \u2018Yes, senator, the best efforts of the cost estimation cell was $7 billion, but we arrived at the final figure by padding it a bit and then doubling the result, on the basis that we\u2019ve never got it right in the past, so why should this be any different?’ Yet it turns out that Mr. Webb\u2019s intuition based \u2018pluck\u2019 was closer to the correct answer\u2014$25 billion\u2014than the results from cost estimation models. (Lest this sound \u2018cheap\u2019 compared to modern aerospace endeavours, the program cost $200 billion in today\u2019s dollars.)<\/p>\n

As my previous posts showed, this wasn\u2019t the first or last time that project cost estimates turned out to be hopelessly optimistic. And the same goes for schedules. Augustine\u2019s 23rd Law<\/a>, based on the average performance of a large sample of major American defence projects, describes it thusly: \u2018Any task can be completed in only one-third more time than is currently estimated\u2019. By that yardstick, the Apollo project was just about average; by delivering in 1969 rather than 1967, it clocked a schedule overrun of 40%.<\/p>\n

We\u2019re not yet finished with the litany of woe that can be extracted from NASA\u2019s historical project data. As well as cost and schedule, there\u2019s also the capability that\u2019s delivered. The Apollo program did what it set out to do, but not without some white knuckle moments and with less margin for error than many would\u2019ve liked. But NASA\u2019s poster boy for underdelivery of capability was undoubtedly the space shuttle. By now in this series we\u2019re used to the schedule and cost overruns (100% and around 200% respectively), but the number of flights delivered over the lifetime of the program was only 1\/6th of the original plan.<\/p>\n

It\u2019s tempting to say that these examples from the cutting edge of technology don\u2019t tell us anything about more mundane deliverables. But a look at the results across the portfolio of projects set in train by the 2000 defence white paper<\/a> suggests otherwise. And if we look at the Collins submarine program, we find\u2014mirabile dictu\u2014schedule slippages, cost overruns and shortfalls in capability, some of which are still being rectified. Similarly, the F-35 Joint Strike Fighter was meant to be in RAAF service by now, but will finally be delivered towards the end of the decade (80% schedule overrun) at a higher price (cost overrun TBD) and the block three aircraft we get will have a weapons suite considerably leaner than was originally the plan. And while the Wedgetail airborne early warning and control aircraft is now in service (six years later than planned), the ANAO\u2019s reporting of DMO data<\/a> shows that there\u2019s a chunk of capability still undelivered.<\/p>\n

Of course, sometimes projects deliver on time, on budget and to specification, but they\u2019re not usually the challenging developmental projects\u2014those end up in Augustine\u2019s notebook, on the Projects of Concern list<\/a> and sometimes on the front page of the newspapers<\/a>. And, as my 2000 white paper analysis showed, there\u2019s an opportunity cost to the budget and to ADF capability more broadly. Engineers are exploring techniques like ‘reference class comparison<\/a>‘ for using historical data to get a handle on systematic underestimation, but it remains an imperfect science at best.<\/p>\n

All of this suggests an approach for future defence planners and governments when they\u2019re putting together white papers and defence capability plans. When the really big ticket development items come up for discussion, a sensible and credible question to ask is:<\/p>\n

If this was to cost twice the number we have before us, be delivered in twice the time estimated and with only 75% of the capability, would we still think it\u2019s a good idea?<\/em><\/p><\/blockquote>\n

There\u2019ll be those who scoff at this notion for a rational planning framework, but it has history on its side. Hugh White understands this principle<\/a> and still wants a late, more costly and less capable F-35 on the grounds that it\u2019ll be better than the alternatives. That\u2019s fine, and it passes the test I propose above. But I wonder if there\u2019ll be any discussion along these lines when decisions are being made about the future submarine? If there isn\u2019t, then we\u2019re not doing it right.<\/p>\n

* Alas, the paper, NASA’s joint cost schedule paradox<\/em> seems to be currently unavailable for download. (It nominally lives here<\/a>.)<\/p>\n

Andrew Davies is a senior analyst for defence capability at ASPI and executive editor of\u00a0 <\/em>The Strategist. Image courtesy of NASA<\/a>.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

In one of my earlier pieces I mentioned the cost growth experienced by the Apollo program. It\u2019s worth digging a little deeper into that story. In a wonderful paper on cost estimation (yes, I need …<\/p>\n","protected":false},"author":6,"featured_media":8607,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"footnotes":""},"categories":[19],"tags":[537],"class_list":["post-8604","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-policy-guns-and-money","tag-project-timelines"],"acf":[],"yoast_head":"\nPick a number. Now add five and double it\u2026 | The Strategist<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.aspistrategist.ru\/pick-a-number-now-add-five-and-double-it\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Pick a number. Now add five and double it\u2026 | The Strategist\" \/>\n<meta property=\"og:description\" content=\"In one of my earlier pieces I mentioned the cost growth experienced by the Apollo program. It\u2019s worth digging a little deeper into that story. In a wonderful paper on cost estimation (yes, I need ...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.aspistrategist.ru\/pick-a-number-now-add-five-and-double-it\/\" \/>\n<meta property=\"og:site_name\" content=\"The Strategist\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/ASPI.org\" \/>\n<meta property=\"article:published_time\" content=\"2013-08-22T20:00:25+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2013-08-25T23:27:49+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.aspistrategist.ru\/wp-content\/uploads\/2013\/08\/space-shuttle1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"768\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Andrew Davies\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@ASPI_org\" \/>\n<meta name=\"twitter:site\" content=\"@ASPI_org\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Andrew Davies\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.aspistrategist.ru\/#website\",\"url\":\"https:\/\/www.aspistrategist.ru\/\",\"name\":\"The Strategist\",\"description\":\"ASPI's analysis and commentary site\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.aspistrategist.ru\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-AU\"},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-AU\",\"@id\":\"https:\/\/www.aspistrategist.ru\/pick-a-number-now-add-five-and-double-it\/#primaryimage\",\"url\":\"https:\/\/www.aspistrategist.ru\/wp-content\/uploads\/2013\/08\/space-shuttle1.jpg\",\"contentUrl\":\"https:\/\/www.aspistrategist.ru\/wp-content\/uploads\/2013\/08\/space-shuttle1.jpg\",\"width\":1024,\"height\":768,\"caption\":\"space-shuttle1\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.aspistrategist.ru\/pick-a-number-now-add-five-and-double-it\/\",\"url\":\"https:\/\/www.aspistrategist.ru\/pick-a-number-now-add-five-and-double-it\/\",\"name\":\"Pick a number. Now add five and double it\u2026 | The Strategist\",\"isPartOf\":{\"@id\":\"https:\/\/www.aspistrategist.ru\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.aspistrategist.ru\/pick-a-number-now-add-five-and-double-it\/#primaryimage\"},\"datePublished\":\"2013-08-22T20:00:25+00:00\",\"dateModified\":\"2013-08-25T23:27:49+00:00\",\"author\":{\"@id\":\"https:\/\/www.aspistrategist.ru\/#\/schema\/person\/08a9125f7af3039520d264e965235a73\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.aspistrategist.ru\/pick-a-number-now-add-five-and-double-it\/#breadcrumb\"},\"inLanguage\":\"en-AU\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.aspistrategist.ru\/pick-a-number-now-add-five-and-double-it\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.aspistrategist.ru\/pick-a-number-now-add-five-and-double-it\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.aspistrategist.ru\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Pick a number. Now add five and double it\u2026\"}]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.aspistrategist.ru\/#\/schema\/person\/08a9125f7af3039520d264e965235a73\",\"name\":\"Andrew Davies\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-AU\",\"@id\":\"https:\/\/www.aspistrategist.ru\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/bbb47ebb41d4978346dbf2e1d21b992a?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/bbb47ebb41d4978346dbf2e1d21b992a?s=96&d=mm&r=g\",\"caption\":\"Andrew Davies\"},\"url\":\"https:\/\/www.aspistrategist.ru\/author\/andrew-davies\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Pick a number. Now add five and double it\u2026 | The Strategist","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.aspistrategist.ru\/pick-a-number-now-add-five-and-double-it\/","og_locale":"en_US","og_type":"article","og_title":"Pick a number. Now add five and double it\u2026 | The Strategist","og_description":"In one of my earlier pieces I mentioned the cost growth experienced by the Apollo program. It\u2019s worth digging a little deeper into that story. In a wonderful paper on cost estimation (yes, I need ...","og_url":"https:\/\/www.aspistrategist.ru\/pick-a-number-now-add-five-and-double-it\/","og_site_name":"The Strategist","article_publisher":"https:\/\/www.facebook.com\/ASPI.org","article_published_time":"2013-08-22T20:00:25+00:00","article_modified_time":"2013-08-25T23:27:49+00:00","og_image":[{"width":1024,"height":768,"url":"https:\/\/www.aspistrategist.ru\/wp-content\/uploads\/2013\/08\/space-shuttle1.jpg","type":"image\/jpeg"}],"author":"Andrew Davies","twitter_card":"summary_large_image","twitter_creator":"@ASPI_org","twitter_site":"@ASPI_org","twitter_misc":{"Written by":"Andrew Davies","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebSite","@id":"https:\/\/www.aspistrategist.ru\/#website","url":"https:\/\/www.aspistrategist.ru\/","name":"The Strategist","description":"ASPI's analysis and commentary site","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.aspistrategist.ru\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-AU"},{"@type":"ImageObject","inLanguage":"en-AU","@id":"https:\/\/www.aspistrategist.ru\/pick-a-number-now-add-five-and-double-it\/#primaryimage","url":"https:\/\/www.aspistrategist.ru\/wp-content\/uploads\/2013\/08\/space-shuttle1.jpg","contentUrl":"https:\/\/www.aspistrategist.ru\/wp-content\/uploads\/2013\/08\/space-shuttle1.jpg","width":1024,"height":768,"caption":"space-shuttle1"},{"@type":"WebPage","@id":"https:\/\/www.aspistrategist.ru\/pick-a-number-now-add-five-and-double-it\/","url":"https:\/\/www.aspistrategist.ru\/pick-a-number-now-add-five-and-double-it\/","name":"Pick a number. Now add five and double it\u2026 | The Strategist","isPartOf":{"@id":"https:\/\/www.aspistrategist.ru\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.aspistrategist.ru\/pick-a-number-now-add-five-and-double-it\/#primaryimage"},"datePublished":"2013-08-22T20:00:25+00:00","dateModified":"2013-08-25T23:27:49+00:00","author":{"@id":"https:\/\/www.aspistrategist.ru\/#\/schema\/person\/08a9125f7af3039520d264e965235a73"},"breadcrumb":{"@id":"https:\/\/www.aspistrategist.ru\/pick-a-number-now-add-five-and-double-it\/#breadcrumb"},"inLanguage":"en-AU","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.aspistrategist.ru\/pick-a-number-now-add-five-and-double-it\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.aspistrategist.ru\/pick-a-number-now-add-five-and-double-it\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.aspistrategist.ru\/"},{"@type":"ListItem","position":2,"name":"Pick a number. Now add five and double it\u2026"}]},{"@type":"Person","@id":"https:\/\/www.aspistrategist.ru\/#\/schema\/person\/08a9125f7af3039520d264e965235a73","name":"Andrew Davies","image":{"@type":"ImageObject","inLanguage":"en-AU","@id":"https:\/\/www.aspistrategist.ru\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/bbb47ebb41d4978346dbf2e1d21b992a?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/bbb47ebb41d4978346dbf2e1d21b992a?s=96&d=mm&r=g","caption":"Andrew Davies"},"url":"https:\/\/www.aspistrategist.ru\/author\/andrew-davies\/"}]}},"_links":{"self":[{"href":"https:\/\/www.aspistrategist.ru\/wp-json\/wp\/v2\/posts\/8604"}],"collection":[{"href":"https:\/\/www.aspistrategist.ru\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.aspistrategist.ru\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.aspistrategist.ru\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/www.aspistrategist.ru\/wp-json\/wp\/v2\/comments?post=8604"}],"version-history":[{"count":15,"href":"https:\/\/www.aspistrategist.ru\/wp-json\/wp\/v2\/posts\/8604\/revisions"}],"predecessor-version":[{"id":8606,"href":"https:\/\/www.aspistrategist.ru\/wp-json\/wp\/v2\/posts\/8604\/revisions\/8606"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.aspistrategist.ru\/wp-json\/wp\/v2\/media\/8607"}],"wp:attachment":[{"href":"https:\/\/www.aspistrategist.ru\/wp-json\/wp\/v2\/media?parent=8604"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.aspistrategist.ru\/wp-json\/wp\/v2\/categories?post=8604"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.aspistrategist.ru\/wp-json\/wp\/v2\/tags?post=8604"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}