SPACE
 		This document should be compatible with the 9 Levels.   If it isn't, you're probably tryng to cheat.
SEVEN STEPS IN SYSTEM PROCUREMENT
 
  1. PROTOTYPE
  Emphasis:   Is it POSSIBLE this will work?
    Examine blue-sky ideas
Examine numerous (up to 10 - 20) potential designs
Use rapid prototyping of simulations
Use rapid prototyping of hardware

Use order-of-magnitude calculations
Emphasis on calculation and simulation rather than hardware
Must still do REAL engineering, not viewgraph engineering

  No concern (at this stage) for  
    Precise answers
  Detailed design
  Integration
  Operational concepts
 
  Manufacturability
  Maintainability
  Training requirements
  System cost 		 
 
  2. NARROW DOWN DESIGNS
  Emphasis:   Is it LIKELY this will work?
    Examine designs from Step One
Estimate performance (via simulation OR hardware) from Step One
Estimate how parts of system will be integrated

Estimate operation of final system
Estimate manufacturability of final system
Estimate maintainability of final system

Estimate training needs of final system
Estimate cost of final system
Choose a handful (5-10) of designs for further analysis 		 
TOP If you lack the detail to narrow down candidates, you must REDO Step One  
 
  3. EVALUATE DESIGNS  
  Emphasis on hardware, detailed calculation, and high-fidelity simulation
    Perform detailed evaluation of designs that pass Step Two
Simulations should be SEI CMM Level 3 or better
Do laboratory tests if feasible
Do flight tests if feasible 		 
  Few lab or flight tests means strong emphasis on high-fidelity simulations.   If you do lab or flight tests, also use them to improve your simulation(s).   If you cannot do flight tests and lack high-fidelity simulations, STOP.  
 
  4. CHOOSE CANDIDATE DESIGNS  
  Emphasis:   Is it VIRTUALLY CERTAIN this will work?
    Examine designs from Step Three
Estimate performance (via simulation or hardware) from Step Three
Estimate how parts of system will be integrated
Estimate operation of final system

Estimate manufacturability of final system
Estimate maintainability of final system
Estimate training needs of final system
Estimate cost of final system 		 
  Make each candidate system fail (in lab or via simulation)
TOP Determine why it failed, how it failed, and how to prevent it
Choose a few (2-5) designs for further analysis
If you lack the detail to narrow down candidates, you must REDO Step Three
 
  5. FIELD TEST CANDIDATE DESIGNS  
  Emphasis:   Is it VIRTUALLY CERTAIN this will work?
    Same as Step Four, except you fill in holes that simulation can't
You must BUILD something
Calculation and simulation is not a substitute for this step
You must test it the SAME WAY it will be used
Believe Murphy's Law — stress-test the system 		 
    Make each candidate system fail in simulaton or hardware
      Determine why it failed, how it failed, and how to prevent it
ASSUME you will have to make some design changes
 
  6. CHOOSE FINAL DESIGN  
    Examine designs from Step Five
Software for any life-critical final system should be SEI CMM Level 5
Calculate performance
Calculate probability of failure

Determine how parts of system will be integrated
Determine operations concepts of final system
Determine manufacturability of final system

Determine maintainability of final system
Determine training needs of final system
Determine cost of final system 		 
    If you lack the detail to choose a design, you must REDO Step Five  
 
  7. FINAL DESIGN  
    Build it
Deploy it
Train people in its use
Operate it
Maintain it 		 
  TOP
 
POINTS TO REMEMBER
    1. Cutting corners on simulation is false economy.   It is usually FAR cheaper to simulate a system than to build, operate, and maintain it.  
    2. Cutting corners on testing is false economy.   It is usually FAR cheaper to test a system than to make it operational.  
    3. The cost of simulating and testing a system is usually FAR cheaper than the cost of a failed system.  
    4. Simulations can be used not just for testing, but also for training.  
    5. You can probably afford to do all of steps one through four NUMEROUS times for less than the cost of doing step five ONCE.   See Point One.  
    6. Building a system is only the start.   Costs and details of operations, maintenance, and training are significant.  
    7. Building a single unit is vastly different from building a hundred, or a thousand.   Mass-producing a technology may be a greater challenge than creating the technology in the first place.  
    8. Building a single functional unit is vastly different from building a set of reliable units.   Making a technology that is reliable, without a lot of maintenance, may be a greater challenge than creating the technology in the first place.  
    9. You can ignore cost, maintenance, training, and operations ONLY in step One.   You CAN'T ignore them after that.  
  10. During each step (after step One) evaluate each factor (cost, maintenance, training, operations, technical feasibility) to the same level of detail.  
  11. Most systems that fail do so because steps were either skipped, or glossed over (usually steps two through six!).  
  12. If you can't afford a system, don't pretend you can afford it anyway.   Instead, decide what you CAN afford.  
  TOP SPACE 9 Levels