Why We Have a Gap
In January 2004, President Bush announced the retirement of the Space Shuttle by 2010, and the onset of the Crew Exploration Vehicle with a goal of 2011, and a threshold of 2014. A cursory glance showed that NASA had between seven and ten years to develop a new human-rated space capsule. Casual space observers wouldn't think that would be too hard. Yet the history since then has showed it to be anything but.
The historical analogies for space capsule development are complex and often contradictory. It is worth noting the amount of time that elapsed from contract award to first manned flight for America's first three capsules:
Mercury: award Feb 1959, manned flight May 1961
Gemini: award Dec 1961, manned flight March 1965
Apollo: award Nov 1961, manned flight October 1968
By today's standards, Mercury was executed on a whirlwind schedule. It was the most simple of America's manned spacecraft, but the program also involved a lot of learning how to do things that no human had ever attempted before. Gemini was a logical evolution of Mercury, and had its development schedule accelerated due to the lessons learned from Mercury. Apollo actually started before America's first orbital space mission, and involved a lot of learning during the design process. While NASA and North American Aviation believed that Apollo was ready for manned flight by February 1967, the Apollo 1 fire tragically demonstrated that the spacecraft had not reached a sufficient level of maturity.
With history as an example, can we truly form a reasonable estimate for the Orion development time? To be fair, Orion requires a significant degree of re-learning forgotten lessons of the past. For instance, the Orion industry team has been unable to reproduce the ablator from Apollo's heat shield. Seven years is a fair estimate for the development time on a lunar-capable Orion, especially in light of the reduced funding levels that Orion receives when compared to Apollo.
I believe that spiral development would have been able to close the gap with Orion, but I disagree with the original strategy laid out by Admiral Craig Steidle when he was running the effort. I don't think it's necessary to have a flyoff between two unmanned "tech demo" spacecraft, as long as you're using technologies that have sufficient tech readiness levels. I believe that a "Block 1" Orion could have been developed to meet the earth-orbital mission requirement by the earliest possible date, with a lunar-capable "Block 2" Orion coming later in the program. NASA uses the "block" nomenclature, but it would appear that the current "Block 1" Orion is overdesigned for its mission of transporting six crew to the space station. It makes the leap to "Block 2" easier, but only prolongs the post-shuttle gap.
In order to meet the 2011 deadline with a seven-year schedule, NASA would have needed to put a prime on contract before the end of 2004. This didn't happen for a number of reasons. For one, presidential administrations tend to avoid big-budget commitments near the end of an election cycle because those decisions may be quickly overturned if the election goes the other way. It must also be noted that it takes a while for government agencies to perform studies, define requirements, issue a request for proposals, and conduct a source selection. While I believe that this could have been performed during 2004, it didn't for several reasons.
Sean O'Keefe's strategy when he was running NASA was to award study contracts for the overall architecture of the lunar mission. In fall 2004, the contractors responded with the results of their "Crew Exploration and Refinement" studies. Many of them utilized EELV's in conjunction with Shuttle-C derived heavy lifters, and favored the Earth-Moon Libration points as places for rendezvous between manned spacecraft, space stations, and landers. Very few of the studies bore any resemblance to the eventual ESAS study.
Rather than select an architecture based on the recommendations of one or many contractors, NASA went through a great upheaval during the end of 2004, continuing into Spring 2005. Sean O'Keefe resigned, and Michael Griffin was confirmed to replace him. The Griffin solution was to ignore the CE&R studies and conduct an internal NASA study in 60 days--Exploration Systems Architecture Study, or ESAS. While it was the administrator's perogative to do so, it rendered most of the work performed in 2004 as wasted.
A good question is whether all of the studies mentioned before were necessary before the Orion contract could be awarded in mid-2006. A lot of work was performed during the Orbital Space Plane program which could have been used to issue a request for proposals on a Block 1, ISS-only Orion spacecraft. The risk is that the Block 1 spacecraft that results might be so unsuited for the lunar mission that the Block 2 spacecraft would be a totally new development. Back in 2004, Americans weren't too concerned that Russia would go rogue and start invading western allies (although Vladimir Putin's curtailment of civil liberties after the Beslan Massacre should have given us reason to pause.) I don't think there was any great rush to get Orion developed, and it was partly due to a naive faith that the Russians would help us get through the gap.
A final factor worth mentioning is that the Ares program relies on taking over shuttle facilities as that program dwindles to a close. Gemini didn't need to wait for Mercury launch pads to open up, and Apollo didn't wait for Gemini launch pads. While the Orion spacecraft is just as much of a pacing item as the Ares I rocket, the need to rebuild shuttle launch pads and reuse other shuttle facilities creates interesting dilemmas. Any extension in the shuttle program (a position now favored, to various degrees, by both presidential candidates) is a likely delay to a first Ares-Orion launch. If one launch pad is modified to support Ares I and Orion, it forces the shuttle program to abandon parallel processing of orbiters and adopt a slower "serial processing" workflow. If a Block 1 Orion could be flown on a Delta IV Heavy, the shuttle program could be extended while retaining parallel-processing and making no impact on the Orion schedule.
It's been a winding road of political and management decisons which got us to The Gap. It will be an even more difficult traverse to get us out of this predicament. The shuttle can't go on forever, and America can no longer rely on Russia's political leadership or buy the Soyuz. Unless Elon Musk can get a manned Dragon and Falcon 9 flying within the next few years, we're going to be in for a lot of trouble.