To get around Earth's atmosphere, astronomers in the1960s began attaching telescopes to huge balloons to carry them above the lower atmosphere's obscuring effects. By the early 1970s, scientists attached small telescopes aboard high-flying Lear jets and sounding rockets, and discovered several thousand infrared sources.

None of these observatories, however, could get completely above the atmosphere, and by the early 1970s, astronomers began to consider the possibility of putting an infrared telescope into orbit. Most of the early ideas involved carrying the telescope on repeated flights on board the NASA Space Shuttle, but this idea was developed at a time when NASA thought that the Shuttle would be making routine flights every week, lasting 30 days or more. More importantly, it assumed that the contamination from the Shuttle (e.g., vapors, small particles, and heat interference) would not be a significant problem.

In 1979, the idea for a Shuttle Infrared Space Facility (SIRTF) was highly recommended in a report by the National Academy of Sciences, and in 1983, NASA announced a call for proposals to build instruments for a large Shuttle-based observatory, to remain attached to the Shuttle during the mission, and returned to the ground for refurbishment prior to re-flight. The first launch was expected to take place around 1990.

As NASA made this announcement, the first infrared telescope was launched into space by a collaborative team of the United States, the United Kingdom, and the Netherlands. The Infrared Astronomical Satellite (IRAS) was an Explorer-class satellite designed to conduct the first infrared survey of the sky. The 10-month mission was a resounding success, and led to huge interest in a follow-up mission from astronomers around the world. The impressive scientific returns from this free-flying observatory led NASA to revise their plans "to provide flexibility for the possibility of a [free-flyer] SIRTF mission."

In 1984, NASA selected a team of astronomers to build the instruments and define the science program for a free-flying mission. The following year, in July 1985, this proved to be the correct decision, as the first Shuttle-based InfraRed Telescope (IRT) found that contamination problems from the Shuttle were considerable. The decision to proceed with a free-flyer observatory led to the first change for Spitzer, transforming the meaning of "SIRTF" to the Space Infrared Telescope Facility.