Skip Navigation: Avoid going through Home page links and jump straight to content
iWorld Web Site Newsstand Trade Shows Corporate Information Search Page webweek.gif magazines.gif
August 18, 1997

Skip Navigation: Avoid going through Home page links and jump straight to content

Mars Attacks the Web

By Todd Spangler

NASA first readied for 32 million hits per day. Private industry helped it handle 46.9 million.

Pathfinder was about to land on Mars, and engineers at NASA suddenly discovered a looming problem.

Sure, everything was going extremely well for the Mars Pathfinder mission itself, and had been ever since NASA's Jet Propulsion Laboratory (JPL) launched the probe on Dec. 2, 1996.

The crisis emerged not in space, but on the Internet: Last year, when NASA's Web site planners had looked forward to the July touch-down date, they knew that NASA's Web server capacity would be miserably inadequate to serve the unprecedented demand for Pathfinder's "Mars, Live" reports.

Kirk Goodall was the NASA Web engineer for the JPL's Mars Pathfinder site team. He had to cobble together the necessary bandwidth and server capacity for what turned out to be the most popular event in the history of the Internet.

From July 1 to Aug. 4, the various Mars Pathfinder sites combined received a stratospheric 566 million hits. On the busiest day, July 8, the Mars Pathfinder sites combined for almost 47 million hits, which was more than double the traffic of any other Web site event.

This year's Kasparov-Deep Blue chess site hosted by IBM had 21 million hits on its busiest day. The 1996 Summer Olympic Games site rang up a peak of 18 million hits on a single day.

The most Internet traffic NASA had ever handled prior to the Mars Pathfinder mission was about 2 million hits per day for the Shoemaker-Levy and Hale-Bopp comets. NASA JPL has two T-3 lines (45 Mbps each) coming into its Pasadena, Calif., facility. The planners knew the Pasadena site alone wouldn't be able to handle the load, but had assumed it would be easy enough to distribute it across the rest of NASA's Web servers.

Goodall remembered discussing the problem in August 1996 with David Dubov, the site's Webmaster, and Bob Anderson, the Pathfinder mission's public outreach coordinator, who oversaw the Web site. Estimating that they would need to serve 25 million hits per day at most--which in hindsight turned out to be about half of the 46.9 million peak--they tallied up the estimated capacity of all of NASA's Web sites.

And then panic kicked in. Goodall's first step was to mirror the site on government and university servers that are part of the Very High Bandwidth Network Service (vBNS), run by MCI under an agreement with the National Science Foundation. That gave him the ability to handle an estimated 32 million hits a day.

But it turned out that not every vBNS location had a high-speed connection to the outside Internet in place, although the administrators reassured Goodall that everything would be fine come July 4.

Goodall was beginning to sweat. "I wasn't feeling too comfortable," he said. "I know from experience that when you're trying to set things up in just a couple of weeks, it's difficult. People go on vacation and things fall through the cracks."

At that point, Anderson suggested to Goodall that he turn to the private sector, but that would introduce a number of complexities because of the restrictions the federal government places on its agencies in dealing with corporations.

Though he knew that with corporate Web servers at NASA's disposal the Mars Pathfinder sites could handle about 80 million hits, Goodall still felt as though he was winging it. To get the necessary approval, he drafted a memorandum of understanding based on text from JPL's legal division. The agreement between NASA and the corporations essentially said their mirrored Web sites could not alter the content in any way, and could contain no advertising.

Goodall approached Silicon Graphics, Sun Microsystems, and Digital, all of which immediately agreed to mirror the Pathfinder site and each of which said they could handle 10 million hits a day. "They all knew that being part of this would give them a lot of visibility," he said.

Then Goodall approached the major Internet service providers and online services for support. He succeeded only with AT&T WorldNet and CompuServe, though America Online eventually added a mirrored site.

Still, the corporate partnerships just about fell through. With two weeks remaining before the landing, Goodall had not received approval from the JPL legal department. The corporations wanted to use their logos, and the legal department was resisting because it implied endorsement.

"I finally said, 'Forget the logos; if you guys want logos, it's going to kill the whole thing,'" Goodall said. "'Just dump the logos and use the American flag.'" The agreements were signed and approved one week before the landing.

Finally came the day of reckoning. Even with all the capacity of 20 mirror sites in place and ready on July 4, some things went wrong. A router that was improperly configured at the NASA Ames site in Northern California caused a crippling bottleneck, rendering the center's network practically useless. Several servers around the world ran out of disk space and crashed.

Even the two main Web servers at JPL started to get bogged down, so Goodall and crew quadrupled the memory in their two Sun SPARC Web servers to 256 Mbytes. "All of our sites used this as an opportunity to tune their networks. They had never seen traffic like this before," Goodall said, without a trace of irony.

NASA now has 46 mirrors for the Mars Pathfinder site, and Goodall said another 50 mirror sites will come online in September. NASA is getting ready for Mars Global Surveyor, another probe that will arrive at Mars Sept. 11, orbit for 4 months, and map the entire surface of the planet. Goodall thinks there will be an even bigger demand for the Surveyor data.

Surveyor will send back a lot more data, too; much more than the 1.2 Gbits of data Pathfinder had transmitted through Aug. 4. Whereas Pathfinder communicated at only 9.6 Kbps, Surveyor can transmit at 56 Kbps through NASA's Deep Space Network, because it has a more powerful antenna and because it can send its signals from orbit rather than from the surface of Mars.

These days, Goodall feels much better about NASA's Web capacity for Surveyor. "We're ready," he said confidently.


Back to Home Page

Keywords: infrastructure
Date: 19970818