Enables Astronomers to Use Web to Search Different Databases
Simultaneously
Taking just two
months to dash through a project once anticipated to need a
year or more of work, a team at The Johns Hopkins University
has created a way to search three different astronomy
databases seamlessly and simultaneously via the World Wide
Web.
The new service, known as SkyQuery, works through a
programming approach known as web services, a revolutionary
new technique that allows various kinds of computers in a
variety of locations across the internet to cooperate in
providing a service to users.
SkyQuery is an important step forward for the National
Virtual Observatory (NVO), a project supported by the
National Science Foundation and currently in development at
Hopkins and many other research instituitions. NVO
scientists want to help astronomers take full advantage of
the massive amounts of astronomical information piling up in
databases worldwide. Astronomers have begun to pull in data
at such an incredible rate that there’s great potential to
make important new discoveries in the backlog of data
without ever looking up at the sky. NVO scientists plan to
use the latest computer technology and data storage and
analysis techniques to establish a unified and relatively
painless way to access as many databases as possible through
one online portal.
“SkyQuery demonstrates this principal by making access to
three large astronomy databases entirely transparent,” says
Alex Szalay, co-principal investigator of the NVO and Alumni
Centennial Professor of Astronomy in Johns Hopkins’ Krieger
School of Arts and Sciences. “You can run a query as if
these three databases, located at sites across the country,
were a single database.”
Szalay, who has a joint appointment in Hopkins’ computer
sciences department, says he and other NVO investigators had
once thought that getting a first, prototypical search
engine online for NVO might take up to 3 years. SkyQuery
came together quickly both as a result of the design team’s
intense efforts and as a result of a Microsoft programming
contest, the Microsoft .Net Best Awards for 2001 for XML
WebServices.
SkyQuery won second prize in that contest, which
challenged college students across the country to write new
programs that took advantage of .NET, a new Microsoft web
services platform.
Microsoft will award contest prizes at a reception on
April 10 in New Orleans at TechEd 2002, a
Microsoft-sponsored conference for software developers and
other information technology professionals. As second-prize
winners, Johns Hopkins team members will receive $10,000,
and Microsoft will donate an additional $10,000 to a Johns
Hopkins scholarship fund.
After an initial call for proposals for the contest last
year, judges from the computer industry selected 100
finalists in January 2002. Like the other contestants, the
Hopkins team, comprised of Szalay; Tanu Malik, a Hopkins
computer sciences graduate student; Tamas Budavari, an
astronomy postdoctoral fellow; and Ani Thakar, an associate
research scientist in astronomy; had to have their finished
program to Microsoft by March 15, little more than two
months later.
“Nothing like a little pressure,” jokes Szalay, who adds
that the bulk of the responsibility for getting the program
put together in time fell on Malik, who served as the team’s
leader, and Budavari. “I have rarely seen people that
exhausted.”
“All of us went to a conference at Microsoft, to learn
what .NET is all about,” Malik recalls. “When we came back
from that conference, then we started implementing it, and
it was like three weeks, day-in and day-out, working on it.”
Thakar credited the extraordinary capabilities of
Microsoft’s .NET platform with making the tight deadline
doable.
“.NET made the job about 10 times easier than it would’ve
been otherwise,” says Thakar. “It supplies the framework
and all the building blocks necessary to develop web
services in a standardized and easy way.”
The team’s finished product can be accessed at
http://contest.eraserver.com/SkyQuery/ . With an
understanding of the language used to run database queries,
any visitor to the page can run a search that simultaneously
accesses data from the Sloan Digital Sky Survey (SDSS) in
Batavia, Illinois; the Faint Images of the Radio Sky at
Twenty centimeters survey in Baltimore, Maryland; and the 2
Micron All-Sky Survey (2MASS) in Pasadena, California. The
SDSS and 2MASS are both currently expanding with new data
that will be accessible through SkyQuery.
One of the bigger challenges of SkyQuery, according to
Budavari, was developing the algorithms that the service
uses to determine when data in different databases come from
the same astronomical object.
“We developed ways to use behind-the-scenes info in all
three databases to help us do probabilistic matches and
figure out which objects are the same,” Budavari explains.
Budavari adds that SkyQuery is “non-static.” Other
databases can and will be added in the future to increase
the power of the search engine. He did, however, have one
caveat about SkyQuery. It seems that in the rush to get
things together for the Microsoft competition, one element
of the SkyQuery portal slipped behind.
“We’re going to be working to make the web portal a
little more aesthetically pleasing,” says Budavari with an
embarrassed smile.
Other top finishers in the Microsoft contest include a
team from the University of Minnesota, Twin Cities, that won
first prize for Renderfarm.NET, a program that transforms
three-dimensional data into animations and images; and a
team from the University of Bridgeport that won third prize
for a program called Brain Webber QA that lets users track
and manage defects in online endeavors and other projects.
Related Web sites: National Virtual Observatory:
