The Astro-Wise System: A Federated Information Accumulator for Astronomy- II

This improvement is achieved by:
• emphasis on project management;
enforcing a global data acquisition and
processing model, while retaining
flexibility
• translating the data model to an object
model, with full registration of all
dependencies
• storing all I/O of the project in a single,
distributed database, containing all
metadata describing the bulk data (eg
images) and derived results in catalogue
form (eg lists of celestial
sources).
• connecting to the database a federated
file server that stores hundreds of
Terabytes of bulk data
• an own compute-GRID which sends
jobs (including clients) to single nodes
or parallel clusters, which then request
data from the distributed database.
The database with all metadata and catalogues
provides the infrastructure to
develop tools for a variety of purposes.
These include rapid trend analysis of
data, complex queries and fast hunting
for ‘needles in the haystack’ of Terabytesized
catalogues. Thus, the system provides
the user with fully integrated,
transparent access to all stages of the
data processing and thereby allows the
data to be reprocessed and the system to
be improved and expanded.
For a given project/instrument, the
system initially starts in a naive, ‘quick
look’ mode, which gradually improves
as various researchers add refined information
to the system under the supervision
of project leaders. Approved calibration
modifications automatically
become public, beyond the project
boundaries. A mechanism for quality
control is implemented which allows for
changes due to one of:
•true physical changes of parameter
values
•improvements in encoded
methods, or
•improved insight in either of
these.