Spidr is a mirror for Solar Terrestrial
Data such as geomagnetic indices, ionospheric soundings, sunsport numbers etc
and DMSP satellite imagery. In total each SPIDR site contains over 20 Gb of
data dynamically served through a web based front end system.
SPIDR sites are kept in synch through a JavaMail based mirror
system.
Each site is not required to host every data set. Data sets which are not
mirroed are accessed remotely through the NGDC site providing data equivalent
access at each site with improved access to some.
There are currently 5 SPIDR site sup and operation with a visit by the Chinese
planned for early October.
SPIDR is designed to be server minimal, a simple site can consist of a single
PC with several Gb of disk space. For sites that wish to maximize performance SPIDR
offers the ability to scale to unlimited parallel servers. The master site in Boulder
is currently running on 8 Linux based PC's the site in South Africa on a single
PC.
Perl is a script programming language that is
similar in syntax to the C language and that includes a number of popular UNIX
facilities such as SED, awk, and tr. Perl is an interpreted language that can
optionally be compiled just before execution into either Ccode or cross-platform bytecode.
When compiled, a Perl program is almost (but not quite) as fast as
a fully precompiled C language program. Perl is regarded as a good choice for
developing common gateway interfac e (CGI) programs because it has good text
manipulation facilities (although it also handles binary files).
Java is a programming
language expressly designed for use in the distributed environment of the
Internet. It was designed to have the "look and feel" of the C++
language, but it is simpler to use than C++ and enforces an object-oriented
programming model. Java can be used to create complete applications that may
run on a single computer or be distributed among servers and clients in a network. It can
also be used to build a small application module or applet for use as
part of a Web page. Applets make it possible for a Web page user to interact
with the page.
Secure Shell (SSH), sometimes known as Secure Socket Shell, is a
UNIX-based command interface and protocol for securely getting access to a remote computer.
It is widely used by network administrators to control Web and other kinds of servers
remotely. SSH is actually a suite of three utilities - slogin, ssh, and scp -
that are secure versions of the earlier UNIX utilities, rlogin, rsh, and rcp. SSH commands
are encrypted and secure in several ways. Both ends of the
client/server connection are authenticated using a digital certificate, and
passwords are protected by being encrypted
Apache is a freely
available Web server that is distributed under an "open source"
license. Version 2.0 runs on most UNIX-based
operating systems (such as Linux, Solaris, Digital UNIX, and AIX), on other
UNIX/POSIX-derived systems (such as Rhapsody, BeOS, and BS2000/OSD), on AmigaOS,
and on Windows 2000. According to the Netcraft (www.netcraft.com) Web server survey
in February, 2001, 60% of all Web sites on the Internet are using Apache (62%
including Apache derivatives), making
Apache more widely used than all other Web servers combined.
rsync is a file
transfer program for Unix systems. rsync uses
the "rsync algorithm" which provides a very fast method for bringing remote files into sync. It does
this by sending just the differences in
the files across the link, without requiring that
both sets of files are present at one of the ends of the link beforehand.