Instructions for adding data

Data Organization

These guidelines are as follows:

• Data are stored by program type (broadscale, modeling, mooring, process and satellite).
• Data, where appropriate, may be "stored" [accessible, really, since data need only be stored once] in more than one program type.
• Data are organized by year of collection, within program type.
• Data are structured hierarchically with the slowest changing variable first (e.g. cruiseid, leg, station, then cast number).
• Use variable/parameter names as defined in the program's thesaurus. If your variable is missing, please contact the Data Management Office.

For more information or when you are ready to add data, contact the Data Management Office.

Nitty Gritty Details

To add data to the JGOFS distributed database system used by the U.S. GLOBEC Georges Bank Program, take the following steps:

1. Decide where data will reside, either on your own server or the GLOBEC server.
2. Decide from which program type (broad-scale, modeling, mooring, process, or satellite) it will be served. This will determine which .remoteobjects file will reference your data. (See more about this below.)
3. Create an entry for the .remoteobjects file as follows:
   • data_id_a=//server.computer.name/directory_path/data_id_b
   • - SI_name
   • - [URL] Brief description of the data

   where data_id_a and data_id_b are names identifying the specific data you are referencing. They need not be the same names, but can be. However, data_id_a does need to be unique with respect to other object names in the same directory. Note that in a future release of the jgofs software, it will be necessary to have data_id_a and data_id_b be the same to take advantage of new capabilities.

   "Server.computer.name" is the internet name of the computer system serving the data such as mycomputer.univ.edu.

   "Directory_path" is the path to the .objects file beginning from the "object root" (defined for each data server). (For example the "object root" might look like /data/objects/ and the "Directory_path" sits below this level.)

   SI_name is the name of the science investigator responsible for this data. Limit the "Brief description" of the object to less than about 40 characters, although an optional URL can be included if you want to have extended text, even images, referenced here.

   Note that the format of these lines is important, in particular the dash (-) must be the first character on the second and third lines and must be followed by a space in each line.

   (There can be qualifiers on the remote name as well but these are not discussed here.)

4. Pass this information along to the U.S. GLOBEC Georges Bank office in order to update the central database directory and make the data available to everyone. Until you do this step, only those people that know the data exist and know its name (data_id_a) will be able to access the data.
5. The .objects file contains an entry for each file (or object) you will be serving from your machine to the user community.

   The .objects file resides only on the system which serves the data. This permits the data contributor to move the data to another location or change the data file name without affecting other users of the database.

   Create an entry for the .objects file as follows if you will be serving the data yourself. The form of this entry is as follows:

   • data_id_b=method_name(parameterlist[,optionalparameters])

   where data_id_b is the same identifier as was used in the .remoteobjects file (see above),

   method_name is the method used to access the data (e.g. def, nm, lm, http...)

   parameterlist usually consist of at least the path and filename of where the data are stored on your system plus any other parameters needed by the method. This is dependent on the requirements of the method.

   Optionalparameters include such things as projection options which cause the method to return only specified field names, such as latitude, longitude, water depth and temperature even though the data file contains many more field names and values. The optionalparameters can also allow methods to restriction data values to specified values or ranges of values.

   Other formats for the .objects entries include:

   • objectname=reference--to--other--obj[(params)]

   or

   • objectname=methodname(parameters[,n1=v1,n2=v2 ...])

     where the n1=v1, etc. optional strings are inserted into the environment before the method is executed. They can be used for selection (n1>v1) or projection (n1,n4).

   The parameters in the .objects file are merged with those in the request.

   Here are two examples of .objects entries. The first uses the default method def to access the mocness.dat file of Mocness data while the second example uses the nm method to access only pressure and salinity data from the file called station.dat restricted to pressure less than 450 and only for station 7.

   • mocness1=def(/data/smith/cruise1/mocness.dat)
   • ctd123=nm(/data/brown/ctd/station.dat,press,sal(press<450,station=7))

6. Create the information necessary for the descript.html file. This file contains information about all the data referenced in the .remoteobjects file. A sample of such a file follows. Provide the data management office
with this information so it can be entered into the system. The information should be sent in plain text (sending via e-mail is fine).

eventlog
< /h1 > [Note: Only the closing tag is necessary to highlight the name.]

     PI:       A. Smith
    Dataset:        CTD data
    Ship:            R/V Albatross IV
    Cruise:         9312-IV

Parameter        Description                      Units 
----------------------------------------------------------
  eventno        event/operation number            nd
  instrument     instrument used to collect data   nd
  cast           cast number                       nd
  station        station number                    nd
.
.
   lat           latitude                  decimal degrees
   lon           longitude                 decimal degrees
   wdepth        depth of the water             meters
   depth         depth of the cast              meters

eventlog

This information is repeated for each object defined in the .remoteobjects file bracketing each description using the name of the object. The specific formatting of the information can vary. This information is made available when a person clicks on the highlighted third entry in the .remoteobjects file.

Note 1: If the above instructions were not clear, click here for another version of these instructions.

Note 2: The JGOFS/GLOBEC system is pretty flexible in how it deals with actual data values. "Data" are typically ASCII text, either numbers or text, although "data" can also be URL links to images and movies. We have found that, with some precautions, the following symbols can be used in the data:

? . : / * - + [ ] { } ' ~ @ ^

However, the following symbols should not be used in the data:

commas, tabs or ; ( ) \ " ` ! # $ % & = , < >
The semicolon appearing in data can cause a problem when selections are specified.