Instructions for upgrading an existing JGOFS V1.73 installation to V2.00
------------------------------------------------------------------------
Dennis Schmidt
November 9, 2008


This file is  README_jgofs-V2.00_INSTALL_UPGRADE.  Distributed in file  
jgofs-V2.00.tar, it provides instructions for upgrading a JGOFS data server 
from V1.73 to V2.00.

If what you want to do is create a new JGOFS data server, rather than upgrade 
an existing one, DON'T use these instructions; instead, use the instructions 
in file  README_jgofs-V2.00_INSTALL_NEW, also distributed in file  
jgofs-V2.00.tar.



- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
NOTES:
	1. Although the procedure is straightforward, these instructions 
are long, due to the amount of explanation provided.  Please follow them 
step by step; if you do, you should have little or no difficulty in upgrading.

	2. The five scripts in this installation -- four of which are 
in the gzipped tar files "jgofs-V2.00_specialwrap1" and 
"jgofs-V2.00_specialwrap2" -- are perl scripts, with names ending with 
".pl".  They assume that perl is  /usr/bin/perl.  If your perl is located 
elsewhere, we recommend creating a symbolic link at  /usr/bin/perl  so that 
you don't have to modify and re-package the scripts.  You might need the 
assistance of a system administrator to create that link if you need it.  
This is the syntax:
		ln  -s  <wherever perl is>  /usr/bin/perl
where "<wherever perl is>" is replaced by the path shown by "which perl".
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -



============================================================================
Preliminary step:  Unpack file  jgofs-V2.00.tar  in the root directory
================   of an existing JGOFS V1.73 data server to upgrade

To upgrade your existing JGOFS V1.73 data server, you will need to put 
the  jgofs-V2.00.tar  file into the server's root directory, if you haven't 
already.  Then, if you haven't already, unpack the tar file there:

	cd  <JGOFS server root directory>
	tar  xfv  jgofs-V2.00.tar

In the "cd" command, replace "<JGOFS server root directory>" with the path to 
your JGOFS server root directory.

This will unpack five files, including this one.

Don't remove the  jgofs-V2.00.tar  file from the directory.



============================================================================
Step (1):  Use  jgofs-V2.00_1install_script.pl  to back up and identify 
========   the site customizations present in your JGOFS V1.73 data server


Run the initial script to back up the existing installation and to get 
the site customizations:

	./jgofs-V2.00_1install_script.pl


Note that this is the only action you will take in this step, except for 
checking the results.  The following paragraphs explain the results and 
how to check them.


Running the  jgofs-V2.00_1install_script.pl  script will create:

	- file  <servername>_jgofs-V1.73_backup_<timestamp>.tz, in the 
    parent directory of the current working (server root) directory, where 

	    "<servername>" is replaced by the name of 
	     the current working directory, and 

	    "<timestamp>" is replaced by a 14-digit string 
	     identifying the date and time of running the script

	    (we will use these notations throughout these instructions); 

    this gzipped tar file is a backup copy of the existing V1.73 data server 
    installation;

	- file  log_jgofs-V2.00_1upgrade_prep_<timestamp>, in the current 
    working (server root) directory; this file is a log of the running of 
    the script;

	- file  <servername>_V1.73diffs.output, in the current working 
    (server root) directory; this file shows the site customizations in 
    the existing V1.73 data server;

	- directory  <servername>_V1.73customcopyroot, under the parent 
    directory of the current working (server root) directory; in the tree 
    under the created directory will be saved copies of all the customized 
    files from your existing V1.73 installation; you will refer to these 
    files in step (3), below, when carrying forward the customizations into 
    the new V2.00 installation.


The script will terminate early if there is a problem at any point.  If 
the script finds that the directory it's running in doesn't look like a 
JGOFS V1.73 root directory with a valid jgofs-V2.00.tar file unpacked in 
it, the script will report the problem on-screen and stop.  If the 
directory looks OK, the script will start the log file 
(log_jgofs-V2.00_1upgrade_prep_<timestamp>) and record the progress of each 
subsequent step there.  You will need to check the log file when the 
script is finished, to assure that each step finished without errors.

This is the sequence of steps recorded in the  
log_jgofs-V2.00_1upgrade_prep_<timestamp>  file:

	- Unpack gzipped tar file  jgofs-V2.00_specialwrap1  into 
	   the current working directory; this will produce a short 
	   list of files in the log file.

	- Back up the existing V1.73 installation to file  
	   ../<servername>_jgofs-V1.73_backup_<timestamp>.tz;
	   this will produce a long list of files in the log file.

	- Create temp directory  /tmp/jgofs-V1.73_<timestamp>  into which 
	   to unpack an unmodified copy of the V1.73 distribution.

	- Unpack an unmodified copy of the V1.73 distribution into the 
	   temp directory; this will produce a long list of files in 
	   the log file.

	- Run script  get_v173_site_diffs.pl  to generate file  
	   <servername>_V1.73diffs.output; that script won't add anything 
	   to the log file.

	- Remove the unpacked unmodified V1.73 distribution and its temp 
	   directory.

	- Remove no-longer-needed files unpacked from  jgofs-V2.00.tar  
	   and  jgofs-V2.00_specialwrap1; this will produce, in the log 
	   file, a list of the files removed.

	- Make "convenience copies" of all site-customized files found 
	   in your installed V1.73 data server, under directory  
	   ../<servername>_V1.73customcopyroot, for use during customizing 
	   your V2.00 upgrade; this will produce, in the log file, a list 
	   of the customized files.


The following paragraphs describe the contents of file  
<servername>_V1.73diffs.output, which details the site customizations in 
your installed V1.73 data server.

The  <servername>_V1.73diffs.output  file has two main sections after its 
header lines.  The first section shows comparisons between installed files 
and their same-named source files in the V1.73 distribution.  The second 
section compares certain installed files (e.g., htmlbin/dir ) with template 
files from which they were originally derived (in the case of  dir, the 
template file is  src/htmlbin/dir_pl.tmp).  In each section of the  
*_V1.73diffs.output  file, if there are no diff-format lines following the 
line stating which files are being compared (that line begins with 
"********"), then the two files compared are identical; that will be the 
case for most files.

Most of the customizations will probably be in a limited number of lines in 
the  build-env  file.  You should note all of these.  We will detail below, 
in step (3), how to carry them forward into V2.00.

You may see two kinds of customization in the files derived from template 
files, listed in the last section of the  *_V1.73diffs.output file, 
involving the following files:
 
	htmlbin/dir
	htmlbin/info
	objects/.objects
	objects/.remoteobjects 
	objects/test.info.  

The first kind of customization is site specification that 
was accomplished automatically by the V1.73 build process.  This 
consists of the replacement of strings such as "perllocation", "jgofsdir", 
"JGOFSDIR", "dmoname", "objectdir", "informationserver", "imagedirectory", 
and "myaddr" with values specific to your server.  The V2.00 build process 
will do this site specification again automatically.  Any other changes you 
see, though, originated outside the build process and will need to be 
restored.  For this upgrade, the easiest way to handle this will be simply 
to restore the "convenience copies" of your current versions (saved by this 
script, under directory  ../<servername>_V1.73customcopyroot/) after you 
build V2.00.  This works for this upgrade because there have been no changes 
in the template files between V1.73 and V2.00.  Again, we'll detail the 
steps below.

You must also note and understand any other customizations, to be able to 
handle them correctly.  There are likely to be none, but if, for example, 
you have changed anything in  src/include/default.h, src/include/inner.h, 
or  src/lib/makefile.jgofs  (or, less likely, any other source file), you 
will need to integrate your changes with the new versions in V2.00.



============================================================================
Step (2):  Use  jgofs-V2.00_2upgrade_script.pl  to clean out the existing 
========   V1.73 installation and unpack the new V2.00 distribution


- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
IMPORTANT NOTE:
Proceed with step (2), below, only when you are satisfied that step (1) 
has produced a good backup copy of your installation, in a gzipped tar file 
whose name has the form  ../<servername>_jgofs-V1.73_backup_<timestamp>.tz.

If something goes wrong later in the upgrade process, it is VERY IMPORTANT 
not to just start over.  First you MUST remove EVERYTHING that has been 
added into the corrupted server tree, and restore the server to EXACTLY 
the state it was in (as V1.73) before you started the attempt to upgrade.  
We provide a script to make it easy to do this.  This thorough cleaning and 
regression is necessary because of the significant restructuring between 
V1.73 and V2.00.  Restarting the upgrade process without restoring V1.73 
EXACTLY is hazardous; omitting the cleanup, or doing only a partial cleanup, 
is likely to leave files in place that would interfere with the correct 
functioning of the V2.00 installation.

(What follows here is not step (2) of the upgrade procedure; 
step (2) is below, following the dashed line.)  
DO THIS ONLY IF YOU NEED TO RECOVER FROM AN UNSUCCESSFUL PARTIAL UPGRADE 
OF THE SERVER:
	chmod u+x XXX_remove_bad_V2.00_restore_V1.73.pl
	./XXX_remove_bad_V2.00_restore_V1.73.pl
from within the server root directory.  If that script isn't in the 
directory, then the V1.73 server hasn't been disturbed yet.

The script will ask you to confirm your intention.  Then if you say 
to go ahead, it will remove all the added files from the corrupted 
server tree, and restore the complete V1.73 server tree saved in file  
../<servername>_jgofs-V1.73_backup_<timestamp>.tz.  If there is more than 
one such backup file, it will use the one with the latest timestamp.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -


Still in the data server root directory, run script  
jgofs-V2.00_2upgrade_script.pl  to clean out the existing installation 
and unpack the new distribution:

	./jgofs-V2.00_2upgrade_script.pl


Note that this is the only action you will take in this step, except for 
checking the results.  The following paragraphs explain the results and 
how to check them.


Running the  jgofs-V2.00_2upgrade_script.pl  script will:

	- create file  log_jgofs-V2.00_2finish_upgrade_<timestamp>, in the 
    current working (server root) directory; this file is a log of the 
    running of the script; note that this is a separate file from the log 
    of running the first script;

	- convert your existing customized JGOFS V1.73 data server into 
    an un-customized V2.00 data server.


The script will terminate early if there is a problem at any point.  If 
the script finds that the directory it's running in doesn't look like a 
JGOFS V1.73 root directory that also contains the files needed to upgrade 
to V2.00 (the gzipped tar files  jgofs-V2.00_dontuntar_file  and  
jgofs-V2.00_specialwrap2), the script will report the problem on-screen 
and stop.  It will do the same if the V1.73 backup that should have been 
made in step (1) is missing.  If the directory and the backup look OK, the 
script will start the log file (log_jgofs-V2.00_2finish_upgrade_<timestamp>) 
and record the progress of each subsequent step there.  You will need to 
check the log file when the script is finished, to assure that each step 
finished without errors.

This is the sequence of steps recorded in the  
log_jgofs-V2.00_2finish_upgrade_<timestamp>  file:

	- Unpack gzipped tar file  jgofs-V2.00_specialwrap2  in 
	   current working directory; this will produce a short 
	   list of files in the log file.

	- Run "./build clean" to clean out generated files from the 
	   V1.73 installation; this will generate extensive output in 
	   the log file; see below for a description of expected error 
	   messages that you can safely ignore.

	- Run script  jgofs-V2.00_finish_v173_clean.pl  to clean out 
	   other files that are obsolete or moved or renamed in V2.00; 
	   that script won't add anything to the log file.

	- Unpack the gzipped tar file containing the actual V2.00 
	   distribution; this will produce a long list of files in the 
	   log file.

	- Install the un-customized V2.00  src/lib/makefile.jgofs.

	- Remove no-longer-needed files unpacked from  jgofs-V2.00.tar, 
	   jgofs-V2.00_specialwrap1, and  jgofs-V2.00_specialwrap1; 
	   this will produce, in the log file, a list of the files removed.

These are the three kinds of error messages expected in the log file from 
the "./build clean" command; you can safely ignore them:
	[1] "rm: cannot remove `./<file name>': No such file or directory" 
		(where "<file name>" is replaced by a file name); followed by:
	    "make: [clean] Error 1 (ignored)"
		(There will be many of these messages.)
	[2] "make: *** No rule to make target `clean'.  Stop."
		(There will be a few of these messages.)
	[3] "make: listvar.o: Command not found
	     make: *** [clean] Error 127"
		(This is the only message of this kind that you should see.)



============================================================================
Step (3):  Customize and build your new V2.00 distribution
========

-----
(3.1)  Edit the new file  build-env.pl  to restore the customizations that 
-----  were in file  build-env  in V1.73, and to specify a few new parameters.  
The following are likely to be all the lines that you will or might need to 
edit.  These summaries show the lines as they appear in the distributed file, 
they briefly explain the options, and they point you to the analogous lines in 
the V1.73  build-env  file.  (Note that the V1.73  build-env  line numbers 
might be offset from those in your installation, if, for example, you have 
added any lines.  Note also that some default values in the distributed V2.00  
build-env.pl  file have changed from the defaults in the distributed V1.73  
build-env  file; make sure the values in your installed V2.00  build-env.pl  
file are correct for your server, even if in V1.73 you had to edit them and 
now you don't, or vice versa.)  You can find a copy of your customized V1.73  
build-env  at  ../<servername>_V1.73customcopyroot/build-env.

	V2.00 build-env.pl                             V1.73 build-env
	------------------                             ---------------
	line 165  ($opsys)                             line   9  (SYS)
	    $opsys = "linux";
	    (The standard values are "linux", 
	     "Solaris", "Irix")

	line 178  ($perl_location)                     line   8  (PERL)
	    $perl_location = "/usr/bin/perl";
	    (You can verify Perl's location by 
	     typing the command, "which perl")

	line 184  ($server_name)                       line   7  (MYADDR)
	    $server_name = "gb9.whoi.edu";
	    (Substitue your server's name)

	line 185  ($port)                              -----
	    $port = 80
	    (This will be correct for most servers)

	line 194  ($button_images_server)              line  12 (BUTTONIMAGESDIR--
            $button_images_server =                    "globec.whoi.edu";
               "http://globec.whoi.edu/images")
			                          
	    (You can serve button images from this URL, or you can 
	     serve local copies, downloaded via anonymous FTP from the 
	     /pub/images  directory on  globec.whoi.edu ; the file is 
	     jgofs_buttons.tar )

	line 196  ($button_images_dir)                 line  12 (BUTTONIMAGESDIR--
	    $button_images_dir = "images";	  "http://globec.whoi.edu/images")
	    
             (This is correct for the globec 
	     button images server)

	line 203  ($display_data_url)		        -----
	    $display_data_url = TRUE;
	    (Set to FALSE if you don't want to see the data 
	     object specifier as a header line on data page displays)

	line 208  ($background_color)                  -----
	    $background_color = "ffffff";
	    (This is white; the successive pairs of characters are the 
	     hexadecimal codes for the intensities of red, green, and 
	     blue components, each ranging from 0 ("00") to 255 ("ff"))

	line 215  ($dmoname)                           line  75  (DMONAME)
	    #$dmoname = "Data Server";		       (For U.S. JGOFS and GLOBEC
	    (Either leave this line commented          programs, see lines 65, 70
	     out (which will set $dmoname to           instead of line 75)
	     the same default used in V1.73 -- 
	     "$server_name -- " (where 
	     $server_name is the string set in line 184)); or 
	     uncomment the line and change the string as desired)
	    (If the server is a node in the JGOFS program or the GLOBEC 
	     program you may leave the line commented out, and $dmoname 
	     will be set up correctly in line 420 or 425)

	line 223  ($program)                            line  10  (PROGRAM)
	    #  $program = "";
	    (Leave this line commented out unless the server is 
	     a node in the JGOFS program or the GLOBEC program; in 
	     either of the latter cases, follow instructions in the file)

	line 431  ($objdir)                             line  76  (OBJDIR)
	    $objdir = "/";                              (For U.S. JGOFS and GLOBEC
	    (Use as in V1.73:  Leave unedited           programs, see lines 66, 71
	     if objects are to go directly              instead of line 76)
	     into the  objects  directory, or
	     name a subdirectory following the '/';
	     For U.S. JGOFS and GLOBEC programs, the correct value is set 
	     automatically in lines 421, 426 instead of using line 431)

If your  *_V1.73diffs.output  file shows any other customizations of your V1.73 
build-env  file, you will also need to work out how to translate them into the 
V2.00  build-env.pl  file.


-----
(3.2)  If any other V1.73 source files were customized -- the two most likely 
-----  candidates being  src/include/default.h  and  src/include/inner.h -- 
integrate those changes with the new V2.00 versions of these files.  Later, in 
step (3.5), you'll restore any customization of files generated from template 
files -- htmlbin/dir,  htmlbin/info,  objects/.objects, 
 objects/.remoteobjects,  objects/test.info.

If you customized any source files that no longer appear in the V2.00 source 
tree, you should contact the provider of the V2.00 distribution or see  
http://www.bco-dmo.org/contacts.html  to learn whether any action is needed.  
Don't just restore the old file, as it might conflict with something in V2.00.


-----
(3.3)  Build the V2.00 data server, saving the output in a log file; 
-----  you might create a new log file for the rest of the installation, 
as illustrated here, or extend the log file created in step (2), whose 
name has the form  log_jgofs-V2.00_2finish_upgrade_<timestamp>:

	./build.pl  >>log_jgofs-V2.00_3build_and_test  2>&1


-----
(3.4)  Check the output in the  log_jgofs-V2.00_3build_and_test  file.
-----  Note that it's normal (i.e., it's not a problem) to see several 
instances of a pair of lines of this form (with various text for the 
various *s in the string, "*/src/*/makefile.*"):

 *** make file */src/*/makefile.* does not contain appropriate
 *** macro name (CONFIG_FILE) to force makes

You will probably also see this pair of lines (once), which also is not a 
problem:

mkdir: cannot create directory `../../objects': File exists
make: [../../objects/.objects] Error 1 (ignored)


-----
(3.5)  If any of your V1.73 files derived from template files were customized 
-----  beyond the build-generated site specification explained above in step 
1, now you should restore your versions, overwriting the versions that the 
V2.00 build has just created.  Your versions are available under directory  
../<servername>_V1.73customcopyroot/; now you can copy them back to their 
original locations:

	cp -p  ../<servername>_V1.73customcopyroot/htmlbin/dir  ./htmlbin
	cp -p  ../<servername>_V1.73customcopyroot/htmlbin/info  ./htmlbin
	cp -p  ../<servername>_V1.73customcopyroot/objects/.objects  ./objects
	cp -p  ../<servername>_V1.73customcopyroot/objects/.remoteobjects  
								./objects
	cp -p  ../<servername>_V1.73customcopyroot/objects/test.info  ./objects



This completes the installation and customization of your V2.00 data server.



============================================================================
Step (4):  Test the V2.00 server
========


-----
(4.1)  Test operation from the command line:
-----	    
	bin/listvar  /test  >>log_jgofs-V2.00_3build_and_test  2>&1

should produce, at the end of the  log_jgofs-V2.00_3build_and_test  file, 
a hierarchical list of variables.

	bin/list  -f  -n  /test  >>log_jgofs-V2.00_3build_and_test  2>&1

should produce, at the end of the  log_jgofs-V2.00_3build_and_test  file, 
a "flattened" table of data using those variables.


-----
(4.2)  Test Web access to the JGOFS system:
-----
Presumably, since you are upgrading, you already have a working server 
setup.  If you don't, see step (4) of file  README_jgofs-V2.00_INSTALL_NEW  
(present in your JGOFS root directory), for rudimentary guidance in getting 
it set up.

Use your browser to go to this address:

	http://<server name>/jg/serv/test.html0

where "<server name>" is your server's name (one of ours, for example, is 
globec.whoi.edu; you specified yours on line 184 of  build-env.pl).

This should produce the "/test ---- Level 0" display, including links and 
buttons that should allow you to examine the test data and metadata in various 
ways.


-----
(4.3)  Test access to and operation of "dir":
-----  You probably have a Web page containing a link to your "dir" program -- 
a link like this one:

	<a href="/jg/dir">...</a>

where "..." may include a text label, an image, etc., associated with the 
appearance of the link on the Web page.
This link gives access to the objects specified in the  .remoteobjects  file 
in the  <JGOFS root directory>/objects  directory, and to objects in 
subdirectories of that directory.

Or you may have a link more like this:

	<a href="/jg/dir/b_objs/">...</a>

This link gives direct access to the objects specified in the  .remoteobjects  
file in the  b_objs  subdirectory of the  <JGOFS root directory>/objects  
directory.

Check that your links to "dir" work as they should.
