Installing the Pipeline
From Tauwiki
I can't understand why a person will take a year or two to write a novel when he can easily buy one from a book store. --Fred Allen
Contents |
[edit] Binary Installation
Precompiled java archive (jar) files of each module and a combined binary distribution are available from TAUVEX website. The binary distribution also contains the parameter files and calibration files along with a few "not so useful" files. These binaries are generated on a nightly basis. All the third party dependencies are packed with the distribution. Download the latest binary distribution from TAUVEX Download area and unpack the file (either zip or tar.gz). Pipeline is ready for use.. provided you have java installed.
[edit] Pre-requisite
UVS pipeline software requires Sun java(TM) runtime environment (JRE), version 1.4.0 or later. java is available for download from http://java.sun.com.
[edit] Organization of the files
If you got the binaries, this section can be skipped safely and you may proceed to Using the Pipeline. Inside the UVS_data_pipeline directory, files and folders are organized as follows:
- calib: Contains a sample flatfield file (fits) along with two ascii (text) files with flux calibration information.
- doc: One fairly outdated copy of documentation (pdf) may be found in this folder.
- jar: The most important directory of the distribution! All the java archive (jar) files of UVS pipeline modules, pipeline test modules and third party dependencies are packed in here. These third party softwares are:
- nom.tam.fits by Tom McGlynn for fits read/write operations
- fits WCS library for various coordinate conversions
- jfreechart for making plots and figures
- javadoc: Technical documentation generated by javadoc, useful for developers.
- params: All the init parameter files reside here. Note that all these are plain text files and very well documented within.
- scripts: The pipeline wrapper script (unix/linux shell script) is in here.
[edit] Building from the Source
Java source code distribution is available from TAUVEX website alongside the binary distribution. Building from source is a bit more involved than using binaries, since a java runtime Env. is not enough. We need java SDK (software development kit) version 1.4.0 or later. That is a bigger download from http://java.sun.com. There is another piece of software which makes life a lot simpler (and less interesting). Ironically, it was named "ant" (another neat tool). Ant is a new generation build tool (equivalent of old make) written in java and it reads xml. I have seen ant packed along with some versions of java SDK. If it is not there, you can get it from http://ant.apache.org. Here is the final briefing:
[edit] Pre-requisite
- Make sure you got the latest distribution of source code.
- Java SDK, version 1.4.0 or later. Always good to get the latest available at http://java.sun.com
- ant - The new age build tool from http://ant.apache.org. Any versions above 1.6 should be fine.
- Make sure that you can execute java and ant from command line. You need to add them into your $PATH (%PATH on windows). There are people who manage without setting PATH, but ant might give you screen full of instructions and complaints.
- And yes, we need about 12 MB of disk space.
[edit] Building with ant
Before we start, do this little test to make sure everything is ok. Try the following commands:
java -version ant -version
If you see things like:
java version "1.6.0_02" Java(TM) SE Runtime Environment (build 1.6.0_02-b05) Java HotSpot(TM) Client VM (build 1.6.0_02-b05, mixed mode) Apache Ant version 1.7.0 compiled on December 13 2006
We are ready for build. Unpack the source code distribution and change over to that directory. There should be a file called build.xml, which is the default input file for ant. Execute the following command to build your code.
ant
You will see ant compiling the modules one by one. Takes a while.. Tasks performed by ant are:
- Make a new directory called jar. Copy all the third party software from src/external to jar
- Make a new directory called build and subdirectories inside build for each module. The compiled java files (.class) of each module are stored in these folders.
- Once the .class files are ready, pack all files from individual modules into a java archive (jar) file and save them in jar folder.
- Make technical documentation using javadoc and save all the generated html files in javadoc.
Once that is done, you can cleanup the area by removing unwanted files. Listed below are some of the useful ant options:
| Ant argument | Purpose |
| ant clean | Removes the temperory folder build which contains the .class files. |
| ant distclean | Delete the folders jar, doc, javadoc useful if you want to restore the source distribution to the initial state. |
| ant cleanBuild | Do a distclean and then build again |
[edit] In the absence of ant
Quite a straight forward job. change over to the folder src. Build the packages one by one. Steps to be taken are listed below:
- Make a directory called jar and copy all the .jar files from src/external into jar. Define the path to each of those jar files in a variable called CLASSPATH. See the howto in java docs for details.
- Build the component called UVSUtil. Change over to src/UVSUtil and try the command javac *.java. You should end up with a .class file for every java file.
- Make a jar: A manual is available in java docs and move it to the folder called jar
- Add UVSUtil.jar to CLASSPATH and proceed to other modules, one by one.
[edit] Setting up the Software
There is nothing to be configured. Once you have all the jar files in place, software is ready to go. Using_the_Pipeline is a long description of running the modules and pipeline as a whole.
[edit] Parameter files
Parameter files are used to hold user defined input values for running pipeline modules, pre-pipeline modules and test modules. They are all plain text files and is available in the folder params in the UVS data pipeline distribution directory. All of them are documented well internally. It is a good practice to check their contents before running the pipeline. If you are unsure about any parameters, it is best to leave it with the default value. A summary of their contents and modules which use them are listed below.
- UVS_Pipeline_initparams.txt: The main parameter file. Hold several parameters which are used by the following modules:
- UVS_create_level1a
- UVS_validate_data
- UVS_calc_xy
- UVS_flat_field
- UVS_create_image
- UVS_Imgheader_keywords.txt: Holds TAUVEX specific keywords which will go into the primary image header of a fits files. Required for UVS_create_level1a.
- UVS_Btableheader_keywords.txt: Contains TAUVEX specific keywords for the fits binary table headers. UVS_create_level1a requires this file to generate level1a data. Do not modify these, unless you have a valid reason!
- UVS_Pipeline_calparams.txt: Lists the locations of available calibration file names. UVS_get_calfiles read this file to select the most appropriate calibration information (closest to obs. date in fits header). Do not modify these, unless you have a valid reason!
- UVS_Pipeline_distcoeff.txt: Contains the geometric distortion coefficients, required by UVS_geom_corr to correct for distortions.
- UVS_Pipeline_PSEinitparams.txt: Holds the required parameters for point source extraction. These values are required for UVS_register_data to identify point source positions and to align images taken over the duration of an observation.
