Running the Pipeline

From Tauwiki

Contents 1 Simulation Data 1.1 Telemetry Information 2 General Notes 3 Steps in Running the Pipeline 4 create_level0_data 4.1 Purpose 4.2 Command Line 4.3 Parameters 4.4 Notes 5 UVS_ingest_data 5.1 Purpose 5.2 Command Line 5.3 Parameters 5.4 Notes 6 UVS_create_level1a 6.1 Purpose 6.2 Command Line 6.3 Parameters 6.4 Notes 7 UVS_calc_xy.jar 7.1 Purpose 7.2 Command Line 7.3 Parameters 7.4 Notes 8 UVS_get_calfiles 8.1 Purpose 8.2 Command Line 8.3 Parameters 8.4 Notes 9 UVS_apply_cal 9.1 Purpose 9.2 Command Line 9.3 Parameters 9.4 Notes 10 UVS_flat_field 10.1 Purpose 10.2 Command Line 10.3 Parameters 10.4 Notes 11 UVS_geom_corr 11.1 Purpose 11.2 Command Line 11.3 Parameters 11.4 Notes 12 UVS_calc_radec 12.1 Purpose 12.2 Command Line 12.3 Parameters 12.4 Notes 13 UVS_register_data 13.1 Purpose 13.2 Command Line 13.3 Parameters 13.4 Notes 14 UVS_create_image 14.1 Purpose 14.2 Command Line 14.3 Parameters 14.4 Notes

[edit] Simulation Data

If simulation data are needed then start with the Online Simulator. The parameters can be changed as needed. Choose the ascii file option and download and uncompress the data in the required directory. The simulation data should then be converted into the Level0 data required by the pipeline using the procedure outlined below in [CREATE_LEVEL0_DATA].

[edit] Telemetry Information

Telemetry comes from the spacecraft and is necessary f The pipeline expects telemetry information in a different file. This file will be created by create_level_0 but needs a telemetry file. This file will be created from the parameter file from the simulation program using [sim_telemetry].

[edit] General Notes

The programs require and are built on java 1.4.1. I am using

java version "1.5.0_07" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_07-164) Java HotSpot(TM) Client VM (build 1.5.0_07-87, mixed mode, sharing)

Either the parameter file must be in the running directory or in a subdirectory called params.

[edit] Steps in Running the Pipeline

java -jar $UVS_DIR/UVS_ingest_data.jar level0_data science_data telemetry_data

java -jar $UVS_DIR/UVS_create_level1a.jar science_data telemetry_data root

java -jar $UVS_DIR/UVS_calc_xy.jar root_T1_1a.fits root_T1_1b.fits

java -jar $UVS_DIR/UVS_apply_cal.jar root_T1_1b.fits root_T1_1c.fits

java -jar $UVS_DIR/UVS_flat_field.jar root_T1_1c.fits root_T1_1d.fits

java -jar $UVS_DIR/UVS_geom_corr.jar root_T1_1d.fits root_T1_1e.fits

java -jar $UVS_DIR/UVS_calc_radec.jar root_T1_1e.fits root_T1_1f.fits

java -jar $UVS_DIR/UVS_register_data.jar root_T1_1f.fits root_T1_1g.fits

java -Xmx256M -jar $UVS_DIR/UVS_create_image.jar root_T1_1g.fits image.fits

[edit] create_level0_data

[edit] Purpose

The simulator produces ASCII data which must be converted into Level 0 data (see TAUVEX Data Levels). In order to do this start log file, end log file, and telemetry file are needed. The telemetry file can be created by sim_telemtery.c program, which can be downloaded from here. The sample log files and telemetry may be downloaded from here.

[edit] Command Line

java -jar $UVS_DIR/create_level0_data.jar ascii_data startlog.txt endlog.txt telemetry.txt level0_data

[edit] Parameters

Input Parameters

Parameter	Description
ascii_data:	Output of the simulation program.
startlog.txt:	Parameters in StartLog.
endlog.txt:	Parameters in EndLog.
telemetry.txt:	Output of telemetry_sim.

Output Parameters

Parameter	Description
level0:	Input to pipeline (from spacecraft)

[edit] Notes

This step will not be needed when actual TAUVEX data is obtained.

[edit] UVS_ingest_data

[edit] Purpose

This program separates the single file we get from Mission into telemetry and data files.

[edit] Command Line

java -jar $UVS_DIR/UVS_ingest_data.jar level0_data science_data telemetry_data

[edit] Parameters

   level0_data: File that comes from Mission Ops (or from create_level0)
   science_data: Science data file in same format as TAUVEX spacecraft data.
   telemetry_data: Contains telemetry data from Mission Ops.

[edit] Notes

I don't know if fillers can be in the data or not.

[edit] UVS_create_level1a

[edit] Purpose

This program will read TAUVEX data and create a Fits Binary Table. Changes in the input data will not propagate to later modules.

[edit] Command Line

java -jar $UVS_DIR/UVS_create_level1a.jar science_data0 telemetry_data root

[edit] Parameters

   science_data: Output of UVS_ingest
   telemetry_data: Output of UVS_ingest
   root: Root name (plus _) to be prepended to data files.

[edit] Notes

Three files will be created:

  root_T1_1a.fits
  root_T2_1a.fits
  root_T3_1a.fits

[edit] UVS_calc_xy.jar

[edit] Purpose

The spacecraft data comes down as W, S, Z. This program converts these to X,Y.

[edit] Command Line

java -jar $UVS_DIR/UVS_calc_xy.jar root_T1_1a.fits root_T1_1b.fits

[edit] Parameters

   root_T1_1_1a.fits: Output of UVS_create_level1a
   root_T1_1_1b.fits: Added x,y columns (Levelb data)

[edit] Notes

From here on the flow splits into different streams for each telescope.

[edit] UVS_get_calfiles

[edit] Purpose

This module gets the locations of the calibration files.

[edit] Command Line

java -jar UVS_get_calfiles.jar root_T1_1_1b.fits

[edit] Parameters

   root_T1_1_1b,fits: Output of UVS_calc_xy

[edit] Notes

The header values are updated in the input file.

Calibration files must exist.

[edit] UVS_apply_cal

[edit] Purpose

This module actually applies the calibration to the Level 1b data.

[edit] Command Line

java -jar UVS_apply_cal.jar root_T1_1_1b.fits root_T1_1_1c.fits

[edit] Parameters

   root_T1_1_1b.fits: Output of UVS_get_calfiles
   root_T1_1_1c.fits: Calibrated file (Level 1c)

[edit] Notes

The original data is still carried through.

[edit] UVS_flat_field

[edit] Purpose

To apply a flat field to the data.

[edit] Command Line

java -jar UVS_flat_field.jar root_T1_1_1c.fits root_T1_1_1d.fits

[edit] Parameters

root_T1_1_1c.fits: Output of UVS_apply_cal. root_T1_1_1d.fits: Flat fielded data.

[edit] Notes

Currently a unit flat field is applied.

[edit] UVS_geom_corr

[edit] Purpose

Correct for geometric distortion in the detector.

[edit] Command Line

java -jar UVS_geom_corr.jar root_T1_1_1d.fits root_T1_1_1e.fits

[edit] Parameters

  root_T1_1_1d.fits: Output of flat file.
  root_T1_1_1e.fits: File with corrected x and y coordinates.

[edit] Notes

The geometric correction is a 49 element correction in x and y

x_prime = sum(aij * x^i * y^j) y_prime - sum(bij * x^i * y^j)

[edit] UVS_calc_radec

[edit] Purpose

Adds coordinate information to each photon based on the nominal header.

[edit] Command Line

java -jar UVS_calc_radec.jar root_T1_1_1e.fits root_T1_1_1f.fits

[edit] Parameters

   root_T1_1_1e.fits: Output of UVS_geom_corr
   root_T1_1_1f.fits: Coordinate information added.

[edit] Notes

The implicit precision of the ra and dec is 0.1 degree.

[edit] UVS_register_data

[edit] Purpose

Shifts the frames to correct for image motion.

[edit] Command Line

java -jar UVS_register_data.jar root_T1_1_1f.fits root_T1_1_1g.fits

[edit] Parameters

   root_T1_1_1f.fits: Output of UVS_calc_radec.
   root_T1_1_1g.fits: Corrected for image motion.

[edit] Notes

[edit] UVS_create_image

[edit] Purpose

Takes level1g as the input and creates an image.

[edit] Command Line

java -jar UVS_create_image.jar root_T1_1_1g.fits image.fits

[edit] Parameters

   root_T1_1_1g.fits: Output of UVS_register_image.
   image.fits: Image file with coadded data.

[edit] Notes

There is still an uncertainty in the data of 0.1 degree.