CAFE Software Tools

CAFE Reduction Pipeline

CAFE Reduction Pipeline
Quick Guide for the CAFE Pipeline Example of the reduced data (17/06/2011) Installation instructions (to use outside CAHA) Installation just the CAFE pipeline (Requires R3D) Installation R3D, Perl version Installation R3D, C version
Quick on-line data reduction: We have created a quick & dirty on-line data reduction based on R3D and E3D to allow the user to visualize the observed object. Detail information of how it works can be found in the Report on the Commissioning Run To use it, we created an account with the user r3duser in the r3dpipe computer. So you need to connect to this computer, and transfer your data from rike to it. ssh -X r3duser@r3dpipe (ask the password to the CAHA staff) Change to the directory: cd /disk-a/r3duser create there the working directory: mkdir cafe_110821 cd cafe_060821 and copy from cafe computer your data there (with scp). The script requires at minimun the raw data to reduce, a continuum exposure and an arc exposure. It assumes a set of master files for the bias, flat-field and flux calibration. Once you have your data in the correct directory, you just need to run the script: R3D_cafe.shift.new.pl SCIENCE.fits CONTINUUM.fits ARC.FITS Where the SCIENCE.fits corresponds to the frame that we want to reduce. CONTINUUM.fits corresponds to an continuum-lamp frame, and ARC.fits corresponds to the ThAr-lamp frame. The result of the reduction is a RSS-file named ARC.disp_cor.fits, with each row corresponding to the spectrum comming from a particular order of the spectrograph. All the science frames in a particular night can be reduced using the auto_redu_CAFE.pl script, which automatically reduce all the frames within a particular directory. To visualize the data, you can use the spec2D_CAFE_plot.pl and spec2D_CAFE_plot_all.pl scripts on the same computer. To derive the S/N of the reduced frames you should use the following versions of the script:R3D_cafe_SN.pl, auto_redu_CAFE_SN.pl and spec2D_CAFE_plot_SN.pl. This reduction creates two different files, one corresponding to the reduced data (.disp_cor.fits), and another corresponding to the S/N spectra (.SN.fits) TBD: Create a 1D version of the full spectra, gluing all the orders. The typical output of the reduction of a frame can bee seen here. Installing the software (outside CAHA) A example of the 2D frames created by the pipeline has been uploaded in the CAHA ftp. These data corresponds to the nights of the 17/06/2011. In this directory it is possible to find fully reduced frames (.disp_cor.fits), for both the observed objects along this night and the corresponding S/N spectra (.SN.fits). A reduced ARC frames is also included (arc.disp_cor.fits). Installing the software (outside CAHA) To install the software you require to install first the R3D package, C and Perl versions, and then, download and install the CAFE pipeline, by the following steps: Download, Requirements and Installation (R3D Perl Version) The last stable Perl version of R3D can be download here. R3D Perl version requires the following perl modules to be installed in your computer: Statistics::OLS Math::FFT Math::Stat Math::Spline qw(spline linsearch binsearch) Math::Derivative qw(Derivative2) Math::Approx Astro::FITS::CFITSIO qw( :longnames :constants ) PGPLOT PDL All this modules can be found at CPAN. It is recomended the lastest possible version of PDL. Once installed all these modules, and assuming that you have untar the files at the directory DIR, you should run the following script: ./change_line.pl /home/sanchez/sda1/perl/MY/my.pl DIR/my.pl You can test the scripts by running: ./test_r3d.pl If you do not get any error or warning message you have suscessfully installed R3D. Then, add the directory DIR to your PATH. If you find any problem, please send an email to S.F.Sanchez Download, Requirements and Installation (R3D, C Version) The last stable C version of R3D can be download here. It required the Lyon-C library, and PGPLOT with Tcl/tk (basically the same requirements than E3D. To install it: For binary releases: 1. Download the binary release for your platform into a directory of your choice: (a) Linux 64bits available (b)Linux 32bits available 2. Unpack the distribution using e.g. tar xvzf R3D_c_linux.gz, this will create a directory R3D containing the R3D programs. 3. Add the new directory to your PATH. For source code tarballs: 1. Download the last stable version of the R3D distribution (e.g. r3d-0.1.tar.gz) from the list of Downloads and copy it into the Euro3D directory (given by the environment variable IFU_PATH, read the preinstalation requeriments). 2. Extracting the archive via tar xvfz r3d-0.1.tar.gz under the $IFU_PATH path will create the directory e3d-1.3a or a newer version number respectively. 3. Change into the created directory and execute the configure and make scripts: r3d> ./configure r3d> make Attention: The make can fail due to a different definition of certain libraries. We have tested the procedure under the two main Linux distributions, RedHat (7.1 and 8.0) and Suse (8.2). They differ in the definition of TCL/TK libraries. You should edit the file add_defs/makedefs.local to select the proper definition of V3D_LIBS for your distribution and add your E3D path to the declaration of the chosen (#HOME) V3D_LIBS as well as to V3D_INC. 4. Add the directory user/bin of r3d to your PATH, e.g. for csh add in .login or .cshrc: setenv PATH=${PATH}:${IFU_PATH}/r3d/user/bin NOTE The command line programs require similar inputs and produce similar outputs in both versions. R3D C version requires to have installed PGPLOT with Tcl/TK Download, Requirements and Installation (CAFEPipe, Perl Version) As indicated before, you need to install R3D, prior to run the pipeline of CAFE. Then, download the following TAR-ball file, select a certain directory to install it (eg., `DIR` and uncompress it. The tar file will create a new directory named `CAFE`. To run the pipeline, you need to add this new directory to your PATH, e.g., setenv PATH=${PATH}:DIR/CAFE To test that everything is ok, just run one of the pipeline commands, e.g., `R3D_cafe.pl`. If you get no error messages, then the pipeline is operative.

Quick Guide for the CAFE Pipeline
Example of the reduced data (17/06/2011)
Installation instructions (to use outside CAHA)

Quick on-line data reduction:

We have created a quick & dirty on-line data reduction based on R3D and E3D to allow the user to visualize the observed object.
Detail information of how it works can be found in the Report on the Commissioning Run
To use it, we created an account with the user r3duser in the r3dpipe computer. So you need to connect to this computer, and transfer your data from rike to it.

ssh -X r3duser@r3dpipe
(ask the password to the CAHA staff)

Change to the directory:

cd /disk-a/r3duser

create there the working directory:

mkdir cafe_110821
cd cafe_060821

and copy from cafe computer your data there (with scp).
The script requires at minimun the raw data to reduce, a continuum exposure and an arc exposure. It assumes a set of master files for the bias, flat-field and flux calibration. Once you have your data in the correct directory, you just need to run the script:

R3D_cafe.shift.new.pl SCIENCE.fits CONTINUUM.fits ARC.FITS

Where the SCIENCE.fits corresponds to the frame that we want to reduce. CONTINUUM.fits corresponds to an continuum-lamp frame, and ARC.fits corresponds to the ThAr-lamp frame. The result of the reduction is a RSS-file named ARC.disp_cor.fits, with each row corresponding to the spectrum comming from a particular order of the spectrograph. All the science frames in a particular night can be reduced using the auto_redu_CAFE.pl script, which automatically reduce all the frames within a particular directory.

To visualize the data, you can use the spec2D_CAFE_plot.pl and spec2D_CAFE_plot_all.pl scripts on the same computer.

To derive the S/N of the reduced frames you should use the following versions of the script:R3D_cafe_SN.pl, auto_redu_CAFE_SN.pl and spec2D_CAFE_plot_SN.pl. This reduction creates two different files, one corresponding to the reduced data (*.disp_cor.fits), and another corresponding to the S/N spectra (*.SN.fits)

TBD: Create a 1D version of the full spectra, gluing all the orders.

The typical output of the reduction of a frame can bee seen here.

Installing the software (outside CAHA)

A example of the 2D frames created by the pipeline has been uploaded in the CAHA ftp. These data corresponds to the nights of the 17/06/2011.

In this directory it is possible to find fully reduced frames (*.disp_cor.fits), for both the observed objects along this night and the corresponding S/N spectra (*.SN.fits).

A reduced ARC frames is also included (arc.disp_cor.fits).

Installing the software (outside CAHA)

To install the software you require to install first the R3D package, C and Perl versions, and then, download and install the CAFE pipeline, by the following steps:

Download, Requirements and Installation (R3D Perl Version)
The last stable Perl version of R3D can be download here.

R3D Perl version requires the following perl modules to be installed in your computer:
Statistics::OLS
Math::FFT
Math::Stat
Math::Spline qw(spline linsearch binsearch)
Math::Derivative qw(Derivative2)
Math::Approx
Astro::FITS::CFITSIO qw( :longnames :constants )
PGPLOT
PDL
All this modules can be found at CPAN. It is recomended the lastest possible version of PDL.
Once installed all these modules, and assuming that you have untar the files at the directory DIR, you should run the following script:
```
./change_line.pl /home/sanchez/sda1/perl/MY/my.pl DIR/my.pl
```
You can test the scripts by running:
```
./test_r3d.pl
```
If you do not get any error or warning message you have suscessfully installed R3D. Then, add the directory DIR to your PATH. If you find any problem, please send an email to S.F.Sanchez

Download, Requirements and Installation (R3D, C Version)
The last stable C version of R3D can be download here.
It required the Lyon-C library, and PGPLOT with Tcl/tk (basically the same requirements than E3D. To install it:
For binary releases:
1. Download the binary release for your platform into a directory of your choice: (a) Linux 64bits available
(b)Linux 32bits available
2. Unpack the distribution using e.g. tar xvzf R3D_c_linux.gz, this will create a directory R3D containing the R3D programs.
3. Add the new directory to your PATH.
For source code tarballs:
1. Download the last stable version of the R3D distribution (e.g. r3d-0.1.tar.gz) from the list of Downloads and copy it into the Euro3D directory (given by the environment variable IFU_PATH, read the preinstalation requeriments).
2. Extracting the archive via tar xvfz r3d-0.1.tar.gz under the $IFU_PATH path will create the directory e3d-1.3a or a newer version number respectively.
3. Change into the created directory and execute the configure and make scripts:
r3d> ./configure
r3d> make

Attention:
The make can fail due to a different definition of certain libraries. We have tested the procedure under the two main Linux distributions, RedHat (7.1 and 8.0) and Suse (8.2). They differ in the definition of TCL/TK libraries. You should edit the file add_defs/makedefs.local to select the proper definition of V3D_LIBS for your distribution and add your E3D path to the declaration of the chosen (#HOME) V3D_LIBS as well as to V3D_INC.
4. Add the directory user/bin of r3d to your PATH, e.g. for csh add in .login or .cshrc:
setenv PATH=${PATH}:${IFU_PATH}/r3d/user/bin
NOTE The command line programs require similar inputs and produce similar outputs in both versions.

R3D C version requires to have installed PGPLOT with Tcl/TK

Download, Requirements and Installation (CAFEPipe, Perl Version)
As indicated before, you need to install R3D, prior to run the pipeline of CAFE.
Then, download the following TAR-ball file, select a certain directory to install it (eg., DIR and uncompress it. The tar file will create a new directory named CAFE.
To run the pipeline, you need to add this new directory to your PATH, e.g.,
setenv PATH=${PATH}:DIR/CAFE
To test that everything is ok, just run one of the pipeline commands, e.g., R3D_cafe.pl. If you get no error messages, then the pipeline is operative.