Interactive Processing of Fabry-Perot Data  using FP_PROC

This routine allows the user to process LWS FP data for AOTs  L03 and L04. For L03 observations an Automatic Least-Squares (ALS) algorithm developed by Gary Davis (see notes LWS/USK/N/2009.01 and LWS/USK/N/2009.01) is used. It allows the  grating position to be shifted until all the mini-scans line up with each other. For L04 observations only the Interactive shifting of the grating is possible, as the only criterion available to the  user is that the ends of a line are  flat, i.e. there should be  no slope on the 'continuum'.

For both L03 and L04 AOTs the user can now interactively optimise the dark current and straylight component. This is done by using the grating L01 observation of the same object, if available, and matching the two continua by adjusting the FP dark/straylight component until good agreement is achieved.

Data for L03/L04 AOTs  are processed from SPD to AAR stage. The results are written to ISAP compatible LSAN FITS files.

You can view the original header of the FP_PROC routine clicking here.

                                      Summary of command line arguments

As with other LIA routines, FP_PROC is invoked from the command line from the ISAP/LIA environment.

Typical example of usage:  fp_proc,tdt='44700574',dir='/mydisk/mydata/'
 

 Argument    I/O  Type        Description
 Name
 --------------------------------------------------------
   tdt                 I      string       the tdt number associated with the observation. For example if the tdt number is 44700574 then the routine will look for data files  l*44700574.fits.  If not specified the routine assumes that files to be read are called l*.fit (viz the SOC processed files  from the CD_ROM).

   dir                 I      string    the directory from which the input files are to be read. It can be a relative or absolute path. e.g.  dir='data/'. If not specified then the data are read from the user's curent working directory.

/newdarks              keyword   If set then no dark current is subtracted from the input SPD data. It is assumed that the dark has already been subtracted interactively.

/newabs                  keyword   If set the absolute response correction takes place using the correction factors stored in lgif file with the suffix'_newabs'. So, if the original lgif file is called lgif12345678.fits then fp_proc expects to read a file with the name lgif12345678_newabs.fits. Such an lgif file would normally be generated using the ia_abscorr interactive routine.

The following five command line arguments are intended for diagnostic purposes only. Typically a user would write the processing results to a FITS file and then continue the data reduction and analysis with ISAP.

 Argument    I/O  Type        Description
 Name
 --------------------------------------------------------
   ospd           O structure  IDL structure containing essentially the  SPD data but with ancillary information such as the extent for each mini-scan (for L03) or line (for L04), detector and FP used, etc.

   ostr            O structure  IDL structure. Each record in it contains averaged data for each mini-scan or line as generated by the standard pipeline.

   rstr            O structure  IDL structure. As for ostr but  contains unaveraged data for each mini-scan or line.

   nostr          O structure  IDL structure. Each record in it contains averaged data for each mini-scan or line but generated by shifting the grating and optimizing the dark current. For L03s a combination of an Automatic Least-Squares (ALS) shift and Interactive shift is possible. For L04s only the interactive  shift option is available.

   nrstr         O  structure  IDL structure. As for nostr but contains unaveraged data for each mini-scan or line.
 


                                                         Description of user interface
 

When the program is started up a two panel plot display widget appears on the screen. The plot windows are initially blank. For raster observations, the user is required to select the Raster Point ID to be processed BEFORE the main display widget comes up.

The display setup and processing steps are slightly different depending on whether the observation is L03 or L04.

   AOT L03:

(a) The title of the widget gives information about the observation. A row of textual information and buttons follows next, one row. For each unique combination of FP and detector. Normally only one or two such lines are expected. The first 4 text boxes describe the FP and detector in use, and the start and end numbers of mini-scans.
 

(b) The next button is the Processing ... button, which is at the heart of the all the FP processing. Clicking on this button reveals a pull-down menu with four options:

1) Standard Pipeline
2) Optimize Darks
3) Automatic Least-Squares (Grating shift)
4) Interactive (Grating shift)

At the start only option 1) is available to the user (the other three are desensitised), because all subsequent processing steps make use of these pipeline results. So the user must first select option 1). Selecting option 1) leads to processing of the SPD product through the interactive pipeline, producing AAR data (in ostr and rstr). Note that nominal darks are used here instead of the measured darks. The results are displayed in the top panel as Flux in W cm^-2 micron^-1 versus wavelength in microns.

Once the pipeline processing has ended the other two processing options also become active:


 

The user can now prepare the displayed data in the top panel for subsequent processing (ALS or Interactive) by zapping (discarding) sections of the data which are known to be glitch artefacts or just noisy data. This step is performed by clicking and dragging the right mouse button over the offending region. The user is asked to confirm whether zapping should go ahead. The results are re-displayed to the user after zapping (using a dot symbol for each data point). The following display shows a zap sequence in operation:

 Zooming on a selected area is also possible by clicking and dragging the left mouse button.

 IMPORTANT: You cannot zap a whole mini-scan!

The remaining 5 buttons in the top row are are as follows:
 


(c) The Optimize Darks Option: Selecting this option results in another widget display appearing:
 


 

Here the user is able to adjust the dark current/straylight component between the standard fixed darks as determined in dedicated calibration observations and the measured darks in the actual observation. A slider widget allows the adjustments to be made. The results are displayed as shown below with original spectrum in red and the adjusted spectrum in yellow. The user can specify a dark value by entering it in the text box labelled User Dark and then clicking on the Try Dark button:
 
 


 
 
 

Interactive dark current optimization is especially effective when a grating L01 observation for the object is also available. It is assumed that the user has used other LIA routines such as ia_dark, ia_drift and ia_abscorr (in consultation with an LIA expert) to reduce the L01 data. The user should click on the Overplot L01 LSAN data button to read and overplot the LSAN data.

The following display shows the original FP data (red),  the FP data after adjusting the background (yellow) and the scan-averaged  L01 detector data - in this case for detectors LW1 (blue), LW2 (pink) and LW3 (pale yellow).
 


 

d) The Automatic Least-Squares Option: Once this selection is made another two-plot display widget appears. The procedure is started up by clicking on the Start/Next Iteration button. The left plot shows a region of the pipeline processed spectrum of interest with the mini-scan of interest plotted in red and the adjacent mini-scans in green. The root-sum squared (RSS) statistic is plotted in the right plot as a function of grating shift - it is the RSS which is being minimised between the overlapping portions of the mini-scans.
 


 

After each iteration the user should examine the Current ,Maximum Shift to see if further iteration is necessary. Normally, as a rule of thumb, if the maximum shift in the current iteration is less than or equal to 10, then the solution can be accepted and no further iterations should be necessary. The user should then Exit from L03 ALS display and return to the main display widget. The results of of the ALS iterations are displayed in the lower panel of the main display widget. Clicking on the Make Prime button ensures that the plot is displayed in the top panel and the results written out to a FITS file.

The user can at this start the procedure of zapping 'bad' data points and running the ALS option again, or to run the Interactive option described below.

(d) The Interactive Option: The user may notice that the first and last mini-scans in the observation do not fit their adjacent mini-scans as well as those in the middle. This is because the mini-scans at the ends have only one adjacent mini-scan, and there is thus insufficient information to constrain the grating shift. In these cases the Interactive option is especially useful to tweak the appropriate mini-scans. This option can also be used for other mini-scans but due care must be taken to avoid introducing spurious features in the overlapping mini-scans. See also the L04 section below.

Upon selection of this option the Interactive Shifter display widget appears:
 


 

The left plot shows the mini-scan neighbourhood and the mini-scan to be shifted shown in red. The right plot shows a close-up of the mini-scan, also in red. A slider widget allows the grating position to be shifted between +/-250 in FP scan units (approx  +/- 1/4 grating resolution element).

As the shift is adjusted the affect on the mini-scan is shown in yellow in both the left and right plot windows. It is then left to the user to decide what the optimum grating shift is. The shift for the adjusted mini-scan be accepted or rejected by using the appropriate buttons, and the user scan switch to  the next mini-scan, previous mini-scan or a particular mini-scan as desired.

At the end of the procedure the user should Exit and return to the main FP widget. The results of of the Interactive processing are displayed in the lower panel of the main display widget. Clicking on the Make Prime button ensures that the plot is displayed in the top panel and the results written out to a FITS file.
 

  AOT L04:

(a) The title of the widget gives information about the observation as for L03, e.g. FPS and FPL Processing for TDT: 12345678 AOT: L04 Object: M40 South

A single row of 7 buttons follows next, the first being the Processing ... button. Clicking it reveals three options:

1) Standard Pipeline
2) Optimize Darks
3) Interactive (Grating shift)

Initially only the Standard Pipeline option is available, as is the case for L03 observations, because all subsequent processing steps make use of  these pipeline results. Once the pipeline processing has been completed the results are displayed in the top panel for ALL the spectral lines observed in the AOT. Using the zoom feature (i.e. by clicking and dragging the left mouse button over the desired region)  will allow lines to be inspected closely. The Select line to display button brings up a pulldown menu from which the line number of interest can be selected  and displayed individually.
 
 

The user can now prepare the displayed data in the top panel for subsequent processing (Interactive or Optimize Darks) by zapping (discarding) sections of the data which are known to be glitch artefacts or noisy data. This step is performed by clicking and dragging the right mouse button over the offending region. The user is asked to confirm whether zapping should go ahead. The results are re-displayed to the user after zapping (using a dot symbol for each data point). The following display shows a zap sequence in operation:
 
 


 
 
 

 IMPORTANT: You cannot zap a whole spectral line!

The remaining 5 buttons in the top row are are as follows:
 

(b) The Optimize Darks Option: Selecting this option results in another widget display appearing:


 

The first line in the AOT is displayed and  the user allowed to enter a dark /straylight value anywhere between the standard fixed dark and the measured dark appropriate  for the detector on which the line is observed. The results are displayed as shown below with original spectrum in red and the adjusted spectrum in yellow. The user can specify a dark value by entering it in the text box labelled User Dark and then clicking on  the Try Dark button:
 


 

Interactive dark current optimization is especially effective when grating L01 observation for the object is also available. It is assumed that the user has used other LIA routines such as ia_dark, ia_drift and ia_abscorr (in consultation with an LIA expert) to reduce the L01 data. The user should click on the Overplot L01 LSAN data button to read and overplot the LSAN data.

The following display shows the original FP data (red),  the FP data after adjusting the background (yellow) and the scan-averaged  L01 detector data - in this case for detector SW3 (blue).  It is recommended that the user only enter a dark value between the fixed and measured ranges.The example below therefore is not realistic!
 


 
 

(c) The Interactive (Grating shift) Option: Only the Interactive option is available for adjusting the grating position to achieve a flat continuum on either side of each spectral line. Upon selection of this option the Interactive Shifter display widget appears:
 


 

The original  line is shown in red and the yellow line is the result of shifting the grating  by 50 units.  In the example shown there is very little shift necessary to obtain a flat continuum, but if a line is highly asymmetric then it may be necessary to adjust the grating position more carefully over the prescribed range of +/- 250 FP scan units (approx +/- 1/4 grating resolution element).  It is then left to the user to decide what the optimum grating shift is. The shift for the adjusted  line be accepted or rejected by using the appropriate buttons, and the user scan switch to  the next  line, previous line or a particular line as desired.

At the end of the procedure the user should Exit and return to the main FP widget. The results of of the Interactive processing are displayed in the lower panel of the main display widget. Clicking on the Make Prime button ensures that the plot is displayed in the top panel and the results can be written to a FITS file after clicking on the Write Fits button.

Document prepared by Sunil Sidher, April 1999.