This is your calibrated spectrum; not very informative at this stage,
but we can note that the noise of the spectrum is higher in the region
with lambda < 93um, corresponding to the SW detectors. There is nothing
wrong with this observation; in fact, this is common for the LWS
and it is due to the combined effect of higher NEPs and narrower
spectral element size for the SW detectors. Let's make a couple of checks
for memory effects and detector intercalibration:
The green and the blue lines represent the two scan directions; you
can zoom and closely inspect different portions of your spectrum. For faint
sources you should not see any systematic differences among the green and
blue lines (for the same detectors); if you instead see a significant difference
(well above the noise of the green and blue lines) consider that the effect
has not been modelled, so that it is not possible to correct for it. In
case a line is present it is strongly advisable to keep the two scan directions
separate and estimate the line parameters separately; the two line fluxes
can be averaged later, but the two determinations will give you the best
estimate of the uncertainty to assign to the average line flux.
In principle, spectral segments from adjacent detectors, and
segments from a single detector should line up nicely; the overlapping
portions of the spectra should match both in shape and in absolute value,
but this generally does not happens e.g., at 128 microns above. What are
the possible reasons for a mismatch ?
Now you should quantify a bit this mismatch. Generally speaking you might be happy with a 10-15% mismatch, but this of course depends on the source. If the source is pointlike, you should expect less mismatch because (see above) the primary calibrator for the LWS is also pointlike and the beam is stable at all wavelengths. Examine your spectrum and take notes about which detectors are more critical in the aspects we have just discussed.
There are 4 steps in the LIA reprocessing of an LWS02 AOT:
While at the ISAP> prompt, type: IA_DARK, tdt='TDT', where TDT is the eight digit number attached to the filename of all you data files. If you are running ISAP in a directory which is different from the one where your data files are stored, an additional parameter has to be given; the call would then be: IA_DARK,tdt='TDT',dir='your directory'. Once the widget is up, click on detector such as LW1. You will see something like this:
The red crosses are the DC measurements, while the white points are your (uncalibrated) data; everything is plotted as a function of time. If your observations was taken at a revolution number earlier than about 400 you may see DC measurements (red crosses) taken in between the observation; however only the first and the last (before and after your observation) DC measurements can actually be used.
Here the various scans of the grating can be recognized as an oscillating pattern on your data. The discontinuities in the oscillation segments above represent the different "lines" scanned in the execution of the LWS02 AOT. Remember that this data is still uncalibrated at this stage and the transmission profiles (called RSRF, 1 with the appropriate RSRF segment or each line/detector combination) of the instrument are still to be divided out. To check where the different lines and cans start and end, go with the mouse on a point and click the middle button of the mouse: the text area at the bottom of the IA_DARK widget will give you all the information regarding the nearest point to the mouse actual position, e.g.:
Raster ID is 1 1 - Line # 0 - Scan # 0 - ITK = 228327855 - Phc = 8.64105e-16 Amps
Again, it is important to remember that at this stage the instrumental transmission is not yet divided out and, if there is sufficient flux from your source, it will manifest itself as a periodic pattern throughout your dataset. It is important to notice that that glitches are present also during the dark current measurements; if you bring up the second DC measurement you will see this:
You will note the gap between the first and second point here - a glitch occurred and was removed possibly leaving a trailing ramp. The median-clipped averaged performed on the whole set of points above gives the DC estimated made by the OLP, which is visible in the box 'OLP Used...' in Fig. 6; it is clear however that this may be an overestimate because at least the point after the gap is artificially high. (This is a fairly mild example of this effect. ) It seems that the best estimate should be based on all but the second point, which we mask out and go ahead and make the DC estimate without it.
When you come to the subtraction of your estimated DCs from the data, a lot of common sense is needed. Take the example in Fig. 7.
The problem here is that the source flux (white) is weaker than the first dark current measurement (first set of red points, with the mean shown as a red line). Yet the source flux includes dark current. This indicates that the first dark current measurement is measuring more dark current than is present during the observation and is not reliable. One possible choice is to rely solely on the second dark, which is weaker than the source flux (red points at the bottom) or else give up on making the correct dark subtraction and instead remove all the dark current and continuum by interpolating a "dark" function following the lower envelope of your data. However, note here how the glitches (SW1 has the worst) are altering the detector gain in time and causing steep falloffs. Compounding the problems here is the fact that their are discontinuities caused by switching to the different line wavelengths. For multi-line LWS02 observations, when dark measurements are unreliable, systematic changes in dark current and gain are difficult to untangle from intrinsic line and continuum fluxes associated with the successively observed lines.
Likewise, with an LWS02 Raster Map , signal variation is bound to occur during the observation due to the source's spatial structure, and between the starting and ending dark observations, it is practically impossible to isolate a systematic trend in the instrument behaviour .
Consider the ia_dark result for the LW1 detector for the same observation...
You should by now be convinced that the oscillations you see in the signal (white points) in Fig. 7a is due to spectral scanning back and forth over the line region interval and is dominated by the detector's relative spectral response (which has not yet been divided-out) - and the oscillation is seen here in 5 different segments, one for each line observed. The oscillations will disappear when we process our spectrum to the AAR stage. The best (and most conservative) thing you want to do at this stage is to assume that the DC is constant through out the observation. In general, if the DC measurements done before and after the observation are comparable (as with the example above), within the noise of the single measurements, we recommend you choose the linear interpolation option which uses the average of the two measurements. This is the option applied by the OLP, and unless the two measurements are significantly different or there is a clear DC trend in your data, you should stick with that option.
In general, the advanced options of ia_dark; the option to add
points and to fit with polynomial functions, are not advised for LWS02....
typically, there is insufficient data to be able to decide to take such
actions. The exceptional case would be where a line was observed by scanning
so many times that an instrumental trend was visible. In such a case,
the user is advised to distinguish between
dark variations, which elevate or lower the mean signal strength, leaving peak to peak variations constant, and responsively trends, which change the span of peak to peak variations range and cause line fluxes obtained in individual scans to change. If your data are of sufficient quality (i.e., you can see the effect described below) then here are the corrective actions:
While there may be responsively drifts within the period of time that an LWS02 observation is taken, at this time there is no way to isolate and correct these drifts. However, absolute responsively changes (a constant detector gain change from the nominal) may be measured and corrected as described below.
Responsivity Correction Factors
The absolute responsively correction factors can revised using the routine IA_ABSCORR, whose tutorial is available at http://www.ipac.caltech.edu/iso/lia/abs_corr.html. Since the estimate of these factors is based on the Illuminator Flash data, the characteristics of the source observed are not important; the tutorial has all the information you need to proceed.
Do not start zapping in the noise; you should only zap what's really systematic otherwise you will modify the statistics of the measurement and the averaged spectrum will be perturbed.
Do this zapping for all detectors; remember to STORE the AAR after finishing zapping each detector. If you SPLIT apart your AAR, it is time now to merge all the pieces back into a single AAR.
Here are some guidelines for averaging. If you have a limited number of scans you may want to increase the bin size to gain a bit in S/N at the expense of the spectral resolution. Consider that unless you are observing, e.g., a supernova remnant where the line can be broader than 1500 km/s, your line will be unresolved and there is nothing band in increasing a bit the bin size to push down a bit the noise. Play also a bit with the sigma clipping threshold; a too high value will not discard enough outliers, while a too low value will start discarding too many points; there should be an optimum value which minimizes the noise level of the averaged spectrum. We cannot give more precise guidelines here because this is a function of the data. Also, the bin size and sigma values which give good results for one detector may be not optimized for another detector; if this is the case you can always send separate detectors to Average and merge the results back into a single AAR later (remember to STORE each averaged piece before you go to the next). The averaging algorithm is another parameter you can play with; the Mean and Median methods do not clip outliers and are not recommended. The Standard Clip & Mean does one single pass to discard outliers, while the Special Clip & Mean is more drastic and does iterative passes until there is nothing more to discard. Essentially play with the bin size, sigma and the averaging method until you are happy with your result; the precise combination of these three parameters will be a function of your data.
Q. "This is all very nice. But my source is not
infinitely extended and it not uniformly bright!! What should I do ?"
A. This is of course a most tricky and common situation, but unfortunately there is no definite indication; the truth will be in the middle. However, from tests done on real cases (extended galaxies, hence non uniform) it seems that the correction works well for sources more extended than 3-4 arcmin.