Derive-SPDs input is Edited Raw Data (ERD ), 24 Hz data containing detector readout voltages of the target's flux, of internal flux standards, of wavelength calibration exposures, of dark currents and of housekeeping data. For each detector, all data for one full reset interval are combined to a single current estimate at single wavelength and are then saved in the Standard Processed Data (SPD ) file for that AOT . No attempt is made within Derive-SPD to average measurements of the same type (e.g. dark current measurements) or to do processing on any time scale longer than one reset interval.
A basic overview of the Derive-SPD processing is given in figure 8.1 and the ERD and SPD files are described in sections 9.2.1 and 9.3.1.
Figure 8.1: SWS Derive-SPD process.
Each 1 second measurement of a given AOT is saved separately inside the same SPD product in chronological order of acquisition. The motivation for this is to offer to the observer the possibility of judging the stability and repeatability of SWS measurements. There is an one-to-one correspondence between AOT's and SPD files. That is, one AOT produces one SPD file.
The individual processing steps are described as follows:
Data is read in for one reset interval (SW and/or LW reset interval). As discussed above, each AOT generates different types of data: astronomical measurements from the source, measurements from the spectral reference sources (grating and FP), measurements with the calibrators and dark currents. Each of these different types of data has a different physical meaning and serves a different purpose. Hence, the data must be flagged according to their type.
Each ERD data record is tagged with a Instrument Time Key (ITK) which points toward a unique record in the Compact Status History file (CSH ), called on the CD-ROM the SSTA file. Cross-correlating the ERD ITK with the CSH, Executed Observation History per ICS (EOHI) and General Housekeeping (GEHK ) files allows an unambiguous determination of the status of the SWS instrument at the time of the measurement and therefore of the type of the data recorded and the actual wavelength range covered (to be verified during the wavelength check stages).
This marks data affected by reset pulses or data out of limits. To do this it uses Cal-G files 3 and 4 .
The detector readouts voltages of between -10 and +10 V are converted into a bit number ranging from 0 to 4095. The midbit is the bit value that corresponds to 0 Volts, approximately 2048, and is different for each detector. This data is stored in Cal-G file 2A .
This module was only active for OLP Version 4.3. During testing, unusual results were obtained that have not been explained yet. Therefore from OLP V5.0 onwards Cal-G file 2A has been filled with neutral values exactly half of 4095, i.e. 2047.50.
Cal-G 2B contains information describing the shape of the reset pulse. This calfile is used to subtract the effect of the reset pulse from the data. Cal-G file 5 is also used in this.
This module was only active for OLP Version 4.3. During testing, unusual results were obtained that have not been explained yet. Therefore from OLP V5.0 onwards Cal-G file 2B has been filled with zeros that result in this module doing nothing.
The amplifier chain contains a high pass filter. For a constant voltage gradient input the response of the amplifier is to asymptotically approach a constant value, i.e.
where V(t) is the output voltage at time t, 
the input voltage
applied on the capacitance due to the accumulation of charges on the detector,
and 
the detector time constant. In order to later reconstruct the true
photo-current, it is first necessary to linearize the ramps, i.e. determine
the actual input voltages 
applied on the detector.
Knowing the RC time constant , it is straightforward to invert the above
formula and derive the input voltage. Each detector i has its own time
constant 
, stored in Cal-G 2 .
Due to parasitic capacitance between neighbouring detectors any (strong) signal in one detector will leak to other detectors in the same array.
Assuming the detector response is a linear function of the intensity of the signal, a set of cross-talk matrices can be determined with constant correction factors. The read-outs D of each individual detector j in the detector array may then be corrected by applying the following formula:
The sum is over all detectors of index i different from j, where i goes from 1 to M. M being the number of detectors in the array (12 for the gratings and 2 for the FP). This cross-talk correction matrix is held in Cal-G file 1 .
Assuming an error matrix 
on the individual values of F 
can
be derived at the same time as the cross-talk matrix, it is then straightforward
to propagate these uncertainties.
where the summation has the same meaning as in the previous formula.
These errors are not calculated in the generation of the array 
and hence their effects on D are not calculated. Also, if the dependance
of on signal intensity is not linear equation 8.2 will not
correctly describe the necessary correction and so any errors will be larger
than given by equation 8.3.
Glitches (ref section 5.4) are recognized in the
differentiated voltages. The median of the 24 Hz bit values is computed and
used, in conjunction with a value 
held in Cal-G 6 , to
define the size of a glitch. If any differentiated 24 Hz bit values are higher
than this they are marked as being affected by a glitch. If the glitch is so
strong that the detector gets saturated all the measurements which follow may
have to be discarded up to the end of the ramp. If the glitch is not too strong
the following measurements do not have to be discarded. However, the offset of
the immediately following ramp will not match that of the ramp which preceded
the glitch (for a definition of offset see sections 8.3.8 and
8.2.9).
The SWS Glitch History (SWGH ) file contains a list of all glitches found during processing.
During a period of enhanced background the glitch frequency may be very high. An attempt could be made at reconstructing the individual measurements disturbed by glitches. This last case however, is not considered in the base-line design of the offline processing software.
The accuracy depends on how well a glitch can be identified. This in turn depends on the strength of the glitch and on the value of the rejection threshold which is related to the precision with which the noise pattern of the SWS detectors can be documented. As experience is gained from ground testing and during the mission, fine tuning of the threshold may be necessary.
Any rest interval disturbed by three or more glitches is thrown away as unusable.
Information of glitches in stored in Cal-G file 6 .
This computes the grating and FP positions.
A least-square fit is applied to the data taken over one reset period in order to estimate its slope S and offset O.
where,
k is a constant which depends on the internal capacitance, put explicitly to be a constant 1. This implies that I has units of Amp per Farad. It does not need to be explicitly measured as it cancels out when the relative flux calibration is applied.
O is the charge offset at the zero-point of each ramp.
The time t is counted in seconds since the start of the ramp.
This module uses the glitch information generated by the deglitching algorithm above to raise a flag when the slopes are affected by 1, 2 or more glitches , are saturated or when there is no data (see SWSPFLAG in Table 9.8 for details). Because of this the routine uses Cal-G file 6 .
Refer to section 8.3.8 for more details on flux calibration.
The electronics amplify the detector read-out voltages with a commandable gain factor before the analog to digital (AD) conversion. Each detector can be operated at three different gains of the amplifier chain. The output voltage must be reconstructed from the digitized read-out's and corrected for the gain factor.
The input to the AD converter is related to the voltage across the detector by
where G is the gain which can be set to one of three values. The slope in V/s is related to the output of the AD converter, in Bits/sample, by
The conversion factor is given by
The numerical values originate from the fact the A/D converter has a maximum input range of 20 V which are converted into 4095 steps, with the input voltage inverted, and that there are 24 samples per second.
Once we have calculated the slope from the digital output we can divide by the gain factor to get the detector voltage.
Values for this are held in Cal-G file 5 .
This generates and fills the flag word with all relevant data.
This module converts the grating position to a grating angle, which is then used to calculate the wavelength of the light falling on the detectors. See section 7.5.
This module determines the Fabry-Pérot gap from the telemetered currents. See section 7.5.
This module calculates the wavelength of light falling on the detectors. Refer to sections 7.5 and 7.6 for a discussion of the philosophy behind this and the likely accuracy.
For the grating, this involves applying equation 7.4. The grating angle, calculated in section 8.2.12 is used along with various calibration files to determine the wavelength of light falling on each detector. Only bands with an unique order falling on their detectors are assigned a wavelength, all others are flagged as confused. It should be noted that all bands with an unique order falling on their detectors are assigned a wavelength, even if the band was not requested or contains useless data (although for non-FP observations the FP data is not transferred to the AAR).
For an AOT 01 measurement, where the grating moves during a reset interval, the grating position in the SPD is set to the grating position at the start of the reset interval. The wavelength in the SPD is that at the middle of the reset interval.
For the Fabry-Pérot , once the grating wavelength is calculated the mechanical gap width (calculated in section 8.2.13 is corrected to get the optical gap width (calibration file 18 ). The wavelength which passes through the FP is the one of which a whole multiple fits exactly in two times the optical gap width.
The data produced by the processing chain above is written out to the SPD and glitch list SWGH files. Each AOT results in one SPD file. For a definition of the SPD file see section 9.3.1. Note that the SPD file, like an ERD file, contains science data interleaved with calibration data.
SPDs of observations containing different reset intervals for the SW and LW gratings will contain zeros for the grating data which has no reset at a second the other grating has a reset.
