2MAPPS Software Interface Specification DBM01 3.6 98/01/23 Interface Name: DBMAN/DB Point Source Data Interface Type of Interface: Database (Informix) Field Data Specification Written By: DBMAN _______________________________ T. Evans Read By: Informix DB Description: DBMAN loads 2MASS data into the Informix 2MASS database. This SIS describes the DB fields (AKA "table columns") for the point source data. Descriptions of the extended source, scan/quality, auxiliary, and calibration DB data are found in the other DBMxx SISs. Also, information from each DBMAN run will also be written to a UNIX summary file. The DB table containing this data from survey scans is named "pt_src", and the table containing this data from calibration scans is named "pt_src_c". Both are in the "tmass" DB. If a field is not applicable or contains a "bad or N/A measurement value" (see previous 2MAPPS SISs for details) for a certain source, the field will contain "NULL" values as defined by the DB. However, some fields behave differently for band-filled data, as indicated by "BF" in the descriptions below. The descriptions below contain the following information: 1. "Name" contains the names of the DB fields or table columns. Note that they are all in lower case, because the DB doesn't distinguish between upper and lower case names. If there is a number in parentheses after the name, please refer to the numbered notes at the end of the descriptions for more information. Many fields are repeated 3 times, once for each band, but are found only once in the description below; these fields contain "" in the name, where "b" is understood to mean "j", "h", and "k". NOTE: The "cntr" column MUST be the first column in each table. 2. "Description" contains short descriptions of the field contents. 3. "Units" contains the units of the values in the fields. 4. "DBType" contains the DB data storage types for the fields. Note that some data types are further defined by values in parentheses following the type name. Also, these types are NOT cast in stone and could change, especially for the position fields. 5. "Null" indicates (y/n) whether the DB fields accept NULL values. 6. "Idx" contains a code indicating whether fields are indexed and what type of indexes they have. If the code is "0", the field is not indexed. If it is "1", the field has a simple index. If there is a "u" after the digit(s), the index is a "unique" one. If the code has a 2-digit number, the first digit indicates to which composite index the field belongs, and the second digit indicates the ordering of the fields within the composite index. 7. "Fmt" contains a C-like description of the best formats for displaying the field contents, *excluding* a leading space for field separation. 8. "Tbl" contains the (sub-)table ID to which the fields belong. NOTE: The "cntr" field will be the first field in each sub-table, and is used as a join index. The 1st sub-table ("01") is the "main" sub-table, and must contain one entry per table cntr number; any other sub-table will not contain an entry for a cntr number if its fields would all be NULL. 9. "Min" and "Std" indicate (y/n) whether the DB fields are found in the "mini" and "standard" table output views. NOTE: A line beginning with "-------" can only be used herein to delineate the section of field/data dictionary lines seen below. Name Description Units DBType Null Idx Fmt Tbl Min Std ----------------------------------------------------------------------------------------------------------------------------------------- cntr entry counter (key) number (unique within table) - integer n 1u 9d 01 y y hemis hemisphere (N/S) of observation - char(1) n 21u 1s 01 n y date observation date - date n 22u yymmdd 01 n y scan scan number (unique within date) - smallint n 23u 3hd 01 n y id source ID number (unique within scan) - integer n 24u 6d 01 n y ra right ascension (J2000 decimal deg) deg decimal(9,6) n 1 10.6f 01 y y dec declination (J2000 decimal deg) deg decimal(8,6) n 1 10.6f 01 y y glon galactic longitude (decimal deg) deg decimal(9,6) n 0 10.6f 01 n y glat galactic latitude (decimal deg) deg decimal(9,6) n 1 10.6f 01 n y x unit sphere x value - decimal(10,9) n 0 12.9f 01 n n y unit sphere y value - decimal(10,9) n 0 12.9f 01 n n z unit sphere z value - decimal(10,9) n 0 12.9f 01 n n err_maj major axis of 1-sigma error ellipse arcsec decimal(3,2) n 0 4.2f 01 y y err_min minor axis of 1-sigma error ellipse arcsec decimal(3,2) n 0 4.2f 01 y y err_ang angle of error ellipse major axis (E of N) deg smallint n 0 3hd 01 y y x_scan x (cross-scan) position (U-Scan coord.) arcsec decimal(5,2) n 0 7.2f 01 n n y_scan y (in-scan) position (U-Scan coord.) arcsec decimal(7,2) n 0 9.2f 01 n n cnf_flg (1) "confusion" flags from band merging - char(6) n 0 6s 01 y y _m (2) selected "default" magnitude mag decimal(5,3) y 1 6.3f 01 y y _msig "default" mag uncertainty mag decimal(4,3) y 0 5.3f 01 y y _m_psf (3) PSF-fit magnitude (BF aperture-photometry mag) mag decimal(5,3) y 0 6.3f 01 n n _msig_psf PSF-fit mag error (BF mag 1-sigma uncertainty) mag decimal(4,3) y 0 5.3f 01 n n _psfchi reduced chi-squared value of fit - decimal(5,2) y 0 6.2f 01 n y _psf_id ID number of PSF used for fitting - smallint y 0 5hd 01 n n _skyval estimated sky (bkgnd) level (BF noise within ap.) DN decimal(7,2) y 0 8.2f 01 n y _scale object-to-PSF comparative width parameter - decimal(4,3) y 0 5.3f 01 n n _sharp object-to-PSF comparative height parameter - decimal(4,3) y 0 5.3f 01 n n _blend number of objects grouped with source - smallint y 0 1hd 01 y y _peakpix peak pixel level above background DN decimal(7,2) y 0 8.2f 01 n n _m_stdap standard aperture magnitude mag decimal(5,3) y 0 6.3f 01 y y _msig_stdap standard ap. mag uncertainty mag decimal(4,3) y 0 5.3f 01 y y _m_lgap large ap. magnitude (BF 95%-confidence upper limit) mag decimal(5,3) y 0 6.3f 01 n n _msig_lgap large ap. mag uncertainty mag decimal(4,3) y 0 5.3f 01 n n _m_r1 R1 aperture magnitude mag decimal(5,3) y 0 6.3f 01 n y _msig_r1 R1 mag uncertainty mag decimal(4,3) y 0 5.3f 01 n y _prob_pers persistence probability - decimal(4,3) n 1 5.3f 01 y y _prnt_id persistence parent ID number - integer y 0 6d 01 n n _prg_flg (4) "purge" flags (BF flag ='fil') - char(3) n 0 3s 01 y y _mrg_flg (5) "merge" flags (BF status flags) - char(6) n 0 6s 01 y y _pix_flg (6) "pixel" src extraction flags (BF # bad pixels in ap.) - char(4) n 0 4s 01 n y _cal mag calibration value for source mag decimal(4,3) n 0 6.3f 01 n n gal_contam src contaminated by galaxy (check blanked/subtracted tbl) - smallint n 0 2hd 01 y y id_opt (7) associated optical source catalog ID ("best" match) - char(13) y 0 13s 01 n y dist_opt distance in arcsec relative to associated optical source arcsec decimal(4,2) y 0 5.2f 01 n y phi_opt position angle relative to optical source (E of N) arcsec smallint y 0 3hd 01 n y b_m_opt catalog blue mag of associated optical source mag decimal(4,2) y 0 5.2f 01 n y r_m_opt catalog red mag of associated optical source mag decimal(4,2) y 0 5.2f 01 n y nopt_mchs number of optical sources matched to this 2MASS source - smallint n 0 2hd 01 n y n2mass_mchs number of 2MASS sources matched to this optical source - smallint n 0 2hd 01 n y j_h default J-H mag color mag decimal(5,3) y 1 7.3f 01 y y j_h_sig sigma for J-H mag color mag decimal(4,3) y 0 5.3f 01 y y h_k default H-K mag color mag decimal(5,3) y 1 7.3f 01 y y h_k_sig sigma for H-K mag color mag decimal(4,3) y 0 5.3f 01 y y j_k default J-K mag color mag decimal(5,3) y 1 7.3f 01 y y j_k_sig sigma for J-K mag color mag decimal(4,3) y 0 5.3f 01 y y b_r B-R mag color of associated optical source mag decimal(5,3) y 0 7.3f 01 n y r_k default R-K mag color mag decimal(5,3) y 0 7.3f 01 n y b_k default B-K mag color mag decimal(5,3) y 0 7.3f 01 n y r_j default R-J mag color mag decimal(5,3) y 0 7.3f 01 n y b_j default B-J mag color mag decimal(5,3) y 0 7.3f 01 n y ext_key_1 (8) key to 1st extended source data DB record - integer y 0 7d 01 y y ext_key_2 key to 2nd extended source data DB record (if necessary) - integer y 0 7d 01 y y mp_key (9) key to minor planet prediction DB record - integer y 0 5d 01 n n exbr_key (10) key to extended bright source DB record - integer y 0 5d 01 n n night_key (11) key to night data record in "scan DB" - integer n 0 4d 01 n n scan_key key to scan data record in "scan DB" - integer n 0 6d 01 n n spt_ind spatial (x,y,z) index key - integer n 1 9d 01 n n clr_ind color index key - integer y 0 9d 01 n n ----------------------------------------------------------------------------------------------------------------------------------------- total # columns = 114, # bytes/row = 459 ----------------------------------------------------------------------------------------------------------------------------------------- NOTES: (Be aware that all flag definitions are subject to change) (1)"Confusion" flag: default (not confused) = "000000" (?) The confusion flag consists of 6 single-digit numbers defined as follows: Digit #1: Number of H matches found by J seed (0 if no J seed) Digit #2: Number of K matches found by J seed (0 if no J seed) Digit #3: Number of J matches found by H seed (0 if no H seed) Digit #4: Number of K matches found by H seed (0 if no H seed) Digit #5: Number of J matches found by K seed (0 if no K seed) Digit #6: Number of H matches found by K seed (0 if no K seed) (2)_m: selected default magnitude for each band For each band, a "default" magnitude is selected by DBMAN. For most srcs, the _m values are the PSF fit mags for each band. However, if a src is matched to a R1 src in a band, and the 1st digit of the _mrg_flg for that band is >0, the _m value for that band is the R1 aperture mag (_m_r1). On the other hand, if a src is band-filled in a band, the value of _m is the band-fill aperture-photometry mag found in the PSF fit mag column. The corresponding mag sigma value is put in the _msig field. For bands with source detections, the _m values are used to calculate the mag colors J-H,...R-K,...etc., along with the color sigmas. If one of the bands used in the calculation is band-filled, the color value becomes an upper or lower limit and is calculated using the 95%-confidence upper limit value. (3)Band-filled quantities: As noted above, some mag fields behave differently for band-filled data, as indicated by "BF" in the descriptions. These fields are as follows: (a) _m_psf (BF aperture-photometry mag): This field is calculated by finding a mag value for the measured flux within the band-fill aperture: BFMag = ZERO_CAL - 2.5 log10(flux) where ZERO_CAL is the zero calibration magnitude. Note that this field may have to represent a negative flux; in such a case, the value is BFMag = 99.0 - [ZERO_CAL - 2.5 log10(|flux|)] This encoding requires fluxes close to zero to be clipped. The clipping distance is set to 0.01 DN above zero for positive fluxes and below zero for negative fluxes. For example, the positive flux 0.0005 DN would be clipped at 0.01 DN and encoded as BFMag = ZERO_CAL - 2.5 log10(0.01) = 24.93 (for ZERO_CAL = 19.93) For a negative flux of -0.0005 DN, the clipping would yield -0.01 DN, and the encoding would be BFMag = 99.0 - [ZERO_CAL - 2.5 log10(0.01)] = 74.07 (for ZERO_CAL = 19.93) BFMag near 50 cannot occur, so 50 marks the boundary between positive and negative flux encoding. (b) _msig_psf (BF mag 1-sigma uncertainty): This field contains the 1-sigma uncertainty in the band-fill mag. (c) _skyval (BF noise within ap.): This field contains the noise measurement in the band-fill aperture. (d) _m_lgap (BF 95%-confidence upper limit): This field is computed as follows: Lim95 = integrated flux + (2 * noise) where the noise in this case means the "local noise", computed as follows (using the DAOPHOT algorithm): noise = sqrt(error1 + error2 + error3) where error1 = variance in the local sky = npix * skyvar npix = number of pixels in the aperture skyvar = the variance (square of the standard deviation) of the sky brightness error2 = photon noise = int. flux * Gain error3 = (skyvar/nsky) * (npix ** 2) nsky = number pixels in sky annulus (skyvar/nsky) = square of the standard error of the mean sky brightness For the case in which the integrated flux is negative, the upper limit is Lim95 = 2 * noise where noise = sqrt(error1 + error3) Different upper limits may be computed by the user. (4)"Purge" flags: default (not purged) = "000" col. 1: glint flag: 1 if src is a filter glint col. 2: diffraction spike flag: 1 if src is accepted as a real src but is contaminated by a diffraction spike 2 if src is part of a diffraction spike col. 3: confusion flag: 1 if src is confused with a brighter src A value of "fil" indicates the src was band-filled in this band. This value can change if we decide to use all integer flags. (5)"Merge" flags: default (not merged) = "000000" col. 1: mag/pos to-use flag: 0 if mag & position to use is from R2 data 1 if mag to use is from R1 data, pos from R2 2 if mag & position to use is from R1 data col. 2: R1-only flag: 0 if source is not R1-only source 1 if source is R1-only source col. 3-4: # R2 matches: # of possible (within radius, del mag) R2 matches to this R1 source col. 5-6: # R1 matches: # of possible (within radius, del mag) R1 matches to this R2 source Band-fill flags are not yet defined. (6)"Pixel" source extraction flags: default (all good) = "0066" col. 1: Nsat: # of frames with saturated pixels within std. aperture col. 2: Nbad: # of frames with bad pixels within std. aperture col. 3: Napmag: # of frames with 3-sigma ap. mags detections col. 4: Nposs: # of ap. mag measurements possible If band-filled, it contains simply the number of bad pixels in the band-fill aperture. (7)Associated optical source catalog ID: default (no assoc. src) = NULL The catalog ID and other association parameters are given for the "best" optical match found. The ID consists of a single-letter abbreviation for the catalog (e.g., 'T' for Tycho, 'U' for USNO-A) followed immediately by the ID of the object within that catalog, with no embedded blanks. (8)Extended source data keys: default (no ext. src data) = "0" or NULL? If there is ext. src data for this src, the ext_key_1 field will be set to the key of the record in the extended source DB containing the ext. src data. If there is a duplicate ext. src extraction in a different coadd (see SIS DBM02), the ext_key_2 field will be set to the key of the record of the 2nd extraction. The ext. src data will also have the same date, hemisphere, scan, id keys. (9)Minor planet record key: default (no minor planet match) = "0" or NULL? If a minor planet (asteroid) is predicted at this src's position, the field will be set to the key of the record containing the minor planet prediction data (see SIS DBM03). The minor planet data will also have the same date, hemisphere, scan, id keys. (10)Ext. bright source data key: default (no ext. br. src data) = "0" or NULL? If there is ext. bright src data for this src, the field will be set to the key of the record containing the ext. bright src data (see SIS DBM05). The ext. br. src data will also have the same date, hemisphere, scan, id keys. (11)Night, scan data keys: no default (non-null) These fields will be set to the keys of the records containing the corresponding night, scan, and band data in the "scan DB" (see SIS DBM06). These records will also have the same date, hemisphere, scan, and band keys. Possible NULL fields -------------------- The following is a list of which groups of fields can have NULL values, with a simple explanation of why and approximately how often it will occur. del_ra, del_dec, m_r_opt, m_b_opt, mch_prob, mch_flg, b_r, r_k, r_k_sig, b_k, b_k_sig, r_j, r_j_sig, b_j, b_j_sig: These parameters are NULL if no optical match to the src is found. This is expected to occur for <50% of srcs; probably more like 2%-25%?? depending on sky location. Additionally, one (or possibly both?) of the optical mags for a matching src could be NULL, so the mag colors that use the missing mag will also be NULL. _m, _msig, _m_psf, _msig_psf, _psfchi, _skyval, _scale, _sharp, _blend, _peakpix: These parameters all have a VERY low probability of being NULL, but it is possible. They won't necessarily all be NULL in the same entry, though. _m_stdap, _msig_stdap: These parameters are NULL if no good std. ap. mag measurement can be made. This happens for ~10-15% of srcs. _m_lgap, _msig_lgap: These parameters are NULL if no good lg. ap. mag measurement can be made. This happens for ~10-15% of srcs. _m_r1, _msig_r1: These parameters are NULL if no R1 src is merged with the R2-R1 src in this band. This will happen often, since only the brightest srcs (up to m~13 -- ~10%?) have R1 extractions. _prnt_id: This parameter is NULL if the src is not a possible persistence src. Fewer than 5% of srcs are possible persistence srcs. j_h, j_h_sig, h_k, h_k_sig, j_k, j_k_sig: One or more of these parameters will be NULL if both mags (_m) in the color calculation are band-fill mag "limits". Otherwise, they will be mag color limits. r_k, r_k_sig, b_k, b_k_sig, r_j, r_j_sig, b_j, b_j_sig: Four or all of these parameters will be NULL if the optical mag(s) in the color calculations are NULL or if there is no optical match to the point src. On the other hand, if the optical mags exist but a 2MASS band is band-filled, the colors involving that band will be mag color limits. ext_key_1, ext_key_2: These parameters might be either 0 or NULL if the src either does not have extended src data, or has only 1 extraction of extended src data. About 1%(?) of srcs will have any extended src data at all. mp_key: This parameter might be either 0 or NULL if the src does not have a minor planet match. Fewer than 1% of srcs will have a match. extbr_key: This parameter might be either 0 or NULL if the src does not have a extended bright src record. Fewer than 1% of srcs will have these records. Band-filling: If a band is band-filled for a src, the following parameters for that band are all NULL: _psfchi _msig_stdap _psf_id _msig_lgap _scale _m_r1 _sharp _msig_r1 _blend _prnt_id _peakpix _cal _m_stdap Additionally, colors (j_h,...b_k,...) that involve 2 NULL or band-filled mags, along with their sigmas, are also NULL (see above).