VI. Analysis of the Release Catalogs

1. Comparison of Achieved Performance of the All-Sky Release Catalogs
with Level 1 Science Specification

f. PSC Reliability

i. Executive Summary

The Level 1 Science Requirements for the PSC states that the differential reliability of sources in the last half-magnitude bin brighter than the completeness limit (J=15.8, H=15.1 and Ks=14.3 mag) should be greater than 99.95% in unconfused regions of the sky.

Reliability in this context, refers to the probability that if a new measurement is made at the position of a PSC source, a source will be detected, within the constraints of possible proper motion or flux variability of the source. The practical definition of reliability in tests on samples of data is the ratio of the number of "real" astronomical sources to the total number of sources in the set. The total number is just the sum of "real" and "false" detections.

We verify the reliability of the PSC using both external and internal comparisons. The external test is based on comparison of sources in the PSC with sources drawn from the Sloan Digital Sky Survey (SDSS) Early Release Area, and indicates a reliability of 99.981% for 2MASS sources in this area. The internal comparison makes use of repeated Survey observations of the sources near the celestial poles, where the Survey Tiles converged and were heavily overlapped, and indicates differential reliability of 99.98%, 99.98% and 99.96% in J, H and Ks, respectively. These tests are described below in subsections ii and iii.

The various tests for reliability have led to a good understanding of the remaining sources of unreliability in the PSC. These are summarized in subsection iv. None of these are believed to produce enough spurious detections in the PSC that would compromise meeting the reliability requirement.

Each of these tests show that the 2MASS PSC meets the Level 1 Requirements for reliability.

ii. Reliability from Comparison with the SDSS Early Data Release (EDR)

In the absence of foreground extinction (i.e., at high galactic latitude), the depth of SDSS should exceed that of 2MASS for all except the intrinsically reddest objects. Thus, all reliable, inertial sources in the 2MASS PSC should have counterparts in the SDSS. We have estimated the reliability of the PSC by measuring the fraction of 2MASS sources in the SDSS EDR area that have SDSS counterparts. The detailed report on this analysis is presented in this memo by D. Kirkpatrick and B. Nelson.

All objects in the 2MASS PSC with 23h28m < RA < 24h or 0h < RA < 3h40m and -1° < DEC < +1° -- an area completely covered by both releases and representing 126 deg2 -- were checked for a corresponding SDSS source within a 1.5´´ radius. There are a total of 140659 2MASS PSC sources in this area that have ph_qual="A" in at least one band, and that have use_src="1". These characteristics define the high reliability Catalog subset of the PSC that we believe meets or surpasses all of the Level 1 Science Requirements. Of these, 231 (0.16%) do not have a corresponding SDSS source within 1.5´´.

The position of the 231 2MASS sources without SDSS counterparts were examined on 2MASS Quicklook, DSS and XDSS images to assess their nature. Over half of these non-matches (140) were found to have optical source counterparts on the DSS or XDSS plates and hence validate the reliability of the 2MASS source. We do not understand their absence from the SDSS EDR lists. Of the remaining 91 2MASS sources (0.07% of the initial list).:

This implies a reliability of 99.981% for the PSC, which is better than the stated requirement of 99.95%.

iii. Reliability from Tile Overlap Regions

2MASS Survey Tiles are oriented along lines of declination, so adjacent Tiles converge as one moves towards the celestial poles. The Tiles in the |dec|>84° declination bands overlap one another heavily, so points on near the poles were many times during the Survey. We have used the repeated measurements of sources in these heavily overlapped regions to gauge the internal reliability of the PSC.

Sources in the overlapping regions between Tiles were positionally correlated using a matching radius of 2´´ during duplicate source resolution, the process that selected which apparition of multiply detected sources were to be passed to the PSC (see this memo for details). Two statistics were generated for each source: the number of Tiles on which each source could have been detected (spos), and the number of Tiles on which it was detected (sdet).

These two statistics have been compiled for sources in the PSC that were covered by six overlapping Tiles (spos="6") to estimate the reliability of the PSC. These sources are distributed in narrow declination bands approximately 0.3° wide, at ±89°. The net reliability for an ensemble of sources, R, is given by:

R = (Ntrue) / (Ntrue + Nfalse/nc)

where Ntrue = Number of true sources (sdet >2), Nfalse = Number of false sources (sdet2), and nc = Max number of possible confirmations, in this case, 6.

The differential reliability was determined for collections of sources in the spos=6 regions in each band separately, and in 0.5-mag-wide bins in each band. The reliability was computed for all spos=6 sources, those with ph_qual="A" in the relevant band, and then for sources with cc_flg="000," regardless of ph_qual value. A summary of the differential reliability in the 0.5 mag bin just above the SNR=10:1 brightness levels, specified in the Level 1 Requirements (J<15.8, H<15.1, Ks<14.3 mag), is given in Table 1.

The reliability for the ph_qual="A" samples is 99.93%, 99.98% and 99.96% in J, H and Ks, respectively. Thus, H and Ks meet the requirements, even without filtering out possible confusion. The J-band reliability of the ph_qual="A" sample falls below the requirement, because of the influence of sources in confused environments. This is illustrated by the fact that the differential J relability of the cc_flg="000" dataset is 0.999794, surpassing the requirement, regardless of the photometric quality of sources in that magnitude bin.

Thus, for this subset of PSC data the differential reliability is >99.95% in all bands, in the absence of confusion.

Diffential Reliability for spos=6 Sources
BandNo filterph_qual="A"cc_flg="000"
J 0.997804 0.999316 0.999794
H 0.995326 0.999782 0.998919
Ks 0.995877 0.999589 0.999286

iv. A Summary of Known Classes of Unreliable Sources

Validation of the PSC, and especially tests of reliability, such as those described above, have allowed us to identify the causes of the majority of remaining unreliable sources in the PSC. A specific example of this is that, from the comparison with the SDSS EDR data (above), we have made the discovery of hot pixel artifacts. One pixel in the southern H-band array had produced so many spurious "Catalog-quality" that led us to identify and remove them from the PSC before the Release, in order to meet the reliability requirements in the H-band. It is important to recognize that, even at the strict reliability levels specified by the Level 1 Requirements, though, there may be >170,000 unreliable sources in the high reliability Catalog component of the PSC, and more in the full dataset.

The major causes of unreliability in the PSC are summarized below, along with estimates of the extent of such objects in the Release. A considerably more detailed discussion of these classes of objects, and tips on how to recognize them are supplied to users in the PSC Cautionary Notes (see I.6b).

In general, unreliable sources exhibit one or more of the following characteristics. Users should exercise some caution when encountering such sources. However, possession of any one or more of these attributes does not necessarily imply that a detection is unreliable, only that it merits examination.

Common Characteristics of Unreliable PSC Sources
ConditionSource Attribute
Single band detection rd_flg="0" in two bands
Contamination or confusion flagging cc_flg != "000"
Large value of profile-fit photometry 2 value rd_flg="2" and [jhk]_psfchi >> 1
Low frame-detection count for SNR>7 sources ndet[1,3,5] 1 and [jhk]_snr >7
Read_1 detection that is fainter than the Read_2 saturation limits sources(rd_flg[1]="1" and j_m>9) or (rd_flg[2]="1" and h_m>8.5) or (rd_flg[3]="1" and k_m>8)
Source is fragment of an extended sourcegal_contam="2"
Source is associated with a known solar system objectmp_flg="1"

The Major Classes of Unreliable Sources are:

[Last Updated: 2003 January 27; by R. Cutri, D. Kirkpatrick and S. Wheelock]

Return to VI.1.