From mazz@ipac.caltech.eduWed Feb 26 07:22:22 1997
Date: Tue, 25 Feb 1997 12:58:22 -0800 (PST)
From: Joe Mazzarella
To: ISIS -- Nian-Ming Chiu ,
Roc Cutri , Tracey Evans ,
John Good , Carol Lonsdale ,
Joe Mazzarella ,
Sherry Wheelock
Cc: Tom Chester
Subject: 12Feb97 2MASS DBWG Meeting Minutes. Meeting today 1:15PM, SCR
***************************************************************************
12-Feb-97 2MASS DBWG (ISISWG) Meeting Minutes
ATTENDED: Nian-Ming, Tracy, John, Joe, Roc
---------------------------------------------------------------------------
GENERAL DISCUSSION/FACTOIDS
Tracy: Loading/indexing test results report:
Method 1: Load the table with indexing on.
==> Requires a re-build of internal pointers/indices.
Method 2: Drop indices; load data; re-create indices.
Comparison: Even with non-clustered indices, Methid 1 takes ~10 times
longer than method 2.
Left to do: Unclear how timing is effected by the number of indices.
Note: Indices add ~20% to the initial space requirements.
(To be clarified later.)
Roc: There will be an end-to-end operational run-through using either
'faked' 3-channel data or a real night of telescope data in mid/end March.
This should include DBMAN to test population of the DB, and hopefully
the ability to run some basic queries. (V1.0 of the pipeline is due ~17 Mar.)
All:
1) Is enough disk space configured to handle the DBMAN output?
Approximate needs:
(<#objects/scan>~5000)*(~75 scans/night) ~ 4e5 objects/night
==> Assume average of ~0.5e6 objects/night.
peak of ~1.0e6 objects/night.
(~1e6 objects/night)*(~600 bytes/object) ~ 6e8 = 0.6 GB/night
==> Assume 1 GB/night to be safe for run-through.
We will have 8*9GB~70 GB (/k*) shoebox disks on karloff available
for Informix as soon as team mambers clear the space.
==> Enough for first ~2 months of the mission.
See actions below.
2) Graphics output for Web pages: To simplify usage of Web HTML for
2MASS analysis docs (e.g., basic in-lined graphics) and ISIS, we need to
produce results in GIF (or JPEG) format in addition to PostScript. We
need to learn how to produce GIF/JPEG directory from our plotting
packages, and/or how to convert PostScript to GIF/JPEG in an
automated way without loss of resolution. (Display of PS with
Ghostview and doing a screen-dump with xv does not cut it!). Mazz
agreed to look into solutions (NETPBM, etc.)
---------------------------------------------------------------------------
RECENTLY COMPLETED ACTIONS
John:
o Email to ipac-staff about 18-20 Informix classes.
o Draft agenda and viewgraphs for IPAC ISIS meeting (w/ Joe)
Nian:
o Discussions with WIRE indicate they will purchase their own
UltraSPARC 2 to be configured as their own informix server.
Tracy:
o Progress on SIS's and timing tests. See discusion above.
Joe:
o Met w/ Sherry regarding 2MASS postage-stamp server.
Link to current server in place. We will make it more
modular and integrated into ISIS later.
o Draft agenda and viewgraphs for IPAC ISIS meeting (w/ John)
o Distributed copies of CDS/ESA "Astronomical Server URL" memo.
Roc:
o Summarized disk space available for 'tmass' db for first few
months of mission.
Answer: 8*9GB~70 GB (/k*) shoebox disks on karloff
---------------------------------------------------------------------------
PRIORITY ONGOING ACTIONS
(Long-term ongoing actions omitted here. See task schedule on our Web.)
Tracy:
o Loading/Index timing experiments:
w/ and w/o various indexes, clustered indexes, etc.
o Refinements on SIS's for DBMAN & WSDB field definitions:
extended sources, mag & color uncertainties and upper limits.
o Treat galaxy data. (See previous memos).
o Calibration scans need SIS's and tables in the DBMS.
~40 fields expected; define indexing scheme.
John:
o Infrastructure (& User System w/ Joe) design docs.
o Design/prototype infrastructure (& User System w/ Joe) code.
o Call Cassini Project and get their story on the raid disk.
o Find out more (w/ Nian) about HP/Informix business.
Does this hardware/software bundle offer significant benefits?
Summarize in email to team. Send email.
o Discussions (w/ Joe) with IPAC mission managers in preparation
for general ISIS req/design 'kick-off' meeting.
Nian:
o Working on high-priority Survey Operations DB (non-Informix) project.
Roc:
o Refine mag/color uncertainties and upper limits (w/ Sci Team)
for DNMAN SIS (to Tracy).
o Firm up agenda and viewgraphs for IPAC ISIS meeting
(w/ Joe&John&Carol)
Joe:
o Discussions (w/ John) with IPAC mission managers in preparation
for general ISIS req/design 'kick-off' meeting.
ISO & SIRTF (George & Rick): Friday, 07-Feb, 11:00
WIRE (Tim & Perry): TBD
MSX (Mehrdad): TBD
o Reschedule ISIS meeting and include (at least draft) agenda.
o User System (& Infrastructure w/ John) design docs.
o Design/prototype User System (& Infrastructure w/ John) code.
o Make an "RTB" for Sybase->Informix db migration (FSC to start)
Carol:
o Firm up agenda and viewgraphs for IPAC ISIS meeting
(w/ Joe&John&Roc)
---------------------------------------------------------------------------
NEW ACTIONS (See also ONGOING above and in Web Task List)
Joe:
o Develope initial "CatScan" interface for end-March run-through.
John:
o Please design and document the data "containers" (partitions) we
will use for the source DB. How will table fields be partitioned
vertically (split into relations), and how will objects=rows be
partitioned in data-space to maximize query efficiency, etc.?
Nian:
o Plan for incorporating /k* disks into Informix space as the
disks are cleared and become available. It would be nice to
have one 9 GB disk totally available for extension of tmass
by ~7 March in preparation for DBMAN run-through by mid/end
March.
Tracy:
o Develope and test initial DBMAN for end-March run-through.
Roc:
Carol:
NOTE: Joe, John, Tracy, Sherry, Nian will be at Informix ESQL/C class
next week, so there will be no DBWG meeting.
---------------------------------------------------------------------------