• Antelope Release 5.5 Linux 2.6.32-220.el6.x86_64 2015-04-21

 

NAME

check_q330pf_db - flag mismatch between open records in database and dataloggers in q3302orb.pf files

SYNOPSIS

check_q330pf_db  [-l] [-m mailto] [-p pf] [-v] -m [mailto] [-s subset]  db

SUPPORT


Contributed code: NO BRTT support.
THIS PIECE OF SOFTWARE WAS CONTRIBUTED BY THE ANTELOPE USER COMMUNITY. BRTT DISCLAIMS ALL OWNERSHIP, LIABILITY, AND SUPPORT FOR THIS PIECE OF SOFTWARE.

FOR HELP WITH THIS PIECE OF SOFTWARE, PLEASE CONTACT THE CONTRIBUTING AUTHOR.

DESCRIPTION

check_q330pf_db reviews the dataloggers section of q3302orb parameter files and compares stations found there to a database to make sure that all open stations are in the pf files and that no closed stations are attempting to be collected. Optionally, it also checks to see if there are stations attempting to be collected that have not had any waveform data written to the database recently.

This script may not see much use for any network other than the USArray Transportable Array. It was written to help alert the network operator of potential mismatches when trying to keep track of the status of 450+ dataloggers who only reside in place for ~18 months. With 25 or more dataloggers either installed or removed per month, a safety net to highlight problems is necessary. For static networks this script is unlikely to be run more than a few times per year, if at all. Note that for the first released version of this script, there is the assumption that all dataloggers in the network are q330s.

OPTIONS

FILES

See q3302orb(1) for a full description of the q3302orb.pf file. This script only reviews the dataloggers section of that parameter file.

ENVIRONMENT

Need to have sourced $ANTELOPE/setup.csh and environment variable $PFPATH set.

PARAMETER FILE

Below is the default check_q330pf_db parameter file.

# pf file for check_q330pf_db

q3302orb_pfs &Tbl{       # list of q3302orb pf file to check , if empty only q3302orb.pf
        q3302orb_1
        q3302orb_2
        q3302orb_prelim
}

subset          "snet=='XX'"  # default subset, full expression

# only used with -l option
maxlag          5        # number of days of no data before complaint
chansub         BHZ.*    # channel(s) for subset expression:  i.e. BHZ|BHZ_00

  • q3302orb_pfs
    List of q3302orb parameter files to check. Most networks only operate a single instance of q3302orb and use the default naming convention for that parameter file so there would only be one item listed: q3302orb. However, at the ANF in San Diego, for the TA project, we have 6 or 7 instances of q3302orb running, each with its own parameter file. Shown here is an example of a multi-q3302orb instance setup.
  • subset
    A full expression for what subset needs to be done. This subset can use fields from the site and/or snetsta table. Note that this subset is only applied to the database, your list of dataloggers will not be subsetted in a similar way. See BUGS AND CAVEATS below.
  • maxlag
    Value is in days. The wfdisc from the input database dbin is reviewed for stations that have no data more recent than now - maxlag (days). Only used with the -l option.
  • chansub
    Channels to look for in the wfdisc table. Subset applied to the wfdisc is chan=~/chansub/. Only used with the -l option.

EXAMPLE

check_q330pf_db -l -m someone@somewhere.edu "snet=='TA'" db/usarray

The above example applies all checks to the database. User someone, might receive an email with a subject of: "Problems - check_q330pf_db hostname /datalag/". It would be up to the user to then review their system to see if the station that is reporting the datalag is a station with comms issues and not sending data, or is a station that has been extracted from the field and thus needs to be closed in both the metadata and removed from the q3302orb.pf file.

DIAGNOSTICS

There are three types of tests performed and if any one of the three tests are failed they are reported in the subject line of the output email, or to STDERR. The three possible failures that could show up individually, or in combination in the mail subject line are: /datalag/, /excess db stations/, or /rm q3302orb stations/. They are explained below.
  • datalag
    Will only be returned if -l option is used. This test is tripped if there are dataloggers listed in the pf file with open records in the metadata, but no recent (see maxlag) data.
  • excess db stations
    If there are open stations in the database that are not found in the q3302orb pf files, this alarm is tripped. It could be a non-issue for you if the station does not have a q330 (see BUGS and CAVEATS below), or it could indicate that you need to update your metadata.
  • rm q3302orb stations
    If there are stations that have been closed in the metadata, but not removed from the q3302orb pf files, this alarm is tripped. For a mobile network like the USArray TA, this is an important case to trip because if the datalogger is reinstalled, it could start collecting data tagged with an incorrect station name. A quick response to this alarm is recommended.

In a perfect world, no mismatches will be found and you will see a subject line that reports a Success.

SEE ALSO

q3302orb(1)

BUGS AND CAVEATS

This program assumes all stations to be collected are q330s. At some point, I need to put in a check to try to ignore a station if it is not using a q330 as the datalogger.

Note that there is no check in this script to see if the sta-datalogger_sn reported in the database matches the sta-datalogger_sn recorded in the q3302orb.pf

If you use a subset like -s snet=='TA'&&sta=~/1.*/, you may see many error messages like:


XXXX exists in a pf but is not found in the db

This is because you subsetted the database only. The same subset is not applied to the dataloggers table so you may see many reports of a station existing in the pf but not the db.

AUTHOR

Jennifer Eakins
Univ. of California, San Diego
Antelope User Group Contributed Software