CHAPTER 8

QUALITY ASSURANCE

8.1 Introduction. Effective quality control (QC) requires actions on the part of observers; communicators; national analysis centers (the principal users of the data) and archival centers; and the upper-air program managers. The QC endeavors of these entities are so interrelated that their activities cannot be carried out independently. This Chapter describes an integrated system for coordination between these entities. Such a system provides Quality Assurance.

The purpose of Quality Assurance is to identify deficiencies in the rawinsonde data and to initiate swift and corrective actions. To identify deficiencies, an efficient monitoring program must be maintained. Subsequent to the detection of problems, procedures must exist to enable the problems to be corrected. Each of the entities listed above has a particular responsibility in the Quality Assurance program.

WMO recommendations and minimum standards for upper-air observations quality control are given in Reference 10.

8.2 Monitoring and Quality Control. Two types of quality control are imposed on U.S. upper-air observations. The primary type is referred to as Operational Quality Control (OQC). The second type is referred to as Administrative Quality Control (AQC). The basis for performing both types can be found in WMO Manuals and Guides for the Global Observing System, the Global Telecommunications System, and the Global Data Processing System. [Refs. 10-13] The requirements and methods for carrying out these two types of quality control are jointly agreed upon by the agencies involved and are specified in this Chapter.

8.2.1 Operational Quality Control. OQC is the term used to describe monitoring which is carried out in real-time to near-real-time (0 to 12 hours after observation). It is defined as the quality control performed in time to salvage missing, incomplete, or erroneous data for immediate use or to allow remedial actions to be taken in time to be of value before the next scheduled observation takes place. The following entities are involved: the observing station (OQC described in Chapter 4); the communication hubs (OQC described in this Chapter); and the national analysis centers (OQC covered in this Chapter).

Successful OQC monitoring includes checks of the following items: the quantity and regularity of observations, the quality of the data, the completeness and timeliness of the collection of observational data at the Center concerned, and adherence to standard codes and communications procedures.

Two very important keys to the success of quality control are awareness and timeliness. These include speed in detecting and correcting deficiencies and promptness in notifying data sources and parent units of deficiencies found in their data.

8.2.1.1 The Communications Centers. The Communications Centers are responsible for monitoring the regular flow of meteorological information. This first level of OQC, after the data leave the observing site, involves keeping a close watch on the receipt and transmission of information, generating requests for missing bulletins and reports, checking communications formats, and arranging for the rerouting of traffic in case of system difficulties.

Established procedures allow for verification of the successful transmission and reception of all parts of the rawinsonde message from each scheduled observation. Normally this verification takes place at a location other than the observing site, to assure that the data have successfully found their way through the telecommunications system, and may occur at a communication center or at an analysis center.

8.2.1.2 OQC at the National Analysis Centers. Procedures performed by the centers consist of a series of computerized and interactive examinations of the decoded upper-air messages. Each center makes a decision as to the quality of each report. Any change or rejection (deletion) of the entire report or a portion thereof is recorded.

8.2.1.3 Reporting Deficiencies. The centers have the authority for reporting suspected deficiencies to individual upper-air units within their purview. The National Centers for Environmental Prediction (NCEP) alert NWS upper-air units of problems with their observations. Air Force Global Weather Center (AFGWC) and Fleet Numerical Meteorology and Oceanography Center (FNMOC) perform a similar function for DoD installations. However, the centers should also exchange OQC information for stations of other agencies.

8.2.2 Administrative Quality Control. AQC is the term used to describe monitoring data after OQC has been completed. AQC includes monitoring data timeliness, quantity, and quality. It is intended to identify and correct chronic or persistent system problems which may not be readily identifiable by OQC. Its purpose is to review the general performance of the system and network and to identify shortcomings which may persist after real-time monitoring is completed. Non-real-time monitoring requires the preparation of summaries and various statistics. It may also include the more immediate record checking accomplished at the station by the following working shift.

AQC is performed at the centers by the upper-air program managers and at the Archival Centers. Most of the centers' programs assign probable values to questionable and erroneous data. This enables the centers to evaluate the magnitude of difference between reported and computed values - an essential tool in AQC. Daily statistics on the differences between the observed variables and the assimilating forecast are accumulated monthly. Also available are the records of the consistency-checking programs and the differences between the observed variables and the analysis in which they were used. Digital (by site) and graphical summaries of the mean difference to the forecast, the root-mean-square difference, and the number of rejections by the analysis (assimilation) scheme are produced. Monthly reports containing a brief summary of this information are prepared for the upper-air program managers.

8.2.3 WMO Reports. Under the World Weather Watch program, the WMO has requested that various WMCs and Regional Meteorological Centers (RMCs) assume the responsibility for global monitoring and quality control of specific types of data. The task of coordinating the monitoring of upper-air data has been undertaken by the European Center for Medium-Range Weather Forecasting (ECMWF), located near Reading, England. The ECMWF, along with the major numerical prediction centers throughout the world, monitors the reception and quality of upper-air data from every synoptic network upper-air station in the world. The ECMWF's responsibility is to collect information from these prediction centers, identify sites whose data by consensus pose problems during the analysis and forecast procedure, and report its findings to the WMO Secretariat. NCEP (Washington D.C., WMC) compiles the U.S. AQC Monthly Quality Monitoring report for the WMO which includes statistics for U.S. stations. The report is then distributed to other U.S. centers, ECMWF, and other WMCs and RMCs.

8.3 Coordination and Quality Assurance. The efficient exchange of information generated by AQC is essential for Quality Assurance. The entire atmospheric science community will have contact with upper-air observations at some time or other. In this Handbook, those units which perform regularly some kind of quality control are listed. Table 8-1 contains a list of the organizational units in the U.S. that have responsibility for some phase of the rawinsonde and pibal observations - either the taking of, the use of, or the management of the program. An alphabetic code is given for each as well as a location and telephone number. The purpose of this list is to enable them to contact one another and exchange information.

Table 8-1. Specific Components Involved in Upper-air Observations

8.3.1 The National and Regional Headquarters. The Headquarters elements of the various agencies must be committed to and directly involved in the AQC aspect of the upper-air program. They should rely on the national centers and the communication centers to provide statistical information for evaluation and action. By assembling statistical data into useful formats, the various Headquarters can:

• identify repetitious occurrences of deficiencies in operational upper-air data,

• determine when and where system breakdowns and shortcomings are occurring,

• ascertain where additional training may be required,

• establish reasonable standards of performance for evaluating the observing and communicating networks as a function of geographic or management criteria, and

• use the objective standards for evaluating various segments of the upper-air network.

Initial corrective action should be taken as rapidly as possible by the observing sites' program manager. Follow-up action to remedy systematic deficiencies should be taken by the agency concerned in a timely fashion. Performance statistics and actions should be based on station groupings according to geographic and management criteria. Similarly, Regional Headquarters should establish standards for the observing sites under their management control.

8.3.2 Exchange of Information Notifying Individual Upper-air Units. The centers should provide the summary statistics to their upper-air program managers for further analysis, review, and action. Individual station supervisors should take action to correct deficiencies. The objective basis for actions should be predicated on the establishment of long term averages of the performance by individual observers. Based on these averages, attention should be focused on substandard performance.

The style and format for any published diagnostic information shall be coordinated among the centers and approved by the individual agencies. This will ensure consistency and compatibility among the methods and formats and will avoid misinterpretation of results. All diagnostics should be summarized, tabulated, printed, and distributed to interested offices within the agencies. They should also be exchanged among the centers for analysis and comparison.

8.4 Documenting Quality Information for Archival Data. The centers responsible for maintaining an archive of rawinsonde and pibal data collect reports that have been subjected to various levels of OQC and AQC. In addition, they perform some types of AQC independently of the centers responsible for forecasts. The results of all quality control activity should be archived as metadata in appropriate formats. Moreover, future users of archival information need to be aware of previously documented problems. Therefore, documentation related to data quality problems that may have been resolved but remain in the archived data should be maintained.

Chapter 7
Appendix A
Table of Contents
Publications List
Home