CHAPTER 5
Technology Transition and Data Management
5.1 Developing Operational Models
Success of the National Space Weather Program (NSWP) requires that the knowledge and models of the space weather system generated by the research community be incorporated into operational models of use to forecasters and/or external customers. The goal of the research model is to demonstrate understanding of the physics appropriate to the bounded system being modeled. Input may or may not include realistic data, and output is typically in the form of scaled parameters that illustrate the behavior of the system under a variety of conditions, mostly idealized. In contrast, an operational model must be able to use existing real data to produce clear-cut results applicable to all types of conditions.
The strict requirements imposed on operational models have made the development process slow and difficult. A high priority in the implementation of the NSWP is the creation of a system to incorporate research results into operational models quickly and efficiently. The NSWP will investigate accomplishing this process through rapid prototyping, as illustrated in Figure 5-1. At the core of the process is a Rapid Prototyping Center (RPC). At the RPC, immediate feedback is provided to the development team as
Figure 5-1. Development Process for Operational Models
concepts are tested in a quasi-operational environment. RPCs allow competing methodologies or techniques to be examined quickly, cheaply, and creatively, often generating a product with more capability than originally envisioned.
RPCs will publish standards to which models must adhere before they are accepted into the rapid prototyping process. Models must be validated--they must properly represent the physical processes, function correctly within the natural range of input variables, and be coded correctly. Standard structured programming techniques will be required, as will appropriate documentation. Standard computer languages will be specified so that code will be transportable between the hardware platforms and operating system environments under use at the centers. RPCs may also specify output formats to facilitate interaction with in-place visualization tools.
As illustrated in Figure 5-1, the rapid prototyping process uses research models in
conjunction with a realistic data stream. The models are evaluated as necessary to check their ability to produce accurate results under the full range of possible input values. They must be suitable to the task at hand, produce reliable results, and be easily usable to the space weather forecaster. The operational models must deal with momentary or extended data dropouts, different data sampling rates, and rapid rates of change in sensor output that may or may not be noise. Sometimes models must knit together sets of data from very different and non-collocated sensors, extrapolating data, if necessary, or filling in with proxy data. Output errors must be controlled as the model moves forward in time or space, and all model outputs must be self-consistent. The successful operational model should also validate itself against actual measurements and reinitialize itself, or notify operators that errors have become unacceptable. Finally the model should accommodate forecasters with various skill levels, adapt to a variable production demand, and be relatively easy to modify if new data, products, or hardware or software upgrades become compelling.
Research models and analysis techniques that "graduate" from the RPC process will have de facto become incorporated into the appropriate operational service center: the 50th Weather Squadron (50 WS) or the Space Environment Center (SEC). The 50 WS, which supports military users of space weather data, produces a set of tailored products from the model outputs for specific military needs and provides unclassified data to SEC. These data, along with other data generated at SEC are available for civilian customers. In addition, these data can be used to help industry and, perhaps, universities tailor civilian products in response to specific customer demands. An exciting possibility is the formation of small businesses aimed specifically at tailored-product development.
5.2 Data Management
The NSWP will guide improvements of the capability to process, assimilate, and analyze increasingly complex data sets. Rapid advances in computer technology have opened a realm of possibilities for development of expert systems, image feature recognition, real-time data access, and database systems. Near-real-time data assimilation is also required for initialization and updating of forecast models.
Space weather services depend on data collection and processing in the same way that tropospheric weather services do. Data need to be collected from a large number of sensors strategically placed on Earth's surface, in Earth orbit, and in interplanetary space. The forecast centers need computer systems that can rapidly process and analyze large volumes of observational data; run fairly complex models in real time; display and manipulate imagery; derive, generate, and disseminate useful products; and facilitate data sharing and backup responsibilities. Acquisition of new data sets and development of advanced models, with complex calculations, will require greatly enhanced computer systems at the core of space weather services. The forecast centers must replace and continually upgrade both hardware and software to deal with the growing computational needs.
Climatological studies and products also need improving to satisfy the need of planners and engineers to know the range of conditions their systems may encounter and the probabilities of those conditions. Critical needs in this area are a system of quality control for the data and an effective method of archiving checked data so that they are readily accessible to the research community.
The NSWP will promote and coordinate development of improved standards for data collection, formatting, communication, and management. This improvement will facilitate processing and use of the increasing volume of observations, model output, and products within and between operational centers and customers.