RPI BinBrowser User's Guide

U. MASS Lowell
Center for Atmospheric Research
Last changed: January 17, 2001

Please e-mail suggestions and corrections to Ivan_Galkin@uml.edu
 

1. RPI Data Background

1.1. RPI Measurement is a Cube

RPI is a radar with the variable frequency. In the most of its measurement modes, RPI sends a pulse and listens for the echoes. On each particular frequency, RPI can send more than one pulse. An RPI measurement can therefore be visualized as a cube:

Figure 1.  RPI Measurement Cube

In this particular example, RPI used 7 frequencies, sampled the reply at 6 ranges for each frequency, and repeated the measurements 4 times per frequency.

There are also listen-only measurement modes available in RPI. where no pulse is transmitted. The data collected in these modes are still arranged in terms of the frequency - range- repetition "cube". See section 1.5 for more details.

1.2. Databin Concept

RPI measurement "cube" consists of databins, where each databin has a unique combination of frequency, range and repetition. An example on Figure 1 contains 7 x 6 x 4 = 168 databins. The databin contents depends on RPI measurement mode:
 
 
Abbreviation
Name
Contents
Diagram
Comments
LTD Linear-scale Time-domain Data 12-bit linear scale quadrature components, 2 per antenna, 3 antennas LTD.GIF
SSD Spectral-domain Science Data Three 8 bit log-scale amplitudes (per antennas) and two 8 bit linear scale phase differences SSD.GIF
SMD Spectral-domain Maximum-amplitude Data Three 8 bit log-scale amplitudes (per antennas), two 8 bit linear scale phase differences, and one 8-bit Doppler number SMD.GIF Only one Doppler line is reported out of all collected
PRD Precision Range Data Three 8 bit log-scale amplitudes (per antennas), five 8 bit linear scale phase differences, and one 8-bit Doppler number PRD.GIF
CAL Calibration One 8 bit log-scale amplitudes (per antennas) and one 8 bit linear scale phase per antenna, 3 antennas CAL.GIF Calibration packet only
DBD Double Byte Data One 8 bit amplitude and one 8 bit Doppler DBD.GIF Low data volume option
SBD Single Byte Data One 5 bit amplitude and one 3 bit Doppler SBD.GIF Low data volume option
TTD Thermal-noise Time-domain Data Eight 8-bit log-scale averaged amplitudes and two cross-power terms per antenna, 3 antennas TTD.GIF

1.3. Auxiliary Information: Preface and Frequency Headers

Apart from the set of databins, each RPI measurement has to be accompanied by auxiliary information that includes

1.4. Telemetry Packaging

RPI data stream has to be partitioned into the fixed length telemetry packets. The design requires robustness to the telemetry drop-outs, i.e. loss of any packet in the sequence of packets from an RPI measurement must not cause loss of the whole packet sequence or any of its remaining parts.

To fulfill telemetry packaging requirements, all databins collected on a frequency are enumerated and the serial number of the first databin in the packet is stored in a data header in the beginning of each packet, so that its frequency, range, and repetition number can be restored independently of previous packets..

1.5. Organization of TTD data

In the thermal noise mode, RPI samples antenna outputs without transmitting; the sampled quadratures undergo averaging and calculation of the cross-power terms. TTD databin stores 8 averaged amplitudes per antenna; there may be more than 8 amplitudes collected on a particular frequency. To follow the standard data arrangement shown on Figure 1, the "cube" of TTD databins has 1 range, and the additional databins for the same frequency are organized in the repetition plane.

2. Data Sources

RPI BinBrowser operates with the following data sources: IMPORTANT: BinBrowser v2.0 reads UDF files in its original "Little Endian" arrangement, as created at IMAGE SMOC. If the UDF Data Installation procedure changes the Endian to "Big" to allow UDF software to read files on your computer, BinBrowser v1.4 will not be able to identify the change of the Endian arrangement and read those files. In this case instead of using the UDF Data installation procedure, un-gzip the original UDF bundles with RPI data and place the files on your disk manually.

The source data can be acquired:

The UMLCAR RPI Data Catalog provides an elaborate means for subsetting the RPI data, as discussed in detail below.

3. RPI Sessions

3.1. Concept of Session

One RPI session contains all raw data collected by RPI during a single frequency scan from the lower frequency limit, [L], to the upper frequency limit, [U], plus the derived, or scaled information (plasmagram traces, electron density profiles, etc.) created interactively by a BinBrowser operator.
 

3.2. Session List

At any given instance, BinBrowser has access to a particular subset of RPI data acquired from the data sources, called session list. The session list is assembled by BinBrowser if the source data are taken from files, or by the RPI Data Catalog server in reply to a query, as discussed below. The session list is used to navigate through the currently available data subset (see Figure 2).
 
 

Figure 2.  Session Selector

After the session list is compiled, the chronologically first entry is displayed as the current session. The droplist control is used to "drop" the list of available sessions to select the current session. Also, four buttons are available to walk the session list forward and backwards.

3.3. Twin Sessions

When on a particular schedule, RPI goes through a variety of measurement programs. When BinBrowser compiles the session list, all sessions are presented in the time-ordered sequence. To help with navigating the list, the concept of twin sessions is introduced, where twin sessions are simply subsets of sessions with the same RPI program number. Stepping through the twin sessions gives a consistent presentation of the RPI similar measurements on orbit.

4. Files and Queries

4.1. Data Management Scenarios

There are three main scenarios of the data management within the BinBrowser 2.0, (A), (B), and (C):
 
Table 1.  Data Management Scenarios
Scenario:
Session List created by:
Raw data coming from:
Format:
Comments:
(A) BinBrowser Local disk, network disk, removable media. L0, L0.5, L1, RSI  Local one-day files directly opened by "Open File" or "Add File" command.
(B) UMLCAR RPI Catalog Local disk, network disk, removable media.  L0 A catalog query issued over the Internet to subset the data and build the session list. The L0 data files are available locally.
(C) UMLCAR RPI Catalog UMLCAR Online RPI Database.  L0 A catalog query issued over the Internet to subset the data, build the session list and retrieve the data, session-by-session, over the Internet.

Scenario (B) is the most convenient as it combines the search/subsetting features with the speed of the local disk access to the raw data. Scenario (C) is used for the situations where the raw data are not available locally, and obtaining them in form of the one day files from the RPI data distributors is not feasible. In this case the current session data is obtained from the UMLCAR RPI Online Database over the Internet and stored in a local disk cache to avoid re-sending of the data next time the session is requested. Scenario (A) is typical for off-line work with the local files.

All scenarios for data management can be utilized simultaneously. For example, a query can be sent to retrieve a session list, and if only a part of session data can be found locally, (B) and (C) scenarios will be combined to acquire all relevant data; then, a local file with RSI information can be added using the (A) scenario.

4.2. File operations

Files are opened using the two available options in the File menu:

4.3. Queries

Figure 3 shows a Query dialog example.
 
 

Figure 3.  Query dialog example.

The Query dialog provides a powerful means for subsetting data using the following search criteria:

4.3.1 Expert Ratings

The expert ratings constitute a powerful mechanism of selecting subsets of RPI data of particular scientific content by the online referring to the knowledge of the established experts in the field. The experts are able to login into the ratings database to submit their ratings, which then become available for BinBrowser users.

The following category names are provided for the ratings:

The available ratings are: The EXPERT RATINGS box becomes enabled after the checkbox "Search only rated plasmagrams" is checked. The ratings conditions are written into the text part of the EXPERT RATINGS box, using a simple expression language described below. If the box is left empty, the database search returns all rated plasmagrams within the specified period of time.

4.3.2 Query Form

A catalog query can be entered in the QUERY FORM box using the expression language described in the previous section. There are two kinds of search variables for the query, RPI Program parameters and Predicted geospace data. The following list of search variables is effective as of January 2001:

RPI program parameters:

Geospace prediction data: The up-to-date list of available search variables for the queries is provided in the Query instructions section of the query dialog.

4.3.3  Expressions

Both queries and expert ratings conditions are written using a simple expression language which contains: An example of query would be:
program_no = 33 and pp.dpla < 2.5

4.3.4 Sending queries

Similar to the File operations, queries are initiated using the two available options in the File menu:

4.4. Configuring BinBrowser for Internet Operations

BinBrowser always makes an attempt to establish connection with the UMLCAR Online RPI Database. There are four categories of data stored centrally at UMLCAR to be retrieved over the Internet:
  1. The latest MET-to-UT conversion table ("Clock Drift" file)
  2. Definitive orbital parameters of the IMAGE spacecraft
  3. Session Lists (in response to the queries)
  4. Raw L0 data
If the Internet connection is not available or Interclient port 3060 is blocked by a firewall, BinBrowser allows only local operations with one-day files (Scenario (A), Table 1). In this case the orbital parameters are taken from the RPI data headers (those are CIDP-generated) and the local copy of clockdrift.asc file is used. Use of an outdated clockdrift.asc file will result in incorrect UT displays in BinBrowser. Clock Drift updates are available from SMOC.

If the Internet connection can be established, BinBrowser:

  1. Requests the latest Clock Drift information (once)
  2. Requests definitive orbital parameters (for each session), unless "Use Orbital Data from Catalog" is disabled in the Options menu
  3. Enables catalog queries
  4. Allows raw L0 data transfers
Transfer of the raw L0 data over Internet is arranged if: The local archive should store the standard Level-0 files from SMOC, named in accordance with the following SMOC convention:
im_l0_rpi_YYYYMMDD_v01.lz,
where YYYY is the Year, MM is the month (01-12), DD is the day (01-31). The files should reside: The archive home directory is specified in the query dialog (see Figure 3), using the platform-independent style:
 
 
/ directory separator
..  one directory level up
. current directory
: not allowed to use in the names or as the directory separator!
Examples:
 
D:/RPI/L0 (Windows)
/usr/local/rpi/data (Unix)
Lowell/Applications/Data/L0 (Macintosh)
../../../Data (platform-independent)

5. BinBrowser Screen Arrangement

 

 


 


Figure 4.  BinBrowser Screen Arrangement


5.1. Data Panel

The major part of the BinBrowser screen is taken by a Data Panel that displays, in a variety of graphics presentation modes and data representations, the contents of databins for the current session. The Data Panel is a tabbed panel that allows to switch between the presentation modes by clicking on the tabs on the top of the panel. The following tabs are provided in BinBrowser v2.0: The graphics presentation modes are discussed in a greater detail in subsequent section of the guide.

5.2. Headers Panel

The triple text panel, left of the data panel, displays Headers Panel contains information useful for the low-level browsing through the data, when the Data Panel shows one frequency at a time (Text, Line Graphs and Range Scans presentation modes). In Plasmagram, Echomap, and Dynamic Spectrogram modes, the Headers Panel displays information from the first available telemetry packet.

5.3. Expert Ratings and Browsing Controls

The bottom left panel of BinBrowser is used for the Expert Ratings in Plasmagram and Echomap tab views, and as the Browsing Controls panel in Text, Line Graphs and Range Scans tab views.

5.3.1. Expert Ratings

The Expert Ratings panel includes: The Ratings Controls are shown in Figure 5.

Figure 5.  Expert Ratings Controls


By default, the Ratings Content combobox is set to AVERAGE, so that the table of ratings shows averaged ratings from all experts that submitted their ratings to the database. The COUNT selection is used to view how many ratings were submitted, for each category. When an expert name is selected, the table shows the ratings submitted by the selected expert, and the Expert View button is enabled to set BinBrowser visualization options (representation, MPA threshold, color scales, etc.) to the settings suggested by the selected expert. Expert menu provides an option that allows to always apply the ExpertView settings (without pressing the button).

The Submit button is enabled only for the registered experts. Please read Section 7 below on Expert operations.

5.3.2. Browsing Controls

Browsing Controls provide means for low-level navigating within the range-frequency-repetition cube and, similar to the Header Panel, operate in the Text, Line Graphs and Range Scans presentation modes only, when one frequency data are displayed in the Data Panel.

Figure 6.  Browsing Controls









One frequency data can be shown in the following ways:

Left and right arrows change frequency of the displayed databins. Up and down arrows behave differently, depending on the choice of (i), (ii), or (iii) scan types: In the Text and Line Graphs presentation modes, the choice between (i) and (ii) scan types is made using the Scan Type Selector.
 

5.4. Presentation Options

Presentation Options include Color Scale combobox allows a choice of color scales for data representation.

Thresholding controls enable cleaning of the data display which is arranged dynamically by evaluating the noise level of each frequency individually and adjusting the cleaning threshold by adding a fixed specified number to the noise level estimate.

Representation selector allows display of the contents of the databins in a variety of representations.
 

6. Plasmagram Trace Editing


Figure 7.  Plasmagram Trace Editing

BinBrowser allows highlighting, or scaling, of the plasmagram traces to further invert them to the plasma density contour. It is possible to scale more than one trace off the same plasmagram, but only one trace can be made visible at a time. There are three controls on the top of plasmagram image:

To scale a trace, first set start and stop frequencies using the drag controls ("rings") below the plasmagram, as shown in Figure 7. To change positions of the start/stop frequencies, position the mouse pointer at the drag control, press left button and drag the mouse. To underscore the desired trace, press and hold the left button, dragging the pointer immediately below the leading edge of the trace.

The scaled infromation is saved to a disk as an RSI file from the Current Session menu. RSI files can be used with an offline profile inversion program.
 

7. Expert Operations

The registered experts are allowed to submit their Ratings and Expert Views. Please contact Ivan_Galkin@uml.edu to register as the expert.

It is necessary to login as the expert and select your name in the Ratings Content combobox to be able to submit information to the ratings database.
In this case the table ratings become selectable, and the Comments section isenabled for editing. Typically, an expert is suggested to

  1. adjust the BinBrowser visualization setting to the optimal view,
  2. select the ratings,
  3. write comments, and
  4. press Submit button.
Alternatively, The Expert menu provides "Submit changes automatically" option, so that BinBrowser detects the changes to the ratings and submit them automatically. Please note that ExpertView settings are submitted together with the ratings. ExpertView data can be submitted without changes to the ratings by pressing the Submit button.
 

8. PostPro1 Utility

The PostProcessorOne, or PostPro1, a part of the RPIAnywhere installation, is a non-interactive ("batch-mode") version of BinBrowser program which reads input Level 0 files and produces: PostPro1 can be used for mass production of the browse products and CDF files.

8.1 Configuring PostPro1

All PostPro1 settings are stored in a PostPro1.ini file located in the RPIAnywhere home directory. PostPro1.ini file is a text file which can be edited using a standard text editor.

8.1.1. Directories

PostPro1 reads Level 0 files from an incoming directory, stores the BPs, CDF files, and Log files in an outgoing directory, and uses one temporary (working) directory for miscellaneous file operations. Corresponding entries in the PostPro1.ini file are:


Figure 8.  PostPro1 testing its incoming directory for new L0 files to process.

Every 10 seconds PostPro1 tests its incoming directory for new L0 input files. If a file is discovered, it is processed and deleted.

Please make sure that PostPro1 is not running while you place files into the incoming directory, otherwise a source file might get
opened and scanned for sessions prematurely.

8.1.2. Browse Products Selection

It is possible to specify which BPs are to be created by setting to true or false the following entries in the PostPro1.ini file:

8.1.3. Plasmagram Creation Options

The following is an example of the plasmagram BP processing options: PostPro1 will be producing 800x600 pictures. The plasmagrams will be cleaned by thresholding them at the level of MPA+4. Each picture bin will be filled with a rectangle taking 100% of the available bin area.

8.1.4. Echomap Creation Options

The following is an example of the echomap BP processing options: PostPro1 will be producing 800x600 pictures. The echomap data will be cleaned by thresholding at the level of MPA+4. The size of the echomap is 20 x 20 Re.

8.1.5. Dynamic Spectrogram Creation Options

The following is an example of the dynamic spectrogram BP processing options: PostPro1 will be producing 800x600 pictures. DS channel can be set to:
0 (Antenna X)
1 (Antenna Y)
2 (Antenna Z)
3 (Combined X and Y antennas)
DS value can be set to:
0 (Amplitude 1 in the databin)
1 (Amplitude 2 in the databin)
2 (Amplitude 3 in the databin)
3 (Amplitude 4 in the databin)
4 (Amplitude 5 in the databin)
5 (Amplitude 6 in the databin)
6 (Amplitude 7 in the databin)
7 (Amplitude 8 in the databin)
8 (Median of 8 amplitudes in the databin)

8.1.6. Movie Frames Creation Options

8.1.7. Picture Format Options

There are two options specifying the format of the pictures: ImageFormat can be set to 0 (GIF) or 1 (PNG). Creation of PNG pictures takes considerably longer time. PngCompressionLevel is a number between 0 and 9 specifying the level of compression.

8.1.8. CDF Options

Set MakeCDFs to false to disable CDF creation.

8.1.9. Other Options