 |
RPI BinBrowser
User's Guide |
|
U. MASS Lowell
Center for Atmospheric Research
Last changed: January 17, 2001
Please e-mail suggestions and corrections
to Ivan_Galkin@uml.edu
|
|
1. RPI Data Background
1.1. RPI Measurement is a Cube
RPI is a radar with the variable frequency. In the most of its measurement
modes, RPI sends a pulse and listens for the echoes. On each particular frequency,
RPI can send more than one pulse. An RPI measurement can therefore be visualized
as a cube:
Figure 1. RPI Measurement Cube
In this particular example, RPI used 7 frequencies, sampled the reply
at 6 ranges for each frequency, and repeated the measurements 4 times per
frequency.
There are also listen-only measurement modes available in RPI.
where no pulse is transmitted. The data collected in these modes are still
arranged in terms of the frequency - range- repetition "cube". See section
1.5 for more details.
1.2. Databin Concept
RPI measurement "cube" consists of databins, where each databin has a unique
combination of frequency, range and repetition. An example on Figure 1 contains
7 x 6 x 4 = 168 databins. The databin contents depends on RPI measurement mode:
|
Abbreviation
|
Name
|
Contents
|
Diagram
|
Comments
|
| LTD |
Linear-scale Time-domain Data |
12-bit linear scale quadrature components, 2 per antenna, 3 antennas |
LTD.GIF |
|
| SSD |
Spectral-domain Science Data |
Three 8 bit log-scale amplitudes (per antennas) and two 8 bit linear
scale phase differences |
SSD.GIF |
|
| SMD |
Spectral-domain Maximum-amplitude Data |
Three 8 bit log-scale amplitudes (per antennas), two 8 bit linear scale
phase differences, and one 8-bit Doppler number |
SMD.GIF |
Only one Doppler line is reported out of all collected |
| PRD |
Precision Range Data |
Three 8 bit log-scale amplitudes (per antennas), five 8 bit linear
scale phase differences, and one 8-bit Doppler number |
PRD.GIF |
|
| CAL |
Calibration |
One 8 bit log-scale amplitudes (per antennas) and one 8 bit linear
scale phase per antenna, 3 antennas |
CAL.GIF |
Calibration packet only |
| DBD |
Double Byte Data |
One 8 bit amplitude and one 8 bit Doppler |
DBD.GIF |
Low data volume option |
| SBD |
Single Byte Data |
One 5 bit amplitude and one 3 bit Doppler |
SBD.GIF |
Low data volume option |
| TTD |
Thermal-noise Time-domain Data |
Eight 8-bit log-scale averaged amplitudes and two cross-power terms
per antenna, 3 antennas |
TTD.GIF |
|
1.3. Auxiliary Information: Preface and Frequency Headers
Apart from the set of databins, each RPI measurement has to be accompanied by
auxiliary information that includes
-
Preface: RPI program settings and CIDP orbital information
-
Frequency Headers: auxiliary data collected at each frequency (e.g.,
antenna currents and voltages), and frequency specific system settings
(gain, frequency search, MPA, etc.)
1.4. Telemetry Packaging
RPI data stream has to be partitioned into the fixed length telemetry packets.
The design requires robustness to the telemetry drop-outs, i.e.
loss of any packet in the sequence of packets from an RPI measurement must not
cause loss of the whole packet sequence or any of its remaining parts.
To fulfill telemetry packaging requirements, all databins collected
on a frequency are enumerated and the serial number of the first
databin in the packet is stored in a data header in the beginning
of each packet, so that its frequency, range, and repetition number can
be restored independently of previous packets..
1.5. Organization of TTD data
In the thermal noise mode, RPI samples antenna outputs without transmitting; the
sampled quadratures undergo averaging and calculation of the cross-power terms.
TTD databin stores 8 averaged amplitudes per antenna; there may be more than 8
amplitudes collected on a particular frequency. To follow the standard data arrangement
shown on Figure 1, the "cube" of TTD databins has 1 range, and the additional
databins for the same frequency are organized in the repetition plane.
2. Data Sources
RPI BinBrowser operates with the following data sources:
-
Raw Data:
-
Level 0 files (instrument packets),
-
Level 0.5 files (UDF)
-
Level 1 file (CDF)
-
Scaled Data:
-
RPI Scaled Information (RSI)
IMPORTANT: BinBrowser v2.0 reads UDF files in its original
"Little Endian" arrangement, as created at IMAGE SMOC. If the UDF Data
Installation procedure changes the Endian to "Big" to allow UDF software
to read files on your computer, BinBrowser v1.4 will not be able to identify
the change of the Endian arrangement and read those files. In this case
instead of using the UDF Data installation procedure, un-gzip the original
UDF bundles with RPI data and place the files on your disk manually.
The source data can be acquired:
-
as one-day files residing on any media: hard disk, CD-ROM, DVD-ROM, etc.,
using one of the following:
-
online via Internet from the UMLCAR Online RPI Database (Level 0 only)
The UMLCAR RPI Data Catalog provides an elaborate means for subsetting
the RPI data, as discussed in detail below.
3. RPI Sessions
3.1. Concept of Session
One RPI session contains all raw data collected by RPI during
a single frequency scan from the lower frequency limit, [L], to the upper
frequency limit, [U], plus the derived, or scaled information (plasmagram
traces, electron density profiles, etc.) created interactively by a BinBrowser
operator.
3.2. Session List
At any given instance, BinBrowser has access to a particular subset of RPI data
acquired from the data sources, called session list. The session list is
assembled by BinBrowser if the source data are taken from files, or by the RPI
Data Catalog server in reply to a query, as discussed below. The session
list is used to navigate through the currently available data subset (see Figure
2).
Figure 2. Session Selector
After the session list is compiled, the chronologically first entry
is displayed as the current session. The droplist control is used to "drop"
the list of available sessions to select the current session. Also, four
buttons are available to walk the session list forward and backwards.
3.3. Twin Sessions
When on a particular schedule, RPI goes through a variety of measurement programs.
When BinBrowser compiles the session list, all sessions are presented in the time-ordered
sequence. To help with navigating the list, the concept of twin sessions
is introduced, where twin sessions are simply subsets of sessions with the same
RPI program number. Stepping through the twin sessions gives a consistent presentation
of the RPI similar measurements on orbit.
4. Files and Queries
4.1. Data Management Scenarios
There are three main scenarios of the data management within the BinBrowser 2.0,
(A), (B), and (C):
Table 1. Data Management Scenarios
|
Scenario:
|
Session List created by:
|
Raw data coming from:
|
Format:
|
Comments:
|
| (A) |
BinBrowser |
Local disk, network disk, removable media. |
L0, L0.5, L1, RSI |
Local one-day files directly opened by "Open File" or "Add File" command. |
| (B) |
UMLCAR RPI Catalog |
Local disk, network disk, removable media. |
L0 |
A catalog query issued over the Internet to subset the data and build
the session list. The L0 data files are available locally. |
| (C) |
UMLCAR RPI Catalog |
UMLCAR Online RPI Database. |
L0 |
A catalog query issued over the Internet to subset the data, build
the session list and retrieve the data, session-by-session, over the Internet. |
Scenario (B) is the most convenient as it combines the search/subsetting
features with the speed of the local disk access to the raw data. Scenario
(C) is used for the situations where the raw data are not available locally,
and obtaining them in form of the one day files from the RPI data distributors
is not feasible. In this case the current session data is obtained from
the UMLCAR RPI Online Database over the Internet and stored in a local
disk cache to avoid re-sending of the data next time the session is requested.
Scenario (A) is typical for off-line work with the local files.
All scenarios for data management can be utilized simultaneously. For
example, a query can be sent to retrieve a session list, and if only a
part of session data can be found locally, (B) and (C) scenarios will be
combined to acquire all relevant data; then, a local file with RSI information
can be added using the (A) scenario.
4.2. File operations
Files are opened using the two available options in the File menu:
-
New File (current Session List is discarded),
-
Add File (sessions in the opened file are added to the current Session
List).
4.3. Queries
Figure 3 shows a Query dialog example.
Figure 3. Query dialog example.
The Query dialog provides a powerful means for subsetting data using
the following search criteria:
-
Time interval
-
Expert Ratings
-
RPI Program parameters
-
Predicted geospace data
4.3.1 Expert Ratings
The expert ratings constitute a powerful mechanism of selecting subsets
of RPI data of particular scientific content by the online referring to the knowledge
of the established experts in the field. The experts are able to login into the
ratings database to submit their ratings, which then become available for BinBrowser
users.
The following category names are provided for the ratings:
-
PP -- plasmapause echoes
-
MP -- magnetopause echoes
-
CUS -- cusp echoes
-
RES -- resonances
-
DIR -- echo directions
-
DUC -- ducted echoes
-
OTH -- other
The available ratings are:
-
NR -- no rating given
-
0 -- no signature
-
1 -- probable signature
-
2 -- good signature
-
3 -- high confidence signature
The EXPERT RATINGS box becomes enabled after the checkbox "Search only rated plasmagrams"
is checked. The ratings conditions are written into the text part of the EXPERT
RATINGS box, using a simple expression language described below. If the box is
left empty, the database search returns all rated plasmagrams within the specified
period of time.
4.3.2 Query Form
A catalog query can be entered in the QUERY FORM box using the expression language
described in the previous section. There are two kinds of search variables for
the query, RPI Program parameters and Predicted geospace data. The following list
of search variables is effective as of January 2001:
RPI program parameters:
-
PROGRAM_NO - program number
-
SCHEDULE_NO - schedule number
-
D.NAME - databin type
-
T.NAME - housekeeping data type (SRD, HK, MSG, ECH)
-
LOWER_FREQ, UPPER_FREQ - frequency limits
-
COARSE_FREQ_STEP, FINE_FREQ_STEP - frequency stepping
-
NO_OF_FINE_FREQ_STEPS - number of fine frequency steps
-
START_RANGE - start range in 960km units
-
MAX_RANGE - maximum range in km
-
WAVEFORM_ID - Tx Waveform
-
ANTENNA_ID - Tx Antenna options
-
COUPLER_ON - Coupler mode: 1-on, 0-off
-
NO_OF_REPEAT - Number of Integrated Repetions
-
PULSE_REP_RATE - Pulse Repetion Rate
-
START_RANGE - Start range 960 km units (1Mm)
-
RANGE_STEP - 24 or 48 (10 km units)
Geospace prediction data:
-
PP.DMP - Distance to magnetopause in RE
-
PP.DPLA - Distance to plasmasphere in RE
-
PP.FCE - local electron cyclotron frequency
in kilo Hertz
-
PP.FPE - local electron plasma frequency
in kilo Hertz
-
PP.FMP - maximum frequency in kilo Hertz
of the echo from the MODEL magnetopause
-
PP.PFMP - maximum power flux in Watts per Meter
squared from the MODEL magnetopause
-
PP.MLT - magnetic local time in hours
-
PP.MLAT - magnetic latitude in degrees
-
PP.ILAT - ivariant latitude in degrees
-
PP.L - dipole lshell value in
units of earth radii
-
PP.R - radial position in RE
-
PP.XSM - x position of spacecraft in RE in
solar magnetic coordinates
-
PP.YSM - y position of spacecraft in RE in
solar magnetic coordinates
-
PP.ZSM - z position of spacecraft in RE in
solar magnetic coordinates
-
PP.XMP - x position of magnetopause in RE
in solar magnetic coordinates
-
PP.YMP - y position of magnetopause in RE
in solar magnetic coordinates
-
PP.ZMP - z position of magnetopause in RE
in solar magnetic coordinates
The up-to-date list of available search variables for the queries is provided
in the Query instructions section of the query dialog.
4.3.3 Expressions
Both queries and expert ratings conditions are written using a simple expression
language which contains:
-
the search variables (QUERY FORM box) or the rating category names (EXPERT
RATINGS box)
-
numbers
-
relational operators:
-
= (equal)
-
> (greater than)
-
< (smaller than)
-
>= (greater or equal)
-
<= (less or equal)
-
!= (not equal)
-
logical operators:
-
and (logical-AND)
-
or (logical-OR)
-
parenthesis
An example of query would be:
program_no = 33 and pp.dpla < 2.5
4.3.4 Sending queries
Similar to the File operations, queries are initiated using the two available
options in the File menu:
-
New Query (current Session List is discarded),
-
Add Query (found sessions are added to the current Session List).
4.4. Configuring BinBrowser for Internet Operations
BinBrowser always makes an attempt to establish connection with the UMLCAR Online
RPI Database. There are four categories of data stored centrally at UMLCAR to
be retrieved over the Internet:
-
The latest MET-to-UT conversion table ("Clock Drift" file)
-
Definitive orbital parameters of the IMAGE spacecraft
-
Session Lists (in response to the queries)
-
Raw L0 data
If the Internet connection is not available or Interclient port 3060 is
blocked by a firewall, BinBrowser allows only local operations with one-day
files (Scenario (A), Table 1). In this case the orbital parameters are
taken from the RPI data headers (those are CIDP-generated) and the local
copy of clockdrift.asc file is used. Use of an outdated clockdrift.asc
file will result in incorrect UT displays in BinBrowser. Clock Drift updates
are available from SMOC.
If the Internet connection can be established, BinBrowser:
-
Requests the latest Clock Drift information (once)
-
Requests definitive orbital parameters (for each session), unless
"Use Orbital Data from Catalog" is disabled in the Options menu
-
Enables catalog queries
-
Allows raw L0 data transfers
Transfer of the raw L0 data over Internet is arranged if:
-
the requested data are not available locally, or
-
the local archive is not properly arranged, or
-
the local archive folder is not specified in BinBrowser.
The local archive should store the standard
Level-0 files from SMOC, named in accordance with the following SMOC
convention:
im_l0_rpi_YYYYMMDD_v01.lz,
where YYYY is the Year, MM is the month (01-12), DD is the day (01-31).
The files should reside:
-
in the archive home directory, or
-
in the subdirectories YYYY_MM of the home directory, where YYYY
is the year, and MM is the month (01-12).
The archive home directory is specified in the query dialog (see Figure
3), using the platform-independent style:
| / |
directory separator |
| .. |
one directory level up |
| . |
current directory |
| : |
not allowed to use in the names or as the directory separator! |
Examples:
| D:/RPI/L0 |
(Windows) |
| /usr/local/rpi/data |
(Unix) |
| Lowell/Applications/Data/L0 |
(Macintosh) |
| ../../../Data |
(platform-independent) |
5. BinBrowser Screen Arrangement
Figure 4. BinBrowser Screen Arrangement
5.1. Data Panel
The major part of the BinBrowser screen is taken by a Data Panel that displays,
in a variety of graphics presentation modes and data representations, the contents
of databins for the current session. The Data Panel is a tabbed panel that
allows to switch between the presentation modes by clicking on the tabs on the
top of the panel. The following tabs are provided in BinBrowser v2.0:
-
Text
-
Line Graphs
-
Range Scans
-
Plasmagram
-
Echomap
-
Dynamic Spectrogram
The graphics presentation modes are discussed in a greater detail in subsequent
section of the guide.
5.2. Headers Panel
The triple text panel, left of the data panel, displays
-
Preface
-
Data Header of the current telemetry packet
-
Current Frequency Header
Headers Panel contains information useful for the low-level browsing through the
data, when the Data Panel shows one frequency at a time (Text, Line Graphs and
Range Scans presentation modes). In Plasmagram, Echomap, and Dynamic Spectrogram
modes, the Headers Panel displays information from the first available telemetry
packet.
5.3. Expert Ratings and Browsing Controls
The bottom left panel of BinBrowser is used for the Expert Ratings in Plasmagram
and Echomap tab views, and as the Browsing Controls panel in Text,
Line Graphs and Range Scans tab views.
5.3.1. Expert Ratings
The Expert Ratings panel includes:
-
Table of ratings,
-
Comments section, and
-
Ratings Controls.
The Ratings Controls are shown in Figure 5.
Figure 5. Expert Ratings Controls
By default, the Ratings Content combobox is set to AVERAGE, so that
the table of ratings shows averaged ratings from all experts that
submitted their ratings to the database. The COUNT selection is used to
view how many ratings were submitted, for each category. When an expert
name is selected, the table shows the ratings submitted by the selected
expert, and the Expert View button is enabled to set BinBrowser
visualization options (representation, MPA threshold, color scales, etc.)
to the settings suggested by the selected expert. Expert menu provides
an option that allows to always apply the ExpertView settings (without
pressing the button).
The Submit button is enabled only for the registered experts. Please
read Section 7 below on Expert operations.
5.3.2. Browsing Controls
Browsing Controls provide means for low-level navigating within the range-frequency-repetition
cube and, similar to the Header Panel, operate in the Text, Line Graphs and Range
Scans presentation modes only, when one frequency data are displayed in the Data
Panel.
Figure 6. Browsing Controls
One frequency data can be shown in the following ways:
-
(i) All ranges for a particular repetition (Text and Line Graphs presentations),
-
(ii) All repetitions for a particular range (Text and Line Graphs presentations),
-
(iii) All ranges and all repetitions (Range Scan presentation).
Left and right arrows change frequency of the displayed databins. Up and
down arrows behave differently, depending on the choice of (i), (ii), or
(iii) scan types:
-
(i) (All ranges are shown): Change repetition.
-
(ii) (All repetition are shown): Change range.
-
(iii) (All ranges and repetitions are shown): Up and down arrows are disabled.
In the Text and Line Graphs presentation modes, the choice between (i)
and (ii) scan types is made using the Scan Type Selector.
5.4. Presentation Options
Presentation Options include
-
Display Options panel with the Color Scale combobox and Thresholding controls,
-
Representation Selector to select one of the available data representations.
Color Scale combobox allows a choice of color scales for data representation.
Thresholding controls enable cleaning of the data display which
is arranged dynamically by evaluating the noise level of each frequency
individually and adjusting the cleaning threshold by adding a fixed specified
number to the noise level estimate.
Representation selector allows display of the contents of the
databins in a variety of representations.
6. Plasmagram Trace Editing
Figure 7. Plasmagram Trace Editing
BinBrowser allows highlighting, or scaling, of the plasmagram
traces to further invert them to the plasma density contour. It is possible
to scale more than one trace off the same plasmagram, but only one trace
can be made visible at a time. There are three controls on the top of plasmagram
image:
-
New trace (to add a new trace)
-
Trace Selector (to select currently visible trace from the list)
-
Remove trace (to delete the currently visible trace from the list)
To scale a trace, first set start and stop frequencies using the drag controls
("rings") below the plasmagram, as shown in Figure 7. To change positions
of the start/stop frequencies, position the mouse pointer at the drag control,
press left button and drag the mouse. To underscore the desired trace,
press and hold the left button, dragging the pointer immediately below
the leading edge of the trace.
The scaled infromation is saved to a disk as an RSI file from the Current
Session menu. RSI files can be used with an offline profile inversion program.
7. Expert Operations
The registered experts are allowed to submit their Ratings and Expert Views. Please
contact Ivan_Galkin@uml.edu to register
as the expert.
It is necessary to login as the expert and select your name in the Ratings
Content combobox to be able to submit information to the ratings database.
In this case the table ratings become selectable, and the Comments
section isenabled for editing. Typically, an expert is suggested to
-
adjust the BinBrowser visualization setting to the optimal view,
-
select the ratings,
-
write comments, and
-
press Submit button.
Alternatively, The Expert menu provides "Submit changes automatically"
option, so that BinBrowser detects the changes to the ratings and submit
them automatically. Please note that ExpertView settings are submitted
together with the ratings. ExpertView data can be submitted without changes
to the ratings by pressing the Submit button.
8. PostPro1 Utility
The PostProcessorOne, or PostPro1, a part of the RPIAnywhere
installation, is a non-interactive ("batch-mode") version of BinBrowser program
which reads input Level 0 files and produces:
-
Browse Products (GIF or PNG pictures):
-
Plasmagrams
-
Echomaps
-
RPI Movie frames
-
Daily Dynamic Spectrograms
-
Level 1 CDF files:
-
Plasmagram/Echomap data (k0)
-
Dynamic Spectrogram data (k1)
-
Fixed Frequency data (k2)
PostPro1 can be used for mass production of the browse products and CDF files.
8.1 Configuring PostPro1
All PostPro1 settings are stored in a PostPro1.ini file located in the
RPIAnywhere home directory. PostPro1.ini file is a text file which can be edited
using a standard text editor.
8.1.1. Directories
PostPro1 reads Level 0 files from an incoming directory, stores the BPs, CDF files,
and Log files in an outgoing directory, and uses one temporary (working) directory
for miscellaneous file operations. Corresponding entries in the PostPro1.ini
file are:
-
DataInputPath
-
DataOutputPath
-
DataWorkPath
Figure 8. PostPro1 testing its incoming directory for
new L0 files to process.
Every 10 seconds PostPro1 tests its incoming directory for new L0 input
files. If a file is discovered, it is processed and deleted.
Please make sure that PostPro1 is not running while you place
files into the incoming directory, otherwise a source file might get
opened and scanned for sessions prematurely.
8.1.2. Browse Products Selection
It is possible to specify which BPs are to be created by setting to true
or false the following entries in the PostPro1.ini file:
-
MakePlasmagrams
-
MakeEchomaps
-
MakeDynamicSpectrograms
-
MakeMovieFrames
8.1.3. Plasmagram Creation Options
The following is an example of the plasmagram BP processing options:
-
PlasmagramWidth=800
-
PlasmagramHeight=600
-
CleanPlasmagrams=true
-
PlasmagramThreshold=4
-
OptifontSizePL=100
PostPro1 will be producing 800x600 pictures. The plasmagrams will be cleaned by
thresholding them at the level of MPA+4. Each picture bin will be filled with
a rectangle taking 100% of the available bin area.
8.1.4. Echomap Creation Options
The following is an example of the echomap BP processing options:
-
EchomapWidth=800
-
EchomapHeight=600
-
CleanEchomaps=true
-
EchomapThreshold=4
-
EchomapSizeInRe=20
PostPro1 will be producing 800x600 pictures. The echomap data will be cleaned
by thresholding at the level of MPA+4. The size of the echomap is 20 x 20 Re.
8.1.5. Dynamic Spectrogram Creation Options
The following is an example of the dynamic spectrogram BP processing options:
-
DynamicspectrWidth=800
-
DynamicspectrHeight=600
-
DynamicSpectrogramChanel=3
-
DynamicSpectrogramValue=0
PostPro1 will be producing 800x600 pictures. DS channel can be set to:
0 (Antenna X)
1 (Antenna Y)
2 (Antenna Z)
3 (Combined X and Y antennas)
DS value can be set to:
0 (Amplitude 1 in the databin)
1 (Amplitude 2 in the databin)
2 (Amplitude 3 in the databin)
3 (Amplitude 4 in the databin)
4 (Amplitude 5 in the databin)
5 (Amplitude 6 in the databin)
6 (Amplitude 7 in the databin)
7 (Amplitude 8 in the databin)
8 (Median of 8 amplitudes in the databin)
8.1.6. Movie Frames Creation Options
-
MovieFrameHeight=150
-
MovieFrameWidth=150
8.1.7. Picture Format Options
There are two options specifying the format of the pictures:
-
ImageFormat
-
PngCompressionLevel
ImageFormat can be set to 0 (GIF) or 1 (PNG). Creation of PNG pictures takes considerably
longer time. PngCompressionLevel is a number between 0 and 9 specifying the level
of compression.
8.1.8. CDF Options
Set MakeCDFs to false to disable CDF creation.
8.1.9. Other Options
-
ReloadAfterThisManyFiles=4
-
UpdateFromCatalog=false
|
