Contact: UNAVCO data guru
BINEX, for "BINary EXchange", is an operational binary format standard for GNSS research and operational purposes. The format has been designed to grow and allow encapsulation of any data or metadata allowed in the common ASCII exchange formats such as RINEX, IONEX, SP3, SINEX, and so on, including GNSS-related data and metadata as encountered.
BINEX is a collaborative format specification, partnering with the UNAVCO community and interested receiver manufacturers. A partial list of participants currently includes:
Although a reality since circa 2000, BINEX should be viewed as an ongoing design, with a long-term goal of a compact binary replacement for existing ASCII exchange formats (e.g. RINEX, SINEX, IONEX, SP3, etc.).
Some of the design goals of BINEX:
There are really two distinct phases to defining BINEX:
The first task had to be done very correctly, because once the generalized record structure was defined and started in use, there could be no returning to correct an earlier oversight. The generalized record structure is fairly simple, yet flexible. Since there is no "version" numbers in BINEX records, a BINEX parser should be able to parse any BINEX file or data stream, i.e. the parser should be able to:
The parcer should identify valid records, but not necessarily be able to "translate" specific records.
Please consult the Web page on BINEX conventions used in this documentation before proceeding, or, if you get confused, come back to this first.
There are two sets of possible records depending on the level of CRC (cyclic redundancy check) needed: one with what is called the regular CRC model and another with what is called the enhanced CRC model.
The two designs for the regular CRC generalized record structure are:
1 byte: synchronization byte, also containing little/big endian bit for record 1-4 bytes: record ID 1-4 bytes: record message length in bytes [1-4 bytes]: bit-flipped 1-4 bytes of record message length (enhanced CRC only!) n bytes: record message 1-16 bytes: checksum or CRC (of ID, length, and message bytes)
1 byte: leading synchronization byte, also containing little/big endian bit for record message 1-4 bytes: record ID 1-4 bytes: record message length in bytes [1-4 bytes]: bit-flipped 1-4 bytes of record message length (enhanced CRC only!) n bytes: record message 1-16 bytes: checksum or CRC (of ID, length, and message bytes) 1-4 bytes: total number of bytes in: leading synchronization byte record ID record message length record message checksum or CRC with bytes in reverse order 1 byte: terminating synchronization byte, also containing little/big endian bit for record message
Without worrying about specific details at this point, there is a fundamental difference between these two designs: the first allows one to easily read a BINEX file in the forward direction (consistent with the model used by nearly all native GPS/GLONASS/SBAS formats)--though reading in a backward direction is still possible; the second allows one to easily read a BINEX file either forwards or backwards, but the size of each record is slightly inflated (probably by 3 bytes, in most cases).
The design of the enhanced CRC generalized record structure allows for yet more secure and reliable data transfer by 1) providing a bit-flipped version of the 1-4 record message length bytes immediately after the 1-4 record message length bytes (i.e. XOR-ing the two fields will give all 1's) and 2) having an upscaled CRC for the entire record.
A quick aside: Why worry about being able to read a file backwards? The analogy here is determining the start and end epochs (times) of a RINEX OBS file. The start time, by definition, is a required RINEX OBS header field, whereas the end time is an optional RINEX OBS header field--and either may correctly or incorrectly represent the actual start and end times in the file. In fact, because the header metadata is unreliable, teqc searches the actual time tags in the file to determine the start and end times (ignoring the header metadata for this). It turns out the one of nice things about RINEX is that one can (usually) read a file backwards. So, determining the start and end epochs directly from the time tags does not require reading an entire file forwards to the end. For some applications, this saves considerable CPU time.
Note: What would be the typical byte-count overhead for each epoch's worth of GPS/GLONASS/SBAS data represented in BINEX? Usually, about 6 bytes (for the first of the above structure models), independent of the number of satellites being tracked for that epoch. For ConanBinary (a fairly compact representation), this compares with 5 x number of satellites being tracked, e.g. if 10 satellites are being tracked, 50 bytes are being used in ConanBinary to define the record structure for a single epoch's worth of data. Additionally, in BINEX the 6 bytes includes a checksum or CRC for record security, which is totally absent in ConanBinary.
BINEX will utilize both models of the generalized record structure: the key is the synchronization bytes. In short, a different synchronization byte is used for the two models. The upshot is that the bulk of a BINEX file could be composed of the "forward" readable records, but a reversible record could be appended on to the end of the file. (If one where to try to read a "forward" readable record backwards, one would quickly find that this was invalid; e.g. the last byte would not be equal to the defined terminating synchronization byte, the CRC for the record would be wrong, etc.)
The current design is that the synchronization byte of any "forward" readable record is:
and the leading synchronization byte of any "reverse" readable record is:
and the terminating synchronization byte of any "reverse" readable record is:
(Note: These values were selected by using the ASCII hexadecimal values for the letters
after adding 0x80 (= 128). The terminating values are obtained by reversing and negating the bits of 0xd2, 0xf2, 0xd8, and 0xf8. More importantly, all these hexadecimal values also turn out not to be in conflict with any usual leading bytes of other common GPS native formats.)
One of the driving philosophies for BINEX is extensibility. Consequently, the design isn't pigeon-holed in the number of possible BINEX record types that are allowable. With the unsigned BINEX integer (ubnxi) method of using one to four bytes for the record ID, a total of 2^29 ~= 500 million different record types are possible, though usually only one or two bytes will be necessary. Additionally, most of these records will probably have a subrecord ID (usually only one byte will be necessary), which essentially allows for almost 3e17 possible record-subrecord combinations--far more than enough for the foreseeable future!
The one-byte record IDs (0-127) are reserved for public domain standardized records, i.e. for the type of information currently in RINEX, IONEX, SP3, SINEX, etc. files.
The multi-byte record IDs (128-536870911) have initially assigned for private use in blocks of 4 record IDs on a request basis, allowing over 4000 such requests before the two-byte record IDs were used up. The idea here is that if JPL, say, wants a block of BINEX record IDs for internal use. They might estimate they need 20 or so records. Five blocks of 4 records IDs would then be assigned to them. Any BINEX parser should be able to parse a file containing these JPL BINEX records once the records were in use; the records would just be recognized as private and skipped. If, at a later date, the organization that requested a private block of IDs might develop one or more of their private records to the point where they felt the general community could make use of them. Then the specified records would move into the public arena with the necessary documentation so that a BINEX reader could actually translate those records.
At this time:
The record message length is also represented using the unsigned BINEX integer (ubnxi) method of using one to four bytes. Thus, individual records of up to 0.5 Gbytes would be possible, requiring 4 bytes to store the length, though most records will probably only require one or two bytes for the record message length. The value for the record message length specifies the number of bytes in the record message which immediately follow the bytes for the record message length.
Immediately following the record message length byte(s) is the record message. The format of each record message depends on which type of record it is (identified, of course, by the record ID).
Each record contains a record checksum that is generated from all of the bytes in the record ID, the record message length, and the record message itself. The decision of which type of checksum to use (and hence, the number of bytes in the checksum) is based on the total number of bytes covered by the checksum, and on whether the record format uses regular CRC or enhanced CRC. The design for this sum is:
C/C++, Perl, and Fortran code for the 1-, 2-, and 4-byte checksum/CRC cases has already been written and tested.
For reversible record structure records, i.e. the leading synchronization byte is either 0xd2 or 0xf2, the record length bytes are repeated in reverse order from what they are at the first part of the record. Note that only the byte order is reversed; the bit-order within each byte remains the same.
Depending on the resolution needed, a variety of time stamps can be used within various BINEX records, subrecords, and/or fields. All measure time from 6.0 Jan 1980, coincident with the start of GPS time. All "minutes" are understood to be standard 60-second minutes. By expressing the basic time in minutes as an uint4 (unsigned 4-byte integer), the time stamps have a range of slightly over 8166 years (so starting in 1980, are usable until 10146 A.D.!). Depending on the time resolution required for a particular use, the seconds are either ignored or represented in a variety of ways:
The assignment of the first few public records IDs are fairly straightforward:
(Only another 121 public records to go!)
At this point, the above BINEX record outlines should be intepreted as complete up to the point that they have been defined, though generally allowing for additions to be made in the future.
The forward parsing of all BINEX files or data streams follows the same algorithm, and makes use of the generalized BINEX record structure. The first valid byte will be the synchronization and endian byte, either:
Search the file or data stream from the start forward until one of these bytes is located. A possible synchonization/endian byte has now been found. At this point, it is already assumed that user knows the "endian-ness" of the processor being used. With the reading of the synchronization/endian byte, the user should establish the values of two other boolean values:
The next 1-4 bytes will establish the record ID of the record. The C/C++ function read_ubnxi() can be used to do this.
The next 1-4 bytes will establish the length of the message in the record. The C/C++ function read_ubnxi() can be used to do this.
Next, the number of bytes specified in the length of message are now actually read. These bytes form the body of message which will have to be decoded on a per record basis.
Next, 1-16 bytes will be read which form the checksum or CRC for this particular record. The number of bytes used in the checksum/CRC depends on the total number of bytes needed for the record ID, length of record message, and the record message itself. The value of the checksum/CRC should be verified against the checksum or CRC generated by the actual bytes collected for this record in the record ID bytes, the length bytes, and the record message bytes. If there is not 100% agreement, integrity of this record is suspect, or the synchronization/endian byte selected was actually another part of a valid record. Start at the byte after the suspect synchronization/endian byte selected for this pass, and search for the next possible synchronization/endian byte and repeat the process until a valid checksum/CRC match occurs.
For a synchronization/endian byte value of
the user has now successfully read in an entire forward readable only BINEX record, and can proceed with decoding the contents of the message bytes according to the value of the record ID value.
For a synchronization/endian byte value of
the user must complete the reading of the tail end of the current record, since these leading synchronization/endian byte values indicate a forward and backward readable record. This involves reading the number of length bytes for the message again. The value of the bytes read should be identical to the value of the bytes read earlier in the record for the length, except the bytes are in reverse order. Next the terminating synchronization/endian byte is read.
If the terminating length bytes and the terminating sychronization/endian byte are what they should be, the user has now successfully read a forward and backward readable BINEX record in the forward direction, and can proceed with decoding the contents of the message bytes according to the value of the record ID value.
Prototype BINEX software is available. This consists of C/C++ functions, which can be used as is or modified to create your own.
Also, Doug Hunt (dhuntucar.edu) at COSMIC/UCAR has modified, added to, and improved on this prototype code for the standardized reading and writing of BINEX files, resulting in a BINEX library which can be used in C, Fortran, and Perl code. See:
Testing of standard UNIX and GNU compression programs has been tested on BINEX files made up of mostly 0x7f-00 records (GPS observables), with a few 0x01-01 records (GPS navigation messages). The compression programs gzip and zip, both with and without the optimal compression using the -9 option, compressed the test files nearly equally and seem to have the most effect, compressing files by 20-30%. The compression program compress has little effect, compressing files by only a few %. In all cases, the test files with big-endian records compressed slightly better than the files with little-endian records of the same data (and we have no idea why this happens).
If you want to be included in the BINEX email forum, please go to the
ls.unavco.org / mailman / listinfo / binex support page and subscribe.
(Remove spaces from the italicized address. We have not hyperlinked the preceding URL to help prevent spammers, web-bots, and harvesters from learning about mail addresses at UNAVCO.) Once subscribed, emails about BINEX questions and issues should then be sent to:
You will be notified of BINEX related issues, etc.
Last modified: Friday, 02-Dec-2016 19:31:47 UTC