Data Recording Using Commercial Servers – Tutorial

Home   >   Tutorials   >   Data Recording Using Commercial Servers – Tutorial
Data Recording Using Commercial Servers – Tutorial

Introduction to Data Recording Using Commercial Servers

Legacy data recorders were originally implemented as custom hardware-based systems. Many of these systems have since migrated to an architecture that combines an industrial PC with data acquisition modules and custom front panels. There are now new software-de ned data recorders on the market that leverage low-cost commercial server platforms for the underlying computing/networking/storage hardware.

Satellite ground systems and range data systems record the telemetry data at one or more points in the downlink processing stream. For satellite systems, command data may also be recorded. Data may be recorded at the antenna site so that the data is not lost in the event of a network outage. Data may also be recorded at the antenna site if the downlink rate exceeds the network capacity. In addition, data is nearly always recorded after decryption and placed into data archives that can be used to support detailed analysis.

SLE Services

Data recorders capture the data streams on each of their input channels first in memory and then write this data to files stored on disk drives. Typically, each channel is written to a separate file. A high rate channel running at 1 Gbps quickly creates large raw data files! To avoid overly large files, long duration recordings may be split across multiple physical files on the disk with the data recorder closing one file and starting another during the recording. This happens under the covers without data loss.

Data recorders store meta data along with the raw input data. This meta data is usually stored in a separate set of files. Meta data allows the data recorder to replay the raw data. For each block of data written to the raw data file, the corresponding meta data would include information such as the arrival time of the first bit, the number of bits in that segment, and the duration of those bits.

This meta data allows the recorder to replay the raw data stream file with the same data rate and timing as when the data was originally recorded. It also allows the replay to start and stop at any point in the recording.


Data Storage

Commercial servers offer a wide range of disk storage options, both in performance (recording speed) and reliability. With a software-de ned recorder, the storage subsystem can be sized and configured to specific needs.

RAID storage can be used for applications that require a near 100% guarantee against data loss.

The RAID controllers in commercial servers also allow the data files to be mirrored (RAID 0) or striped (RAID 5) across multiple disks to greatly increase the data recording bandwidth when compared to a single disk drive. With RAID 5, parity information can also be striped across the disks to automatically handle a drive failure without data loss.

File Management

Data recorders include file management functions that enable the user to assign names to each recording and allow files to be Exported and Imported. Exporting a file from one data recorder and Importing it to another allows the second recorder to replay the recorded data.

File management functions also include what to do in the event that the disk storage is becoming too full. For example, the data recorder can be configured to delete the oldest recordings first if necessary to make room for the current recording. Data recorders allow the user to “protect” specific recordings so that they are never deleted as part of the file management process.

With a commercial server, files can also be moved using the operating system’s capabilities to copy, delete, and relocate files. Recordings can be copied to removable media (e.g. CD, removable drive) or transferred across the network (e.g. FTP).


Playback from the data recorder is used to regenerate and output data streams. The user can select all or a subset of the channels captured in a particular recording to be included in the playback. Typically playback is performed at the original data rates, but faster or slower playback rates can be set. This allows a low data rate stream to be processed in less time.

An important feature of the meta data files is that they allow the data recorder to accurately reproduce the data stream, including dead clock periods, static data, and data rates shifts.

Software-Defined Data Recorders

Software-de ned data recorders offer several nice features over point solutions. Other processing functions are easily added to the software-defined data recorder. One example is the ability to monitor data quality during the recording. If the telemetry data is unencrypted, the data recorder can include a frame synchronizer that provides a real time indication of whether or not the data is good.

Custom data quality checks can also be implemented. Data quality statistics can be displayed and stored with the meta data.

And this works both ways. Data recording can be added to other software-de ned systems such as modems, gateways, and front end processors. The modem becomes a modem/data recorder. The front end processor becomes a FEP with integrated data recording.

Other Advantages of Commercial Servers

Commercial servers have multiple advantages over industrial PCs for applications and environments that do not require a ruggedized data recorder. These include:

Current Technology: The commercial server market dwarfs both the industrial PC and the data recorder markets. There’s a much higher level of capital investment in being competitive by offering the latest technology. Commercial servers are always one or two technology generations ahead of industrial PCs.

System Management: Commercial servers from companies such as Dell, HP, and IBM have system’s management tools that run diagnostics, report issues, perform monitoring, etc.

Lower Cost: The market size again creates an advantage in that commercial servers are lower in initial cost. They also have worldwide support systems in place that provide service convenience and reduce the long-term cost of maintenance.

Technology Refresh: Software-de ned recorders running on commercial servers afford a simpler approach to technology refresh when the existing system has grown tired. The data recording application and any signal acquisition modules are simply migrated to a current server platform.

Can we help? AMERGINT’s expertise is available to assist in your systems engineering and design

Randy Culver