Tips for Digital Signal Record and Playback
This article highlights some tips Viewpoint has learned in the design and build of digital record and playback systems based on NI FlexRIO equipment
Background
At an ever increasing rate, vast amounts of data are moved between device subsystems. Some examples of such subsystems are cameras, video displays, hard drives, and medical imaging. Even devices that don’t deal with massive levels of data still utilize digital signals at growing amounts, often for digital communications. This trend is creating a real need in device design and test to record and playback digital data streams. These streams are now vital for successful product design verification and debug and can produce better results than relying solely on simulations.
This article highlights some tips Viewpoint has learned in the design and build of digital record and playback systems based on NI FlexRIO equipment. The data rates achievable in practice are pretty phenomenal, especially while keeping all the signals tightly synchronized and aligned across multiple channels and clocks.
System Component Tips
Backplane
For perhaps obvious reasons, we recommend the PXI Express backplane for data rates faster than about 100 MB/s. One application Viewpoint developed is able to record and playback at up to 800 MB/s for each RAID drive
FlexRIO: Off the Shelf and Customization
Three components comprise a FlexRIO-based application.
- FlexRIO FPGA Modules: based on Xilinx FPGAs surrounded by onboard memory and streaming connections over PXI and PXI Express.
- FlexRIO Adapter Modules: front-ends to the FPGA typically containing circuitry for high-speed, high-channel analog and digital I/O.
- LabVIEW FPGA: enables design of the FPGA fabric to perform onboard processing and real-time analysis.
We recommend using the FlexRIOs with speed-grade FPGAs. These FlexRIOs handle higher clock speeds that can come in very handy for higher data rates and pre-processing prior to record or playback.
For custom needs, NI offers engineers the ability to create custom FlexRIO Adapter Modules (FAMs) and to incorporate CLIP nodes into the LabVIEW FPGA code. Custom FAMs can be incredibly useful in a digital record and playback system for steps as simple as connecting to your devices with product cables and connectors and as complex as signal preprocessing.
Also, since CLIP nodes are specially wrapped snippets of VHDL code, you can configure the innards of the FPGA in ways not exposed in LabVIEW and you can perform some basic signal manipulations before handing off the data to LabVIEW.
RAID Drive Array
The combination of large amounts of data and fast data rates require a digital record and playback system to use a RAID array. These drives can store TBs of data and accommodate multiple GB/s sustained reads and writes. These drives are typically external to the PXI chassis and connect to the chassis through an interface card, such as a MXI-Express. Also, in our experience, the faster RAID drives are worth the cost difference.
Software
The three LabVIEW flavors, LabVIEW, LabVIEW RT, and LabVIEW FPGA, are used in these applications. The host PC runs LabVIEW to monitor and configure the realtime application. The realtime layer manages the data collection (or playback) by interfacing with the FlexRIO FPGAs. The host layer running Windows manages the RAID array reading and writing. The FPGA layer manages all the bytes streams and bit processing. Any engineer tackling digital record and playback applications needs to be very familiar with these tools.
Software Tips
As might be expected, most of the challenges appear with code around the FPGA. Special care is needed to assure efficient use of the FPGA fabric. Furthermore, efficient methods must be used to get data off the FPGA into PC memory (and eventually onto the RAID drives) to handle high data rates and preventing lost data from buffer overflows.
FPGA Layer
Technically, the biggest challenge in these Digital Record and Playback systems deals with the integration of the CLIP nodes with the LabVIEW FPGA code. Developing the CLIP nodes requires good understanding of VHDL and, potentially, a good understanding of the Xilinx chipset capabilities.
If you are using external clocking, avoid a mistake we made in early projects: use only internal clocks or FAM-based oscillators as base clocks for the top-level FPGA application. Otherwise, as we discovered the hard way, the FPGA can lock-up when an external clock stops.
Also, if multiple clocks are needed, follow design rules carefully by using FIFOs to decouple clock domains.
Using external clocks or DDS clocks (from a timing module) is ok for some loops as long as controls/indicators are not used in those loops. Instead, input/outputs should be buffered through global variables or registers to the controls/indicators executing in a loop clocked by a permanent clock.
Use the provided base clocks for high speed loops whenever possible. Derived clocks may not be able to achieve the same timing as the provided base clocks.
Be aware of how many DMA channels will be required before purchasing hardware. Some FlexRIO models have significantly fewer channels than others, and don’t assume that a newer FPGA model has more DMA channels than an older model.
Regarding schedule management, the biggest lesson we learned early on in these projects is the compile time. Do not underestimate the FPGA compile times. Complex FPGA coding coupled with CLIP nodes will require multiple recompiles and the hours add up, which obviously extends the project delivery date.
Host Layer
The host layer provides the user interface and communication with the RT layer for messaging and data. Interfacing with the RAID drives might have made sense in the RT layer, but the RT OS does not support the RAID drives.
Also, our initial thought of pushing all the data into on giant TDMS file turned out to be slower than using multiple file streams. Of course, applications that need external clocking will likely require multiple file streams due to the possibly different data rates.
Hardware Tips
As might be expected, most of the challenges appear with connectivity to the FlexRIO and configuring clock and data lines. These issues appear whether you are using an off-the-shelf FAM from NI or a custom FAM.
Data and Clock Signals
If your application needs to be externally clocked, be careful of routing clock signals using the FPGA resources. At high data and clock frequencies, skew and even accommodation of the fast signals can be challenging. For example, IO skew can be accommodated in LabVIEW FPGA, but there are limitations which can be overcome only in VHDL CLIP nodes.
Custom FAM Design
When designing a custom FAM, we found that power consumption is the biggest concern. Only about 3 W at 3.3 VDC are available through the FlexRIO connector. Of course, as the signal data and clock rates increase, this power budget becomes more critical.
At high data and clock rates, trace lengths are important to maintain synchronization between all the signals connected to the FlexRIO. As an approximation, 10 ps corresponds to about 1.5 mm of trace length.
Summary
Designing a digital record and playback system is fairly straightforward until data rates get “fast” and external clocks are needed. The FlexRIO platform offers a beneficial combination of off-the-shelf hardware performance and software FPGA-based programming, coupled with enough openness to enhance the performance through software CLIP nodes and hardware custom FAMs.