Example of FPGA-based Aurora 8B/10B communication

The SFP ports on imperix controllers are typically used for interconnecting devices in a RealSync network. However, when customizing the FPGA firmware, imperix designed the system to allow these SFP ports to be repurposed for other communication protocols. Aurora 8B/10B or Aurora 64B/66B can be used to communicate with hardware-in-the-loop (HIL) simulators that support the Aurora protocol, such as OPAL-RT, TYPHOON HIL, SPEEDGOAT and RTDS.

This note provides an example of loopback communication using the Aurora 8B/10B protocol over a fiber optic link. It provides a step-by-step guide demonstrating how the Aurora 8B/10B protocol can be seamlessly integrated into the imperix controller FPGA.

An example of SFP communication with a hardware-in-the-loop (HIL) simulator is available on the page SFP communication with an RTDS MMC simulator.

What is Aurora 8B/10B?

Aurora 8B/10B is a serial link layer communication protocol developed by Xilinx/AMD. The protocol is open and provides lightweight, high-speed point-to-point communication between devices. The following example uses Aurora 8B/10B, but Aurora 64/66B can also be used. Implementing Aurora communication is particularly useful for establishing high-throughput, low-latency communication with other power controllers or HIL simulators.

Aurora 8B/10B loopback example

For demonstration purposes, a loopback connection is established between two ports on the same controller, as shown in the diagram below:

• SBI and SBO blocks are used to move data between the CPU and the FPGA.
• FIFOs are used to move data between the IXIP clock domain (250 MHz) and the Aurora communication clock domain (78.125 MHz in this example).
• Aurora 8B10B IPs encode/decode data.
• TX/RX serial links are connected to physical SFP ports to transmit data over the optical fiber.

Vivado project

The Vivado block design of the Aurora 8B/10B loopback example is provided below.

The following zip file contains scripts to automatically generate this design.

To generate the design using the script, please do the following:

• Download the FPGA sandbox template 3.10 or later, available on the FPGA download page
• Unzip it and save the content somewhere on the PC
• Rename the folder to something more explicit

• Download Aurora_8b10b_example_gen_scripts.zip using the button above
• Unzip it and copy the content to the scripts folder of the FPGA sandbox template

• Set the vivado_path variable to match the Vivado version installed on the PC

• Double-click on create_aurora_8b10b_example.bat
  Windows Defender SmartScreen may display a warning pop-up. Simply click More info, then Run anyway.

The Vivado Aurora 8B/10B project will be created and configured. The step-by-step section below explains how to recreate it manually.

MATLAB model

The model below is used to test the design. It generates a sinusoidal waveform and sends it to the FPGA using SBI 0 and 1. It reads the received on SBO 2 and 3. Probe variables are used to observe the sent and received signals on Cockpit.

Experimental results

Observing the signals on Cockpit validates that the data sent on port SFP 0 using Aurora 8B/10B is properly received on port SFP 1.

Thanks to the Aurora 8B/10B low latency, one single control task period delay is observed between the sent and received data.

Using the Integrated Logic Analyzer (ILA) in Vivado, the propagation delays of the data in the FPGA can be measured. With a lane rate of 3.125 Gbps, the total measured delay is 560 ns. It consists in:

• 84 ns for the TX FIFO data
• 424 ns for the Aurora communication
• 52 ns for the RX FIFO

Step-by-step procedure to create the Aurora 8B/10B loopback example

Instantiating the Aurora 8B/10B IP

The first step is to instantiate 2 Aurora IPs. The Aurora 8B/10B IP is available for free from the Vivado IP Catalog.

Configuring the Aurora 8B/10B IP

The screenshots below show the settings used in this example. The two following parameters must be set to specific values to work on imperix hardware:

• GT Reflck must be set to 250 MHz, because the clock is generated outside of the FPGA
• “include Shared Logic in example design” must be checked, because the Shared Logic is already instantiated inside imperix firmware

The other Core Options settings can be modified freely. Below are the settings used in this example:

• A lane width of 4 Bytes was chosen to be easily interfaced to the AXI-Stream interface.
• The default line rate of 3.125 Gbps was kept. This will result in a data clock of
  line_rate * 0.8 / lane_width_in_bits = (3.125 Gbps * 0.8) / (32 bits) = 78.125 MHz
• In this example, making packets is unnecessary, so the interface is set to streaming mode.

Other Core Options settings are not detailed in this note and are left as defaults.

Configuring the imperix IP

The second step is to configure the imperix firmware IP to make the SFP ports available from the sandbox. In this example, SFP 0 and 1 are used.

This makes the following ports visible on the imperix IP.

• TX and RX signals are the differential serial signals connected between the transceiver and the physical SFP ports.
• The GT interface gives access to the shared logic instantiated in the imperix IP.

Connecting the clocks of the Aurora 8B/10B IP

The Aurora IP provides a tx_out_clk (78.125 MHz in this example), which is used as the user_clock. A BUFG buffer is required between tx_out_clk and the clock inputs. The init clocks and DRP clocks are connected to clk_50_mhz provided by the imperix firmware IP.

To learn more about the different clocks, please refer to the Aurora 8B10B IP and the GTX transceiver user guides.

Connecting the data interfaces of the Aurora 8B/10B IP

The AXI-Stream interface module (ix_axis_interface) is used to exchange data with the Aurora IPs. As the AXI-Stream interface module and the Aurora IPs are not in the same clock domain, AXI4-Stream Data FIFO with independent clocks are used to manage the clock domain crossing. The figure below shows how the FIFOs are connected in the system.

To learn more about the AXI-Stream interface module, please refer to the getting started page.

Connecting the shared logic interface of the Aurora 8B/10B IP

The following signals, connected to the shared logic of Aurora, must be connected between the GT interface of the imperix firmware IP and each Aurora IP:

• gt0_qplllock
• gt0_qpllrefclklost
• gt_qpllclk_quad1
• gt_qpllrefclk_quad1
• gt_refclk1

Connecting the TX/RX signals of the Aurora 8B/10B IP

Finally, the rxn, rxp, txn, and txp signals of each Aurora IP must be connected to the corresponding pins of the imperix firmware IP to map each Aurora IP to a physical SFP port on the board.

Finally, the bitstream can be generated and loaded in an imperix controller, as explained on the getting started page.

Going further

The page high-level synthesis for FPGA developments shows how automated code generation tools such as Model Composer and Vitis HLS can be used to facilitate the development of FPGA modules. Like the Aurora 8B/10B IP, they use AXI4-Streams to move data around.

The FPGA development on imperix controllers summarizes all the other FPGA-related pages.