+Gigabit Ethernet is a verified and a weel known standard for data transmission over Ethernet networks at speed of 1Gbps using both optic fibers and standard UTP cables. It's implementation on TRB Addon boards is a gateway for collected data into event builders via Ethernet network. Entity trb_net16_gbe_buf is the interface between TRBNet streaming API and the UDP packet construction logic.
+
+Current design allows to send data with maximum speed of 400Mbps. Each subevent is sent in one UDP packet. The maximum size of a packet is 65535B. In order to control the size of incoming subevents, two registers can be set which describe the minimum and the maximum size of an accepted subevent. Te ones that will not fit between those two values, will be marked as incorrect and sent to the network. GbE logic performes the check of incoming subevent headers consistency. When the structure or the values are incorrect, the subevent is also marked and sent to event builders. The bit nr. 26 in error vector at the end of subevent signalizes incorrect subevent.
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-\subsection{Memory Map}
+\subsection{Configuration}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-The configuration memory of the GbE interface consists of two parts. Registers 0x8300 to 0x83FF are used to configure the behaviour the SubEventBuilder and the packing of data into UDP packets. The registers implemented are listed in table \ref{GbEConfigRegisters}.
+The configuration memory of the GbE interface consists of two parts. Registers 0x8300 to 0x830B are used to configure the behaviour the SubEventBuilder and the packing of data into UDP packets. The registers implemented are listed in table \ref{GbEConfigRegisters}.
The second part is located at addresses 0x8100 to 0x81F0. Each block of 16 addresses describes the addresses and ports used to send data to one of the Eventbuilders (see table \ref{GbEEBSettings}. Which entry in the memory is used, is selected by the lower 4 bits of the IPU readout information as shown in table \ref{IPUInformationBits}.
Register 0x8309 is used to insert an additional delay between sending two UDP frames. Figure \ref{fig:gbedelay} shows the relationship between this setting and the achievable data rate.
-The GbE interface itself has a control and a status register, available at addresses 0xc020 and 0xc021 respectively:
-\begin{description}
- \item[0x8020: Control Register] PHY control register. Each of the bits has to be reset manually after use
- \begin{description}
- \item[Bit 0]Restart Autonegotiation
- \item[Bit 1]PHY mode (must be 0)
- \item[Bit 3]PHY reset
- \end{description}
- \item[0x8021: Status Register] GbE link status register (detailed explanation: t.b.d.)
- \begin{description}
- \item[Bit 15..0] Link partner page
- \item[Bit 16] Link partner page received (strobe signal)
- \item[Bit 17] Autonegotiation completed
- \item[Bit 23..20] Reset state machine status
- \item[Bit 27..24] Link status
- \end{description}
-\end{description}
+The GbE interface itself has two registers to control the link status, available at addresses 0x8000 and 0x8200.
-\begin{figure}
- \centering
- \includegraphics[width=12cm]{delaygbe.png}
- \caption[Delay on GbE Interface]{Additional delay can be inserted between sending two UDP packets to reduce the total bandwidth. With no delay, 400 MBit/s (50 MByte/s) can be reached.}
- \label{fig:gbedelay}
-\end{figure}
+- Behaviour configuration:
\begin{table}[hbtp]
\begin{center}
\end{center}
\end{table}
+- Event builder addresses:
+
\begin{table}[hbtp]
\begin{center}
\begin{tabularx}{\textwidth}{|l|l|X|}
\textbf{Address} & \textbf{Name} & \textbf{Description} \\
\hline\hline
0x81S0 & DestMacLsb & Lower 32 bits of the destination MAC address \\
-0x81S1 & DestMacMsb & Bit 15..0: Higher 16 bit of the destinatioon MAC, Bit 31..16: reserved\\
+0x81S1 & DestMacMsb & Bit 15..0: Higher 16 bit of the destination MAC, Bit 31..16: reserved\\
0x81S2 & DestIP & Destination IP \\
0x81S3 & DestUdpPort & Bit 15..0: Destination UDP port, Bit 31..16: reserved \\
0x81S4 & SrcMacLsb & Lower 32 bits of the source MAC address \\
\caption{Memory map for GbE Ethernet settings. The third digit is the EventBuilder number, allowing to stor 16 different settings that are selected by the IPU request information word.}
\label{GbEEBSettings}
\end{center}
-\end{table}
\ No newline at end of file
+\end{table}
+
+- Link control
+
+\begin{description}
+ \item[0x8000: Control Register] PHY control register. Each of the bits has to be reset manually after use
+ \begin{description}
+ \item[Bit 0]Restart Autonegotiation
+ \item[Bit 1]PHY mode (must be 0)
+ \item[Bit 3]PHY reset
+ \end{description}
+ \item[0x8200: Status Register] GbE link status register (detailed explanation: t.b.d.)
+ \begin{description}
+ \item[Bit 15..0] Link partner page
+ \item[Bit 16] Link partner page received (strobe signal)
+ \item[Bit 17] Autonegotiation completed
+ \item[Bit 23..20] Reset state machine status
+ \item[Bit 27..24] Link status
+ \end{description}
+\end{description}
+
+\begin{figure}
+ \centering
+ \includegraphics[width=12cm]{delaygbe.png}
+ \caption[Delay on GbE Interface]{Additional delay can be inserted between sending two UDP packets to reduce the total bandwidth. With no delay, 400 MBit/s (50 MByte/s) can be reached.}
+ \label{fig:gbedelay}
+\end{figure}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+\subsection{Debugging}
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+For debug purpose a set of registers has been prepared, started from address 0x83E0 to 0x83F2 with meaning described in table \ref{GbEDebugRegisters}. While those registers can be read by hand using trbcmd command, a tool gbe_debug_parser.pl is provided to parse the output and display status in clear text. The parser reads out the registers of all GbE enabled and actually connected hubs.
+
+\begin{table}[hbtp]
+\begin{center}
+\begin{tabularx}{\textwidth}{|l|l|X|}
+\hline
+\textbf{Address} & \textbf{Name} & \textbf{Description} \\
+\hline\hline
+0x83E0 & ipu2gbe1 & Bit 0: sf_full, Bit 1: sf_empty, Bit 2: sf_afull, Bit 3: sf_aempty, Bit 7..4: load machine state, Bit 11..8: save machine state, Bit 15..12: bank selected, Bit 16: conf_done, Bit 17: remove done, Bit 18: read_done, Bit 19: padding needed, Bit 20: load sub done \\
+0x83E1 & ipu2gbe2 & Bit 7..0: cts information, Bit 31..16: actual message size \\
+0x83E2 & packetConstr1 & Bit 3..0: construction state, Bit 7..4: save headers state, Bit 11..8: load state, Bit 27..12: queue size, Bit 28: df full, Bit 29: df empty, Bit 30: shf full, Bit 31: shf empty \\
+0x83E3 & packetContsr2 & Bit 15..0: size left, Bit 24..16:all ctr, Bit 25: pc ready \\
+0x83E4 & frameConstr1 & Bit 7..0: construction state, Bit 11..8: transmission state, Bit 27..12: sent frames counter, Bit 28: fpf full, Bit 29: fpf empty, Bit 30: ready, Bit 31: headers ready \\
+0x83E5 & frameConstr2 & Bit 15..0: ready frames counter \\
+0x83E6 & tsmac & Bit 31..0: tsmac statistics \\
+0x83E7 & sgmii/gbe & same as register 0x8200 \\
+0x83E8 & ipu2gbe3 & Bit 15..0: pc sub size \\
+0x83E9 & ipu2gbe4 & Bit 31..0: incoming data counter \\
+0x83EA & ipu2gbe5 & Bit 15..0: dropped small events counter, Bit 31..16: dropped large events counter \\
+0x83EB & ipu2gbe6 & Bit 31..0: invalid headers counter \\
+0x83EC & ipu2gbe7 & Bit 15..0: cts trigger number, Bit 31..16: cts random code \\
+0x83ED & ipu2gbe8 & Bit 15..0: cts endpoint address, Bit 31..16: cts length \\
+0x83EE & ipu2gbe9 & Bit 15..0: internal trigger counter, Bit 31..16: internal endpoint address \\
+0x83EF & ipu2gbe10 & Bit 15..0: saved events, Bit 31..16: loaded events \\
+0x83F0 & ipu2gbe11 & Bit 15..0: constructed events, Bit 31..16: dropped events \\
+0x83F2 & ipu2gbe12 & Bit 15..0: invalid header size counter \\
+\hline
+\end{tabularx}
+\caption{Memory map for GbE Ethernet debug.}
+\label{GbEDebugRegisters}
+\end{center}
+\end{table}
+
+Gbe_monitor.pl is another tool to monitor the status of GbE hubs in the system. It provides information about transmission rates, link status and constructed events counters. Transmission rate is measured as the number of bytes sent from board into the network, it can vary from the one measured at the receiving point due to network issues.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+\subsection{Implementation notes}
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+Construction and transmission logic runs on 125MHz clock which has to be provided into gbe_buf entity in one of two provided ways:
+- 125MHz oscillator is directly connected to SERDES unit: USE_125MHZ_EXTCLK generic value has to be set to 1, then the input port CLK_125_IN is not used, and signal is received from reference clock output of SERDES inside gbe_buf logic (the case of HUB2 addon).
+- 125MHz clock signal is connected to FPGA pins: USE_125MHZ_EXTCLK generic value has to be set to 0 and the port CLK_125_IN has to be connected to signal provided by FPGA logic (the case of Shower, MDC and CTS addons).
+
+While generating a new PCS unit in IPExpress there are two things to care about:
+- Gigabit Ethernet has to be chosen as the quad protocol mode
+- CTC_BYP entry in generated .txt file has to be changed by hand from NORMAL to BYPASS in order to allow the proper operation of SGMII/GBE PCS IPCore.
jspc29 / daqdocu.git / commitdiff