From: Jan Michel Date: Wed, 23 Apr 2014 21:31:11 +0000 (+0200) Subject: updated gbe section X-Git-Url: https://jspc29.x-matter.uni-frankfurt.de/git/?a=commitdiff_plain;h=9aa9abb0047ca50608b0fb458756c270378fd59c;p=daqdocu.git updated gbe section --- diff --git a/trb3/GbeDataReadout.tex b/trb3/GbeDataReadout.tex new file mode 100644 index 0000000..73270a1 --- /dev/null +++ b/trb3/GbeDataReadout.tex @@ -0,0 +1,141 @@ +\label{sec:GbeDataReadout} +Communication with TRB3 is handled by the Gigabit Ethernet interface (SPF8 by default). It can be used for Slow Control connection (see next section) and for the readout of collected data. In order to act as standard network device, there are several protocols that share the same link. The basic ones for the network discovery are DHCP, ARP and ICMP. Protocols typical for data handling in standard TRB3 implementation are SCTRL and TrbNetData. Even though they all can run in parallel, processing data, they all share the same input and output link, distributing 125MBps bandwidth. + +\begin{figure}[!htbp] + \begin{center} + \includegraphics[width=0.7\textwidth]{figures/gbe_scm.png} + \caption[Block diagram of the GbE module.]{Block diagram of the GbE module.} + \end{center} +\end{figure} + +Default buffer depths allow the following data sizes configuration: +\begin{table}[!htbp] +\begin{center} + \begin{tabular}{|l|l|} + \hline + Size & Description \\ \hline + 1400 Bytes & Single reception only possible, no reassembly mechanism \\ \hline + 4000 Bytes & Maximum MTU of outgoing Ethernet frames \\ \hline + 64000 Bytes & Maximum size of a single UDP TrbNetData packet \\ \hline + 64000 Bytes & Maximum size of a single UDP SlowControl packet \\ \hline + \end{tabular} + \caption[Frame and packet sizes]{Default maximum sizes of frames and packets} +\end{center} +\end{table} + +\subsection{Data Readout} +TrbNetData module constructs Hades data packets out of the fragments received from the TrbNet endpoints. In standard case, those are the edge FPGAs, but it's possible to use the TRB3 board as a HUB and collect also data from "slave" boards. A Hades packet is formed as an entity called HadesTransportUnitQueue and there are several ways of constructing it. In general event fragments from all the connected endpoints are buffered one after the other, encapsulated with proper headers on several levels: subevent headers, queue headers, UDP headers, IP headers and Ethernet as a final step. +\subsection{Addressing} +Each TRB3 board has a unique MAC address which is constructed in following way: \newline 02:00:BE:UNIQUE$\_$ID(31 downto 8), where UNIQUE$\_$ID(31 downto 8) is a value read out from the temperature sensor and differs between boards, the first part is constant. This address is used only for the network and SlowControl packets. Data readout addressing is stored in a block memory under base address 0x8100 and has to be configured manually. As there is a way to distribute packets to several event building machines or processes, all of those addresses need to be written into this memory with the following structure: + +\begin{table}[!htbp] +\begin{center} + \begin{tabular}{|l|l|} + \hline + 0x81X0 + offset & Description \\ \hline + 0 & destination MAC address 32 lower bits \\ \hline + 1 & destination MAC address 16 upper bits \\ \hline + 2 & destination IP address \\ \hline + 3 & destination UDP port \\ \hline + 4 & source MAC address 32 lower bits \\ \hline + 5 & source MAC address 16 upper bits \\ \hline + 6 & source IP address \\ \hline + 7 & source UDP port \\ \hline + \end{tabular} + \caption[Addressing registers map]{Addressing registers map} +\end{center} +\end{table} + +Each such block of addresses corresponds to one destination event builder. There can be up to 16 destinations configured, where each one has an offset in addressing of 0x10. + +\subsection{Configuration} + +Some header values as well as operation mechanics can be changed and adjusted, here's the table of control registers (R/W): + +\begin{table}[!htbp] +\begin{center} + \begin{tabular}{|l|l|l|} + \hline + 0x8300 + offset & Description & Default value\\ \hline + 0 & Subevent ID value for the header field & 0x000000cf \\ \hline + 1 & Subevent decoding value for the header field & 0x00020001 \\ \hline + 2 & Queue decoding valuke for the header field & 0x00030064 \\ \hline + 4 & Max Ethernet frame size (MTU) can be set up to 4kB & 0x00000578 \\ \hline + 5 & Enable GbE data transport & 0x0 (DISABLED BY DEFAULT) \\ \hline + 7 & Enable multievent mode & 0x0 \\ \hline + 8 & Update readout counter value & 0x000000 \\ \hline + 9 & Enable RX channel & 0x1 \\ \hline + A & Include additional SlowControl data header & 0x1 \\ \hline + B & Include trigger type in the decoding field & 0x0 \\ \hline + FF & Reset value the their default values & 0x0 \\ \hline + \end{tabular} + \caption[Control registers map]{Control registers map} +\end{center} +\end{table} + +\newpage + +\subsection{Monitoring} + +The operation of the entire GbE module as well as individual protocols can be monitored using the following registers (R only): + +\begin{table}[!htbp] +\begin{center} + \begin{tabular}{|l|l|} + \hline + 0x8300 + offset & Description \\ \hline + e0 & Received bytes counter \\ \hline + e1 & Received Eth frames counter \\ \hline + e2 & Transmitted bytes counter \\ \hline + e3 & Transmitted Eth frames counter \\ \hline + e4 & Transmitted packets counter \\ \hline + e5 & Dropped RX frames counter \\ \hline + a0 & SlowControl received frames counter \\ \hline + a1 & SlowControl received bytes counter \\ \hline + a2 & SlowControl transmitted frames counter \\ \hline + a3 & SlowControl transmitted bytes counter \\ \hline + a4[0] & SlowControl rx fifo full \\ \hline + a4[1] & SlowControl rx fifo empty \\ \hline + a4[2] & SlowControl tx fifo full \\ \hline + a4[3] & SlowControl tx fifo empty \\ \hline + a4[7:4] & SlowControl state machine \\ \hline + b0 & TrbNetData received frames counter = 0 \\ \hline + b1 & TrbNetData received bytes counter = 0 \\ \hline + b2 & TrbNetData transmitted frames counter \\ \hline + b3 & TrbNetData transmitted bytes counter \\ \hline + b4[3:0] & IpuInterface receiving state machine \\ \hline + b4[7:4] & IpuInterface loading state machine \\ \hline + b4[8] & Split fifo empty flag \\ \hline + b4[9] & Split fifo almost empty flag \\ \hline + b4[10] & Split fifo full flag \\ \hline + b4[11] & Split fifo almost full flag \\ \hline + b5[3:0] & Packet constructor constructing state machine \\ \hline + b5[7:4] & Packet constructor loading state machine \\ \hline + b5[11:8] & Packet constructor headers state machine \\ \hline + b5[12] & Data fifo full flag \\ \hline + b5[13] & Data fifo empty flag \\ \hline + b5[14] & Headers fifo full flag \\ \hline + b5[15] & Headers fifo empty flag \\ \hline + f3 & Same as e2 for backwards compatibility \\ \hline + f4 & Same as e3 for backwards compatibility \\ \hline + \end{tabular} + \caption[Monitoring registers map]{Monitoring registers map} +\end{center} +\end{table} + +\newpage + +Additionally there are two register groups, one for SlowControl and another for TrbNetData that can be used for histograming of outgoing packet sizes. Each group consists of 32 x 32bits registers, where each register represents a counter of packets with size within a specified range. The size difference between two regiters is 2kB, so the first register is a counter for packets with sizes from 0 to 2kB and so on. + +\begin{table}[!htbp] +\begin{center} + \begin{tabular}{|l|l|} + \hline + Register block base & Description \\ \hline + 0x8360 & Start of the 32 registers block for SlowControl packet histograming \\ \hline + 0x8380 & Start of the 32 registers block for TrbNetData packet histograming \\ \hline + \end{tabular} + \caption[Histograming registers map]{Histograming registers map} +\end{center} +\end{table} + diff --git a/trb3/figures/gbe_scm.png b/trb3/figures/gbe_scm.png new file mode 100644 index 0000000..f7239d8 Binary files /dev/null and b/trb3/figures/gbe_scm.png differ diff --git a/trb3/main.tex b/trb3/main.tex index 4c1a23c..18cde0c 100644 --- a/trb3/main.tex +++ b/trb3/main.tex @@ -174,9 +174,6 @@ \section{Related Boards} \subsection{CBM-RICH} \subsection{CBM-TOF} - \clearpage - \section{Glue Hardware} - \input{GlueHardware} \cleardoublepage \part{Design Components} @@ -200,7 +197,8 @@ \input{TriggerModule} \clearpage - \section[GbE Data Read-out]{GbE Data Read-out\footnote{This space to be filled by Grzegorz Korcyl. At the moment: read the TrbNet manual.}} + \section{GbE Data Read-out} + \input{GbeDataReadout} \subsection{Building Blocks} \subsection{Slow Control Registers}