For a beam time, please be aware, that everything can happen! Any sorts of non
experienced failures happen according to Murphy always when you really don't
need them :-)...
-So, please don't go to a beam time with {\bf one} TRB3 and no replacement module and
+So, please don't go to a beam time with {\bf one} TRB and no replacement module and
always be sure that you have a Lattice Download Cable available to re-flash the
-TRB3 if something goes wrong in the fever of a beam time.
+boards if something goes wrong in the fever of a beam time.
\subsection{Hardware Information}
\begin{itemize*}
\item TRB3 \href{http://jspc29.x-matter.uni-frankfurt.de/trbweb/?action=page&url=schematics}{Schematics}
- \item Pin-out file for all FPGAs vary based on the AddOns in use and can be found \href{http://http://jspc29.x-matter.uni-frankfurt.de/git/?p=trb3.git;a=tree;f=base}{in the git repository} (*.lpf files)
+ \item Pin-out file for all FPGAs vary based on the AddOns in use and can be found in the git repositories, \verb!pinout/*.lpf!.
\end{itemize*}
\subsection{Board Identification}
\label{Trb3BoardID}
-The TRB3 boards are equipped with 5 temperature sensors, one for each FPGA. They contain a unique ID that is used to identify each FPGA. Additionally, the peripheral FPGAs have to bits identification (endpoint ID) corresponding to their position on the board.
+Each FPGA is equipped with a temperature sensor. They contain a unique ID that is used for identification. Additionally, on multi-FPGA boards, FPGA have two bits identification (endpoint ID) corresponding to their position on the board.
This information can be read out after the first initial programming of the FPGAs.
-The board itself has a sticker with a three-digit serial number to identify it (we are looking into a bright future of the TRB3 ;-)).
-The combination of serial number and unique ids is given in the file \files{serials\_trb3.db} available in the main directory of the cvs repository. For each board it contains five lines
+The board itself has a sticker with a serial number to identify it.
+The combination of serial number and unique ids is given in the file \files{serials\_trb3.db} available in the daqtools.git repository. For each FPGA it contains one line:
\begin{verbatim}#SID Unique ID
0015 0x08000002e2e22b28
The first three digits of the SID is the serial number as written on the board, the fourth digit is an identifier for the FPGA number as printed on the PCB (central FPGA is FPGA 5, the others are numbered 1 to 4, but mapped to 0 to 3 in the file). Based on this file, a second file \files{addresses.db} can be written for each individual set-up to assign each board the necessary network addresses.
+Similar files exist for all other board flavors.
\subsection{Flash Programming}
\label{flashprog}
-Typically only the first programming of a board is done with a JTAG cable, all later upgrades can be done directly via TrbNet to the Flash ROMs. The advantage is the increased speed (about a factor 10) and that no physical access to the board is necessary. The software needs some settings in the FPGA code to function properly:
+Typically only the first programming of a board is done with a JTAG cable, all later upgrades can be done directly via TrbNet to the Flash ROMs. The advantage is the increased speed (about a factor 10) and that no physical access to the board is necessary. There are a couple of safety features to assure no wrong firmware is loaded.
+
+First, the name of the design has to contain a certain sub-string like \verb!trb3_central! or \verb!dirich!, depending on the target platform (can be overridden by software).
+Second, the upper 16 Bit of the Hardware Version register as described below is checked (must be set correctly in the VHDL project).
-First, the name of the design has to contain a certain sub-string:
-\begin{description}
- \item[trb3\_central] if the design is targeted to the central FPGA
- \item[trb3\_periph] if the design is targeted to either of the peripheral FPGA
-\end{description}
-Second, the upper 16 Bit of the Hardware Version register as described below is checked.
\begin{warning}
Always ensure that your trbflash accesses the TrbNet over RPC (i.e.
\subsection{Design Identification}
\label{sec:Identification}
The TrbNet endpoint has a generic setting \signal{Regio\_Hardware\_Version} (register 0x42) that has to be set according to the following rules:
-The upper 16 Bit are used by the software to identify the hardware before programming the Flash to prevent loading invalid designs and
-have to contain one of the following values. The last digit should be used to denote the hardware revision.
+The upper 16 Bit are used by the software to identify the hardware before programming the Flash to prevent loading invalid designs and have to contain one of the following values. The last digit should be used to denote the hardware revision.
\begin{description*}
- \item[9000] design for Trb3 central FPGA
- \item[9100] design for Trb3 peripheral FPGAs
- \item[9200] design for CBM Rich
- \item[9300] design for CBM Tof
- \item[9500] design for Trb3sc
- \item[9600] design for DiRich
- \item[9700] design for DiRich Combiner module
+ \item[9000] design for Trb3 central FPGA, \verb!trb3_central!
+ \item[9100] design for Trb3 peripheral FPGAs, \verb!trb3_periph!
+ \item[9200] design for CBM Rich, \verb!cbm_rich!
+ \item[9300] design for CBM Tof, \verb!cbm_tof!
+ \item[9500] design for Trb3sc, \verb!trb3sc!
+ \item[9600] design for DiRich, \verb!dirich!
+ \item[9700] design for DiRich Combiner module, \verb!combiner!
\item[9900] design for Munich Skyroc boards
- \item[9A00] design for DiRICH5s
- \item[9B00] design for FaRICH1
- \item[9C00] design for DiRICH1
+ \item[9A00] design for DiRICH5s, \verb!dirich5s!
+ \item[9B00] design for FaRICH1, \verb!farich!
+ \item[9C00] design for DiRICH1, \verb!dirich!
- \item[A500] design for TRB5sc
- \item[A600] design for MDC central
- \item[A700] design for MDC TDC
+ \item[A500] design for TRB5sc, \verb!trb5sc!
+ \item[A600] design for MDC central, \verb!mdcoep!
+ \item[A700] design for MDC TDC, \verb!mdctdc!
\end{description*}
The lower 16 Bit are used to identify the contents of the design and the AddOn boards they should be used with. Combine
register should only be used to describe hardware-related features, like the needed AddOn Board.
\begin{description*}
-\item[Central FPGA]~
+\item[Trb3 central]~
\begin{description*}
\item[cXX0] contains a CTS
\item[cXX1] contains a CTS, use with AddOn for trigger signals
- \item[8XXX] {\color{darkgray}uses RX clock as main internal clock}
- \item[0e00] {\color{darkgray}contains a GbE link for slow control and read-out}
- \item[0d00] {\color{darkgray}contains a GbE link for read-out only}
- \item[0010] {\color{darkgray}accepts triggers from optical link SFP1}
- \item[0020] {\color{darkgray}accepts slow-control from optical link SFP1}
- \item[0040] {\color{darkgray}sends triggers to optical link SFP1}
- \item[0080] {\color{darkgray}sends slow-control to optical link SFP1}
\end{description*}
-\item[Peripheral FPGA (also CBM-RICH and other derivatives)]~
+\item[Trb3 peripheral, Trb3sc]~
\begin{description*}
\item[0XXX] use with ADA adapter board AddOn version 1
\item[1XXX] use with ADA adapter board AddOn version 2
\item[9XXX] use with ADC AddOn
\item[AXXX] use with SFP AddOn v2
\item[BXXX] onboard connectors
- \item[X0nX] {\color{darkgray}contains $2^n$ TDC channels, single edge, n<8}
- \item[X1nX] {\color{darkgray}contains $2^n$ TDC channels, double edge, n<8}
- \item[X2XX] {\color{darkgray}contains a network hub}
- \item[X4XX] {\color{darkgray}SPI interface on AddOn connector}
- \item[X8XX] {\color{darkgray}Double edge TDC realized with two single edge channels}
- \item[XX8X] {\color{darkgray}Non-TDC (because of bad choice of encoding)}
\item[XX9X] for MVD converter board 2013
- \item[XXX1] {\color{darkgray}uses RX clock as main internal clock}
+\end{description*}
+
+\item[Trb5sc]~
+\begin{description*}
+ \item[1000] use with ADA adapter (80pin KEL)
+ \item[2000] use with 4conn adapter (40 pin KEL)
+ \item[4000] use with RJ45 adapter
+ \item[0100] use with HDMI adapter
\end{description*}
\end{description*}
\end{center}
\subsection{Network Addresses}
-The network addresses in a TRB3 set-up (not in HADES) follow a simple scheme:
+For proper automatic detection of data in software, the network addresses in a TRB set-up need to have their first hex digit representing their function: 'C' for the CTS, '8' for a hub, '0' or '1' for TDCs. See \ref{Data_Unpacking}.
-All network addresses are of the form ABBC, where:
-\begin{itemize*}
-\item C is the FPGA number. 0 for the central FPGA, 1-4 for the peripheral FPGAs.
-\item A denotes the function of the FPGA:\\C: CTS, 8: Hub, 0--1: TDC, D--E other purposes
-\item BB is a number identifying the TRB in the full system. BB is equal on all 5 FPGA of one board.
-\end{itemize*}
+The middle two digits are often chosen to equal the serial number of the board while the last digit enumerates the FPGAs on one board / one crate.
-The FPGA with the CTS has an address starting with 'C'. For data unpacking schemes see also \ref{Data_Unpacking}.
-
-All boards of a given type are accessible by a broadcast address at the same time. This is set by \signal{Broadcast\_Special\_Addr} in the TrbNet endpoint:
+All boards of a given type are accessible in parallel by a broadcast address. This is set by \signal{Broadcast\_Special\_Addr} in the TrbNet endpoint:
\begin{itemize*}
\item 0x40 for the central FPGA
\item 0x45 for the peripheral FPGA
\item 0x91 MDC TDC
\end{itemize*}
-The initial address set with \signal{Regio\_Init\_Address} can be chosen from the following set:
+The initial address set with \signal{Regio\_Init\_Address} can be chosen freely. Some suggestions:
\begin{itemize*}
\item 0xF300 for the central FPGA
\item 0xF305 for the peripheral FPGA
\subsection{JTAG}
Since programming of FPGAs can be done via GbE, the JTAG connector is usually not used. In case of corrupted designs it provides the only access to reconfigure FPGAs. Loading a design directly to the FPGA is quite fast (25 s) but loading it to the Flash ROM is deadly slow (5 minutes) - better: first load the design to the FPGA via JTAG, then flash it using TrbNet.
-The pin-out of the JTAG connector (1x8 pin-header near the power supply).
+The pin-out of the JTAG connector on TRB3 (1x8 pin-header near the power supply):
\begin{description*}
\item[1] VCC (3.3V, red)
\item[2] TMS (violet)
\item[7] GND (n/c)
\item[8] GND (n/c)
\end{description*}
-Pin 1 is next to the 2x6 pin-header. Note that TDO and TDI are switched compared to the layout on all other boards. If you experience strange behaviour of the programming procedure and think you might have destroyed the cable: It's most likely a software issue - reboot your PC!
+
+Pin 1 is next to the 2x6 pin-header. Note that TDO and TDI are switched compared to the layout on all other boards.
+
+If you experience strange behaviour of the programming procedure and think you might have destroyed the cable: It's most likely a software issue - reboot your PC!
Note that the programming cable can only be used as root by default in most Linux flavours. You can either change permissions on the device in /dev/ by hand or put this (as one line!)
\\\verb!SUBSYSTEMS=="usb", ATTRS{idVendor}=="1134", SYMLINK+="lattice_cable",!
\\\verb!GROUP="users", MODE="0666"'!
-\\into
-\\\files{/etc/udev/rules.d/91-usb-hardware.rules}
+\\into \files{/etc/udev/rules.d/91-usb-hardware.rules}
\\and run
\\\verb!udevadm control --reload-rules; udevadm trigger!
\\\verb!/etc/init.d/boot.udev restart!
\\Commands and files might be different on your machine.
-There's also an alternative to the quite expensive Lattice Programming
-cable, if you have some time left, you can buy the FT2232H mini module
-and solder a cable yourself. This costs roughly 50 Euro and one hour
-soldering fun. If the programmer is not able to find a valid JTAG chain or
+The FT2232 based JTAG cables might have a conflict with automatically loaded drivers. If the programmer is not able to find a valid JTAG chain or
errors are reported during programming, the problem may be related to the Linux ftdi\_sio kernel module.
+You can remove the module manually by running \\\verb!rmmod ftdi_sio! after plugging in the cable.
You can unload it automatically using a udev-rule by adding (as one line)
\\\verb!SUBSYSTEM=="usb",ATTRS{idVendor}=="0403",ATTRS{idProduct}=="6010",MODE=="0666",!
\\\verb!OWNER=="root",GROUP="users",RUN+="/bin/sh -c 'RESULT=$(basename %p); !
\\\verb!echo $RESULT:1.0 >/sys/bus/usb/drivers/ftdi_sio/unbind"!
-\\into
-\\\files{/etc/udev/rules.d/91-usb-hardware.rules}
-\\and restart udev or unload it by hand with
-\\\verb!rmmod ftdi_sio!
-\\after plugging in the cable.
+\\into \files{/etc/udev/rules.d/91-usb-hardware.rules}
+
+
\subsection{Data Unpacker}
\label{Data_Unpacking}
\item[5555] The only SubSubEvent-ID starting with 5 is 5555 at the end of each SubEvent, marking the beginning of the status information word for this SubEvent.
\item[7] Special TDCs (like beam detectors, not part of the system under test)
\item[8] Skip this SubSubEvent-Header. Note: Only the header, the next word should be another valid SubSubEvent-Header.
-\item[C000] Unpack as CTS data
+\item[C] Unpack as CTS data
\item[D..E] Other read-out formats, no TDC data included
\end{description*}
The data format is shown in later sections.
-\subsection{Trigger \& Clock Input}
+\subsection{TRB3 Trigger \& Clock Input}
Both trigger and clock input (RJ-45 connectors) are equipped with a configurable fan-out chip. Its control signals can be set in slow-control register 0xd300 (currently in CTS only, but should also be present in other central FPGA).
\begin{description*}
\item[Bit 0] Trigger select (local signal: 1, external signal: 0
\item[Bit 27 -- 24] 4 User signals on clock fan-out
\end{description*}
-\subsection{Power Consumption}
+\subsection{TRB3 Power Consumption}
The TRB3 with linear supplies and typical TDC designs needs
\begin{itemize*}
\item 3.3V - 1.25A
\label{fig:trb3scsfpnumbers}
\end{center}
\end{figure}
-\subsubsection{Powering Schemes}
+\subsection{Powering Schemes}
\begin{description*}
\item[External 5V] Connect a 1.5A power supply with not more than 5.5V to the black 3pin input.
- \item[Backplane 5V] Connect a fitting power supply to the backplane. See the backplane description for further details. DC/DC converters are bypassed for lower noise. Power-LEDs will be off.
+ \item[Backplane 5V] Connect a fitting power supply to the backplane.
+ \item[Backplane 48V] The backplane can be fitted with a DCDC module accepting 15-48V input voltage and up to 120W total power.
\item[External 3.5V/1.4V] Remove jumpers J12, J14 and J16. Connect a 1A power supply with 3.5-4.0V and 2A 1.4-1.7V to the black 4pin input. The 2.7V rail can be powered individually or connected to the 3.5V rail.
\item[Backplane 3.5V/1.4V] Remove jumpers J12, J14 and J16. Close fuses F1V2L, F3V3L and F2V5L. Connect a proper power supply to the backplane. DC/DC converters are bypassed for lower noise. Power-LEDs will be off.
\end{description*}
-\subsubsection{Clock Inputs}
+\subsection{Clock Inputs}
The current system clock configuration is shown on two LEDs on the front-panel.
\begin{itemize*}
- \item The board has own 240 MHz and 125 MHz oscillators. The internal 240 MHz will be selected if there is no external clock available at power-up.
+ \item The board has own 200 MHz and 125 MHz oscillators. The internal 200 MHz will be selected if there is no external clock available at power-up.
\item A external clock can be fed in via the RJ-45 connector (left, pair 2) or from the backplane. The source is selected with a switch. At power-up the board searches for an external clock on the selected input. If none is found, the internal is used.
- \item The system clock can be recovered from the SFP1 input signal. This is selected at compile-time.
+ \item The system clock can be recovered from the SFP1 input signal. This is selected at compile-time (only in special setups).
\end{itemize*}
-The native frequency of the board are 240 MHz, logic running at 120 MHz. Nevertheless, running at 200/100 MHz will be used for compatibility with existing setups. Most logic should be able to run at the higher speed when selected during compilation.
-\subsubsection{Trigger Input/Output}
+\subsection{FEE: Reference time / Trigger Input/Output}
\begin{itemize*}
- \item The default trigger input is pair 1 on the left RJ-45 on the front-panel
- \item The front-panel trigger input can be rerouted to the second RJ-45 if separation of trigger and clock is required.
- \item The trigger can be supplied from the backpanel, selected by the switch (same as clock setting)
+ \item The default reference time input is pair 1 on the inner RJ-45 on the front-panel
+ \item The reference time can be supplied from the backpanel, selected by the switch (same as clock setting)
+ \item The outer RJ-45 supports four trigger output signals.
\end{itemize*}
-\subsubsection{Other I/O}
+\subsection{CTS: Reference time / Trigger Input/Output}
+\begin{itemize*}
+ \item The default reference time output is on pair 1 on the outer RJ-45 on the front-panel
+ \item The inner RJ-45 supports two trigger input signals on pairs 3 and 4.
+\end{itemize*}
+
+
+\subsection{Other I/O}
HDR\_IO is available for any general purpose I/O. All lines are LVCMOS25. By default, SPI and UART are available:
\begin{center}
\begin{tabular}{|l|l|}
Optionally, a LCD can be connected. In this case SPI channel 8 is not used.
-\subsubsection{Serial Links}
+\subsection{Serial Links}
By default, SFP1 is used for GbE, SFP2 for TrbNet. SFP2 must be removed if the board is to be used on a backplane as slave module. Removing the SFP selects the backpanel as TrbNet input.
\end{tabularx}
% \caption[Trb3sc Serdes Mapping]{Mapping of Serdes channels}
\end{center}
-The TrbNet uplink (either Sfp or backplane) is synchronous, i.e. no clock tolerance compensation is enabled. Hence, both boards connected with the link must share a common clock source. For low-accuracy applications, the clock can be recovered from the optical link. In this case, no additional clock distribution is needed. If running in a crate, clock recovery is available, but not necessary as the backplane distributes a clock signal to all boards connected.
-\subsubsection{Modifications}
+
+\subsection{Modifications}
\begin{figure}
\begin{center}
\includegraphics[width=0.7\textwidth]{figures/trb3scpatch.png}
jspc29 / daqdocu.git / commitdiff
