Readout Firmware of the Vertex Locator for LHCb Run 3 and beyond

Karol Hennessy; Antonio Fernandez Prieto; Pablo Vazquez Regueiro; Jan Buytaert; Martin Van Beuzekom; Edgar Lemos Cid; Lars Eklund; Kristof De Bruyn; Sneha Naik; Manuel Schiller; Donal Murray; Alexander Leflat; Giovanni Bassi; Giovanni Punzi; Federico Lazzari; Michael J. Morello; Oscar Boente Garcia; Abraham Gallas Torreira; Beatriz Garcia Plana; Themis Bowcock; Francesco Dettori; Karlis Dreimanis; Vinicius Franco Lima; David Hutchcroft; Kurt Rinnert; Tara Shears; Oscar Augusto; Victor Coco; Paula Collins; Tim Evans; Massi Ferro-Luzzi; Heinrich Schindler; Kazu Akiba; Elena Dall' Occo; Cristina Sanchez Graz; Wouter Hulsbergen; Daniel Hynds; Igor Kostiuk; Marcel Merk; Aleksandra Snoch; Dana Seman Bobulska; Silvia Borghi; Stefano De Capua; Deepanwita Dutta; Marco Gersabeck; Chris Parkes; Peter Svihra; Mark Williams; Galina Bogdanova; Vladimir Volkov; Pawel Kopciewicz; Maciej Majewski; Agnieszka Oblakowska-Mucha; Bartlomej Rachwal; Tomasz Szumlak; Lucas Meyer Garcia; Franciole Marinho; Larissa Helena Mendes; Irina Nasteva; Juan Otalora; Gabriel Rodrigues; Jaap Velthuis; Pawel Jalocha; Malcolm John; Nathan Jurik; Luke Scantlebury-Smead; John Back; Tim Gershon; Tom Latham; Andrew Morris

doi:10.1109/TNS.2021.3085018

Readout Firmware of the Vertex Locator for LHCb Run 3 and beyond

Karol Hennessy, Antonio Fernandez Prieto, Pablo Vazquez Regueiro, Jan Buytaert, Martin Van Beuzekom, Edgar Lemos Cid, Lars Eklund, Kristof De Bruyn, Sneha Naik, Manuel Schiller, Donal Murray, Alexander Leflat, Giovanni Bassi, Giovanni Punzi, Federico Lazzari, Michael J. Morello, Oscar Boente Garcia, Abraham Gallas Torreira, Beatriz Garcia Plana, Themis BowcockFrancesco Dettori, Karlis Dreimanis, Vinicius Franco Lima, David Hutchcroft, Kurt Rinnert, Tara Shears, Oscar Augusto, Victor Coco, Paula Collins, Tim Evans, Massi Ferro-Luzzi, Heinrich Schindler, Kazu Akiba, Elena Dall' Occo, Cristina Sanchez Graz, Wouter Hulsbergen, Daniel Hynds, Igor Kostiuk, Marcel Merk, Aleksandra Snoch, Dana Seman Bobulska, Silvia Borghi, Stefano De Capua, Deepanwita Dutta, Marco Gersabeck, Chris Parkes, Peter Svihra, Mark Williams, Galina Bogdanova, Vladimir Volkov, Pawel Kopciewicz, Maciej Majewski, Agnieszka Oblakowska-Mucha, Bartlomej Rachwal, Tomasz Szumlak, Lucas Meyer Garcia, Franciole Marinho, Larissa Helena Mendes, Irina Nasteva, Juan Otalora, Gabriel Rodrigues, Jaap Velthuis, Pawel Jalocha, Malcolm John, Nathan Jurik, Luke Scantlebury-Smead, John Back, Tim Gershon, Tom Latham, Andrew Morris

Research output: Contribution to Journal › Article › Academic › peer-review

Abstract

The new LHCb Vertex Locator (VELO) for LHCb, comprising a new pixel detector and readout electronics, will be installed in 2021 for data taking in Run 3 at the LHC. The electronics centers around the 'VeloPix' ASIC at the front-end operating in a trigger-less readout at 40MHz. A custom serializer, called gigabit wireline transmitter (GWT), and associated custom protocol have been designed for the VeloPix. The GWT data are sent from the serializers of the VeloPix at a line rate of 5.12 Gb/s, reaching a total data rate of 2-3 Tb/s for the full VELO detector. Data are sent over 300-m optic-fiber links to the control and readout electronics cards for deserialization and processing in Intel Arria 10 FPGAs. Because of the VeloPix trigger-less design, latency variances up to 12 $\mu \text{s}$ can occur between adjacent datagrams. It is therefore essential to buffer and synchronize the data in firmware prior to onward propagation or suffer a huge CPU-processing penalty. This article will describe the architecture of the readout firmware in detail with focus given to the resynchronization mechanism and techniques for cauterization. Issues found during readout commissioning, and scaling resource utilization, along with the their solutions, will be illustrated. The latest results of the firmware data-processing chain can be presented as well as the verification procedures employed in simulation. Challenges for the next generation of the detector will also be presented with ideas for a readout processing solution.

Original language	English
Pages (from-to)	2472-2479
Journal	IEEE Transactions on Nuclear Science
Volume	68
Issue number	10
DOIs	https://doi.org/10.1109/TNS.2021.3085018
Publication status	Published - 1 Oct 2021
Externally published	Yes

Funding

This work was supported in part by CERN and the National Agencies: CAPES, CNPq, FAPERJ, and FINEP (Brazil); in part by INFN (Italy); in part by NWO (Netherlands); in part by MEiN and NCN (Poland) under Grant UMO-2018/31/B/ST2/03998; in part by MSHE (Russia); in part by MICINN (Spain); and in part by STFC (U.K.).

Funders	Funder number
MEiN
MSHE
CERN
Science and Technology Facilities Council
Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
Nederlandse Organisatie voor Wetenschappelijk Onderzoek
Conselho Nacional de Desenvolvimento Científico e Tecnológico
Instituto Nazionale di Fisica Nucleare
Narodowe Centrum Nauki	UMO-2018/31/B/ST2/03998
Narodowe Centrum Nauki
Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro
Financiadora de Estudos e Projetos
Ministerio de Ciencia e Innovación

Access to Document

10.1109/TNS.2021.3085018

Persistent URL (handle)

Copy link address

Cite this

Hennessy, K., Prieto, A. F., Regueiro, P. V., Buytaert, J., Van Beuzekom, M., Cid, E. L., Eklund, L., De Bruyn, K., Naik, S., Schiller, M., Murray, D., Leflat, A., Bassi, G., Punzi, G., Lazzari, F., Morello, M. J., Garcia, O. B., Torreira, A. G., Plana, B. G., ... Morris, A. (2021). Readout Firmware of the Vertex Locator for LHCb Run 3 and beyond. IEEE Transactions on Nuclear Science, 68(10), 2472-2479. https://doi.org/10.1109/TNS.2021.3085018

@article{695bf0cf2caf48a6b4bf69263236eb17,

title = "Readout Firmware of the Vertex Locator for LHCb Run 3 and beyond",

abstract = "The new LHCb Vertex Locator (VELO) for LHCb, comprising a new pixel detector and readout electronics, will be installed in 2021 for data taking in Run 3 at the LHC. The electronics centers around the 'VeloPix' ASIC at the front-end operating in a trigger-less readout at 40MHz. A custom serializer, called gigabit wireline transmitter (GWT), and associated custom protocol have been designed for the VeloPix. The GWT data are sent from the serializers of the VeloPix at a line rate of 5.12 Gb/s, reaching a total data rate of 2-3 Tb/s for the full VELO detector. Data are sent over 300-m optic-fiber links to the control and readout electronics cards for deserialization and processing in Intel Arria 10 FPGAs. Because of the VeloPix trigger-less design, latency variances up to 12 $\mu \text{s}$ can occur between adjacent datagrams. It is therefore essential to buffer and synchronize the data in firmware prior to onward propagation or suffer a huge CPU-processing penalty. This article will describe the architecture of the readout firmware in detail with focus given to the resynchronization mechanism and techniques for cauterization. Issues found during readout commissioning, and scaling resource utilization, along with the their solutions, will be illustrated. The latest results of the firmware data-processing chain can be presented as well as the verification procedures employed in simulation. Challenges for the next generation of the detector will also be presented with ideas for a readout processing solution.",

author = "Karol Hennessy and Prieto, {Antonio Fernandez} and Regueiro, {Pablo Vazquez} and Jan Buytaert and {Van Beuzekom}, Martin and Cid, {Edgar Lemos} and Lars Eklund and {De Bruyn}, Kristof and Sneha Naik and Manuel Schiller and Donal Murray and Alexander Leflat and Giovanni Bassi and Giovanni Punzi and Federico Lazzari and Morello, {Michael J.} and Garcia, {Oscar Boente} and Torreira, {Abraham Gallas} and Plana, {Beatriz Garcia} and Themis Bowcock and Francesco Dettori and Karlis Dreimanis and Lima, {Vinicius Franco} and David Hutchcroft and Kurt Rinnert and Tara Shears and Oscar Augusto and Victor Coco and Paula Collins and Tim Evans and Massi Ferro-Luzzi and Heinrich Schindler and Kazu Akiba and Occo, {Elena Dall'} and Graz, {Cristina Sanchez} and Wouter Hulsbergen and Daniel Hynds and Igor Kostiuk and Marcel Merk and Aleksandra Snoch and Bobulska, {Dana Seman} and Silvia Borghi and {De Capua}, Stefano and Deepanwita Dutta and Marco Gersabeck and Chris Parkes and Peter Svihra and Mark Williams and Galina Bogdanova and Vladimir Volkov and Pawel Kopciewicz and Maciej Majewski and Agnieszka Oblakowska-Mucha and Bartlomej Rachwal and Tomasz Szumlak and Garcia, {Lucas Meyer} and Franciole Marinho and Mendes, {Larissa Helena} and Irina Nasteva and Juan Otalora and Gabriel Rodrigues and Jaap Velthuis and Pawel Jalocha and Malcolm John and Nathan Jurik and Luke Scantlebury-Smead and John Back and Tim Gershon and Tom Latham and Andrew Morris",

year = "2021",

month = oct,

day = "1",

doi = "10.1109/TNS.2021.3085018",

language = "English",

volume = "68",

pages = "2472--2479",

journal = "IEEE Transactions on Nuclear Science",

issn = "0018-9499",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "10",

}

Hennessy, K, Prieto, AF, Regueiro, PV, Buytaert, J, Van Beuzekom, M, Cid, EL, Eklund, L, De Bruyn, K, Naik, S, Schiller, M, Murray, D, Leflat, A, Bassi, G, Punzi, G, Lazzari, F, Morello, MJ, Garcia, OB, Torreira, AG, Plana, BG, Bowcock, T, Dettori, F, Dreimanis, K, Lima, VF, Hutchcroft, D, Rinnert, K, Shears, T, Augusto, O, Coco, V, Collins, P, Evans, T, Ferro-Luzzi, M, Schindler, H, Akiba, K, Occo, ED, Graz, CS, Hulsbergen, W, Hynds, D, Kostiuk, I, Merk, M, Snoch, A, Bobulska, DS, Borghi, S, De Capua, S, Dutta, D, Gersabeck, M, Parkes, C, Svihra, P, Williams, M, Bogdanova, G, Volkov, V, Kopciewicz, P, Majewski, M, Oblakowska-Mucha, A, Rachwal, B, Szumlak, T, Garcia, LM, Marinho, F, Mendes, LH, Nasteva, I, Otalora, J, Rodrigues, G, Velthuis, J, Jalocha, P, John, M, Jurik, N, Scantlebury-Smead, L, Back, J, Gershon, T, Latham, T & Morris, A 2021, 'Readout Firmware of the Vertex Locator for LHCb Run 3 and beyond', IEEE Transactions on Nuclear Science, vol. 68, no. 10, pp. 2472-2479. https://doi.org/10.1109/TNS.2021.3085018

TY - JOUR

T1 - Readout Firmware of the Vertex Locator for LHCb Run 3 and beyond

AU - Hennessy, Karol

AU - Prieto, Antonio Fernandez

AU - Regueiro, Pablo Vazquez

AU - Buytaert, Jan

AU - Van Beuzekom, Martin

AU - Cid, Edgar Lemos

AU - Eklund, Lars

AU - De Bruyn, Kristof

AU - Naik, Sneha

AU - Schiller, Manuel

AU - Murray, Donal

AU - Leflat, Alexander

AU - Bassi, Giovanni

AU - Punzi, Giovanni

AU - Lazzari, Federico

AU - Morello, Michael J.

AU - Garcia, Oscar Boente

AU - Torreira, Abraham Gallas

AU - Plana, Beatriz Garcia

AU - Bowcock, Themis

AU - Dettori, Francesco

AU - Dreimanis, Karlis

AU - Lima, Vinicius Franco

AU - Hutchcroft, David

AU - Rinnert, Kurt

AU - Shears, Tara

AU - Augusto, Oscar

AU - Coco, Victor

AU - Collins, Paula

AU - Evans, Tim

AU - Ferro-Luzzi, Massi

AU - Schindler, Heinrich

AU - Akiba, Kazu

AU - Occo, Elena Dall'

AU - Graz, Cristina Sanchez

AU - Hulsbergen, Wouter

AU - Hynds, Daniel

AU - Kostiuk, Igor

AU - Merk, Marcel

AU - Snoch, Aleksandra

AU - Bobulska, Dana Seman

AU - Borghi, Silvia

AU - De Capua, Stefano

AU - Dutta, Deepanwita

AU - Gersabeck, Marco

AU - Parkes, Chris

AU - Svihra, Peter

AU - Williams, Mark

AU - Bogdanova, Galina

AU - Volkov, Vladimir

AU - Kopciewicz, Pawel

AU - Majewski, Maciej

AU - Oblakowska-Mucha, Agnieszka

AU - Rachwal, Bartlomej

AU - Szumlak, Tomasz

AU - Garcia, Lucas Meyer

AU - Marinho, Franciole

AU - Mendes, Larissa Helena

AU - Nasteva, Irina

AU - Otalora, Juan

AU - Rodrigues, Gabriel

AU - Velthuis, Jaap

AU - Jalocha, Pawel

AU - John, Malcolm

AU - Jurik, Nathan

AU - Scantlebury-Smead, Luke

AU - Back, John

AU - Gershon, Tim

AU - Latham, Tom

AU - Morris, Andrew

PY - 2021/10/1

Y1 - 2021/10/1

N2 - The new LHCb Vertex Locator (VELO) for LHCb, comprising a new pixel detector and readout electronics, will be installed in 2021 for data taking in Run 3 at the LHC. The electronics centers around the 'VeloPix' ASIC at the front-end operating in a trigger-less readout at 40MHz. A custom serializer, called gigabit wireline transmitter (GWT), and associated custom protocol have been designed for the VeloPix. The GWT data are sent from the serializers of the VeloPix at a line rate of 5.12 Gb/s, reaching a total data rate of 2-3 Tb/s for the full VELO detector. Data are sent over 300-m optic-fiber links to the control and readout electronics cards for deserialization and processing in Intel Arria 10 FPGAs. Because of the VeloPix trigger-less design, latency variances up to 12 $\mu \text{s}$ can occur between adjacent datagrams. It is therefore essential to buffer and synchronize the data in firmware prior to onward propagation or suffer a huge CPU-processing penalty. This article will describe the architecture of the readout firmware in detail with focus given to the resynchronization mechanism and techniques for cauterization. Issues found during readout commissioning, and scaling resource utilization, along with the their solutions, will be illustrated. The latest results of the firmware data-processing chain can be presented as well as the verification procedures employed in simulation. Challenges for the next generation of the detector will also be presented with ideas for a readout processing solution.

AB - The new LHCb Vertex Locator (VELO) for LHCb, comprising a new pixel detector and readout electronics, will be installed in 2021 for data taking in Run 3 at the LHC. The electronics centers around the 'VeloPix' ASIC at the front-end operating in a trigger-less readout at 40MHz. A custom serializer, called gigabit wireline transmitter (GWT), and associated custom protocol have been designed for the VeloPix. The GWT data are sent from the serializers of the VeloPix at a line rate of 5.12 Gb/s, reaching a total data rate of 2-3 Tb/s for the full VELO detector. Data are sent over 300-m optic-fiber links to the control and readout electronics cards for deserialization and processing in Intel Arria 10 FPGAs. Because of the VeloPix trigger-less design, latency variances up to 12 $\mu \text{s}$ can occur between adjacent datagrams. It is therefore essential to buffer and synchronize the data in firmware prior to onward propagation or suffer a huge CPU-processing penalty. This article will describe the architecture of the readout firmware in detail with focus given to the resynchronization mechanism and techniques for cauterization. Issues found during readout commissioning, and scaling resource utilization, along with the their solutions, will be illustrated. The latest results of the firmware data-processing chain can be presented as well as the verification procedures employed in simulation. Challenges for the next generation of the detector will also be presented with ideas for a readout processing solution.

UR - http://www.scopus.com/inward/record.url?scp=85107329620&partnerID=8YFLogxK

U2 - 10.1109/TNS.2021.3085018

DO - 10.1109/TNS.2021.3085018

M3 - Article

SN - 0018-9499

VL - 68

SP - 2472

EP - 2479

JO - IEEE Transactions on Nuclear Science

JF - IEEE Transactions on Nuclear Science

IS - 10

ER -

Readout Firmware of the Vertex Locator for LHCb Run 3 and beyond

Abstract

Funding

Access to Document

Persistent URL (handle)

Other files and links

Fingerprint

Cite this