Objective: Trans-synaptic axonal degeneration is a mechanism by which neurodegeneration can spread from a sick to a healthy neuron in the central nervous system. This study investigated to what extent trans-synaptic axonal degeneration takes place within the visual pathway in multiple sclerosis (MS). Methods: A single-centre study, including patients with long-standing MS and healthy controls. Structural imaging of the brain (MRI) and retina (spectral-domain optical coherence tomography) were used to quantify the extent of atrophy of individual retinal layers and the primary and secondary visual cortex. Generalised estimation equations and multivariable regression analyses were used for comparisons. Results: Following rigorous quality control (OSCAR-IB), data from 549 eyes of 293 subjects (230 MS, 63 healthy controls) were included. Compared with control data, there was a significant amount of atrophy of the inner retinal layers in MS following optic neuritis (ON) and also in absence of ON. For both scenarios, atrophy stopped at the level of the inner nuclear layer. In contrast, there was significant localised atrophy of the primary visual cortex and secondary visual cortex in MS following ON, but not in MS in absence of ON. Interpretation: These data suggest that retrograde (trans-synaptic) axonal degeneration stops at the inner nuclear layer, a neuronal network capable of plasticity. In contrast, there seems to be no neuroplasticity of the primary visual cortex, rendering the structure vulnerable to anterograde (trans-synaptic) degeneration.