Driving is a complex task that requires the ability to receive sensory information, process the information, and to make proper, timely judgments and responses. Older drivers or drivers suffering brain pathologies may have deficits in their driving ability (motor, visual, cognitive or perceptual) and these deficits may lead to reduced driver fitness and increased accident probability. These particularities of impaired driver’s behaviour and safety characteristics make the assessment of their driving ability a very challenging task especially regarding driver distraction which is an important cause of vehicle accidents. Especially in the elderly and people with brain pathologies, because of the degradation of cerebral function, they may be particularly vulnerable to the effects of distraction. The objective of this study is the analysis of reaction time and accident probability of drivers with cognitive impairments due to various brain pathologies, in combination with in-vehicle distraction, through a driving simulator experiment, which was carried out by an interdisciplinary research team of neurologists, neuropsychologists and transportation engineers. The driving scenario concerned driving in rural area, with and without distraction (conversation with passenger and mobile phone use). The brain pathologies examined include early Alzheimer’s disease, Parkinson’s disease, and Mild Cognitive Impairment. 140 participants (out of which 109 were patients) completed the experimental procedure. A statistical analysis was carried out by means of mixed generalized linear modelling and the results indicated significant differences between the driving performance of healthy drivers and patients. Patients with cerebral diseases reacted significantly slower at unexpected incidents than the healthy ones and were more likely to be involved in an accident. The mobile phone use had a significant negative effect on both reaction time and accident probability. Finally, effects of conversation with passengers were less pronounced, although several differences in driving behaviour and safety of group of patients were identifiable.