This study tries to identify and review risk factors that impact road safety and driving performance focusing more on three types of users, namely elderly drivers, truck operators, and office workers. The present review is part of research conducted within the EU H2020 HADRIAN project, which aims at developing an innovative Human Machine Interface (HMI) that will provide seamless “fluid” interactions between the driver and the automated vehicle. The literature review was conducted using popular databases such as Google Scholar, Science Direct, and Scopus using specific search terms and prioritization criteria for studies coming mainly from Europe and the U.S, and after 2005. Then, the reviewed risk factors were discussed based on how they can be extended and adapted at the different future AD levels and what risk factors should be considered in future safety analysis depending on the AD level. The aforementioned review will be exploited by the HADRIAN project, like any other HMI stakeholder, in order to develop a human-centered assessment methodology that will evaluate the way the human interacts with potential HMI configurations. Finally, the reviewed risk factors could guide stakeholders in accomplishing a safer transition from manual to autonomous driving for all road users.