The advent of autonomous vehicles (AVs) is a breakthrough innovation in the field of transportation. At the cornerstone of autonomous vehicles (AVs) research lies the challenge of ensuring that the future vehicles can react properly and efficiently in all situations and especially in emergencies. The present work analyzes autonomous vehicle’s “driver”/operator behavior and conceptualizes the changes that should be introduced to the existing behavioral driving models in order to address the requirements of autonomous vehicles and increase the acceptance of autonomous driving. For this purpose, empirical evidence and qualitative experience from over 20 relevant projects and pilots are critically reviewed. Moreover, the conceptualization and potential parameterization of AV behavioral models are analyzed based on three popular modeling alternatives: Summala’s Multiple Comfort Zone Model, Fuller’s Risk Allostasis Model (RAT), Vaa’s Risk Monitor Model (RMM). The findings are critically discussed to reveal future research directions.