Autonomous point to point shuttles are an emerging paradigm of a future mobility-on-demand ecosystem. However, the traffic and environmental impacts of their operation are largely under researched especially in relation to influential infrastructure related factors and service-related specifications. The scope of this study is to reveal the factors that may affect the degree and magnitude of the road segment level impacts of an autonomous urban shuttle service (AUSS) operating in a city using microsimulation and structural equation modeling (SEM). For the purposes of this research, a systematic framework is developed and applied in the city center of Athens (Greece), which encompasses different scenarios of operations including: (i) Baseline (no AUSS operation), (ii) AUSS operation with a dedicated lane during peak hour, (iii) AUSS operation mixed with regular traffic during peak hour and (iv) AUSS operation mixed with regular traffic during off-peak hour. Two connected automated vehicle (CAV) profiles were used to model the advent of automation in the overall traffic: a cautious profile is introduced first, followed by a more aggressive profile. SEM findings indicate that the AUSS operation has a significant effect on cumulative travel time per segment and CO2 emissions per segment only during the scenario of mixed operation with traffic during off-peak hours. Additionally, the influence of the network geometry is correlated with reduced travel time and with increased CO2 emissions. Road traffic density was found to be positively correlated with both travel time and CO2 emissions, while the penetration of both cautious and aggressive CAVs was found to be negatively correlated with both indicators.