I don't very much trust Elon's vision in this regard, with respect to the near-to-medium term (say, the next 5 years or so). It seems very unlikely that within the next 5 years technology will have advanced sufficiently to enable that level of automated driving in day-to-day traffic.

He might have intuition in this area, but one individual's intuition is a poor substitute for actual data.