解读 Waymo Public Road Safety Performance Data

[toc]

在 2018 年 3 月的亚利桑那州坦佩市,一辆 Uber 自动驾驶汽车撞死一名行人的事件至今让人们心有余悸,安全问题自那次事件之后饱受争议。

Uber 自动驾驶撞死行人事故

2020 年 10 月 末,自动驾驶领域的领跑者 -- Waymo,在其 官网 首次公开披露了其在凤凰城运营自动驾驶汽车的里程和碰撞数据,本文基于这份报告,提炼一些观点和数据。

报告传送门:Public Road Safety Performance Data

Waymo 自动驾驶车队

在这份总结了过去 21 个月(2019 年全年和 2020 年前 9 个月)凤凰城郊区自动驾驶汽车运营报告中,Waymo 承认其自动驾驶汽车卷入了 18 起碰撞事故,还有 29 次是 “差点出事故”(在模拟器中),并表示 “几乎所有” 这些碰撞事件都是人类司机或行人的过错,而且没有造成任何 “严重或危及生命的伤害。

报告将碰撞方式分为 “单车事件” 和 “多车辆事件”,其中单车事件包含: 1. 涉及道路偏离、与道路环境 / 基础设施或其他固定对象接触的事件; 2. 涉及机动车辆行驶的事件,碰到行人或骑自行车的人; 3. 车辆被行人或骑自行车的人撞到的事件。

前两种情况相加,能占到致命碰撞贡献率的 60%。

多车辆事件则包括:

  1. 倒车碰撞
  2. 侧擦碰撞
  3. 正面碰撞
  4. 或对方逆行、追尾碰撞
  5. 在十字路口或拐角处的碰撞等等。

在论文中,Waymo case by case 地概述了其他人类驾驶员的 “违反道路规则” 是如何导致 “严重” 碰撞的。

专业名词介绍

  • automated vehicles (AVs)
  • automated driving system (ADS)
  • operational design domain (ODD)

Abstract

  • 这份报告总结了 Waymo 在凤凰城 (Phoenix) 测试区域的 610 万英里的自动驾驶测试数据,包括有车上安全员监管 ( trained operator behind the steering wheel) 的自动驾驶,和 65000 英里的没有安全驾驶员无人驾驶 (driverless operation)。 > The data presented in this paper represents more than 6.1 million miles of automated driving in the Phoenix, Arizona metropolitan area, including operations with a trained operator behind the steering wheel from calendar year 2019 and 65,000 miles of driverless operation without a human behind the steering wheel from 2019 and the first nine months of 2020.

  • 报告中承认,这段测试期间 Waymo 自动驾驶汽车卷入了 18 起真实的碰撞事故和 29 次仿真事故 (被安全驾驶员及时接管,但是在模拟器中发生碰撞事故),所幸均没有造成致命伤害。 > There were 47 contact events that occurred over this time period, consisting of 18 actual and 29 simulated contact events, none of which would be expected to result in severe or life-threatening injuries

  • Waymo 表示,绝大多数事故都和人类驾驶员的不守规则和其他错误相关;由其他司机不谨慎行为诱发的事故频率清楚地提醒人们,只要自动驾驶汽车与人类司机共享道路,避免碰撞就是一大挑战。 > Nearly all the events involved one or more road rule violations or other errors by a human driver or road user. > The presence of collisions that resulted from challenging situations induced by other drivers serves as a reminder of the limits of AV collision avoidance as long as AVs share roadways with human drivers.

  • 在 Waymo 看来,报告不只是向公众公开测试里程和事故,更是做了一个向公众分享自动驾驶行驶安全的一个例子。 > The long-term contributions of this paper are not only the events and mileages shared, but the example set by publicly sharing this type of safety information.

目前,很多自动驾驶的公司可能都在努力创造一个 “黑匣子”,仅在最受控制的环境下向公众展示其技术领先性、跑了多少里程、拿了多少牌照,很少像 Waymo 这样向外界详细披露自动驾驶汽车在现实生活中短板,开始直面问题。

Introduction

  • 报告的目的是公开更多数据,来尽力提高公众对自动驾驶车辆的信心和接受程度 (acceptance)。 > The purpose of this paper is to make available relevant data to promote awareness and discussions that ultimately foster greater public confidence in AVs.

Public Road Testing

  • 在测试初期,为了保证 AVs 在公开道路测试时的安全和行为合理,会有经过培训的驾驶员坐在驾驶员的位置,随时准备接管车辆。 > In order to perform initial public road testing of AVs in a safe and responsible manner, trained vehicle operators are seated in the driver’s seat and can take over the driving task at any time.

The Role of Counterfactual (“What If”) Simulation

Waymo 统计了经过培训的安全驾驶员控制车辆以避免碰撞的事件,然后工程师们会模拟如果驾驶员没有接管车辆自动驾驶系统的话会发生什么,从而产生一个反事实或 “假设” 出来的推断情景。他们利用这些事件来不断检验调整汽车的临场决策反应,然后利用这些数据改进其自动驾驶软件。

  • 反现实的仿真用来预测自车在接管后的一段时间内的表现,给我们提供如果安全员没有接管,那么自车会发生什么的视角。 > Counterfactual disengagement simulation is used to represent the predicted vehicle response for a brief period (seconds) after disengagement, and the simulation outcome provides insight into what could have happened had the trained operator not intervened. > MC: 只能提供极短时间的仿真,否则容易和真实世界背离。另外,需要有精细的模型,可以预期如果没有接管发生,交通参与者的行为。

  • 反事实的接管仿真可以独立使用 (individually) 也可以聚合使用 (in aggregate)。 > The outcomes of counterfactual disengagement simulations are used both individually and in aggregate.

    • 独立仿真时间:场景库的虚拟仿真,可以作为评估软件开发的测试集。 > Individual counterfactual disengagement simulation: If the simulation outcome reveals an opportunity to improve the behavior of the ADS, then the simulation is used to develop and test changes to software algorithms. The disengagement event is also added to a library of scenarios, so that future software can be tested against the scenario. > 在细分领域,自动驾驶的策略会随着接管场景而发生调整和变化,容易造成回滚,而这些场景集合可以作为软件的回滚测试的一部分。
    • 聚合使用:评估 AVs 的路测表现。 > At an aggregate level, Waymo uses results from counterfactual disengagement simulations to produce metrics relevant to the AV’s on-road performance.
  • 软件进化带来的仿真不可复现性问题 > Waymo’s models will continue to evolve, and even for these brief simulations, future models may result in different simulated outcomes.

Aims and Contributions of this Paper

  • 报告包括在 610 万英里测试中的事故总计和事故描述 (包括真实的事故和实际接管但发生仿真事故)。这部分里程,相当于美国普通持证司机 500 多年的驾驶里程。 > This paper includes safety data in the form of event counts and event descriptions from over 6.1 million miles of driving conducted in the Waymo Driver’s driverless ODD. This mileage figure represents over 500 years of driving for the average U.S. licensed driver.

  • 本文提供了两种信息:一种是在或无安全驾驶员情况下的真实碰撞(actual contact event);一种是驾驶员接管,但是在后续的仿真中发生碰撞的,但是因为安全员接管实际没有发生碰撞。 > For these miles, this paper provides information regarding

    1. every actual contact event that vehicles were involved in during driverless operation with and without trained operators, as well as
    2. events in which the vehicle’s trained operator disengaged and subsequent counterfactual simulation resulted in any contact between the AV and the other agent, had the disengagement not occurred

在 2019 年 1 月至 12 月期间,Waymo 在自动驾驶模式下行驶了 610 万英里。从 2019 年 1 月到 2020 年 9 月期间,Waymo 在无人驾驶模式下行驶了 6.5 万英里。该公司表示,综合来看这相当于 “美国普通持证司机 500 多年的驾驶时间”。

Methods

Data sources

  • Waymo 自动驾驶软件的最高速度是 45 英里每小时,换算成公里数是 72.4km/h。除了大雨和沙尘暴天气,会在白天和夜间进行无人驾驶测试。 > The ODD includes roadways with speed limits up to and including 45 miles per hour. Driverless operations occur at all times day and night, except during inclement weather including heavy rain and dust storms.

报告分享的数据来源于两方面: - 无人驾驶(Driverless operation),即车上没有驾驶员,在整个过程中由自动驾驶系统 (automated driving system) 控制车辆。 > Driverless operation, in which the automated driving system controls the vehicle for the entire trip without a human driver behind the wheel or otherwise being available to assume any part of the driving task.
> MC: 这种模式下,在 2019 年初到 2020 年九月底的测试里程是 6.5 万英里。

  • 由驾驶员的自动驾驶(Self-driving with trained operators):虽然由自动驾驶系统控制车辆,但是车上有安全驾驶员可以随时接管车辆。 > Self-driving with trained operators , in which the automated driving system controls the vehicle but there is a trained vehicle operator in the driver’s seat who can disengage and take over the driving task.
    > MC: 这种模式下,2019 年全年的测试里程是 6.1 百万里程。提供一整年的测试数据,可以控制和规避潜在的季节影响 (controlling for potential seasonality effects)。在评估自车表现时,不能忽略季节影响,比如秋天有落叶误感,方向盘急打要比冬季严重。

Data from Actual Collisions and Minor Contacts

  • 数据包括自动驾驶 (self-driving with trained operators mode) 或无人驾驶 (driverless mode) 模式下每一个真实碰撞 (actual collision) 和小事故 (minor contact event),甚至包括行人撞上静止自车的事故。 > This definition encompasses not only every severity of collision, but also events such as a pedestrian walking into the side of the stationary AV.

Data from Counterfactual (“What If”) Simulation

Simulation of the AV motion post-disengagement

  • 接管后的仿真 (post-disengage simulation) 第一步,就是仿真自车的表现 (AV’s counterfactual post-disengage motion)。这样的仿真比较简单,容易快速实现。 > The first step in post-disengage simulation is therefore to simulate the AV’s counterfactual post-disengage motion. > This is performed by providing a simulation running Waymo self-driving software with the AV’s pre-disengage position, attitude, velocity, and acceleration along with the AV’s recorded sensor observations and simulating the response of the software and resulting motion of the Waymo vehicle.
  • 检测是否和其他交通参与者有碰撞,重叠的位置 (overlapping positions) 意味着有潜在的碰撞 (potential collision)。 > Overlapping positions indicate a potential collision. After the AV’s post-disengage motion is simulated, a check is performed to determine if the simulated positions of the AV overlap at any point with the recorded positions of other agents.

  • 仿真第二步:在接管和仿真两种情况下,自车的行为存在不一致 (比如位置和速度不一样),那么就会影响其他参与者的行为,所以做仿真时,参与者的行为可能和实际不符,需要为其他交通参与者重新建模 (modeling the behavior of other agents)。这是比较困难的,需要有比较丰富的模型和交互设计。 > This may not be realistic in cases where the other agents would likely have responded differently to the AV’s counterfactual simulated motion than they did to the AV’s actual post-disengage motion. In such cases, further simulation is required.

  • 总结一下,接管后的仿真,不仅需要使用恰当的软件,仿真自车的行为,也需要仿真其他交通参与者的行为,而这是比较困难的。

Modeling of other agents

  • 为其他交通参与者长时间建模是比较有挑战性的,但是在接管后的短时间内建立冲突避免 (conflict-avoidance) 或避障行为 (collision-avoidance) 是可行的做法。 > While modeling agent behavior over long periods of time is challenging, understanding plausible conflict-avoidance or collision-avoidance behavior over the short time horizon following a disengagement is a more feasible task.

  • Waymo 使用人类避障行为模型 (human collision avoidance behavior models) 来反映短期内障碍物的表现;可以使用离散的多因素 (反应时间、刹车和打方向盘的力度) 来反映交通参与者可行的反应空间 (the space of plausible reactions)。 > Waymo expresses short-term agent responses using human collision avoidance behavior models. > These models aim to capture the responses of human drivers, motorcyclists, cyclists, and pedestrians to collision avoidance situations, such as braking by a lead vehicle or being cut-off by another agent who fails to yield right-of-way. > Because only the agent’s short-term response needs to be modeled, the space of plausible reactions to such stimuli can be defined using a discrete set of factors such as response times to specific inputs and brake or swerve ability.

  • 因为上述的多因素因环境因人而异 (比如反应时间),Waymo 在开发和评估 AV 行为时,使用一个范围来表示可能的人类驾驶表现 (a broad spectrum of potential human driving performance). 为了透明度和简单性,本文使用确定的模型 (deterministic model) 来对一个给定的输入产生给定的输出。 > Waymo considers a broad spectrum of potential human driving performance in developing and evaluating the AV, but for transparency and simplicity, the results reported in this paper are based on deterministic models that generate a single response to a given input.
    > Other methods can be used to capture a range of possible human responses, such as probabilistic counterfactual outcomes, but they are more complex.

  • Waymo 独有的人类避碰模型是基于现在已有的道路使用者的行为模型 (road user behavior modelling frameworks),并基于人类自然的避碰模型 (naturalistic human collision) 和差点碰撞的数据 (near-collision) 来校正。 > Waymo’s proprietary human collsion avoidance behavior models are based on existing road user behavior modelling frameworks and calibrated using naturalistic human collision and near-collision data.

  • 交通参与者的反应被其刹车和转向性能所限制。Waymo 针对不同类型的交通参与者以及不同的触发场景 (different stimuli),使用不同的模型。 > The agent’s response is further constrained by human braking and steering limitations. Waymo uses different models for different types of agents, including heavy trucks, pedestrians, and cyclists, and for different stimuli such as a forward agent braking or an agent emerging from behind an occlusion.
    > MC: 如果单车智能比较高,贴近于人类驾驶员的表现,那么是否可以使用自动驾驶软件的算法模拟车辆的行为;也就是说其他车和自车的表现拟合度比较高,能否使用自车模型来仿真其他车的表现呢?

  • 人类避碰模型的使用场合和条件:当仿真接管后的 AV 轨迹和交通参与者的实际轨迹有重叠,那么就会在接管后的仿真中使用人类避障模型 (human collision avoidance behavior models)。 > Human collision avoidance behavior models are employed for disengagements in which there is overlap between the simulated post-disengage trajectory of the AV and the actual post-disengage trajectory of another agent.
    > In these cases, instead of using the agent’s recorded post-disengage trajectory, the post-disengage trajectory of the other agent is determined by applying the relevant human collision avoidance behavior model.

Contact analysis of simulated collisions

  • Waymo 仿真分析表明,接管很少会造成实际上的碰撞 (大部分是防御性接管)。事实上,超过 99% 的接管没有仿真的接触 (simulated contact) 发生。 > Our simulation analysis indicates that disengagements would rarely result in contact. In fact, in more than 99.9% of disengagements, no simulated contact is found to occur.
    > MC: 因为 Waymo 的软件开发程度比较高,才可以有这样的把握;对于初创期的软件,没有充分的道路测试,切不可使用这样的结论。

Waymo 根据伤害可能性、碰撞物体、撞击速度和撞击几何结构分析,以确定某些事故的严重性,。

  • 如何碰撞发生,那么如何确定碰撞的严重程度 (event severity) 呢?Waymo 根据碰撞障碍物的类型 (collision object),相对速度 (impact velocity) 和碰撞位置 (impact geometry) 来确定可能的伤害程度 (likelihood of injury)。Waymo 使用国家碰撞数据库 (national crash databases) 来为事故严重程度分级 (event severity category),事件的统计等级分为预计无伤害 (S0) 到可能的严重伤害 (S1、S2 和 S3)。 > This determination categorizes collisions based on likelihood of injury and is based on the collision object (e.g., other vehicles, static objects, or vulnerable road users such as pedestrians or cyclists), impact velocity, and impact geometry.
    > Waymo’s methods for determining event severity category are developed using national crash databases and are periodically refined to reflect updated data.

Results: Collisions and Minor Contacts

Collision Categories

上表中的碰撞类型分类 (collision typology) 是根据美国国家机构的专业分类 (using the Manner of Collision categories from National Highway Traffic Safety Administration (NHTSA) collision databases such as the Fatality Analysis Reporting System); 碰撞程度分类是基于国际标准 ISO 26262 来评估碰撞的严重程度,从没有伤害的 S0 (no injury expected) 到有关键伤害的 S3 (possible critical injuries expected),碰撞的伤害逐渐增大。 > categorized in rows according to their collision typology using the Manner of Collision categories from National Highway Traffic Safety Administration (NHTSA) collision databases such as the Fatality Analysis Reporting System.
> columns categorized by estimated event severity using the ISO 26262 severity classes: S0, S1, S2, and S3, ranging from no injury expected (S0) to possible critical injuries expected (S3). * MC: 虽然中美两国的交通法规和驾驶习惯不同,但是这样的交通事故分类是可以拿来参考的。

Waymo 报告中的碰撞事故中,没有 S2 或 S3 级别事故发生,最严重的事故是 S1 级别 (airbag-deployment-level),有 3 次保护气囊弹出。 > There were no actual or predicted S2 or S3 events. One actual event involved deployment of another vehicle’s frontal airbags and the Waymo vehicle’s side airbags.

Waymo 通过和人类的事故相比较,来表明其自动驾驶软件在减少人员伤亡上的可能性。 > Comparison between these human collision statistics and Waymo event counts provides insight into the Waymo Driver’s opportunity for reducing injuries and fatalities due to collisions.

  • In total, the Waymo vehicle was involved in 20 events involving contact with another object and experienced 27 disengagements that resulted in contact in post-disengagement simulation, for a total of 47 events (actual and simulated).
    • 我的困惑点:上述表格中统计的实际碰撞是 18 次,仿真碰撞是 29 次,这和 Waymo 的文字描述是不统一的 (实际碰撞 20 次,虚拟碰撞 27 次);那么是有数据修正吗,还是我粗心搞错了呢?

报告以图片形式 (diagrams have been provided),重点关注了 3 个和弱势交通群体的交互事故,以及 8 个有安全气囊弹出的严重事故。 > Specifically, diagrams have been provided for every actual or simulated event in which a pedestrian or cyclist was involved (three events) and every event with actual or simulated airbag deployment for any involved vehicle (eight events).

Single Vehicle Events

根据 Manner of Collision 的分类标准,交通事故可以分为单车事故 (single vehicle events) 和多车事故 (multiple-vehicle events)。 > The Manner of Collision categories within the NHTSA crash database can be broadly classified as either single vehicle events, which involve a single motorized vehicle in transport, or multiple-vehicle events, which involve the impact of at least two motorized vehicles in transport.

Waymo 自动驾驶车辆没有发生偏离车道 (road departure) 和撞上行人的单车事故 (struck a pedestrian or cyclist);而这类事故在人类驾驶数据中占比约为 60%。 > The Waymo Driver did not have any events (actual or simulated) in this data that involved road departure, contact with the roadway environment/infrastructure or other fixed objects, or rollover. There were also no collisions (actual or simulated) in which the Waymo Driver struck a pedestrian or cyclist.

Waymo 在减速或者静止时,被行人或滑板车从右侧撞上。 > In each instance, the Waymo Driver decelerated and stopped, and a pedestrian or cyclist made contact with the right side of the stationary Waymo vehicle while the pedestrian or cyclist was traveling at low speeds. * MC:我怀疑自车突然急刹,行人来不及反应而撞上去。

Multiple Vehicle Events: Reversing Reversing

倒车碰撞 (reversing collisions) 事故经常发生在停车场,很少出现在交警报告数据库中。 > Reversing collisions (e.g., rear-to-front, rear-to-side, rear-to-rear) are usually associated with parking lot events or occur on local ( ≤ 25 mph) roadways and do not frequently appear in databases of police-reported crashes.

In both scenarios, the Waymo vehicle was stopped or traveling forward at low speed and the other vehicle was reversing at a speed of less than 3 mph at the moment of contact to the side of the Waymo vehicle.

Multiple Vehicle Events: Same Direction Sideswipe

同方向剐蹭 (sideswipe) 主要发生在变道 (lane changing) 和并道 (lane merging) 行为时。 > These events are typically experienced during lane changing or merging maneuvers.
> The Waymo Driver was involved in ten simulated same direction sideswipe collisions.

Other vehicle changing lanes, Waymo vehicle straight

The other vehicle changed lanes into the area occupied by the Waymo vehicle, which resulted in simulated or actual sideswipe collisions.

Other vehicle straight, Waymo vehicle changing lanes

In both of these simulations, the Waymo Driver made a lateral movement in front of a vehicle traveling straight in an adjacent lane.

Multiple Vehicle Events: Head-on or Opposite Direction Sideswipe

对头碰撞极易发生严重的交通事故 (high severity)。 > Head-on collisions have the potential for high severity outcomes.

不能期待驾驶员是理性的,或者一定会有避碰动作:道路情况复杂,你永远不知道司机是什么状态,可能是车辆失控 (impaired) 或疲劳驾驶 (fatigued) 等异常情况。 > The absence of simulated collision avoidance movement by the other vehicle reflects our assumption based on driving behavior and circumstances that the other driver was significantly impaired or fatigued.

如何区别示意图中哪部分是真实,哪部分是仿真的:actual collisions are represented in color, while simulated ones feature a black and white background. Solid trajectory lines represent those observed in real life, while dashed trajectories and shaded poses represent simulated conditions. Diagrams are intended for visual reference only, and are not drawn to scale.

Multiple Vehicle Events: Rear End

追尾碰撞 (rear end collisions) 是最常见的人类驾驶员的碰撞行为。 > Rear End collisions are the most common collision type in human-driven collisions.
> The Waymo Driver was involved in fourteen actual and two simulated rear end collisions

Rear end struck event group, Waymo vehicle stopped or gradually decelerating for traffic controls or traffic ahead while traveling straight

上面的事故,是唯一一个发生在没有安全驾驶员的无人驾驶模式下 (driverless mode)。 > Sole collision in driverless mode, without a trained operator in the driver’s seat.

Rear end struck event group, Waymo vehicle moving slower while traveling straight

In the other collision, the Waymo vehicle, traveling straight at the speed limit, was struck by a vehicle traveling 23 (57-35) mph over the posted speed limit.

Rear end struck event group in right turning maneuvers

These collisions occurred while the Waymo was stationary or near stationary waiting for crossing traffic to clear after having gradually slowed to account for this traffic.

Rear end struck event with braking of lead vehicle during left turn

  • 自车在路口左转急刹停 (a deceleration to a near stop),后面车辆跟车距离不够来不及刹车。 > The remaining rear end struck collision involved a deceleration to a near stop by the Waymo Driver while making a left turn in an intersection with a following vehicle that was traveling at a speed and following distance that did not allow for the following driver to successfully respond to the Waymo Driver’s braking.

Rear end striking event

背景车有挑衅行为 (antagonistic motive),在前方没有障碍物的情况下,故意插入 (cut in) 自车前方后急刹 (braked hard immediately),自车来不及急刹,从后方撞上。这是唯一一起在仿真中,自车追尾其他车的事故。 > The single simulated event in this grouping involved a vehicle that swerved into the lane in front of the Waymo and braked hard immediately after cutting in despite lack of any obstruction ahead (consistent with antagonistic motive).

Multiple Vehicle Events: Angled

Angled collisions, those that are typically seen at intersections and involve crossing or turning vehicles, account for approximately one quarter of all human-driven collisions and a similar fraction of the contribution to all human-driven fatalities.

Angled event group with the other vehicle not yielding to Waymo right-of-way

The collisions in this grouping (ten simulated, one actual) involve the Waymo vehicle traveling straight in a designated lane at or below the speed limit. In all scenarios, the turning/crossing other vehicle either disregarded traffic controls or otherwise did not properly yield right-of-way.

路权的定义: > Right-of-way is determined based on the positions of vehicles prior to contact with respect to the intersection geometry, roadway markings, and the status of traffic control devices. Right-of-way is useful as a means of categorizing some events, but it can be insufficient to determine collision responsibilities since it does not reflect all road rule violations (e.g. speeding), nor does it provide information regarding collision avoidability.

为了减少碰撞风险,Waymo 车辆即使在有路权时也会让行 (yielding)。 > In order to avoid collisions, the Waymo Driver recognizes that yielding even when the Waymo vehicle is entitled to right-of-way may be more appropriate to decrease the risk of collision, for example when encountering an incautious other agent.

In both instances, when the simulated Waymo Driver became aware of the other vehicle's intention to enter the travel lane, the simulated Waymo Driver initiated braking in an attempt to avoid/mitigate impact. * 如果检查到其他车不让行,那么 Waymo 自车会开始刹车来避免碰撞。

上面这个碰撞不常见,需要认真思考下。

左转车辆应该礼让直行车辆,如果不礼让,怎么办呢? Waymo 目前的做法也只能是安全员接管。 > The simulated collision in Figure 9 (Event H) depicts a vehicle making a left turn across the Waymo vehicle’s travel path. The Waymo Driver’s simulated response to the vehicle’s action was the initiation of braking just prior to entering the intersection.

这是最严重的一类碰撞事故 (the most severe collision)。 > It is the most severe collision (simulated or actual) in the dataset and approaches the boundary between S1 and S2 classification.

Angled event group with Waymo vehicle crossing another vehicle’s path

The collisions in this grouping involve four simulated collisions, where the Waymo Driver was making a right turn from a rightmost lane that was either splitting to an additional lane, or had been the result of two lanes merging to one.

Waymo 向右变道时和右侧车道的直行车辆发生碰撞,类似于国内的自车右转,与自行车道的交互。 > In each event, a passenger vehicle attempted to pass the Waymo vehicle on the right while the Waymo Driver was slowing to make the right turn with the right turn signal activated.

Discussion

这份报告的目的是:供工业界,政策制定者和公众学习;促进关注和讨论,以及加快公众对自动驾驶的接受程度。 > The goal of this transparency is to contribute to broad learning with the industry, policymakers, and the public; promote awareness and discussions; and foster greater public confidence in automated vehicles.

Of the fifteen angled events, eleven events were characterized by the other vehicle failing to properly yield right-of-way to the Waymo vehicle traveling straight at or below the speed limit. > MC: 针对自车有路权,其他车辆不让行的情况,Waymo 也依赖安全驾驶员及时接管。

  • 人类驾驶行为的不稳定性和不安全性,导致了绝大多数的碰撞。几乎所有的事故都是因为其他交通参与者违背交通规则或者交通表现有偏差。 > Humans exhibit a large variation of driving behaviors including deviations from traffic rules and safe driving performance that can lead to collisions.
    > Nearly all events summarized above involved one or more road rule violations or other driving performance deviations by another road user.

  • Waymo 在努力实现不因为自身原因造成碰撞的同时,也尽量减少因为人类走神 (inattention),激进驾驶 (aggressive driving) 和超速 (speeding) 带来的可能碰撞。 > In addition to Waymo's key focus on not causing collisions, Waymo also works to mitigate possible collisions due to human behaviors such as inattention, aggressive driving, and speeding.

  • Although many of these situations would not be present in a future with a high proportion of AVs, we envision sharing roads with human drivers for the foreseeable future. The rare contact events described in this paper are used to develop enhanced collision avoidance to improve traffic safety, and we will continue to focus on enhancing avoidance of human-induced collisions. > 这些场景在道路上不常见,但是也无法避免,自动驾驶同行可以反思遇到的这些 corner cases.

  • Waymo 在努力减少事故发生的可能性 (likelihood),而不仅仅是避免碰撞。自车的行为,要能够被其他交通参与者 (other road users) 可判断 (interpretable) 和可预测 (predictable)。 > Beyond collision avoidance, Waymo also continually investigates improvements to the Waymo Driver’s behaviors to reduce the likelihood of conflict with human-driven vehicles and other road users.
    > This illustrates a key challenge faced by AVs operating in a predominantly human traffic system and underscores the importance of driving in a way that is interpretable and predictable by other road users.

  • 相比于人类,Waymo 的自动驾驶能力是可以不断提升的,适用到整个车队上。 > Unlike human drivers, who primarily improve through individual experience, the learnings from an event experienced by a single AV can be used to permanently improve the safety performance of an entire fleet of AVs.

Aggregate Safety Performance

Waymo 车辆在单车表现 (single-vehicle collision typology) 和 追尾问题 (rear-end collisions) 上的良好表现,已经优于人类。 > The Waymo Driver experienced zero actual or simulated events in the “road departure, fixed object, rollover” single-vehicle collision typology, reflecting the system's ability to navigate the ODD in a highly reliable manner.
> In addition, while rear-end collisions are one of the most common collision modes for human drivers, the Waymo Driver only recorded a single front-to-rear striking collision (simulated) and this event involved an agent cutting in and immediately braking without allowing for adequate separation distance (consistent with antagonistic motive).

Lower-severity collision risk

无论是人类驾驶员,还是自动驾驶车辆,轻微事故 (lower-severity events) 发生的频率要高于严重事故 (higher-severity) 的发生频率。 > In both human-driven and automated vehicles, lower-severity events (S0 and S1) occur at significantly higher frequency than higher-severity (S2 and S3) events. As a result, fewer miles are needed to draw statistical conclusions about S0 and S1 rates.
> When comparing driving data, the mileage needed to reveal statistically significant differences also depends on the magnitude of the differences in the actual rates being compared.

For a given metric, the larger the difference in performance, the fewer miles that are required to establish statistical confidence in a hypothesis of non-inferiority or superiority.

如何统计和判责 > low-severity data, when evaluated in the context of each event’s collision geometry, may be informative of high-severity risk.

现有的道路公开测试,可以为 S0 or S1 提供统计上的支持 (sufficient statistical signal)。 > The 6.1 million miles in self-driving with trained operators mode underlying the data in Section 3 provide sufficient statistical signal to detect moderate-to-large differences in S0 and S1 event frequencies, and Waymo makes use of these event rates for tracking longer-term improvements to the Waymo Driver.

Higher-severity collision risk

现有的道路公开测试,无法为 S2 or S3 提供强有力的统计支持。 > 6.1 million miles does not provide statistical power to draw meaningful conclusions about the frequencies of events of severity S2 or S3.
> MC:目前的测试里程,可以为 Lower-severity 提供支持,但是不能为偶发的 Higher-severity 提供支持,里面有统计噪声 (statistical noise)。
> At this mileage scale, the statistical noise is extremely large and zero or low event counts only provide performance bounds, which necessitates the consideration of other metrics to fully assess the safety of the Waymo Driver.

  1. 通过仿真和封闭场测试来评估高风险的表现. > Waymo uses other methods to evaluate the higher-severity performance, including both simulation-based and closed-course scenario-based collision-avoidance testing.

  2. 从低风险事故中发掘高风险事故的信息。 > Low-severity data, when evaluated in the context of each event’s collision geometry, may be informative of high-severity risk.

Comparison benchmarks

Human driver collision rates have been widely discussed as providing a benchmark for AVs.

警方统计的交通事故可能会忽略一部分低风险事件 (non-police-reportable contact),所以 police-reported 不足以代表人类的真实事故发生频率。 > By including low-speed events involving non-police-reportable contact (e.g. a less than 2 mph vehicle-to-vehicle contact or a pedestrian walking into the side of a stationary vehicle), the scope of events is considerably greater than the scope of police-reported or insurance-reported collisions commonly used to generate performance baselines. As such, comparing the data presented in this paper to police-reported collision numbers is not an apt comparison.

Obtaining reliable event counts that include such minor events typically requires analysis of naturalistic driving data.

Limitations and Future Work

Limitations related to the statistical power of the mileages reported have been discussed in the above section on aggregate collision frequencies. > 即,目前的测试里程,可以为 Lower-severity 提供支持,但是不能为偶发的 Higher-severity 提供支持,里面有统计噪声 (statistical noise)。

Limitations of counterfactual simulations

  • 反事实仿真只是预测,但不是绝对的准确。 > Due to the nature of human agent behavior, disengagement simulations are not definitive: counterfactual simulations predict what could have occurred, but cannot definitively predict exactly what would have occurred.
    > As a result, had the driver not disengaged, some of the reported simulated collisions may not have actually occurred (e.g. other agents may have behaved differently). Conversely, other events that, in simulation did not result in contact, may have actually resulted in collisions (e.g. if the other agent had been distracted at the critical moment).

  • Waymo therefore takes a cautious approach to interpreting both the outcomes of individual collisions and aggregate performance metrics, and considers them in the context of other indicators of AV performance.

Secondary collision in simulated events

The severities ascribed to the simulated collisions are based on the single impact depicted in the simulation. Owing to complexities in accurately modeling post-impact vehicle dynamics (which may or may not involve subsequent steering and braking maneuvers from the other vehicle), the outcome of any secondary collisions that might occur subsequent to the simulated event are not explicitly modeled.

Waymo 的第一次碰撞,可以包含绝大多数的严重事故,二级事故不是很重要;二次碰撞需求显得不是那么紧迫。 > In Waymo’s ODD, the vast majority of primary vehicle-to-vehicle collisions (99% for all collisions, 95% for fatal collisions) included in police-reported crash databases involve either a single vehicle-to-vehicle collision event or a subsequent collision event of equal or lesser severity.

Interpreting disengage performance

Waymo 安全驾驶员的避障表现,不代表人类驾驶员的避障表现。 > Care should be taken in drawing conclusions based on the collision-avoidance performance of Waymo’s trained operators during disengagements, which for the reasons described below, is not predictive of the collision-avoidance performance of the overall population of human drivers.

Waymo 驾驶员和普通人类驾驶员的区别在哪里呢?

  1. 受过专业培训: Waymo vehicle operators are selected from a subset of the driving population with good driving records and receive instruction specific to Waymo AVs, defensive driving training, and education regarding fatigue.
  2. 避免被打扰:When operating a vehicle, strict rules are in place regarding handheld devices including cell phones and operators are continually monitored for signs of drowsiness.
  3. 注意力更集中:Unlike drivers in human-driven vehicles, while the AV is in self-driving mode, Waymo’s trained operators do not execute navigation, path planning, or control tasks, but instead are focused on monitoring the environment and the Waymo Driver’s response to it.

Trained vehicle operators are therefore able to focus their full attention on being ready to disengage and execute collision avoidance, and their performance at this task is expected to be superior to that of a human in a traditional driving role.

Future work

We expect and invite other safety researchers to review the events and mileages presented here and make their own findings regarding the safety performance of Waymo’s operations demonstrated in this data.

Conclusion

Taken together, these 47 lower severity (S0 and S1) events (18 actual and 29 simulated, one during driverless operation) show significant contribution from other agents, namely human-related deviations from traffic rules and safe driving performance.

The frequency of challenging events that were induced by incautious behaviors of other drivers serves as a clear reminder of the challenges in collision avoidance so long as AVs share roadways with human drivers.

由其他司机不谨慎行为诱发的事故频率清楚地提醒人们,只要自动驾驶汽车与人类司机共享道路,避免碰撞就是一大挑战。只要和人类共享道路 (share roads),完全安全的自动驾驶是不可行的,需要降低过高的期望 (inflated expectations)。 > Statistics regarding the high percentage of human collisions that are attributed to human error may lead to inflated expectations of the potential immediate safety benefits of AVs. AVs will share roads with human drivers for the foreseeable future, and significant numbers of collisions due to human driver errors that are simply unavoidable should be expected during this period.

  • 和人类现有数据比较,碰撞分布发生变化;可以认清楚自动驾驶的优点 (机器稳定以及不知疲倦) 以及劣势,优点可以及早加以利用,但是如何规避缺点却是一个难点。 > Due to the typology of those collisions initiated by other actors as well as the Waymo Driver’s proficiency in avoiding certain collision modes, the data presented shows a significant shift in the relative distributions of collision types as compared to national crash statistics for human drivers.

  • 这是业界首次发布百万英里自动驾驶中的碰撞事故。评估自动驾驶汽车安全性没有标准的方法,Waymo 作为行业领头羊的担当,希望通过公布这些数据,推动政策制定者、研究人员甚至其他公司承担制定通用框架的任务。 > This is the first time that information on every actual and simulated collision or contact has been shared for millions of miles of automated driving.
    > The most significant long-term contributions of this paper will likely not be the actual data shared, but the example set by publicly sharing this type of safety performance data and the dialogs that this paper fosters.

References