自动驾驶汽车测试安全员综合能力评价方法

缪明月; 裴舜嗣; 顾礼源; 赵方霞; 黄利华; 赖见辉

doi:10.3963/j.issn.1674-4861.2026.04.007

自动驾驶汽车测试安全员综合能力评价方法

doi: 10.3963/j.issn.1674-4861.2026.04.007

1.
首都经济贸易大学管理工程学院北京 100070
2.
北京工业大学城市交通学院北京 100124

基金项目:

教育部人文社科项目 21YJCZH060

北京市教委科技一般项目 KM202410038001

详细信息

通讯作者:
缪明月（1977—），博士，副教授. 研究方向: 交通安全、交通仿真、交通法律. E-mail：miaoming_f@cueb.edu.cn

中图分类号: U491.2
计量
- 文章访问数: 18
- HTML全文浏览量: 10
- PDF下载量: 0
- 被引次数: 0
出版历程
- 收稿日期: 2026-01-27
- 网络出版日期: 2026-11-06

Comprehensive Ability Evaluation Method for Autonomous Vehicle Test Safety Officers

1.
School of Management Engineering, Capital University of Economics and Business, Beijing 100070, China
2.
College of Metropolitan Transportation, Beijing University of Technology, Beijing 100124, China

摘要

摘要: 针对当前自动驾驶测试安全员（以下简称"安全员"）综合能力评价缺乏可量化、可操作的评估体系，以及现有驾驶人评价框架无法覆盖"监控-接管-反馈"全职责链条的问题，从自动驾驶测试安全员的岗位特性出发，构建了涵盖反应能力、驾驶能力和描述能力3个维度的综合能力评价指标体系。选取50名一线安全员，设计涵盖无驾驶任务、人工驾驶和自动驾驶3种模式的分阶段实验，采集反应时间、驾驶操作和问题描述等6类13项原始数据。在反应能力评价方面，以反应时间均值、方差和错误次数为核心指标，对高斯混合模型（Gaussian mixture model，GMM）与极端随机树（extremely randomized trees，ET）、随机森林（random forest，RF）、极端梯度提升（eXtreme gradient boosting，XGBoost）等6种算法组合进行比较，经"聚类质量-信息保留度-变量利用充分性—分类精度"四级递进筛选，通过轮廓系数（0.82）、赤池信息准则（186.3）、权重分布均匀度和PRF（查准率、查全率、$F_{1}$分数）等多维指标综合筛选，确定GMM+ET为反应能力最优评价模型，查准率、查全率和$F_{1}$分数均达到1.00，十折交叉验证平均准确率为0.99。在驾驶能力评价方面，构建以事故发生次数为因变量的多元线性回归模型（调整后$R^{2}=0.552, \operatorname{Prob} $（F-statistic）$<0.01$），识别出最大制动值、平均油门程度和最佳速度差值为影响驾驶能力的3个核心指标。在描述能力评价方面，采用多专家分级评分法，从准确度与精炼度2个维度对安全员的问题反馈质量进行量化评定。依据事故归因统计与德尔菲法专家校准，确定反应能力、驾驶能力和描述能力的权重分别为0.402、0.496、0.102。结果表明: 3项能力维度相互独立，反应能力与驾驶能力的皮尔逊相关系数为- 0.123；驾驶能力得分与综合排名的相关性最强（$r=0.87$），是影响综合能力的核心因素；GMM+ ET融合模型的十折交叉验证平均准确率为$0.99 、F_{1}$分数为1.00，在变量利用充分性与评价精度上显著优于GMM+RF（F₁均值0.63）与k-means+XGBoost（F₁均值0.58）。
- 交通安全 /
- 自动驾驶测试安全员 /
- 综合能力评价方法 /
- 高斯混合模型与极端随机树 /
- 反应时间 /
- 多元线性回归
Abstract: To address the absence of a quantifiable and operational evaluation framework for the comprehensive competence of autonomous vehicle test safety officers, and the inability of existing driver assessment systems to cover the full"monitor-takeover-feedback"responsibility chain, a three-dimensional evaluation index system encompassing reaction ability, driving ability, and description ability was constructed based on the job characteristics of test safety officers.Fifty frontline test safety officers participated in a staged experiment covering three modes: no-driving task, manual driving, and autonomous driving.Six categories comprising thirteen raw data indicators were collected, including reaction time, driving operations, and problem descriptions.For reaction ability evaluation, reaction time mean, variance, and error count were adopted as core indicators.Six algorithmic combinations involving Gaussian Mixture Model (GMM), Extremely Randomized Trees (ET), Random Forest (RF), and eXtreme Gradient Boosting (XGBoost) were preliminarily compared and screened through a four-level progressive strategy of clustering quality, information retention, variable utilization sufficiency, and classification accuracy, based on the silhouette coefficient (0.82), Akaike information criterion (AIC) of 186.3, weight distribution uniformity, and PRF (Precision, Recall, $F_{1}$-score).The GMM+ET fusion model was identified as the optimal evaluation model, achieving precision, recall, and $F_{1}$-score all equal to 1.00, with a mean accuracy of 0.99 under ten-fold cross-validation. For driving ability evaluation, a multiple linear regression model (adjusted $R^{2}=0.552, \operatorname{Prob}$ (F-statistic) $<0.01$) with accident frequency as the dependent variable was constructed, identifying maximum braking value, average throttle input, and optimal speed deviation as the three core influencing indicators.For description ability evaluation, a multi-expert grading method was employed to quantify the accuracy and conciseness of safety officers'problem feedback.Based on accident attribution statistics and Delphi method expert calibration, the weights of reaction ability, driving ability, and description ability were determined as 0.402, 0.496, and 0.102, respectively.The results demonstrate that the three ability dimensions are mutually independent, with a Pearson correlation coefficient of-0.123 between reaction and driving abilities.Driving ability score exhibits the strongest correlation with comprehensive ranking $(r=0.87)$, representing the core factor influencing overall competence.With a ten-fold cross-validation mean accuracy of 0.99 and $F_{1}$-score of 1.00, the GMM+ET fusion model significantly outperforms GMM+RF (mean $F_{1}$-score of 0.63) and k-means+XGBoost (mean $F_{1}$-score of 0.58) in variable utilization adequacy and evaluation accuracy.
- traffic safety /
- safety officers for autonomous driving tests /
- comprehensive ability evaluation method /
- Gaussian mixture model and extremely randomized trees /
- reaction time /
- multiple linear regression

HTML全文

图 1 测试实景图

Figure 1. Actual test picture

下载: 全尺寸图片幻灯片

图 2 反应能力测试流程图

Figure 2. Flowchart of the reaction capability evaluation framework

下载: 全尺寸图片幻灯片

图 3 驾驶能力评价流程

Figure 3. Evaluation ideas of driving ability

下载: 全尺寸图片幻灯片

图 4 安全员各能力考核维度得分分布

Figure 4. Distribution of competency dimension scores for test safety officers

下载: 全尺寸图片幻灯片

图 5 各维度得分随综合排名的变化趋势

Figure 5. Trends of competency scores across comprehensive rankings

下载: 全尺寸图片幻灯片

图 6 反应能力与驾驶能力的相关性分析

Figure 6. Correlation analysis between reaction ability and driving ability

下载: 全尺寸图片幻灯片

表 1 实验中直接获取的数据

Table 1. Data obtained directly in the experiment

测试数据	衍生指标	数据获取方式(人工记录、系统自动记录)	数据获取阶段
脚反应时间	平均时间(A)、最短时间(A)、标准差(A)	系统自动记录	一、二、三
手反应时间	平均时间(A)、最短时间(A)、标准差(A)	系统自动记录	一、三
脚反应错误次数	错误次数(A)	系统自动记录	一、二、三
手错误反应次数	错误次数(A)	系统自动记录	一、二、三
驾驶与操作数据	转动转向盘角度和次数(B)、踩踏油门踏板角度和次数(B)、踩踏刹车踏板角度和次数(B)、车速(B)	系统自动记录	二
发生事故及错误的次数	发生事故的次数(B)	人工记录	二
问题描述能力评价分值	描述准确度(C)	人工记录	三
注：A、B、C分别为应用于反应能力评价、驾驶能力评价和问题描述能力评价的指标。

下载: 导出CSV

表 2 被试安全员基本情况

Table 2. Basic information of test safety officers

基本情况	类别	人数(比例/%)
性别	男	46(92)
性别	女	4(8)
年龄/岁	≥18~30	16(32)
	> 30~40	32(64)
	> 40~50	2(4)
	> 50	0(0)
学历	初中及以下	0(0)
	高中	6(12)
	大专	43(86)
	大学及以上	1(2)
驾龄/年	≥1~5	3(6)
	> 5~10	16(32)
	> 10~15	20(40)
	> 15~20	7(14)
	> 20	4(8)

下载: 导出CSV

表 3 算法排名表

Table 3. Algorithm ranking table

排名	模型名称	变量选择个数	权重是否均匀	类别	查准率	查全率	F₁分数
1	GMM+ET	28	是	0	1.00	1.00	1.00
				1	1.00	1.00	1.00
				2	1.00	1.00	1.00
2	GMM+RF	28	是	0	1.00	1.00	1.00
				1	0.50	0.00	0.00
				2	0.80	1.00	0.89
3	k-means+	28	否	0	1.00	1.00	1.00
				1	0.00	0.00	0.00
				2	0.60	1.00	0.75

下载: 导出CSV

表 4 指标权重表

Table 4. Model weight setting table

序号	指标$X_{i}$	权重$a_{i}$	序号	指标$X_{i}$	权重$a_{i}$
0	脚平均反应时间	0.0006	14	一阶段脚错误反应次数	0.1054
1	脚最短反应时间	0.0008	15	一阶段手错误反应次数	0.0642
2	脚反应时间标准差	0.0299	16	二阶段脚平均反应时间	0.0168
3	手平均反应时间	0.1387	17	二阶段脚最短反应时间	0.0173
4	手最短反应时间	0.0049	18	二阶段脚反应时间标准差	0.0119
5	手反应时间标准差	0.0587	19	二阶段脚错误反应次数	0.0374
6	脚错误反应次数	0.2045	20	三阶段脚平均反应时间	0.0248
7	手错误反应次数	0.0203	21	三阶段脚最短反应时间	0.0121
8	一阶段脚平均反应时间	0.0074	22	三阶段脚反应时间标准差	0.0058
9	一阶段脚最短反应时间	0.0203	23	三阶段手平均反应时间	0.0568
10	一阶段脚反应时间标准差	0.0074	24	三阶段手最短反应时间	0.0321
11	一阶段手平均反应时间	0.0279	25	三阶段手反应时间标准差	0.0478
12	一阶段手最短反应时间	0.0345	26	三阶段脚错误反应次数	0.0485
13	一阶段手反应时间标准差	0.0369	27	三阶段手错误反应次数	0.0741

下载: 导出CSV

表 5 驾驶能力评价指标

Table 5. Driving ability evaluation index

序号	驾驶能力	符号
0	最大制动值/(°)	$ B_{\max} $
1	最大制动频数/次	BF
2	平均油门程度/(°)	$ \overline{G} $
3	最佳速度差值/(km/h)	V_d
4	平均速度/(km/h)	$ \overline{V} $
5	转向盘最大转角/(°)	$ SW_{\max} $
6	转向盘最大转角频数/次	SF

下载: 导出CSV

表 6 驾驶能力指标线性回归结果

Table 6. Results of linear regression results of driving ability index

Variable(变量)	Coef(系数)	std err(标准差)	t	P
截距项	-2.638	1.344	-1.963	0.006
$ B_{\max} $	3.892	1.781	2.185	0.037
BF	-0.403	0.503	1.300	0.231
$ \overline{G} $	4.174	3.389	1.235	0.227
V_d	2.159	0.853	2.535	0.017
$ \overline{V} $	-1.962	1.329	-1.476	0.151
$ SW_{\max} $	-13.973	17.214	-0.808	0.426
SF	3.236	3.501	0.924	0.364
$\operatorname{Prob}(F \text {-statistic })=1.360 \times 10^{-5}, R^2=0.535$

下载: 导出CSV

参考文献(27)

[1]	CHU M D, ZONG K Y, SHU X, et al. Work with AI and work for AI: autonomous vehicle safety drivers' lived experiences[C]. The 2023 CHI Conference on Human Factors in Computing Systems (CHI'23), New York: ACM, 2023.
[2]	California Department of Motor Vehicles. Autonomous vehicle tester (AVT) program for manufacturer's testing permit[EB/OL]. (2021-03) [2025-10-11]. https://www.dmv.ca.gov/portal/file/autonomous-vehicle-tester-avt-program-for-manufacturers-testing-permit-pdf/.
[3]	中国道路交通安全协会. 智能网联汽车安全员能力要求与培训考核规范: T/CTS 15—2023[S]. 北京: 2023. China Road Traffic Safety Association. Competency requirements and specification for training and assessment of safety inspectors for intelligent and connected vehicle: T/CTS 15—2023[S]. Beijing, 2023. (in Chinese)
[4]	JOHANSSON G, RUMAR K. Drivers' brake reaction times[J]. Human Factors: The Journal of the Human Factors and Ergonomics Society, 1971, 13(1): 23-27. doi: 10.1177/001872087101300104
[5]	ČULÍK K, KALAŠOVÁ A, ŠTEFANCOVÁ V. Evaluation of driver's reaction time measured in driving simulator[J]. Sensors, 2022, 22(9): 3542. doi: 10.3390/s22093542
[6]	BAUDER M, PAULA D, PFEILSCHIFTER C, et al. Influences of vehicle communication on human driving reactions: a simulator study on reaction times and behavior for forensic accident analysis[J]. Sensors, 2024, 24(14): 4481. doi: 10.3390/s24144481
[7]	CHOUHAN R, DHAMANIYA A, RAO A M, et al. Quantifying the relationship between auditory distractions, reaction time and crash probability[J]. IATSS Research, 2025, 49(1): 60-71. doi: 10.1016/j.iatssr.2025.02.001
[8]	MARMOLEJO-RAMOS F, BARRERA-CAUSIL C, KUAN G S, et al. Generalised exponential-Gaussian distribution: a method for neural reaction time analysis[J]. Cognitive Neurodynamics, 2023, 17(1): 221-237. doi: 10.1007/s11571-022-09813-2
[9]	郭柏苍, 谢宪毅, 金立生, 等. 考虑多维驾驶特性的制动反应时间预测模型[J]. 汽车工程, 2021, 43(11): 1683-1692. GUO B C, XIE X Y, JIN L S, et al. Braking reaction time pre-diction model considering multidimensional driving characteristics[J]. Automotive Engineering, 2021, 43(11): 1683-1692. (in Chinese)
[10]	张骏, 吴志敏, 潘雨帆, 等. 基于SVR模型的驾驶简单反应时间预测方法[J]. 中国公路学报, 2017, 30(4): 127-132. ZHANG J, WU Z M, PAN Y F, et al. Predicting method of simple reaction time of driver based on SVR model[J]. China Journal of Highway and Transport, 2017, 30(4): 127-132. (in Chinese)
[11]	BAO Y L, WANG X S. Optimization of forward collision warning algorithm considering truck driver response behavior characteristics[J]. Accident Analysis & Prevention, 2024, 198: 107450.
[12]	BIONDI F N, MCDONNELL A S, MAHMOODZADEH M, et al. Vigilance decrement during on-road partially automated driving across four systems[J]. Human Factors, 2024, 66(9): 2179-2190. doi: 10.1177/00187208231189658
[13]	王丹, 朱曰莹, 张策, 等. 基于多重客观指标表征的驾驶人接管绩效评估方法[J]. 交通信息与安全, 2024, 42(6): 55-63. doi: 10.3963/j.jssn.1674-4861.2024.06.006 WANG D, ZHU Y Y, ZHANG C, et al. An evaluation model for driver takeover performance based on multi-objective indicators representation[J]. Journal of Transport Information and Safety, 2024, 42(6): 55-63. (in Chinese) doi: 10.3963/j.jssn.1674-4861.2024.06.006
[14]	谢厅, 刘星良, 刘唐志, 等. 基于DE-EL的城市快速路合流区危险驾驶行为识别方法[J]. 交通信息与安全, 2024, 42(6): 23-30. doi: 10.3963/j.jssn.1674-4861.2024.06.003 XIE T, LIU X L, LIU T Z, et al. A recognition method for risky driving behaviors of urban expressway merging area based on DE-EL model[J]. Journal of Transport Information and Safety, 2024, 42(6): 23-30. (in Chinese) doi: 10.3963/j.jssn.1674-4861.2024.06.003
[15]	孙博华, 邓伟文, 吴坚, 等. 驾驶人纵侧向驾驶能力评价方法研究[J]. 北京理工大学学报, 2019, 39(11): 1149-1154. SUN B H, DENG W W, WU J, et al. Research on evaluation method of driver's longitudinal and lateral driving ability[J]. Transactions of Beijing Institute of Technology, 2019, 39(11): 1149-1154. (in Chinese)
[16]	涂辉招, 崔航, 鹿畅, 等. 面向自动驾驶路测驾驶能力评估的避险脱离率模型[J]. 同济大学学报(自然科学版), 2020, 48(11): 1562-1569. TU H Z, CUI H, LU C, et al. Risk avoidance disengagement rate model for driving ability assessment in autonomous vehicle road testing[J]. Journal of Tongji University (Natural Science), 2020, 48(11): 1562-1569. (in Chinese)
[17]	吴付威, 杨慧, 马勇, 等. L3级自动驾驶车辆两阶段预警对接管绩效影响研究[J]. 交通运输系统工程与信息, 2025, 25(5): 40-49. WU F W, YANG H, MA Y, et al. Impact of two-stage warning on takeover performance of L3 autonomous vehicles[J]. Journal of Transportation Systems Engineering and Information Technology, 2025, 25(5): 40-49. (in Chinese)
[18]	CHEN L J, LI D F, WANG T, et al. Takeover performance prediction based on LSTM-BiLSTM-ATTENTION deep learning method[J]. Systems, 2025, 13: 46. doi: 10.3390/systems13010046
[19]	秦雅琴, 胡浩玮, 蒋玉博. 动、静态驾驶适宜性相关性分析[J]. 安全与环境学报, 2019, 19(5): 1643-1648. QIN Y Q, HU H W, JIANG Y B. Correlation analysis of dynamic and static driving aptitude[J]. Journal of Safety and Environment, 2019, 19(5): 1643-1648. (in Chinese)
[20]	赵红专, 张继康, 潘佳雯, 等. 基于改进视觉算法的自动驾驶风险预判模型[J]. 交通运输系统工程与信息, 2024, 24(5): 79-87. ZHAO H Z, ZHANG J K, PAN J W, et al. Automatic driving risk prediction model based on improved vision algorithm[J]. Journal of Transportation Systems Engineering and Information Technology, 2024, 24(5): 79-87. (in Chinese)
[21]	HORN S, ROSSNER P, MADIGAN R, et al. User evaluation of comfortable deceleration profiles for highly automated driving: findings from a test track study[J]. Transportation Research Part F: Traffic Psychology and Behaviour, 2024, 105: 206-221. doi: 10.1016/j.trf.2024.05.025
[22]	LIU Y J, WU C Z, ZHANG H, et al. Safety evaluation and prediction of takeover performance in automated driving considering drivers' cognitive load: a driving simulator study[J]. Transportation Research Part F: Traffic Psychology and Behaviour, 2024, 103: 35-52. doi: 10.1016/j.trf.2024.03.021
[23]	WU G M, WU D. Identifying drivers' perception-reaction time (PRT) in car-following processes via two different methods using vehicle trajectory data[J]. Safety and Reliability, 2024, 43(4): 207-224. doi: 10.1080/09617353.2024.2369960
[24]	SCHLENOFF C I, KOOTBALLY Z, RACHA-KONDA P, et al. Standards and performance metrics for on-road automated vehicles: workshop report[R]. Gaithersburg, MD: National Institute of Standards and Technology, 2024. NIST IR 8527.
[25]	PFISTER M, LUE J C L, STEFANINI F R, et al. Comparison of reaction response time between hand and foot controlled devices in simulated microsurgical testing[J]. Biomed Research International, 2014, 2014: 769296.
[26]	HOFFMANN E R. A comparison of hand and foot movement times[J]. Ergonomics, 1991, 34(4): 397-406. doi: 10.1080/00140139108967324
[27]	SALUNKHE G, FEIGE B, SAVILLE C W N, et al. Dissociating slow responses from slow responding[J]. Frontiers in Psychiatry, 2020, 11: 505800. doi: 10.3389/fpsyt.2020.505800