Volume 43 Issue 6
Dec.  2025
Turn off MathJax
Article Contents
Article Navigation >  Journal of Transport Information and Safety  > 2025  >  43(6): 11-20
FAN Bo, ZHOU Chongwei, ZHANG Sinan, YANG Jun, CHEN Yanyan, LI Tongfei. Scenario-based Testing and Evaluation Systems for Autonomous Vehicles: Research Status, Challenges, and Trends[J]. Journal of Transport Information and Safety, 2025, 43(6): 11-20. doi: 10.3963/j.jssn.1674-4861.2025.06.002
Citation: FAN Bo, ZHOU Chongwei, ZHANG Sinan, YANG Jun, CHEN Yanyan, LI Tongfei. Scenario-based Testing and Evaluation Systems for Autonomous Vehicles: Research Status, Challenges, and Trends[J]. Journal of Transport Information and Safety, 2025, 43(6): 11-20. doi: 10.3963/j.jssn.1674-4861.2025.06.002
shu

Scenario-based Testing and Evaluation Systems for Autonomous Vehicles: Research Status, Challenges, and Trends

doi: 10.3963/j.jssn.1674-4861.2025.06.002

  • College of Metropolitan Transportation, Beijing University of Technology, Beijing 100124, China

  • Received Date: 2025-06-15
    Available Online: 2026-03-13
    Abstract
  • Autonomous driving technology is accelerating toward large-scale testing and commercial application. Consequently, constructing systematic scenario frameworks for testing and robust evaluation metrics is crucial for safe deployment. This paper reviews the research status, challenges, and future trends of these systems. The study analyzes complexities introduced by vehicle-road-cloud integration and dynamic mixed traffic. It finds that traditional"mileage-failure"statistical models are insufficient for end-to-end performance assessment. Regarding test scenarios, the paper outlines the evolution toward scenario-driven paradigms. It summarizes semantic description methods based on the ISO 34501 standard and the PEGASUS six-layer model. Mainstream scenario generation technologies are also reviewed. Current frameworks show insufficient coverage of long-tail and edge scenarios. Standards are highly fragmented. Furthermore, existing frameworks often under-represent vehicle-to-everything (V2X) collaborative elements due to an excessive focus on single-vehicle intelligence. Regarding evaluation metrics, existing methodologies are categorized into three dimensions, including competition-based, closed-track/simulation hybrid, and theory-oriented approaches. The review identifies several deficiencies in current systems. Specifically, current metrics insufficiently assess the use of V2X collaborative information. Evaluation dimensions and workflows are fragmented, and objective quantitative metrics for interactive experience are lacking. To address these challenges, next-generation testing systems should focus on four research paths. ①Unified scenario description languages and data-sharing frameworks are needed to establish benchmarks for measuring scenario risk criticality and realism. ② Hierarchical scenario systems should be built to cover nominal conditions as well as long-tail boundaries for full-domain coverage. ③Comprehensive metrics should integrate communication latency, system resilience, and social ethics. ④World models and generative AI, combined with causal inference, can simulate extreme conditions and explore unknown failure modes to validate the system's generalization capability.

     

    • FullText(HTML)
  • loading
    • References(47)
  • [1]
    FENG S, FENG Y H, SUN G W, et al. Testing scenario library generation for connected and automated vehicles, part Ⅰ: methodology[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22(3): 1573-1582. doi: 10.1109/TITS.2020.2972211
    [2]
    孙航, 李志军, 张琳琳, 等. 基于OEDR和ODC的自动驾驶汽车实际道路测试评价技术研究[J]. 汽车工程, 2022, 44 (6): 842-850.

    SUN H, LI Z J, ZHANG L L, et al. Research on the test and evaluation technique of real roads for automated driving vehicles based on OEDR and ODC[J]. Automotive Engineering, 2022, 44(6): 842-850. (in Chinese)
    [3]
    LEE D H. Efficient perception, planning, and control algorithm for vision-based automated vehicles[J]. Applied Intelligence, 2024, 54(17): 8278-8295.
    [4]
    ZHANG X R, XIONG L, ZHANG P Z, et al. Real-world troublemaker: a 5G cloud-controlled track testing framework for automated driving systems in safety-critical interaction scenarios[J]. IEEE Internet of Things Journal, 2025, 12(23): 50617-50631. doi: 10.1109/JIOT.2025.3609958
    [5]
    HUANG T, LIU J, ZHOU X, et al. Vehicle-to-Everything cooperative perception for autonomous driving[R/OL]. (2025-09-02)[2025-10-01]. https://arxiv.org/abs/2310.03525.
    [6]
    SCHLENOFF C, KOOTBALLY Z, RACHAKONDA P, et al. Standards and performance metrics for on-road automated vehicles[R/OL]. (2024-06-04)[2025-09-13]. https://doi.org/10.6028/NIST.IR.8527.
    [7]
    SCHOLTES M, WESTHOFEN L, TURNER L R, et al. 6-layer model for a structured description and categorization of urban traffic and environment[J]. IEEE Access, 2021(9): 59131-59147.
    [8]
    蒋拯民, 党少博, 李慧云, 等. 自动驾驶汽车场景测试研究进展综述[J]. 汽车技术, 2022(8): 10-22.

    JIANG Z M, DANG S B, LI H Y, et al. A survey on the research progress of scenario-based testing for autonomous vehicles[J]. Automobile Technology, 2022(8): 10-22. (in Chinese)
    [9]
    SON W, HA Y, OH T, et al. PG-based vehicle-in-the-loop simulation for system development and consistency validation[J]. Electronics, 2022, 11(24): 4073. doi: 10.3390/electronics11244073
    [10]
    余卓平, 邢星宇, 陈君毅. 自动驾驶汽车测试技术与应用进展[J]. 同济大学学报(自然科学版), 2019, 47(4): 540-547.

    YU Z P, XING X Y, CHEN J Y. Review on automated vehicle testing technology and its application[J]. Journal of Tongji University(Natural Science), 2019, 47(4): 540-547. (in Chinese)
    [11]
    MENZEL T, BAGSCHIK G, MAURER M. Scenarios for development, test and validation of automated vehicles[C]. 2018 IEEE Intelligent Vehicles Symposium, Changshu, China: IEEE, 2018.
    [12]
    朱冰, 范天昕, 赵文博, 等. 自动驾驶汽车连续测试场景复杂度评估方法[J]. 吉林大学学报(工学版), 2025, 55(2): 456-467.

    ZHU B, FAN T X, ZHAO W B, et al. Continuous test scenario complexity evaluation method for automated driving vehicles[J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(2): 456-467. (in Chinese)
    [13]
    CROSATO L, TIAN K, SHUM H, et al. Social interaction-aware dynamical models and decision-making for autonomous vehicles[R/OL]. (2023-10-31)[2025-10-01]. https://doi.org/10.48550/arXiv.2310.18891.
    [14]
    孙乐乐, 黄松, 郑长友, 等. 自动驾驶仿真测试场景生成技术研究进展[J]. 计算机工程与应用, 2025, 61(1): 59-79.

    SUN L L, HUANG S, ZHENG C Y, et al. Research progress on autonomous driving simulation test scenario generation technology[J]. Computer Engineering and Applications, 2025, 61(1): 59-79. (in Chinese)
    [15]
    赵祥模, 承靖钧, 徐志刚, 等. 基于整车在环仿真的自动驾驶汽车室内快速测试平台[J]. 中国公路学报, 2019, 32(6): 124-136.

    ZHAO X M, CHENG J J, XU Z G, et al. An indoor rapid-testing platform for autonomous vehicle based on vehicle-in-the-loop simulation[J]. China Journal of Highway and Transport, 2019, 32(6): 124-136. (in Chinese)
    [16]
    THORN E, KIMMEL S, CHAKA M. A framework for automated driving system testable cases and scenarios[R]. Washington D C: National Highway Traffic Safety Administration, 2018.
    [17]
    ISO. Road vehicles-test scenarios for automated driving systems-Vocabulary: ISO 34501[S]. Geneva, Switzerland: ISO, 2022.
    [18]
    熊璐, 吴建峰, 冯天悦, 等. 面向自动驾驶逻辑场景评价的危险域识别方法[J]. 中国公路学报, 2024, 37(5): 371-382.

    XIONG L, WU J F, FENG T Y, et al. Danger field identification method for evaluation of autonomous driving logic scenarios[J]. China Journal of Highway and Transport, 2024, 37 (5): 371-382. (in Chinese)
    [19]
    WINNER H, LEMMER K, FORM T, et al. PEGASUS— First steps for the safe introduction of automated driving[C]. Road Vehicle Automation 5, Cham, Switzerland: Springer, 2019.
    [20]
    ISO. Road vehicles—safety of the intended functionality: ISO/DIS 21448[S]. Geneva, Switzerland: ISO, 2019.
    [21]
    李鹏辉, 董倩茹, 袁赫男, 等. 面向自动驾驶仿真测试的高覆盖切入场景库生成方法[J]. 中国公路学报, 2024, 37(7): 237-249.

    LI P H, DONG Q R, YUAN H N, et al. High-coverage cut-in scenario library generation for automated driving simulation testing[J]. China Journal of Highway and Transport, 2024, 37 (7): 237-249. (in Chinese)
    [22]
    刘法旺, 曹建永, 张志强, 等. 基于场景的智能网联汽车"三支柱"安全测试评估方法研究[J]. 汽车工程学报, 2023, 13 (1): 1-7.

    LIU F W, CAO J Y, ZHANG Z Q, et al. A scenario-based "three-pillar"safety testing and assessment method for intelligent and connected vehicles[J]. Chinese Journal of Automotive Engineering, 2023, 13(1): 1-7. (in Chinese)
    [23]
    赵祥模国家重点研发计划(2021YFB2501200)团队. 自动驾驶测试与评价技术研究进展[J]. 交通运输工程学报, 2023, 23(6): 10-77.

    ZHAO X M Team of National Key Research and Development Program of China(2021YFB2501200). Research progress in testing and evaluation technologies for autonomous driving[J]. Journal of Traffic and Transportation Engineering, 2023, 23(6): 10-77. (in Chinese)
    [24]
    庞劭荣, 张诗波, 罗龙浩, 等. 基于改进云组合赋权的智能网联驾驶场景安全性评价方法[J]. 交通信息与安全, 2023, 41(5): 35-42. doi: 10.3963/j.jssn.1674-4861.2023.05.004

    PANG S R, ZHANG S B, LUO L H, et al. A method for evaluating safety of driving scenes with intelligent connected vehicles based on an improved cloud combination weighting[J]. Journal of Transport Information and Safety, 2023, 41 (5): 35-42. (in Chinese) doi: 10.3963/j.jssn.1674-4861.2023.05.004
    [25]
    FINKELDEI F, THEES C, WEGHORN J N, et al. Scenario factory 2.0: scenario-based testing of automated vehicles with commonroad[J]. Automotive Innovation, 2025, 8(2): 207-220. doi: 10.1007/s42154-025-00360-0
    [26]
    ZHANG X, KHASTGIR S, TIELE J K, et al. ODD and behavior based scenario generation for automated driving systems[J]. IEEE Access, 2024, 12: 10652-10663. doi: 10.1109/ACCESS.2024.3350512
    [27]
    FENG S, YAN X, SUN H, et al. Intelligent driving intelligence test for autonomous vehicles with naturalistic and adversarial environment[R/OL]. (2021-02-02)[2025-10-01]. https://doi.org/10.1038/s41467-021-21007-8.
    [28]
    MORADLOO N, MAHDINIA I, KHATTAK A J. Safety in higher level automated vehicles: investigating edge cases in crashes of vehicles equipped with automated driving systems[J]. Accident Analysis & Prevention, 2024, 203: 107607.
    [29]
    杜德慧, 叶振, 郑成行, 等. 面向自动驾驶系统的场景建模及边缘关键场景生成[J]. 软件学报, 2025, 36(8): 3512-3530.

    DU D H, YE Z, ZHENG C H, et al. Scenario modeling and edge-critical scenario generation for autonomous driving system[J]. Journal of Software, 2025, 36(8): 3512-3530. (in Chinese)
    [30]
    LIN Y, RATZEL M, ALTHOFF M. Automatic traffic scenario conversion from OpenSCENARIO to commonroad[C]. IEEE 26th International Conference on Intelligent Transportation Systems, Bilbao, Spain: IEEE, 2023.
    [31]
    WEI Z, ZHOU H, ZHOU R. Risk and complexity assessment of autonomous vehicle testing scenarios[J]. Applied Sciences, 2024, 14(21): 9866. doi: 10.3390/app14219866
    [32]
    YAN S, ZHANG X, HAO K, et al. On-demand scenario generation for testing automated driving systems[R/OL]. (2025-05-25)[2025-06-11]. https://doi.org/10.48550/arXiv.2505.14053.
    [33]
    刘刚, 宋健. EMB硬件在环仿真试验台[J]. 汽车工程, 2006(10): 929-932.

    LIU G, SONG J. Hardware-in-the-loop test bench for electromechanical brake system[J]. Automotive Engineering, 2006 (10): 929-932. (in Chinese)
    [34]
    HAYWARD J C. Near-miss determination through use of a scale of danger[J]. Highway Research Record, 1972, 384: 24-34.
    [35]
    ALLEN B L, SHIN B T, COOPER P J. Analysis of traffic conflicts and collisions[J]. Transportation Research Record, 1978, 667: 67-74.
    [36]
    HILLENBRAND J, SPIEKER A M, KROSCHEL K. A multilevel collision mitigation approach—its situation assessment, decision making, and performance tradeoffs[J]. IEEE Transactions on Intelligent Transportation Systems, 2006, 7(4): 528-540. doi: 10.1109/TITS.2006.883115
    [37]
    ZHOU J, WANG L, WANG X. Scalable evaluation methods for autonomous vehicles[J]. Expert Systems with Applications, 2024, 249: 123603. doi: 10.1016/j.eswa.2024.123603
    [38]
    修海林. 有条件自动驾驶汽车测试与综合评价研究[D]. 重庆: 重庆大学, 2019.

    XIU H L. Study on the test and comprehensive evaluation of conditional autonomous vehicles[D]. Chongqing: Chongqing University, 2019. (in Chinese)
    [39]
    WANG W, WU L, LI X, et al. An evaluation method for automated vehicles combining subjective and objective factors[R/OL]. (2023-01-01)[2025-10-01]. https://doi.org/10.3390/machines11060597.
    [40]
    ZHANG P, ZHU B, ZHAO J, et al. Performance evaluation method for automated driving system in logical scenario[J]. Automotive Innovation, 2022, 5(3): 299-310. doi: 10.1007/s42154-022-00191-3
    [41]
    ROESENER C, SAUERBIER J, ZLOCKI A, et al. A comprehensive evaluation approach for highly automated driving[C]. 25th International Technical Conference on the Enhanced Safety of Vehicles (ESV), Detroit, USA: NHTSA, 2017.
    [42]
    PEREIRA J, PREMEBIDA C, ASVADI A, et al. Test and evaluation of connected and autonomous vehicles in real-world scenarios[C]. 2019 IEEE Intelligent Vehicles Symposium, Paris, France: IEEE, 2019.
    [43]
    王书辉. 基于仿真的自动驾驶汽车测试评价研究[D]. 西安: 长安大学, 2024.

    WANG S H. Research on testing and evaluation of autonomous vehicles based on simulation[D]. Xi'an: Chang'an University, 2024. (in Chinese)
    [44]
    ZHAO S J, WEI Z R, WANG P Y, et al. An objective evaluation method for automated vehicle virtual test[J]. Expert Systems with Applications, 2022, 206: 117940. doi: 10.1016/j.eswa.2022.117940
    [45]
    LIN D J, LIU H H, CHOU A C, et al. Evaluation of difficulty for autonomous vehicles testing roads based on multiple criteria decision analysis[C]. SAE Technical Paper Series, Detroit, USA: SAE International, 2024.
    [46]
    李茹, 马育林, 田欢, 等. 基于熵值和G1法的自动驾驶车辆综合智能定量评价[J]. 汽车工程, 2020, 42(10): 1327-1334.

    LI R, MA Y L, TIAN H, et al. Comprehensive intelligent quantitative evaluation of autonomous vehicle based on entropy and G1 methods[J]. Automotive Engineering, 2020, 42 (10): 1327-1334. (in Chinese)
    [47]
    李振龙, 潘梦妞, 曲彦菘, 等. 基于IAHP-EWM-LDM的L3级自动驾驶接管过程安全性评价方法[J]. 交通信息与安全, 2023, 41(4): 14-23, 100. doi: 10.3963/j.jssn.1674-4861.2023.04.002

    LI Z L, PAN M N, QU Y S, et al. A method for evaluating the safety over the takeover process of the level 3 automated vehicles based on IAHP-EWM-LDM[J]. Journal of Transport Information and Safety, 2023, 41(4): 14-23, 100. (in Chinese) doi: 10.3963/j.jssn.1674-4861.2023.04.002
    • Relative Articles
    • Supplements(0)
    • Cited By
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Tables(3)

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (22) PDF downloads(3) Cited by()
    Proportional views
    Related

    Copyright Editorial Office of Transport Information and Safety鄂ICP备05003336号-2

    Address：No. 1178，Heping Avenue, Wuchang, Wuhan, CHINA (430063)

    Tel：027-86580355 Email：jtjsj@vip.163.com

    Supported by: Beijing Renhe Information Technology Co. Ltd  

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return