Associate Professor

The University of Tokyo, Japan

University of Alberta, Canada

Email: ma.lei [at] acm [dot] org



Prospective Students:

My research group is continuously recruiting. Currently, our lab have multiple scholarship & RA positions for Ph.D., MSc. candidates, and postdocs with competitive salary support, at both The University of Tokyo, Japan and University of Alberta, Canada. If you have a strong background in the topics relevant (but not limited) to my research with strong self-motivation & strong will to achieve, I strongly encourage you to apply and join us. Note that to inquiry about opportunities to join our research group, please send emails only to contact@momentum-lab.com to inquiry and apply!
For students already with Todai, or students who can come to Todai Hongo Campus area, please simply drop me an email to schedule a meet up at the Campus, to discuss the possibilities.





img


  1. (03/2024) Our paper Active Code Learning: Benchmarking Sample-Efficient Training of Code Models is accepted at IEEE Transactions on Software Engineering (TSE).
  2. (02/2024) Our paper PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinement is accepted at ACM CHI 2024. This paper proposes PromptCharm, a very early mixed-initiative and interactive system that facilitates text-to-image creation through multi-modal prompt engineering and refinement. It helps designers in prompting, automatically refines and optimizes the the initial prompt, and further supports in exploring and selecting different image styles within a large database with iterative refinments.
  3. (01/2024) Our paper ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planning is accepted at ICRA 2024. LLMs are advantageous in offering the potential to enhance the generalizability as task-agnostic planners and facilitate flexible interaction between human instructors and planning systems. However, task plans generated by LLMs often lack feasibility and correctness. To address this challenge, in this exploratory work, we designed a novel framework that improves LLM-based planning through an iterative self-refinement process towards appraoching better fesibility and trustworthiness.
  4. (01/2024) Our papers LRR: Language-Driven Resamplable Continuous Representation against Adversarial Tracking Attacks and Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalization are accepted at ICLR 2024.
  5. (12/2023) Our paper MultiTest: Physical-Aware Object Insertion for Testing Multi-sensor Fusion Perception Systems is accepted at ICSE 2024. This is our exploratory work that proposes a testing framework for muti-sensor system by considering the multi-modality characteristics of such systems.
  6. (10/2023) Our paper LUNA: A Model-Based Universal Analysis Framework for Large Language Models is available on Arxiv. This paper introduces the inaugural mode-based, general-purpose analysis framework designed to scrutinize, comprehend, and rationalize the behaviors and risks associated with inferences made by Large Language Models (LLMs) following a human-centered manner. LUNA is crafted with high extensibility in mind, and our ongoing efforts are geared towards adapting LUNA for various domain-specific tasks and ensuring its compliance with trustworthiness standards. We would regularly updating more extensive findings. Should you have an interest in LUNA, for both researchers or industry practitioners, please do not hesitate to contact us. We welcome collaboration and are keen to explore the potential of LUNA together.
  7. (08/2023) Our paper Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models is available on Arxiv. This paper investigates blackbox techniques to understand the confidence of inference by LLMs. We found that even under the blackbox setting, we can still gain intuition and understanding of the level of confidence in an LLM's inference at industrial scale. We will continuously update and release more comprehensive results.
  8. (08/2023) Our paper Is Model Attention Aligned with Human Attention? An Empirical Study on Large Language Models for Code Generation is available on Arxiv. This is an study for better understanding and constructing human-centered process of Large Language Models for Code Generation.
  9. (08/2023) Our paper Towards Building AI-CPS with NVIDIA Isaac Sim: An Industrial Benchmark and Case Study for Robotics Manipulation is available on Arxiv. This is our exploratory study towards sim to real, via efficiently building AI-CPS with Nvidia Isaac Sim as an early step.
  10. (08/2023) Three papers are accepted by ESEC/FSE 2023.
  11. (08/2023) Three papers are accepted by ASE 2023.
  12. (06/2023) Our paper SIEGE: A Semantics-Guided Safety Enhancement Framework for AI-Enabled Cyber-Physical Systems is accepted at TSE 2023 . This paper performs an early explorative study for the safety enhancement of AI-CPS via ensembling approach, which is also a continuous series of work towards investigating safety assurance after our previous work on AI-CPS benchmarking (ICSE 2022), Falsifai (TSE 2022), etc.
  13. (06/2023) Our paper Faire: Repairing Fairness of Neural Networks via Neuron Condition Synthesis is accepted at ACM TOSEM 2023 .
  14. (04/2023) Our work ArchRepair on the Repair of Deep Neural Network and the recent research activities of JST eAI project has been reported by 日本経済新聞 (Nikkei News) , also see here.
  15. (04/2023) On April 2023, I joined computer science department of The University of Tokyo, Japan as Associate Professor.
  16. (03/2023) Our paper Evading DeepFake Detectors via Adversarial Statistical Consistency is accepted at CVPR 2023 .
  17. (02/2023) Our paper PatchCensor: Patch Robustness Certification for Transformers via Exhaustive Testing is accepted at ACM Transactions on Software Engineering and Methodology (TOSEM 2023) .
  18. (02/2023) Our paper ArchRepair: Block-Level Architecture-Oriented Repairing for Deep Neural Networks is accepted at ACM Transactions on Software Engineering and Methodology (TOSEM 2023) .
  19. (01/2023) Our papers on human-centered interactive data debugging DeepLens and model debugging DeepSeer are accepted at CHI 2023 .
  20. (01/2023) Our paper Neural Episodic Control with State Abstraction is accepted at ICLR 2023 as a spotlight paper .
  21. (12/2022) Our paper GraphSearchNet: Enhancing GNNs via Capturing Global Dependencies for Semantic Code Search is accepted at IEEE Transactions on Software Engineering (TSE 2023) .
  22. (12/2022) Four papers accepted at ICSE 2023 (two on research track, one on SEIP track, one NIER track).
  23. (11/2022) Our paper DeepGemini: Verifying Dependency Fairness for Deep Neural Network is accepted at AAAI 2023 .
  24. (07/2022) Join IEEE CIS Neural Networks Committee on Secure Learning as a Task Force Member.
  25. (07/2022) Our paper Decompiling x86 Deep Neural Network Executables is accepted at The 32nd USENIX Security Symposium (USENIX Security '23 Summer)
  26. (07/2022) Our paper FalsifAI: Falsification of AI-Enabled Hybrid Control Systems Guided by Time-Aware Coverage Criteria is accepted at IEEE Transactions on Software Engineering (TSE)
  27. (06/2022) Our paper DARTSRepair: Core-Failure-Set Guided DARTS for Network Robustness to Common Corruptions is accepted at Pattern Recognition 2022
  28. (06/2022) Our paper xFuzz: Machine Learning Guided Cross-Contract Fuzzing is accepted at IEEE Transactions on Dependable and Secure Computing, TDSC 2022
  29. (03/2022) Our paper Countering Malicious DeepFakes: Survey, Battleground, and Horizon is accepted at International Journal of Computer Vision, IJCV 2022. We present a comprehensive survey covering the latest DeepFake generation and detection methods, as well as the battleground landscape.
  30. (01/2022) Our paper When Cyber-Physical Systems Meet AI: A Benchmark, an Evaluation, and a Way Forward is accepted at ICSE SEIP 2022, to the best of our knowledge, this is the very first systematic benchmark construction and study of AI-enabled Cyber Physical Systems (AI-CPS). We make our dataset and benchmark publically available, and hope this would inspire lots of future new reserach and investigation along this exciting direction!
  31. (01/2022) Our paper An Empirical Study on Data Distribution-Aware Test Selection for Deep Learning Enhancement is accepted at ACM Transactions on Software Engineering and Methodology (TOSEM 2022).
  32. (12/2021) Our paper An Empirical Study of the Impact of Hyperparameter Tuning and Model Optimization on DNN is accepted at ACM Transactions on Software Engineering and Methodology (TOSEM 2021).
  33. (11/2021) Our paper Towards Robustness of Deep Program Processing Models -- Detection, Estimation and Enhancement is accepted at ACM Transactions on Software Engineering and Methodology (TOSEM 2021).
  34. (09/2021) Our paper NPC: Neuron Path Coverage via Characterizing Decision Logic of Deep Neural Networks is accepted at ACM Transactions on Software Engineering and Methodology (TOSEM 2021).
  35. (08/2021) Our paper DeepRepair: Style-Guided Repairing for Deep Neural Networks in the Real-World Operational Environment is accepted at IEEE Transactions on Reliability 2021.
  36. (07/2021) Our paper Learning to Adversarially Blur Visual Object Tracking is accepted at ICCV 2021.
  37. (07/2021) Join IJCAI 2022 as a PC Member.
  38. (07/2021) Our paper Predictive Perturbation-aware Filtering against Adversarial Attack via Multi-domain Learning is accepted at ACM MM 2021.
  39. (07/2021) Our paper An Empirical Study of GUI Widget Detection for Industrial Mobile Games is accepted at FSE 2021 Industry Track.
  40. (06/2021) Join ICLR 2022 as a reviewer, AISec 2021 and ESEC/FSE 2022 as a PC member. Please consider submitting your high quality papers.
  41. (05/2021) Our paper Automatic RNN Repair via Model-based Analysis is accepted at ICML 2021.
  42. (04/2021) Our paper Effective Hybrid System Falsification Using Monte Carlo Tree Search Guided by QB-Robustness is accepted at CAV 2021.
  43. (03/2021) Our survey paper Countering Malicious DeepFakes: Survey, Battleground, and Horizon that makes comprehensive coverage regarding malicious DeepFakes is available on Arxiv.
  44. (12/2020) Media coverage: Our work DeepRhythm that uses heartbeat detection to identify deepfake videos is reported in English (Link1, Link2), Japanese, and Chinese!
  45. (12/2020) Our paper Automatic Web Testing using Curiosity-Driven Reinforcement Learning is accepted at ICSE 2021.
  46. (12/2020) Our paper A Search-Based Testing Framework for Deep Neural Networks of Source Code Embedding is accepted at ICST 2021.
  47. (12/2020) Join ICML 2021 and CVPR 2021 as reviewers.
  48. (12/2020) Our papers EfficientDeRain: Learning Pixel-wise Dilation Filtering for High-Efficiency Single-Image Deraining and Decision-Guided Weighted Automata Extraction from Recurrent Neural Networks are accepted at AAAI 2021.
  49. (11/2020) Join ICSE 2022 as PC Member.
  50. (11/2020) Our paper Oracle-Supported Dynamic Exploit Generation for Smart Contracts after two years' countinous effort is finally accepted at IEEE Transactions on Dependable and Secure Computing (TDSC)!
  51. (11/2020) Join DeepTest 2021 as PC Member.
  52. (11/2020) Join GECCO 2021 as PC Member.
  53. (10/2020) Join ASE 2021 as PC Member.
  54. (10/2020) Join ICPC 2021 as PC Member.
  55. (10/2020) Join IJCAI 2021 as PC Member.
  56. (09/2020) Our paper Watch out! Motion is Blurring the Vision of Your Deep Neural Networks is accepted at NeurIPS 2020. This paper performs intensitive investigation on the impact of potential camera blurring effects to the decision of a DNN!
  57. (09/2020) Join AAAI 2021 as PC Member.
  58. (08/2020) Our empirical study on Autonomous Driving Systems and its ML Components is accepted at Industry Track of ESEC/FSE 2020.
  59. (07/2020) Our papers MARBLE: Model-Based Robustness Analysis of Stateful Deep Learning Systems and Cats Are Not Fish: Deep Learning Testing Calls for Out-Of-Distribution Awareness are accepted at ASE 2020.
  60. (07/2020) Two papers are accepted at ICSME 2020, NIER Track.
  61. (07/2020) Our DeepFake Series works DeepRhythm, DeepSonar, and Amora are accepted at ACM MM 2020.
  62. (07/2020) Our paper SPARK: Spatial-aware Online Incremental Attack Against Visual Tracking is accepted at ECCV 2020. This paper proposes a novel and efficient attack to mislead online trackers along an incorrect or specified trajectory!
  63. (06/2020) Join ESEC/FSE 2021 as PC Member.
  64. (06/2020) Join NeurIPS’20 as a reviewer.
  65. (05/2020) Our paper FakeSpotter is accepted at IJCAI 2020, a previous verison is reported as AI vs AI: ‘FakeSpotter’ Studies Neurons to Bust DeepFakes, by the Synced AI Industry Review!
  66. (05/2020) Our paper DeepSonar: Towards Effective and Robust Detection of AI-Synthesized Fake Voices is available on CoRR arXiv!
  67. (04/2020) Join AISec’20, ASE'20, ICSME'20 as PC member.
  68. (03/2020) Give tutorials regarding Testing and Analysis of Stateful Deep Learning Systems at IVADO Training Program on Software Engineering for Machine Learning Applications, at Montreal, Canada, March 2-6 2020.
  69. (02/2020) Give Invited talk Automatic Video Game Testing via Evolutionary Deep Reinforcement Learning at Polytechnique Montreal, Canada, Feb.6, 2020
  70. (02/2020) Give Invited talk Quality and Reliability Assurance for Deep Learning Systems at School of Computer Science, McGill University, Canada, Feb.5, 2020
  71. (02/2020) Our paper Towards Byzantine-resilient Learning in Decentralized Systems is available on CoRR arXiv!
  72. (02/2020) Our paper ABBA: Saliency-Regularized Motion-Based Adversarial Blur Attack is available on CoRR arXiv!
  73. (01/2020) Our paper FakeLocator: Robust Localization of GAN-Based Face Manipulations via Semantic Segmentation Networks with Bells and Whistles is available on CoRR arXiv!
  74. (12/2019) I am honored to be invited to Dagstuhl Seminar 20091:SE4ML - Software Engineering for AI-ML-based Systems to be held on February 23 – 28, 2020
  75. (12/2019) I am honored to be invited to give lectures at IVADO Training Program on Software Engineering for Machine Learning Applications, to be held at Montreal, Canada, March 2-6 2020.
  76. (12/2019) Our paper Amora: Black-box Adversarial Morphing Attack is available on CoRR arXiv!
  77. (12/2019) Our paper Towards Characterizing Adversarial Defects of Deep Learning Software from the Lens of Uncertainty is accepted at ICSE'2020, This work is the very early work that discovers the new uncommon adversarial examples that represent the challenging vulnerability of deep learning software, we are now working to seek new ways to counteract these issues!
  78. (11/2019) Our survey paper Machine Learning Testing: Survey, Landscapes and Horizons, on the emerging direction of testing and anlaysis of machine learning is accepted at IEEE Transactions on Software Engineering (TSE), a previous early draft is available at CoRR arXiv, we are working on the camera-ready version and would make it available soon!
  79. (11/2019) Our research framework DeepStudio: a Framework of Securing Deep Learning Systems receives the 2019 CCF Research Prototype Competition Award (Second Prize).
  80. (11/2019) Our papers Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning and Generating Adversarial Examples for Holding Robustness of Source Code Processing Models are accepted at AAAI 2020.
  81. (10/2019) Our paper Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement Learning receives the ACM SIGSOFT Distinguished Paper Award in ASE'19.
  82. (10/2019) Media coverage: AI vs AI: ‘FakeSpotter’ Studies Neurons to Bust DeepFakes, our work FakeSpotter is reported by the media of AI Industry Review!
  83. (10/2019) Our paper Model-based Testing of Apache ZooKeeper: Fundamental API Usage and Watchers is finally accepted at Software Testing, Verification AND Reliability (STVR) Journal after two years continous effort !
  84. (09/2019) Our papers DeepMutation++, DeepHunter, and DeepStellar are accepted and will be demonstrated at ASE 2019 as tool papers !
  85. (09/2019) Our papers Wuji and Deep Learning Platform/Framework Compatibility Study are accepted at ASE 2019 as full research papers !
  86. (08/2019) Our paper CDA: Characterising Deprecated Android APIs is accepted at Empirical Software Engineering (EMSE) Journal !
  87. (07/2019) Our paper An Empirical Study of Common Challenges in Developing Deep Learning Applications is accepted at ISSRE'19 !
  88. (07/2019) Two papers: Safe Inputs Generation for Black-box Systems and MobiDroid: A Performance-Sensitive Malware Detection System on Mobile Platform are accepted at ICECCS 2019!
  89. (06/2019) Our survey paper Machine Learning Testing: Survey, Landscapes and Horizons is available on CoRR arXiv!
  90. (05/2019) Our paper DiffChaser: Detecting Disagreements for Deep Neural Networks is accepted at IJCAI'19 !
  91. (05/2019) Our paper DeepStellar: Model-Based Quantitative Analysis of Stateful Deep Learning Systems is accepted at FSE'19 !
  92. (05/2019) I am invited to join AISec'19 Program Committee collocated with CSS'19 this year, please considier to submit your high quality papers!
  93. (05/2019) I am invited to visit and give a talk on Testing Deep Learning System Towards Intelligence of Things at Google Waterloo on June 5!
  94. (05/2019) Our paper DeepHunter: A Coverage-Guided Fuzz Testing Framework for Deep Neural Networks is accepted at ISSTA'19 !
  95. (04/2019) I am happy to be invited to join and give a talk on NII Shonan Meeting on Software Engineering for Machine Learning Systems (SEMLS) which will be held on October this year.
  96. (04/2019) I am invited by Machine Learning and Software Engineering Community of Japan (MLSE) to give tutorial on "Software Testing of Deep Learning System: from the Past to the Future"
  97. (04/2019) I am co-organizing the first MLST Workshop Collocated with ICST!
  98. (03/2019) I am happy to be invited to join and give a talk on The Software Engineering for Machine Learning Applications (SEMLA) International Symposium, Polytechnique Montreal which will be held on May this year.
  99. (02/2019) I am co-organizing the first AI4Mobile Workshop Collocated with SANER!
  100. (12/2018) Our paper VULTRON: Catching Vulnerable Smart Contracts Once and for All ICSE'19 as an NIER Track Paper!
  101. (12/2018) Our paper DeepCT: Tomographic Combinatorial Testing for Deep Learning Systems SANER'19 as an ERA Track Paper!
  102. (09/2018) Our paper Practical Fault Attack on Deep Neural Networks is accepted at CCS'18 as a Poster Paper!
  103. (09/2018) Our paper Metamorphic Relation Based Adversarial Attacks on Differentiable Neural Computer is available on CoRR arXiv. This is the very first work towards in-depth analysis on the robustness of Differtentiable Neural Computer!
  104. (08/2018) Our paper DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems receive the ACM SIGSOFT Distinguished Paper Award in ASE'18.
  105. (08/2018) Our paper DeepMutation: Mutation Testing of Deep Learning Systems is accepted at ISSRE'18
  106. (07/2018) Our paper DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems is accepted at ASE'18
  107. (06/2018) Our paper Combinatorial Testing for Deep Learning Systems is available on CoRR arXiv. This is the first combinatorial testing for Deep Neural Networks!
  108. (06/2018) Our paper Practical Fault Attack on Deep Neural Networks is available on CoRR arXiv. This is the first practical attacks on Deep Neural Networks using laser technology!
  109. (05/2018) Our paper Characterising Deprecated Android APIs won the FOSS Impact Paper Award in MSR'18.
  110. (05/2018) Our paper DeepMutation: Mutation Testing of Deep Learning Systems is available on CoRR arXiv. This is the first mutation testing technique for Deep Neural Networks!
  111. (03/2018) Our paper DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems is available on CoRR arXiv. This paper proposes a set of testing critiera with multi-granuality for Deep Neural Networks!
  112. (01/2018) Our paper Characterising Deprecated Android APIs has been accepted to appear in MSR'18.
  113. (03/2017) Our paper Model-based API Testing of Apache ZooKeeper won the IEEE Best Presentation Award at ICST'17.
  114. (07/2016) Our paper Runtime monitoring for concurrent systems has been accepted to appear in RV'16.
  115. (09/2016) Our paper An empirical study on the effects of code visibility on program testability has been accepted to appear in Springer Software Quliaty Journal (SQJ).
  116. (03/2016) Our paper Efficient and Precise Dynamic Slicing for Client-Side JavaScript Programs has been nominated for the IEEE Best Candidate Paper Award in SANER'16.
  117. (11/2015) Our paper GRT: Program-analysis-guided Random Testing won the ACM SIGSOFT Distinguished Paper Award at ASE'15.
  118. (11/2015) Our paper Incremental Web Application Testing using Page Object won the IEEE Best Paper Award at HotWeb'15.

About Me

I am currently an associate professor with The University of Tokyo, and University of Alberta (with a shared appointment time from April 2023), leading Momentum Lab. Previously, I held the assistant professor position with Kyushu University from Jan. 2019, and was promoted to associate professor on April 2020. From April 2021, I joined University of Alberta and was honored to be selected as a Canada CIFAR AI Chair, and a Fellow with Alberta Machine Intelligent Institute (Amii). From April 2023, I joined The computer science department of The University of Tokyo as an associate professor.
My research spans a wide range of research topics, and comes with a special focus centering on the interdisciplinary research fields of Software Engineering and Artificial Intelligence, in the design and development of quality assurance and engineering support for building trustworthy AI systems.
In my early career as a student, I received a B.E. degree from Shanghai Jiaotong University (SJTU) in 2009, M.E. and Ph.D. degrees from The University of Tokyo, in 2011 and 2014, respectively. During my Ph.D. program, I studied for half a year in the Mathematics and Computer Science department of  Technische Universität München(TUM)

I love theories, but I like even more to transfer theories into practice!
I have several research interests and working directions. My current major research interest mainly focuses on providing general purpose quality assurance solutions and engineering support for intelligent software systems. I believe the trustworthy intelligent software system would be the key driving force for technology innovation of the future. It does not solely rely on the foundational research from the AI community, but also requires the contribution from the many communities such as software engineering, control, etc., upbringing a new interdisciplinary emerging field Machine Learning System Engineering, an extremely exciting and demanding area of the near future! We have conducted consecutive works along the general-purpose quality assurance research for intelligent software systems (e.g., testing, analysis, verification, interpretation, attacks, defenses), to bridge vast demands from industry. Exciting research results would be continuously updated!

In the past, my research interest mainly focuses around software engineering and programming language, in particular automated software testing, verification, analysis, evolution, mining, etc. I am particularly intersted in the interdisplinary fields of software and various foundational science and engineering process. For example, I worked on green computation and optimization solutions for cloud and big data centers I am also interested in genomic data analsyis to uncover life mysteries and to search for health solutions through trustworthy AI solutions.

  • TSE 2022 Annual Best Paper Award, IEEE Transactions on Software Engineering (TSE)
  • ACM SIGSOFT Distinguished Paper Award (ESEC/FSE'23)
  • Canada CIFAR AI Chair, Pan-Canadian AI Strategy (2021)
  • Fellow, Alberta Machine Intelligence Institute (Amii) (2021)
  • ACM SIGSOFT Distinguished Paper Award (ASE'19)
  • ACM SIGSOFT Distinguished Paper Award (ASE'18)
  • ACM SIGSOFT Distinguished Paper Award (ASE'15)
  • Research Prototype Competition Award (Second Prize, CCF NASAC'19)
  • 2018 Baidu-NASAC Academic Star Award
  • ACM FOSS Impact Paper Award (MSR'18)
  • Leadership Award (ICECCS'17)
  • IEEE Best Presentation Award (ICST'17)
  • IEEE Best Paper Candidate Award (SANER'16)
  • IEEE Best Paper Award (HotWeb'15)
  • IEEE Best Testing Tool Award (SBST'15)
  • JASSO Hornor scholoarship (2011 and 2013)
  • MEXT Scholarship sponsored by Japanese Government (2009)
  • Excellent Academic Scholarship of Shanghai Jiao Tong University (2006)

    See also at Google Scholar or DBLP.

  1. Zhijie Wang, Yuheng Huang, Da Song, Lei Ma, Tianyi Zhang
    PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinement
    The ACM CHI conference on Human Factors in Computing Systems. (ACM CHI 2024, CORE RANK A*)
  2. Qiang Hu, Yuejun Guo, Xiaofei Xie, Maxime Cordy, Lei Ma, Mike Papadakis, Yves Le Traon
    Active Code Learning: Benchmarking Sample-Efficient Training of Code Models
    IEEE Transactions on Software Engineering, 2024. (TSE, Core Rank A*, Impact Factor:9.322)
  3. Zhehua Zhou, Jiayang Song, Kunpeng Yao, Zhan Shu, Lei Ma
    ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planning
    2024 IEEE International Conference on Robotics and Automation, Yokohama, Japan . (ICRA 2024,CORE RANK A*)
  4. Jianlang Chen, Xuhong Ren, Qing Guo, Felix Juefei-Xu, Di Lin, Wei Feng, Lei Ma, Jianjun Zhao
    LRR: Language-Driven Resamplable Continuous Representation against Adversarial Tracking Attacks
    The Twelfth International Conference on Learning Representations (ICLR 2024, CORE RANK A*)
  5. Yibing Liu, Chris XING TIAN, Haoliang Li, Lei Ma, Shiqi Wang
    Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalization
    The Twelfth International Conference on Learning Representations (ICLR 2024 Spotlight, CORE RANK A*)
  6. Zhehua Zhou, Jiayang Song, Xuan Xie, Zhan Shu, Lei Ma, Dikai Liu, Jianxiong Yin, Simon See
    Towards Building AI-CPS with NVIDIA Isaac Sim: An Industrial Benchmark and Case Study for Robotics Manipulation
    The 46th International Conference on Software Engineering, SEIP Track (ICSE 2024, CORE RANK A*)
  7. XinyuGao , Zhijie Wang, Yang Feng, Lei Ma, Zhenyu Chen, Baowen Xu
    MultiTest: Physical-Aware Object Insertion for Testing Multi-sensor Fusion Perception Systems
    The 46th International Conference on Software Engineering (ICSE 2024, CORE RANK A*)
  8. Paolo Arcaini, Fuyuki Ishikawa, Lei Ma, Yuta Maezawa, Nobukazu Yoshioka, Fuyuan Zhang
    Technical Briefing on Deep Neural Network Repair
    The 46th International Conference on Software Engineering (ICSE 2024, CORE RANK A*)
  9. Fuyuan Zhang, Xinwen Hu, Lei Ma, Jianjun Zhao
    DeepRover: A Query-efficient Blackbox Attack for Deep Neural Networks
    The 31th the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. (ESEC/FSE 2023, CORE Rank A*)
  10. Kaifeng Huang, Bihuan Chen, Susheng Wu, Junming Cao, Lei Ma, Xin Peng
    Demystifying Dependency Bugs in Deep Learning Stack
    The 31th the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. (ESEC/FSE 2023, CORE Rank A*)
  11. Xinyu Gao, Zhijie Wang, Yang Feng, Lei Ma, Zhenyu Chen, Baowen Xu
    Benchmarking Robustness of AI-enabled Multi-sensor Fusion Systems: Challenges and Opportunities The 31th the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. (ESEC/FSE 2023, CORE Rank A*)
    ACM SIGSOFT Distinguished Paper Award.
  12. Jiaming Ye, Shangzhou Xia, Fuyuan Zhang, Paolo Arcaini, Lei Ma, Jianjun Zhao, Fuyuki Ishikawa
    QuraTest: Integrating Quantum Specific Features in Quantum Program Testing
    The 38th IEEE/ACM International Conference on Automated Software Engineering. (ASE 2023, CORE Rank A*)
  13. Yi Liu, Yuekang Li, Gelei Deng, Felix Juefei-Xu, Cen Zhang, Chengwei Liu, Yeting Li, Lei Ma, Yang Liu
    Aster: Automatic Speech Recognition System Accessibility Testing for Stutterers
    The 38th IEEE/ACM International Conference on Automated Software Engineering. (ASE 2023, CORE Rank A*)
  14. Zhuo Li, Xiongfei Wu, Derui Zhu, Mingfei Cheng, Siyuan Chen, Fuyuan Zhang, Xiaofei Xie, Lei Ma, Jianjun Zhao
    Generative Model-Based Testing on Decision-Making Policies
    The 38th IEEE/ACM International Conference on Automated Software Engineering. (ASE 2023, CORE Rank A*)
  15. Jiayang Song, Xuan Xie, Lei Ma
    SIEGE: A Semantics-Guided Safety Enhancement Framework for AI-Enabled Cyber-Physical Systems
    IEEE Transactions on Software Engineering, 2023. (TSE, Core Rank A*, Impact Factor:9.322)
  16. Tianlin Li, Xiaofei Xie, Jian Wang, Qing Guo, Aishan Liu, Lei Ma, Yang Liu
    Faire: Repairing Fairness of Neural Networks via Neuron Condition Synthesis
    ACM Transactions on Software Engineering and Methodology (TOSEM 2023, CORE Rank A*, Impact Factor 2.674)
  17. Yang Hou, Qing Guo, Yihao Huang, Xiaofei Xie, Lei Ma, Jianjun Zhao
    Evading DeepFake Detectors via Adversarial Statistical Consistency
    The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2023, CORE Rank A*)
  18. Yuheng Huang, Lei Ma, Yuanchun Li
    PatchCensor: Patch Robustness Certification for Transformers via Exhaustive Testing
    ACM Transactions on Software Engineering and Methodology (TOSEM 2023, CORE Rank A*, Impact Factor 2.674)
  19. Hua Qi, Zhijie Wang, Qing Guo, Jianlang Chen, Felix Juefei-Xu, Fuyuan Zhang, Lei Ma, Jianjun Zhao
    ArchRepair: Block-Level Architecture-Oriented Repairing for Deep Neural Networks
    ACM Transactions on Software Engineering and Methodology (TOSEM 2023, CORE Rank A*, Impact Factor 2.674)
  20. Zhijie Wang, Yuheng Huang, Da Song, Lei Ma, Tianyi Zhang
    DeepSeer: Interactive RNN Explanation and Debugging via State Abstraction
    The ACM CHI Conference on Human Factors in Computing Systems (CHI 2023, CORE Rank A*)
  21. Da Song, Zhijie Wang, Yuheng Huang, Lei Ma, Tianyi Zhang
    DeepLens: Interactive Out-of-Distribution Data Detection in NLP Models
    The ACM CHI Conference on Human Factors in Computing Systems (CHI 2023, CORE Rank A*)
  22. Zhuo Li, Derui Zhu, Yujing Hu, Xiaofei Xie, Lei Ma, Yan Zheng, Yan Song, Yingfeng Chen, Jianjun Zhao
    Neural Episodic Control with State Abstraction
    The Eleventh International Conference on Learning Representations (ICLR 2023, spotlight CORE Rank A*)
  23. Shangqing Liu, Xiaofei Xie, Jingkai Siow, Lei Ma, Guozhu Meng, Yang Liu
    GraphSearchNet: Enhancing GNNs via Capturing Global Dependencies for Semantic Code Search
    IEEE Transactions on Software Engineering (TSE 2023, CORE Rank A*, Impact factor 6.22)
  24. Qiang Hu, Yuejun Guo, Xiaofei Xie, Maxime Cordy, Lei Ma, Mike Papadakis, Yves Le Traon
    Aries: Efficient Testing of Deep Neural Networks via Labeling-Free Accuracy Estimation
    The 45th International Conference on Software Engineering (ICSE 2023, CORE Rank A* )
  25. Jiongchi Yu, Yuechen Wu, Xiaofei Xie, Wei Le, Lei Ma, Yingfeng Chen, Yujing Hu, Fan Zhang
    GameRTS: A Regression Testing Framework for Video Games
    The 45th International Conference on Software Engineering (ICSE 2023, CORE Rank A* )
  26. Xiongfei Wu, Jiaming Ye, Ke Chen, Xiaofei Xie, Yujing Hu, Ruochen Huang, Lei Ma and Jianjun Zhao
    Widget Detection-based Testing for Industrial Mobile Games
    The 45th International Conference on Software Engineering, SEIP Track (ICSE 2023, CORE Rank A* )
  27. Qiang Hu, Yuejun Guo, Xiaofei Xie, Maxime Cordy, Lei Ma, Mike Papadakis, Yves Le Traon
    CodeS: Towards Code Model Generalization Under Distribution Shift
    The 45th International Conference on Software Engineering, NIER Track (ICSE 2023, CORE Rank A* )
  28. Xuan Xie, Fuyuan Zhang, Xinwen Hu, Lei Ma
    DeepGemini: Verifying Dependency Fairness for Deep Neural Network
    The Thirty-Seventh AAAI Conference on Artificial Intelligence (AAAI 2023, CORE Rank A* )
  29. Zhibo Liu, Yuanyuan Yuan, Shuai Wang, Xiaofei Xie, Lei Ma
    Decompiling x86 Deep Neural Network Executables
    The 32nd USENIX Security Symposium (USENIX Security 2023 Summer, CORE Rank A* )
  30. Shuncheng Tang, Zhenya Zhang, Yi Zhang, Jixiang Zhou, Yan Guo, Shuang Liu, Shengjian Guo, Yanfu Li, Lei Ma, Yinxing Xue, and Yang Liu
    A Survey on Automated Driving System Testing: Landscapes and Trends
    ACM Transactions on Software Engineering and Methodology (TOSEM 2023, CORE Rank A*, Impact Factor 2.674)
  31. Amin Eslami Abyane, Derui Zhu, Roberto Medeiros de Souza, Lei Ma, Hadi Hemmati
    Towards Understanding Quality Challenges of the Federated Learning: A First Look from the Lens of Robustness
    Empirical Software Engineering, Nov. 2022 (EMSE 2022, CORE Rank A)
  32. Zihe Song, Yingfeng Chen, Lei Ma, Shangjie Lu, Honglei Lin, Changjie Fan, Wei Yang
    An Empirical Analysis of Compatibility Issues for Industrial Mobile Games
    The 33rd IEEE International Symposium on Software Reliability Engineering, Oct. 2022 (ISSRE 2022, CORE Rank A)
  33. Xiongfei Wu, Jinqiu Yang, Lei Ma, Yinxing Xue, and Jianjun Zhao
    On the Usage and Development of Deep Learning Compilers: an Empirical Study on TVM
    Empirical Software Engineering volume 27, 172, Sep. 2022 (EMSE 2022, CORE Rank A)
  34. Zhuo Li, Yuechen Wu, Lei Ma, Xiaofei Xie, Yingfeng Chen, and Changjie Fan
    GBGallery : A Benchmark and Framework for Game Testing
    Empirical Software Engineering volume 27, 140, July 2022 (EMSE 2022, CORE Rank A)
  35. Zhenya Zhang, Deyun Lyu, Paolo Arcaini, Lei Ma, Ichiro Hasuo, and Jianjun Zhao
    FalsifAI: Falsification of AI-Enabled Hybrid Control Systems Guided by Time-Aware Coverage Criteria
    IEEE Transactions on Software Engineering (TSE 2022, CORE Rank A*, Impact factor 6.22 )
  36. Liming Zhai, Qing Guo, Xiaofei Xie, Lei Ma, Yi Estelle Wang, Yang Liu
    A3GAN: Attribute-Aware Anonymization Networks for Face De-identification
    The 30th ACM International Conference on Multimedia, 2022 (ACM MM 2022, CORE Rank A*)
  37. Xuhong Ren, Jianlang Chen, Felix Juefei-Xu, Wanli Xue, Qing Guo, Lei Ma, Jianjun Zhao, Shengyong Chen
    DARTSRepair: Core-Failure-Set Guided DARTS for Network Robustness to Common Corruptions
    Pattern Recognition, Elsevier, 2022 (CORE Rank A*, Impact factor 7.74)
  38. Yinxing Xue, Jiaming Ye, Wei Zhang, Jun Sun, Lei Ma, Haijun Wang, and Jianjun Zhao
    xFuzz: Machine Learning Guided Cross-Contract Fuzzing
    IEEE Transactions on Dependable and Secure Computing, 2022 (TDSC, CORE Rank A*, Impact factor 7.329)
  39. Felix Juefei-Xu, Run Wang, Yihao Huang, Qing Guo, Lei Ma, Yang Liu
    Countering Malicious DeepFakes: Survey, Battleground, and Horizon
    International Journal of Computer Vision, Springer, 2022 (IJCV, CORE rank A*, Impact Factor: 11.81)
  40. Jiayang Song, Deyun Lyu, Zhenya Zhang, Zhijie Wang, Tianyi Zhang, Lei Ma
    When Cyber-Physical Systems Meet AI: A Benchmark, an Evaluation, and a Way Forward
    The 44th International Conference on Software Engineering, SEIP Track (ICSE 2022, CORE Rank A*)
  41. Qiang Hu, Yuejun Guo, Maxime Cordy, Xiaofei Xie, Lei Ma, Mike Papadakis, Yves Le Traon
    An Empirical Study on Data Distribution-Aware Test Selection for Deep Learning Enhancement
    ACM Transactions on Software Engineering and Methodology (TOSEM 2022, CORE Rank A*, Impact Factor 2.674)
  42. Lizhi Liao, Heng Li, Weiyi Shang, Lei Ma
    An Empirical Study of the Impact of Hyperparameter Tuning and Model Optimization on DNN
    ACM Transactions on Software Engineering and Methodology (TOSEM 2021, CORE Rank A*, Impact Factor 2.674)
  43. Huangzhao Zhang, Zhiyi Fu, Ge Li, Lei Ma, Zhehao Zhao, Hua'an Yang, Yizhe Sun, Yang Liu, and Zhi Jin
    Towards Robustness of Deep Program Processing Models -- Detection, Estimation and Enhancement
    ACM Transactions on Software Engineering and Methodology (TOSEM 2021, CORE Rank A*, Impact Factor 2.674)
  44. Xiaofei Xie, Tianlin Li, Jian Wang, Lei Ma, Qing Guo, Felix Juefei-Xu, Yang Liu
    NPC: Neuron Path Coverage via Characterizing Decision Logic of Deep Neural Networks
    ACM Transactions on Software Engineering and Methodology (TOSEM 2021, CORE Rank A*, Impact Factor 2.674)
  45. Bing Yu, Hua Qi, Qing Guo, Felix Juefei-Xu, Xiaofei Xie, Lei Ma, Jianjun Zhao
    DeepRepair: Style-Guided Repairing for Deep Neural Networks in the Real-World Operational Environment
    IEEE Transactions on Reliability (Impact Factor 3.177)
  46. Qing Guo, Ziyi Cheng, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Yang Liu, and Jianjun Zhao
    Learning to Adversarially Blur Visual Object Tracking
    International Conference on Computer Vision, Montreal, Canada, 2021 (ICCV 2021, CORE Rank A*)
  47. Yihao Huang, Qing Guo, Felix Juefei-Xu, Lei Ma, Weikai Miao, Yang Liu, Geguang Pu
    Predictive Perturbation-aware Filtering against Adversarial Attack via Multi-domain Learning
    The 29th ACM International Conference on Multimedia (ACM MM 2021, CORE Rank A*)
  48. Jiaming Ye, Ke Chen, Xiaofei Xie, Lei Ma, Ruochen Huang, Yingfeng Chen, Yinxing Xue, Jianjun Zhao
    An Empirical Study of GUI Widget Detection for Industrial Mobile Games
    The ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Industry Track (ESEC/FSE 2021, CORE Rank A*)
  49. Xiaofei Xie, Wenbo Guo, Lei Ma, Wei Le, Jian Wang, Lingjun Zhou, Yang Liu, Xinyu Xing
    Automatic RNN Repair via Model-based Analysis
    The 38th International Conference on Machine Learning, 2021 (ICML 2021, CORE Rank A*)
  50. Zhenya Zhang, Deyun Lyu, Paolo Arcaini, Lei Ma, Ichiro Hasuo and Jianjun Zhao
    Effective Hybrid System Falsification Using Monte Carlo Tree Search Guided by QB-Robustness
    The 33rd International Conference on Computer-Aided Verification, 2021 (CAV 2021, CORE Rank A*)
  51. Alvin Chan, Lei Ma, Felix Juefei-Xu, Xiaofei Xie, Yang Liu, Yew Soon Ong
    Breaking Neural Reasoning Architectures with Metamorphic Relation-Based Adversarial Examples
    IEEE Transactions on Neural Networks and Learning Systems, 2021 (TNNLS, Impact Factor: 11.683)
  52. Xiaolu Hou, Jakub Breier, Dirmanto Jap, Lei Ma, Shivam Bhasin, Yang Liu
    Physical Security of Deep Learning on Edge Devices: Comprehensive Evaluation of Fault Injection Attack Vectors
    Elsevier Microelectronics Reliability Journal, 2021 (Impact Factor: 1.535)
  53. Yan Zheng, Yi Liu, Xiaofei Xie, Yepang Liu, Lei Ma, Jianye Hao, and Yang Liu
    Automatic Web Testing using Curiosity-Driven Reinforcement Learning
    The 43rd International Conference on Software Engineering (ICSE 2021, CORE Rank A*)
  54. Maryam V. Pour, Li Zhuo, Lei Ma and Hadi Hemmati
    A Search-Based Testing Framework for Deep Neural Networks of Source Code Embedding
    IEEE International Conference on Software Testing, Verification and Validation (ICST 2021, CORE Rank A)
  55. Xiyue Zhang, Xiaoning Du, Xiaofei Xie, Lei Ma, Yang Liu, Meng Sun
    Decision-Guided Weighted Automata Extraction from Recurrent Neural Networks
    Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI 2021, CORE Rank A*)
  56. Qing Guo, Jingyang Sun, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Wei Feng, Yang Liu, and Jianjun Zhao
    EfficientDeRain: Learning Pixel-wise Dilation Filtering for High-Efficiency Single-Image Deraining
    Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI 2021, CORE Rank A*)
  57. Haijun Wang, Yi Li, Shang-Wei Lin, Cyrille Artho, Lei Ma, and Yang Liu.
    Oracle- Supported Dynamic Exploit Generation for Smart Contracts
    IEEE Transactions on Dependable and Secure Computing (TDSC 2020, Impact factor 6.864)
  58. Qing Gu, Felix Juefei-Xu, Xiaofei Xie, Lei Ma, Jian Wang, Bing Yu, Wei Feng, Yang Liu
    Watch out! Motion is Blurring the Vision of Your Deep Neural Networks
    Thirty-fourth Conference on Neural Information Processing Systems (NeurIPS 2020, CORE Rank A*)
  59. Zi Peng, Jinqiu Yang, Tse-Hsun Chen, Lei Ma
    A First Look at the Integration of Machine Learning Models in Complex Autonomous Driving Systems - A Case Study on Apollo
    The 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Industry Track (ESEC/FSE 2020, CORE Rank A*/CCF A)
  60. Xiaoning Du, Yi Li, Xiaofei Xie, Lei Ma, Yang Liu, Jianjun Zhao
    MARBLE: Model-Based Robustness Analysis of Stateful Deep Learning Systems
    The 35th IEEE/ACM International Conference on Automated Software Engineering. (ASE 2020, CORE Rank A/CCF A)
  61. David Berend, Xiaofei Xie, Lei Ma, Lingjun Zhou, Yang Liu, Chi Xu, Jianjun Zhao
    Cats Are Not Fish: Deep Learning Testing Calls for Out-Of-Distribution Awareness
    The 35th IEEE/ACM International Conference on Automated Software Engineering. (ASE 2020, CORE Rank A/CCF A)
  62. Xuhong Ren, Bing Yu, Hua Qi, Felix Juefei-Xu, Zhuo Li, Wanli Xue, Lei Ma and Jianjun Zhao
    Few-Shot Guided Mix for DNN Repairing
    The 36th IEEE International Conference on Software Maintenance and Evolution, NIER Track (ICSME 2020, CORE Rank A)
  63. Yuechen Wu, Yingfeng Chen, Xiaofei Xie, Bing Yu, Changjie Fan and Lei Ma
    Regression Testing of Massively Multiplayer Online Role-Playing Games
    The 36th IEEE International Conference on Software Maintenance and Evolution, NIER Track (ICSME 2020, CORE Rank A)
  64. Hua Qi, Qing Guo, Felix Juefei-Xu, Xiaofei Xie, Lei Ma, Wei Feng, Yang Liu, Jianjun Zhao
    DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms
    The 28th ACM International Conference on Multimedia. (ACM MM 2020, CORE Rank A*/CCF A)
  65. Run Wang,Felix Juefei-Xu, Yihao Huang, Qing Guo, Xiaofei Xie, Lei Ma, Yang Liu
    DeepSonar: Towards Effective and Robust Detection of AI-Synthesized Fake Voices
    The 28th ACM International Conference on Multimedia. (ACM MM 2020, CORE Rank A*/CCF A)
  66. Run Wang,Felix Juefei-Xu, Yihao Huang, Qing Guo, Xiaofei Xie, Lei Ma, Yang Liu
    Amora: Black-box Adversarial Morphing Attack
    The 28th ACM International Conference on Multimedia. (ACM MM 2020, CORE Rank A*/CCF A)
  67. Qing Guo, Xiaofei Xie, Felix Juefei-Xu, Lei Ma, Zhongguo Li, Wanli Xue, Wei Feng, Yang Liu
    SPARK: Spatial-aware Online Incremental Attack Against Visual Tracking
    The 16th European Conference on Computer Vision (ECCV 2020, CORE Rank A)
  68. Run Wang, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Yihao Huang, Jian Wang, Yang Liu
    FakeSpotter: A Simple Baseline for Spotting AI-Synthesized Fake Faces
    The 30th International Joint Conference on Artificial Intelligence - Paciffic Rim International Conference on Artificial Intelligence, 7 pages, Japan, July 2020. (IJCAI 2020, CORE Rank A*/CCF A) (Media Coverage by Synced AI Industry Review)
  69. Xiyue Zhang, Xiaofei Xie, Lei Ma, Xiaoning Du, Qiang Hu, Yang Liu, Jianjun Zhao, Meng Sun
    Towards Characterizing Adversarial Defects of Deep Learning Software from the Lens of Uncertainty
    The 42nd International Conference on Software Engineering, 12 pages, 23-29 May 2020, Seoul, South Korea (ICSE’20, CORE Rank A*/CCF A)
  70. Jie M. Zhang, Mark Harman, Lei Ma, Yang Liu
    Machine Learning Testing: Survey, Landscapes and Horizons
    IEEE Transactions on Software Engineering, 2020. (TSE, CORE Rank A*, Impact Factor:4.778)
    TSE 2022 Anual Best Paper Award.
  71. Huangzhao Zhang, Zhuo Li, Ge Li, Lei Ma, Yang Liu, Zhi Jin
    Generating Adversarial Examples for Holding Robustness of Source Code Processing Models
    The 34th AAAI Conference on Artificial Intelligence, New York, USA, Feb 7-12, 2020. (AAAI'20, CCF Rank A)
  72. Jianwen Sun, Tianwei Zhang, Xiaofei Xie, Lei Ma, Yan Zheng, Kangjie Chen, Yang Liu
    Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning.
    The 34th AAAI Conference on Artificial Intelligence, New York, USA, Feb 7-12, 2020. (AAAI'20, CCF Rank A)
  73. Li Li, Jun Gao, Tegawendé F. Bissyandé, Lei Ma, Xin Xia and Jacques Klein
    CDA: Characterising Deprecated Android APIs
    Empirical Software Engineering Journal, 2020 (EMSE, SCI-indexed, Impact Factor: 4.457)
  74. Cyrille Artho, Kazuuki Banzai, Quentin Gros, Guillaume Rousset, Lei Ma, Takashi Kitamura, Masami Hagiya, Yoshinori Tanabe and Mitsuharu Yamamoto
    Model-based Testing of Apache ZooKeeper: Fundamental API Usage and Watchers
    Software Testing, Verification and Reliability, 2019 (STVR, SCI-indexed, Impact Factor: 1.171)
  75. Yan Zheng, Xiaofei Xie, Ting Su, Lei Ma, Jianye Hao, Zhaopeng Meng, Yang Liu, Ruimin Shen, Yinfeng Chen, Changjie Fan
    Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement Learning
    In Proc. 34th IEEE/ACM Conference on Automated Software Engineering (ASE 2019), San Diego, California, USA, November 11-15, 2019. (ASE'19, CCF Rank A)
    ACM SIGSOFT Distinguished Paper Award.
  76. Qianyu Guo, Sen Chen, Xiaofei Xie, Lei Ma, Qiang Hu, Hongtao Liu, Yang Liu, Jianjun Zhao, Xiaohong Li
    An Empirical Study towards Characterizing Deep Learning Development and Deployment across Different Frameworks and Platforms.
    In Proc. 34th IEEE/ACM Conference on Automated Software Engineering (ASE 2019), San Diego, California, USA, November 11-15, 2019. (ASE'19, CCF Rank A)
  77. Qiang Hu, Lei Ma, Xiaofei Xie, Bing Yu, Yang Liu, and Jianjun Zhao
    DeepMutation++: a Mutation Testing Framework for Deep Learning Systems
    In Proc. 34th IEEE/ACM Conference on Automated Software Engineering (ASE 2019), (Tool Demo Track), San Diego, California, USA, November 11-15, 2019. (ASE'19, CCF Rank A)
  78. Xiaoning Du, Xiaofei Xie, Yi Li, Lei Ma, Yang Liu, and Jianjun Zhao.
    A Quantitative Analysis Framework for Recurrent Neural Network.
    In Proc. 34th IEEE/ACM Conference on Automated Software Engineering (ASE 2019), (Tool Demo Track), San Diego, California, USA, November 11-15, 2019. (ASE'19, CCF Rank A)
  79. Xiaofei Xie, Hongxu Chen, Yi Li, Lei Ma, Yang Liu, and Jianjun Zhao
    Coverage-guided Fuzzing for Feedforward Neural Networks
    In Proc. 34th IEEE/ACM Conference on Automated Software Engineering (ASE 2019), (Tool Demo Track), San Diego, California, USA, November 11-15, 2019. (ASE'19, CCF Rank A)
  80. Tianyi Zhang, Cuiyun Gao, Lei Ma, Michael R. Lyu and Miryung Kim
    An Empirical Study of Common Challenges in Developing Deep Learning Applications
    The 30th International Symposium on Software Reliability Engineering (ISSRE'19, CORE Rank A)
  81. Bai Xue, Yang Liu, Lei Ma, Xiyue Zhang, Meng Sun and Xiaofei Xie
    Safe Inputs Generation for Black-box Systems
    The 24th International Conference on Engineering of Complex Computer Systems (ICECCS'19, CORE Rank A)
  82. Ruitao Feng, Sen Chen, Xiaofei Xie, Lei Ma, Shangwei Lin and Yang Liu
    MobiDroid: A Performance-Sensitive Malware Detection System on Mobile Platform
    The 24th International Conference on Engineering of Complex Computer Systems (ICECCS'19, CORE Rank A)
  83. Xiaofei Xie, Lei Ma, Haijun Wang, Yuekang Li, Yang Liu, Xiaohong Li
    DiffChaser: Detecting Disagreements for Deep Neural Networks
    The 29th International Joint Conference on Artificial Intelligence, Macau, China, August 2019 (IJCAI'19, CCF Rank A)
  84. Xiaoning Du, Xiaofei Xie, Yi Li, Lei Ma, Yang Liu and Jianjun Zhao.
    DeepStellar: Model-Based Quantitative Analysis of Stateful Deep Learning Systems
    The 27th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Tallinn, Estonia, August 2019 (FSE'19, CCF Rank A)
  85. Xiaofei Xie, Lei Ma, Felix Juefei-Xu, Minhui Xue, Hongxu Chen, Yang Liu, Jianjun Zhao, Bo Li, Jianxiong Yin, Simon See
    DeepHunter: A Coverage-Guided Fuzz Testing Framework for Deep Neural Networks
    The 28th International Symposium on Software Testing and Analysis, Beijing, China, July, 2019 (ISSTA'19, CCF Rank A)
  86. Haijun Wang, Yi Li, Shang-Wei Lin, Lei Ma, Yang Liu
    VULTRON: Catching Vulnerable Smart Contracts Once and for All
    The 41st ACM/IEEE International Conference on Software Engineering, Montreal, QC, Canada, May, 2019, (ICSE'19, CCF Rank A)
  87. Chao Xie, Hua Qi,Lei Ma, Jianjun Zhao
    DeepVisual: A Visual Programming Tool for Deep Learning Systems
    The 27th IEEE/ACM International Conference on Program Comprehension, Montreal, QC, Canada, May, 2019, (ICPC'19, CCF Rank B)
  88. Lei Ma, Felix Juefei-Xu, Minhui Xue, Bo Li, Li Li, Yang Liu, Jianjun Zhao
    DeepCT: Tomographic Combinatorial Testing for Deep Learning Systems
    2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering, pp. 614-618, Hangzhou, China, 2019 (SANER'19, CCF Rank B)
  89. Yanjie Zhao, Haoyu Wang, Lei Ma, Yuxin Liu, Li Li, John Grundy
    Knowledge Graphing Git Repositories: A Preliminary Study
    2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering, pp. 599-603, Hangzhou, China, 2019 (SANER'19, CCF Rank B)
  90. Sen Chen, Minhui Xue, Lingling Fan, Lei Ma,, Yang Liu, Lihua Xu
    How Can We Craft Large-Scale Android Malware? An Automated Poisoning Attack.
    2019 IEEE International Workshop on Artificial Intelligence for Mobile, pp. 21-24, Hangzhou, China, 2019. (AI4Mobile'19)
  91. Sen Chen, Lingling Fan, Ting Su, Lei Ma,, Yang Liu, Lihua Xu
    Automated Cross-Platform GUI Code Generation for Mobile Apps.
    IEEE International Workshop on Artificial Intelligence for Mobile, pp. 13-16, Hangzhou, China, 2019. (AI4Mobile'19)
  92. Jakub Breier, Xiaolu Hou, Dirmanto Jap, Lei Ma, Shivam Bhasin, Yang Liu
    Practical Fault Attack on Deep Neural Networks.
    The 25th ACM Conference on Computer and Communications Security, Poster Paper (CCS'18 CCF Rank A)
  93. Lei Ma, Felix Juefei-Xu, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Chunyang Chen, Ting Su, Li Li, Yang Liu, Jianjun Zhao, Yadong Wang
    DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems.
    The 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE'18, CORE Rank A)
    ACM SIGSOFT Distinguished Paper Award.
  94. Lei Ma, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Felix Juefei-Xu, Chao Xie, Li Li, Yang Liu, Jianjun Zhao, Yadong Wang
    DeepMutation: Mutation Testing of Deep Learning Systems.
    The 29th IEEE International Symposium on Software Reliability Engineering (ISSRE'18 CORE Rank A)
  95. Alvin Chan, Lei Ma, Felix Juefei-Xu, Xiaofei Xie, Yang Liu, Yew Soon Ong
    Metamorphic Relation Based Adversarial Attacks on Differentiable Neural Computer.
    arXiv:1809.02444
  96. Qiang Hu, Lei Ma, Jianjun Zhao
    DeepGraph: A PyCharm Tool for Visualizing and Understanding Deep Learning Models.
    The 25th Asia-Pacific Software Engineering Conference (APSEC'18 CORE Rank B)
  97. Li Li, Jun Gao, Tegawendé F. Bissyandé, Lei Ma, Xin Xia and Jacques Klein
    Characterising Deprecated Android APIs. The 15th International Conference on Mining Software Repositories, Gothenburg, Sweden (MSR'18, CORE Rank A)
    ACM FOSS Impact Paper Award.
  98. Cyrille Valentin Artho, Quentin Gros, Guillaume Rousset, Kazuaki Banzai, Lei Ma, Takashi Kitamura, Masami Hagiya, Yoshinori Tanabe and Mitsuharu Yamamoto
    Model-based API Testing of Apache ZooKeeper.
    The 10th IEEE International Conference on Software Testing, Verification and Validation, Tokyo, Japan (ICST'17, CORE Rank A)
    IEEE Best Presentation Award.
  99. Lei Ma,Bing Yu, Hiroyuki Sato and Yadong Wang
    Collaborative Computation Offloading in Heterogeneous Asynchronous Cloud Environment.
    2017 IEEE 41st Annual Computer Software and Applications Conference (COMPSAC'17, CORE Rank B)
  100. Yoriyuki Yamagata, Cyrille Artho, Masami Hagiya, Jun Inoue, Lei Ma, Yoshinori Tanabe, Mitsuharu Yamamoto
    Runtime monitoring for concurrent systems.
    The 16th International Conference in Runtime Verification, Madrid, Spain, September 2016, Springer.(RV'16, CORE Rank C)
  101. Lei Ma, Cheng Zhang, Bing Yu, Hiroyuki Sato
    An empirical study on the effects of code visibility on program testability.
    Software Quality Journal, 2016, Springer.(SQJ, SCI-Indexed, Impact Factor: 2.141)
  102. Lei Ma, Cheng Zhang, Bing Yu, Jianjun Zhao
    Retrofitting Automatic Testing through Library Tests Reusing.
    The IEEE 24th International Conference on Program Comprehension, May 2016, Austin, USA. (ICPC'16, CCF B)
  103. Lei Ma, Cyrille Artho, Masami Hagiya, Yoshinori Tanabe, Yoriyuki Yamagata, Alexander Kohan and Mitsuharu Yamamoto
    Monitoring Distributed Applications with Java Pathfinder.
    2016 JPF Workshop 2016, November 2016, Seattle, USA. (JPF Workshop'16)
  104. Alexander Kohan, Mitsuharu Yamamoto, Cyrille Artho, Yoriyuki Yamagata, Lei Ma, Masami Hagiya and Yoshinori Tanabe
    Java Pathfinder on Android Devices.
    2016 JPF Workshop 2016, November 2016, Seattle, USA. (JPF Workshop'16)
  105. Jiabin Ye, Cheng Zhang, Lei Ma, Haibo Yu, Jianjun Zhao
    Efficient and Precise Dynamic Slicing for Client-Side JavaScript Programs.
    The IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering, March 2016, Osaka, Japan. (SANER'16, CCF B)
    IEEE Best Paper Candidate Award.
  106. Cyrille Artho, Lei Ma
    Classification of Randomly Generated Test Cases.
    The first IEEE International Workshop on Validating Software Tests, March, 2016, Osaka, Japan. (VST'16 colocated with SANER'16)
  107. Lei Ma, Cyrille Artho, Cheng Zhang, Hiroyuki Sato, Johannes Gmeiner, Rudolf Ramler
    GRT: Program-analysis-guided random testing.
    The 30th IEEE/ACM International Conference on Automated Software Engineering, November 2015, Lincoln, USA. (ASE'15, CCF A)
    ACM SIGSOFT Distinguished Paper Award.
  108. Lei Ma, Cyrille Artho, Cheng Zhang, Hiroyuki Sato, Johannes Gmeiner, Rudolf Ramler
    GRT: An automated test generator using orchestrated program analysis.
    The 30th IEEE/ACM International Conference on Automated Software Engineering, November 2015, Lincoln, USA. (ASE'15, CCF A)
  109. Lei Ma, Cheng Zhang, Bing Yu, Hiroyuki Sato
    An empirical study on effects of code visibility on code coverage of software testing.
    The IEEE/ACM 10th International Workshop on Automation of Software Test, May 2015, Florence, Italy. (AST'15, CORE Rank C)
  110. Lei Ma, Cyrille Artho, Cheng Zhang, Hiroyuki Sato,Masami Hagiya, Yoshinori Tanabe, Mitsuharu Yamamoto
    GRT at the SBST 2015 tool competition. The IEEE 8th International Workshop on Search-based Software Testing, May 2015, Florence, Italy.
    IEEE Best Testing Tool Award.
  111. Bing Yu, Lei Ma, Cheng Zhang
    Incremental Web Application Testing using Page Object.
    The IEEE International Workshop on Hot Topics in Web Systems and Technologies, November 2015, Washington, D.C, USA. (HotWeb'15, CORE Rank B)
    IEEE Best Paper Award.
  112. Lei Ma, Cyrille Artho, Cheng Zhang, and Hiroyuki Sato
    Efficient testing of software product lines via centralization.
    The ACM International Conference on Generative Programming: Concepts and Experiences, September 2014, Vasteras, Sweden. (GPCE'14, CORE Rank B)
  113. Lei Ma, Cyrille Artho, Cheng Zhang, and Hiroyuki Sato
    When Project Centralization and Random Testing Meet: Efficient Automatic Testing of Multiple Software Product Variants.
    The 2014 JSSST Software Conference, July 2014, Nagoya, Japan. (JSSST'14)
  114. Lei Ma, Cyrille Artho, and Hiroyuki Sato
    Managing Product Variants by Project Centralization.
    Lecture Notes on Software Engineering (LNSE), January 2014.
  115. Lei Ma, Cyrille Artho, and Hiroyuki Sato
    Project centralization based on graph coloring.
    The ACM 29th International Symposium on Applied Computing, March 2014, Gyeongju, South Korea. (SAC'14, CORE Rank B)
  116. Lei Ma, Cyrille Artho, and Hiroyuki Sato
    Improving automatic centralization by version separation.
    IPSJ Transactions on Programming, 6(4), December 2013.
  117. Kentaro Ishii, Haipeng Mi, Lei Ma, Natsuda Laokulrat, Masahiko Inami, Takeo Igarashi
    Pebbles: Tangible Devices for Designing Robot Navigation Routes.
    Computer Software Journal, August 2013.
  118. Lei Ma, Cyrille Artho, Hiroyuki Sato
    Analyzing distributed Java applications by automatic centralization.
    The IEEE 37th Annual Computer Software and Applications Conference Workshops, July 2013, Kyoto, Japan.(COMPSACW'13)
  119. Kentaro Ishii, Haipeng Mi, Lei Ma, Natsuda Laokulrat, Masahiko Inami, Takeo Igarashi
    Pebbles: User-Configurable Device Network for Robot Navigation.
    The 14th IFIP International Conference on Human-Computer Interaction, September 2013, Cape Town, South Africa. (INTERACT'13, CORE Rank A)
  120. Kentaro Ishii, Haipeng Mi, Lei Ma, Natsuda Laokulrat, Masahiko Inami, Takeo Igarashi
    Pebbles: an interactive configuration tool for indoor robot navigation.
    The 25th Annual ACM Symposium on User Interface Software and Technology, October 2012, Cambridge, MA, USA. (UIST'12, CORE Rank A)

  • Tutorial on AI Quality Assurance
    collocated with the 37th IEEE/ACM International Conference on Automated Software Engineering (ASE), Oct., 2022
  • Quality Assurance for Data-driving Intelligent System Engineering
    The CCF Advanced Disciplines Lectures (ADL), July 31, 2022
  • Towards Quality and Reliability Engineering in the era of Data-Driven Complex AI Systems
    The University of Tokyo, July 20, 2022
  • AI Debugging and Testing Tutorial
    Machine Learning System Engineering Summer Training Camp, Tokyo, July 1, 2022
  • Towards Quality and Reliability Engineering for Data-Driven AI Systems
    Zhejiang University, Dec. 29, 2021
  • An Orchestrated Reflection of Quality Assurance in the era of Data-Driven Intelligent Systems
    The 15th International Conference on Verification and Evaluation of Computer and Communication Systems (VECoS 2021), Virtual, Nov. 22, 2021
  • Quality Assurance and Engineering in the era of Data-driven Complex AI Systems
    Shanghai Jiaotong University, Oct. 18, 2021
  • Towards Testing and Analysis of Data-driven AI-enabled Software Systems
    Guest Lecture at The Australian National University, Virtual, Sep 22, 2021
  • Quality Assurance in the era of Data-driven Complex Intelligent Systems
    Machine Learning System Engineering Summer Training Camp, Virtual, July 1, 2021
  • Quality Assurance for Data-driven Intelligent Systems from Testing, Analysis to Repairing
    JST Mirai eAI Project Colloquium, Virtual, June 11, 2021
  • Challenges and Opportunities of Software Engineering in the era of Data-Driven Intelligent Systems
    AIST Software Engineering workshop, Virtual, March 26, 2021
  • Reliable and Trustworthy AI Systems In the Data-Driven Era
    DIY AI Technical Talk, ISA Edmonton Section, Virtual, March 15, 2021
  • Explainability-driven Analysis in the era of Data-Driven Intelligent Systems
    The 3rd International Workshop on Intelligent Bug Fixing, Virtual, March 9, 2021
  • Testing and Analysis Towards Reliable Deep Learning Systems
    Nvidia Corporation, Virtual, Nov.19, 2020
  • Testing and Analysis of Deep Learning Systems
    University of Science and Technology of China, Virtual, Nov.13, 2020
  • Testing and Analysis of Deep Learning Systems
    Tsinghua University, Virtual, Oct.14, 2020
  • Towards Characterizing Adversarial Defects of Deep Learning Software from the Lens of Uncertainty
    The 37th JSSST Annual Conference, Virtual, Sep.8, 2020
  • Testing and Analysis of Stateful Deep Learning Systems
    Institute for Data Valorisation (IVADO) Training Program on the Software Engineering of Machine Learning-based Systems, Montreal, Canada, March 3-6, 2020
  • Automatic Video Game Testing via Evolutionary Deep Reinforcement Learning
    Polytechnique Montreal, Canada, Feb.6, 2020
  • Quality and Reliability Assurance for Deep Learning Systems
    School of Computer Science, McGill University, Canada, Feb.5, 2020
  • Towards Analyzing Stateful Deep Learning System
    University of British Columbia, Canada, Dec.13, 2019
  • Testing and Analysis of Deep Learning Systems
    National Software Application Conference of China (NASAC'19), Nov. 24, Hangzhou, China, 2019
  • Quality and Reliability for Machine Learning Systems
    Celebration of ASE 2019, 10 Nov. San Diego, California, USA, 2019
  • DeepStellar: Analysis of Stateful Deep Learning Systems
    JST ERATO MMSD (Metamathematics for System Design) Project, Tokyo, Japan, 7 Nov. 2019
  • Towards Quality and Reliability Engineering of Deep Learning Systems
    The 8th Asian Workshop on Advanced Software Engineering (AWASE 2019), Fukuoka, Japan, 28-29 Oct., 2019
  • Towards Testing and Analysis of Deep Learning Systems
    MLSE International Symposium 2019, Tokyo, Japan, 18 Oct., 2019
  • Towards Testing Deep Learning Systems
    Shonan Meeting SEMLS (Software Engineering for Machine Learning Systems), Shonan Village, Japan, 13-17 Oct., 2019
  • Quality Assurance of Deep Learning Engineering
    NAIST Data Science Day, Nara, Japan, 11 Oct., 2019
  • Quality Assurance of Machine Learning Systems
    Intelligent Software Testing Symposium, Nanjing, China, 17 Sep., 2019
  • Testing Deep Learning System Towards Intelligence of Things
    Google Waterloo, Canada, June 5, 2019
  • Towards Quality Assurance of Deep Learning Systems
    The Software Engineering for Machine Learning Applications (SEMLA) International Symposium, Polytechnique Montreal, Canada, May 23 – 24, 2019
  • Software Testing of Deep Learning System: from the Past to the Future
    Japan Machine Learning Systems Engineering Community (MLSE, 日本機械学習工学研究会), Tokyo, Japan, April 26, 2019
  • DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems
    2018 National Software Application Conference of China (NASAC'18), Nov. 23, ShenZhen, China, 2018
  • Quality Assurance for Deep Learning System
    KTH Royal Institute of Technology, Stockholm, Sweden, Sep. 13, 2018
  • Deep Learning Quality Assurance in Industrial Scale
    2018 Nvidia NVAITC AI Symposium, Singapore, August 30, 2018
  • Program Analysis Guided Intelligent Automated Testing
    2017 National Software Testing Competition Symposium, Nanjing, China, October 25, 2017
  • A Comprehensive Study on Automated Testing
    National Institute of Advanced Industrial Science and Technology (AIST), Osaka, Japan, March 24, 2017
  • Program Analysis Guided Software Verification and Testing
    National Institute of Advanced Industrial Science and Technology (AIST), Osaka, Japan, March 13, 2016
  • Guided Random Testing: When Program Analysis Meets Automatic Test Generation
    Static Analysis Meets Runtime Verification NII Shonan Meeting, Shonan Village Center, Japan, March 15-19, 2015

Standards and Journals


  • Associate Editor, IEEE Transactions on Software Engineering (TSE)
  • Associate Editor, Automated Software Engineering Journal, Springer (AUSE)
  • Associate Editor, IEEE Transactions on Reliability (ToR)
  • Advisory Board Member, Empirical Software Engineering Journal, Springer (EMSE)
  • Guest Editor, Computers & Security, ELSEVIER
    Special Issue on "Adversarial Machine Learning towards Advanced Intelligent Systems (Impact Factor: 5.105)
  • Task Force Members, IEEE CIS Neural Networks on Secure Learning
  • Member of Standards Council of Canada, ISO/IEC JTC 1/SC 42 Artificial intelligence
  • Reviewer, ACM Transactions on Software Engineering and Methodology (TOSEM)
  • Reviewer, Journal of Systems and Software (JSS)
  • Reviewer, The international journal of Software Testing, Verification and Reliability (STVR)
  • Reviewer, Journal of Software: Evolution and Process
  • Reviewer, Software and Systems Modeling, Springer
  • Reviewer, IEEE Transactions on Dependable and Secure Computing (TDSC)
  • Reviewer, IEEE Transactions on BigData
  • Reviewer, Transactions on Machine Learning Research (TMLR)
  • Reviewer, IEEE Transactions on Neural Networks and Learning Systems (TNNLS)
  • Reviewer, IEEE Transactions on Multimedia (TMM)
  • Reviewer, IEEE Software
  • Reviewer, ACM Computing Surveys

Conference


2025

  • PC Member, The 47th International Conference on Software Engineering (ICSE 2025)
  • PC Member, The 34th The ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2025)

2024

  • PC Co-Chair, The 35th IEEE International Symposium on Software Reliability Engineering (ISSRE 2024)
  • Area Chair, Thirty-seventh Conference on Neural Information Processing Systems, Research Track & Benchmark Track (NeurIPS 2024)
  • PC member, The 39th IEEE/ACM International Conference on Automated Software Engineering (ASE 2024)
  • PC Member, The 33rd The ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2024)
  • PC Member, The ACM International Conference on the Foundations of Software Engineering, Poster Track (FSE 2024)
  • PC member, The 32nd IEEE/ACM International Conference on Program Comprehension (ICPC 2024)
  • PC member, The 3rd International Conference on AI Engineering/Software Engineering for AI, Industry Track (CAIN 2024)
  • PC member, The 1st ACM International Conference on AIware (AIware 2024)
  • PC member, The Genetic and Evolutionary Computation Conference (GECCO 2024)
  • PC member, The 24th IEEE International Conference on Software Quality, Reliabiilty, and Security & Applications (QRS 2024)
  • PC member, The 4th International Workshop on Software Engineering and AI for Data Quality in Cyber-Physical Systems/Internet of Things (SE4DQ 2024)
  • PC Member, The 33rd International Joint Conference on Artificial Intelligence (IJCAI 2024)
  • Reviewer, The Twelfth International Conference on Learning Representations (ICLR 2024)
  • Reviewer, The IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023 (CVPR 2024)
  • Reviewer,The 41st International Conference on Machine Learning (ICML 2024)
  • Reviewer,The ACM International Conference on Multimedia (ACM MM 2023)
  • PC member, IEEE Computer Society Signature Conference on Computers, Software and Applications (COMPSAC 2024)
  • PC member, The 28th International Conference on Engineering of Complex Computer Systems (ICECCS 2024)
  • PC member, The 15th Asia-Pacific Symposium on Internetware (Internetware'24)
  • Reviewer, The IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2024)

2023

  • Area Chair, Thirty-seventh Conference on Neural Information Processing Systems, Datasets and Benchmarks Track (NeurIPS 2023)
  • PC member, The 45th International Conference on Software Engineering (ICSE 2023)
  • PC member, The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2023)
  • PC member, The 38th IEEE/ACM International Conference on Automated Software Engineering (ASE 2023)
  • PC member, International Symposium on Formal Methods (FM 2023)
  • PC member, The 34rd IEEE International Symposium on Software Reliability Engineering (ISSRE 2023)
  • PC member, The 2nd International Conference on AI Engineering/Software Engineering for AI (CAIN 2023)
  • PC member, The International Workshop on Dependability of Safety-Critical Systems with Machine Learned Components (D-SyMLe 2023)
  • PC member, The 4th International Workshop on Testing for Deep Learning and Deep Learning for Testing (DeepTest 2023)
  • PC member, The 31st IEEE/ACM International Conference on Program Comprehension (ICPC 2023)
  • PC member, The Genetic and Evolutionary Computation Conference (GECCO 2023)
  • PC member, The 27th International Conference on Engineering of Complex Computer Systems (ICECCS 2023)
  • PC member, IEEE Computer Society Signature Conference on Computers, Software and Applications (COMPSAC 2023)
  • PC member, The 16th International Conference on Verification and Evaluation of Computer and Communication Systems (VECOS 2023)
  • PC Member, AAAI Spring Symposium on AI Trustworthiness Assessment (AITA 2023)
  • PC Member, The 32nd International Joint Conference on Artificial Intelligence (IJCAI 2023)
  • Reviewer, The IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2023)
  • Reviewer, The Eleventh International Conference on Learning Representations (ICLR 2023)
  • Reviewer, The IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023 (CVPR 2023)
  • Reviewer,The 40th International Conference on Machine Learning (ICML 2023)
  • Reviewer,The ACM International Conference on Multimedia (ACM MM 2023)
  • Reviewer, Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS 2023)

2022

  • Area Chair, Thirty-sixth Conference on Neural Information Processing Systems, Datasets and Benchmarks Track (NeurIPS 2022)
  • Mentor, Student Mentoring Workshop (SMeW @ ICSE 2022)
  • PC member, The 44th International Conference on Software Engineering (ICSE 2022)
  • Co-organizer, The 2nd International Workshop on AI and Software Testing/Analysis (AISTA 2022)
  • PC Member, 15th ACM Workshop on Artificial Intelligence and Security (AISec 2022)
  • PC member, The 33rd IEEE International Symposium on Software Reliability Engineering (ISSRE 2022)
  • PC member, The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2022)
  • PC member, The 37th IEEE/ACM International Conference on Automated Software Engineering (ASE 2022)
  • PC member, The 38th IEEE International Conference on Software Maintenance and Evolution (ICSME 2022)
  • PC member,16th International Conference Verification and Evaluation of Computer and Communication Systems (VECoS 2022)
  • SPC member,Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI 2022)
  • PC member,The 31th International Joint Conference on Artificial Intelligence (IJCAI 2022)
  • Reviewer,The 39th International Conference on Machine Learning (ICML 2022)
  • Reviewer,IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2022)
  • Reviewer, Tenth International Conference on Learning Representations (ICLR 2022)
  • Reviewer,The European Conference on Computer Vision (ECCV 2022)

2021

  • PC Member, 14th ACM Workshop on Artificial Intelligence and Security (AISec 2021)
  • Reviewer, The Thirty-fifth Conference on Neural Information Processing Systems (NeurIPS 2021)
  • Reviewer, The 38th International Conference on Machine Learning (ICML 2021)
  • Reviewer, The International Conference on Computer Vision (ICCV 2021)
  • Reviewer, The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021)
  • Student Volunteers Chair & PC member of Tool Demo Track, The ACM/IEEE 24th International Conference on Model Driven Engineering Languages and Systems (MODELS 2021)
  • Organizing Committee, The Workshop on AI and Software Testing/Analysis (AISTA 2021)
  • PC member, The 3rd International Workshop on Testing for Deep Learning and Deep Learning for Testing (DeepTest 2021)
  • PC member, The 22nd Genetic and Evolutionary Computation Conference (GECCO 2021)
  • PC member, The 36th IEEE/ACM International Conference on Automated Software Engineering (ASE 2021)
  • PC member, The 29th IEEE/ACM International Conference on Program Comprehension (ICPC 2021)
  • PC member, The 45th IEEE Computer Society International Conference on Computers, Software & Applications (COMPSAC 2021)
  • PC member, The 21th IEEE International Conference on Software Quality, Reliabiilty, and Security & Applications (QRS'2021)
  • PC member, The 30th International Joint Conference on Artificial Intelligence (IJCAI 2021)
  • PC member, The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI 2021)
  • PC member, The ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2021)

2020

  • PC member, The 13th ACM Workshop on Artificial Intelligence and Security (AISec 2020)
  • Reviewer, The Thirty-fourth Conference on Neural Information Processing Systems (NeurIPS 2020)
  • PC member, The 35rd IEEE/ACM International Conference on Automated Software Engineering (ASE 2020)
  • PC member, The 36th IEEE International Conference on Software Maintenance and Evolution (ICSME’20)
  • PC member, The 21st Genetic and Evolutionary Computation Conference (GECCO 2020)
  • PC member, This 27th IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER 2020)
  • Workshop Chair, The 27th Asia-Pacific Software Engineering Conference (APSEC 2020)
  • Doctoral Symposium Co-Chair, THE 22nd International Conference on Formal Engineering Methods (ICFEM 2020)
  • Publicity and Social Media Chair, The 12th Asia-Pacific Symposium on Internetware (Internetware 2020)
  • PC member, The 25th International Conference on Engineering of Complex Computer Systems (ICECCS 2020)
  • PC member, The 8th International Workshop on Realizing Artificial Intelligence Synergies in Software Engineering (RAISE 2020)
  • PC member, The IEEE Intl. Conf. on Softw.Testing, Verification and Validation, Poster Track (ICST 2020)
  • PC member, The 14th International Symposium on Theoretical Aspects of Software Engineering (TASE 2020)
  • PC member, The 44th IEEE Computer Society International Conference on Computers, Software & Applications (COMPSAC 2020)
  • PC member, The 20th IEEE International Conference on Software Quality, Reliabiilty, and Security & Applications (QRS'2020)
  • PC member, The IEEE International Workshop on Intelligent Bug Fixing (IBF 2020)

    • 2019

    • PC member, The 34rd IEEE/ACM International Conference on Automated Software Engineering (ASE'2019)
    • Organizing Comitte Member, The IEEE International Workshop on Machine Learning and Software Testing(MLST'2019)
    • Organizing Comitte Member, The IEEE International Workshop on Artificial Intelligence for Mobile (AI4Mobile)
    • Local Chair & PC member, The 11th Asia-Pacific Symposium on Internetware (Internetware'19)
    • Local Chair, The 8th Asia-Pacific Workshop of Advanced Software Engineering (AWASE'19)
    • PC Member, The 26th IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER'19 ERA)
    • PC member, The 19th IEEE International Conference on Software Quality, Reliabiilty, and Security(QRS'2019)
    • PC member, 19th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM'2019)
    • PC member, The 43th IEEE Computer Society International Conference on Computers, Software & Applications (COMPSAC'19)
    • PC Member, International Symposium on Computing and Networking (CANDAR'19)
    • PC member, The 24th International Conference on Engineering of Complex Computer Systems(ICECCS'19)
    • PC member, The First International Workshop on Intelligent Bug Fixing (IBF'2019))
    • PC member, The 42th IEEE Computer Society International Conference on Computers, Software & Applications (COMPSAC'19)
    • PC Member, IEEE International Conference on Mobile Cloud Computing, Services, and Engineering (Mobile Cloud'19)
    • PC Member, The 19th IEEE International Conference on Scalable Computing and Communications (ScalCom'19)
    • Reviewer, The 16th International Conference on Mining Software Repositories (MSR'2019)

    2018

    • Reviewer, The 11th ACM Workshop on Artificial Intelligence and Security (AISec'18)
    • Reviewer, IEEE Transactions on Reliability
    • Reviewer, The 15th International Conference on Managed Languages & Runtimes (ManLang'18)
    • Reviewer, The 25th ACM Conference on Computer and Communications Security (CCS'18)
    • Reviewer, The 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE'18)
    • PC member, The 42th IEEE Computer Society International Conference on Computers, Software & Applications (COMPSAC'18)
    • Reviewer, ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA'18)
    • Reviewer, ETAPS 24th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS'18)
    • PC Member, The 18th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM'18)
    • PC Member, IEEE International Conference on Mobile Cloud Computing, Services, and Engineering (Mobile Cloud'18)
    • PC Member, The 18th IEEE International Conference on Scalable Computing and Communications (ScalCom'18)
    • PC Member, IEEE International Workshop on Validating Software Tests (VST'18)
    • PC Member, The 26th European Conference On Information Systems (ECIS'18)
    • PC member, The Exellent Ph.D Research Forum (NASAC'2018)
    • Reviewer, Springer Journal of Computer Virology and Hacking Techniques
    • PC member, The 18th International Conference on Advances in System Testing and Validation Lifecycle (VALID'18)
    • PC Member, International Symposium on Computing and Networking (CANDAR'18)

    2017

    • Publicity Chair, The 24th International Conference on Engineering of Complex Computer Systems(ICECCS'17)
    • PC member, The 17th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM'17)
    • Reviewer, Elsevier Journal of Systems and Software (JSS)
    • PC member, The 41th IEEE Computer Society International Conference on Computers, Software & Applications (COMPSAC'17)
    • Publication Chair & PC Member, IEEE International Conference on Mobile Cloud Computing, Services, and Engineering (Mobile Cloud'17)
    • PC member, The Eighth International Conference on Advances in System Testing and Validation Lifecycle (VALID'17)
    • PC Member, International Symposium on Computing and Networking (CANDAR'17)
    • IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER): 2018, 2019

    2016 & 2015

    • External reviewer tool track, The 31st IEEE/ACM International Conference on Automated Software Engineering (ASE'16)
    • PC member, The Eighth International Conference on Advances in System Testing and Validation Lifecycle (VALID'16)
    • PC Member, IEEE International Workshop on Validating Software Tests (VST'16)
    • Publication Chair & PC Member, IEEE International Conference on Mobile Cloud Computing, Services, and Engineering (Mobile Cloud'16)
    • Technical reviewer, International Conference on Formal Aspects of Component Software (FACS'16)
    • Reviewer, Elsevier Science of Computer Programming Journal (FTSCS'16)
    • Tool Panel Member, International Workshop on Search-Based Software Testing (SBST'15)