Associate Professor

University of Alberta, Canada &

Kyushu University, Japan

Canada CIFAR AI Chair

Fellow at Alberta Machine Intelligent Institute (Amii)

Email: ma.lei [at] acm [dot] org

Trulli Trulli Trulli

Prospective Students:

Our research group is continuously recruiting. Currently, we have multiple scholarship & RA positions for Ph.D., MSc. candidates, and postdocs with competitive salary support, at both University of Alberta, Canada; and Kyushu University, Japan. If you have a strong background in the topics relevant (but not limited) to my research with strong self-motivation to achieve, I strongly encourage you to apply and join us. Note that to inquiry about opportunities to join our research group, please send emails only to contact@momentum-lab.com to apply!





img


  1. Call for Papers: Computers & Security Special Issue on "Adversarial Machine Learning towards Advanced Intelligent Systems (Impact Factor: 5.105). Consider submit your high quality papers! (Open until April 2023)
  2. (03/2023) Our paper Evading DeepFake Detectors via Adversarial Statistical Consistency is accepted at CVPR 2023 .
  3. (02/2023) Our paper PatchCensor: Patch Robustness Certification for Transformers via Exhaustive Testing is accepted at ACM Transactions on Software Engineering and Methodology (TOSEM 2023) .
  4. (02/2023) Our paper ArchRepair: Block-Level Architecture-Oriented Repairing for Deep Neural Networks is accepted at ACM Transactions on Software Engineering and Methodology (TOSEM 2023) .
  5. (01/2023) Our papers on human-centered interactive data debugging DeepLens and model debugging DeepSeer are accepted at CHI 2023 .
  6. (01/2023) Our paper Neural Episodic Control with State Abstraction is accepted at ICLR 2023 as a spotlight paper .
  7. (12/2022) Our paper GraphSearchNet: Enhancing GNNs via Capturing Global Dependencies for Semantic Code Search is accepted at IEEE Transactions on Software Engineering (TSE 2023) .
  8. (12/2022) Four papers accepted at ICSE 2023 (two on research track, one on SEIP track, one NIER track).
  9. (11/2022) Our paper DeepGemini: Verifying Dependency Fairness for Deep Neural Network is accepted at AAAI 2023 .
  10. (07/2022) Join IEEE CIS Neural Networks Committee on Secure Learning as a Task Force Member.
  11. (07/2022) Our paper Decompiling x86 Deep Neural Network Executables is accepted at The 32nd USENIX Security Symposium (USENIX Security '23 Summer)
  12. (07/2022) Our paper FalsifAI: Falsification of AI-Enabled Hybrid Control Systems Guided by Time-Aware Coverage Criteria is accepted at IEEE Transactions on Software Engineering (TSE)
  13. (06/2022) Our paper DARTSRepair: Core-Failure-Set Guided DARTS for Network Robustness to Common Corruptions is accepted at Pattern Recognition 2022
  14. (06/2022) Our paper xFuzz: Machine Learning Guided Cross-Contract Fuzzing is accepted at IEEE Transactions on Dependable and Secure Computing, TDSC 2022
  15. (03/2022) Our paper Countering Malicious DeepFakes: Survey, Battleground, and Horizon is accepted at International Journal of Computer Vision, IJCV 2022. We present a comprehensive survey covering the latest DeepFake generation and detection methods, as well as the battleground landscape.
  16. (01/2022) Our paper When Cyber-Physical Systems Meet AI: A Benchmark, an Evaluation, and a Way Forward is accepted at ICSE SEIP 2022, to the best of our knowledge, this is the very first systematic benchmark construction and study of AI-enabled Cyber Physical Systems (AI-CPS). We make our dataset and benchmark publically available, and hope this would inspire lots of future new reserach and investigation along this exciting direction!
  17. (01/2022) Our paper An Empirical Study on Data Distribution-Aware Test Selection for Deep Learning Enhancement is accepted at ACM Transactions on Software Engineering and Methodology (TOSEM 2022).
  18. (12/2021) Our paper An Empirical Study of the Impact of Hyperparameter Tuning and Model Optimization on DNN is accepted at ACM Transactions on Software Engineering and Methodology (TOSEM 2021).
  19. (11/2021) Our paper Towards Robustness of Deep Program Processing Models -- Detection, Estimation and Enhancement is accepted at ACM Transactions on Software Engineering and Methodology (TOSEM 2021).
  20. (09/2021) Our paper NPC: Neuron Path Coverage via Characterizing Decision Logic of Deep Neural Networks is accepted at ACM Transactions on Software Engineering and Methodology (TOSEM 2021).
  21. (08/2021) Our paper DeepRepair: Style-Guided Repairing for Deep Neural Networks in the Real-World Operational Environment is accepted at IEEE Transactions on Reliability 2021.
  22. (07/2021) Our paper Learning to Adversarially Blur Visual Object Tracking is accepted at ICCV 2021.
  23. (07/2021) Join IJCAI 2022 as a PC Member.
  24. (07/2021) Our paper Predictive Perturbation-aware Filtering against Adversarial Attack via Multi-domain Learning is accepted at ACM MM 2021.
  25. (07/2021) Our paper An Empirical Study of GUI Widget Detection for Industrial Mobile Games is accepted at FSE 2021 Industry Track.
  26. (06/2021) Join ICLR 2022 as a reviewer, AISec 2021 and ESEC/FSE 2022 as a PC member. Please consider submitting your high quality papers.
  27. (05/2021) Our paper Automatic RNN Repair via Model-based Analysis is accepted at ICML 2021.
  28. (04/2021) Our paper Effective Hybrid System Falsification Using Monte Carlo Tree Search Guided by QB-Robustness is accepted at CAV 2021.
  29. (04/2021) I joined University of Alberta as as Associate Professor and was honored to be selected as a Canada CIFAR AI Chair under the Pan-Canadian AI Strategy. I am also honored to be affiliated with Alberta Machine Intelligent Institute (Amii) as an Amii Fellow.
  30. (03/2021) Our survey paper Countering Malicious DeepFakes: Survey, Battleground, and Horizon that makes comprehensive coverage regarding malicious DeepFakes is available on Arxiv.
  31. (12/2020) Media coverage: Our work DeepRhythm that uses heartbeat detection to identify deepfake videos is reported in English (Link1, Link2), Japanese, and Chinese!
  32. (12/2020) Our paper Automatic Web Testing using Curiosity-Driven Reinforcement Learning is accepted at ICSE 2021.
  33. (12/2020) Our paper A Search-Based Testing Framework for Deep Neural Networks of Source Code Embedding is accepted at ICST 2021.
  34. (12/2020) Join ICML 2021 and CVPR 2021 as reviewers.
  35. (12/2020) Our papers EfficientDeRain: Learning Pixel-wise Dilation Filtering for High-Efficiency Single-Image Deraining and Decision-Guided Weighted Automata Extraction from Recurrent Neural Networks are accepted at AAAI 2021.
  36. (11/2020) Join ICSE 2022 as PC Member.
  37. (11/2020) Our paper Oracle-Supported Dynamic Exploit Generation for Smart Contracts after two years' countinous effort is finally accepted at IEEE Transactions on Dependable and Secure Computing (TDSC)!
  38. (11/2020) Join DeepTest 2021 as PC Member.
  39. (11/2020) Join GECCO 2021 as PC Member.
  40. (10/2020) Join ASE 2021 as PC Member.
  41. (10/2020) Join ICPC 2021 as PC Member.
  42. (10/2020) Join IJCAI 2021 as PC Member.
  43. (09/2020) Our paper Watch out! Motion is Blurring the Vision of Your Deep Neural Networks is accepted at NeurIPS 2020. This paper performs intensitive investigation on the impact of potential camera blurring effects to the decision of a DNN!
  44. (09/2020) Join AAAI 2021 as PC Member.
  45. (08/2020) Our empirical study on Autonomous Driving Systems and its ML Components is accepted at Industry Track of ESEC/FSE 2020.
  46. (07/2020) Our papers MARBLE: Model-Based Robustness Analysis of Stateful Deep Learning Systems and Cats Are Not Fish: Deep Learning Testing Calls for Out-Of-Distribution Awareness are accepted at ASE 2020.
  47. (07/2020) Two papers are accepted at ICSME 2020, NIER Track.
  48. (07/2020) Our DeepFake Series works DeepRhythm, DeepSonar, and Amora are accepted at ACM MM 2020.
  49. (07/2020) Our paper SPARK: Spatial-aware Online Incremental Attack Against Visual Tracking is accepted at ECCV 2020. This paper proposes a novel and efficient attack to mislead online trackers along an incorrect or specified trajectory!
  50. (06/2020) Join ESEC/FSE 2021 as PC Member.
  51. (06/2020) Join NeurIPS’20 as a reviewer.
  52. (05/2020) Our paper FakeSpotter is accepted at IJCAI 2020, a previous verison is reported as AI vs AI: ‘FakeSpotter’ Studies Neurons to Bust DeepFakes, by the Synced AI Industry Review!
  53. (05/2020) Our paper DeepSonar: Towards Effective and Robust Detection of AI-Synthesized Fake Voices is available on CoRR arXiv!
  54. (04/2020) Join AISec’20, ASE'20, ICSME'20 as PC member.
  55. (03/2020) Give tutorials regarding Testing and Analysis of Stateful Deep Learning Systems at IVADO Training Program on Software Engineering for Machine Learning Applications, at Montreal, Canada, March 2-6 2020.
  56. (02/2020) Give Invited talk Automatic Video Game Testing via Evolutionary Deep Reinforcement Learning at Polytechnique Montreal, Canada, Feb.6, 2020
  57. (02/2020) Give Invited talk Quality and Reliability Assurance for Deep Learning Systems at School of Computer Science, McGill University, Canada, Feb.5, 2020
  58. (02/2020) Our paper Towards Byzantine-resilient Learning in Decentralized Systems is available on CoRR arXiv!
  59. (02/2020) Our paper ABBA: Saliency-Regularized Motion-Based Adversarial Blur Attack is available on CoRR arXiv!
  60. (01/2020) Our paper FakeLocator: Robust Localization of GAN-Based Face Manipulations via Semantic Segmentation Networks with Bells and Whistles is available on CoRR arXiv!
  61. (12/2019) I am honored to be invited to Dagstuhl Seminar 20091:SE4ML - Software Engineering for AI-ML-based Systems to be held on February 23 – 28, 2020
  62. (12/2019) I am honored to be invited to give lectures at IVADO Training Program on Software Engineering for Machine Learning Applications, to be held at Montreal, Canada, March 2-6 2020.
  63. (12/2019) Our paper Amora: Black-box Adversarial Morphing Attack is available on CoRR arXiv!
  64. (12/2019) Our paper Towards Characterizing Adversarial Defects of Deep Learning Software from the Lens of Uncertainty is accepted at ICSE'2020, This work is the very early work that discovers the new uncommon adversarial examples that represent the challenging vulnerability of deep learning software, we are now working to seek new ways to counteract these issues!
  65. (11/2019) Our survey paper Machine Learning Testing: Survey, Landscapes and Horizons, on the emerging direction of testing and anlaysis of machine learning is accepted at IEEE Transactions on Software Engineering (TSE), a previous early draft is available at CoRR arXiv, we are working on the camera-ready version and would make it available soon!
  66. (11/2019) Our research framework DeepStudio: a Framework of Securing Deep Learning Systems receives the 2019 CCF Research Prototype Competition Award (Second Prize).
  67. (11/2019) Our papers Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning and Generating Adversarial Examples for Holding Robustness of Source Code Processing Models are accepted at AAAI 2020.
  68. (10/2019) Our paper Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement Learning receives the ACM SIGSOFT Distinguished Paper Award in ASE'19.
  69. (10/2019) Media coverage: AI vs AI: ‘FakeSpotter’ Studies Neurons to Bust DeepFakes, our work FakeSpotter is reported by the media of AI Industry Review!
  70. (10/2019) Our paper Model-based Testing of Apache ZooKeeper: Fundamental API Usage and Watchers is finally accepted at Software Testing, Verification AND Reliability (STVR) Journal after two years continous effort !
  71. (09/2019) Our papers DeepMutation++, DeepHunter, and DeepStellar are accepted and will be demonstrated at ASE 2019 as tool papers !
  72. (09/2019) Our papers Wuji and Deep Learning Platform/Framework Compatibility Study are accepted at ASE 2019 as full research papers !
  73. (08/2019) Our paper CDA: Characterising Deprecated Android APIs is accepted at Empirical Software Engineering (EMSE) Journal !
  74. (07/2019) Our paper An Empirical Study of Common Challenges in Developing Deep Learning Applications is accepted at ISSRE'19 !
  75. (07/2019) Two papers: Safe Inputs Generation for Black-box Systems and MobiDroid: A Performance-Sensitive Malware Detection System on Mobile Platform are accepted at ICECCS 2019!
  76. (06/2019) Our survey paper Machine Learning Testing: Survey, Landscapes and Horizons is available on CoRR arXiv!
  77. (05/2019) Our paper DiffChaser: Detecting Disagreements for Deep Neural Networks is accepted at IJCAI'19 !
  78. (05/2019) Our paper DeepStellar: Model-Based Quantitative Analysis of Stateful Deep Learning Systems is accepted at FSE'19 !
  79. (05/2019) I am invited to join AISec'19 Program Committee collocated with CSS'19 this year, please considier to submit your high quality papers!
  80. (05/2019) I am invited to visit and give a talk on Testing Deep Learning System Towards Intelligence of Things at Google Waterloo on June 5!
  81. (05/2019) Our paper DeepHunter: A Coverage-Guided Fuzz Testing Framework for Deep Neural Networks is accepted at ISSTA'19 !
  82. (04/2019) I am happy to be invited to join and give a talk on NII Shonan Meeting on Software Engineering for Machine Learning Systems (SEMLS) which will be held on October this year.
  83. (04/2019) I am invited by Machine Learning and Software Engineering Community of Japan (MLSE) to give tutorial on "Software Testing of Deep Learning System: from the Past to the Future"
  84. (04/2019) I am co-organizing the first MLST Workshop Collocated with ICST!
  85. (03/2019) I am happy to be invited to join and give a talk on The Software Engineering for Machine Learning Applications (SEMLA) International Symposium, Polytechnique Montreal which will be held on May this year.
  86. (02/2019) I am co-organizing the first AI4Mobile Workshop Collocated with SANER!
  87. (12/2018) Our paper VULTRON: Catching Vulnerable Smart Contracts Once and for All ICSE'19 as an NIER Track Paper!
  88. (12/2018) Our paper DeepCT: Tomographic Combinatorial Testing for Deep Learning Systems SANER'19 as an ERA Track Paper!
  89. (09/2018) Our paper Practical Fault Attack on Deep Neural Networks is accepted at CCS'18 as a Poster Paper!
  90. (09/2018) Our paper Metamorphic Relation Based Adversarial Attacks on Differentiable Neural Computer is available on CoRR arXiv. This is the very first work towards in-depth analysis on the robustness of Differtentiable Neural Computer!
  91. (08/2018) Our paper DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems receive the ACM SIGSOFT Distinguished Paper Award in ASE'18.
  92. (08/2018) Our paper DeepMutation: Mutation Testing of Deep Learning Systems is accepted at ISSRE'18
  93. (07/2018) Our paper DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems is accepted at ASE'18
  94. (06/2018) Our paper Combinatorial Testing for Deep Learning Systems is available on CoRR arXiv. This is the first combinatorial testing for Deep Neural Networks!
  95. (06/2018) Our paper Practical Fault Attack on Deep Neural Networks is available on CoRR arXiv. This is the first practical attacks on Deep Neural Networks using laser technology!
  96. (05/2018) Our paper Characterising Deprecated Android APIs won the FOSS Impact Paper Award in MSR'18.
  97. (05/2018) Our paper DeepMutation: Mutation Testing of Deep Learning Systems is available on CoRR arXiv. This is the first mutation testing technique for Deep Neural Networks!
  98. (03/2018) Our paper DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems is available on CoRR arXiv. This paper proposes a set of testing critiera with multi-granuality for Deep Neural Networks!
  99. (01/2018) Our paper Characterising Deprecated Android APIs has been accepted to appear in MSR'18.
  100. (03/2017) Our paper Model-based API Testing of Apache ZooKeeper won the IEEE Best Presentation Award at ICST'17.
  101. (07/2016) Our paper Runtime monitoring for concurrent systems has been accepted to appear in RV'16.
  102. (09/2016) Our paper An empirical study on the effects of code visibility on program testability has been accepted to appear in Springer Software Quliaty Journal (SQJ).
  103. (03/2016) Our paper Efficient and Precise Dynamic Slicing for Client-Side JavaScript Programs has been nominated for the IEEE Best Candidate Paper Award in SANER'16.
  104. (11/2015) Our paper GRT: Program-analysis-guided Random Testing won the ACM SIGSOFT Distinguished Paper Award at ASE'15.
  105. (11/2015) Our paper Incremental Web Application Testing using Page Object won the IEEE Best Paper Award at HotWeb'15.

About Me

I am currently an associate professor and Canada CIFAR AI Chair at the University of Alberta, leading Momentum Lab (website to be launched), and holds a shared appointment as an associate professor with Kyushu University, Japan. I am also honored to be selected as a Fellow and affiliated with Alberta Machine Intelligent Institute (Amii).
My current research mainly focuses on the interdisciplinary research fields of Software Engineering and Artificial Intelligence, and especially on proposing quality assurance and engineering support for building trustworthy AI systems.
I received a B.E. degree from Department of Computer Science and Engineering of Shanghai Jiaotong Unviersity in 2009, M.E. and Ph.D degrees from The University of Tokyo in 2011 and 2014, respectively. During my Ph.D. program, I studied for half a year in the Mathematics and Computer Science department of Technische Universität München(TUM). After receiving the Ph.D., I worked as a Research Fellow in the joint projects by Chiba University, The University of Tokyo, Advanced Industrial Science and Technology (AIST/ITRI) and National Institute of Informatics, and later joined in Harbin Institute of Technology as an Associate Professor in 2016, during which in 2018 I got the chance to visit NTU Singapore. In 2019, I joined Kyushu University as an Assistant Professor and later got promoted as a Tenured Associate Professor in 2020. In 2021, was selected to receive the honorship of Canada CIFAR AI Chair and Amii Fellow under the Pan-Canadian AI Strategy.

I love theories, but I like even more to transfer theories into practice!
I have several research interests and working directions. My current major research interest mainly focuses on providing general purpose quality assurance solutions and engineering support for intelligent software systems. I believe the trustworthy intelligent software system would be the key driving force for technology innovation of the future. It does not solely rely on the foundational research from the AI community, but also requires the contribution from the many communities such as software engineering, control, etc., upbringing a new interdisciplinary emerging field Machine Learning System Engineering, an extremely exciting and demanding area of the near future! We have conducted consecutive works along the general-purpose quality assurance research for intelligent software systems (e.g., testing, analysis, verification, interpretation, attacks, defenses), to bridge vast demands from industry. Exciting research results would be continuously updated!

In the past, my research interest mainly lies around software engineering and programming language, in particular automated software testing, verification, analysis, evolution, mining, etc. I also worked on green computation and optimization solutions for cloud and big data centers. In addition, I am also interested in genomic data analsyis to uncover life mysteries and to search for health solutions through trustworthy AI solutions.

    See also at Google Scholar or DBLP.

  1. Yang Hou, Qing Guo, Yihao Huang, Xiaofei Xie, Lei Ma, Jianjun Zhao
    Evading DeepFake Detectors via Adversarial Statistical Consistency
    The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2023, CORE Rank A*)
  2. Yuheng Huang, Lei Ma, Yuanchun Li
    PatchCensor: Patch Robustness Certification for Transformers via Exhaustive Testing
    ACM Transactions on Software Engineering and Methodology (TOSEM 2023, CORE Rank A*, Impact Factor 2.674)
  3. Hua Qi, Zhijie Wang, Qing Guo, Jianlang Chen, Felix Juefei-Xu, Fuyuan Zhang, Lei Ma, Jianjun Zhao
    ArchRepair: Block-Level Architecture-Oriented Repairing for Deep Neural Networks
    ACM Transactions on Software Engineering and Methodology (TOSEM 2023, CORE Rank A*, Impact Factor 2.674)
  4. Zhijie Wang, Yuheng Huang, Da Song, Lei Ma, Tianyi Zhang
    DeepSeer: Interactive RNN Explanation and Debugging via State Abstraction
    The ACM CHI Conference on Human Factors in Computing Systems (CHI 2023, CORE Rank A*)
  5. Da Song, Zhijie Wang, Yuheng Huang, Lei Ma, Tianyi Zhang
    DeepLens: Interactive Out-of-Distribution Data Detection in NLP Models
    The ACM CHI Conference on Human Factors in Computing Systems (CHI 2023, CORE Rank A*)
  6. Zhuo Li, Derui Zhu, Yujing Hu, Xiaofei Xie, Lei Ma, Yan Zheng, Yan Song, Yingfeng Chen, Jianjun Zhao
    Neural Episodic Control with State Abstraction
    The Eleventh International Conference on Learning Representations (ICLR 2023, spotlight CORE Rank A*)
  7. Shangqing Liu, Xiaofei Xie, Jingkai Siow, Lei Ma, Guozhu Meng, Yang Liu
    GraphSearchNet: Enhancing GNNs via Capturing Global Dependencies for Semantic Code Search
    IEEE Transactions on Software Engineering (TSE 2023, CORE Rank A*, Impact factor 6.22)
  8. Qiang Hu, Yuejun Guo, Xiaofei Xie, Maxime Cordy, Lei Ma, Mike Papadakis, Yves Le Traon
    Aries: Efficient Testing of Deep Neural Networks via Labeling-Free Accuracy Estimation
    The 45th International Conference on Software Engineering (ICSE 2023, CORE Rank A* )
  9. Jiongchi Yu, Yuechen Wu, Xiaofei Xie, Wei Le, Lei Ma, Yingfeng Chen, Yujing Hu, Fan Zhang
    GameRTS: A Regression Testing Framework for Video Games
    The 45th International Conference on Software Engineering (ICSE 2023, CORE Rank A* )
  10. Xiongfei Wu, Jiaming Ye, Ke Chen, Xiaofei Xie, Yujing Hu, Ruochen Huang, Lei Ma and Jianjun Zhao
    Widget Detection-based Testing for Industrial Mobile Games
    The 45th International Conference on Software Engineering, SEIP Track (ICSE 2023, CORE Rank A* )
  11. Qiang Hu, Yuejun Guo, Xiaofei Xie, Maxime Cordy, Lei Ma, Mike Papadakis, Yves Le Traon
    CodeS: Towards Code Model Generalization Under Distribution Shift
    The 45th International Conference on Software Engineering, NIER Track (ICSE 2023, CORE Rank A* )
  12. Xuan Xie, Fuyuan Zhang, Xinwen Hu, Lei Ma
    DeepGemini: Verifying Dependency Fairness for Deep Neural Network
    The Thirty-Seventh AAAI Conference on Artificial Intelligence (AAAI 2023, CORE Rank A* )
  13. Zhibo Liu, Yuanyuan Yuan, Shuai Wang, Xiaofei Xie, Lei Ma
    Decompiling x86 Deep Neural Network Executables
    The 32nd USENIX Security Symposium (USENIX Security 2023 Summer, CORE Rank A* )
  14. Shuncheng Tang, Zhenya Zhang, Yi Zhang, Jixiang Zhou, Yan Guo, Shuang Liu, Shengjian Guo, Yanfu Li, Lei Ma, Yinxing Xue, and Yang Liu
    A Survey on Automated Driving System Testing: Landscapes and Trends
    ACM Transactions on Software Engineering and Methodology (TOSEM 2023, CORE Rank A*, Impact Factor 2.674)
  15. Amin Eslami Abyane, Derui Zhu, Roberto Medeiros de Souza, Lei Ma, Hadi Hemmati
    Towards Understanding Quality Challenges of the Federated Learning: A First Look from the Lens of Robustness
    Empirical Software Engineering, Nov. 2022 (EMSE 2022, CORE Rank A)
  16. Zihe Song, Yingfeng Chen, Lei Ma, Shangjie Lu, Honglei Lin, Changjie Fan, Wei Yang
    An Empirical Analysis of Compatibility Issues for Industrial Mobile Games
    The 33rd IEEE International Symposium on Software Reliability Engineering, Oct. 2022 (ISSRE 2022, CORE Rank A)
  17. Xiongfei Wu, Jinqiu Yang, Lei Ma, Yinxing Xue, and Jianjun Zhao
    On the Usage and Development of Deep Learning Compilers: an Empirical Study on TVM
    Empirical Software Engineering volume 27, 172, Sep. 2022 (EMSE 2022, CORE Rank A)
  18. Zhuo Li, Yuechen Wu, Lei Ma, Xiaofei Xie, Yingfeng Chen, and Changjie Fan
    GBGallery : A Benchmark and Framework for Game Testing
    Empirical Software Engineering volume 27, 140, July 2022 (EMSE 2022, CORE Rank A)
  19. Zhenya Zhang, Deyun Lyu, Paolo Arcaini, Lei Ma, Ichiro Hasuo, and Jianjun Zhao
    FalsifAI: Falsification of AI-Enabled Hybrid Control Systems Guided by Time-Aware Coverage Criteria
    IEEE Transactions on Software Engineering (TSE 2022, CORE Rank A*, Impact factor 6.22 )
  20. Liming Zhai, Qing Guo, Xiaofei Xie, Lei Ma, Yi Estelle Wang, Yang Liu
    A3GAN: Attribute-Aware Anonymization Networks for Face De-identification
    The 30th ACM International Conference on Multimedia, 2022 (ACM MM 2022, CORE Rank A*)
  21. Xuhong Ren, Jianlang Chen, Felix Juefei-Xu, Wanli Xue, Qing Guo, Lei Ma, Jianjun Zhao, Shengyong Chen
    DARTSRepair: Core-Failure-Set Guided DARTS for Network Robustness to Common Corruptions
    Pattern Recognition, Elsevier, 2022 (CORE Rank A*, Impact factor 7.74)
  22. Yinxing Xue, Jiaming Ye, Wei Zhang, Jun Sun, Lei Ma, Haijun Wang, and Jianjun Zhao
    xFuzz: Machine Learning Guided Cross-Contract Fuzzing
    IEEE Transactions on Dependable and Secure Computing, 2022 (TDSC, CORE Rank A*, Impact factor 7.329)
  23. Felix Juefei-Xu, Run Wang, Yihao Huang, Qing Guo, Lei Ma, Yang Liu
    Countering Malicious DeepFakes: Survey, Battleground, and Horizon
    International Journal of Computer Vision, Springer, 2022 (IJCV, CORE rank A*, Impact Factor: 11.81)
  24. Jiayang Song, Deyun Lyu, Zhenya Zhang, Zhijie Wang, Tianyi Zhang, Lei Ma
    When Cyber-Physical Systems Meet AI: A Benchmark, an Evaluation, and a Way Forward
    The 44th International Conference on Software Engineering, SEIP Track (ICSE 2022, CORE Rank A*)
  25. Qiang Hu, Yuejun Guo, Maxime Cordy, Xiaofei Xie, Lei Ma, Mike Papadakis, Yves Le Traon
    An Empirical Study on Data Distribution-Aware Test Selection for Deep Learning Enhancement
    ACM Transactions on Software Engineering and Methodology (TOSEM 2022, CORE Rank A*, Impact Factor 2.674)
  26. Lizhi Liao, Heng Li, Weiyi Shang, Lei Ma
    An Empirical Study of the Impact of Hyperparameter Tuning and Model Optimization on DNN
    ACM Transactions on Software Engineering and Methodology (TOSEM 2021, CORE Rank A*, Impact Factor 2.674)
  27. Huangzhao Zhang, Zhiyi Fu, Ge Li, Lei Ma, Zhehao Zhao, Hua'an Yang, Yizhe Sun, Yang Liu, and Zhi Jin
    Towards Robustness of Deep Program Processing Models -- Detection, Estimation and Enhancement
    ACM Transactions on Software Engineering and Methodology (TOSEM 2021, CORE Rank A*, Impact Factor 2.674)
  28. Xiaofei Xie, Tianlin Li, Jian Wang, Lei Ma, Qing Guo, Felix Juefei-Xu, Yang Liu
    NPC: Neuron Path Coverage via Characterizing Decision Logic of Deep Neural Networks
    ACM Transactions on Software Engineering and Methodology (TOSEM 2021, CORE Rank A*, Impact Factor 2.674)
  29. Bing Yu, Hua Qi, Qing Guo, Felix Juefei-Xu, Xiaofei Xie, Lei Ma, Jianjun Zhao
    DeepRepair: Style-Guided Repairing for Deep Neural Networks in the Real-World Operational Environment
    IEEE Transactions on Reliability (Impact Factor 3.177)
  30. Qing Guo, Ziyi Cheng, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Yang Liu, and Jianjun Zhao
    Learning to Adversarially Blur Visual Object Tracking
    International Conference on Computer Vision, Montreal, Canada, 2021 (ICCV 2021, CORE Rank A*)
  31. Yihao Huang, Qing Guo, Felix Juefei-Xu, Lei Ma, Weikai Miao, Yang Liu, Geguang Pu
    Predictive Perturbation-aware Filtering against Adversarial Attack via Multi-domain Learning
    The 29th ACM International Conference on Multimedia (ACM MM 2021, CORE Rank A*)
  32. Jiaming Ye, Ke Chen, Xiaofei Xie, Lei Ma, Ruochen Huang, Yingfeng Chen, Yinxing Xue, Jianjun Zhao
    An Empirical Study of GUI Widget Detection for Industrial Mobile Games
    The ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Industry Track (ESEC/FSE 2021, CORE Rank A*)
  33. Xiaofei Xie, Wenbo Guo, Lei Ma, Wei Le, Jian Wang, Lingjun Zhou, Yang Liu, Xinyu Xing
    Automatic RNN Repair via Model-based Analysis
    The 38th International Conference on Machine Learning, 2021 (ICML 2021, CORE Rank A*)
  34. Zhenya Zhang, Deyun Lyu, Paolo Arcaini, Lei Ma, Ichiro Hasuo and Jianjun Zhao
    Effective Hybrid System Falsification Using Monte Carlo Tree Search Guided by QB-Robustness
    The 33rd International Conference on Computer-Aided Verification, 2021 (CAV 2021, CORE Rank A*)
  35. Alvin Chan, Lei Ma, Felix Juefei-Xu, Xiaofei Xie, Yang Liu, Yew Soon Ong
    Breaking Neural Reasoning Architectures with Metamorphic Relation-Based Adversarial Examples
    IEEE Transactions on Neural Networks and Learning Systems, 2021 (TNNLS, Impact Factor: 11.683)
  36. Xiaolu Hou, Jakub Breier, Dirmanto Jap, Lei Ma, Shivam Bhasin, Yang Liu
    Physical Security of Deep Learning on Edge Devices: Comprehensive Evaluation of Fault Injection Attack Vectors
    Elsevier Microelectronics Reliability Journal, 2021 (Impact Factor: 1.535)
  37. Yan Zheng, Yi Liu, Xiaofei Xie, Yepang Liu, Lei Ma, Jianye Hao, and Yang Liu
    Automatic Web Testing using Curiosity-Driven Reinforcement Learning
    The 43rd International Conference on Software Engineering (ICSE 2021, CORE Rank A*)
  38. Maryam V. Pour, Li Zhuo, Lei Ma and Hadi Hemmati
    A Search-Based Testing Framework for Deep Neural Networks of Source Code Embedding
    IEEE International Conference on Software Testing, Verification and Validation (ICST 2021, CORE Rank A)
  39. Xiyue Zhang, Xiaoning Du, Xiaofei Xie, Lei Ma, Yang Liu, Meng Sun
    Decision-Guided Weighted Automata Extraction from Recurrent Neural Networks
    Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI 2021, CORE Rank A*)
  40. Qing Guo, Jingyang Sun, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Wei Feng, Yang Liu, and Jianjun Zhao
    EfficientDeRain: Learning Pixel-wise Dilation Filtering for High-Efficiency Single-Image Deraining
    Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI 2021, CORE Rank A*)
  41. Haijun Wang, Yi Li, Shang-Wei Lin, Cyrille Artho, Lei Ma, and Yang Liu.
    Oracle- Supported Dynamic Exploit Generation for Smart Contracts
    IEEE Transactions on Dependable and Secure Computing (TDSC 2020, Impact factor 6.864)
  42. Qing Gu, Felix Juefei-Xu, Xiaofei Xie, Lei Ma, Jian Wang, Bing Yu, Wei Feng, Yang Liu
    Watch out! Motion is Blurring the Vision of Your Deep Neural Networks
    Thirty-fourth Conference on Neural Information Processing Systems (NeurIPS 2020, CORE Rank A*)
  43. Zi Peng, Jinqiu Yang, Tse-Hsun Chen, Lei Ma
    A First Look at the Integration of Machine Learning Models in Complex Autonomous Driving Systems - A Case Study on Apollo
    The 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Industry Track (ESEC/FSE 2020, CORE Rank A*/CCF A)
  44. Xiaoning Du, Yi Li, Xiaofei Xie, Lei Ma, Yang Liu, Jianjun Zhao
    MARBLE: Model-Based Robustness Analysis of Stateful Deep Learning Systems
    The 35th IEEE/ACM International Conference on Automated Software Engineering. (ASE 2020, CORE Rank A/CCF A)
  45. David Berend, Xiaofei Xie, Lei Ma, Lingjun Zhou, Yang Liu, Chi Xu, Jianjun Zhao
    Cats Are Not Fish: Deep Learning Testing Calls for Out-Of-Distribution Awareness
    The 35th IEEE/ACM International Conference on Automated Software Engineering. (ASE 2020, CORE Rank A/CCF A)
  46. Xuhong Ren, Bing Yu, Hua Qi, Felix Juefei-Xu, Zhuo Li, Wanli Xue, Lei Ma and Jianjun Zhao
    Few-Shot Guided Mix for DNN Repairing
    The 36th IEEE International Conference on Software Maintenance and Evolution, NIER Track (ICSME 2020, CORE Rank A)
  47. Yuechen Wu, Yingfeng Chen, Xiaofei Xie, Bing Yu, Changjie Fan and Lei Ma
    Regression Testing of Massively Multiplayer Online Role-Playing Games
    The 36th IEEE International Conference on Software Maintenance and Evolution, NIER Track (ICSME 2020, CORE Rank A)
  48. Hua Qi, Qing Guo, Felix Juefei-Xu, Xiaofei Xie, Lei Ma, Wei Feng, Yang Liu, Jianjun Zhao
    DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms
    The 28th ACM International Conference on Multimedia. (ACM MM 2020, CORE Rank A*/CCF A)
  49. Run Wang,Felix Juefei-Xu, Yihao Huang, Qing Guo, Xiaofei Xie, Lei Ma, Yang Liu
    DeepSonar: Towards Effective and Robust Detection of AI-Synthesized Fake Voices
    The 28th ACM International Conference on Multimedia. (ACM MM 2020, CORE Rank A*/CCF A)
  50. Run Wang,Felix Juefei-Xu, Yihao Huang, Qing Guo, Xiaofei Xie, Lei Ma, Yang Liu
    Amora: Black-box Adversarial Morphing Attack
    The 28th ACM International Conference on Multimedia. (ACM MM 2020, CORE Rank A*/CCF A)
  51. Qing Guo, Xiaofei Xie, Felix Juefei-Xu, Lei Ma, Zhongguo Li, Wanli Xue, Wei Feng, Yang Liu
    SPARK: Spatial-aware Online Incremental Attack Against Visual Tracking
    The 16th European Conference on Computer Vision (ECCV 2020, CORE Rank A)
  52. Run Wang, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Yihao Huang, Jian Wang, Yang Liu
    FakeSpotter: A Simple Baseline for Spotting AI-Synthesized Fake Faces
    The 30th International Joint Conference on Artificial Intelligence - Paciffic Rim International Conference on Artificial Intelligence, 7 pages, Japan, July 2020. (IJCAI 2020, CORE Rank A*/CCF A) (Media Coverage by Synced AI Industry Review)
  53. Xiyue Zhang, Xiaofei Xie, Lei Ma, Xiaoning Du, Qiang Hu, Yang Liu, Jianjun Zhao, Meng Sun
    Towards Characterizing Adversarial Defects of Deep Learning Software from the Lens of Uncertainty
    The 42nd International Conference on Software Engineering, 12 pages, 23-29 May 2020, Seoul, South Korea (ICSE’20, CORE Rank A*/CCF A)
  54. Jie M. Zhang, Mark Harman, Lei Ma, Yang Liu
    Machine Learning Testing: Survey, Landscapes and Horizons
    IEEE Transactions on Software Engineering, 2020. (TSE, CCF Rank A, SCI-indexed, Impact Factor:4.778)
  55. Huangzhao Zhang, Zhuo Li, Ge Li, Lei Ma, Yang Liu, Zhi Jin
    Generating Adversarial Examples for Holding Robustness of Source Code Processing Models
    The 34th AAAI Conference on Artificial Intelligence, New York, USA, Feb 7-12, 2020. (AAAI'20, CCF Rank A)
  56. Jianwen Sun, Tianwei Zhang, Xiaofei Xie, Lei Ma, Yan Zheng, Kangjie Chen, Yang Liu
    Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning.
    The 34th AAAI Conference on Artificial Intelligence, New York, USA, Feb 7-12, 2020. (AAAI'20, CCF Rank A)
  57. Li Li, Jun Gao, Tegawendé F. Bissyandé, Lei Ma, Xin Xia and Jacques Klein
    CDA: Characterising Deprecated Android APIs
    Empirical Software Engineering Journal, 2020 (EMSE, SCI-indexed, Impact Factor: 4.457)
  58. Cyrille Artho, Kazuuki Banzai, Quentin Gros, Guillaume Rousset, Lei Ma, Takashi Kitamura, Masami Hagiya, Yoshinori Tanabe and Mitsuharu Yamamoto
    Model-based Testing of Apache ZooKeeper: Fundamental API Usage and Watchers
    Software Testing, Verification and Reliability, 2019 (STVR, SCI-indexed, Impact Factor: 1.171)
  59. Yan Zheng, Xiaofei Xie, Ting Su, Lei Ma, Jianye Hao, Zhaopeng Meng, Yang Liu, Ruimin Shen, Yinfeng Chen, Changjie Fan
    Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement Learning
    ACM SIGSOFT Distinguished Paper Award.
    In Proc. 34th IEEE/ACM Conference on Automated Software Engineering (ASE 2019), San Diego, California, USA, November 11-15, 2019. (ASE'19, CCF Rank A)
  60. Qianyu Guo, Sen Chen, Xiaofei Xie, Lei Ma, Qiang Hu, Hongtao Liu, Yang Liu, Jianjun Zhao, Xiaohong Li
    An Empirical Study towards Characterizing Deep Learning Development and Deployment across Different Frameworks and Platforms.
    In Proc. 34th IEEE/ACM Conference on Automated Software Engineering (ASE 2019), San Diego, California, USA, November 11-15, 2019. (ASE'19, CCF Rank A)
  61. Qiang Hu, Lei Ma, Xiaofei Xie, Bing Yu, Yang Liu, and Jianjun Zhao
    DeepMutation++: a Mutation Testing Framework for Deep Learning Systems
    In Proc. 34th IEEE/ACM Conference on Automated Software Engineering (ASE 2019), (Tool Demo Track), San Diego, California, USA, November 11-15, 2019. (ASE'19, CCF Rank A)
  62. Xiaoning Du, Xiaofei Xie, Yi Li, Lei Ma, Yang Liu, and Jianjun Zhao.
    A Quantitative Analysis Framework for Recurrent Neural Network.
    In Proc. 34th IEEE/ACM Conference on Automated Software Engineering (ASE 2019), (Tool Demo Track), San Diego, California, USA, November 11-15, 2019. (ASE'19, CCF Rank A)
  63. Xiaofei Xie, Hongxu Chen, Yi Li, Lei Ma, Yang Liu, and Jianjun Zhao
    Coverage-guided Fuzzing for Feedforward Neural Networks
    In Proc. 34th IEEE/ACM Conference on Automated Software Engineering (ASE 2019), (Tool Demo Track), San Diego, California, USA, November 11-15, 2019. (ASE'19, CCF Rank A)
  64. Tianyi Zhang, Cuiyun Gao, Lei Ma, Michael R. Lyu and Miryung Kim
    An Empirical Study of Common Challenges in Developing Deep Learning Applications
    The 30th International Symposium on Software Reliability Engineering (ISSRE'19, CORE Rank A)
  65. Bai Xue, Yang Liu, Lei Ma, Xiyue Zhang, Meng Sun and Xiaofei Xie
    Safe Inputs Generation for Black-box Systems
    The 24th International Conference on Engineering of Complex Computer Systems (ICECCS'19, CORE Rank A)
  66. Ruitao Feng, Sen Chen, Xiaofei Xie, Lei Ma, Shangwei Lin and Yang Liu
    MobiDroid: A Performance-Sensitive Malware Detection System on Mobile Platform
    The 24th International Conference on Engineering of Complex Computer Systems (ICECCS'19, CORE Rank A)
  67. Xiaofei Xie, Lei Ma, Haijun Wang, Yuekang Li, Yang Liu, Xiaohong Li
    DiffChaser: Detecting Disagreements for Deep Neural Networks
    The 29th International Joint Conference on Artificial Intelligence, Macau, China, August 2019 (IJCAI'19, CCF Rank A)
  68. Xiaoning Du, Xiaofei Xie, Yi Li, Lei Ma, Yang Liu and Jianjun Zhao.
    DeepStellar: Model-Based Quantitative Analysis of Stateful Deep Learning Systems
    The 27th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Tallinn, Estonia, August 2019 (FSE'19, CCF Rank A)
  69. Xiaofei Xie, Lei Ma, Felix Juefei-Xu, Minhui Xue, Hongxu Chen, Yang Liu, Jianjun Zhao, Bo Li, Jianxiong Yin, Simon See
    DeepHunter: A Coverage-Guided Fuzz Testing Framework for Deep Neural Networks
    The 28th International Symposium on Software Testing and Analysis, Beijing, China, July, 2019 (ISSTA'19, CCF Rank A)
  70. Haijun Wang, Yi Li, Shang-Wei Lin, Lei Ma, Yang Liu
    VULTRON: Catching Vulnerable Smart Contracts Once and for All
    The 41st ACM/IEEE International Conference on Software Engineering, Montreal, QC, Canada, May, 2019, (ICSE'19, CCF Rank A)
  71. Chao Xie, Hua Qi,Lei Ma, Jianjun Zhao
    DeepVisual: A Visual Programming Tool for Deep Learning Systems
    The 27th IEEE/ACM International Conference on Program Comprehension, Montreal, QC, Canada, May, 2019, (ICPC'19, CCF Rank B)
  72. Lei Ma, Felix Juefei-Xu, Minhui Xue, Bo Li, Li Li, Yang Liu, Jianjun Zhao
    DeepCT: Tomographic Combinatorial Testing for Deep Learning Systems
    2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering, pp. 614-618, Hangzhou, China, 2019 (SANER'19, CCF Rank B)
  73. Yanjie Zhao, Haoyu Wang, Lei Ma, Yuxin Liu, Li Li, John Grundy
    Knowledge Graphing Git Repositories: A Preliminary Study
    2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering, pp. 599-603, Hangzhou, China, 2019 (SANER'19, CCF Rank B)
  74. Sen Chen, Minhui Xue, Lingling Fan, Lei Ma,, Yang Liu, Lihua Xu
    How Can We Craft Large-Scale Android Malware? An Automated Poisoning Attack.
    2019 IEEE International Workshop on Artificial Intelligence for Mobile, pp. 21-24, Hangzhou, China, 2019. (AI4Mobile'19)
  75. Sen Chen, Lingling Fan, Ting Su, Lei Ma,, Yang Liu, Lihua Xu
    Automated Cross-Platform GUI Code Generation for Mobile Apps.
    IEEE International Workshop on Artificial Intelligence for Mobile, pp. 13-16, Hangzhou, China, 2019. (AI4Mobile'19)
  76. Jakub Breier, Xiaolu Hou, Dirmanto Jap, Lei Ma, Shivam Bhasin, Yang Liu
    Practical Fault Attack on Deep Neural Networks.
    The 25th ACM Conference on Computer and Communications Security, Poster Paper (CCS'18 CCF Rank A)
  77. Lei Ma, Felix Juefei-Xu, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Chunyang Chen, Ting Su, Li Li, Yang Liu, Jianjun Zhao, Yadong Wang
    DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems. ACM SIGSOFT Distinguished Paper Award.
    The 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE'18, CORE Rank A)
  78. Lei Ma, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Felix Juefei-Xu, Chao Xie, Li Li, Yang Liu, Jianjun Zhao, Yadong Wang
    DeepMutation: Mutation Testing of Deep Learning Systems.
    The 29th IEEE International Symposium on Software Reliability Engineering (ISSRE'18 CORE Rank A)
  79. Alvin Chan, Lei Ma, Felix Juefei-Xu, Xiaofei Xie, Yang Liu, Yew Soon Ong
    Metamorphic Relation Based Adversarial Attacks on Differentiable Neural Computer.
    arXiv:1809.02444
  80. Qiang Hu, Lei Ma, Jianjun Zhao
    DeepGraph: A PyCharm Tool for Visualizing and Understanding Deep Learning Models.
    The 25th Asia-Pacific Software Engineering Conference (APSEC'18 CORE Rank B)
  81. Li Li, Jun Gao, Tegawendé F. Bissyandé, Lei Ma, Xin Xia and Jacques Klein
    Characterising Deprecated Android APIs. ACM FOSS Impact Paper Award.
    The 15th International Conference on Mining Software Repositories, Gothenburg, Sweden (MSR'18, CORE Rank A)
  82. Cyrille Valentin Artho, Quentin Gros, Guillaume Rousset, Kazuaki Banzai, Lei Ma, Takashi Kitamura, Masami Hagiya, Yoshinori Tanabe and Mitsuharu Yamamoto
    Model-based API Testing of Apache ZooKeeper.IEEE Best Presentation Award.
    The 10th IEEE International Conference on Software Testing, Verification and Validation, Tokyo, Japan (ICST'17, CORE Rank A)
  83. Lei Ma,Bing Yu, Hiroyuki Sato and Yadong Wang
    Collaborative Computation Offloading in Heterogeneous Asynchronous Cloud Environment.
    2017 IEEE 41st Annual Computer Software and Applications Conference (COMPSAC'17, CORE Rank B)
  84. Yoriyuki Yamagata, Cyrille Artho, Masami Hagiya, Jun Inoue, Lei Ma, Yoshinori Tanabe, Mitsuharu Yamamoto
    Runtime monitoring for concurrent systems.
    The 16th International Conference in Runtime Verification, Madrid, Spain, September 2016, Springer.(RV'16, CORE Rank C)
  85. Lei Ma, Cheng Zhang, Bing Yu, Hiroyuki Sato
    An empirical study on the effects of code visibility on program testability.
    Software Quality Journal, 2016, Springer.(SQJ, SCI-Indexed, Impact Factor: 2.141)
  86. Lei Ma, Cheng Zhang, Bing Yu, Jianjun Zhao
    Retrofitting Automatic Testing through Library Tests Reusing.
    The IEEE 24th International Conference on Program Comprehension, May 2016, Austin, USA. (ICPC'16, CCF B)
  87. Lei Ma, Cyrille Artho, Masami Hagiya, Yoshinori Tanabe, Yoriyuki Yamagata, Alexander Kohan and Mitsuharu Yamamoto
    Monitoring Distributed Applications with Java Pathfinder.
    2016 JPF Workshop 2016, November 2016, Seattle, USA. (JPF Workshop'16)
  88. Alexander Kohan, Mitsuharu Yamamoto, Cyrille Artho, Yoriyuki Yamagata, Lei Ma, Masami Hagiya and Yoshinori Tanabe
    Java Pathfinder on Android Devices.
    2016 JPF Workshop 2016, November 2016, Seattle, USA. (JPF Workshop'16)
  89. Jiabin Ye, Cheng Zhang, Lei Ma, Haibo Yu, Jianjun Zhao
    Efficient and Precise Dynamic Slicing for Client-Side JavaScript Programs. IEEE Best Paper Candidate Award.
    The IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering, March 2016, Osaka, Japan. (SANER'16, CCF B)
  90. Cyrille Artho, Lei Ma
    Classification of Randomly Generated Test Cases.
    The first IEEE International Workshop on Validating Software Tests, March, 2016, Osaka, Japan. (VST'16 colocated with SANER'16)
  91. Lei Ma, Cyrille Artho, Cheng Zhang, Hiroyuki Sato, Johannes Gmeiner, Rudolf Ramler
    GRT: Program-analysis-guided random testing. ACM SIGSOFT Distinguished Paper Award.
    The 30th IEEE/ACM International Conference on Automated Software Engineering, November 2015, Lincoln, USA. (ASE'15, CCF A)
  92. Lei Ma, Cyrille Artho, Cheng Zhang, Hiroyuki Sato, Johannes Gmeiner, Rudolf Ramler
    GRT: An automated test generator using orchestrated program analysis.
    The 30th IEEE/ACM International Conference on Automated Software Engineering, November 2015, Lincoln, USA. (ASE'15, CCF A)
  93. Lei Ma, Cheng Zhang, Bing Yu, Hiroyuki Sato
    An empirical study on effects of code visibility on code coverage of software testing.
    The IEEE/ACM 10th International Workshop on Automation of Software Test, May 2015, Florence, Italy. (AST'15, CORE Rank C)
  94. Lei Ma, Cyrille Artho, Cheng Zhang, Hiroyuki Sato,Masami Hagiya, Yoshinori Tanabe, Mitsuharu Yamamoto
    GRT at the SBST 2015 tool competition. IEEE Best Testing Tool Award.
    The IEEE 8th International Workshop on Search-based Software Testing, May 2015, Florence, Italy.
  95. Bing Yu, Lei Ma, Cheng Zhang
    Incremental Web Application Testing using Page Object.IEEE Best Paper Award.
    The IEEE International Workshop on Hot Topics in Web Systems and Technologies, November 2015, Washington, D.C, USA. (HotWeb'15, CORE Rank B)
  96. Lei Ma, Cyrille Artho, Cheng Zhang, and Hiroyuki Sato
    Efficient testing of software product lines via centralization.
    The ACM International Conference on Generative Programming: Concepts and Experiences, September 2014, Vasteras, Sweden. (GPCE'14, CORE Rank B)
  97. Lei Ma, Cyrille Artho, Cheng Zhang, and Hiroyuki Sato
    When Project Centralization and Random Testing Meet: Efficient Automatic Testing of Multiple Software Product Variants.
    The 2014 JSSST Software Conference, July 2014, Nagoya, Japan. (JSSST'14)
  98. Lei Ma, Cyrille Artho, and Hiroyuki Sato
    Managing Product Variants by Project Centralization.
    Lecture Notes on Software Engineering (LNSE), January 2014.
  99. Lei Ma, Cyrille Artho, and Hiroyuki Sato
    Project centralization based on graph coloring.
    The ACM 29th International Symposium on Applied Computing, March 2014, Gyeongju, South Korea. (SAC'14, CORE Rank B)
  100. Lei Ma, Cyrille Artho, and Hiroyuki Sato
    Improving automatic centralization by version separation.
    IPSJ Transactions on Programming, 6(4), December 2013.
  101. Kentaro Ishii, Haipeng Mi, Lei Ma, Natsuda Laokulrat, Masahiko Inami, Takeo Igarashi
    Pebbles: Tangible Devices for Designing Robot Navigation Routes.
    Computer Software Journal, August 2013.
  102. Lei Ma, Cyrille Artho, Hiroyuki Sato
    Analyzing distributed Java applications by automatic centralization.
    The IEEE 37th Annual Computer Software and Applications Conference Workshops, July 2013, Kyoto, Japan.(COMPSACW'13)
  103. Kentaro Ishii, Haipeng Mi, Lei Ma, Natsuda Laokulrat, Masahiko Inami, Takeo Igarashi
    Pebbles: User-Configurable Device Network for Robot Navigation.
    The 14th IFIP International Conference on Human-Computer Interaction, September 2013, Cape Town, South Africa. (INTERACT'13, CORE Rank A)
  104. Kentaro Ishii, Haipeng Mi, Lei Ma, Natsuda Laokulrat, Masahiko Inami, Takeo Igarashi
    Pebbles: an interactive configuration tool for indoor robot navigation.
    The 25th Annual ACM Symposium on User Interface Software and Technology, October 2012, Cambridge, MA, USA. (UIST'12, CORE Rank A)

  • Tutorial on AI Quality Assurance
    collocated with the 37th IEEE/ACM International Conference on Automated Software Engineering (ASE), Oct., 2022
  • Quality Assurance for Data-driving Intelligent System Engineering
    The CCF Advanced Disciplines Lectures (ADL), July 31, 2022
  • Towards Quality and Reliability Engineering in the era of Data-Driven Complex AI Systems
    The University of Tokyo, July 20, 2022
  • AI Debugging and Testing Tutorial
    Machine Learning System Engineering Summer Training Camp, Tokyo, July 1, 2022
  • Towards Quality and Reliability Engineering for Data-Driven AI Systems
    Zhejiang University, Dec. 29, 2021
  • An Orchestrated Reflection of Quality Assurance in the era of Data-Driven Intelligent Systems
    The 15th International Conference on Verification and Evaluation of Computer and Communication Systems (VECoS 2021), Virtual, Nov. 22, 2021
  • Quality Assurance and Engineering in the era of Data-driven Complex AI Systems
    Shanghai Jiaotong University, Oct. 18, 2021
  • Towards Testing and Analysis of Data-driven AI-enabled Software Systems
    Guest Lecture at The Australian National University, Virtual, Sep 22, 2021
  • Quality Assurance in the era of Data-driven Complex Intelligent Systems
    Machine Learning System Engineering Summer Training Camp, Virtual, July 1, 2021
  • Quality Assurance for Data-driven Intelligent Systems from Testing, Analysis to Repairing
    JST Mirai eAI Project Colloquium, Virtual, June 11, 2021
  • Challenges and Opportunities of Software Engineering in the era of Data-Driven Intelligent Systems
    AIST Software Engineering workshop, Virtual, March 26, 2021
  • Reliable and Trustworthy AI Systems In the Data-Driven Era
    DIY AI Technical Talk, ISA Edmonton Section, Virtual, March 15, 2021
  • Explainability-driven Analysis in the era of Data-Driven Intelligent Systems
    The 3rd International Workshop on Intelligent Bug Fixing, Virtual, March 9, 2021
  • Testing and Analysis Towards Reliable Deep Learning Systems
    Nvidia Corporation, Virtual, Nov.19, 2020
  • Testing and Analysis of Deep Learning Systems
    University of Science and Technology of China, Virtual, Nov.13, 2020
  • Testing and Analysis of Deep Learning Systems
    Tsinghua University, Virtual, Oct.14, 2020
  • Towards Characterizing Adversarial Defects of Deep Learning Software from the Lens of Uncertainty
    The 37th JSSST Annual Conference, Virtual, Sep.8, 2020
  • Testing and Analysis of Stateful Deep Learning Systems
    Institute for Data Valorisation (IVADO) Training Program on the Software Engineering of Machine Learning-based Systems, Montreal, Canada, March 3-6, 2020
  • Automatic Video Game Testing via Evolutionary Deep Reinforcement Learning
    Polytechnique Montreal, Canada, Feb.6, 2020
  • Quality and Reliability Assurance for Deep Learning Systems
    School of Computer Science, McGill University, Canada, Feb.5, 2020
  • Towards Analyzing Stateful Deep Learning System
    University of British Columbia, Canada, Dec.13, 2019
  • Testing and Analysis of Deep Learning Systems
    National Software Application Conference of China (NASAC'19), Nov. 24, Hangzhou, China, 2019
  • Quality and Reliability for Machine Learning Systems
    Celebration of ASE 2019, 10 Nov. San Diego, California, USA, 2019
  • DeepStellar: Analysis of Stateful Deep Learning Systems
    JST ERATO MMSD (Metamathematics for System Design) Project, Tokyo, Japan, 7 Nov. 2019
  • Towards Quality and Reliability Engineering of Deep Learning Systems
    The 8th Asian Workshop on Advanced Software Engineering (AWASE 2019), Fukuoka, Japan, 28-29 Oct., 2019
  • Towards Testing and Analysis of Deep Learning Systems
    MLSE International Symposium 2019, Tokyo, Japan, 18 Oct., 2019
  • Towards Testing Deep Learning Systems
    Shonan Meeting SEMLS (Software Engineering for Machine Learning Systems), Shonan Village, Japan, 13-17 Oct., 2019
  • Quality Assurance of Deep Learning Engineering
    NAIST Data Science Day, Nara, Japan, 11 Oct., 2019
  • Quality Assurance of Machine Learning Systems
    Intelligent Software Testing Symposium, Nanjing, China, 17 Sep., 2019
  • Testing Deep Learning System Towards Intelligence of Things
    Google Waterloo, Canada, June 5, 2019
  • Towards Quality Assurance of Deep Learning Systems
    The Software Engineering for Machine Learning Applications (SEMLA) International Symposium, Polytechnique Montreal, Canada, May 23 – 24, 2019
  • Software Testing of Deep Learning System: from the Past to the Future
    Japan Machine Learning Systems Engineering Community (MLSE, 日本機械学習工学研究会), Tokyo, Japan, April 26, 2019
  • DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems
    2018 National Software Application Conference of China (NASAC'18), Nov. 23, ShenZhen, China, 2018
  • Quality Assurance for Deep Learning System
    KTH Royal Institute of Technology, Stockholm, Sweden, Sep. 13, 2018
  • Deep Learning Quality Assurance in Industrial Scale
    2018 Nvidia NVAITC AI Symposium, Singapore, August 30, 2018
  • Program Analysis Guided Intelligent Automated Testing
    2017 National Software Testing Competition Symposium, Nanjing, China, October 25, 2017
  • A Comprehensive Study on Automated Testing
    National Institute of Advanced Industrial Science and Technology (AIST), Osaka, Japan, March 24, 2017
  • Program Analysis Guided Software Verification and Testing
    National Institute of Advanced Industrial Science and Technology (AIST), Osaka, Japan, March 13, 2016
  • Guided Random Testing: When Program Analysis Meets Automatic Test Generation
    Static Analysis Meets Runtime Verification NII Shonan Meeting, Shonan Village Center, Japan, March 15-19, 2015

Standards and Journals


  • Guest Editor, Computers & Security, ELSEVIER
    Special Issue on "Adversarial Machine Learning towards Advanced Intelligent Systems (Impact Factor: 5.105) (Open until April 2023)
  • Task Force Members, IEEE CIS Neural Networks on Secure Learning
  • Member of Standards Council of Canada, ISO/IEC JTC 1/SC 42 Artificial intelligence
  • Advisory Board Member, Empirical Software Engineering Journal, Springer (EMSE)
  • Review Board Member, Automated Software Engineering Journal, Springer (AUSE)
  • Guest Editor, IEEE Transactions on Reliability (ToR)
  • Reviewer, IEEE Transactions on Software Engineering (TSE)
  • Reviewer, ACM Transactions on Software Engineering and Methodology (TOSEM)
  • Reviewer, Journal of Systems and Software (JSS)
  • Reviewer, The international journal of Software Testing, Verification and Reliability (STVR)
  • Reviewer, Journal of Software: Evolution and Process
  • Reviewer, Software and Systems Modeling, Springer
  • Reviewer, IEEE Transactions on Dependable and Secure Computing (TDSC)
  • Reviewer, Transactions on Machine Learning Research (TMLR)
  • Reviewer, IEEE Transactions on Neural Networks and Learning Systems (TNNLS)
  • Reviewer, IEEE Software
  • Reviewer, ACM Computing Surveys

Conference


2023

  • PC member, The 45th International Conference on Software Engineering (ICSE 2023)
  • PC member, The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2023)
  • PC member, The 38th IEEE/ACM International Conference on Automated Software Engineering (ASE 2023)
  • PC member, International Symposium on Formal Methods (FM 2023)
  • PC member, The 2nd International Conference on AI Engineering/Software Engineering for AI (CAIN 2023)
  • PC member, The International Workshop on Dependability of Safety-Critical Systems with Machine Learned Components (D-SyMLe 2023)
  • PC member, The 4th International Workshop on Testing for Deep Learning and Deep Learning for Testing (DeepTest 2023)
  • PC member, The 31st IEEE/ACM International Conference on Program Comprehension (ICPC 2023)
  • PC member, The Genetic and Evolutionary Computation Conference (GECCO 2023)
  • PC member, The 27th International Conference on Engineering of Complex Computer Systems (ICECCS 2023)
  • PC member, IEEE Computer Society Signature Conference on Computers, Software and Applications (COMPSAC 2023)
  • PC member, The 16th International Conference on Verification and Evaluation of Computer and Communication Systems (VECOS 2023)
  • PC Member, AAAI Spring Symposium on AI Trustworthiness Assessment (AITA 2023)
  • PC Member, The 32nd International Joint Conference on Artificial Intelligence (IJCAI 2023)
  • Reviewer, The IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2023) (CVPR 2023)
  • Reviewer, The Eleventh International Conference on Learning Representations (ICLR 2023)
  • Reviewer, The IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023 (CVPR 2023)
  • Reviewer,The 40th International Conference on Machine Learning (ICML 2023)

2022

  • Area Chair, Thirty-sixth Conference on Neural Information Processing Systems, Datasets and Benchmarks Track (NeurIPS 2022)
  • Mentor, Student Mentoring Workshop (SMeW @ ICSE 2022)
  • PC member, The 44th International Conference on Software Engineering (ICSE 2022)
  • PC Member, 15th ACM Workshop on Artificial Intelligence and Security (AISec 2022)
  • PC member, The 33rd IEEE International Symposium on Software Reliability Engineering (ISSRE 2022)
  • PC member, The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2022)
  • PC member, The 37th IEEE/ACM International Conference on Automated Software Engineering (ASE 2022)
  • PC member, The 38th IEEE International Conference on Software Maintenance and Evolution (ICSME 2022)
  • PC member,16th International Conference Verification and Evaluation of Computer and Communication Systems (VECoS 2022)
  • SPC member,Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI 2022)
  • PC member,The 31th International Joint Conference on Artificial Intelligence (IJCAI 2022)
  • Reviewer,The 39th International Conference on Machine Learning (ICML 2022)
  • Reviewer,IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2022)
  • Reviewer, Tenth International Conference on Learning Representations (ICLR 2022)
  • Reviewer,The European Conference on Computer Vision (ECCV 2022)

2021

  • PC Member, 14th ACM Workshop on Artificial Intelligence and Security (AISec 2021)
  • Reviewer, The Thirty-fifth Conference on Neural Information Processing Systems (NeurIPS 2021)
  • Reviewer, The 38th International Conference on Machine Learning (ICML 2021)
  • Reviewer, The International Conference on Computer Vision (ICCV 2021)
  • Reviewer, The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021)
  • Student Volunteers Chair & PC member of Tool Demo Track, The ACM/IEEE 24th International Conference on Model Driven Engineering Languages and Systems (MODELS 2021)
  • Organizing Committee, The Workshop on AI and Software Testing/Analysis (AISTA 2021)
  • PC member, The 3rd International Workshop on Testing for Deep Learning and Deep Learning for Testing (DeepTest 2021)
  • PC member, The 22nd Genetic and Evolutionary Computation Conference (GECCO 2021)
  • PC member, The 36th IEEE/ACM International Conference on Automated Software Engineering (ASE 2021)
  • PC member, The 29th IEEE/ACM International Conference on Program Comprehension (ICPC 2021)
  • PC member, The 45th IEEE Computer Society International Conference on Computers, Software & Applications (COMPSAC 2021)
  • PC member, The 21th IEEE International Conference on Software Quality, Reliabiilty, and Security & Applications (QRS'2021)
  • PC member, The 30th International Joint Conference on Artificial Intelligence (IJCAI 2021)
  • PC member, The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI 2021)
  • PC member, The ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2021)

2020

  • PC member, The 13th ACM Workshop on Artificial Intelligence and Security (AISec 2020)
  • Reviewer, The Thirty-fourth Conference on Neural Information Processing Systems (NeurIPS 2020)
  • PC member, The 35rd IEEE/ACM International Conference on Automated Software Engineering (ASE 2020)
  • PC member, The 36th IEEE International Conference on Software Maintenance and Evolution (ICSME’20)
  • PC member, The 21st Genetic and Evolutionary Computation Conference (GECCO 2020)
  • PC member, This 27th IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER 2020)
  • Workshop Chair, The 27th Asia-Pacific Software Engineering Conference (APSEC 2020)
  • Doctoral Symposium Co-Chair, THE 22nd International Conference on Formal Engineering Methods (ICFEM 2020)
  • Publicity and Social Media Chair, The 12th Asia-Pacific Symposium on Internetware (Internetware 2020)
  • PC member, The 25th International Conference on Engineering of Complex Computer Systems (ICECCS 2020)
  • PC member, The 8th International Workshop on Realizing Artificial Intelligence Synergies in Software Engineering (RAISE 2020)
  • PC member, The IEEE Intl. Conf. on Softw.Testing, Verification and Validation, Poster Track (ICST 2020)
  • PC member, The 14th International Symposium on Theoretical Aspects of Software Engineering (TASE 2020)
  • PC member, The 44th IEEE Computer Society International Conference on Computers, Software & Applications (COMPSAC 2020)
  • PC member, The 20th IEEE International Conference on Software Quality, Reliabiilty, and Security & Applications (QRS'2020)
  • PC member, The IEEE International Workshop on Intelligent Bug Fixing (IBF 2020)

    • 2019

    • PC member, The 34rd IEEE/ACM International Conference on Automated Software Engineering (ASE'2019)
    • Organizing Comitte Member, The IEEE International Workshop on Machine Learning and Software Testing(MLST'2019)
    • Organizing Comitte Member, The IEEE International Workshop on Artificial Intelligence for Mobile (AI4Mobile)
    • Local Chair & PC member, The 11th Asia-Pacific Symposium on Internetware (Internetware'19)
    • Local Chair, The 8th Asia-Pacific Workshop of Advanced Software Engineering (AWASE'19)
    • PC Member, The 26th IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER'19 ERA)
    • PC member, The 19th IEEE International Conference on Software Quality, Reliabiilty, and Security(QRS'2019)
    • PC member, 19th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM'2019)
    • PC member, The 43th IEEE Computer Society International Conference on Computers, Software & Applications (COMPSAC'19)
    • PC Member, International Symposium on Computing and Networking (CANDAR'19)
    • PC member, The 24th International Conference on Engineering of Complex Computer Systems(ICECCS'19)
    • PC member, The First International Workshop on Intelligent Bug Fixing (IBF'2019))
    • PC member, The 42th IEEE Computer Society International Conference on Computers, Software & Applications (COMPSAC'19)
    • PC Member, IEEE International Conference on Mobile Cloud Computing, Services, and Engineering (Mobile Cloud'19)
    • PC Member, The 19th IEEE International Conference on Scalable Computing and Communications (ScalCom'19)
    • Reviewer, The 16th International Conference on Mining Software Repositories (MSR'2019)

    2018

    • Reviewer, The 11th ACM Workshop on Artificial Intelligence and Security (AISec'18)
    • Reviewer, IEEE Transactions on Reliability
    • Reviewer, The 15th International Conference on Managed Languages & Runtimes (ManLang'18)
    • Reviewer, The 25th ACM Conference on Computer and Communications Security (CCS'18)
    • Reviewer, The 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE'18)
    • PC member, The 42th IEEE Computer Society International Conference on Computers, Software & Applications (COMPSAC'18)
    • Reviewer, ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA'18)
    • Reviewer, ETAPS 24th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS'18)
    • PC Member, The 18th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM'18)
    • PC Member, IEEE International Conference on Mobile Cloud Computing, Services, and Engineering (Mobile Cloud'18)
    • PC Member, The 18th IEEE International Conference on Scalable Computing and Communications (ScalCom'18)
    • PC Member, IEEE International Workshop on Validating Software Tests (VST'18)
    • PC Member, The 26th European Conference On Information Systems (ECIS'18)
    • PC member, The Exellent Ph.D Research Forum (NASAC'2018)
    • Reviewer, Springer Journal of Computer Virology and Hacking Techniques
    • PC member, The 18th International Conference on Advances in System Testing and Validation Lifecycle (VALID'18)
    • PC Member, International Symposium on Computing and Networking (CANDAR'18)

    2017

    • Publicity Chair, The 24th International Conference on Engineering of Complex Computer Systems(ICECCS'17)
    • PC member, The 17th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM'17)
    • Reviewer, Elsevier Journal of Systems and Software (JSS)
    • PC member, The 41th IEEE Computer Society International Conference on Computers, Software & Applications (COMPSAC'17)
    • Publication Chair & PC Member, IEEE International Conference on Mobile Cloud Computing, Services, and Engineering (Mobile Cloud'17)
    • PC member, The Eighth International Conference on Advances in System Testing and Validation Lifecycle (VALID'17)
    • PC Member, International Symposium on Computing and Networking (CANDAR'17)
    • IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER): 2018, 2019

    2016 & 2015

    • External reviewer tool track, The 31st IEEE/ACM International Conference on Automated Software Engineering (ASE'16)
    • PC member, The Eighth International Conference on Advances in System Testing and Validation Lifecycle (VALID'16)
    • PC Member, IEEE International Workshop on Validating Software Tests (VST'16)
    • Publication Chair & PC Member, IEEE International Conference on Mobile Cloud Computing, Services, and Engineering (Mobile Cloud'16)
    • Technical reviewer, International Conference on Formal Aspects of Component Software (FACS'16)
    • Reviewer, Elsevier Science of Computer Programming Journal (FTSCS'16)
    • Tool Panel Member, International Workshop on Search-Based Software Testing (SBST'15)
  • Canada CIFAR AI Chair, Pan-Canadian AI Strategy (2021)
  • Fellow, Alberta Machine Intelligence Institute (Amii) (2021)
  • ACM SIGSOFT Distinguished Paper Award (ASE'19)
  • ACM SIGSOFT Distinguished Paper Award (ASE'18)
  • ACM SIGSOFT Distinguished Paper Award (ASE'15)
  • Research Prototype Competition Award (Second Prize, CCF NASAC'19)
  • 2018 Baidu-NASAC Academic Star Award
  • ACM FOSS Impact Paper Award (MSR'18)
  • Leadership Award (ICECCS'17)
  • IEEE Best Presentation Award (ICST'17)
  • IEEE Best Paper Candidate Award (SANER'16)
  • IEEE Best Paper Award (HotWeb'15)
  • IEEE Best Testing Tool Award (SBST'15)
  • JASSO Hornor scholoarship (2011 and 2013)
  • MEXT Scholarship sponsored by Japanese Government (2009)
  • Excellent Academic Scholarship of Shanghai Jiao Tong University (2006)