Unifying Speaker Matthew B. Dwyer
Matthew B. Dwyer is the Robert Thomson Distinguished Professor in the Department of Computer Science at the University of Virginia and an Amazon Scholar. He has authored more than 140 scholarly publications in program analysis, software specification, and automated formal methods. These research contributions have been recognized with five “test of time” (ICSE 2010, SIGSOFT 2010, FSE 2018, SIGSOFT 2021, ISSTA 2022). He was named a Fulbright Research Scholar (2011), an IEEE Fellow (2013), a Parnas Fellow (2018), and an ACM Fellow (2019), and has received the IEEE Computer Society Harlan D. Mills Award (2022).
Talk
Leveraging Abstractions for Validation of Machine Learning Models
Time: TBA
Room: TBA
Machine learning models have grown significantly in terms of their ability to process complex inputs and perform challenging inferences with accuracy that exceeds alternative methods. This has led developers to begin to consider deploying them as system components. Validating model behavior, however, especially to the standards one might expect in a mission or safety critical system, presents significant obstacles. Chief among these is the difficulty of describing correctness properties expressing constraints in the typically sparse, high-dimensional, and uninterpretable model input domain.
We describe recent work that leverages scene graphs that define graph abstractions of raw sensor inputs that include only salient detail, capture semantic features of the problem domain, and are amenable to interpretation and analysis. We describe how such abstractions can be used to define a coverage domain to judge the adequacy of validation, how specifications can be formulated over such abstractions, and how such specifications can be used to monitor model behavior during deployment, bias training to maximize specification conformance, and to drive targeted testing based on the specification.