The Vehicle Tutorial: Neural Network Verification with Vehicle

5 pages•Published: October 23, 2023

Matthew Daggitt, Wen Kokke, Ekaterina Komendantskaya, Robert Atkey, Luca Arnaboldi, Natalia Slusarz, Marco Casadio, Ben Coke and Jeonghyeon Lee

Abstract

Machine learning components, such as neural networks, gradually make their way into software; and, when the software is critically safe, the machine learning components must be verifiably safe. This gives rise to the problem of neural network verification. The community has been making rapid progress in developing methods for incorporating logical specifications into neural networks, both in training and verification. However, to truly unlock the ability to verify real-world neural network-enhanced systems we believe the following is necessary:
1. The specification should be written once and should automatically work with training and verification tools. 2. The specification should be written in a manner independent of any particular neural network training/inference platform. 3. The specification should be able to be written as a high-level property over the problem space, rather than a property over the input space (of the neural network). 4. After verification the specification should be exportable to general interactive theorem provers so that its proof can be incorporated into proofs about the larger systems around the neural network.
In this tutorial we presented Vehicle, a tool that allows users to do all of this. We provided an introduction to the Vehicle specification language, and then walked attendees through using it to express a variety of famous and not-so-famous specifications. Subsequently we demonstrate how a specification can be compiled down to i) queries for network verifiers, ii) Tensorflow graphs to be used as loss functions during training and iii) cross-compiled to Agda, a main-stream interactive theorem prover. Finally we discussed some of the technical challenges in the implementation as well as some of the outstanding problems.

Keyphrases: adversarial training, domain specific languages, neural network verification, programming languages, types

In: Nina Narodytska, Guy Amir, Guy Katz and Omri Isac (editors). Proceedings of the 6th Workshop on Formal Methods for ML-Enabled Autonomous Systems, vol 16, pages 1-5.

Links:	https://easychair.org/publications/paper/Rkrv
	https://doi.org/10.29007/5s2x

BibTeX entry

@inproceedings{FoMLAS2023:Vehicle_Tutorial_Neural_Network,
  author    = {Matthew Daggitt and Wen Kokke and Ekaterina Komendantskaya and Robert Atkey and Luca Arnaboldi and Natalia Slusarz and Marco Casadio and Ben Coke and Jeonghyeon Lee},
  title     = {The Vehicle Tutorial: Neural Network Verification with Vehicle},
  booktitle = {Proceedings of the 6th Workshop on Formal Methods for ML-Enabled Autonomous Systems},
  editor    = {Nina Narodytska and Guy Amir and Guy Katz and Omri Isac},
  series    = {Kalpa Publications in Computing},
  volume    = {16},
  publisher = {EasyChair},
  bibsource = {EasyChair, https://easychair.org},
  issn      = {2515-1762},
  url       = {/publications/paper/Rkrv},
  doi       = {10.29007/5s2x},
  pages     = {1-5},
  year      = {2023}}

Download PDF Open PDF in browser