How Generalizable is My Behavior Cloning Policy? A Statistical Approach to Trustworthy Performance Evaluation

Abstract

With the rise of stochastic generative models in robot policy learning, end-to-end visuomotor policies are increasingly successful at solving complex tasks by learning from human demonstrations. Nevertheless, since real-world evaluation costs afford users only a small number of policy rollouts, it remains a challenge to accurately gauge the performance of such policies. This is exacerbated by distribution shifts causing unpredictable changes in performance during deployment. To rigorously evaluate behavior cloning policies, we present a framework that provides a tight lower-bound on robot performance in an arbitrary environment, using a minimal number of experimental policy rollouts. Notably, by applying the standard stochastic ordering to robot performance distributions, we provide a worst-case bound on the entire distribution of performance (via bounds on the cumulative distribution function) for a given task. We build upon established results to ensure that the bounds hold with a user-specified confidence level and tightness, and are constructed from as few policy rollouts as possible. In experiments we evaluate policies for visuomotor manipulation in both simulation and hardware. Specifically, we i) empirically validate the guarantees of the bounds in simulated manipulation settings, ii) find the degree to which a learned policy deployed on hardware generalizes to new real-world environments, and iii) rigorously compare two policies tested out-of-distribution. Our data, code, and implementation of confidence bounds are open-source.

Video

BibTex

@article{nishimura2021ratilqr,
  author={Vincent, Joseph A. and Nishimura, Haruki and Itkina, Masha and Shah, Paarth and Schwager, Mac and Kollar, Thomas},
  journal={IEEE Robotics and Automation Letters}, 
  title={RAT iLQR: A Risk Auto-Tuning Controller to Optimally Account for Stochastic Model Mismatch}, 
  year={20224},
  volume={9},
  number={10},
  pages={8619-8626}
}