Coverage Evaluation and Analysis of Post-Silicon Tests with Virtual Prototypes

Abstract

High-quality tests for post-silicon validation should be ready before a silicon device becomes available in order to save time spent on preparing, debugging, and fixing tests after the device is available. Thorough analysis of post-silicon validation tests should be conducted early even without a silicon device. We propose an approach to coverage evaluation and analysis of post-silicon validation tests. Test coverage is an important metric for evaluating the quality and readiness of post-silicon tests. We propose an online-capture offline-replay approach to coverage evaluation of post-silicon validation tests with virtual prototypes for estimating silicon device test coverage. We first capture necessary data from a concrete execution of the virtual prototype within a virtual platform under a given test, and then compute the test coverage by efficiently replaying this execution offline on the virtual prototype itself. Our approach provides early feedback on quality of post-silicon validation tests before silicon is ready. To ensure fidelity of early coverage evaluation, our approach has been further extended to support coverage evaluation and conformance checking in the post-silicon stage. Moreover, we propose a runtime analysis approach to monitor and analyze device behaviors triggered by post-silicon tests. We developed a shadow execution framework to execute a virtual prototype in a shadow environment while device behaviors can be well observed and debugged. Such shadow execution does not change normal execution flow; however, it provides capability for developers to observe any device transaction and state changes. We have applied our approach to evaluate a suite of common tests on virtual prototypes of five network adapters. Our approach was able to better help developers observe device behaviors triggered by the tests, and reliably estimate that this suite achieves high functional coverage on all five silicon devices.