Automatic Verification of Iterated Separating Conjunctions Using Symbolic Execution

Abstract

In permission logics such as separation logic, the iterated separating conjunction is a quantifier denoting access permission to an unbounded set of heap locations. In contrast to recursive predicates, iterated separating conjunctions do not prescribe a structure on the locations they range over, and so do not restrict how to traverse and modify these locations. This flexibility is important for the verification of random-access data structures such as arrays and data structures that can be traversed in multiple ways such as graphs. Despite its usefulness, no automatic program verifier natively supports iterated separating conjunctions; they are especially difficult to incorporate into symbolic execution engines, the prevalent technique for building verifiers for these logics.

In this paper, we present the first symbolic execution technique to support general iterated separating conjunctions. We propose a novel representation of symbolic heaps and flexible support for logical specifications that quantify over heap locations. Our technique exhibits predictable and fast performance despite employing quantifiers at the SMT level, by carefully controlling quantifier instantiations. It is compatible with other features of permission logics such as fractional permissions, recursive predicates, and abstraction functions. Our technique is implemented as an extension of the Viper verification infrastructure.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Permission logics such as separation logic [18] and implicit dynamic frames [19] associate an access permission with each memory location in order to reason about shared mutable state. Dynamic heap data structures require specifications to denote access permissions to a statically-unknown set of locations. Such specifications are typically expressed in existing tools using recursive predicates [15], which work well so long as the traversal of the data structure matches the definition of the predicate. However, access patterns that do not follow the predicate structure (e.g., traversing a doubly-linked list from the end) or that follow no specific order (e.g., random access into an array) are difficult to handle in existing program verifiers, requiring programmers to provide substantial manual proof steps (for instance, as ghost code) to bridge the mismatch between the program’s access pattern and the imposed predicate structure.

Iterated separating conjunction [18] (hereafter, ISC) is an alternative way to denote properties of a set of heap locations, which has for instance been used in by-hand proofs to denote locations of arrays [18], cyclic data structures [3, 23], the objects stored in linked lists [7], and graph algorithms [23]. Unlike recursive predicates, an ISC does not prescribe any particular traversal order.

Despite its usefulness and inclusion in early presentations of separation logic, no existing program verifier supports general ISCs directly. Among the tools based on symbolic execution, Smallfoot [2] does not support ISC; VeriFast [22] and jStar [7] allow programmers to encode some forms of ISC via abstract predicates that can be manipulated by auxiliary operations and lemmas (in VeriFast) or tailored rewrite rules (in jStar). For arrays, this encoding is partially supported by libraries. However, in the general case, programmers need to provide the extra machinery, which significantly increases the necessary manual effort.

Among the verifiers based on verification condition generation, Chalice [12] supports only a restricted form of ISC (ranging over all objects stored in a sequence), and VeriCool uses an encoding that leads to unreliable behaviour of the SMT solver [21, p. 46]. The GRASShopper tool [16] does not provide built-in or general support for ISC, but some ingredients of the technique we present (particularly, the technical usage of inverse functions) have been employed there to specify particular random access to data structures (e.g., arrays). The Dafny verifier [10] can be used to write similar set and quantifier-based specifications, but does not support permission-based reasoning or concurrency.

In this paper, we present the first symbolic execution technique that directly supports general forms of ISC. Our technique is compatible with other features of permission logics: it supports fractional permissions [5], such that a heap location may be ranged over by several ISCs, and allows ISC to occur in predicate bodies and in preconditions of abstraction functions [8].

This combination of features allows one to specify and verify challenging examples such as graph-marking algorithms that so far were beyond the scope of automated verifiers based on permission logics.

Our main technical contributions are: (1) a novel representation of the partial heaps that are denoted by an ISC, along with algorithms to manipulate this representation; (2) a technique to preserve across heap changes (to frame) the values of expressions that depend on the unbounded set of heap locations denoted by ISCs; (3) an SMT encoding that carefully controls quantifier instantiations; (4) an implementation of our approach in the Viper verification infrastructure [14]. Our implementation and several interesting examples are available online [1].

Outline. In the next section, we explain the main technical challenges our work addresses, and illustrate them with a simple motivating example. Our design for a symbolic heap that can represent permissions described by ISCs is presented in Sect. 3. We explain the symbolic evaluation of expressions and framing with respect to this heap representation in Sect. 4. In Sect. 5, we discuss how we control quantifier instantiations. Section 6 presents an evaluation of our implementation. We conclude in Sect. 7.

2 Technical Challenges

Permission logics ensure that a heap location is accessed only when the corresponding permission is held. Dedicated assertions denote the permission to a heap location e.f, written as ${e.f}\mapsto {\_}$ in separation logic and as the accessibility predicate in implicit dynamic frames; we use the latter in this paper. These logics include a separating conjunction , expressing that the permissions denoted by the two conjuncts must be disjoint. For instance, an assertion implies the disequality $x\ne y$. Many permission logics allow permissions to be split into fractions, and to re-assemble fractions into a full permission. In these logics, any non-zero permission allows read access to a location, whereas write access requires the full permission. When appropriate permissions are held, assertions may also constrain the value of a heap location (for instance, $x.f > 3$); assertions that do not contain accessibility predicates are called pure. We use the terms pure assertion and expression synonymously.

Verification of many program constructs can be modelled by two basic operations. Inhaling an assertion A adds the permissions denoted by A to the current state and assumes the pure assertions in A. Exhaling an assertion A checks that the current state satisfies the pure assertions in A; it also checks that the state contains the permissions denoted by A and removes them. As soon as permission to a heap location is no longer held, information about its value cannot be retained. Inhale and exhale can be seen as the permission-aware analogues of assume and assert statements [12]; they are sometimes called produce and consume [20]. Using these operations, a method call (for example) can be encoded by exhaling the method precondition and then inhaling its postcondition.

Building a verification tool for a permission logic requires effective solutions to the following technical challenges:

• 1. How to model the program state, including permissions and values?

• 2. How to check for a permission in a state?

• 3. How to add and remove permissions to and from a state?

• 4. How to evaluate (heap-dependent) expressions in a state?

• 5. When to preserve (frame) an expression’s value across heap changes?

In the remainder of this section, we summarize how existing verifiers solve these challenges for logics without ISC and then explain how providing support for ISC complicates these challenges.

2.1 Smallfoot-Style Symbolic Execution

Smallfoot [2] introduced a symbolic execution technique that has become the state-of-the-art way of building verifiers for permission logics. It provides simple and efficient solutions to the technical challenges above: (1) A symbolic state consists of a set of heap chunks, and a set of path conditions. A heap chunk has the form ${o.f}\mapsto {[}v,p]$, mimicking separation logic’s points-to predicates. It records a receiver value o, a field name f, a location value v representing the value stored in location o.f, and a permission amount p. A permission amount is a value between 0 and 1 (inclusive); intermediate values can be used to support fractional permissions. Here, o, v, and p are (immutable) symbolic values. Path conditions are boolean constraints on the symbolic values collected while verifying a program path such as the branch conditions on that path. Path conditions may constrain heap values and may be quantified. An SMT solver is used to answer queries about the path conditions, for instance, equality of symbolic values. (2) Checking for permission to a heap location entails iterating through the heap chunks and finding those with matching receiver-field pairs. (3) Removing permissions is modelled by subtracting permissions from the corresponding chunk(s), and adding a permission is modelled by adding a heap chunk (with a fresh symbolic location value) that provides the added permission amount. (4) Evaluating a heap lookup e.f yields the location value of the chunk for e.f (and is not permitted if no such chunk exists). (5) Framing the value of such expressions happens implicitly so long as the same heap chunk provides non-zero permission to the location. When a chunk no longer provides any permission, it gets removed and its location value becomes inaccessible.

In order to specify unbounded heap structures, the Smallfoot approach has been extended to handle user-defined recursive predicates. In successor tools such as VeriFast [22], jStar [7], and Viper [14], heap chunks may also represent predicate instances. Smallfoot-style symbolic execution has also been extended to support heap-dependent pure functions in the assertion language [20]. For example, the operations of a list class may be specified in terms of an function. Such functions include a precondition that requires permission to all locations read by the function body; this information is used to frame function applications.

These extensions increase the expressiveness of permission logics significantly, but are not sufficient to simply specify and automatically reason about important data structures such as arrays and graphs: this requires support for ISCs.

2.2 Iterated Separating Conjunction

Figure 1 illustrates the usage of ISCs: method replaces all occurrences of integer by integer in the segment of array between and . The recursive calls to smaller array segments are performed concurrently using parallel composition $\parallel$. The second precondition requires access permissions for all elements in the array segment, and the first postcondition returns these permissions to the caller; both are expressed using ISC. The second postcondition specifies the functional behaviour of the method using an -expression to refer to the prestate of a method; this pure assertion needs heap-dependent expressions under a quantifier.

Fig. 1.

A parallel replace operation on array segments. The second precondition and the first postcondition denote access permissions to the elements of the array. The quantifier in these conditions denotes an ISC: the body of the quantifier includes accessibility predicates (of the form ). The second postcondition uses a regular (pure) quantifier to specify the functional behaviour of the method. Here, expressions let the postcondition refer to values in the prestate; the access permissions for these expressions come from the second precondition.

Verifying the example entails splitting the symbolic state described by the ISC in the precondition in order to exhale the preconditions of the recursive calls, and to re-combine the states resulting from inhaling the postconditions of these calls after the parallel composition, in order to prove the callee’s postcondition.

Providing support for ISCs complicates each of the five technical challenges discussed above:

• 1. Heap chunks must be generalised to denote permission to an unbounded number of locations simultaneously, and encode a symbolic value per location (for instance, to represent the values of each array location in Fig. 1).

• 2. Exhaling an ISC requires checking permission for an unbounded number of heap locations; these could be spread across multiple heap chunks, as in the case of exhaling the postcondition of .

• 3. Removing permissions from a generalised chunk may affect only some of the locations to which it provides permission. For example, when exhaling the precondition of the first recursive call to , the permissions required for the second call must be retained in the symbolic state.

• 4. Evaluating heap-dependent expressions under quantifiers may rely on symbolic values from multiple heap chunks. For example, proving the second postcondition of requires information from both recursive calls.

• 5. Framing in existing Smallfoot-style verifiers requires that heap-dependent expressions depend only on a bounded number of symbolic values (which can include representations of predicate instances [20]). However, this requirement is too strong for pure quantifiers over heap locations and for functions whose preconditions use ISCs to require access to an unbounded set of locations (see for instance the client in the online version of our running example [1]).

Our technique is the first to provide automatic solutions to these challenging problems. Section 3 tackles the first 3 problems; Sect. 4 tackles the remaining 2.

3 Treatment of Permissions

We consider the following canonical form of source-level assertion for denoting an ISC:

, in which c(x) is a boolean expression, e(x) a reference-typed expression, and p(x) an expression denoting a permission amount. More complex assertions can be desugared into this canonical form, for instance, iterating over the conjunction of two accessibility predicates can be encoded by repeating the quantification over each conjunct. For simplicity, we do not consider nested ISCs, but an extension is possible. Our canonical form is sufficient to directly model quantifying over all receivers in a set (useful for graph examples) or over integer indices into an array, as shown in Fig. 1.

The permission expression p(x) may be a complex expression including conditionals, and need not evaluate to the same value for each instantiation of x. This enables us to model complex access patterns such as requiring non-zero permission to every nth slot of an array, which is for instance important for the verification of GPU programs [4]. ISCs are complemented by unrestricted pure quantifiers over potentially heap-dependent expressions, which are essential for specifying functional properties.

In this section, we present the first key ingredient of our symbolic execution technique: a representation for ISCs as part of the verifier’s symbolic state along with algorithms to manipulate this representation.

3.1 Symbolic Heap Representation

As explained in Sect. 2.1, Smallfoot-style heap chunks ${o.f}\mapsto {[}v,p]$ consist of a receiver value o, a field name f, a location value v and a permission amount p. A naïve generalisation of this representation would be to make o, v, and p functions of the bound variable of an ISC. However, such a representation has severe drawbacks. Checking whether a heap chunk provides permission to a location y.f (challenge 2 above) amounts to the existential query $\exists x. o(x) = y$; SMT solvers provide poor support for such existential queries. In the presence of fractional permissions, determining how much permission such a heap chunk provides is worse still, requiring to calculate the sum of all $p(x_i)$ such that $x_i$ satisfies the existential query.

Our design avoids these difficulties with a simple restriction: we require the receiver expressions e(x) in an ISC to be injective in x, for all values of x to which the ISC provides permission. Under this restriction, we can soundly assume that the mapping between the bound variable x and receiver expression e(x) is invertible for such values, by some function ${{e}^{-1}}$. We can then represent an ISC over receivers $r = e(x)$ directly, essentially by replacing x by ${{e}^{-1}}(r)$ throughout.

Our resulting design is to use quantified chunks of the form , in which r (which is implicitly bound in such a chunk) plays the role of a quantified (reference-typed) receiver. Such a quantified chunk represents p(r) permission to all locations r.f; p(r) may be any expression denoting a permission amount. The domain of a quantified chunk is the set of field locations $r'.f$ for which $p(r') > 0$. The values of these locations are modelled by the function v, which we call a value map and explain in Sect. 4. A symbolic heap is a set of quantified chunks; a symbolic state is a symbolic heap plus a set of path conditions, as usual.

Fig. 2.

Symbolic execution rules for inhaling and exhaling ISCs. The instruction submits a query to the SMT solver. If the proof obligation does not hold, it aborts with a verification failure. The Eval function evaluates an expression in a symbolic state and yields updated path conditions and the resulting symbolic expression, see Sect. 4. In both rules, the constraint $\underline{c}(y)$ is temporarily added to the path conditions used during the evaluation of e(y) and p(y); these expressions may be well-formed only under this additional constraint.

Under our injectivity restriction, we represent a source-level assertion of the form using a quantified chunk of the form for a suitable value map v and inverse function $e^{-1}$. Whenever necessary to avoid ambiguity, we use underlined expressions to denote the results of symbolically evaluating corresponding source-level expressions; with the exception of heap-dependent expressions (see Sect. 4.1), this evaluation is orthogonal to the contributions of this paper.

Our injectivity restriction does not limit the data structures that can be handled by our technique, provided specifications are expressed appropriately. The restriction applies to memory locations, not to the values stored in the locations. Many examples such as ISCs ranging over array indices or elements of a set naturally satisfy the restriction. Ranges that may contain duplicates (for instance, the fields of all objects stored in an array) can be encoded by mapping them to a set (thereby ignoring multiplicities) or by using complex permission expressions p that reflect multiplicities appropriately.

3.2 Inhaling and Exhaling Permissions

Using the symbolic heap design explained above, we define the operations for inhaling and exhaling ISCs in Fig. 2. The operation takes a symbolic heap $h _0$, path conditions $\pi _0$, and an ISC, and returns an updated heap and path conditions. Following the encoding described in the previous subsection, the operation introduces a (fresh) inverse function $e^{-1}$, which is constrained as the partial inverse of the (evaluated) receiver expression $\underline{e}(x)$ by adding the constraints Inv-1 and Inv-2 to the path conditions. We will discuss controlling the instantiation of these quantifiers (and others introduced by our technique) in Sect. 5. The fresh value map v models the (thus far unknown) values of the heap locations in the domain of the new quantified chunk, which is added to the symbolic heap $h _0$.

To encode our example (Fig. 1) in a tool without native array support, we model the array slots as a set of ghost objects, each with a field (representing the slot’s value). That is, an array location a[i] is modelled by the location , where is an injective function mapping indices to these ghost objects. Full details of the encoding of the running example are given online [1, Example ]. Following Fig. 2, inhaling the second precondition (at the start of checking the method body) entails introducing an inverse function ${{a}^{-1}}$ mapping array locations back to corresponding indices, and then adding a quantified chunk . Correspondingly, at the program point after the two recursive calls, the symbolic heap will contain two quantified chunks: one for each array segment.

The operation is initially similar to , one difference being that the injectivity of the receiver expression is checked before defining the inverse function. Removing permissions is more complex than adding permissions because it may involve updates to many existing quantified chunks in the symbolic state. This operation is delegated to the auxiliary operation , shown in Fig. 3.

The injectivity check performed by guarantees that the introduced inverse functions exist and satisfy the constraints added to the path conditions, which is required for soundness. We assume here that each operation has a corresponding ; for instance, inhaling a method precondition at the beginning of a method body corresponds to exhaling the precondition at the call site. Therefore, the check performed by also covers the inverse functions introduced in corresponding operations.

Fig. 3.

The remove operation. The argument q maps references to permission amounts. The operation checks that the symbolic heap contains at least q(r) permission for each location r.f and removes it.

takes as inputs an initial symbolic heap $h _0$ and path conditions $\pi _0$, a field name f, and a function q that yields for each reference r the permission amount for location r.f to be removed. fails with a verification error if the initial heap does not contain the permissions in q, and otherwise returns an updated symbolic state. This is achieved by iterating over all available chunks for field f, greedily taking as much of the still-required permissions ($q_{ needed }$) as possible from the current chunk ($q_{ current }$). Updating the chunks is expressed via pointwise-defined functions describing the corresponding permission amounts; they involve permission arithmetic, but no existential quantifiers, and can be handled efficiently by the underlying SMT solver. After this iteration, checks that all requested permissions have been removed.

In our array example (Fig. 1), we exhale the second precondition before each recursive call; this requires finding the appropriate permissions from the (single) quantified chunk in the state at this point, and removing them. Dually, when exhaling the postcondition at the end of the method body, all permissions from both of the two quantified chunks yielded by the recursive calls must be removed: the iteration in the algorithm achieves this.

Note that ’s permission accounting is precise, which is important for soundness and completeness: it maintains the invariant that (for all r), the difference between the permissions held in the original state and those requested via parameter q is equal to the difference between those held in the updated state and those still needed. If the operation succeeds, we know (from the last check) that those still needed are exactly 0, from which we conclude that precisely the correct amounts were subtracted.

3.3 Integrating Predicates with Iterated Separating Conjunctions

Predicates are a standard feature of verification tools for permission logics (including the Viper infrastructure on which our implementation is built); they integrate simply with our support for ISCs. Figure 4 shows an example of a predicate definition, parameterised by a set of nodes, that defines a graph in terms of ISCs and closure properties over the given set of nodes. Viper requires explicit ghost operations to exchange a predicate instance P(e) for its body (via an operation ), and vice versa (via an operation ); this is a standard way to handle possibly-recursive predicates. In terms of the underlying verifier, an operation essentially corresponds to followed by , and dually for . Since our support for ISCs is expressed in terms of inhale and exhale rules, it naturally integrates with Viper’s existing way of handling predicates; our implementation supports predicates with ISCs and pure quantifiers in their bodies, as illustrated by the graph predicate.

Our implementation does not yet support predicates inside ISCs, but our presented technique extends straightforwardly to support this. Inhaling an ISC which ranges over predicate instances yields, just as for accessibility predicates for fields, a new quantified chunk. An of a predicate belonging to such a chunk can be handled by exhaling the predicate instance (removing it from the chunk’s permissions), and then inhaling the predicate’s body. Folding an instance inhales a quantified predicate chunk that provides permissions to the single instance. We plan to extend our implementation to also support this feature combination, which will allow one to denote an unbounded number of predicate instances.

Fig. 4.

A predicate defining a graph in terms of ISCs and closure properties over a given set of nodes (that form the graph).

4 Treatment of Symbolic Values

So far we have addressed the first three technical challenges described in Sect. 2 by presenting a novel heap representation for ISCs together with algorithms that let the verifier efficiently add, as well as check for and remove permissions. In this section we present our solution to the remaining two challenges, concerned with the evaluation and framing of expressions.

4.1 Symbolic Evaluation of Heap-Dependent Expressions

Quantified chunks represent value information via the value map v. The existence of such a chunk in a symbolic heap allows the evaluation of a read of field f for any receiver in the domain of the heap chunk, to an application of the value map. Intuitively, v represents a partial function from this domain to values (of the type of the field f). Since SMT solvers typically do not natively support partial functions, we model value maps as under-specified total functions from the receiver reference (the field f is fixed) to the type of f. We apply these functions only to references whose f field location is in the chunk’s domain. This is why the algorithm (Fig. 2) does not need to explicitly remove information about the values stored in the locations whose permissions are removed; the underlying total function still represents appropriate values for the new (smaller) domain.

Summarising Value Maps. Inhaling permissions adds a fresh heap chunk with a fresh value map (see Fig. 2). Therefore, a symbolic heap may contain multiple chunks for the same field, each with its own value map. In the presence of fractional permissions, the domains of these chunks may overlap such that the value of one location x.f may be represented by multiple value maps. Similarly, the value of x.f may be represented by multiple maps when the receiver x is quantified over and the permissions to different instantiations of the quantifier are recorded in different chunks. Therefore, all of these value maps need to be considered when evaluating such a field access.

Fig. 5.

The operation introduces a fresh value map for field f and constrains it according to the value maps of all heap chunks for f. It also returns a function summarising the permissions held for the field f.

In order to incorporate information from all relevant chunks, and provide a simple translation for field-lookups, we summarise the value maps for all chunks for a field f lazily before we evaluate an expression e.f. This summarisation is defined by the operation in Fig. 5. For each quantified chunk with the appropriate field, it equates a newly-introduced value map with the value map in the chunk at all locations in the chunk’s domain. Analogously, it builds up a permission expression summarising the permissions held per receiver, across all heap chunks for the field f; we use this permission expression to check whether a field access is permitted.

Note that the definition of does not depend on path conditions, only on the symbolic heap; it can be computed without querying the SMT solver. Our implementation memoizes , avoiding the duplication of the function declarations and path conditions defining the value and permission maps.

Symbolic Evaluation. Symbolic evaluation of expressions is defined by an operation , which takes a symbolic heap, path conditions, and an expression, and yields updated path conditions and the symbolic value of the expression; the cases for field lookup and pure quantifiers are given in Fig. 6 (some additional cases can be found in Appendix A). Using the operation, we can simply define the evaluation of a field lookup, as shown first in Fig. 6. To evaluate such an expression, we check that at least some permission to the field location is held in the current symbolic heap, and use the value map generated by to define the value of the field lookup. Via the path conditions generated by , any properties known about the value maps of any of the corresponding quantified chunks will also be known about the resulting symbolic value. In each reachable state, these properties are consistent, which implies in particular that there exists a value for the field lookup that satisfies all of them. Viper regularly checks for inconsistent path conditions and prunes the current program path if it detects an unreachable state.

Evaluating pure quantifiers is handled by replacing the bound variable with a fresh constant and evaluating the quantifier body. Additional path conditions generated during this recursive evaluation might mention the fresh constant; these are universally quantified over when returning the path conditions.

Inhale, Exhale, and Field Writes. Inhaling and exhaling pure boolean expressions is implemented by first symbolically evaluating the expression and then either adding the resulting symbolic expression to the path conditions or checking it, respectively (see Appendix A).

A field write

is desugared as:

. The exhale checks that the heap has the required permission and removes it; the inhales create a new chunk with the previously-removed permission and constrain the associated value map such that it maps receiver $\underline{e_1}$ to the value of $\underline{e_2}$. For example, the field write in Fig. 1 is executed in a symbolic heap with a single quantified chunk that provides full permissions to each array location. After the field write has been executed, the heap contains two quantified chunks: the initial one, still providing full permissions to each array location except for (and with an unchanged value map), and a second one that provides permissions to only, with a fresh value map representing the updated value.

Fig. 6.

Symbolic evaluation of field reads and pure quantifiers.

4.2 Framing Heap-Dependent Expressions

Permissions provide a straightforward story for framing the values of heap locations (and pure quantifiers over these): so long as the symbolic state contains some permission to a field location, its value will be preserved. However, framing heap-dependent functions is more complicated [8, 20]. The value of a function can be framed so long as all locations the function depends on remain unchanged. To express a function’s dependency on the heap, its precondition must require permission to all locations its implementation may read. For any given function application, the symbolic values of these locations are called the snapshots of the function application. Consequently, two function applications yield the same result if they take the same arguments and have equal snapshots. One can thus model a heap-dependent function at the SMT level by a function taking snapshots as additional arguments [20].

ISCs complicate this approach because a function whose precondition contains an ISC may depend on an unbounded set of heap locations. The values of these locations cannot be represented by a fixed number of snapshots. It is also not possible to represent them as a value map since these are modelled at the SMT level as total functions, causing two problems. First, requiring equality of total functions would include locations the heap-dependent function does not actually depend on; since the values for these locations are under-specified, the equality check would often fail even when the function value could be soundly framed. Second, a function cannot be used as a function argument, nor compared for equality in the first-order logic supported by SMT solvers.

We address the first problem by modelling snapshots as partial functions called partial value maps, and the second by applying defunctionalisation [17]. That is, we model a partial value map for a field f of type T as a value of an (uninterpreted) type ${PVM}$, together with a function $domain _f:PVM \rightarrow Set [ Ref ]$ for the domain of the partial value map, and a function $apply _f:PVM \times Ref \rightarrow T$ for the result of applying a partial value map to a receiver reference. We also include an extensionality axiom for partial value maps, allowing us to prove equality when two partial value maps are equal as partial functions.

Following the prior work, we model a heap-dependent function via a function at the SMT level, with a partial value map as additional snapshot argument for each ISC required in the function’s precondition. For each application of such a function, we check that the current state contains all permissions required by the function precondition. If this is the case, we process each ISC in the precondition in turn. For an ISC for a field f, we employ the operation (Fig. 5) to summarise the value information v for the field f in the current symbolic state, and introduce a fresh constant $pvm$ of type ${PVM}$. We constrain $domain _f(pvm)$ to yield the set of references in the domain of the ISC, and for all receivers r in this domain, assume $apply _f(pvm,r) = v(r)$. $pvm$ is then used as a snapshot argument to the translated function.

5 Controlling Quantifier Instantiations

When generating quantifiers for an SMT solver, it is important to carefully control their instantiation [8, 11, 13] by providing syntactic triggers. A quantifier $\forall x \cdot P(x)$ may be decorated with a trigger $\{f(x)\}$, which instructs the solver to instantiate x with a term e only if f(e) is a term encountered by the solver during the current proof effort. Triggers must be chosen carefully: enabling too few instantiations may cause examples to fail unexpectedly, while too many may lead to unreliable performance or even non-termination of the solver (see also Sect. 6).

Fig. 7.

Example triggers used in our SMT encoding.

We carefully select triggers for all quantifiers generated by our technique (although we have omitted them from the presentation so far). Figure 7 shows three representative examples. The path condition VmDefEq relates the value map introduced by the operation to the value maps of heap chunks (Fig. 5). The two triggers express alternatives: they allow instantiating the path condition if either of the two value maps have been applied to the term instantiating r. This design allows us to derive relationships between two evaluations of an expression, which introduce two summary value maps. Instantiating VmDefEq in both directions allows us to relate these value maps via the value maps of heap chunks.

The next two examples define the inverse function of a receiver expression (see Fig. 2). The trigger $e^{-1}(r)$ for Inv-1 is essential for relating occurrences of the inverse function to the original expression e. The case of Inv-2 is almost symmetrical, but with extra technicalities. Since e comes from the source program, it may not be an expression allowed as a trigger. Trigger terms must typically include at least one function application (if e(x) were simply x, this could not be used), and no built-in operators such as addition. In the former case, we use v(x) as a trigger, where v is the value map of the relevant chunk; the quantifier will then be instantiated whenever we look up a value from the chunk, which is when we need the definition of the inverse function. In the latter case, we resort to allowing the underlying tools select trigger terms, which may lead to incompleteness. However, we did not observe any such incompletenesses in our experiments.

Instantiating either of the two axioms Inv-1 and Inv-2 gives rise to potentially new function application terms suitable for triggering the other axiom. For example, when instantiating Inv-2 due to a term of the shape e(x), we learn the equality $e^{-1}(\underline{e}(x)) = x$ in which the function application $e^{-1}(\underline{e}(x))$ matches the trigger for the Inv-1 axiom. Instantiating this axiom, in turn, will provide the equality $\underline{e}(e^{-1}(\underline{e}(x))) = \underline{e}(x)$. Note however, that this will not cause an indefinite sequence of instantiations of these two axioms (a so-called matching loop): SMT solvers consider quantifier instantiations modulo known equalities. Thus, the function application $\underline{e}(e^{-1}(\underline{e}(x)))$ does not give rise to a new instantiation of Inv-2, since the term to be matched against the quantified variable ($e^{-1}(\underline{e}(x))$) is already known to be equal to x, which was used for the prior instantiation.

6 Evaluation

We have implemented our technique as an extension of the Viper verification infrastructure [14]; the implementation is open source and can be tried online [1]. To evaluate the performance of our technique, we ran experiments with three kinds of input programs: (1) 9 hand-coded verification problems involving arrays and graphs, including our running example (see the Viper examples page [1] for details), (2) 65 examples generated by the VerCors project at the University of Twente [4], which uses our implementation to encode GPU verification problems, and (3) 82 additional regression tests.

Fig. 8.

Performance evaluation of our implementation on verification challenges. Lines of code (LOC) does not include blank lines and comments. Column “Time (s)” gives runtimes of the base version of our implementation; columns “w/o memoization” and “w/o triggers” show the % difference in time relative to the base version.

Figure 8 shows the results for (1), and Fig. 9 those for (2) and (3). We performed our experiments on an Intel Core i7-4770 3.40 GHz with 16 GB RAM machine running Windows 7$\,\times \,$64 with an SSD. The reported times are averaged over 10 runs of each verification (with negligible standard deviations). Timings do not include JVM start-up: we persist a JVM across test runs using the Nailgun tool.

Fig. 9.

Performance evaluation of our implementation on two sets of programs: the “VerCors” set contains (non-trivial) programs generated by the VerCors tool, “Regressions” contains (usually simple) regression tests; column “No. Files” displays the number of files per program set. All input files are available as part of the Viper test suite.

Our experiments show that our implementation is consistently fast: all examples verify in a few seconds. Since SMT encodings sometimes exhibit worse performance for failed verification attempts, we also tested 4 variants of each example from Fig. 8 in which we seeded errors; in all cases the errors were detected with lower runtimes (the verifier halts as soon as an error is detected).

To measure the effect of memoizing calls to , we disabled this feature and measured the difference in runtimes over the same inputs. As shown in the “w/o memoization” columns, disabling this optimisation typically increases the runtime, but not enormously; a likely explanation for the relatively small difference is that performs the iteration over quantified chunks efficiently, without querying the SMT solver. The number of quantified chunks in a given symbolic state is also typically kept small: the tool performs modular verification per method/loop body, and we eagerly remove any quantified chunks that no longer provide permissions (after an exhale).

To evaluate the importance of our use of triggers for controlling quantifier instantiations (see Sect. 5), we also compare with a variant of our implementation in which triggers are omitted, leaving this task to the underlying tools (that is, Viper and Z3 [6]). The relative times are shown in the “w/o triggers” columns. We observe that this variant typically improves verification time. However, the triggers chosen automatically by Viper and Z3 are too strict: 7 % of the programs (11 out of the 156 original programs) fail spuriously in this version. This, as well as a general reduction in quantifier instantiations, explains the effect on the runtime: the longest-running example in our base implementation (averaging 11.82 s) takes only 3 s without our triggers, but wrongly fails to verify. The longest-running example in the variant without triggers takes 8.83 s but also has a high standard deviation of 4.71 s, suggesting that performance also becomes unpredictable when triggers are selected automatically. The triggers that we choose thus avoid spurious errors and provide predictable, fast performance.

7 Conclusions and Future Work

We have presented the first symbolic execution technique that supports ISCs. This feature provides the possibility of specifying random-access data structures and provides an alternative mechanism to recursive definitions which is essential in the common case when a data structure can be traversed in multiple ways. Our technique generalises Smallfoot-style symbolic execution and is, thus, applicable to other verifiers for permission logics using this common implementation technique.

Two of the authors participated in the recent VerifyThis verification competition at ETAPS’16 (see http://etaps2016.verifythis.org/) using our implementation, and won the Distinguished User-assistance Tool Feature for the ISC support described in this paper: this prize was awarded for a feature that proved particularly useful during the competition.

As future work, we plan to build on our verification technique in four ways. First, we plan to extend our technique to support predicates under ISCs, as discussed in Sect. 3.3. Second, we plan to combine our verification technique with inference techniques that make use of ISCs, such as the shape analysis developed by Lee et al. [9]. Third, we plan to support statements that perform an operation (e.g., unfolding a predicate) on each instance of a quantifier without requiring a loop (and invariant). Such statements require permissions that can be expressed using ISCs. Fourth, we plan to integrate support for aggregates in pure assertions [11], which provide another means for specifying functional properties over locations described by an ISC.

A Additional Definitions and Symbolic Execution Rules

Partial Value Maps. Figure 10 shows background definitions related to partial value maps (see Sect. 4.2), which are emitted to the SMT solver before the verification starts. The background definitions include a type ${PVM}$ and, per field declaration, a function $domain _f$ that denotes the domain of a partial value map, a function $apply _f$ that denotes applying a partial value map to a receiver to obtain the value of the corresponding field location, and an extensionality axiom stating that two partial value maps are equal if their domains agree and if they agree on the values in their domain.

Fig. 10.

Background definitions related to partial value maps (see Sect. 4.2). $domain _f$ denotes the domain of a partial value map, $apply _f$ its application to a reference.

The trigger of the extensionality axiom $\mathbf \{ toSnap (pvm_1), toSnap (pvm_2)\mathbf \}$ ensures that the extensionality axiom is instantiated whenever it is necessary to reason about the equality of partial value maps that are used as snapshots. Wrapping partial value maps by $toSnap$ is necessary because Viper requires snapshots to uniformly be of type $toSnap$; function $toSnap$ embeds values into the $toSnap$ type (a corresponding inverse function exists as well). This external requirement (of Viper, not of our technique) turned out to be beneficial for us, since it allows choosing triggers that are permissive, yet yield good performance.

Inhaling and Exhaling Pure Assertions. Figure 11 shows the symbolic execution rules for inhaling and exhaling potentially heap-dependent (but pure) assertions such as pure quantifiers. Both rules use to evaluate the assertion; the result is then added to the path conditions or asserted to hold in the current state, respectively.

Fig. 11.

Symbolic execution rules for inhaling and exhaling pure assertions.

Symbolic Evaluation of Expressions. Figure 12 shows selected symbolic execution rules for evaluating expressions. Evaluating an implication $e_1 \Rightarrow e_2$ starts by evaluating $e_1$, and temporarily assuming $\underline{e_1}$ while evaluating $e_2$ (see also the discussion of Fig. 2 in Sect. 3.1). From the path conditions obtained from evaluating $e_1$ ($\pi _{\delta }$), all instances of VmDefEq are extracted ($\pi _v$). The final set of path conditions, with which the verification proceeds ($\pi _3$), includes the path conditions obtained from the evaluation of $e_1$, all instances of VmDefEq that were obtained from evaluating $e_2$ (this allows memoizing because value map definitions are always in scope, that is, are not nested under implications), and — conditionally on $\underline{e_1}$ — the remaining path conditions from evaluating $e_2$.

Fig. 12.

Additional symbolic execution rules for evaluating pure expressions.

Viper’s remaining symbolic execution rules for evaluating expressions did not need to be changed when we implemented our technique. For illustrative purposes, we show the rule for evaluating heap-independent functions (including arithmetic and other operators), and for evaluating short-circuiting conjunction.