Homework 10: Formal Methods with Z3 Due Wednesday, April 24, 2024, 11:59PM AoE

High level goals

The high-level goal of this assignment is to learn about automated theorem provers (Z3 in particular), and how they can be used to model programs, prove program properties, and generate test cases.

You may work with a partner for this assignment. If you do you must use the same partner for all sub-components of this assignment.

Setup

Download and unzip the hw10-z3.zip archive.

Open the Z3 web interface (or this or this) in your browser. (Alternatively, download the latest Z3 release and run the examples from the command line.)

Check out the Z3 Tutorial if you want more background. However, you should be able to do most of this exercise with targeted queries as opposed to reading the entirety of it.

Instructions

The hw10-z3.zip archive contains four folders (three pairs and one set of programs). Each pair contains the original program, a mutated program, and z3 starter code for comparing the two. The set contains the original program, a set of four mutants, and z3 starter code.

Follow these steps:

1. Examine the original and mutated program in the pair1 directory. Are they equivalent? (See question 1, below.)
2. Complete Z3startercode.pair1.smt2 (see hints below!). Your completed code should either prove equivalence of the two programs or generate a model (i.e., a test case) that proves non-equivalence.
3. Test your code by running it with Z3. Run the .smt2 file with Z3. You can copy the entire contents of the file into the web interface and run it there.
4. ** Repeat steps 2 and 3 for the pair2 and pair3 directories.** Note that we are using width-8 BitVecs instead of the normal width-32 BitVec. This is for ease of reading/writing/reasoning: the principals we care about are the same, but are a little easier to grok at this lower precision. Also, recall the BitVec operators, such as bvadd and bvsgt, that replace Int operators, such as + and >; if you need more information about the available BitVec operators, you can consult the documentation.
5. Familiarize yourself with Z3 scopes.
6. Examine the four mutants in the set1 directory.
7. Use scopes to complete Z3startercode.set1.smt2. Your completed code should, for each mutant, either prove equivalence or generates a model (test case) that proves non-equivalence. By using scopes you can leverage the fact that much of the execution is identical between the original program and the mutant programs. (See Question 2 and 3)
8. Test your code by running it with Z3.

Hints

When modifying the .smt2 templates, only modify what is between the following two lines. Do not alter anything that is not between the following two lines, including the lines themselves!

;;;;;;;;;;;;;;;;; START STUDENT CODE ;;;;;;;;;;;;;;;

and

;;;;;;;;;;;;;;;;; END STUDENT CODE ;;;;;;;;;;;;;;;

These anchor lines are used for grading.

Submission: Written Report

Answer the following questions:

1. Which of the mutants in pair1, pair2, and pair3 are equivalent? Provide, for each non-equivalent mutant, a test case that detects it. The test case should be the output of the (get-model) instruction in your smt2 code. [5 points, -2 per incorrect answer and -1 per missing or incorrect test case, to a minimum of 0]
2. Which of the four mutants in set1 are equivalent? Provide, for each non-equivalent mutant, a test case that detects it. [3 points, -1 per incorrect answer or missing test case to a minimum of zero]
3. Briefly explain how scopes work in Z3 and when they are useful? [1 point for explanation, 1 point for when they are useful]

Submit a plain-text file or PDF with your answers to the above questions to the HW10 assignment on Canvas. Include in that file:

• the name and UCID of your partner, if applicable
• your final versions of the four Z3startercode.<xyz>.smt2 templates.
• citations for any outside sources that you used

FAQ

Coming soon!