Norm conflict identification in contracts

Abstract

The exchange of goods and services between individuals is often formalised by a contract in which the parties establish norms to define what is expected of each one. Norms use deontic statements of obligation, prohibition, and permission, which may be in conflict. The task of manually detecting norm conflicts can be time–consuming and error-prone since contracts can be vast and complex. To automate such tasks, we develop an approach to identify potential conflicts between norms. We show the effectiveness of our approach and its individual components empirically using two publicly available corpora, and contribute with a new annotated test corpus for norm conflict identification.

Appendices

Appendix 1: Instructions used by human volunteers for conflict creation

The following text was used to guide the volunteers during the norm conflict insertion. The document consists of a description for each step of the system.

This README explains how to execute contract_data_structure.py to introduce conflicts randomly in contracts from our corpus.⁴ The main goal is to create contracts containing norm conflicts independently from a conflict detection algorithm.

Execute:

• To execute, run the following command: python -B contract_data_structure.py

• After login, there will be two initial options:

  • (1) Pick a random contract; and

  • (4) Finish.

Options:

• (1) This option chooses a contract from the corpus at random. This step may return an error due to the choice of a contract without norms; if that happens, ignore the error and press (1) again. If no error occurs, the program will display information, such as the total number of norms extracted and the parties identified;

• (4) This option just clears the output folder and exits.

• When the user selects a contract, the program adds a new option:

  • (1) Pick a random contract;

  • (2) Pick a random norm; and

  • (4) Finish.

• Option (1) restarts the process with a new contract;

• Option (2) chooses a random norm among the extracted ones.

• When the user selects a norm, the program adds yet another option:

  • (1) Pick a random contract;

  • (2) Pick a random norm;

  • (3) Make a conflict; and

  • (4) Finish.

• Options (1,2,4) are the same as before;

• Option (3) displays the last chosen norm and asks you to alter it in order to create a conflict.

Process:

• In this manual conflict insertion, you are intended to follow a series of steps.

  • 1. Execute contract_data_structure.py;

  • 2. Insert your first name, and pick a contract with option (1);

  • 3. Choose a random norm using option (2);

  • 4. Choose the option to create a conflict (3).

• You have to create between 70 and 100 conflicts of 3 types. These types are:

  • Permission $\times$ Obligation (33%);

  • Permission $\times$ Prohibition (33%);

  • Obligation $\times$ Prohibition (33%).

• A regular norm has the following structure:

• Example 1:

  • “Purchaser must pay the product taxes.”

  Given a regular norm, you will choose option 3, which allows you to alter such norm. Then you have to alter it in order to generate a conflict, e.g., if you got the Example 1, you may choose to create either a Permission $\times$ Obligation conflict or an Obligation × Prohibition conflict. In the first case (Permission $\times$ Obligation), a possible modification can be described as follows:

• Example 2:

  • “Purchaser MAY pay the product taxes.”

    To ensure that you are really making a conflict, use Table 7 as a guide:

    Table 7 Modal verbs and their deontic modalities

  You may also modify the structure after the modal verb creating a conflict and altering the conflict structure (obviously maintaining the same meaning), as the Example 3 shows.

• Example 3:

  • “Purchaser may choose to pay the taxes related to the product.”

• We recommend you to use more than three contracts to create conflicts, it allows us to test our approach in different contexts.

• At the end of the process, choose option (4) and that’s it!

  Thanks in advance.

Appendix 2: Performance measures used in the paper

Below, we summarize the performance measures from the literature used in this paper.

$$accuracy = \frac{{tp + tn}}{{tp + tn + fp + fn}}$$

(2)

$$precision = \frac{{tp}}{{tp + fp}}$$

(3)

$$recall = \frac{{tp}}{{tp + fn}}$$

(4)

$$f - score = 2 \times \frac{{precision \times recall}}{{precision + recall}}$$

(5)

$$tpr = \frac{{tp}}{{tp + fn}}$$

(6)

$$specificity = \frac{{tn}}{{tn + fp}}$$

(7)

$$fpr = \frac{{fp}}{{fp + tn}}$$

(8)