Keywords

1 Introduction

Verifying personal identity through the inherent characteristics of individuals, biometric verification technology is attracting great attention as a more trustable alternative to token/knowledge-based security system. Some physiological biometric attributes like fingerprint or face are already familiar to the public. There is another biometric type called behavioral biometric attributes which are related to the pattern of behavior of a person, such as voice or signature [1]. Compared to physiological ones, they are more accessible and less intrusive. Among them, signature remains the most widespread and recognized socially and legally individual verification approach. Moreover, signing is a rapid movement driven by long-term writing habit, which will lead to the differences in both the signing process and the appearance of a signature. Therefore, it’s not possible that a forgery is exactly the same as the genuine signatures. That is, verification based on signature is feasible theoretically. Practically, affected by either environment condition or mental state variations among the signatures from a user occur inevitably, making it a challenging task.

Signature verification technique may be split into two categories: offline and online [20]. Offline signature verification works on the static digital signature images acquired after the signing process. The input of online signature verification is temporal signals captured by electronic devices like tablets, smart phones during the signing process. Usually, online signature verification system ensures a higher accuracy and security owing to dynamic information collected in the writing process. It makes the signature more unique and more difficult to forge.

Based on employed features, online signature verification techniques can broadly be divided into two groups: global features based and function features based approaches. In the framework of global features based methods, a signature is characterized as a vector of elements, each one representative of the value of a feature [1]. Examples of such attributes include width, height, etc. As for function features based methods, a signature is described in terms of a set of time function whose values constitute the feature set. Examples are position trajectory, velocity, pressure, etc. With regards to the classifiers, approaches like neural network (NN), support vector machine (SVM), dynamic time warping (DTW), hidden Markov Model (HMM) are adopted. Among them, DTW is considered as the most common technique. Sharma and Sundaram [3] explore the utility of information derived from DTW cost matrix and devise a novel score that describes the characteristic of the warping paths. Then they incorporate the derived warping path score with DTW score to make decision. Yanikoglu and Kholmatov [4] present a novel system based on the fusion of the Fast Fourier Transform and DTW. Kholmatov and Yanikoglu [8] match the test signatures against a set of reference signatures using DTW. Then, using the alignment scores, the test signatures are classified by standard pattern classification techniques.

Regardless of the method mentioned, the focus of them is mainly the dynamics of the signatures instead of shape. In online system, the shape of a signature is represented by its x-y coordinate. Gupta and Joyce [6] capture the shape using the position extrema points of a signature and propose the edit-distance-based string matching algorithm for comparing the sequence of two signatures. Shape context proposed by Belogie and Malik [16] offers a globally characterization to shape, making it a robust, compact and highly discriminative descriptor. That inspires us to exploit shape context based signature characterization. Besides, more distinguish characteristics usually exist in the signing process. In order to improve the accuracy further, function features based approach is utilized subsequently to form a cascade framework.

The rest of this paper is organized as follows: Sect. 2 gives our proposed method in detail. Section 3 shows the database used in our experiment and experimental results. Section 4 offers the conclusion.

2 Proposed Method

The proposed method for online signature verification is detailed in the following subsections. Figure 1 shows the diagram representation of the proposed system. The input signature is first passed through preprocessing model. After that, we use shape context to capture its shape feature and the calculated shape distance is fed into a classifier. Only after passing the first stage test will it enter the second stage. In this stage, its function features are extracted and SC-DTW is employed to get a distance. Then the distance is used to verify the authenticity of the signature.

Fig. 1.
figure 1

Diagram of proposed verification system

Full size image

2.1 Signature Preprocessing

Because signatures are captured by electronic devices, noises and fluctuations are interrupted unavoidably. And there’s no guarantee that signatures acquired at different time or places of one individual will always be the same. Those variations will decrease the similarities between test and reference signatures. In order to address those issues, preprocessing comprising of smoothing and normalization is adopted as the first step.

Gaussian smoothing can be employed to reduce those artifacts. The normalization step standardizes the size and location of signature at different inputs. The commonly used method of size normalization is the utilization of maxi-min normalization. The size depends on the maximum and minimum in the horizontal and vertical directions. Despite relative simplicity, it cannot represent the exact size as is attested by Fig. 2. Figure 2(a) shows the genuine signatures and skilled forgery from two user. As we can see, some people tend to write longer strokes at times, for example, some downward and upward strokes. Our solution is the introduction of moment-based normalization [9]. The size of a signature depends on the width and height of the window calculated by its moment, and Fig. 2(b) shows the window. Some research shows that y coordinate provides more distinctive information. Therefore, the height of the signature is normalized to a predefined value and the width is adjusted accordingly in order to keep the aspect ratio. For location normalization, the signature is centered at (0, 0). Figure 2 shows the original signatures and corresponding preprocessed signature. After preprocessing, the signatures have the same size and location.

Fig. 2.
figure 2

Examples for signature preprocessing. (a) Original signatures in SVC 2004. (b) Window calculated by moment of corresponding signatures. (c) Corresponding preprocessed signatures.

Full size image

2.2 Shape Context Based Signature Characterization

The shape context captures the distribution over relative positions of other shape points and thus summarizes global shape in a rich, local descriptor. The shape is represented by a set of points sampled from the shape contours which in this work is (xi, yi), i = 1, 2,…, N, N is the number of points. Different people writes at different speed and the data acquisition equipment samples the signature at fixed interval, which means the number and the distribution of sample points acquired varies with person. The speed information that is proved to be one of the most discriminative feature is implicit in the (xi, yi). Shape Context is capable of extract those differences.

As we can see from Fig. 3(a–c), the number and the distribution of the sample points from two genuine signatures are more similar. Taking one point as the origin of polar coordinate, the shape context of this point is calculated as illustrated. Log-polar histogram bins are used to represent the shape contexts and we choose five bins for log r and twelve bins for \( {\theta } \). The number of neighboring points that fall into the very bin is just the histogram value.

Fig. 3.
figure 3

Shape contexts computation and matching. (a–b) Two genuine signatures. (c) A skilled forgery. They are all from the same person. Note that the square marker points represents trend-transition-points (TTPs) extracted. (d) Diagram of log-polar histogram bins used in computing the shape contexts. (e-g) Example shape contexts histograms for certain trend-transition-point marked in (a–c).

Full size image

Consider a point pi on the first shape and a point qj on the second shape. Denote Cij = C(pi, qj) as the matching cost of these two points, given by

$$ C_{ij} = C\left( {p_{i} ,q_{j} } \right) = \frac{1}{2}\sum\nolimits_{k = 1}^{K} {\frac{{\left[ {h_{i} \left[ k \right] - h_{j} \left[ k \right]} \right]^{2} }}{{h_{i} \left[ k \right] + h_{j} \left[ k \right]}}} $$
(1)

where hi[k] and hj[k] denote the \( {\text{K}} \)-bin histogram at pi and qj respectively.

Given the set of costs Cij between all pairs of points pi on the first shape and qj on the second shape, the Hungarian method is implemented to find the optimal alignment. The cost between shape contexts is based on the chi-square test statistic, so thin plate spline (TPS) model is adopted for transformation. After that, the distance between two shapes can be measured generally.

2.3 Trend-Transition-Point (TTP) Extraction

The presentation for signature shape should not only captures the essentials of the shape but allows considerable variation. Besides, the more points, the greater the computational load. There’s no need to use all points to represent the shape, and just a few selected points could do it. Based on the above, we propose a trend-transition-point (TTP) extraction method and note that the shape context is calculated on every point, but only extracted points are involved in computing the shape distance.

TTPs include local extrema points and corner points. The trends before and after the TTPs are completely different while between two successive TTPs, its shape approximates to a straight line. So TTPs keep its shape thus the signature could be reconstructed with these selected points. The method of corner point detection we adopted is proposed in [7], which makes use of eigenvalues of covariance matrices of different support regions.

Let Sk(si) denotes the region of support (ROS) of point si, which contains itself and k points in its left and right neighborhoods. That is Sk(si) = {sj | j = i − k, i − k + 1, …, i + k − 1, i + k}, sj = {xj, yj}. λL and λS are the eigenvalues obtained from the covariance matrix of Sk(si). Sharper the corner is, larger λS is. When the shape in an ROS is close to a straight line, its λS will approaches to zero. So corners can be determined according to λS exceeding a threshold.

As a summary, the algorithm is implemented as follows. The start and end points as extrema points are chosen. As to the others, if its corresponding x or y coordinate value is higher or lower than both the left and right one, it is an extreme point. If not, its λS would be calculated. In order to avoid the determination of threshold, whether the point is a TTP is decided by its neighborhood. That is, unless the λS of this point is greater than the left and right points, it can be categorized as a TTP. After the rough extraction mentioned, one of the two successive points whose distance is lower than a threshold will be deleted depending on its λS.

2.4 Function Features Based Signature Characterization

The signature is considered as a ballistic movement. The dissimilarity of shape can be helpful to tell the genuine signature from random and minor skilled forgeries while not that discriminative to well-skilled forgeries. A set of function features are shown in Table 1.

Table 1. Function features of online signature verification
Full size table

2.5 DTW Based Matching

In general, dynamic time warping or DTW is a method that calculate an optimal match between two given sequences based on the dynamic programming (DP) algorithm such that the overall distance between them is minimized.

Classical DTW.

For the past a few years, DTW has become a major technique in signature verification. It is employed extensively along with function features, since it enables the time axis of two temporal functions to be compressed or expanded locally, in order to obtain the minimum of a given distance measure [3].

More specifically, denote \( T = \left\{ {t_{1} ,t_{2} , \ldots ,t_{N} } \right\}\,\,{\text{and}}\,\,S{ = }\left\{ {s_{1} ,s_{2} , \ldots ,s_{M} } \right\} \) as two time series of different length N and M respectively. A matrix termed “cost matrix” denoted by d(n, m) is constructed whose (n, m)th cell represents the dissimilarity between the \( n th \) point of T and the \( m th \) point of S. The cost matrix is defined as:

$$ d\left( {n,m} \right) = \left\| {t_{n} - s_{m} } \right\| $$
(2)

The overall distance is calculated as shown in the following equation:

$$ D\left( {n,m} \right) = d\left( {n,m} \right) + min\left\{ {\begin{array}{*{20}l} {C\left( {n,m - 1} \right) } \hfill \\ {C\left( {n - 1,m - 1} \right)} \hfill \\ {C\left( {n - 1,m} \right) } \hfill \\ \end{array} } \right. $$
(3)

where D(n, m) is the cumulative distance up-to the current element.

SC-DTW.

DTW has been an effective method of finding the alignment between two signatures with different length. However, DTW usually warps time series according to their numerical characteristics as Eq. (2) but ignores their shape nature, and can lead to abnormal alignment sometimes. Inspired by Zhang and Tang’s idea [5], we apply shape context to DTW in order to obtain a more feature to feature alignment. In this method, time series is considered as a 1-D array and a 2-D shape.

When finding the alignment between two time series, the cost matrix is replaced by the cost between shape contexts, which means

$$ d\left( {n,m} \right) = C_{ij} $$
(4)

where Cij is defined in Eq. (1).

It is worth noting that shape context is merely used to find the alignment between two time series and the cumulative distance is still obtained by the original cost matrix. The SC-DTW is computed on each signature segments divided by TTPs. The final dissimilarity of two signatures is the average of distances computed on each function features (Fig. 4).

Fig. 4.
figure 4

SC-DTW. (a) The time series of total velocity v from two signature and a pair of corresponding points found by shape context. (b) The shape context histograms of the points marked in (a).

Full size image

2.6 Verification

Due to a limited number of genuine signatures and rare well-skilled forgeries available in practical applications, we, in this paper, propose a cascade verification system. In the first stage, the test signature is verified on account of the dissimilarity in shape. Random forgeries and minor skilled forgeries can be distinguished easily while well skilled forgeries not. That promotes a second stage for further verification. Dynamic features symbolize the information during the signing process, even though an adept forger couldn’t imitate them completely. Hence, they could be helpful when verifying skilled forgeries.

During enrolment, the user supplies several signatures as reference signatures. They are pairwise coupled to get the distances between shape context and function features. When verifying a test signature, the signature is compared with all the reference signatures belonging to the claimed ID in terms of shape context firstly. After normalized by the corresponding averages of the reference signatures, the shape distances’ average is used in classification. Unless classified as a genuine signature, this test signature steps into next stage. At the second stage, the difference is that test signature is compared with regards to function features distances through SC-DTW and others are alike. The average DTW distances is the basis of classification.

3 Experiments and Evaluation

In this section, we present the database we used and then analyze the experimental results we get. The performance is evaluated with Equal Error Rate (EER), which is calculated as the point at which the False Rejection Rate (FRR) of genuine signatures equals the False Acceptance Rate (FAR) of forgery signatures. We average the EERs obtained across all users enrolled.

And the writing pressure has been recognized as one of the most effective and discriminative feature and is quite helpful to make decision. However, some small pen-based input devices such as personal digital assistants (PDA) cannot collect the pressure information, and in the mobile scenario where finger is used as writing-tool, pressure is hard to obtain. The study of interoperability between devices and the effects of mobile conditions has been a hot topic in the field of online signature verification. So we test our proposed method in the case of with and without pressure information being used.

3.1 Database

The database we use is publicly available SVC 2004, which contains two tasks—task1 and task2. The signature data for task1 contain coordinate information only, but the signature data for task2 also contain additional information including pressure. Every task has 40 users respectively and for each user, 20 genuine and 20 skilled forgeries were collected using a graphic tablet (WACOM Intuos). And the genuine signatures are collected in two sessions, spaced apart by at least one week. The signatures are mostly in either English or Chinese.

3.2 Verification Results

In our experiment, we randomly select 5 genuine signatures for enrolment as reference signatures from each user. The resting 15 genuine signatures and 20 skilled forgeries of the users are employed for testing. For the random forgeries scenario, 20 signatures from other users are randomly selected. The trial is conducted for ten times for each user and the average EER is computed as the measure of performance. And we evaluate the method on the condition of common threshold and user-dependent threshold. In the common threshold set-up, the matching scores from all users are compared with a predefined threshold. While in user-dependent threshold case, the threshold is taken from 1.0 to 2.0 with a step size 0.1. We also test the system both with and without pressure information. Task1 and task2 are all usable in the absence of pressure information, while in the case of considering pressure information, only task2 is used.

In the first stage, shape distance is fed into a distance-based classifier. It is not a discriminative enough feature to tell the skilled signatures. However, it is good at distinguishing the random forgeries from genuine signatures. As we can see from Table 2, the EER between genuine signature and random forgeries is lower and approaches to zero in the user threshold set-up. It can classify the signatures roughly in this stage. Because the damage brought by rejecting a genuine signature is higher than by accepting a forgery, the test signature judged as a genuine one will be fed into next stage. In the second stage, the test signature is classified on the ground of DTW distance which is more reliable. In Table 3, we compare the results from classical DTW and our proposed SC-DTW, showing that SC-DTW could get lower EER. The test signature is classified again and it is a determinate genuine signature unless it is accepted. The final EER we get is 2.85% when the threshold is user-dependent. In Table 4, we list the SC-DTW and cascade structure verification results in the presence and absence of pressure information in the condition of user threshold. The pressure is an enough effective dynamic feature and the EER could be significantly decreased. Given the circumstances of user threshold and cascade structure, the EER is 2.45%.

Table 2. Verification results (%) based on shape distance for SVC2004
Full size table
Table 3. Verification results (%) of different stages with common threshold and user threshold for SVC2004
Full size table
Table 4. Verification results (%) in the presence and absence of pressure information with user threshold for SVC2004
Full size table

3.3 Comparisons

In this subsection, we give the results of prior works tested on SVC2004 as is showed in Table 5. The number of reference signatures is all five in the listed works. And their results are obtained based on SVC2004 task2. With pressure information help, the EER of our proposed method is slightly lower than the state-of-the-art, demonstrating its effectiveness and competitiveness. But without pressure information, the result is not the best but still can be acceptable.

Table 5. Comparisons between proposed and prior works on the SVC 2004
Full size table

4 Conclusion

In this paper, we present a novel online signature verification technique based on shape context and function features. When only x and y coordinates are considered, shape context is a robust descriptor to capture the shape of signature. The shape distance is computed accordingly. In order to reduce the computational complexity, we propose a trend-transition-point (TTP) extraction algorithm and only these point is participated in the calculation of shape distance. In order to improve the performance further, a set of dynamic function features are derived and we use SC-DTW to get the similarity. We incorporate shape context into DTW to measure the dissimilarity between two points, thus getting a more feature-to-feature alignment. And the process takes place in the segments divided by TTPs. Then we talk about the effect of pressure information. The results we get is competitive given the absence of pressure and pen inclination and the like which are not available in certain scenario. It provide a possibility of the usability of shape in online system and a prospect for the study of interoperability between devices and the effects of mobile conditions, which will be our future work.