# A modified subgradient extragradient method for solving monotone variational inequalities

- 702 Downloads
- 3 Citations

## Abstract

In the setting of Hilbert space, a modified subgradient extragradient method is proposed for solving Lipschitz-continuous and monotone variational inequalities defined on a level set of a convex function. Our iterative process is relaxed and self-adaptive, that is, in each iteration, calculating two metric projections onto some half-spaces containing the domain is involved only and the step size can be selected in some adaptive ways. A weak convergence theorem for our algorithm is proved. We also prove that our method has \(O(\frac{1}{n})\) convergence rate.

## Keywords

variational inequalities subgradient extragradient method Lipschitz-continuous mapping level set half-spaces convergence rate## MSC

47J20 90C25 90C30 90C52## 1 Introduction

*H*be a real Hilbert space with inner product \(\langle\cdot,\cdot \rangle\) and norm \(\Vert \cdot \Vert \). The variational inequality problem (

*VIP*) is aimed to finding a point \(x^{*}\in C\), such that

*C*is a nonempty closed convex subset of

*H*and \(f:C\rightarrow H\) is a given mapping. This problem and its solution set are denoted by \(\operatorname{VI}(C,f)\) and \(\operatorname {SOL}(C,f)\), respectively. We also always assume that \(\operatorname{SOL}(C,f)\neq\emptyset\). The variational inequality problem \(\operatorname{VI}(C,f)\) has received much attention due to its applications in a large variety of problems arising in structural analysis, economics, optimization, operations research and engineering sciences; see [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14] and the references therein.

*λ*is an arbitrary positive constant. Many algorithms for the problem (1.1) are based on the fixed point problem (1.2). Korpelevich [2] proposed an algorithm for solving the problem (1.1) in Euclidean space \(R^{n}\), known as the extragradient method (EG):

*λ*is some positive number and \(P_{C}\) denotes the metric projection of

*H*onto

*C*. She proved that if

*f*is

*κ*-Lipschitz-continuous and

*λ*is selected such that \(\lambda\in(0,1/\kappa)\), then the two sequences \(\{ x_{n}\}\) and \(\{\tilde{x}_{n}\}\) generated by the EG method, converge to the same point \(z\in\operatorname{SOL}(C,f)\).

In 2006, Nadezhkina and Takahashi [3] generalized the above EG method to general Hilbert spaces (including infinite-dimensional spaces) and they also established the weak convergence theorem.

In each iteration of the EG method, in order to get the next iterate \(x_{k+1}\), two projections onto *C* need to be calculated. But projections onto a general closed and convex subset are not easily executed and this might greatly affect the efficiency of the EG method. In order to overcome this weakness, Censor *et al.* developed the subgradient extragradient method in Euclidean space [4], in which the second projection in (1.4) onto *C* was replaced with a projection onto a specific constructible half-space, actually which is one of the subgradient half-spaces. Then, in [5, 6], Censor *et al.* studied the subgradient extragradient method for solving the *VIP* in Hilbert spaces. They also proved the weak convergence theorem under the assumption that *f* is a Lipschitzian continuous and monotone mapping.

The main purpose of this paper is to propose an improved subgradient extragradient method for solving the Lipschitz-continuous and monotone variational inequalities defined on a level set of a convex function [13], that is, \(C:=\{x\in H \mid c(x)\leq0\}\) and \(c:H\rightarrow R\) is a convex function. In our algorithm, two projections \(P_{C}\) in (1.3) and (1.4) will be replaced with \(P_{C_{k}}\) and \(P_{T_{k}}\), respectively, where \(C_{k}\) and \(T_{k}\) are half-spaces, such that \(C_{k}\supset C\) and \(T_{k}\supset C\). \(C_{k}\) is based on the subdifferential inequality, the idea of which was proposed firstly by Fukushima [14], and \(T_{k}\) is the same one as Censor’s method [5].

It is also worth pointing out that the step size in our algorithm can be selected in some adaptive way, that is, we have no need to know or to estimate any information as regards the Lipschitz constant of *f*, therefore, our algorithm is easily executed.

Our paper is organized as follows. In Section 2, we list some basic definitions, properties and lemmas. In Section 3, the improved subgradient extragradient algorithm and its corresponding geometrical intuition are presented. In Section 4, the weak convergence theorem for our method is proved. Finally, we prove that our algorithm has \(O(\frac {1}{n})\) convergence rate in the last section.

## 2 Preliminaries

*H*be a real Hilbert space with inner product \(\langle\cdot,\cdot \rangle\) and norm \(\Vert \cdot \Vert \) and let

*C*be a closed convex subset of

*H*. We write \(x_{k}\rightharpoonup x\) and \(x_{k}\rightarrow x\) to indicate that the sequence \(\{x_{k}\}_{k=0}^{\infty}\) converges weakly and strongly to

*x*, respectively. For each point \(x\in H\), there exists a unique nearest point in

*C*, denoted by \(P_{C}(x)\), such that

*H*onto

*C*. It is well known that \(P_{C}\) is characterized by the following inequalities:

*c*at

*x*. We say

*c*is Gâteaux differentiable on

*H*, if for each \(x\in H\),

*c*is Gâteaux differentiable at

*x*.

A function \(c:H\rightarrow R\) is said to be weakly lower semicontinuous (w-lsc) at \(x\in H\), if \(x_{k}\rightharpoonup x\) implies \(c(x)\leq\liminf_{k\rightarrow\infty}c(x_{k})\). We say *c* is weakly lower semicontinuous on *H*, if for each \(x\in H\), *c* is weakly lower semicontinuous at *x*.

*c*is said to be subdifferentiable at a point \(x\in H\) if the set

*c*at

*x*, \(\partial c(x)\) is subdifferential of

*c*at

*x*and the inequality in (2.4) is said to be the subdifferential inequality of

*c*at

*x*. We say

*c*is subdifferentiable on

*H*, if

*c*is subdifferentiable at each \(x\in H\). It is well known that if

*c*is Gâteaux differentiable at

*x*, then

*c*is subdifferentiable at

*x*and \(\partial c(x)=\{c'(x)\}\), namely, \(\partial c(x)\) is just a set of the simple points [17].

*κ*, such that

*f*is also said to be a

*κ*-Lipschitzian-continuous mapping.

*H*, if

### Definition 2.1

Normal cone

*C*at \(v\in C\),

*i.e.*

### Definition 2.2

Maximal monotone operator

*H*.

*T*is called a maximal monotone operator if

*T*is monotone,

*i.e.*

*T*,

It is clear that a monotone mapping *T* is maximal iff for any \((x,u)\in H\times H\), if \(\langle u-v,x-y\rangle\geq0\), \(\forall(y,v)\in G(T)\), then it follows that \(u\in T(x)\).

*T*is maximal monotone and \(0\in Tv\) if and only if \(v\in \operatorname{SOL}(C,f)\) [18].

The next property is known as the Opial condition and all Hilbert spaces have this property [19].

### Lemma 2.3

*For any sequence*\(\{x_{k}\}_{k=0}^{\infty}\)

*in*

*H*

*that converges weakly to*

*x*(\(x_{k}\rightharpoonup x\)),

*the inequality*

*holds for any*\(y\in H\)

*with*\(x\neq y\).

The following lemma was proved in [20].

### Lemma 2.4

*Let*

*H*

*be a real Hilbert space and let*

*C*

*be a nonempty*,

*closed and convex subset of*

*H*.

*Let the sequence*\(\{x_{k}\}_{k=0}^{\infty}\subset H\)

*be Fejér*-

*monotone with respect to*

*C*,

*i*.

*e*.,

*for any*\(u\in C\),

*Then*\(\{P_{C}(x_{k})\}_{k=0}^{\infty}\)

*converges strongly to some*\(z'\in C\).

## 3 The modified subgradient extragradient method

*C*is a level set given as follows:

In the rest of this paper, we always assume that the following conditions are satisfied.

### Condition 3.1

The solution set of \(\operatorname{VI}(C,f)\), denoted by \(\operatorname {SOL}(C,f)\), is nonempty.

### Condition 3.2

The mapping \(f:H\rightarrow H\) is monotone and Lipschitz-continuous on *H* (but we have no need to know or to estimate the Lipschitz constant of *f*).

### Condition 3.3

- (i)
\(c(x)\) is a convex function;

- (ii)
\(c(x)\) is weakly lower semicontinuous on

*H*; - (iii)
\(c(x)\) is Gâteaux differentiable on

*H*and \(c'(x)\) is a \(M_{1}\)-Lipschitzian-continuous mapping on*H*; - (iv)
there exists a positive constant \(M_{2}\) such that \(\Vert f(x) \Vert \leq M_{2} \Vert c'(x) \Vert \) for any \(x\in\partial C\), where

*∂C*denotes the boundary of*C*.

Next, we present the modified subgradient extragradient method as follows.

### Algorithm 3.4

The modified subgradient extragradient method

- Step 1:
- select an initial guess \(x_{0}\in H\) arbitrarily, set \(k=0\) and construct the half-space$$ C_{k}:=\bigl\{ w\in H \mid c(x_{k})+\bigl\langle c'(x_{k}),w-x_{k}\bigr\rangle \leq0\bigr\} ; $$
- Step 2:
- given the current iteration \(x_{k}\), computewhere$$ y_{k}=P_{C_{k}}\bigl(x_{k}- \beta_{k}f(x_{k})\bigr), $$(3.2)and \(m_{k}\) is the smallest nonnegative integer, such that$$\begin{aligned} \beta_{k}=\sigma\rho^{m_{k}},\quad\sigma>0, \rho\in(0,1) \end{aligned}$$(3.3)where \(M=M_{1}M_{2}\) and \(\nu\in(0,1)\).$$\begin{aligned} \beta_{k}^{2} \bigl\Vert f(x_{k})-f(y_{k}) \bigr\Vert ^{2}+2M \beta_{k} \Vert x_{k}-y_{k} \Vert ^{2}\leq\nu^{2} \Vert x_{k}-y_{k} \Vert ^{2}, \end{aligned}$$(3.4)
- Step 3:
- calculate the next iterate,where$$ x_{k+1}=P_{T_{k}}\bigl(x_{k}- \beta_{k}f(y_{k})\bigr), $$(3.5)which is the same half-space as Censor’s method [5].$$ T_{k}=\bigl\{ w\in H \mid\bigl\langle x_{k}- \beta _{k}f(x_{k})-y_{k},w-y_{k}\bigr\rangle \leq0\bigr\} , $$(3.6)

At the end of this section, we list the alternating theorem [21, 22] for the solutions of \(\operatorname{VI}(C,f)\), where *C* is given by (3.1). This result will be used to prove the convergence theorem of our algorithm in the next section.

### Theorem 3.5

*Assume that the solution set*\(\operatorname{SOL}(C,f)\)

*of*\(\operatorname {VI}(C,f)\)

*is nonempty*.

*Given*\(x^{*}\in C\).

*Then*\(x^{*}\in\operatorname{SOL}(C,f)\)

*iff we have either*

- 1.
\(f(x^{*})=0\),

*or* - 2.
\(x^{*}\in\partial C\)

*and there exists a positive constant**β**such that*\(f(x^{*})=-\beta c'(x^{*})\).

## 4 Convergence theorem of the algorithm

In this section, we prove the weak convergence theorem for Algorithm 3.4. First of all, we give the following lemma, which plays a crucial role in the proof of our main result.

### Lemma 4.1

*Let*\(\{x_{k}\}_{k=0}^{\infty}\)

*and*\(\{y_{k}\}_{k=0}^{\infty}\)

*be the two sequences generated by Algorithm*3.4.

*Let*\(u\in\operatorname{SOL}(C,f)\)

*and let*\(\beta_{k}\)

*be selected as*(3.3)

*and*(3.4).

*Then*,

*under the Conditions*3.1, 3.2

*and*3.3,

*we have*

### Proof

*f*, we have

The subsequent proof is divided into following two cases.

Case 1: \(f(u)\neq0\).

*M*is defined as before. Substituting (4.9) into the last inequality of (4.4), we obtain

Case 2: \(f(u)=0\).

*m*be the smallest nonnegative integer, such that

*κ*is the Lipschitz constant of

*f*. Noting that \(\Vert f(x_{k})-f(y_{k}) \Vert \leq\kappa \Vert x_{k}-y_{k} \Vert \), we assert from (4.12) and (4.13) that \(m_{k}\leq m\), which implies

### Theorem 4.2

*Assume that Conditions*3.1-3.3

*hold*.

*Then the two sequences*\(\{x_{k}\} _{k=0}^{\infty}\)

*and*\(\{y_{k}\}_{k=0}^{\infty}\)

*generated by Algorithm*3.4

*converge weakly to the same point*\(z \in\operatorname{SOL}(C,f)\),

*furthermore*

### Proof

*i.e.*,

*H*, so there exists \(M'>0\) such that \(\Vert c'(x_{k}) \Vert \leq M'\) for all \(k\geq0\), and then

*T*is a maximal monotone operator.

*T*is a maximal monotone operator, (4.24) means that \(0\in T(z)\) and consequently \(z\in T^{-1}(0)=\operatorname{SOL}(C,f)\).

## 5 Convergence rate of the modified method

### Lemma 5.1

*Let*\(\{x_{k}\}_{k=0}^{\infty}\)

*and*\(\{y_{k}\}_{k=0}^{\infty}\)

*be the two sequences generated by Algorithm*3.4

*and let*\(\beta_{k}\)

*be selected as*(3.3)

*and*(3.4).

*Assume that the Conditions*3.1, 3.2

*and*3.3

*are satisfied*.

*Then*,

*for any*\(u\in C\),

*we have*

### Theorem 5.2

*For any integer*\(n>0\),

*we have a*\(z_{n}\in H\),

*which satisfies*\(z_{n}\rightharpoonup z\), \(z\in\operatorname{SOL}(C,f)\)

*and*

*where*

### Proof

On the other hand, since \(z_{n}\) is a convex combination of \(y_{0}, y_{1}, \ldots, y_{n}\), it is easy to see that \(z_{n}\rightharpoonup z\in\operatorname {SOL}(C,f)\) due to the fact that \(y_{k}\rightharpoonup z\in\operatorname{SOL}(C,f)\) proved by Theorem 4.2. The proof is complete. □

## 6 Results and discussion

Since the modified subgradient extragradient method proposed in this paper is relaxed and self-adaptive, it is easily implemented. A weak convergence theorem for our algorithm is proved due to the alternating theorem for the solutions of variational inequalities. Our results in this paper effectively improve the existing related results.

## 7 Conclusion

Although the extragradient methods and the subgradient extragradient methods have been widely studied, the existing algorithms all face the problem that the projection operator is hard to calculate. The problem can be solved effectively by using the modified subgradient extragradient method proposed in this paper, since two projections onto the original domain are all replaced with projections onto some half-spaces, which is very easily calculated. Besides, the step size can be selected in some adaptive ways, which means that we have no need to know or to estimate the Lipschitz constant of the operator. Furthermore, we prove that our method has \(O(\frac{1}{n})\) convergence rate.

## Notes

### Acknowledgements

This work was supported by the Foundation of Tianjin Key Lab for Advanced Signal Processing (2016 ASP-TJ02).

## References

- 1.Kinderlehrer, D, Stampacchia, G: An Introduction to Variational Inequalities and Their Applications. Society for Industrial and Applied Mathematics, Philadelphia (2000) CrossRefMATHGoogle Scholar
- 2.Korpelevich, GM: An extragradient method for finding saddle points and for other problems. Èkon. Mat. Metody
**12**, 747-756 (1976) MathSciNetMATHGoogle Scholar - 3.Nadezhkina, N, Takahashi, W: Weak convergence theorem by an extragradient method for nonexpansive mappings and monotone mappings. J. Optim. Theory Appl.
**128**, 191-201 (2006) MathSciNetCrossRefMATHGoogle Scholar - 4.Censor, Y, Gibali, A, Reich, S: Extensions of Korpelevich’s extragradient method for the variational inequality problem in Euclidean space. Optimization
**61**, 1119-1132 (2012) MathSciNetCrossRefMATHGoogle Scholar - 5.Censor, Y, Gibali, A, Reich, S: The subgradient extragradient method for solving variational inequalities in Hilbert space. J. Optim. Theory Appl.
**148**, 318-335 (2011) MathSciNetCrossRefMATHGoogle Scholar - 6.Censor, Y, Gibali, A, Reich, S: Strong convergence of subgradient extragradient methods for the variational inequality problem in Hilbert space. Optim. Methods Softw.
**26**, 827-845 (2011) MathSciNetCrossRefMATHGoogle Scholar - 7.Yao, YH, Postolache, M, Liou, YC, Yao, ZS: Construction algorithms for a class of monotone variational inequalities. Optim. Lett.
**10**, 1519-1528 (2016) MathSciNetCrossRefMATHGoogle Scholar - 8.Yao, YH, Liou, YC, Kang, SM: Approach to common elements of variational inequality problems and fixed point problems via a relaxed extragradient method. Comput. Math. Appl.
**59**, 3472-3480 (2010) MathSciNetCrossRefMATHGoogle Scholar - 9.Yao, YH, Noor, MA, Liou, YC, Kang, SM: Iterative algorithms for general multivalued variational inequalities. Abstr. Appl. Anal.
**2012**, 768272 (2012) MathSciNetMATHGoogle Scholar - 10.Yao, YH, Noor, MA, Liou, YC: Strong convergence of a modified extragradient method to the minimum-norm solution of variational inequalities. Abstr. Appl. Anal.
**2012**, 817436 (2012) MathSciNetMATHGoogle Scholar - 11.Zegeye, H, Shahzad, N, Yao, YH: Minimum-norm solution of variational inequality and fixed point problem in Banach spaces. Optimization
**64**, 453-471 (2015) MathSciNetCrossRefMATHGoogle Scholar - 12.Yao, YH, Shahzad, N: Strong convergence of a proximal point algorithm with general errors. Optim. Lett.
**6**, 621-628 (2012) MathSciNetCrossRefMATHGoogle Scholar - 13.He, S, Yang, C: Solving the variational inequality problem defined on intersection of finite level sets. Abstr. Appl. Anal.
**2013**, 942315 (2013) MathSciNetMATHGoogle Scholar - 14.Fukushima, M: A relaxed projection method for variational inequalities. Math. Program.
**35**, 58-70 (1986) MathSciNetCrossRefMATHGoogle Scholar - 15.Takahashi, W: Nonlinear Functional Analysis. Yokohama Publishers, Yokohama (2000) MATHGoogle Scholar
- 16.Goebel, K, Reich, S: Uniform Convexity, Hyperbolic Geometry and Non-expansive Mappings. Dekker, New York (1984) MATHGoogle Scholar
- 17.Hiriart-Urruty, JB, Lemarchal, C: Fundamentals of Convex Analysis. Springer, Berlin (2001) CrossRefGoogle Scholar
- 18.Rockafellar, RT: On the maximality of sums of nonlinear monotone operators. Trans. Am. Math. Soc.
**149**, 75-88 (1970) MathSciNetCrossRefMATHGoogle Scholar - 19.Opial, Z: Weak convergence of the sequence of successive approximations for nonexpansive mappings. Bull. Am. Math. Soc.
**73**, 591-597 (1967) MathSciNetCrossRefMATHGoogle Scholar - 20.Takahashi, W, Toyoda, M: Weak convergence theorems for nonexpansive mappings and monotone mappings. J. Optim. Theory Appl.
**118**, 417-428 (2003) MathSciNetCrossRefMATHGoogle Scholar - 21.He, S, Xu, HK: Uniqueness of supporting hyperplanes and an alternative to solutions of variational inequalities. J. Glob. Optim.
**57**, 1375-1384 (2013) MathSciNetCrossRefMATHGoogle Scholar - 22.Nguyen, HQ, Xu, HK: The supporting hyperplane and an alternative to solutions of variational inequalities. J. Nonlinear Convex Anal.
**16**, 2323-2331 (2015) MathSciNetMATHGoogle Scholar - 23.Facchinei, F, Pang, JS: Finite-Dimensional Variational Inequalities and Complementarity Problems, vols. I and II. Springer Series in Operations Research. Springer, New York (2003) MATHGoogle Scholar
- 24.Cai, XJ, Gu, GY, He, BS: On the \(O(\frac{1}{t})\) convergence rate of the projection and contraction methods for variational inequalities with Lipschitz continuous monotone operators. Comput. Optim. Appl.
**57**, 339-363 (2014) MathSciNetCrossRefMATHGoogle Scholar

## Copyright information

**Open Access** This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.