Generalized Arithmetic Mean, Harmonic Mean and Geometric Mean Divergence Measures Having One Parameter

Generalized Arithmetic Mean, Harmonic Mean and Geometric Mean Divergence Measures Having One Parameter

*Prince Goyal

Assistant Professor, Department of Community Medicine, Dr. YSP Govt. Medical College, Nahan (HP)

Abstract:- Whenever the reliability of the system is considered then there is a birth of divergence, according to ones need. We mainly use the divergence measure to achieve the reliability of the system. In some situations like population dynamics, exponential distributions, growth models etc. where Shannons measure gives insufficient results we use directed-divergence measures and their generalizations. Entropy based measures have been frequently used as divergence measures. Therefore, information theoretic divergence measures either probabilistic or non-probabilistic are of great importance and have a major effect to get the target. There are a lot of information and divergence measures used in coding theory, information theory and also in mathematics and statistics. A symmetrized and smoothed form of K-L divergence, the Jensen-Shannon divergence (JSD), is of particular interest because of its sharing properties with families of other divergence measures. The uniqueness and versatility of this measure arise because of a number of attributes including generalization to any number of probability distributions. Furthermore, its entropic formulation allows its generalization in different statistical frameworks. We revisit these generalizations and propose a new generalized AM, HM and GM divergence measure in the integrated mathematical framework. We show that this generalization can be interpreted in terms of mutual information divergence measures.

Keywords:- Divergence, Csiszars f-divergence, Arithmetic Mean, Harmonic Mean, Geometric Mean, Generalized Mean Divergence Measure, Generalized Square Root Mean Divergence Measure

INTRODUCTION

Probabilistic mean divergence is a necessary process of collecting useful information from different and relevant sources. It extensively exists in many areas including medical diagnosis, business (share market), engineering, insurance, decision making and so on. In the literature of information theory so many techniques have been developed and discussed to gather relevant information. We know that the first measure of information theory was given by C E Shannon in 1948 while working in Bell laboratories. The most popular divergence is K-L relative information measure or cross entropy. Jenson-Shannon (JS), J- divergence and Arithmetic Geometric (AG) mean and some other divergences are also famous classical measures. All these measures have some interesting inequalities. In some situations like population dynamics, exponential distributions, growth models etc. where Shannons measure and its generalizations are not applicable we use directed-divergence measures and their generalizations. Aczel [1948] defined some mean values. Bhatia [1990] studied some quantitative-qualitative entropy measures and did their applications in coding theory. Jain [1987] studied some characteristics of a relative useful information measure. Tuteja and Bhaker [1993], defined mean values of some useful information measures. Eve [2003] studied seven means from a mathematical and geometrical point of view, which are Harmonic, Geometric, Arithmetic, Heronian, Contra-Harmonic, Root- mean Square and Centroidal. For any type of divergence, the main relation is for the strategies which one has taken or used to achieve the goal or target. Basically the strategies are related with probabilities of that event.

Let P = (p1, p2,.,pn) be a discrete probability distribution with n outcomes and Q = (q1, q2,.,qn) be another probability distribution then the well known Kullback-Leibler [1951] measure of directed-divergence was defined as:

=1

D(P,Q) =

The KL divergence is non-symmetric in nature. If qi is zero then the corresponding is also zero. The KL divergence measure satisfies the following conditions (which are commonly same for divergence measure).

1. D (P : Q) 0

2. D (P : Q) = 0 iff P = Q

3. D (P : Q) is a convex function of P i.e. of p1, p2,..,pn

n

Now let

Un P (p1, p2 ,…, p 3 ) pi 0,

pi 1, n 2

i 1

is the set of all finite discrete probability distributions for all P, Q Un. Then the measures are defined as:

n p

n 2p

K(P : Q)

pi log

i ;

H(P : Q)

pi log

i

and

i 1

qi

i 1

pi qi

n p q

p q

G(P : Q) i i log i i

i 1

2

2pi

are called Relative-Information, Relative Jenson Shannon Divergence and Relative Arithmetic Geometric Divergence. A generalization of above measures have studied by Taneja [2005] and he called these type s divergence. It is important to define a generalization of a divergence measure by introducing some real parameter that yields a number of new divergences. A lot of work in this area have done by Jain and Tuteja [1986], Sharma [1978], Taneja [1985], Taneja and Hooda [1983], Dragomir [2001, 2001a], Taneja [1989, 1995, 2001, 2013], Burbea and Rao [1982], etc.

A few years back Eve [2003] studied the geometrical interpretation of the seven means as: Let a > 0, b > 0 then the means are as follows:

a b

Arithmetic Mean : A (a : b) =

; Geometric Mean : G (a : b) = ab Harmonic Mean : H (a :

2

b) =

2ab a b

; Heronian Mean : H (a : b) =

a 2 b2

(a

ab b) 3

Contra-Harmonic Mean : (a : b) =

2(a2 ab b2 )

a2 b2 a b

; R-

M Square : R (a : b) =

and Centroidal Mean : C (a : b) =

2

3(a b)

Now we can verify the following result for the above means.

Now if we take M (a : b) = b fm (a/b), where M denotes any of the above seven means, then we have

2x (x

x 1)

Where

fH (x)

, fG (x)

x, fH ' (x)

x 1 3

(x 1) 2(x2 x 1) (x2 1)

fA (x)

, fC (x)

, fR (x)

fCH (x)

2 3(x 1) 2

x2 1

x > 0, x 1

x 1

If we take a = x and b = 1 in any of the above mentioned means then we get the above results. We can also write harmonic, heronian, contra-harmonic, centroidal means and root-mean-square in terms or arithmetic and geometric means.

Csiszars f-Divergence

Given a function f : [0, ) R, Csiszar [1967] introduced the f-divergence measure as:

n p

cf (P : Q)

qif

i

P, Q Un

i 1

qi

Corollary 1: If the function f is convex normalized, i.e. if f (1) = 0, then the f-divergence cf (P : Q) is non-negative and convex

in the pair of probability distribution (P : Q) Un Un.

Corollary 2: Let f : R+ R be differentiable convex and normalized i.e., f(1) = 0, then

f

0 cf (P : Q) Ec (P : Q) ; where

n p

Ecf (P : Q)

(p

i qi )f '

i

P, Q Un and i (1, 2,.,n)

i 1

qi

Some New Generalized Probabilistic Mean Divergence Measures

Let us consider the following mean of order m

1

am bm M

m 0

Sm (a : b) 2

; a, b R

(1)

Now we see that the values of the above equation varies as

ab f m 0

S (a : b) max(a : b) if m

min(a : b) if m

m

a, b R

It can be also easily verified that Sm(a : b) is a monotonic non decreasing function in relation to m.

Thus the following inequality holds as

S (a : b) a b A(a : b)

where

S (a : b) 2ab H(a : b)

1 a b

1

and

2

a2 b2

S0 (a : b)

ab G(a : b)

S2 (a : b) S(a : b) 2

are called Harmonic mean, Geometric mean, Arithmetic mean and Square root mean. So it can be said that H (a : b) G (a : b) A (a : b) S (a : b)

Taneja [2005, 2005a, 2005b] has also proposed some mean divergence measures as:

1. Square root AM Divergence

   n p2 q2 n n

   MSA (P || Q)

   i i 1

   pi

   1, q

   i 1 and P,Q Un

2. Square root GM Divergence

   i 1 2

   i1 i1

   n p2 q2 n n

   MSG (P || Q)

   i i

   piqi

   pi

   1, q

   i 1 and P,Q Un

   i 1 2

   i1 i1

3. Square root HM Divergence

   n p2 q2

   2p q n n

   MSH (P || Q)

   i i

   i i

   pi

   1, q

   i 1 and P,Q Un

   i 1

   2 pi qi

   i1 i1

4. Arithmetic Geometric Mean Divergence

   n n n

   MAG (P || Q) 1

   i 1

   piqi

   pi 1, qi 1 and P,Q Un

   i1 i1

5. Arithmetic Harmonic Mean Divergence

   n 2p q n n

   MAH (P || Q) 1

   i i

   pi

   1, q

   i 1 and P,Q Un

   i 1 pi qi

6. Geometric Harmonic Mean Divergence

i1 i1

2

n 2p q

n piqi

pi

qi n n

MGH (P || Q)

piqi

• i i

  pi

  1, q

  i 1 and P,Q Un

  i1

  pi qi

  i1

  pi qi

  i1 i1

  from the above results we can say that the following inequalities hold

  Some studies have done by Bhattacharyya [1943], Osterreicher and Vajda [2003], etc. related to the mean divergence measures. Now we propose some new probabilistic divergence measures having one or two parameters.

  First, we study divergence measures having one parameter The measure is defined as

  1

  n a2 b2 a b

  M (a : b)

  i i

  i 1 ai bi

• i i

  2

  where ] , 0 [U] 0, [ (2)

  which is known as generalized mean divergence measure.

  The equation (2) becomes AM, HM and GM divergence for different values of as:

  n a2 b2

  a b

  A

  M (a : b)

  i i

  i 1 ai bi

• i i

  2

  ; when = 1

  n a2 b2

  2a b

  H

  M (a : b)

  i i i i

  ; when = 1

  i 1 ai bi

  ai bi

  n a2 b2

  MG (a : b)

  i i

  aibi

  ; when 0

  i 1 ai

  bi

  The above results can be easily verified according to the values of (for = 1, -1, 0)

  Bhatia and Singh [2013] have proposed some AM, HM and GM measures. The proposed measure satisfies all the conditions of a probabilistic divergence measure. Thus we can say that the proposed generalized measure is valid and reliable.

  Again we discuss some other one parametric probabilistic divergence measures The proposed measure is

  n a2 b2

  1

  a b

  s

  M (a : b)

  i i

  i i

  where ] , 0 [U] 0, [ (3)

  i 1 2

  2

  which is known as generalized square root mean divergence measure and satisfies the properties of a divergence measure. Again we find that the equation (3) varies for different values of and contains some particular cases. Particular Cases:

  n

  SA

  a2 b2

  a b

  M (a : b)

  i i

  • i i ; when = 1

i 1

2 2

M (a : b)

ai

• bi

a b

n 2 2

SG i i

; when 0

i 1 2

n a2 b2

2a b

MSH (a : b)

i i

i i

; when = 1

i 1 2

ai bi

Similarly we can make generalized Arithmetic Geometric mean divergence measure, Arithmetic Harmonic mean divergence measure and Geometric Harmonic mean divergence measure which will satisfy some properties and also have some limiting cases.

Now we prove non negativity and convexity of the proposed measure (equation (3)).

take a = x, b = 1 and = 1

Let us consider

x2 1 x 1

DSA (x :1) ; x (0, )

2 2

We take 1st and 2nd derivative of above function and get

D (x :1) 1 . 2x 1 x 1

2 2

SA 1

x2 1 2

2

2 x2 1 2

And the 2nd derivative is

2

1 2

x

1 1

2(x2 1)2 2(x2 1) 2 .4x

1

2(x2 1)2 2x

DSA

(x :1)

2 ;

2(x2 1)

2(x2 1) 2×2 1

2(x2 1) 2

2(x2 1)

Thus we have DSA (x :1) 0

non negative and convex.

2 2 x2 1(x2 1) 2 x2 1(x2 1)

x (0,) . Also we have DSA(x : 1) = 0 for x = 1. Thus we can say that the function is

Again we have

x2 1

DSG (x :1)

2

x; x (0, )

where a = x, b = 1 and 0

Again we find the 1st and 2nd derivative of the above function and get

D (x :1) 1 . 2x 1

SG 1 1

x2 1 2 2

2

2x 2

2

x 1

1

x

1

x2 1

2

2

2 x 2

x2 1

2x

Again differentiate w. r. t. x and we get

x

3

2

1 x 1 .2x

2

1 x

DSG

(x :1)

2

2 x2 1

x2 1

2

2

2

x2 1

1

x

x2 1

1 3

x 2

2 x2 1

2 2

x2 1 x2

1 x2 1

1 3

1 1

x 2

2 x2 1

2 2

2(x2 1) x2 1

4x x

Again, we find that DSG (x :1) 0

x (0,). We have also DSG(x : 1) = 0 for x = 1. Thus it can be said that the above

equation is non negative and convex in the pair of probability distributions. Also, we have

x2 1 2x

DSH (x :1)

2 x 1

; x (0, )

When a = x, b = 1 and = 1 ; we again do derivative of the equation and find

D (x :1) 1 . 2x 2 x 1 x

2

2

SH

2 x 1

2

(x 1)2

x 2

Again differentiate, we get

2 x2 1

(x 1)2

.

x2 1 x 2x

DSH

(x :1) 1

2

2 x2 1

x2 1

2.2

(x 1)3

1 x2 1 x2

4 1 4

Again we see that

2 (x2 1) x2 1

(x 1)3

2(x2 1) x2 1

(x 1)3

DSH (x :1) 0

x (0,) , and DSH(x : 1) = 0 for x = 1. Thus it can be also said that equation fulfills the

condition of non-negativity and convexity. Now we prove some theorems.

Theorem 1:- The folowing inequalities hold for arithmetic and harmonic mean differences.

0 M (P || Q) 1 M (P || Q)

1. SA

   3 SH

   and

2. 0 SA

(P || Q) 1

3 SH

(P || Q)

Proof: Let us consider

K (x)

DSA (x)

SASH

DSH (x)

1

2(x2 1) x2 1

1 4

; x (0, )

2(x2 1) x2 1

(x 1)3

(x 1)3

3

(x 1)3 4 2 x2 12

Again differentiate the above equation w. r. t. x, we get

KSA SH

D (x)

1. SA

   DSH (x)

   3

   1

   (x 1)3 4 2(x2 1)2 3(x 1)2.1 (x 1)3

   3(x 1)2 4 2. 3 (x2 1) 2 .2x

   2

   2

   3

   (x 1)3 4 2(x2 1)2

   3

   3(x 1)2 (x 1)3 4 2(x2 1)2 (x 1)3 3(x 1)2 12 2x

   2

   x2 1

   3

   (x 1)3 4 2(x2 1)2

   12 2(x 1)2 (x2 1) (x2 1) 12 2x(x 1)3

   2

   (x 2 1)

   3

   12 2(x 1)2

   2

   (x 1)3 4 2(x2 1)2

   (x2 1)(x 1)

   3

   (x 1)3 4 2(x2 1)2

   24(x 1)(x2 1)(x 1)2

   2

   3

   2(x2 1) (x 1)3 4 2(x2 1)2

   The value of equation varies as

   Thus we can say that the equation given above increases in x (0, 1) and decreases in x (1, ).

   Theorem 2:- The inequalities related to arithmetic and geometric mean holds as

   0 M (P || Q) 1 M (P || Q)

   1. SA

      2 SG

      and

   2. 0 SA

Proof: Let us take

(P || Q) 1

2 SG

(P || Q)

K (x)

DSA (x)

SASH

DSG (x)

1

2(x2 1) x2 1

1 1

2(x2 1) x2 1

1

4x x

2 x2 1(x2 1)

4x x

2(x2 1) x2 1

2(x2 1) x2 1 4x x

3

4x 2

3

4x 2

2(x2 1) (x2 1)

Again, differentiate the above w. r. t. x, we have

KSA SG

D (x)

1. SA

   DSG (x)

   4x 2

   2(x2 1) (x2 1) 4. 3 x 2 4x 2 6x 2

   2(x2 1). 2x

   2 (x2 1).2x

   3 1 3 1

   2

   3

   2 (x2 1)

   2

   4x 2

   2(x2 1) (x2 1)

   5

   6 2(x2 1) x(x2 1) 12 2x 2

   2

   (x2 1)

   3 3

   4x 2

   2(x2 1)2

   Again we find that the value of equation variates as

   Again we can say that the above equation or function increases in x (0, 1) and decreases in x (1, ).

   Theorem 3:- The following inequalities related to harmonic mean and geometric mean holds

   0 M (P || Q) 3 M (P || Q)

   1. SH

      2 SG

   2. 0 SH

(P || Q) 3

2 SG

(P || Q)

Proof: Let us again consider

K (x)

DSH (x)

SHSG

DSG (x)

1 4

2(x2 1) x2 1

(x 1)3

1 1

2(x2 1) x2 1

4x x

(x 1)3 4 2(x2 1) (x2 1)

2(x2 1) (x2 1)(x 1)3

4x x

2(x2 1) (x2 1)

2(x2 1) (x2 1) 4x x

4x x (x 1)3 4 2(x2 1) (x2 1)

(x 1)3 4x x

3

2(x2 1)2

3 3

4x 2 (x 1)3 4 2(x2 1)2

3 3

(x 1)3 4x 2

2(x2 1)2

By adopting the same procedure again differentiate the above function w. r. t. x we get

KSH SG

D (x)

(x) SH

DSG (x)

3 3 3

1

(x 1)3 4x 2

2(x2 1)2 4x 2 (3(x 1)2 12 2(x2 1) 2 .x

3

1 3

3

(x 1)3 4 2(x2 1)2 .6x 2 4x 2 (x 1)3 4 2(x2 1) 2

1 1

3 3

(x 1)3 6x 2 3 2(x2 1) 2 .x 4x 2

2(x2 1) 2 3(x 1)2

3

2

3

(x 1)3 4x 2

2(x2 1)2

1 1 3 1

192 2×4 (x2 1)2 (x 1)3 6 2x 2 (x2 1) 2 (x 1)6 48x 2 (x 2 1)3 (x 1)3

5 1 3 3

12 2x 2 (x2 1)2 (x 1)6 192 2×3 (x2 1) 2 (x 1)2 96x 2 (x 2 1)3 (x 1)

2

3 3

(x 1)3 4x 2

2(x2 1)2

1 1

5 3 5

6 2x 2 (x2 1)2 (x 1)2 32x 2 (x 1) (x 1)4 (x2 1) 2x 2 4 2(x2 1) 2 (1 x)

2

3 3

(x 1)3 4x 2

2(x2 1)2

We again see that the value of the above function varies as follows

0 for x 1 and

0 for x 1

So, we conclude that the function increases in x (0, 1) and decreases in x (1, ).

CONCLUSION

In the present chapter we have proposed some new probabilistic mean divergence measures containing one parameter. Here, we have studied their limiting cases and properties also. Bounds on these proposed measures have also defined by using csiszars f-divergence. Then we discuss some theorems related to the proposed measures. The concept of weightage can also be used these proposed measures. All these proposed measures can be used in various decision making problems and better results can be found due to the flexibility of parameters. Comparison of one measure can also be done with some other measures and may get some beneficial and fruitful results. These newly developed measures can also be made in fuzzy or generalized fuzzy set theory. Then application can be done in some decision making situations.

REFERENCES

[1] Aczel J., On Mean Values; Bulletin of The American Mathematical Society, Vol. 54, (1948), pp. 392-400.

[2] Bhatia P. K., Summetry and Quantitative-Qualitative Measures and Their Applications to Coding; Ph. D. Thesis, Maharshi Dayanand University, Rohtak, India (1990).

[3] Bhatia P. K. and Singh S., On Some Divergence Measures between Fuzzy Sets and Aggregation Operators; Advanced Modeling and Optimization, Vol. 15 (2), (2013), pp. 235-248.

[4] Bhattacharyya A. (1943), On a Measure of Divergence between Two Statistical Populations Defined by their Probability Distributions; Bull. Cal.

Math. Soc., Vol. 35, pp. 99-109.

[5] Burbea J. and Rao C. R. (1982), On the Convexity of Some Divergence Measures Based on Entropy Functions; IEEE Trans. On Inform. Theory, IT- 28, pp. 489-494.

[6] Csiszar I. (1967), Information Type Measures of Differences of Probability Distribution and Indirect Observation; Studia Math. Hungarica, Vol. 2, pp. 229-318.

[7] Dragomir S. S. (2001), Some Inequalities for the Csiszars f-Divergence Inequalities for Csiszars f-Divergence in Information Theory; Monograph

Chapter 1, Article 1, http://rgmia.vu.edu.au/monographs/csiszar.htm.

[8] Dragomir S. S. (2001a), Other Inequalities for the Csiszars Divergence and Applications Inequalities for Csiszars f-Divergence in Information Theory; Monograph Chapter 1, Article 4, http://rgmia.vu.edu.au/monographs/csiszar.htm.

[9] Eve H. (2003), Means Appearing in Geometrical Figures; Math. Mag., Vol. 76, pp. 292-294.

[10] Jain P. (1987), On Axiomatic Characterization of Generalized Measure of Relative Useful Information; Tamkang Journal of Mathematics, Vol. 18, No. 3, pp. 61-68.

[11] Jain P. and Tuteja R. K. (1986), An Axiomatic Characterization of Relative Useful Information; JIOS, Vol. 7, pp. 49-57. [12] Kullback S. and Leibler R. A. (1951), On Information and Sufficiency; Ann. Mathematical Staistics, Vol. 22, pp.79-86.

[13] Osterreicher F. and Vajda I. (2003), A New Class of Metric Divergences Measure of Csiszars f-Divergence Class and Its Bounds; Computer and Mathematics with Applications, Vo. 49(4), pp. 575-588.

[14] Shannon C. E. (1948), The Mathematical Theory of Communication; Bells System Technical Journal, Vol. 27, pp. 379-423.

[15] Sharma B. D. and Mittal D. P. (1975), New Non-Additive Measures of Entropy for a Discrete Probability Distributions; Journal of Mathematical Science, Vol. 10, pp. 28-40.

[16] Sharma B. D. et al. (1978), On Measures of Useful Information; Information and Control, Vol. 29, pp. 323-326. [17] Taneja H. C. (1985), On Measures of Relative Useful Information Measures; Kybernetika, Vol. 21, pp. 148-156.

[18] Taneja H. C. and Hooda D. S. (1983), On Characterization of Generalized Measure of Useful Information; Soochow Journal of Mathematics, Vol. 9, pp. 221-230.

[19] Taneja I. J. (1989), On Generalized Information Measures and their Applications; Chapter in: Advances in Electronics and Electron Physics, Ed.

P.W. Hankes, Academic Press, Vol. 76, pp. 327-413.

[20] Taneja I. J. (1995), New Developments in Generalized Information Measures; Chapter in: Advances in Imaging and Electron Physics, Ed. P.W. Hankes, Vol. 91, pp. 37-136.

[21] Taneja I. J. (2001), Generalized Information Measures and their Applications; On Line Book, http://www.mtm.ufsc.br/ taneja/book.html. [22] Taneja I. J. (2005), On Mean Divergence Measures; On line available at arXiv:math/0501298v2v [math.ST].

[23] Taneja I. J. (2005a), Generalized Arithmetic and Geometric Mean Divergence Measure and Their Statistical Aspects; On line available at arXiv:math/0501297v1 [math.ST].

[24] Taneja I. J. (2005b), On Undefined Generalizations of Relative Jensen-Shannon and Arithmetic-Geometric Divergence Measures, and Their Properties; On line available at arXiv:math/0501299v1 [math.ST].

[25] Taneja I. J. (2013), Seven Means, Generalized Triangular Discrimination, and Generating Divergence Measures; In Information (open acess), Vol. 4, dio: 10.3390/info4020198, pp. 198-239.

[26] Tuteja R. K. and Bhaker U. S. (1993), Mean Value Characterization of Useful Information Measures; Tamkang Jr. Math., Vol. 24, pp. 383-394.