 Open Access
 Total Downloads : 767
 Authors : Okonkwo Evelyn Nkiruka., Onyeagu Sidney I., Okeke Joseph Uchenna., Nwabueze Joy Chioma., Ogbonna Blessing
 Paper ID : IJERTV1IS9210
 Volume & Issue : Volume 01, Issue 09 (November 2012)
 Published (First Online): 29112012
 ISSN (Online) : 22780181
 Publisher Name : IJERT
 License: This work is licensed under a Creative Commons Attribution 4.0 International License
Comparative Study Of Double Discriminant Analysis And Logistic Regression Based On Binary And Contnuous Variables
By
Okonkwo, Evelyn Nkiruka
Nnamdi Azikiwe University, Awka, Nigeria
Onyeagu, Sidney I.
Nnamdi Azikiwe University, Awka, Nigeria
Okeke, Joseph Uchenna
Anambra State University, Uli, Nigeria
Nwabueze, Joy Chioma.
Micheal Okpala University of Agriculture,Umudike Nigeria
Ogbonna, Blessing
Nnamdi Azikiwe University, Awka, Nigeria
ABSTRACT
In the classification of an observation consisting of both binary and continuous variables, double discriminant analysis and logistic regression had been considered appropriate by most researchers. In this study, these two techniques were extensively discussed and compared using two real life data sets. The average value of PMC for the two data sets showed that logistic regression is optimal to double discriminant analysis in classifying objects whose exogenous variable consist of discrete and continuous variables.
Key words: Double discriminant analysis (DDA), Logistic regression, Regressor, and Probability of misclassification.
1.0 INTRDUCTION
Logistic regression allows one to predict outcome such as group membership from a set of variables that may be continuous, discrete, dichotomous, or a mix (Tabachnick and Fidell, 1996). The problem of discriminant analysis arises when one wants to predict group membership on
the basis of feature vector x. From the above two sentences, it is obvious that the same research questions can be answered by both methods. The logistic regression may be better suitable for cases when the dependant variable is dichotomous such as yes/no, pass/fail, infected/not infected, defective/good life/death, etc., while the independent variable can be on any scale. The discriminant analysis might be better suited when the dependant variable has two groups or more with the requirement that the independent variables will be normally distributed, linearly related, and each group has the same variance and covariance for the variables.
Several authors have formally compared these two techniques. For example, Halperin et al (1971) obtained results from several attributed type explanatory variable and noted only small difference in the classification ability between the two analytic procedures. Press and Wilson (1978) concluded that each analytic technique served a unique function: discriminant analysis was useful for classification of observations into one or two or more populations, whereas logistic regression was useful for relating a qualitative (binary) dependent variable to one or more explanatory variables by a logistic distribution functional form of P(E). Kleinbaum et al (1982) compared classification ability of logistic regression and discriminant analysis and noted that logistic model was slightly superior. Afuecheta et al (2010) compared these two methods on three data sets of normal and non normal data and concluded that logistic regression is the more flexible and more robust method in the case of violation of linear discriminant assumptions.
The objective of this paper is to compare the performance of double discriminant function obtained using pointbiserial model developed by Chang and Afifi (1974) , and logistic regression based on binary and continuous explanatory variables for classifying subjects into one of two populations.
2.0 DOUBLEDISCRIMINANT ANALYSIS
An observation consisting of both binary and continuous variables may be classified into one of two populations by the doublediscriminant function based on the pointbiserial model. When the parameters are unknown or partially known, a sample doublediscriminant function is obtained by replacing the unknown parameters by their sample estimates.
Suppose an observation
W X
Y
is to be classified into one of two
populations i , i= 1,2 where Y is px1 vector of continuous variates and X is a
Bernoulli variate with P(X=1)
i
and P(X=0) = 1i
if W belongs to i . We
assume that W follows a pointbiserial model, that is, that the conditional
distribution of Y given X x
(0 or 1 ) is
N (ix, x ) when W i,
where x
is a
pxp positive definite matrix.. Under the pointbiserial model, and given X x , the likelihood ratio procedure is to classify the observation W into 1 if
C [Y 1
]' 1
k
x 2 1x
2 x x
1x 2 x x
(2.1)
1
x
1 x
where
x ln
2
(1
1 ) / 1
2 ,
x = 0,1, and k is a given constant.
Otherwise the observation is classified into 2
.The discrininant function Cx
in (2.1) is called the doublediscriminant function by Chang and Afifi (1974).
If the parameters are unknown, we may replace them by their sample estimates. Let
Xij
Wij Y
, i 1, 2and j 1, 2,…Ni
ij
be two sequences of observation vectors
independently drawn from
1 . We shall add a subscript x on
Y ij
and Ni to
indicate those values corresponding to Xij x . The sample double
discriminant function is defined according to whether x
is known or
unknown. When x
is known but
ix,
and
x are unknown, the sample
doublediscriminant is
T [Y 1 Y Y ]' S 1 Y Y
k
where
x 2 1x 2 x x
1x 2 x x
(2.2)
Y N
Nix
1 Y ,

ix ijx j 1
i
N
Ni1 Xij
j 1
Ni0Ni Ni1,
2 Nix
S M
1 Y
Y Y
Y ',
x x ijx ix ijx ix
i1 j 1
Mx N1x N2x 2.
Ni is number of observations in population i, and
Nix is the number of
observations in x(1or 0) class of ith population.
When all the parameters are unknown, the doublediscriminant function is
U [Y 1 Y Y ]' S 1 Y Y
k

2 1x 2 x x
1x 2 x x
(2.3)
N N
x] ln
1x 2
N N
2 x
1 .
It can be easily shown that Tx
and
U x are invariant under any nonsingular
linear transformation of the Ys.
As sample sizes
N tend to infinity,Yix ix ,
Sx x , and
x x in
ix
probability. Hence the limiting distribution of Tx
tends to that of Cx
(Tu
N 1 D2 , D2 N 1 D2 , D2
2 x x x 2 x x x
1978),
or
depending on whether W comes
from 1 or 2 ,where
D
2
x = ' 1
(2.4)
1x 2x x 1x 2x
It should be noted that some of the
N1x
may assume small values if i is
close to zero or one. Effort should be to ensure that parameters).
N1x
> p (the number of

Logistic Regression
Logistic regression is part of a category of statistical models called generalized linear models (Agresti, 1996). Logistic regression, more commonly called logit model, deals with the binary case, when the response variable is dichotomous (i.e., binary 0 or 1). The predictor variables may be quantitative, categorical, or a mixture of the two. This model is mainly used
to identify the reltionship between one or more explanatory variables Xi and response variable Y. It has been used for prediction and determining the most influencing explanatory variable(s) on the variables (Cox and Snell, 1994). Instead of a straight line, logistic regression seems preferable to fit some kind of sigmoid curve to the observed points. The tails of sigmoid curve level off before reaching P(E) =0 or P(E) = 1, so that the problem of impossible values of P(E) is avoided.
The basic form of the logistic function is
1
P 1e Z
(3.1)
where Z is the predictor variable(s) and e is the base of the natural logarithm, equal 2.71828 ,P is estimated probability of event occurring.
In the multivariate case, Z instead of being a single predictor variable, is a linear function of a set of predictor variables:
Z b0 b1 X1 b2 X2 … bp X p . (3.2)

One Regressor
Assuming that we have a single regressor, let us try to write a simple linear regression model as
Y 0 1 X
(3.3)
We would logically let Yi = 0 if the unit does not have the characteristic that Y represents, and Yi = 1 if the unit does have the characteristic.
It thus follows that i
can also take on only two values:
1 0 1 X if
Yi = 1 and – 0 1 X
if Yi = 0. Therefore i
cannot be approximately
normally distributed. Consequently, the model (3.3) is inapplicable for a binary dependent variable.
In simple linear regression, the starting point in determining a model is a scatter plot of Y. Consequently; it is necessary to consider other plots. One such plot is a plot of E(Y\X) against X. Rather than plotting points, we must postulate a relationship between Y and X variables since the ordinate of the
plot is not related to the data. It is customary to let E(Yi\Xi) = i ,which is
P(Yi = 1), where Y is binomial. Here i
represents the probability of, for
example, someone dying within a stated time period who has B.P given by
Xi. Given one regressor the probability of an event, say E, for a given value of X, P(E) = i ( X ) is
X
exp 0 1 X
i 1 exp
0 1 X
(3.4)
The model given by (3.4) satisfies the important requirement that 0 i 1 and will be satisfactory model in many applications. The model in terms of Y would be written as
Y X
It follows from (3.4) that
exp
X
so 1 0 1
log X
1 0 1
(3.5)
Since (3.5) results from using a logistic transform (also called a logit transform), the model is called a logistic regression model. The left side of (3.5) is called log odds ratio, and this can be explained as follows. Since =
P(Y=1), it follows that 1 = P(Y=0), and so 1
is the ratio of the two
probabilities, which, when stated in the form the of odds, gives the odds of having Y=1, for a given value of X. Odds are frequently stated in terms of against rather than for so the odds against having Y=1 would be
1
obtained from .
The absence of error on the right side of (3.5) is because the left side is a function of (Y/X), instead of Y, which serves to remove the error term.
The interpretation of
1 is naturally somewhat different from the
interpretation in the linear regression. In (3.5)
1 obviously represents the
amount by which the log odds change per unit change in X. This implies that a unit increase in X increases the odds by the multiplicative factor e1 .

DATA AND THEIR ANALYSIS

The Data for this study are two real life data sets. One set was collected from Amaku General Hospital, Awka, Anambra state, Nigeria. The data is on fasting and nonfasting blood sugar level (FBS and NFBS respectively)
of diabetics and nondiabetics patients with their gender randomly selected from the cases reported at the Hospital in 2009.
The laboratory reference ranges for adults are:
Glucose (fasting): 9 4mmol/l and Glucose (nonfasting):4 8mmol/l
The other set is on four albino CD1 SpragueDawley rats at the weaning age with similar weights. The data is available in Tu and Han (1982).

FINDINGS

Result of diabetic and nondiabetic patient data
Double discriminant analysis
The sample double discriminant functions (DDF) are:
T0 = 1.47732y1 + 1.32279y2 19.55786 for male. T1 = 1.33301y1 + 2.04684y2 24.06988 for female.
The above two sample double discriminant functions when applied on the original data gave the probability of misclassification as: 0.05
Result of Logistic Regression
The logistic regression model for the data is
Z= – 960.615 + 125.379y1 +26.1447 y2 13.6178x
The p value of the model is: 0.0000
The probability of correct classification is 1.00.
Here y1=fastings blood glucose level, y2= nonfastings blood glucose level, and x = sex

Result of four albino CD1 SpragueDawley rats
The sample double discriminant functions (DDF) are:
T0 = 0.020544y1 + 0.04306y2 7.585704 for male. T1 = 0.040258y1 + 0.214881y2 + 3.596135 for female.
The above two sample double discriminant functions when applied on the original data gave the probability of misclassification as: 0.327
Result of Logistic Regression
The logistic regression model for the data is
Z= 6.93092 0.0228665y1 + 0.0044232 y2 2.67589X
The p value of the model is: 0.0000
The probability of correct classification is 0.74 with ties of 0.04 and the probability 0f misclassification is 0.22
Here y1=body weight, y2= total length, and x = sex
Summary of findings
The frequencies of misclassifications are listed in the following table. Table 4.1
Sample data 
DDF 
Logistic 
Diabetic and nondiabetic patients 
0.05 
0.00 
Albino CD1SpragueDawley rats 
0.327 
0.26 
Average 
0.1885 
0.13 

CONCLUSIONS AND DISCUSSION.
Double discriminant analysis (DDA) and logistic regression are used when the observations of exogenous variable consist of binary and continuous
variables with dichotomous dependent variable. We gave extensive discussion on the similarities and dissimilarities of the two methods in the literature. From the average value of probability of misclassification of 0.1885 and 0.13 for DDA and logistic regression analysis respectively, we can conclude that logistic regression is optimal to DDA in classifying objects whose exogenous variable consist of discrete and continuous variables.
Reference

Afuecheta, E.O., Ogum, G.E.O., Osuji, G.A., and Utazi, C.E. (2010). Comparison of Linear Discriminant Analysis and Logistic regression in Classification Problems. Conference Proceedings Nigeria Statistical Association, 8189.

Agresti, A. (1996). An introduction to Categorical Data Analysis. John Wiley& Sons, Inc., New York.

Chang, P.C., and Afifi, A. A.(1974). Classification Based on Dichotomous and Continuous Variables. Journal of the American Statistical Association, 69: 336339.

Cox, D.R., and Snell, E.J. (1994). Analysis of Binary Data. Chapman& Hall, London.

Halperine, M., Blackwelder W.E., and Verter, J.I (1971). Estimation of the Multivariate Logistic Risk Function and Maximum Likelihood Approach. Journal of Chron. Dis. 24:125158.

Kleinbaum D.G., Kupper, L.L., and Morgenstern, H. (1982). Epidemiology Research: Principles and Quantitative Methods.
Toronto: Lifetime Learning 461470.

Krzanowski, W.J. (1993). Priciples of Multivariate Analysis. Oxford University Press Inc., New York. 337345.

Press, S.J., and Wilson, S. (1978). Choosing Between Logistic Regression and Discriminant Analysis. Journal of American Statistical Association 73:699705.

Tabachnick, B.G. and Fidell, L.S. (1996). Using Multivariate Statistics Harper Collins, New York.

Tu, C.T. (1978). Discriminant Analysis Based on Binary and Continuou Variables Unpublished Ph.D Dissertation, Iowa State University.

11 Tu, C.T., and Han C.P. (1982). Discriminant analysis Based on Binary and continuous variable. Journal of American
Statistical Association 77:447454.