Comparison of Optim, Nleqslv and MaxLik to Estimate Parameters in Some of Regression Models

Our main goal in this article is to present the approaches and examples of three functions in R consist of optim, nleqslv and maxLik function to detect the optimization solution of the estimating function in the regression models. We then compare the results with numerous sample sizes (n=150, 300 and 500), the execution time of R code, as well as Normal Q Q plots of three approaches through some of regression models such as the zeroin ated Binomial (ZIB) regression model, logistic regression model, the zero-in ated Poisson (ZIP) regression model and the zero-in ated Bernoulli (ZIBer) regression model. Finally, we discuss potential research directions in the coming times.


Introduction
The R software is developed by Ihaka and Gentleman [15] and it continues to be developed by the R Development Core Team. Software R is one of the statistical analysis tools as well as data analysis in general. In the past 10 years, R has been largely utilized by several universities around the world. This is open source software (free). It carries all the features of other existing commercial software such as SPSS, AMOS, STATA or EViews. About this regard, readers may refer in (see e.g. [3], [4], [5], [7], [11], [19], [20], [24], [26], [27], [28] and [33] etc.) The R software contains numerous types of statistical techniques and graphics. The R software, like S, is designed around a real machine language, and it allows users to add additional features by dening new functions. S software comes before R software. S is a statistical programming language developed primarily by Chambers [6]. In this time, two books were published by the research team at Bell Laboratories: S: An Interactive Environment for Data Analysis and Graphics [1] and Extending the S System [2]. Hence there are also some vital dif-532 c 2019 Journal of Advanced Engineering and Computation (JAEC) VOLUME: 3 | ISSUE: 4 | 2019 | December ferences for S, but numerous codes are written in S can run without modication. Several systems in R are written in its own language, making it easy for users to follow algorithms. In order to perform computational tasks, R can connect to C, C ++ and Fortran languages to be called at runtime. Procient users can write C code to directly handle R software objects.
In several elds of statistical inference, social science and numerous other areas, the issues in- For the available function in R to nd the optimization solutions is very diverse and abundant.
For instance, The optim function is oered by Nash [21]. Eberhart and Kennedy [10] introduce the Particle Warm Optimization (PSO) algorithm. Ter [25] provides the genetic algorithm Dierential Evolution (DE) for Markov Chain Monte Carlo. The nleqslv function is developed by Hasselman [12]. The maxLik function is proposed by Henningsen and Toomet [13]. Scrucca [23] presents to a package for Genetic Algorithms (GA), etc. It has been seen that, there are many packages in R to nd the optimization solutions. gives some concluding remarks of these functions being discussed in our paper.

Literature review
In this section, we rst revisit of the optimization problem and we then revisit of three functions in R include optim, nleqslv and maxLik function to detect the optimization solution of the estimating function in the regression models.
We now discuss about the optimization problem in the next section.

Optimization problem
Let g(u) be a function of a vector u. Optimization problems are related to the task of nding u * such that g(u * ) is a local maximum (or minimum). In the case of maximization and in the case of minimization, u * = argmin g(u) It will be known that statistical estimation prob- We now turn on discuss to three functions in R such as optim, nleqslv and maxLik function.
We rst present to optim function in the next sub-section.

Optim function
The optim function is a widespread available function to nd the optimization solution of the estimating function in the regression models.
The optim function is rst introduced by Nash [21]. This function is already included in R by default, we just need to type the function name and the right structure to use it.
The simplest structure of the optim function can be described as follows: where start is a precursory value for the parameters to be estimated and g is the objective func-

Solution
It can be seen that Choosing (-1,1) vector is an initial value and perform the statistical software R to write code: If we use the simplest formula of optim function, optim(c(−1, 1), g), then the result is provided as follows: In addition, we also use the derivative of the objective function in the formula of optim function. We now present about the nleqslv function in the next sub-section

Nleqslv function
Likewise the optim function, the nleqslv function is an ubiquitous available function in R. The nleqslv function is rst developed by Hasselman [12]. The simplest structure of the nleqslv function can be illustrated as follows: where start is a precursory value for the parameters to be optimized and g is the objective function. The complete formula can be expressed as follows: optim(start, g, jac = NULL, method, where jac is a function to return the Jacobian for the g function. If not supplied numerical derivatives will be used. For method, we can choose method =Broyden" or method =Newton", jacobian is a jacobian matrix of an objective func- Example 2. Assuming that one needs to employ the nleqslv function to detect the maximum value of the following system of equations: Choosing (2,0.5) vector is an initial value and perform the statistical software R to write code: If we use the simplest formula of nleqslv function, nleqslv(c(1.5, 1), g), then the result is provided as follows: To see the jacobian matrix in the result, one only needs to add jacobian = TRUE" in the above formula.
We now present about the maxLik function in the next sub-section

MaxLik function
Similarly to optim function and nleqslv function, the maxLik function is a popular available function to get the optimization solution of the estimating function in the regression models. The maxLik function is rst proposed by Henningsen and Toomet [13]. The simplest structure of this function is given by: where logLik is the log-likelihood function of an objective function and start is a precursory value of parameters need to be estimated.
The complete formula can be written as follows: where grad is a gradient of an objective function.
If NULL, numerical gradient will be executed, hess is a hessian matrix of an objective function. If NULL, numerical Hessian will be performed, method : we can select NM" (Nelder-Mead), CG" (Conjugate Gradients), BFGS" is a quasi-Newton method (Broyden-Fletcher-Goldfarb-Shanno), etc, constraints : if we can select NULL for unconstrained maximization.
It has been seen that: To perform a maxLik function in practice, one needs to have an objective function (a log-likelihood function) and an initial value (real value or vector). To help readers easily perform the maxLik function in practice, we provide an example about this issue. We next investigate an example in paper of Henningsen et al. [13].
Example 3. Supposing that u is drawn from N (2, 3) and the sample size is 1000. Find the parameters are estimated from this distribution.
Generating the data set as follows: Let X i1 = 1, X i4 ∼ N (−1, 1) and X i5 ∼ B(1, 0.5) are independently. In this model, assuming that X i2 = W i2 and When n = 300, selecting (k 4 , k 5 , k 6 ) = (120, 100, 80) and with n = 500, using (k 4 , k 5 , k 6 ) = (200, 170, 130). Note: bias is the distance between the estimator and its true value, SD is the standard deviation, ASE is the asymptotic standard error, and CP is the coverage probability of 95% level condence intervals. All results are simulated with the number of repetition is N = 1000.
The log-likelihood function of logistic regression model has the form For this example, we also investigate the results with several sample sizes (n =150, 300 and 500),

Conclusions
In  550 "This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium provided the original work is properly cited (CC BY 4.0)."