ISYE 6402 Final Exam Review Lessons on Statistical Concepts & Models Questions With Correct Solutions, Already Passed!!

Table of Contents Review Lessons 2 Lesson 1: Basic Statistical Concepts 9 Lesson 2: Basic Statistical Modeling 19 1 Lesson 1: Basic Statistical Concepts I will begin this module by overviewing a series of basic concepts of statistical modeling – the concepts covered in this lesson are essential for understanding the new concepts introduced in this course. Review of Basic Statistical Concepts This is a list of concepts that I'll briefly cover in this lesson. If you do not have a good understanding of these concepts, I recommend you review them before starting on the lectures for this course. The topics that are important for this course include: - Moments of a Distribution which fully characterizes the distribution. - Estimation Methods of parameters in statistical modeling, including method of moments and maximum likelihood estimation. - Basic Estimators for example, for the mean and variance of a random variable given its distribution. - Multivariate distributions along with an understanding of the difference between marginal, conditional, and joint distribution. - And very important for this course, Statistical Inference, including confidence intervals and hypothesis testing. Moments of Distribution What are the moments of a distribution for a random variable X with density function f(x)? The l-th moment of the distribution of X is the expectation of X to the l-th power as defined on the slide. Similarly, we can define the l-th central moment, which is the expectation of X minus the mean of x to the power l. Thus, for this moment, we center the random variable around the mean. Two classic examples of moments are the expectation and the variance. The expectation is the first moment and the variance is the second central moment. Other examples include, the skewness which is the third central moment divided by the standard deviation at the power of 3, measuring how symmetric the distribution of x is. Another example is the Kurtosis which is the fourth central moment divided by the standard deviation at the power of 4, measuring how fat the tails of the distribution are. Here, I described the moments assuming X has a continuous distribution. We can also define moments for discrete distributions. In this class, we primarily focus on continuous distributions. Statistical Estimation Statistical estimation refers to identifying a function of the random data, also called a statistic, to be used in obtaining estimates for one or more parameters, or statistical summaries of a distribution. Thus, first we start with realizations or observations denoted with small letters call them x1, x2, xn. These are realizations of random data, denoted in statistical modeling with capital letters X1, X2 up to big Xn. The random data or random variables, the X’s, are assumed here to have the same distribution, which is f(x) with the parameter theta. The distribution is known up to the parameter theta. That is, the true parameter theta is unknown. Because theta is unknown, we use statistical estimation to estimate it or to obtain approximates for theta that are accurate with respect to the true parameter. Specifically, we develop a function of the data to estimate theta, such that it has good statistical properties, for example, small bias and small variability. Two common approaches in statistics to obtain estimates in statistical estimation are the method of moments and the maximum likelihood approach. Examples of Classical Estimators Some classic estimators commonly studied in the basic statistical courses are the sample mean and the sample variance. Specifically, we begin with data X1 to Xn and estimate a sample mean using the average of the X's, denoted X bar. This is called the sample mean. For the sample variance we first center the X’s by the estimated mean, or sample mean. Then we take the sum of the squared centered data and divide it by n minus 1. We can also construct similar estimator for the skewness and kurtosis using the sample skewness and sample kurtosis as provided on the slide. For both we need to first obtain an estimator for the mean and the variance parameters, given by the sample mean and the sample variance. Then we center the X’s by the sample mean, take the power 3 for Sample skewness and power 4 for the Sample kurtosis and sum them up. We divide those sums by n-1 and the sample standard deviation to the power of 3 for the skewness and power of 4 for the kurtosis. Similar estimators can be constructed for other central moments. Estimators as Random Variables I will note here that we are using empirical data to obtain estimates of parameters or statistical summaries of a distribution; the realizations or empirical data are plugged in these formulas to get the corresponding estimated values for one set of realizations. Because every time we observe from the data, we have a different sample of realizations. We do not get the true values for the statistical summaries, the mean, variance, skewness, or kurtosis, but approximate values or estimates, which will change with each different set of observed data from the random data. Thus, the estimators are random also; they vary with the sample of data used to obtain them. Because of this, we commonly denote the estimators of parameters, or statistical summaries, with hat. For example, mu hat, sigma hat and so on. In this course, I will distinguish between the estimators with hats and the true parameters. We use mu hat for the estimator of mu. And we use the mu without the hat for the true parameter. Sampling Distributions of Estimators Because the estimators of a parameter or statistical summaries are functions of the random data, thus they are also random variables with a distribution. For example, the estimator of the mean is big X bar, which reflects all possible realizations of the sample mean. If the data are normally distributed then the distribution of X bar is also normal with mean mu, the true parameter, mu, and the variance, sigma square divided by n, where sigma square is the true variance of the data X. Moreover, even if the data are not normal, for a large sample size, the distribution of X is approximately normal according to the central limit theorem. The distribution of the estimator for the variance sigma squared is provided as on this slide. S-squared is the sample estimator for the variance. If multiplied by n-1 and divided by the true variance, the sample distribution is a chi-square (χ2) distribution with n-1 degrees of freedom. Based on the sampling distribution of an estimator, we can derive important statistical properties for the estimator. For example, two important statistical properties are Unbiasedness and Consistency. Unbiasedness refers to the property of an estimator, meaning its expectation is exactly equal to the true parameter, whereas consistency means that for large sample data, the estimator is similar to the true parameter, where similarity is in a probabilistic sense.