Matrix Algebra And Ols, And Simulations Of Key Properties Of The Mean

PS 200C - Problem Set 1 Chad Hazlett Due April 6, 11:59pm This problem set is intended to (a) help remind you of some essential material which you are expected to know; (b) help you to clarify whether you have the necessary background for the course. Instructions: • Responses should be typeset in something nice, like TeX or Rmarkdown. • Submit your completed problem set as a single PDF via the course website. If you are not using RMarkdown or similar, please include a copy of your code in your write-up (e.g. using the verbatim environment). If in doubt about formatting issues, please check with the TA. 1. Probability Theory I: Events The 1982 (original) movie Blade Runner (starring Harrison Ford) is set in a future world where there are robots that are designed to look and behave exactly like human beings. The only way to tell if a randomly selected individual (who appears to be human) is in fact a human being or a robot is to administer a test. In our version of the test, suppose that if an individual taking the test is in fact a robot, the test will report that the individual is a robot 95% of the time. If an individual taking the test is not a robot, the test will report that the individual is a robot 3% of the time. Based on the number of robots manufactured, we can estimate that about 2% of all individuals we would test are actually robots. a) (4pts) What is the probability that a randomly selected individual given the test will be classified as a robot? b) (4pts) Given that an individual is classified as a robot by the test, what is the probability that individual is actually a robot? c) (4pts) Given that an individual is classified as a human by the test, what is the probability that individual is actually a robot? 2. Probability Theory II: Random Variables a) (2pt) Consider continuous random variable X with probability distribution p(X).1 How is E[X] defined? (Give the definition, not an estimator you’d use given a sample). b) (2pt) How is Var(X) defined? c) (2pt) Further suppose X and Y have joint density p(X,Y ). How is E[Y |X] defined? (Write it out in terms of an integral and density function). 1In problem sets we will maintain the notation used in class, in which p(Z) is a density function for random variable Z. Note that we use p instead of f , and that we use this for both probability density functions and probability mass functions. Further, we drop the subscript and assume that the density is for the random variable referenced in the parentheses (i.e. fX(X) is simply p(X)) 1 d) (2pt) If X and Y were independent, what does E[X|Y ] reduce to? (Show why this is, writing out the definition of E[X|Y ] first). For the following questions, draw random variables X1, X2,..., XN , all independently from p(X). e) (2pt) Suppose you have scalars, a, b, c. What is E[aX1 + bX2 + cX3]? What is Var[aX1 + bX2 + cX3]? f) (4pt) Let X = 1 N N∑ i Xi. Is X unbiased for E[X]? Prove it. (Do not just cite a theorem!) g) (4pt) Derive the variance of X. What happens to it as N →∞? 3. Matrix Algebra and OLS Consider random variables Y ∈ R and X ∈ RP, drawn from joint density p(X,Y ). You collect a sample of draws from this distribution, {(Y1, Xi), ..., (YN , XN )}. Let X be a N × (1 + P ) matrix, with row i equal to [1 X>i ] (i.e., there is an intercept and then a column for each “covariate’ ’). Consider a model, Y = Xβ + �, where we assume E[�|X] = 0. a) (5pt) Using matrix notation at each step, derive the ordinary least squares estimator for β: βOLS = argmin β∈RP +1 (Y−Xβ)>(Y−Xβ) b) (5pt) Show R code that would achieve the following (there is no need to submit this code in a separate file; just include it in your problem set write-up using an environment such as verbatim or a non-evaluated code chunk in Rmd, etc.): i. Construct a matrix X to represent X in the above, with N = 100, one column of ones, and two columns of randomly drawn numbers (from any distribution you like). ii. Using β = [1 2 3]>, compute vector Y equal to Xβ + �, where � is drawn from a standard normal distribution. iii. Compute (X>X)−1(X>Y ). Use the solve function in R. iv. Compare the result to the coefficients obtained using lm with the data you have constructed. c) (5pt) Show the unbiasedness of β̂OLS for β. (Hint: compute β̂OLS , but replacing Y with Xβ + �). d) (5pt) Compute the variance, with X taken as given, i.e. V[β̂OLS |X], again sticking with matrix notation. You may assume E[��>|X] = σ2IN , where IN is the N ×N identity matrix. e) (2pt) What meaning would you give to the matrix E[��>|X]? Give an intuitive explanation of what the assumption that this matrix equals σ2I implies. 4. Some Statistics Review True or false? For credit, explain your choice briefly. a) (3pt) If there is perfect collinearity, the OLS estimator will give biased and inconsistent estimates. b) (3pt) A very large p-value for the estimated coefficient for an explanatory variable provides strong evidence that the variable has zero effect on the outcome. c) (3pt) If an estimator is unbiased it is also consistent. 2 Giovanni Castro Giovanni Castro Giovanni Castro Giovanni Castro d) (3pt) You have a model Yi = X>i β + �, and you fit it by OLS. If the OLS residuals are uncorrelated with X, our estimate of β are unbiased. 5. Simulations of Key Properties of the Mean Suppose X1 ∼ N(5, 2) and X2 ∼ exp(λ = 1) (where exp indicates the exponential distribution). In R, construct two vectors, X1 with 10000 draws from the same distribution as X1, and X2 with 10000 draws from the same distribution as X2. We will take sub-samples from these two variables to evaluate the coverage probability of 95% confidence intervals using different types of data and different sample sizes. (a)(3pt) Describe the distribution of X1 and X2 using your favorite graphical approach for looking at continuous distributions. Mark the true expectation on your plot using a line. (b)(4pts) Consider for a moment the random variables X1 and X2, representing sample means you could get from taking the average of 8 randomly sampled values of X1 or X2 respectively. Using math (not R), give solutions for: • E[X1]? • E[X2]? • V ar(X1)? • V ar(X2)? (c)(5pts). Now, for X1, draw 5000 samples of size N=6. Get the mean and compute the 95% confidence interval each time. What portion of the confidence intervals you computed include the true expectation? Repeat this for X2. (d)(5pt) Repeat (c) for samples of size 6, 20, 50, and 500. Report the coverage probability for each of your eight simulations in a table. How do your results change? What differences do you see between X1 and X2? (e)(2pt) Explain your findings in parts (c) and (d). 3 Giovanni Castro Giovanni Castro