This course covers fundamental topics in probability and statistics. Topics in probability include: random variables, probability distributions, conditional distributions, joint distributions, moments, covariance, correlation, conditional expectation, central limit theorem. Topics in statistics include: statistical models, Bayesian estimation, conjugate priors, maximum likelihood estimation, confidence intervals, hypothesis testing, p-values, t-test, goodness of fit, simple/multiple linear regression, hypothesis testing on regression coefficients. Throughout the course, we will encounter various special classes of probability distributions, and learn the typical random processes that each distribution often describes.
At the end of the term, students will be able to:
- Apply key concepts of probability, including Bayes’ theorem, discrete and continuous random variables, probability distributions, conditioning, independence, expectations, and moments.
- Define and use different probability distributions (e.g., binomial, Poisson, exponential, normal) and the typical phenomena that each distribution often describes.
- Understand joint vs marginal vs conditional distributions of multiple random variables.
- Compute the covariance and correlation of two random variables.
- Apply the law of large numbers and the Central Limit Theorem (CLT).
- Define and demonstrate the concepts of estimation for statistical inference.
- Use conjugate priors to simplify Bayesian estimation.
- Apply the concepts of interval estimation and confidence intervals.
- Design hypothesis tests, select appropriate thresholds for the tests, and compute p-values.
- Design linear regression models and apply hypothesis testing on regression coefficients.
- Evaluate probabilities and conditional probabilities.
- Compute probability mass/density functions, cumulative distribution functions, moments, moment generating function, and conditional expectations of random variables.
- Determine independence, covariance, correlation of two random variables.
- Approximate the distribution of sum of random variables using CLT.
- Construct estimators using Bayesian estimation and maximum likelihood estimation.
- Perform hypothesis tests, determine significance level, and compute p-values.
- Compute regression lines and perform hypothesis tests on regression coefficients.
- Basic concepts of probability (events, axioms of probability, sample space, conditional probability, independence, Bayes theorem, etc.)
- Random variables, probability distributions, pmf, pdf, cdf, joint distributions, conditional distributions, independence, expectation, variance, moments, moment generating function
- Special classes of probability distributions: Bernoulli, binomial, geometric, exponential, Poisson, normal, bivariate normal, gamma, beta, chi-squared, t-distribution
- Bivariate distributions, covariance, correlation, conditional expectation
- Law of large numbers, central limit theorem, convergences
- Statistical models, Bayesian estimation, conjugate priors, maximum likelihood estimation, statistical inference, confidence intervals, sampling distributions of estimators
- Hypothesis testing, type I/II errors, power, t-test, p-value, two-sample t-statistic, chi-squared test of goodness of fit
- Least-squares method, simple/multiple linear regression, hypothesis testing on regression coefficients
Textbook(s) and/or Other Required Material
- J. L. Devore, Probability and Statistics for Engineering and the Sciences, 8th ed. Boston, MA: Brooks/Cole, 2011.