Probability & Statistics

Sample spaces, Events, Axiomatic definition of Probability. Permutations and Combinations in complex counting problems. Conditional Probability, Independence of events, and Total Probability Theorem. Bayes’ Theorem and its applications in classification and diagnostic systems (Naïve Bayes foundation). Introduction to Random Variables: Definition, Cumulative Distribution Functions (CDF), and Probability Mass/Density Functions (PMF/PDF).

Theoretical distributions essential for data modeling. Discrete: Bernoulli, Binomial, Geometric, and Poisson (modeling rare events/cyber attacks). Continuous: Uniform, Exponential (reliability/waiting times), Gamma, Beta, and **Normal (Gaussian) Distribution** (properties and the Empirical Rule). Expectation, Variance, Skewness, Kurtosis, and Moment Generating Functions (MGF). Transformation of Random Variables.

Jointly distributed random variables (Discrete and Continuous). Marginal and Conditional distributions. Independent Random Variables. Covariance, Correlation Coefficient, and Rank Correlation. Information Theory for ML: Self-Information, Entropy (Shannon), Joint and Conditional Entropy, Mutual Information, and **Kullback-Leibler (KL) Divergence** (foundations for Cross-Entropy Loss in Neural Networks).

Sampling distributions: Chi-square, t-distribution, and F-distribution. Central Limit Theorem (CLT) and Law of Large Numbers. Parameter Estimation: Point estimation using **Maximum Likelihood Estimation (MLE)** and **Maximum A Posteriori (MAP)** (Bayesian approach). Interval Estimation: Confidence Intervals for mean, variance, and proportions. Hypothesis Testing: Null/Alternative hypotheses, Type I/II errors, p-values, Z-test, t-test, and ANOVA.

Introduction to Stochastic processes. Markov Chains: State space, Transition Probability Matrix (TPM), Chapman-Kolmogorov equations, Stationary distributions, and Ergodicity (Mathematical foundation of **Reinforcement Learning**). Random Walks and Poisson Processes (Network traffic modeling). Introduction to Monte Carlo Methods and Sampling techniques.