Machine Learning for Robotics syllabus

Introduction to Machine Learning and PAC Framework: Definition and scope of Machine Learning, Supervised learning, Unsupervised learning, Semi-supervised learning, Reinforcement learning, PAC (Probably Approximately Correct) Learning framework, Hypothesis space, Sample complexity, Generalization error vs. training error, Bias-variance tradeoff. Robot Architecture for Learning: Random variables and probability distributions, Conditional probability and Bayes' theorem, Expectation, Variance and covariance, Maximum Likelihood Estimation (MLE) basics, Concept learning and hypothesis spaces, Find-S algorithm, Candidate Elimination algorithm, General-to-specific ordering and boundary sets.

Linear and Multilinear Regression: Predicting continuous outputs using regression, Simple Linear Regression, MSE cost function, Closed-form solution, Multilinear Regression, Gradient Descent (Batch, Stochastic, Mini-batch), Normal equation method, Ridge (L2) regularization, Lasso (L1) regularization, Overfitting and underfitting. Naïve Bayes and Decision Trees: Conditional independence assumption, Bayes' rule for classification, Gaussian, Multinomial and Bernoulli Naïve Bayes, Laplace smoothing, ID3 algorithm using Information Gain and Entropy, CART algorithm using Gini Index, Pre-pruning and post-pruning, Error bounds and generalization.

Instance-Based and Linear Classifiers: K-Nearest Neighbors (K-NN), Euclidean, Manhattan and Minkowski distances, Curse of dimensionality, Weighted K-NN, Logistic Regression, Sigmoid function, Cross-entropy loss, Softmax regression, Single-layer perceptron, Perceptron convergence, XOR problem limitation. Multi-Layer Perceptrons and Support Vector Machines: Multi-Layer Perceptrons (MLP) and hidden layers, Backpropagation algorithm, ReLU, Tanh, Sigmoid and Leaky ReLU activations, Vanishing and exploding gradients, Linear SVM (Hard margin, Soft margin, Hinge loss), Nonlinear SVM using Kernel trick (Polynomial, RBF, Sigmoid kernels), Support vectors.

Clustering Fundamentals and K-Means: Partitional clustering (K-Means, K-Modes), Hierarchical clustering (Agglomerative, Divisive, Dendrograms), Density-based clustering (DBSCAN with eps and minPts), K-Means algorithm steps, Elbow method and Silhouette score, Self-Organizing Maps (SOM) for topological mapping. Expectation Maximization and Principal Component Analysis: Soft vs. hard clustering, Gaussian Mixture Models (GMM), Expectation Maximization (EM) algorithm (E-step and M-step), Dimensionality reduction using PCA, Eigenvalues and eigenvectors, Principal components and explained variance, Reconstruction error, Applications of PCA.

Evaluation Metrics and Significance Tests: Confusion Matrix (TP, TN, FP, FN), Accuracy, Precision, Recall, F1-Score, ROC Curves and AUC, MSE, RMSE, R-squared, k-fold cross-validation, Learning curves. Ensemble Learning and ML Process in Practice: Bagging and Random Forests (bootstrapping, Outof-bag error), Boosting (Adaboost, XGBoost), Data preprocessing (handling missing values, Normalization, Encoding), Outlier analysis using Z-Score, Hyperparameter tuning (Grid Search, Random Search), Train-validation-test split, Visualization of results.