Theory summary: Data mining

Complete theory summary for the open-book theory part of the Data Mining Important: this document contains only the THEORY part of the course. It does not represent the complete practical part of the course. The theory section of the exam was open book, and I personally used this summary during the exam as my reference document. The summary is structured to make concepts easy to locate quickly during an open-book exam and contains: Chapter 1: Introduction Big Data and the 4 V’s, data and feature types, dataset types, data mining, data mining vs statistics, supervised vs unsupervised learning, classification, regression, pattern discovery and outlier detection. Chapter 2: Processing Principles Structured and unstructured data, feature extraction, transformations, log transformation, normalization, discretization, aggregation, noise and outlier removal, sampling, duplicate data, missing values (MCAR/MAR/MNAR), dimensionality reduction, PCA/SVD and processing of images, surveys, sequences, molecular structures, networks and text. Chapter 3: Univariate Techniques Differential analysis, hypothesis testing, t-tests, parametric vs non-parametric testing, false positives and false negatives, FDR/FWER, multiple-testing corrections, Bonferroni, Hochberg, Benjamini-Hochberg, GWAS, ORA and GSEA. Chapter 4: Unsupervised Clustering Clustering principles, similarity and distance measures, dendrograms, hierarchical clustering, linkage methods, partitioning clustering and k-means. Chapter 5: Principal Component Analysis Multivariate data, covariance and correlation, data projection, PCA, eigenvectors/eigenvalues, loadings and scores, scree plots, PCA examples, t-SNE and UMAP. Chapter 6: Linear Models Simple and multiple linear regression, supervised learning, ANOVA/ANCOVA, mixed models, generalized linear models, nonlinear regression and logistic regression. Chapter 7: Molecular Data Analysis Quantitative analysis, omics data matrices, transcriptomics, RNA-seq, differential analysis, gene-expression downstream analysis, proteomics, protein identification, mass spectrometry, quantitative LC/MS, metabolomics and flow cytometry.