6.2 Supervised vs. Unsupervised Learning

Machine learning comes in two main flavors: supervised and unsupervised. Supervised learning uses labeled data to train models that can make predictions. Unsupervised learning finds hidden patterns in unlabeled data without predefined outputs.

Choosing between them depends on your data and goals. Supervised learning is great for specific predictions, while unsupervised learning uncovers underlying structures. Understanding their differences helps you pick the right approach for your machine learning project.

Supervised vs Unsupervised Learning

Key Differences

Top images from around the web for Key Differences

• 

  Supervised learning - Wikipedia View original

  Is this image relevant?

• 

  Lab 10 - Machine Learning [CS Open CourseWare] View original

  Is this image relevant?

• 

  Introduction to generative adversarial network | Opensource.com View original

  Is this image relevant?

• 

  Supervised learning - Wikipedia View original

  Is this image relevant?

• 

  Lab 10 - Machine Learning [CS Open CourseWare] View original

  Is this image relevant?

1 of 3

Top images from around the web for Key Differences

• 

  Supervised learning - Wikipedia View original

  Is this image relevant?

• 

  Lab 10 - Machine Learning [CS Open CourseWare] View original

  Is this image relevant?

• 

  Introduction to generative adversarial network | Opensource.com View original

  Is this image relevant?

• 

  Supervised learning - Wikipedia View original

  Is this image relevant?

• 

  Lab 10 - Machine Learning [CS Open CourseWare] View original

  Is this image relevant?

1 of 3

• Supervised learning trains models on labeled data with known outputs to make predictions on new data
• Unsupervised learning works with unlabeled data to discover hidden patterns and structures
• Supervised learning maps input features to output labels while unsupervised learning finds inherent data structures
• Goal of supervised learning involves generalizing from training data for accurate predictions
• Unsupervised learning aims to uncover underlying patterns or groupings in the data
• Supervised learning requires labeled training datasets (can be time-consuming and costly to obtain)
• Unsupervised learning uses raw unlabeled data
• Performance evaluation for supervised learning uses direct metrics (accuracy, mean squared error)
• Unsupervised learning often relies on indirect evaluation methods

Data and Model Characteristics

• Supervised learning utilizes labeled training data with known outputs
• Unsupervised learning works with unlabeled data lacking predefined outputs
• Supervised models learn mapping functions from inputs to known outputs
• Unsupervised models identify inherent structures without output labels
• Labeled data for supervised learning can be expensive and time-intensive to obtain
• Unsupervised learning can leverage larger amounts of raw unlabeled data
• Supervised models directly optimize predictive performance on labeled examples
• Unsupervised models aim to capture underlying patterns and relationships

Evaluation and Applications

• Supervised learning performance measured by prediction accuracy on test data
• Unsupervised learning evaluated indirectly (cluster quality, dimensionality reduction)
• Supervised learning suited for predictive modeling (sales forecasting, image classification)
• Unsupervised learning used for exploratory analysis and pattern discovery
• Supervised models can make precise predictions on new data points
• Unsupervised models provide insights into data structure and groupings
• Supervised learning requires domain expertise to label training data
• Unsupervised learning can reveal unexpected patterns with minimal assumptions

Supervised Learning Tasks and Algorithms

Classification Tasks

• Classification predicts discrete class labels for new instances based on labeled training data
• Binary classification involves two possible class outcomes (spam vs not spam)
• Multi-class classification handles multiple possible class labels (animal species)
• Common classification algorithms include:
  • Logistic regression for probabilistic binary classification
  • Decision trees that partition the feature space into regions
  • Random forests combining multiple decision trees
  • Support vector machines (SVM) finding optimal decision boundaries
  • Neural networks with multiple layers for complex classifications
• Ensemble methods combine multiple classifiers to improve overall accuracy:
  • Bagging creates multiple training sets by sampling with replacement
  • Boosting iteratively improves predictions by focusing on difficult examples

Regression Tasks

• Regression predicts continuous numerical values based on input features and labeled examples
• Linear regression models relationships as linear combinations of input features
• Polynomial regression fits non-linear relationships using polynomial terms
• Popular regression algorithms include:
  • Simple linear regression with one input variable
  • Multiple linear regression with multiple input variables
  • Ridge regression adding L2 regularization to prevent overfitting
  • Lasso regression using L1 regularization for feature selection
  • Gradient boosting machines combining weak learners sequentially
• Time series forecasting predicts future values based on historical time-dependent data:
  • ARIMA models for stationary time series
  • Prophet for business forecasting with seasonal patterns

Model Optimization

• Hyperparameter tuning optimizes model settings not learned during training:
  • Grid search exhaustively tries combinations of hyperparameters
  • Random search samples hyperparameter space more efficiently
• Cross-validation assesses model performance and generalization:
  • K-fold cross-validation partitions data into k subsets for validation
  • Leave-one-out cross-validation for small datasets
• Regularization techniques prevent overfitting:
  • L1 regularization (Lasso) encourages sparse feature selection
  • L2 regularization (Ridge) shrinks coefficients toward zero
• Learning curves diagnose bias-variance tradeoff:
  • Plot training and validation error vs training set size
  • Identify underfitting, overfitting, or need for more data

Unsupervised Learning Tasks and Algorithms

Clustering Techniques

• Clustering groups similar data points based on inherent characteristics or distances
• K-means partitions data into k clusters by minimizing within-cluster variances:
  • Iteratively assigns points to nearest centroid and updates centroids
  • Requires specifying number of clusters k in advance
• Hierarchical clustering builds a tree of clusters (dendrogram):
  • Agglomerative (bottom-up) or divisive (top-down) approaches
  • No need to specify number of clusters beforehand
• DBSCAN (Density-Based Spatial Clustering of Applications with Noise):
  • Finds arbitrarily shaped clusters based on density
  • Robust to noise and outliers in the data
• Gaussian Mixture Models represent clusters as Gaussian distributions:
  • Soft clustering assigning probabilities of belonging to each cluster
  • Can model clusters with different shapes and orientations

Dimensionality Reduction

• Dimensionality reduction techniques reduce feature space while preserving information
• Principal Component Analysis (PCA):
  • Finds orthogonal directions of maximum variance in the data
  • Projects data onto lower-dimensional subspace
• t-SNE (t-Distributed Stochastic Neighbor Embedding):
  • Visualizes high-dimensional data in 2D or 3D space
  • Preserves local structure and reveals clusters
• Autoencoders use neural networks for nonlinear dimensionality reduction:
  • Encoder compresses input to lower-dimensional representation
  • Decoder reconstructs original input from compressed representation

Other Unsupervised Techniques

• Anomaly detection identifies rare items or events differing from the majority:
  • Isolation Forest for efficient anomaly detection in large datasets
  • One-class SVM for novelty detection
• Association rule learning discovers relations between variables in databases:
  • Apriori algorithm for mining frequent itemsets
  • FP-growth algorithm for efficient pattern mining
• Generative models learn underlying data distribution:
  • Variational autoencoders for generating new samples
  • Generative Adversarial Networks (GANs) for realistic image generation

Choosing Supervised or Unsupervised Learning

Problem Characteristics

• Supervised learning requires clear target variable to predict
• Unsupervised learning explores data structure without specific target
• Availability of labeled data influences choice:
  • Abundant labeled data favors supervised learning
  • Lack of labels or expensive labeling process suggests unsupervised approach
• Problem domain impacts decision:
  • Predictive modeling tasks typically use supervised learning
  • Exploratory analysis often employs unsupervised techniques
• Desired outcomes guide selection:
  • Precise predictions on new data point to supervised learning
  • Discovering hidden patterns indicates unsupervised learning

Practical Considerations

• Data quality affects choice:
  • High-quality labeled data enables effective supervised learning
  • Noisy or unreliable labels may necessitate unsupervised approaches
• Computational resources influence decision:
  • Supervised learning often requires more computational power for training
  • Some unsupervised algorithms (k-means) can be more computationally efficient
• Interpretability requirements impact selection:
  • Some supervised models (decision trees) offer clear interpretability
  • Unsupervised techniques may provide less straightforward interpretations
• Human expertise availability factors in:
  • Supervised learning needs domain experts for accurate data labeling
  • Unsupervised learning can proceed without extensive domain knowledge

Hybrid Approaches

• Semi-supervised learning combines limited labeled data with larger unlabeled dataset:
  • Self-training iteratively labels confident predictions
  • Co-training uses multiple views of the data for mutual learning
• Transfer learning adapts knowledge from related tasks:
  • Fine-tuning pre-trained models on specific datasets
  • Useful when labeled data is scarce in target domain
• Active learning strategically selects instances for labeling:
  • Reduces labeling effort by focusing on most informative examples
  • Combines benefits of supervised and unsupervised approaches