🤟🏼Natural Language Processing Unit 1 – Intro to Natural Language Processing

What's NLP All About?

• Natural Language Processing (NLP) focuses on enabling computers to understand, interpret, and generate human language
• Combines techniques from computer science, artificial intelligence, and linguistics to analyze and process natural language data
• Aims to bridge the gap between how humans communicate and how computers process information
• Involves tasks such as text classification, sentiment analysis, machine translation, and question answering
• Enables applications like chatbots, voice assistants (Siri, Alexa), and automated customer support systems
• Helps in extracting insights and knowledge from unstructured text data (social media posts, customer reviews)
• Facilitates human-computer interaction by allowing users to communicate with machines using natural language

Key Concepts and Terminology

• Tokenization: The process of breaking down text into smaller units called tokens (words, phrases, or subwords)
• Part-of-Speech (POS) Tagging: Assigning grammatical categories (noun, verb, adjective) to each word in a sentence
• Named Entity Recognition (NER): Identifying and classifying named entities (person names, locations, organizations) in text
• Stemming: Reducing words to their base or root form (e.g., "running" to "run")
• Lemmatization: Determining the dictionary form (lemma) of a word (e.g., "better" to "good")
• Corpus: A large collection of text data used for training and evaluating NLP models
• N-grams: Contiguous sequences of n items (words or characters) from a given text
  • Unigrams: Single words or tokens
  • Bigrams: Pairs of adjacent words or tokens
  • Trigrams: Triples of adjacent words or tokens

Text Processing Basics

• Text Normalization: Converting text into a standardized format
  • Lowercasing: Converting all characters to lowercase
  • Removing punctuation and special characters
  • Handling abbreviations and acronyms
• Stop Word Removal: Eliminating common words (the, is, and) that carry little meaning
• Text Representation: Converting text into numerical representations for machine learning models
  • Bag-of-Words (BoW): Representing text as a vector of word frequencies
  • TF-IDF (Term Frequency-Inverse Document Frequency): Assigning weights to words based on their importance in a document and rarity across the corpus
• Text Similarity: Measuring the similarity between two pieces of text
  • Cosine Similarity: Calculating the cosine of the angle between two vector representations of text
• Text Preprocessing Pipeline: A series of steps applied to raw text data before feeding it into NLP models

Language Models and Probability

• Language Models: Probabilistic models that estimate the likelihood of a sequence of words
• N-gram Language Models: Predicting the probability of the next word based on the previous n-1 words
  • Unigram Language Model: Considers each word independently
  • Bigram Language Model: Considers the probability of a word given the previous word
  • Trigram Language Model: Considers the probability of a word given the previous two words
• Smoothing Techniques: Handling unseen or rare word sequences in language models
  • Laplace Smoothing (Add-one Smoothing): Adding a small constant to the count of each word
  • Good-Turing Smoothing: Adjusting the counts of low-frequency words based on the counts of higher-frequency words
• Perplexity: Measuring the quality of a language model by evaluating how well it predicts a held-out test set

Syntactic Analysis and Parsing

• Syntactic Analysis: Analyzing the grammatical structure of sentences
• Constituency Parsing: Identifying the hierarchical structure of a sentence by breaking it down into constituent phrases
  • Context-Free Grammar (CFG): A set of rules that define the structure of a language
  • Parse Trees: Visual representations of the syntactic structure of a sentence
• Dependency Parsing: Identifying the relationships between words in a sentence based on their grammatical roles
  • Dependency Relations: Labeled edges representing the grammatical relationships between words (subject, object, modifier)
• Part-of-Speech (POS) Tagging: Assigning grammatical categories to each word in a sentence
  • Hidden Markov Models (HMMs): Probabilistic models used for POS tagging
• Chunking: Identifying and grouping words into meaningful phrases (noun phrases, verb phrases)

Semantic Analysis and Understanding

• Semantic Analysis: Extracting meaning and understanding the context of text
• Word Sense Disambiguation (WSD): Determining the correct meaning of a word based on its context
  • Lesk Algorithm: Selecting the word sense that has the highest overlap with the context words
• Named Entity Recognition (NER): Identifying and classifying named entities in text
  • Conditional Random Fields (CRFs): Probabilistic models used for NER
• Coreference Resolution: Identifying and linking mentions of the same entity across a text
• Semantic Role Labeling (SRL): Identifying the semantic roles (agent, patient, instrument) played by words in a sentence
• Sentiment Analysis: Determining the sentiment (positive, negative, neutral) expressed in a piece of text
  • Lexicon-based Approaches: Using pre-defined sentiment lexicons to assign sentiment scores to words
  • Machine Learning Approaches: Training classifiers on labeled sentiment data

Applications and Use Cases

• Machine Translation: Automatically translating text from one language to another (Google Translate)
• Text Summarization: Generating concise summaries of longer texts while preserving key information
  • Extractive Summarization: Selecting important sentences from the original text to form a summary
  • Abstractive Summarization: Generating new sentences that capture the essence of the original text
• Information Retrieval: Retrieving relevant documents or information based on user queries (search engines)
• Chatbots and Conversational Agents: Building systems that can engage in human-like conversations and provide assistance
• Sentiment Analysis: Analyzing the sentiment expressed in customer reviews, social media posts, or feedback
• Text Classification: Assigning predefined categories or labels to text documents (spam detection, topic classification)
• Named Entity Recognition: Extracting and classifying named entities (person names, locations, organizations) from text

Tools and Libraries for NLP

• Natural Language Toolkit (NLTK): A widely-used Python library for NLP tasks
  • Provides modules for tokenization, stemming, POS tagging, parsing, and more
• spaCy: A fast and efficient NLP library in Python
  • Offers pre-trained models for various NLP tasks and supports multiple languages
• Stanford CoreNLP: A comprehensive NLP toolkit developed by Stanford University
  • Provides tools for tokenization, POS tagging, NER, parsing, and coreference resolution
• Gensim: A Python library for topic modeling and document similarity retrieval
  • Implements algorithms like Latent Dirichlet Allocation (LDA) and Word2Vec
• Hugging Face Transformers: A library providing state-of-the-art pre-trained models for NLP tasks
  • Includes models like BERT, GPT, and XLNet for various downstream tasks
• TensorFlow and PyTorch: Deep learning frameworks commonly used for building and training NLP models
• Scikit-learn: A machine learning library in Python that offers tools for text preprocessing and feature extraction