Glossary of AI Terms (Plain‑Language)

21.2 Model Families and Techniques

• Linear Models: Simple models that combine input features linearly. Easy to train and explain. Baselines for many tabular problems.
• Logistic Regression: A linear model for classification. Outputs probabilities via a logistic function. Despite the name, it is a classifier, not a regression in the everyday sense.
• Decision Tree: A flow‑chart model that splits data by rules (“if amount > X then …”). Intuitive and interpretable; can overfit without pruning.
• Random Forest: An ensemble of decision trees trained on random subsets. Reduces overfitting; good accuracy on many tabular tasks.
• Gradient Boosting (e.g., XGBoost, LightGBM, CatBoost): Builds trees sequentially, each correcting the errors of the previous ones. Strong performance on structured data.
• Support Vector Machine (SVM): A classifier that finds a boundary maximizing the margin between classes. Effective in moderate‑dimension spaces; less used for very large datasets.
• Naïve Bayes: A simple probabilistic classifier that assumes feature independence. Fast and surprisingly effective for some text problems.
• k‑Nearest Neighbors (k‑NN):  Classifies or regresses based on the closest examples in feature space. Requires efficient indexing; sensitive to scaling.
• Clustering (k‑Means, Hierarchical): Unsupervised grouping of similar items. Used for exploration, customer segmentation, or deduplication.
• Topic Models (e.g., LDA): Unsupervised methods that uncover themes in document collections. Partially superseded by embedding‑based methods.
• Hidden Markov Model (HMM): A probabilistic model for sequences with hidden states. Classic for speech and simple time series; often replaced by neural models.
• Collaborative Filtering / Matrix Factorization: Recommendation methods that factorize user–item interactions into latent factors; represent tastes and attributes in vector spaces.
• Neural Network: A model built from layers of simple functions (neurons) whose weights are learned. Flexible enough to approximate complex mappings.
• Perceptron / Multilayer Perceptron (MLP): The perceptron is a single‑layer neural classifier; an MLP stacks fully connected layers to model nonlinear patterns.
• Convolutional Neural Network (CNN): A neural network that uses “filters” to detect local patterns; central to image recognition and also used in audio and some text tasks.
• Recurrent Neural Network (RNN), LSTM, GRU: Sequence models that process inputs one step at a time while carrying state. LSTM and GRU units mitigate vanishing gradients. Largely supplanted by transformers for language, though still used in smaller on‑device models.
• Attention: A mechanism that lets models weigh which parts of the input matter for the current prediction. Key to modern language and vision models.
• Transformer: The dominant neural architecture for language and many multimodal tasks. Uses attention to process sequences in parallel. Underlies most large language models (LLMs).
• Encoder / Decoder / Encoder–Decoder: Transformer variants: an encoder reads inputs (useful for classification, embeddings); a decoder generates outputs (useful for text generation); encoder–decoder pairs are common in translation and vision–language tasks.
• Language Model (LM): A model trained to predict the next token (causal) or fill masked tokens (masked LM). Causal LMs are used for generation; masked LMs are common for embeddings and classification.
• Causal vs. Masked LM: Causal LMs read left‑to‑right and generate text; masked LMs see the whole sentence with some tokens hidden and learn to fill them in.
• Diffusion Model: A generative model that learns to remove noise step by step, producing images or audio from random noise guided by text or other signals.
• Variational Autoencoder (VAE): A model that compresses inputs into a latent space and reconstructs them. Useful for anomaly detection and generative tasks with smooth latent controls.
• Generative Adversarial Network (GAN): Two networks—generator and discriminator—trained in a contest. Historically strong for images; less common for text.
• Mixture of Experts (MoE): A model with many “expert” sub‑networks where only a subset is active per input. Increases capacity without proportional compute at inference.
• Multimodal Model: A model that jointly processes more than one modality—text + image, text + audio, etc.—to answer questions, describe scenes, or follow instructions grounded in visuals.
• Vision–Language Model (VLM): A multimodal model specialized for tasks like image captioning, visual question answering, and document understanding.
• Automatic Speech Recognition (ASR): Models that transcribe audio into text.
• Text‑to‑Speech (TTS): Models that synthesize speech from text in specific voices or styles.
• Embedding Model: A model that maps text, images, or other inputs into vectors (lists of numbers) so that semantic similarity can be measured by distance.
• Bi‑Encoder vs. Cross‑Encoder (Retrieval / Re‑Ranking): A bi‑encoder embeds query and document separately for fast approximate search. A cross‑encoder scores a query–document pair jointly for higher precision; used as a re‑ranker.
• Knowledge Graph: A network of entities and relationships represented as nodes and edges. Useful for reasoning, constraints, and explainability.
• Graph Neural Network (GNN): A neural model that operates on graph structures, passing messages along edges to learn from connections as well as features.
• Reinforcement Learning (RL): Learning by acting and receiving rewards or penalties. Useful for control and sequential decision‑making; adapted to tune language models via preferences (RLHF).
• Policy / Value Function: In RL, the policy maps states to actions; the value function estimates expected reward from a state or state–action pair.
• Contextual Bandit: A simplified RL setting where each decision is a one‑shot choice with feedback, used for recommendations and pricing experiments.
• Self‑Supervised Learning: Learning from the structure within unlabeled data by predicting parts of the input from other parts (e.g., next token). The backbone of modern pretraining.
• Contrastive Learning: A method that brings similar pairs closer in embedding space and pushes dissimilar pairs apart. Central to many vision and text–image models.
• Curriculum Learning: Training that starts with easier examples and gradually increases difficulty, improving stability and speed.
• Knowledge Distillation: Training a smaller “student” model to mimic a larger “teacher,” preserving most performance with reduced size and cost.
• Quantization: Representing weights and activations with fewer bits (e.g., 8‑bit or 4‑bit) to reduce memory and increase speed, with careful calibration to limit accuracy loss.
• Pruning: Removing weights or neurons that contribute little, shrinking models with minimal quality loss.
• Adapters / LoRA / PEFT: Parameter‑efficient fine‑tuning methods that add small trainable modules or rank‑decomposed updates to a frozen base model, enabling task customization with modest compute.
• Prompt Tuning (Soft Prompts): Training virtual tokens (not visible words) that steer a frozen model toward a task or brand voice without updating all weights.
• Token Merging / KV Compression: Techniques that reduce the number of active tokens or compress attention memories to speed up inference in long contexts.
• Decoding Strategies (Greedy, Beam, Top‑k, Top‑p, Temperature): Rules for picking the next token when generating text. Greedy chooses the most likely token; beam keeps multiple partial candidates; top‑k and top‑p sample from the most likely subset; temperature adjusts randomness.
• Speculative Decoding: A small draft model proposes several tokens; a larger model quickly veri