Basic AI Glossary
This glossary was written with the help of Microsoft Copilot, Chat GPT, and Claude
Agentic AI
A type of artificial intelligence (AI) that can make decisions, take actions, and learn on its own. It's designed to work with limited human supervision and can perform tasks that previously required human input
AI literacy
Refers to the understanding and knowledge needed to effectively use, interpret, and critically assess artificial intelligence (AI) technologies. Just as digital literacy involves the ability to use digital tools, AI literacy enables individuals to understand how AI systems work, how they impact society, and how to interact with AI-based applications responsibly and ethically.
Algorithm
A set of instructions or rules designed to solve a specific problem or perform a particular task.
Artificial General Intelligence (AGI)
The hypothetical level of machine intelligence that that equals or exceeds that of a human, and possessing the ability to understand or learn any intellectual task that a human being can. There is some debate about the exact definition and whether current AI technology has reached this point or not.
Artificial Intelligence (AI)
A field of computer science focused on creating systems capable of performing tasks that require human intelligence, such as speech recognition, decision-making, and language translation.
Bias in AI
Refers to systematic errors in AI outputs that are influenced by the training data. In generative AI, biased outputs can arise from patterns in the dataset that reflect human prejudices or imbalances.
Big Data
Extremely large data sets that can be analyzed computationally to reveal patterns and trends, often used to train AI models.
Chatbot
An AI program designed to simulate human conversation through text or voice interactions. Chatbots use NLP to understand and respond to user input.
Computer Vision
A field of AI that enables machines to interpret and make decisions based on visual data.
Context Window
A context window refers to the amount of text or input data that a large language model (LLM) can "see" or consider at once when generating predictions or responses. It defines the maximum number of tokens (words, subwords, or characters) that the model can process and use to make predictions. If the input text exceeds the context window size, earlier parts of the text may be "forgotten" or ignored. The size of the context window affects how well the model can maintain coherence in long conversations or texts.
Data Mining
The process of discovering patterns and knowledge from large amounts of data.
Data Scraping
Data scraping, or web scraping, is a process of importing data from websites or programs into files or spreadsheets. There are numerous software applications for automating data scraping. This process can be used to provide massive amounts of data to train AI models.
Fine-Tuning
The process of further training a pre-trained model on a smaller, specific dataset to adapt it to a more specialized task.
Generative AI
A type of artificial intelligence focused on creating new content, such as text, images, audio, or video, based on existing data it has been trained on. Common examples include language models (like GPT), image generation models, and music creation systems.
Hallucination
Refers to a phenomenon where an AI system generates information that is false or misleading but presents it as if it were true. This can happen in various AI applications, such as chatbots or language models, where the AI might produce responses that seem plausible but are actually incorrect or nonsensical.
Large Language Models (LLMs)
A type of deep learning model designed to generate, comprehend, and manipulate human language. They are trained on vast amounts of text data and contain billions (or even trillions) of parameters, making them capable of performing a wide range of tasks such as text completion, summarization, translation, and answering questions. GPT-4, Llama, Claude, BERT, T5, and PaLM are examples of large language models. LLMs are used in many domains, from customer service chatbots and content generation tools to aiding in medical research or legal document processing.
Machine Learning (ML)
A subset of AI where algorithms improve automatically through experience and by using data. In generative AI, ML models are trained to recognize patterns in large datasets to generate new, similar data.
Multimodal
Refers to models or systems that can process and integrate multiple types of data, such as text, images, audio, and video, to perform tasks or generate outputs. These models combine information from different "modalities" to create a more comprehensive understanding or generate richer, more complex results. For example, a multimodal AI system might generate a caption for an image based on both visual and textual inputs.
Natural Language Processing (NLP)
A branch of AI that focuses on enabling machines to understand, interpret, and generate human language. NLP is a core technology behind text-based generative AI models.
Neural Networks
A system of algorithms modeled after the human brain that can recognize patterns. Neural networks form the basis of deep learning models used in generative AI.
Pre-trained Models
Generative AI models that have been trained on a large dataset before being fine-tuned for specific tasks. They can be adapted for new tasks without needing to be trained from scratch.
Prompt
An input provided by a user that initiates the generation of text, images, or other content by a generative AI model. The prompt typically describes what the user wants the AI to produce.
Prompt Chaining
A technique where multiple prompts are used sequentially, with each prompt building on the response from the previous one, to guide the AI toward a complex or refined final output.
Prompt Engineering
The process of designing and optimizing input prompts to guide a generative AI model, like a large language model, to produce desired or high-quality outputs. It involves crafting specific instructions, questions, or examples in a way that helps the model understand the task and generate more accurate or relevant results.
Prompt Template
A reusable structure for prompts that can be customized with different input variables, often used in automation or for specific tasks like summarization or question-answering.
Test Data
A separate dataset used to evaluate the performance of a trained machine learning model, providing an indication of how well the model generalizes to new, unseen data.
Text-to-Image Models
Generative AI models that create images based on textual descriptions. Examples include DALL·E and Stable Diffusion.
Text-to-Text Models
Models that generate text based on input text (prompts). Examples include GPT (like ChatGPT) and T5, used for tasks like text completion, translation, or summarization.
Tokenization
The process of breaking down text into smaller units (tokens), such as words or subwords, for use by NLP models. Tokenization helps models process and understand language input more efficiently.
Traditional AI (Symbolic AI)
An early approach to artificial intelligence that uses explicitly programmed rules, logic, and symbols to represent knowledge and solve problems. Unlike modern machine learning, it relies on hand-coded instructions rather than learning from data.
Training Data
The dataset used to train a generative AI model. This data serves as the foundation for the model to learn patterns and features for generating new content.
Advanced AI Glossary
This glossary was written with the help of Microsoft Copilot, Chat GPT, and Claude
Batch Size
The number of training examples used in one iteration of model training. It affects both the model's learning dynamics and the memory requirements.
Chain-of-Thought Prompting
A method where prompts encourage the model to "think" step-by-step, which can lead to more logical, multi-step problem-solving in the model’s responses.
Deep Learning
A subset of machine learning that uses neural networks with many layers (called deep neural networks). It is particularly effective for generative AI tasks like image generation and natural language processing (NLP).
Few-shot Learning
The ability of LLMs to perform tasks with only a few examples.
Generative Adversarial Networks (GANs)
A framework where two neural networks, a generator and a discriminator, compete to create new, synthetic data that resembles real data.
Gradient Descent
An optimization algorithm used to minimize a model's loss function by iteratively adjusting the model’s parameters in the direction of the steepest descent.
Hyperparameters
Parameters in machine learning models that are set before the training process begins, influencing the behavior and performance of the model (e.g., learning rate, batch size).
Learning Rate
A hyperparameter that controls how much to change the model in response to the estimated error each time the model weights are updated. It significantly impacts training speed and performance.
Loss Function
A mathematical function that measures how far a machine learning model’s predictions are from the actual outcomes; minimizing the loss function improves the model.
One-shot Learning
A scenario where only a single example is provided in the prompt to help guide the AI's understanding of the desired output.
Reinforcement Learning
A learning paradigm where AI agents learn to take actions in an environment to maximize a reward. In generative AI, reinforcement learning can fine-tune models to produce more relevant or high-quality outputs.
Reinforcement Learning from Human Feedback (RLHF)
A method used to fine-tune models by using human evaluations to improve the relevance and quality of responses based on prompts.
Supervised Learning
A type of machine learning where the AI is trained on labeled data. Each input comes with a corresponding output (label), and the AI learns to predict the correct output based on the input. For example, training an AI to recognize images of cats by providing many labeled images of cats and non-cats.
Temperature
A hyperparameter that controls the randomness of the AI model's output. Lower temperatures lead to more deterministic results, while higher temperatures make the output more creative and varied.
Top-k Sampling
A decoding technique that limits the AI's next word prediction to the top-k most likely options, allowing for more control over randomness and creativity.
Top-p (Nucleus) Sampling
A sampling method that selects from the smallest possible set of words whose cumulative probability exceeds a certain threshold (p), balancing between creativity and coherence.
Transfer Learning
A technique where a pre-trained model developed for one task is adapted for a different but related task, reducing the need for large amounts of new data.
Transformer Model
A deep learning model that adopts the mechanism of self-attention, differentially weighting the significance of each part of the input data.
Unsupervised learning
LLMs are typically trained using unsupervised learning, meaning they learn patterns from massive text corpora without being explicitly told the "correct" output. The goal is to predict the next word (or token) in a sequence, which leads to the model understanding the structure of human language.
Zero-Shot Learning
When a generative AI model can perform a task without having been explicitly trained on it. The model generalizes its learning to new, unseen situations.