AI Fundamentals Study Guide | Module 2

← Back to Module 📝 Take Quiz

📖 Use this study guide to review key concepts, dates, people, and exam tips for Module 2

🎯 Key Concepts to Master

GPU (Graphics Processing Unit) - Parallel processing for AI training
TPU (Tensor Processing Unit) - Google's custom AI chips
Neural Networks - Layers of interconnected nodes with weights
Backpropagation - Algorithm for adjusting weights based on errors
Vector Embeddings - Converting data to numerical representations
Transformer Architecture - Attention mechanisms for LLMs
Tokens - Basic units of text (~1.3 per word)
Hallucinations - LLMs stating false information confidently
Model Drift - Changes in LLM behavior over time
Context Window - Memory limits (200K tokens for Claude)
Prompt Engineering - Crafting effective AI inputs
Fine-Tuning - Training models on specific data
RAG (Retrieval-Augmented Generation) - External knowledge access
RLHF - Reinforcement Learning from Human Feedback

📅 Important Dates & Events

2017 - "Attention is All You Need" paper introduces Transformers
2022 - ChatGPT launches, brings LLMs to mainstream
2023 - Claude, Gemini, and GPT-4 compete
Training GPT-4 cost ~$100 million in compute
GPT-3 training used 1,287 MWh (120 homes for a year)

👤 Key People to Know

Vaswani et al. - "Attention is All You Need" (Transformer paper)
Geoffrey Hinton - Backpropagation, deep learning pioneer
Jensen Huang - NVIDIA CEO (GPU dominance in AI)
Dario Amodei - Anthropic CEO (Claude, RLHF)
Sam Altman - OpenAI CEO (ChatGPT, GPT-4)

⭐ Exam Tips & Must-Know Facts

Know the difference between GPUs (NVIDIA parallel processing) and TPUs (Google's custom tensor chips)
Understand backpropagation: how neural networks learn by adjusting weights backward through layers
Memorize vector embedding concept: data converted to numbers, similar concepts = similar vectors
Know what LLMs are: Large Language Models trained on massive text to predict next tokens
Understand CRITICAL limitations: hallucinations (can't be eliminated), model drift (behavior changes)
Transformer architecture: attention mechanism lets models understand context
Token economics: ~1.3 tokens per word, API pricing per million tokens
RAG vs Fine-Tuning: RAG uses external knowledge, fine-tuning trains on your data
RLHF: humans rate outputs, model learns to produce higher-rated responses (how ChatGPT became helpful)
Best practices: Always verify LLM outputs, implement human-in-the-loop for high-stakes decisions
Environmental cost: Training large models uses enormous energy (GPT-3 = 552 tons CO2)

📢 Share This Free Course

𝕏 Twitter f Facebook in LinkedIn 📌 Pinterest