Large language models bootcamp

Key topics

• Fundamentals of the LLM Landscape: From visual perception and natural language understanding to tokenization, embeddings, and how large language models generate text; the canonical end-to-end architecture connecting every layer.
• Vector Databases and Similarity Search: Why vector databases exist, how embeddings represent meaning as geometry, and how similarity search powers retrieval at scale.
• Prompt Engineering Best Practices: Structuring prompts that produce consistent, grounded outputs — and the failure modes that undermine them.
• Building Custom LLM Applications: Three distinct paths — training from scratch, fine-tuning foundation models, and in-context learning — with clear tradeoffs between each.
• Retrieval-Augmented Generation (RAG): How the RAG pipeline connects retrieval to generation, where it breaks down, and what a production-ready implementation actually requires.

Key topics

• Adoption Reality: Why most LLM projects stall between proof-of-concept and production; the competing pressures of cost, accuracy, latency, and infrastructure constraints that make scaling harder than building.
• Technology Challenges: Context-window limits, dataset quality and PII handling, the real cost of fine-tuning (including PEFT approaches), and how inference costs compound with RAG and token volume.
• Business and Risk Challenges: Compliance exposure, legal risk, misaligned customer expectations, and the organizational culture gaps that prevent meaningful KPI alignment.
• Human Behavior Challenges: Fear of AI, resistance to change, biased prompting, and the subjectivity in feedback that makes evaluation harder than it looks.
• Prompting Challenges: Prompt sensitivity, fatigue, overengineering, jailbreaking and injection risks, hallucinations from lack of grounding, and how ambiguity creates silent failures.
• Best Practices: Guardrails and defensive UX design, LLM caching for cost control, structured feedback collection, and evaluation frameworks built around fairness and explainability.

Running your LLM agent safely with Docker sandboxes

60 min · Lecture · Discussion

• LLM Agent Security Fundamentals: Understand why agent execution environments matter when agents can call tools, access files, and interact with APIs.
• Docker Sandboxes (SBX): Set up and run LLM agents inside isolated sandbox environments.
• Secure Agent Execution: Explore how sandboxing limits risk and reduces the attack surface of autonomous agents.
• Policy Enforcement: Configure and tighten permissions governing agent actions, file access, and external integrations.
• Hands-on Agent Deployment: Launch a sandboxed agent and assign it real-world tasks in a controlled environment.
• Practical Exercise: Implement secure execution patterns and harden an agent runtime for your own LLM and agent applications.
• Take-Home Architecture: Learn a reusable pattern for securely deploying and operating LLM agents in production-like environments.

Key topics

• Introduction to LLMs: Strengths and weaknesses of large language models; discriminative versus generative AI; how predictive and generative models differ in what they’re actually doing.
• Transformer Architecture: Tokenization, embeddings, positional encoding, and the attention mechanism that holds it all together — explained from the ground up.
• Embeddings and Similarity: How words become vectors, what proximity in that space actually means, and why it matters for retrieval and reasoning.
• Attention Mechanism: Keys, queries, and values in self-attention; how the model decides what to focus on when generating each token.
• Softmax and Probabilities: How raw attention scores become a probability distribution, and what that means for next-token prediction.
• Training and Fine-Tuning: Adapting pre-trained models with curated data — what changes, what doesn’t, and where overfitting quietly creeps in.
• Search and Retrieval: Building a semantic search engine with embeddings; connecting retrieval to generation for grounded, factual answers.
• Hands-On Exercises: Sentence Transformers, semantic search, attention scoring, and implementing attention mechanisms directly.

Key topics

• Overview and Rationale: Why vector databases exist, how they differ from traditional search, and the role they play in grounding LLM applications.
• Search Types: Vector search, text search, and hybrid search — when to use each and what you lose by picking the wrong one.
• Indexing Techniques: Product Quantization (PQ), Locality Sensitive Hashing (LSH), and Hierarchical Navigable Small World (HNSW) — the tradeoffs between speed, memory, and recall.
• Retrieval Techniques: Cosine similarity, nearest neighbor search, and how retrieval quality degrades at scale without the right index design.
• Advanced RAG Techniques: Chunking and filtering strategies, query rewriting, hybrid search, auto-cut, and re-ranking for precision at the top of the result set.
• Embedding and Model Selection: Domain-specific embeddings, fine-tuned retrieval models, and compression approaches including scalar quantization, product quantization, and binary and matryoshka methods.
• Adaptive Retrieval and Multi-Tenancy: Multi-phase rescoring, preserving recall under compression, tenant isolation, and resource allocation patterns for shared environments.
• Production Challenges: Scaling, reliability, and cost optimization — the problems that don’t surface until you’re running real query volumes.
• Hands-On Exercises: Vector search, similarity search, hybrid search, generative search, Weaviate Query Agent, multi-tenancy, vector compression, and semantic caching.

Key topics

• Privacy, Security, and Compliance: PII-safe ingestion through anonymization and differential privacy; authenticated access and user filtering at the retrieval layer; logging and data retention for compliance.
• Multimodal and Multilingual RAG: Vision models (CLIP, BLIP-2, Gemini), audio pipelines (Whisper, audio embeddings, noise-tolerant search), and cross-lingual retrieval with LaBSE, mBERT, and XLM-R.
• Advanced Architectures: DSPy for prompt compilation and retrieval chain optimization; KG-RAG for graph-grounded, traceable answers; self-improving retrieval through feedback loops, ranking callbacks, and adaptive scoring.
• Retrieval Layer and Index Optimization: FAISS, Qdrant, Weaviate, and Pinecone compared; flat, HNSW, IVF, and PQ index types; improving search quality with hybrid BM25 + dense retrieval and re-rankers like BGE and ColBERT.
• Evaluation and Metrics: Retrieval metrics (Precision@K, Recall@K, Hit Rate@K), generation metrics (BLEU, ROUGE, LLM-as-a-Judge), and tooling with TruLens, Phoenix, and EvalGen.
• Latency, Cost, and Scalability: Identifying bottlenecks across retrieval, re-ranking, and generation; caching strategies at the embedding, prompt, and disk layers; cost controls through batch queries, static prompts, and quantized models.

Key topics

• Introduction to LangChain: What LangChain is for, what it abstracts away, and the class of RAG challenges it was built to address.
• Core Components: LLMs and chat models, prompt templates, example selectors, document loaders, and transformers as the building blocks of LLM-powered applications.
• Output Parsers: Structured data extraction, consistent formatting across model responses, and error handling when outputs don’t conform.
• Retrieval and Vector Stores: Embedding, vectorization, metadata filtering, parent document retrieval, and efficient similarity search optimized for large datasets.
• Chains: Sequential prompt logic with pre- and post-LLM steps; integrating tools and retrieval into coherent, composable workflows.
• Tool Use and Memory: Connecting APIs and external actions, passing results back into the workflow, managing conversation history, storing state between calls, and maintaining context persistence for agents.
• Callbacks and Observability: Event hooks during runs, monitoring and logging, and custom actions on success or failure.
• LCEL (LangChain Expression Language): Piping components with runnables, parallel branches, and modular workflow composition.
• LangGraph and Agents: Graph-based workflows for complex agent orchestration, dynamic control flows, decision-making steps, and combining tools with context for non-linear reasoning.

Key topics

• Why Context Engineering: What the model can see determines what it can do; why a single well-written prompt stops being enough once an agent loops, calls tools, and accumulates history across many steps.
• Graphs Instead of Chains: The state graph as the unit of composition; nodes as steps that return partial state updates rather than mutating shared state; edges as explicit control flow; start and end as entry and exit points; compiling a graph before it can run.
• Designing State: State as the shared memory every node reads from and writes to; typed state schemas; storing raw data and formatting prompts inside the nodes that need them rather than passing formatted strings between steps.
• Reducers: How a field is merged when more than one node writes to it — overwrite by default, accumulate with a reducer — and why a message history declared without one silently keeps only the last write.
• Control Flow: Static edges for deterministic sequence, conditional edges for routing on state, combined update-and-route returns for decisions taken inside a node, and fan-out to parallel workers whose results are aggregated back into one state.
• Short-Term Memory: Checkpointers that persist state at every step; thread IDs that isolate one conversation from another; resuming, replaying, and forking a run from an earlier checkpoint — and why an in-memory checkpointer is a development tool rather than a production one.
• Long-Term Memory: A store for the preferences and facts that outlive a single conversation; namespacing memory per user; deciding what belongs in graph state versus what belongs in the store.
• Keeping Context Under Control: Trimming and summarising message history, passing each node only what it needs, and scoping subgraphs so one agent’s context does not leak into another’s.
• Durability and Failure: Retry policies for transient failures, returning tool errors to the model so it can recover on its own, and pausing for a human when a failure is not the model’s to fix.
• Observability: Streaming state snapshots, incremental updates, and tokens as a graph runs, so a multi-step agent is inspectable rather than a black box between question and answer.
• Hands-On Exercises: Building a stateful graph with reducers; adding a checkpointer and resuming a thread; routing with conditional edges; fanning out work to parallel workers; and instrumenting a run with streamed state to see where the context budget is actually going.

Key topics

• Origins & Motivation: Addressing fragmented integrations and brittle bespoke adapters; introducing a unified, interoperable interface — the “USB-C for AI.”
• Protocol Structure: Client–server handshake model; defining resources, tools, and prompts; JSON-RPC transport with structured, schema-driven messages.
• Context Exposure: How MCP surfaces tools, data, and metadata through a consistent schema to enable discoverability, governance, and controlled access.
• Agentic Integration: Connecting MCP endpoints to reflection, planning, tool-use, and multi-agent coordination patterns for modular and scalable systems.
• Hands-On Labs: Setting up an MCP client in Streamlit; discovering and registering tools; automating workflows through data retrieval and validation; logging traces for monitoring and review.

Key topics

• Core Concepts: Transfer learning and why fine-tuning works; full fine-tuning versus LoRA and QLoRA; parameter-efficient tuning and quantization as practical paths to adapting large models without prohibitive compute.
• Key Considerations: Data quality and relevance as the primary lever for fine-tuning outcomes; overfitting risks and the limitations fine-tuning cannot resolve; when fine-tuning is the right choice versus RAG.
• Hands-On Exercises: Instruction fine-tuning, deploying, and evaluating a LLaMA2-7B 4-bit quantized model in class; fine-tuning and deploying OpenAI and Llama models on Azure AI Studio as a take-home project.

Key topics

• Need for Evaluation: Why reliability, accuracy, and safety can’t be assumed — and how business alignment, ethical accountability, and user trust depend on structured measurement.
• Challenges in Evaluation: Hallucinations, prompt sensitivity, and weak context handling; the difficulty of evaluating outputs where multiple valid answers exist; navigating tradeoffs between accuracy, fluency, and creativity.
• Benchmarking Approaches: MMLU for multitask accuracy, HELM for holistic metrics across accuracy, robustness, and fairness, BBH and HotpotQA for reasoning and multi-hop question answering.
• Text Quality Metrics: BLEU for n-gram precision, ROUGE for recall-based evaluation, BERTScore for semantic similarity, METEOR for synonym and stem alignment, and perplexity as a measure of prediction confidence.
• RAG-Specific Evaluation (RAGAs): Faithfulness, answer relevance, context precision, and context recall — metrics that score retrieval and generation jointly rather than in isolation.
• Open-Ended Output Evaluation (G-Eval): Fluency, faithfulness, answer relevance, and claim-level scoring for outputs that don’t have a single correct answer.
• Additional Benchmarks and Metrics: GLUE for NLU tasks, TriviaQA for multi-hop QA, RealToxicityPrompts for safety evaluation, MRR and MAP for ranking performance, and ROSCOE for reasoning quality across semantic alignment, logical integrity, and commonsense coverage.

Project tracks

• Conversational Workflow Orchestration: Design a multi-turn assistant coordinating tasks across specialized agents.
• Knowledge-Enhanced Agent: Integrate search and APIs for grounding, fact-checking, and real-time data access.
• Document-Aware Action Agent: Retrieve and reason over documents; trigger external tools or services based on insights.
• Orchestrated Collaboration (MCP): Build coordinated multi-agent systems using the Model Context Protocol for seamless tool and enterprise integration.

Attendees will receive

• Comprehensive Datasets: Industry-spanning document collections for robust development and testing.
• Step-by-Step Implementation Guides: Clear instructions from environment setup to deployment.
• Ready-to-Use Code Templates: Prebuilt templates within Data Science Dojo’s sandbox for accelerated development.

Learners can choose to implement

• Virtual Assistant
• Content Generation (Marketing Co-pilot)
• Conversational Agent (Legal & Compliance Assistant)
• Content Personalizer
• MCP Chatbot — an AI agent with calendar, CRM, and API integrations

Outcome

A production-ready multi-agent application demonstrating mastery of reasoning, retrieval, tool use, and protocol-driven interoperability.

Securing multi-agent systems

90 min · Lecture · Discussion · Practical exercise

• Sensitive Data Protection: Configure identification and obfuscation of sensitive data.
• Secure GenAI agents: Implement the “Defense-in-Depth” pattern to secure agents at the application layer.
• Red Team: Audit and verify security controls using Red Teaming techniques.
• Practical exercise: Harden security of the real multi-agent application in a cloud environment.