Large Language Models Bootcamp

Agentic AI Applications

Generative AI fundamentals

Transformers and Attention

Hybrid Search

LLM Evaluation

LLM Fine Tuning

Observability and Monitoring

Guardrails and Responsible AI

Master building AI applications and agentic workflows.

Learn to build agentic AI applications

Learn the entire LLM application stack

In just one week, we will teach you how to build agentic AI applications.

Learn the entire LLM application stack.

LLM fundamentals, Transformers, attention mechasim
Vector databases, semantic and hybrid search
Retrieval-augmented generation (RAG)
Langchain fundamentals
Building multi-agent applications
Observability and monitoring
Evaluation datasets, tasks and metrics
Guardrails and responsible AI
Knowledge graphs and graph RAG
Fine tuning and deploying a large lagnuage model

Instructors and guest speakers

Learn from though leaders at the forefront of building agentic AI applications

Luis Serrano

Founder, Serrano Academy

Raja Iqbal

Founder, Ejento AI

Sebastian Witalec

Director of Developer Relations, Weaviate

John Gilhuly

Head of Developer Relations, Arize AI

Kartik Talamadupula

Head of AI, Wand AI

Jerry Liu

CEO/Co-founder, LlamaIndex

Zain Hasan

Senior DevRel Engineer, Together AI

Sage Elliot

AI Engineer, Union AI

Sophie Daly

Staff Data Scientist, Stripe

Rehan Jalil

Co-Founder | CEO, Securiti AI

Adam Cowley

Developer Advocate, Neo4j

Hamza Farooq

Founder, Travesaal AI

Loved by customers and partners

More than 10,000 working professionals have gone through our training program and recommend us.

"Partnering with Data Science Dojo aligns with our mission to make data science accessible. Their bootcamps contribute to safe AI deployment education."

"LLM bootcamp provides exceptional, hands-on initiation into LLMs and practical applications. A deep dive into the subject with theoretical knowledge."

"Six months of course material access post-training is valuable. Instructors ensure success, making it a top-notch learning experience in AI."

"Instructors simplified complex topics effectively. Hands-on learning enhanced my LLM understanding in 5 days. Intensive but immeasurable rewards."

"A rewarding opportunity to help students of all backgrounds learn new techniques. Being part of a passionate data science community was a real privilege."

"LLM Bootcamp provides hands-on experience with expert instructors, a comprehensive framework, and extensive resources for substantial upskilling."

"The effort in providing resources was commendable. Teachers and assistants were helpful, making it one of the best courses. I learned a lot for future use."

"Top-notch speakers, hands-on workshops, and networking make it a wonderful tech experience. Raja's in-depth teaching focuses on learning concepts."

"Comprehensive curriculum in generative AI, prompt engineering, and data retrieval. Excited about practical training opportunities."

"Collaborating with Data Science Dojo nurtures the next generation of LLM developers, commendable for fostering a dedicated creator community."

"This boot camp provided great content, equipping me with skills and confidence for efficient job execution. Enjoyable class and projects, a rewarding decision."

"The LLM Bootcamp offered an enlightening, well-structured learning experience, invaluable networking, and insights into real-world AI challenges. "

"Seamless navigation, invaluable hands-on exercises, and well-structured technical aspects for optimal, cost-effective results. Highly recommended for practical LLM knowledge."

"Comprehensive curriculum for in-depth understanding, seamless hands-on learning with cloud-based tools, and insightful talks by industry experts. Highly recommended."

"Outstanding boot camp, engaging discussions on coding and data security. A valuable and refreshing experience, recommended for advancing data science skills."

"Explored Data Science Dojo's LLM Bootcamp, gaining confidence with a comprehensive understanding. In-person experience, insightful instructors, and diverse insights."

"LLM Bootcamp surpassed expectations, bridging theory with practical examples. Engaging with cohorts, invaluable insights, and balanced intensity elevated understanding. Highly recommended for language models."

"LLM Bootcamp enriched my understanding of the language-tech intersection. Collaborating with peers was wonderful. Sophie DA's insightful presentation on practical applications is recommended for foundational knowledge."

"LLM Bootcamp shifted my problem-solving approach, sparking creativity with limitless applications. Highly recommended for anyone building their toolkit, fostering creativity in problem solvers, and providing invaluable insights for diverse applications."

"Dedicated a week to a profound learning experience with Data Science Dojo, sparking innovative ideas. The practical approach revolutionized our models. Highly recommended."

"Engaging with diverse participants and the Dojo staff provided valuable insights. Emphasizes the importance of understanding model performance. Highly recommended for all roles."

"The bootcamp's blend of theory and hands-on experience suited learners of all backgrounds. In just five days, I gained invaluable knowledge and tools for navigating the AI landscape. "

"The boot camp surpassed my expectations with insightful talks, practical insights, and comprehensive resources, essential for accelerating knowledge in LLMs."

"The Data Science Dojo Bootcamp exceeded expectations, offering comprehensive learning, hands-on experience, and valuable insights, transforming my confidence in AI."

"The Bootcamp accelerated my text analysis skills by engaging in in-person experience, industry insights, and valuable hands-on learning. Highly recommended!"

"The Bootcamp helped me bridge the gap between academia and industry, offering insightful talks and practical problem-solving exercises. "

"The Bootcamp exceeded my expectations, offering comprehensive training for both beginners and experienced data scientists. Highly recommended for all!"

Technologies and Tools

Explore the bootcamp curriculum

Overview of the topics and practical exercises.

Duration: 120 mins

LLM Application Architectures

Understanding the components of a large-scale enterprise LLM application

Lecture | In-class discussion

In this module, we will understand the common use cases of large language models and the fundamental building blocks of such applications. Learners will be introduced to the following topics:

Large language models and foundation models
Prompts and prompt engineering
Context window and token limits
Embeddings and vector databases
Build custom LLM applications by:
- Training a new model from scratch
- Fine-tuning foundation LLMs
- In-context learning
Canonical architecture for an end-to-end LLM application

Duration: 60 mins

Challenges and Risks

Key challenges and risks in enterprise adoption of large language models

Lecture | In-class discussion

In this module, we will explore the primary challenges and risks associated with adopting generative AI technologies. Learners will be introduced to the following topics at a very high level without going into the technical details:

Misaligned behavior of AI systems
Handling complex datasets
Limitations due to context length
Managing cost and latency
Addressing prompt brittleness
Ensuring security in AI applications
Achieving reproducibility
Evaluating AI performance and outcomes

Duration: 60 mins

Transformers and Attention

A comprehensive introduction to attention mechanism and transformer architecture

Lecture | In-class discussion | Practical Exercise

Dive into the world of large language models, discovering the potent mix of text embeddings, attention mechanisms, and the game-changing transformer model architecture.

Review of neural networks, deep learning and other fundamentals
Encoder/decoder
Transformer networks: tokenization, embedding, positional encoding and transformers block
Attention mechanism
- Self-Attention
- Multi-head Attention
- Transformer models

Duration: 240 mins

Vector Databases

A comprehensive introduction to vector databases

Lecture | In-class discussion | Practical Exercise

Learn about efficient vector storage and retrieval with vector database, indexing techniques, retrieval methods, and hands-on exercises.

Overview
- Rationale for vector databases
- Importance of vector databases in LLMs
- Popular vector databases
Different types of search
- Vector search, text search, hybrid search
Indexing techniques
- Product Quantization (PQ), Locality Sensitive Hashing (LSH) and Hierarchical Navigable Small World (HNSW)
Retrieval techniques
- Cosine Similarity, Nearest Neighbor Search
Advanced Retrieval Augmented Generation techniques
- Limitations of embeddings and similarity in semantic search
- Query transformation for better retrieval
- Relevance scoring in hybrid search using Reciprocal Rank Fusion (RRF)
- Using auto-cut feature to remove irrelevant results dynamically
- Improving search relevance by using language understanding to re-rank search results
Challenges using vector databases in production
- Scaling optimization
- Reliability optimization
- Cost optimization
Hands-on Exercise
- Learn how to perform similarity searches with vectors as input.
- Learn how to perform queries using vector similarity searches with embedding models and vectors.
- Learn how to combine the results of a vector search and a keyword (BM25F) search using hybrid search approach.
- Learn how to use multi-tenancy features for the efficient and secure management of data across multiple users or tenants.
- Learn how to compress vectors using product quantization to reduce memory footprint.

Duration: 60 mins

Prompt Engineering

An introduction to prompt engineering fundamentals

Lecture | In-class discussion | Practical Exercise

Unleash your creativity and efficiency with prompt engineering. Seamlessly prompt models, control outputs, and generate captivating content across various domains and tasks.

Prompt Design and Engineering
- Crafting Instructions for Effective Prompting
- Utilizing Examples to Guide Model Behavior
Innovative Use Case Development
- Tailoring Prompts to Goals, Tasks, and Domains
- Practical Examples:
  - Summarizing Complex Reports
  - Extracting Sentiment and Key Topics from Texts
Understanding and Mitigating Prompt Engineering Risks
- Identifying Common Risks: Prompt Injection, Prompt Leaking, Jailbreaking
- Best Practices for Secure Prompt Engineering
Advanced Prompting Techniques
- Enhancing Performance with Few-Shot and Chain-of-Thought (CoT) Prompting
- Exploring Program-aided Language Models (PAL) and ReAct Methods

Duration: 240 mins

Fine Tuning

A practical introduction to fine tuning

Lecture | In-class discussion | Practical Exercise

In-depth discussion on fine-tuning of large language models through theoretical discussions, exploring rationale, limitations, and Parameter Efficient Fine Tuning.

Fine Tuning Foundation LLMs
- Transfer learning, knowledge distillation and Fine-tuning
- Different fine-tuning techniques
- Limitations for fine-tuning
- Parameter-efficient fine-tuning in depth.
  * Quantization of LLMs
  * Low-Rank Adaptation (LoRA) and QLoRA
- Fine-tuning vs. RAG: When to use one or the other. Risks and limitations.
Hands-on Exercise:
- In-Class: Instruction fine-tuning, deploying, and evaluating a LLaMA2-7B 4-bit quantized model

Duration: 240 mins

Model Context

Introduction to prompt templates, retrievals, document loaders, and memory for context

Lecture | In-class discussion | Practical Exercise

Introduction to prompt templates, retrievals, document loaders, and memory for context

Build LLM Apps using LangChain. Learn about LangChain’s key components such as models, prompts, parsers, memory, chains, and Question-Answering.

Introduction to LangChain:
- Why do we need an orchestration tool for LLM application development?
- What is LangChain?
- Different components of LangChain
Why are orchestration frameworks needed?
- Eliminate the need for foundation model retraining
- Overcoming token limits
- Connecters for data sources
Interface with any LLM using model I/O
- Model I/O overview
- Components of model I/O: Language models, chat models, prompts, example selectors, and output parsers
- Overview of prompts, prompt templates, and example selectors
- Different types of models: language, chat, and embedding models
- Structuring language model responses using various types of output parsers
Connecting external data with LLM application with retrieval
- Retrieval overview
- The rationale for the requirement of retrieval and how does it work with LangChain
- Components of retrieval: Document loaders, text splitters, vector stores, and retrievers
- Loading public, private, structured, and unstructured data with document loaders
- Transforming documents to fewer chunks and extracting metadata using document transformers
- Embedding and vector stores for converting documents into vectors and for efficient storage and retrieval
- Optimizing retrieval using different retrieval techniques available in LangChain
Creating complex LLM workflows with chains
- Chains overview
- Various foundational chain types: LLM, router, sequential, and transformation
- Summarizing large documents using different document chains like stuff, refine, and map-reduce
Retain context and refer to past interactions with the memory component
- How memory can empower AI applications
- Different types of memories: simple buffer memory, conversation summarization, vector-store-backed-memory
- Overcoming token limit by using memory based on summarization of past conversations
- Utilize vector stores for memory

Duration: 240 mins

Agentic Workflows

Introduction to tools and multi-agent workflows

Lecture | In-class discussion | Practical Exercise

Dynamic decision-making with LLMs using agents
- Agents overview
- Components of agents: Tools, toolkits, prompt, and memory
- Different types of agents: Self-ask with search, ReAct, JSON chat, structured chat
- Working with agents using LangGraph
Monitoring and logging using callbacks
- Monitoring LLM application using callbacks
- Understanding how callbacks work with different events

Dynamic decision-making with LLMs using agents
- Agents overview
- Components of agents: Tools, toolkits, prompt, and memory
- Different types of agents: Self-ask with search, ReAct, JSON chat, structured chat
- Working with agents using LangGraph
Monitoring and logging using callbacks
- Monitoring LLM application using callbacks
- Understanding how callbacks work with different events
A Practical Guide to Coordinated LLM Agents Using LangGraph
- Nodes (functions or agents)
- Edges (data/control flow)
- Cycles (iteration, self-correction)
- State management (memory, messages, tools)
- Create 2–3 simple agents (e.g., Researcher, Critic, Summarizer)
- Use `Runnable` interfaces or ToolRunnable` for integration
- Add memory or context passing between agents

Duration: 240 mins

Retrieval Augmented Generation (RAG)

Challenges in building an enterprise RAG pipeline

Lecture | In-class discussion | Practical Exercise

In this module, we’ll explore the challenges in developing RAG-based enterprise-level Large Language Model (LLM) applications. We will discuss the following:

Basic RAG pipeline: Limitations of naïve approach
Indexing: Chunking size optimization. Embedding Models
Querying Challenges: Large Document Slices. Query Ambiguity
Query Optimizations: Multi-Query Retrieval. Multi-Step Retrieval. Step-Back Prompting. Query Transformations
Retrieval Challenges: Inefficient Retrieval of Large Documents. Lack of Conversation Context.
Complex Retrieval from Multiple Sources.: Hybrid Search and Meta-data integration. Sentence window retrieval. Parent-child chunk retrieval. Hierarchical Index Retrieval. Hypothetical Document embeddings (HyDE).
Generation Challenges: Information Overload. Insufficient Context Window. Chaotic Contexts. Hallucination. Inaccurate Responses.
Generation Optimization: Information Compression. Thread of Thought (ThoT). Generator Fine-tuning. Adapter methods. Chain of Note (CoN). Expert Prompting
Access control and governance

Duration: 180 mins

LLM Evaluation

Evaluating RAG-based LLM applications

Lecture | In-class discussion | Practical Exercise

Dive into large language model (LLM) evaluation, examining its importance, common issues, benchmark datasets, and key metrics such as BLEU, ROUGE, and RAGAs, and apply these insights through a hands-on summarization exercise.

Introduction to LLM evaluation
- What is evaluation and why is it important for LLMs?
- Overview of common mistakes made by LLMs
- A brief introduction to benchmark datasets and metrics
- Common LLM evaluation tasks
Benchmark datasets
- Explore datasets for different tasks including natural language understanding, reasoning, knowledge retrieval, etc.
- Learn about different datasets such as MMLU, HELM, and BBH.
Evaluation metrics
- Explain commonly used automatic metrics (BLEU, ROUGE, BERTScore)
- Compare strengths and weaknesses of different metrics
- Discuss the role of human evaluation and techniques (Likert scale)
RAGAS
- Introduction and basic workflow
- Evaluation metrics: Faithfulness. Context precision. Answer relevancy. Context recall
- Detailed workflow stages
- Practical Applications: Summarization. Open-domain QA. Fact-checking
Hands-on exercise
- Evaluating LLMs summarization using metrics like ROUGE, METEOR, and Bertscore
- Evaluation using G-Eval
- Evaluation of end-to-end RAG pipeline with RAGAs

Duration: 240 mins

Capstone Project

Building and deploying an LLM application

Lecture | In-class discussion | Practical Exercise

On the last day of the LLM bootcamp, the learners will apply the concepts and techniques learned during the bootcamp to build an LLM application. Learners will choose to implement the following:

Naive RAG assistant: A simple RAG assistant designed to answer general queries.
Tool connection: An advanced agent that integrates with your data to provide more tailored responses.
Connect web search tool: Extending your agent by adding web search and other sources.

Attendees will receive the following:

Comprehensive Datasets: Access a vast collection of documents from a variety of industries to support your project’s data needs and ensure robust functionality.
Step-by-Step Implementation Guides: Detailed instructions that guide you through each phase of your project, from initial setup to final deployment.
Ready-to-Use Code Templates: Utilize code templates available in Data Science Dojo’s sandbox environments to streamline the development process and get your application up and running quickly.
Cloud-Based Resources: Gain exclusive access to powerful cloud resources, including your own OpenAI key, facilitating the hassle-free deployment of your application on platforms like Streamlit.

At the culmination of the bootcamp, you will have a fully operational LLM application deployed on a public cloud platform, such as Streamlit. This deployment process includes setting up a continuous integration and continuous deployment (CI/CD) pipeline to ensure that your application can be updated and maintained effortlessly. By the end of the bootcamp, you’ll be equipped not only with a finished project but also with the knowledge and skills to deploy and scale applications in real-world scenarios.

Course Schedule

Daily schedule: 9 am - 5 pm PT | Breakfast, lunch and beverages | Breakout sessions and in-class activities

Seattle / Instructor-Led Online June 09-13, 2025

Can't find your location?

Request your location

Attend the LLM Bootcamp for free

We Accept Tuition Benefits

All of our programs are backed by a certificate from The University of New Mexico, Continuing Education. This means that you may be eligible to attend the bootcamp for FREE.

Not sure? Fill out the form so we can help.

Get a certificate from The University of New Mexico Continuing Education with 5 CEUs

Frequently Asked Questions

Are there any prerequisites?

Yes, a very basic level of python programming language.

What software do I need to install on my laptop?

Just bring your laptop. We will provide all software, subscriptions, and browser-based sandboxes.

Is the program accredited?

Yes. You will receive a certificate from The University of New Mexico with 5 CEUs.

Are GPU costs and cloud subscriptions included?

Yes. During the bootcamp, you will be given all resources needed for completing the labs and exercises.

What is the refund policy?

Registrations are 100% refundable for requests received 5 business days before the bootcamp.

What is the duration of the bootcamp

The bootcamp is a 5-day, 40-hour program.

LLM - Online Courses

Reviews

Consulting

Community

Large Language Models Bootcamp

Learn the entire LLM application stack

Instructors and guest speakers

Luis Serrano

Raja Iqbal

Sebastian Witalec

John Gilhuly

Kartik Talamadupula

Jerry Liu

Zain Hasan

Sage Elliot

Sophie Daly

Rehan Jalil

Adam Cowley

Hamza Farooq

Loved by customers and partners

Technologies and Tools

Explore the bootcamp curriculum

LLM Application Architectures

Challenges and Risks

Transformers and Attention

Vector Databases

Prompt Engineering

Fine Tuning

Model Context

Agentic Workflows

Retrieval Augmented Generation (RAG)

LLM Evaluation

Capstone Project

Course Schedule

We Accept Tuition Benefits

Frequently Asked Questions

Are there any prerequisites?

What software do I need to install on my laptop?

Is the program accredited?

Are GPU costs and cloud subscriptions included?

What is the refund policy?

What is the duration of the bootcamp

Training Programs

Enterprise

Community

About