Sequential Decision Making
- 00. Mind Map
- 00. What is Sequential Decision Making
- 01. Reward Hypotesis - Sutton and Barto 1998
- 01. Example - Load Unload MDP
- 01. Implementations for Value and Policy Iteration
- 01. Improving policies
- 01. MDP Recap
- 01. Methods of learning - classical, model planning
- 01. Methods of learning - experience, model free learning
- 01. Online vs Offline planning
- 01. Policies
- 01. Policy Iteration
- 01. Value Iteration
- 01. What are the Q and V functions
- 02. Aproximate POMDP algorithms
- 02. Faster Exact POMDP Solvers - Linear Programming
- 02. Piece-wise Linear Function
- 2. Piecewise Linear Funtion - Vector Pruning
- 02. Planning under uncertainty
- 02. Policy Types
- 02. POMDP - Piecewise Linear Solving
- 02. POMDP Theoretical Solver Implementations
- 03. Example - Partially Observable Monte Carlo Planning
- 03. Online Planning
- 03. Online Planning in Partially Oservable MDP
- 03. Partially Observable Monte Carlo Planning
- 03. Recap - How does MPC work
- 03. Sparse UTC
- 04. Constraint Satisfaction Problem
- 04. Control
- 04. Foundations of MPC
- 04. Mathematics of MPC
- 04. Model Predictive Control
- 04. Model Predictive Control Advanced Maths
- 04. Receding Horizon Decision Making
- 04. RL vs MCP
- 04. Robustness - Positive robust invariant sets
- 04. Smoothness - Locally Lipschitz functions
- 04. Stability - Schur & Lyapunov Stability of linear systems
- 05. Economic MPC
- 05. Closed Loop Property of MPC
- 05. Constraints in MPC
- 05. Exercise - Reference Tracking MPC
- 05. Mathematical groundness of MPC
- 05. MCP Implementation
- 05. MPC Terminal ingredients choices - Q and R
- 05. MPC Terminal Set
- 05. Recursive feasibility
- 05. Stability
- 05. The MPC optimisation problem
- 06. Choosing K Matrix for Tube MPC
- 06. Closed-loop implementation of TMPC
- 06. Constraints tightening Tube MPC
- 06. Double System MPC - Nominal vs Actual System
- 06. Optimisation Problem Tube MPC
- 06. Robust stability in TMPC
- 06.Robust Tube-based MPC
- 07. Abstract MDP
- 07. Causal Bounds in Posterior Sampling RL
- 07. Cofounded Data
- 07. Learning The Model - Deep RL
- 07. Model Based Reinforcement Learning
- 07. Model Learning in Tabular MDPs
- 07. Simulators
- 07. SOTA Model RL
- 07. Stochastic bisimulation of Abstract MDPs
- 08 . Safe policy improvement
- 08. Aleatoric vs Epistemic Uncertainty
- 08. Bayesian Q Networks
- 08. Constraint RL
- 08. Dealing with External Undertainty - Bad Model
- 08. Dealing with Internal Undertainty
- 08. Deep Exploration
- 08. Diverse Projection Ensembles for Distributional RL
- 08. Ensamble Q Networks
- 08. Epistemic MCTS
- 08. How to deal with uncertainty
- 08. New Criteria
- 08. Safe and Robust Reinforcement Learning
- 08. Sequential Monte Carlo
- 08. Sequential Monte Carlo DQN
- 08. Teacher - Safe scenario model
- 08. Vulnearbility of Average
- 09. Bayesian Q-learning
- 09. Bootstrapped DQN - Ensamble of Q Networks
- 09. Causal Models for BA-POMDPs
- 09. Fully observable Model Based RL
- 09. Model based Bayesian RL
- 09. Model Free Bayesian RL
- 09. More powerful planning POMCP for BA-POMDPs
- 09. Partially Observable Model Based Bayesian RL - Bayesian Adaptive MDPs for POMDPs
- 09. Solving BA-POMDPs using POMCP
- 09. SOTA Bayesian RL
- 10. Human Compatible Decision Making
- 10. Adversarial Approaches
- 10. Applications of Inverse Reinforcement Learning
- 10. Behavioural Cloning
- 10. Comparison of Learning Techniques
- 10. DAGs for MDP
- 10. Deep Maximum Entropy Inverse RL
- 10. Direct Policy Learning
- 10. Dynamic programming for MDP
- 10. How Google Maps works. How it learns from past user behaviour
- 10. Imitation Learning
- 10. Inverse Reinforcement Learning
- 10. Inverse Reinforcement Learning Overview
- 10. Limitations of Inverse RL
- 10. Maximum Entropy Inverse RL
- 10. Monte Carlo Tree Search (MCTS)
- 10. Other approaches
- 11. Active Learning
- 11. Bi-directional Human-AI Alignment
- 11. Challenges & Critiques of RLHF
- 11. Direct Preference Optimization
- 11. Evaluation with Reward Learning
- 11. Explaining learned reward functions
- 11. Issues with Reward Learning
- 11. Multi-Objective Reinforced Active Learning - MORAL
- 11. Preference-based RL Overview
- 11. Reinforcement Learning from Human Feedback RLHF
- 11. RL from AI Feedback
- 11. RLHF Feedback
- 11. RLHF for LLMs
- 11. Text2Reward
- 12. Abstraction based Verification
- 12. AI Verification in Practice
- 12. Conclusion of Verificability of AI
- 12. Constraint-Based AI Verification - Satisfiability Modulo Theory (SMT)
- 12. Decision Tree AI Verification
- 12. Example - ACAS Xu System
- 12. Formal Definition of AI Verification
- 12. Linear temporal Logic (LTL) Basics
- 12. Lyapunov Based Verification
- 12. Transforming a NN into a SMT problem
- 12.Challenges of SMT-Based Verification
- 12.Verifiable Decisions in Machine Learning - Intuition
- 4110. Sequential Decision Making
- Lecture 2
- Lecture 3
- SDM_25_26_AI_Verification_II.pdf
- Untitled