Summary Lecture #1 Airline schedules (Aircraft, crew, passengers) are optimized leading to: Little slacksidle time) Schedule dependencies Delay chain effects Causes of schedule disruptions Shortages of airline resources Shortages of airport resources Complex airline resource regulations Aircraft maintenance

Outline Airline planning complexity Schedule disruptions Schedule dependability CDM Ground delay Program How do airlines recover the schedule? Traditional ARM; Model shortcomings Interdependency of passengers and aircraft operations

Outline Objectives Agents and Their building Blocks Principles for building agents Modeling formalisms Algorithmic Principles Building an agent: The Mars Exploration rover

Markov Decision Processes and Reinforcement Learning · Motivation earning policies through reinforcement Q values

How Might a mouse search a Maze for Cheese? heese · State Space Search? As a Constraint Satisfaction Problem? Goal-directed Planning As a rule or production System? What is missing? Ideas in this lecture Objective is to accumulate rewards rather than goal states Task is to generate policies for how to act in all situations rather than a plan for a single starting situation

Robot Localization using SIR Take sample set X+_11 ll. iterate For each sample xt-1 1) Sample from p(X,1 X+1, a) 2) Attach importance weights

Probabilistic model Integrate over al!p(x1 Bel(x,)=ap(=;|x,)p(x1a1,=12a1-2,=0) ap(E, x )p(x, Ix, 1,a, p(|x,)p(x,1x1,a1)p(x1)d

Assignment problem Persons and tasks connected by links Objective: Minimize total task time minutes Carburetor Manny 4 minutes Valves

Courtesy or Eric Feron and Sommer Gentry. Used with permission. Introduction to Linear Programming Eric Feron (updated Sommer Gentry)

Integer Programs LP: Maximize 3x1+ 4x2 IP: Maximize 3x1+ 4x2 Subiect to: Subject te