Agents

• Motivation: Modeling decision-making

• Agents as probabilistic programs
  • Expected utility
  • (Stochastic) transition functions
  • Softmax-optimal decision making
• Sequential decision-making
  • Markov decision processes
  • Expected utility recursion
  • Dynamic programming
  • Gridworld
• Sequential decision-making, part 2
  • Policies
  • Expected action values
  • A neat example: visualizing planned trajectories for a hyperbolic agent
• Multi-agent scenarios
  • Schelling coordination games
  • Game playing: tic-tac-toe
  • Language understanding
    • Scalar implicature
    • Hyperbole

Advanced topics:

• Agents learning from observations
  • POMDPs
  • Expected utility of state recursion
  • Bandit problems
  • Scalar implicature with beliefs
• Biased and bounded agents, intro
• Hyperbolic discounting
• Myopic planning
• Reasoning about agents
  • Inferring beliefs from observed behavior
  • Inferring preferences
  • Inferring biases (discounting parameter, planning horizon)
  • Joint inference of beliefs, biases, and preferences