Agent Program

This is the implementation of the agent function. Maps Percepts to actions.

Types

Simple Reflex Agent

These agents take actions on the current percept, not the historical percepts.
No memory
Simplest Agent
Works best on fully observable environments
Does not work in many situations

\text{Action} = f(\text{Current Percept})

def Simple_Reflex_Agent(percept):
    state = interpret_input(percept)
    rule = rule_match(state, rules)
    return rule.action

Model based Reflex Agent

Simple reflex agent + Memory
Keeps track of the percepts in past, action is made based on current state.
Past percepts are used for figuring out the current state, action is made based on current state only
No planning, only following rules

\text{Action} = f(\text{Current Internal State})

def Model_Based_Agent(percept):
    state = update_state(state, action, percept, model)
    rule = rule_match(state, rules)
    action = rule.action
    return action

All reflex agents have no planning, they only follow rules based on current state.

Goal based Agent

make actions to achieve a defined goal
has planning capability
has Memory
use world state and end state (goal) to find a possible action
all actions are considered same, choosing one depends on the search algorithm
Goals are binary - either reach it or not (takes 1ms or 1yr does not matter)

\text{Action} = \text{Search/Plan}(\text{Current Internal State}, \text{Goal})

function GOAL-BASED-AGENT(percept) returns an action
    persistent: 
	    state, the agent's current internal model of the world
        model, rules explaining how the world evolves and how actions affect it
        goal, description of the desired end state
        plan, a sequence of actions, initially empty

    state <- UPDATE-STATE(state, percept, model)

    # Re-plan if we have no current plan or if the plan has been invalidated
    if plan is empty OR plan is invalid for current state:
        plan <- SEARCH(state, goal, model)  # Returns action sequence where GOAL(state) == True

    if plan is not empty:
        action <- FIRST(plan)
        plan <- REST(plan)
        return action
    
    return NO-OP

Utility based Agent

goal based agent + best action possible
has a Utility function that determines how good each action numerically
actions taken to reach a goal, but in the best way possible

\text{Action} = \text{Optimize}(\text{Current State}, \text{World Model}, \text{Utility Function})

Utility Function

Maps a state or a sequence of states into a numerical value

Happiness

Aka. satisfiction. The returned value of the utility function. Means how good the action is.

function UTILITY-BASED-AGENT(percept) returns an action
    persistent: 
	    state, the agent's current internal model of the world
        model, transition probabilities and world rules
		utility_function, mapping from state 's' to scalar value U(s)

    state <- UPDATE-STATE(state, percept, model)
    
    best_action <- null
    max_expected_utility <- -infinity

    for each action 'a' in POSSIBLE-ACTIONS(state):
        expected_utility <- 0
        
        # Sum over all possible outcome states s' from taking action 'a'
        for each possible outcome state s':
            prob <- TRANSITION-PROBABILITY(s', state, a, model)
            expected_utility <- expected_utility + (prob * utility_function(s'))
            
        if expected_utility > max_expected_utility:
            max_expected_utility <- expected_utility
            best_action <- a

    return best_action

Environment State Representation

Representation	Internal Structure	Core Concept	Example Paradigms
Atomic	None (Black Box)	World is a set of distinct, indivisible states	BFS, DFS, $A^*$ Search, MDPs
Factored	Fixed Vector of Variables	States share attributes but differ in values	Constraint Satisfaction Problems (CSPs), Propositional Logic
Structured	Objects + Relationships	States contain explicit objects and dynamic relations	First-Order Logic, Knowledge Graphs, Planning (PDDL)
![[environment-state.png]]

Agent Program

On this page