Task Environment

This is the entire external problem space of the agent. Agent gets percepts, produce outcomes within this environment. This serves as the problem statement that the agent is trying to solve.

Boundary

the line between the agent and task environment is decided by what control agent has. If the agent has control over something with 100% certainty, it belongs to the agent. Anything the agent cannot directly change with 100% certainty is part of the environment.

Ex: A robot's motor driver is part of the agent, but the physical motor gears are part of the environment, because a gear can jam, slip, or break regardless of what the code commands.

Dimensions of Task Environment

The task environment is defined using these $6$ dimensions.

Fully Observable vs. Partially Observable

1. Fully Observable - Allows agent to detect complete state of the system using sensors at any given time.
2. Partially Observable - contain missing data, noise, or unobserved variables

Single-Agent vs. Multi-Agent

1. Single-Agent - agent works alone without other entities affecting its performance measures.
2. Multi-Agent - other entities work with, or against the agents performance measure

Deterministic vs. Stochastic

1. Deterministic - Next State = AgentAction(Previous State)
2. Stochastic - Next State might be interfered by external bodies, involve uncertainty

Episodic vs. Sequential

1. Episodic - Environment is broken into isolated, independent episodes. each episode does NextState = AgentAction(PrevState), after that everything resets.
2. Sequential - Every action alters environment and they sequentially adds up

Static vs. Dynamic

1. Static - Environment remains unchanged until no action taken by agent
2. Dynamic - Environment will change continuously regardless of action by agent (like a fluid flow)

Discrete vs. Continuous

1. Discrete - No. of different states in the system are finite
2. Continuous - No. of different states are not finite and continuous (like time, velocity)