Multi-Agent systems
CONTENTS
Q-learning 8
Markov games 9
A CONCISE INTRODUCTION TO MAS AND Independent learning 9
DISTRIBUTED AI 2 Coupled learning 9
Sparse Cooperative Q-learning 9
Introduction 2 The problem of exploration 9
Characteristics of Multi Agent Systems 2
Agent design 2
Environment 2 AN INTRODUCTION TO AGENT-BASED
Perception 2 MODELING 10
Control 2
Knowledge 2 Why agent-based modeling (ABM)? 10
Communication 2
Applications 3 What is Agent-based modeling? 11
Challenging issues 3 EBM (Equation-based modelling) vs. ABM 11
For optimal decision making 3 When is ABM most useful? 11
Trade-offs 11
Rational agents 4
The past 4 Exploring and extending abms 12
The future 4
Creating agent-based models 12
Strategic games 5 Designing a model 12
Strategic games / game in normal form 5 Choose the agents 12
Solution concepts 5 Examining the model 12
Iterated elimination of strictly dominated actions
(IESDA) 5 The components of agent-based modelling 13
Nash Equilibrium (NE) 5 Agents 13
Agent cognition: 13
Coordination 6 Environments 13
A proposed general convention: 6 Spatial environments 13
Role assignment: 6 Network-based environments 14
The two solution methods for coordination graphs: 6 Special environments 14
Interactions 14
Learning 8 Observer/UI 14
Markov Decision Processes (MDP) 8 Schedule 14
Discounted future reward 8
Value iteration 8 Analyzing agent-based models 15
Statistics 15
, A CONCISE INTRODUCTION TO MAS AND DISTRIBUTED AI
INTRODUCTION
Agents Perceive their environment through sensors and act upon it through actuators
They have knowledge, perceptions, communication, and control (actions)
Rational agents An agent that always tries to optimize an appropriate performance measure.
AI The study of the principles and design of artificial rational agents
Multi-agent systems MAS Systems where agents coexist with other agents
Distributed AI The field that focusses on Multi-Agent systems
CHARACTERISTICS OF MULTI AGENT SYSTEMS
Agent design
Agent heterogeneity affects all functional aspects of an agent, from perception to decision making
Heterogeneous MAS a MAS whose agents are based on different hardware, or that implement different behaviors
Homogeneous MAS a MAS whose agents are designed in an identical way with a priori on the same capabilities
Environment
Multi Agent Systems can exist in both static and dynamic environments.
Single agents The AI is usually developed for static environments, as this is easier, and it allows for more
rigorous mathematical treatments
Multi agent Systems The environment appears dynamic to individual agents. This can be problematic with e.g.
simultaneously learning agents where non-stable behaviors can be observed.
It’s also difficult to define what parts of a dynamic system one agent should treat as other agents
Perception
Collective information is distributed: The information that reaches different agents in a MAS can differ spatially,
temporally, or semantically, each of which requires different interpretations.
The world is then partially observable. This can cause problems for Optimal Multi Agent Systems planning.
Another issue is about sensor fusion: How to optimally combine perceptions to increase collective knowledge.
Control
Decentralized control The decision-making of each agent lies within that agent (this is common in MAS).
This is robust and fault-tolerant, but it’s not always easy to distribute the protocols
Game theory The general problem of Multi-Agent decision making
Collaborative MAS Team MAS: all agents share the same interest.
Distributed decision making offers asynchronous computation and speedups.
A downside is that appropriate coordination mechanisms must be created.
Knowledge
Common knowledge Every agent knows a fact, and knows that every other agent also knows it
In Single Agent Systems we assume that the agent knows its own actions, but not the corresponding consequences on
the world. In Multi Agent Systems, the levels of this knowledge can differ substantially.
Homogeneous agents are more likely to be aware of the other agents’ perceptions and actions compared to
heterogeneous agents.
Communication
Communication is used for (1) coordinating with other agents, or (2) for negotiating among self-interested agents.
Issues can be what network protocols to use for the information to arrive safely and timely, and what language to use.
Communication in a MAS This is a two-way process, where all agents can be senders and receivers of messages.
CONTENTS
Q-learning 8
Markov games 9
A CONCISE INTRODUCTION TO MAS AND Independent learning 9
DISTRIBUTED AI 2 Coupled learning 9
Sparse Cooperative Q-learning 9
Introduction 2 The problem of exploration 9
Characteristics of Multi Agent Systems 2
Agent design 2
Environment 2 AN INTRODUCTION TO AGENT-BASED
Perception 2 MODELING 10
Control 2
Knowledge 2 Why agent-based modeling (ABM)? 10
Communication 2
Applications 3 What is Agent-based modeling? 11
Challenging issues 3 EBM (Equation-based modelling) vs. ABM 11
For optimal decision making 3 When is ABM most useful? 11
Trade-offs 11
Rational agents 4
The past 4 Exploring and extending abms 12
The future 4
Creating agent-based models 12
Strategic games 5 Designing a model 12
Strategic games / game in normal form 5 Choose the agents 12
Solution concepts 5 Examining the model 12
Iterated elimination of strictly dominated actions
(IESDA) 5 The components of agent-based modelling 13
Nash Equilibrium (NE) 5 Agents 13
Agent cognition: 13
Coordination 6 Environments 13
A proposed general convention: 6 Spatial environments 13
Role assignment: 6 Network-based environments 14
The two solution methods for coordination graphs: 6 Special environments 14
Interactions 14
Learning 8 Observer/UI 14
Markov Decision Processes (MDP) 8 Schedule 14
Discounted future reward 8
Value iteration 8 Analyzing agent-based models 15
Statistics 15
, A CONCISE INTRODUCTION TO MAS AND DISTRIBUTED AI
INTRODUCTION
Agents Perceive their environment through sensors and act upon it through actuators
They have knowledge, perceptions, communication, and control (actions)
Rational agents An agent that always tries to optimize an appropriate performance measure.
AI The study of the principles and design of artificial rational agents
Multi-agent systems MAS Systems where agents coexist with other agents
Distributed AI The field that focusses on Multi-Agent systems
CHARACTERISTICS OF MULTI AGENT SYSTEMS
Agent design
Agent heterogeneity affects all functional aspects of an agent, from perception to decision making
Heterogeneous MAS a MAS whose agents are based on different hardware, or that implement different behaviors
Homogeneous MAS a MAS whose agents are designed in an identical way with a priori on the same capabilities
Environment
Multi Agent Systems can exist in both static and dynamic environments.
Single agents The AI is usually developed for static environments, as this is easier, and it allows for more
rigorous mathematical treatments
Multi agent Systems The environment appears dynamic to individual agents. This can be problematic with e.g.
simultaneously learning agents where non-stable behaviors can be observed.
It’s also difficult to define what parts of a dynamic system one agent should treat as other agents
Perception
Collective information is distributed: The information that reaches different agents in a MAS can differ spatially,
temporally, or semantically, each of which requires different interpretations.
The world is then partially observable. This can cause problems for Optimal Multi Agent Systems planning.
Another issue is about sensor fusion: How to optimally combine perceptions to increase collective knowledge.
Control
Decentralized control The decision-making of each agent lies within that agent (this is common in MAS).
This is robust and fault-tolerant, but it’s not always easy to distribute the protocols
Game theory The general problem of Multi-Agent decision making
Collaborative MAS Team MAS: all agents share the same interest.
Distributed decision making offers asynchronous computation and speedups.
A downside is that appropriate coordination mechanisms must be created.
Knowledge
Common knowledge Every agent knows a fact, and knows that every other agent also knows it
In Single Agent Systems we assume that the agent knows its own actions, but not the corresponding consequences on
the world. In Multi Agent Systems, the levels of this knowledge can differ substantially.
Homogeneous agents are more likely to be aware of the other agents’ perceptions and actions compared to
heterogeneous agents.
Communication
Communication is used for (1) coordinating with other agents, or (2) for negotiating among self-interested agents.
Issues can be what network protocols to use for the information to arrive safely and timely, and what language to use.
Communication in a MAS This is a two-way process, where all agents can be senders and receivers of messages.
