🎛️
Intelligent Control Systems
(Summary)
Week 1: Non-linear Dynamics & Control Learning
1.1: Introduction to ICS
Intelligent Control: Deals with using machine learning to solve control problems
Machine Learning: The study of computer algorithms that improve automatically
through experience
Regression
Classification
Dimensionality Reduction
Control Theory: A branch of engineering and applied mathematics stydying the
design and analysis of control algorithms-often using the language of dynamical
systems
Why should control theorists care about ML?
Suffiecient a priori knowledge may not be available
May be hard or impossible to model all possible working conditions in advance
May be hard to formulate problem in a way that fits standard control technique
In non-linear case, control theory is good at ensuring stability
Machine learning is exciting
How to include ML into the control loop?
Model Learning (non-linear identification)
Model is (pre-)learned.
This demonstrates first the learning loop that takes the control input into the
system dynamics and the learned model and compares the output to iterate.
Once model is learned, the system dynamics can be removed.
Intelligent Control Systems (Summary) 1
, Direct Control Learning
Controller is learned to drive the error signal to zero
Learned to maximise the reward
Examples
Learning Feedforward (ILC, IL)
Learning Feedback (AC, RL, IL)
1.2: Learning the Model
Assumptions: State is measurable
Intelligent Control Systems (Summary) 2
, TC and TD stand for Time Continuous and Time Discrete
Affine function: Is a composition of a linear function with a translation.
TD Model learning :
xk , uk − −f − − > xk+1
Using current state and control action, the model f learns the state of the next time
step.
Dataset Generation:
All relevant of portions of the state space must be explored
Generate varying signals
Chirp
Sequence of steps with random amplitude
Random walks
Varying initial conditions & letting system evolve
TC Model Learning:
Intelligent Control Systems (Summary) 3
, Same idea, simply using gradients.
Assumption: State is not measurable
Only access to set of measurements in time & control inputs
Only have access to output y
One cannot simply input the output as the state… Doesn’t always work like that.
Instead, we can take multiple derivatives of the output and use:
Normal Form
More general form is called NARX Model
(Non-linear Auto-Regressive eXogenous model)
Intelligent Control Systems (Summary) 4
Intelligent Control Systems
(Summary)
Week 1: Non-linear Dynamics & Control Learning
1.1: Introduction to ICS
Intelligent Control: Deals with using machine learning to solve control problems
Machine Learning: The study of computer algorithms that improve automatically
through experience
Regression
Classification
Dimensionality Reduction
Control Theory: A branch of engineering and applied mathematics stydying the
design and analysis of control algorithms-often using the language of dynamical
systems
Why should control theorists care about ML?
Suffiecient a priori knowledge may not be available
May be hard or impossible to model all possible working conditions in advance
May be hard to formulate problem in a way that fits standard control technique
In non-linear case, control theory is good at ensuring stability
Machine learning is exciting
How to include ML into the control loop?
Model Learning (non-linear identification)
Model is (pre-)learned.
This demonstrates first the learning loop that takes the control input into the
system dynamics and the learned model and compares the output to iterate.
Once model is learned, the system dynamics can be removed.
Intelligent Control Systems (Summary) 1
, Direct Control Learning
Controller is learned to drive the error signal to zero
Learned to maximise the reward
Examples
Learning Feedforward (ILC, IL)
Learning Feedback (AC, RL, IL)
1.2: Learning the Model
Assumptions: State is measurable
Intelligent Control Systems (Summary) 2
, TC and TD stand for Time Continuous and Time Discrete
Affine function: Is a composition of a linear function with a translation.
TD Model learning :
xk , uk − −f − − > xk+1
Using current state and control action, the model f learns the state of the next time
step.
Dataset Generation:
All relevant of portions of the state space must be explored
Generate varying signals
Chirp
Sequence of steps with random amplitude
Random walks
Varying initial conditions & letting system evolve
TC Model Learning:
Intelligent Control Systems (Summary) 3
, Same idea, simply using gradients.
Assumption: State is not measurable
Only access to set of measurements in time & control inputs
Only have access to output y
One cannot simply input the output as the state… Doesn’t always work like that.
Instead, we can take multiple derivatives of the output and use:
Normal Form
More general form is called NARX Model
(Non-linear Auto-Regressive eXogenous model)
Intelligent Control Systems (Summary) 4
