Prof .
Dr .
Christian B .
Mend) IN2388
Tensor Networks
Tensor
Networks Lecture
, Mathematical framework
2
. and
graphical diagrams
2
.1 Linear
Algebra fundamentals
·
Vector
Spaces : # ,
R ,
v = (le ,
or <(20 1] , ,
1) :
space of continuous functions : 10 17.
+
K
·
Matrices : AckM , A =
Ca
·
Matrix-vector product : A .
v
=
Can
where A be
interpreted linear from K"-kM
can as a
operator
Remark :
We will formulate definitions ,
theorems, . . .
in terms of
complex numbers ;
this includes real numbers as
a specialcase
V
r)
coss) in =
V C > VxV K
Inner product and :
vector
-
nor m on ...
space
=
· a
can be defined
abstractly in this course : (V .
W) = W
; Vrwe -W
↑ Note 7, ) linear its second but it anti-linear in its first Car W) &
*
S Wh
argument argument
is
=
:
in is :
,
, ,
C ,
> induces a
nam on V via NVI = Fo =K
= Euclidean length
eg .
c (20 17 .
,
4) :
<
fig)
=
So fix) g(x) dx
/ IvII I wll
inequality Kr Vv V
>
Cauchy-Schwartz :
w <
w =
.
, ,
Adjoint (conjugate transpose) of a matrix : At =
( ** )
>
-
A =
Cam
intritioMinor
set
Note :
<v
,
Aw) =< Atr ,
w ) wekm req
,
column
A matrix At KUM is called Hermitian (or self-adjoint) if At = A
nu2 !
Unitary Matrices : He
It is
Kath
the inverse
is called
of
unitary
U
if UTU =
I
cidentity maries ,
=>
U describes a
change
of basis which preserves the length of rectors
Note :
UTH =
I Unt =
I
Normal Matrix At CMM called it commutes A A += At A
normal if with its adjoint
: :
is
-
,
unitary and heritian normal
particular matrix is
~>
in :
every
(1 *, KV
Eigenvalues and
Eigenvectors Let A them vector called
:
a non-zero ve is an
eigenvector of A
with 1 K if Av 1
corresponding eigenvalue
=
Determinant of a
square Matrix At QUXU :
det (A) =
rEsu Sgn (0) apoc
with Su of permutations of [1 n)
group
:
....
,Properties :
·
A is invertible (column or now rectors a re
linearly independent
if det (A) + 0
it and
only
·
det CAT) =
det (A)
·
for all A ,
Be K" :
det (A
.
B) =
det(A) -
det (B)
Relevance for
eigenvalues
:
Av =
Av El
(AI-A)v =
0 = >
XI -
A is not invertible
= v
det (Al-A) =
0
=
"characteristic polynomial" of A
T
fundamental theorem of
algebra guarantees that there exist n
complex roots of
the characteristic polynomial =
eigenvalves ...
SPECTRAL DECOMPOSITION THEOREM
Any normal matrix At Kan is
unitarily diagonalizable ,
i .
e .
there exists a
unitary Matrix Ue purn
and
eigenvalves 1 .... An D such that A U .
= U .
(4 ...an ) ,
equivalently
:
An)
F
A =
U .
( ·
Ut ·
Conversely , every
matrix
representable in this form is nor mal .
A eart
:
The column rectors of U =
(Un : ...: Un) are a basis of
eigenvectors of A ,
since
AU U ( "... (n) that A
AjUj for
stating Uj jo
= =
is
.
.
. . .
n
Remark For form different
general matrices
cigenrectors
basis
eigenvectors
of
:
not
might
a
,
,
-
eigenvalves
a re not
necessarily orthogonal to each other
Homitian
Any Hermitian Matrix A
unitarily diagonalizable ( ...a ) =
=>
is
,
and its real
eigenvalues
:
a re
The At Dr
spectral radius of a
square Matrix is the
largest absolute value
of the of
eigenvalues A :
p(A) [Ixel In 19 on of A
with A
eigenvalves
:
=
Max .... ...
* herritian Matrix At Dr called scridefinite if Ar > 20 Free ,
is
positive sv ,
thus the of likewise (v , x)
eigenvales non-negative for :
such Matrix are since
any eigenpair
a
,
* 11vK2 =
1 .
(v v> ,
=
<V ,
1v> =
<V , Arl 20
SINGULAR VALUE DECOMPOSITION THEOREM -
for
square natrices
Let Ae path be matrix then there exists unitary matrices U,V and list
square
a ,
a
real with On =0 called values ,
of numbers ....
On EF22 . . .
singular
such that A =
U (0 :
on) V
+
roof :seelecturenotea matrix Coven
singular
ones) !
, to
The SVD be
generalized Matrices :
Remark
-
:
can non-square
SINGULAR VALUE DECOMPOSITION THEOREM -
general
(MXK
men
Let A set k =
min (M ,
n) ,
then there exists linear isometries UE
and Venk and real values On
,
non-negative On . . .
with z -. On 20 ,
such that A =
U ( "... on) vt
#te Isometries are
generalizations of
unitary matrices
: :
Uekman with Man is called an
isometry if UtU = In
The columns of U =
(U
:
...: un) a re orthonormal
(CUjueL =
Gje) ,
but
if m -n
they cannot form a basis of I" .
of A
The rank is the dimension of the rector
space generated by
the column reators of A = number of values)
n o n -ze ro
singular
Remark SVD) becomes low-rank factorization if values
singular
:
a are ze ro
- many
land can hence be omitted) :
↑
A u +
.... V
↑
=
↑
-onl
+w +
diag( .
MATRIX FUNCTIONS
Let f : -D function and At pren be
arbitrary be an normal
,
with
spectral decomposition A
=
U( n) Ut.
Then we define f(A) =
U(f(v) fan) :
Ut
Equivalently ,
if f admits a
power
series
expansion f(x) = 9X" * xeK,
then f(A) =
[09 : Al
for At C
Examples :
normal ....
·
et =
exp(A) = Al (note : et * =
AetA for teR)
·
# >
-
knen satisfies A =
A
DP
*
9
KRONECKER PRODUCT of two matrices At CMA ,
Be :
anT
. . .
CimB
a i (mPx nq
AB c
= . . .
am B
-
similarly for two rectors rel , we D :
vow
= ena
Dr .
Christian B .
Mend) IN2388
Tensor Networks
Tensor
Networks Lecture
, Mathematical framework
2
. and
graphical diagrams
2
.1 Linear
Algebra fundamentals
·
Vector
Spaces : # ,
R ,
v = (le ,
or <(20 1] , ,
1) :
space of continuous functions : 10 17.
+
K
·
Matrices : AckM , A =
Ca
·
Matrix-vector product : A .
v
=
Can
where A be
interpreted linear from K"-kM
can as a
operator
Remark :
We will formulate definitions ,
theorems, . . .
in terms of
complex numbers ;
this includes real numbers as
a specialcase
V
r)
coss) in =
V C > VxV K
Inner product and :
vector
-
nor m on ...
space
=
· a
can be defined
abstractly in this course : (V .
W) = W
; Vrwe -W
↑ Note 7, ) linear its second but it anti-linear in its first Car W) &
*
S Wh
argument argument
is
=
:
in is :
,
, ,
C ,
> induces a
nam on V via NVI = Fo =K
= Euclidean length
eg .
c (20 17 .
,
4) :
<
fig)
=
So fix) g(x) dx
/ IvII I wll
inequality Kr Vv V
>
Cauchy-Schwartz :
w <
w =
.
, ,
Adjoint (conjugate transpose) of a matrix : At =
( ** )
>
-
A =
Cam
intritioMinor
set
Note :
<v
,
Aw) =< Atr ,
w ) wekm req
,
column
A matrix At KUM is called Hermitian (or self-adjoint) if At = A
nu2 !
Unitary Matrices : He
It is
Kath
the inverse
is called
of
unitary
U
if UTU =
I
cidentity maries ,
=>
U describes a
change
of basis which preserves the length of rectors
Note :
UTH =
I Unt =
I
Normal Matrix At CMM called it commutes A A += At A
normal if with its adjoint
: :
is
-
,
unitary and heritian normal
particular matrix is
~>
in :
every
(1 *, KV
Eigenvalues and
Eigenvectors Let A them vector called
:
a non-zero ve is an
eigenvector of A
with 1 K if Av 1
corresponding eigenvalue
=
Determinant of a
square Matrix At QUXU :
det (A) =
rEsu Sgn (0) apoc
with Su of permutations of [1 n)
group
:
....
,Properties :
·
A is invertible (column or now rectors a re
linearly independent
if det (A) + 0
it and
only
·
det CAT) =
det (A)
·
for all A ,
Be K" :
det (A
.
B) =
det(A) -
det (B)
Relevance for
eigenvalues
:
Av =
Av El
(AI-A)v =
0 = >
XI -
A is not invertible
= v
det (Al-A) =
0
=
"characteristic polynomial" of A
T
fundamental theorem of
algebra guarantees that there exist n
complex roots of
the characteristic polynomial =
eigenvalves ...
SPECTRAL DECOMPOSITION THEOREM
Any normal matrix At Kan is
unitarily diagonalizable ,
i .
e .
there exists a
unitary Matrix Ue purn
and
eigenvalves 1 .... An D such that A U .
= U .
(4 ...an ) ,
equivalently
:
An)
F
A =
U .
( ·
Ut ·
Conversely , every
matrix
representable in this form is nor mal .
A eart
:
The column rectors of U =
(Un : ...: Un) are a basis of
eigenvectors of A ,
since
AU U ( "... (n) that A
AjUj for
stating Uj jo
= =
is
.
.
. . .
n
Remark For form different
general matrices
cigenrectors
basis
eigenvectors
of
:
not
might
a
,
,
-
eigenvalves
a re not
necessarily orthogonal to each other
Homitian
Any Hermitian Matrix A
unitarily diagonalizable ( ...a ) =
=>
is
,
and its real
eigenvalues
:
a re
The At Dr
spectral radius of a
square Matrix is the
largest absolute value
of the of
eigenvalues A :
p(A) [Ixel In 19 on of A
with A
eigenvalves
:
=
Max .... ...
* herritian Matrix At Dr called scridefinite if Ar > 20 Free ,
is
positive sv ,
thus the of likewise (v , x)
eigenvales non-negative for :
such Matrix are since
any eigenpair
a
,
* 11vK2 =
1 .
(v v> ,
=
<V ,
1v> =
<V , Arl 20
SINGULAR VALUE DECOMPOSITION THEOREM -
for
square natrices
Let Ae path be matrix then there exists unitary matrices U,V and list
square
a ,
a
real with On =0 called values ,
of numbers ....
On EF22 . . .
singular
such that A =
U (0 :
on) V
+
roof :seelecturenotea matrix Coven
singular
ones) !
, to
The SVD be
generalized Matrices :
Remark
-
:
can non-square
SINGULAR VALUE DECOMPOSITION THEOREM -
general
(MXK
men
Let A set k =
min (M ,
n) ,
then there exists linear isometries UE
and Venk and real values On
,
non-negative On . . .
with z -. On 20 ,
such that A =
U ( "... on) vt
#te Isometries are
generalizations of
unitary matrices
: :
Uekman with Man is called an
isometry if UtU = In
The columns of U =
(U
:
...: un) a re orthonormal
(CUjueL =
Gje) ,
but
if m -n
they cannot form a basis of I" .
of A
The rank is the dimension of the rector
space generated by
the column reators of A = number of values)
n o n -ze ro
singular
Remark SVD) becomes low-rank factorization if values
singular
:
a are ze ro
- many
land can hence be omitted) :
↑
A u +
.... V
↑
=
↑
-onl
+w +
diag( .
MATRIX FUNCTIONS
Let f : -D function and At pren be
arbitrary be an normal
,
with
spectral decomposition A
=
U( n) Ut.
Then we define f(A) =
U(f(v) fan) :
Ut
Equivalently ,
if f admits a
power
series
expansion f(x) = 9X" * xeK,
then f(A) =
[09 : Al
for At C
Examples :
normal ....
·
et =
exp(A) = Al (note : et * =
AetA for teR)
·
# >
-
knen satisfies A =
A
DP
*
9
KRONECKER PRODUCT of two matrices At CMA ,
Be :
anT
. . .
CimB
a i (mPx nq
AB c
= . . .
am B
-
similarly for two rectors rel , we D :
vow
= ena
