Entry
Math: Matrix: Eigenvector: Eigenvalue: Operation: Calculate: How to calculate?
Jan 21st, 2006 18:44
Knud van Eeden,
----------------------------------------------------------------------
--- Knud van Eeden --- 15 January 2006 - 04:22 am --------------------
Math: Matrix: Eigenvector: Eigenvalue: Operation: Calculate: How to
calculate?
===
Steps: Overview:
1. -Given the eigenvector equation
- - -
A . x = lambda . x
1. -This equation says thus:
1. -If I multiply a given matrix with any vector, for which of
this vectors do I get the result that only the length of
that vectors is changed (or thus scaled)?
1. -As a result of finding lambda, you find thus what this
scaling factor is (e.g. the length of that particular
vector is scaled 2 times, 3 times, 4.1 times, ...,
after you applied that given matrix)
2. -You are given thus some matrix, you put any vector in,
and calculate the result vector.
This process is similar to a function, where you put an x
value in, and get an y value out.
If the result vector is such that only the length is
changed (and not the direction, because the vector is
rotated or translated), or thus in other words the result
vector is parallel to the original vector, you have found
an eigenvector.
2. -Writing this equation in components
[ a11 a12 a13 ... a1N ]
[ ]
[ a21 a22 a23 ... a2N ] _ -
[ ] . x = lambda . x
[ a31 a32 a33 ... a3N ]
...
[ aN1 aN2 aN3 ... aNN ]
3. -Writing this out further
[ a11 a12 a13 ... a1N ] [ x1 ] [ x1 ]
[ ] [ ] [ ]
[ a21 a22 a23 ... a2N ] [ x2 ] [ x2 ]
[ ] . [ ] = lambda . [ ]
[ a31 a32 a33 ... a3N ] [ x3 ] [ x3 ]
... ... ...
[ aN1 aN2 aN3 ... aNN ] [ xN ] [ xN ]
4. -Similar to writing
4 = 1 . 4
or
10 = 1 . 10
or
x = 1 . x
you can introduce a unit vector,
and so equivalently write this as
[ a11 a12 a13 ... a1N ] [ x1 ]
[ ] [ ]
[ a21 a22 a23 ... a2N ] [ x2 ] - -
[ ] . [ ] = lambda . 1 . x
[ a31 a32 a33 ... a3N ] [ x3 ]
... ...
[ aN1 aN2 aN3 ... aNN ] [ xN ]
5. -Writing this out further
[ a11 a12 a13 ... a1N ] [ x1 ]
[ ] [ ]
[ a21 a22 a23 ... a2N ] [ x2 ]
[ ] . [ ] =
[ a31 a32 a33 ... a3N ] [ x3 ]
... ...
[ aN1 aN2 aN3 ... aNN ] [ xN ]
[ 1 0 0 ... 0 ] [ x1 ]
[ ] [ ]
[ 0 1 0 ... 0 ] [ x2 ]
lambda . [ ] . [ ]
[ 0 0 1 ... 0 ] [ x3 ]
... ...
[ 0 0 0 ... 1 ] [ xN ]
6 -Writing this out further, you will have to multiply lambda
with all the components (because you, by design, want to
have uniform scaling, this means you have to apply the
scaling to all the components)
[ a11 a12 a13 ... a1N ] [ x1 ]
[ ] [ ]
[ a21 a22 a23 ... a2N ] [ x2 ]
[ ] . [ ]
[ a31 a32 a33 ... a3N ] [ x3 ]
... ...
[ aN1 aN2 aN3 ... aNN ] [ xN ]
[ lambda 0 0 ... 0 ] [ x1 ]
[ ] [ ]
[ 0 lambda 0 ... 0 ] [ x2 ]
= [ ] . [ ]
[ 0 0 lambda ... 0 ] [ x3 ]
]
... ] ...
]
[ 0 0 0 ... lambda ] [ xN ]
7. -Put everything on one side of the equation
[ a11 -lambda a12 a13 ... a1N ]
[ ]
[ a21 a22 - lambda a23 ... a2N ]
[ ] - -
[ a31 a32 a33 - lambda ... a3N ] . x = 0
...
[ aN1 aN2 aN3 aNN - lambda ]
7. -This equation will only be generally true if the determinant is
zero
-
(as the vector x is in general not zero)
| a11 -lambda a12 a13 ... a1N |
| |
| a21 a22 - lambda a23 ... a2N |
| |
| a31 a32 a33 - lambda ... a3N | = 0
...
| aN1 aN2 aN3 aNN - lambda |
8. -Thus you have to work out the equation that the determinant is
zero.
9. -Working that equation out according to the determinant rules shows
that you will need to solve an Nth degree polynomial in the
unknown lambda.
This equation is called the 'characteristic polynomial',
or also 'secular polynomial'.
1. -There are thus always exactly N solutions for the eigenvalues
(as according to the fundamental theorem of algebra, an Nth
degree polynomial has exactly N solutions)
1. -According to Abel and Galois theory for Nth degree
polynomials there are thus exact solutions for lambda
possible for the
1 x 1 determinant
2 x 2 determinant (using the root formula)
3 x 3 determinant (using the Cardano formula)
4 x 4 determinant (using the Ferrari formula)
---
But no exact solutions for lambda in 5 x 5, or higher,
determinants.
---
You can otherwise use numeric method solutions for all
cases.
2. -The characteristic polynomial can have
1. real roots
1. -This is indicates that it is business as usual, the
scaling can take place and good eigenvectors can be
found
2. non real roots (=complex roots)
1. -This indicates usually that you will not be able to
find good eigenvectors
1. -E.g. the rotation matrix over 90 degrees will by
definition rotate over 90 degrees all vectors
which you put in.
[ 0 -1 ]
[ ]
[ 1 0 ]
So you will not be able to find any
eigenvector here (except the null vector).
So its eigenvalues will be complex
To quickly calculate this,
from the trace and determinant rule
(see video lecture Gilbert Strang)
lambda1 + lambda2 = 0 + 0 = 0
lambda1 . lambda2 = 0 . 0 - (-1 . 1) = 1
you get
lambda1 = i
lambda2 = -i
3. -The polynomial can have
1. Non-repeating (=unique) roots
1. -This is indicates that it is business as usual, the
scaling can take place and good eigenvectors can be
found
2. Repeating (=unique) roots
1. -This indicates usually that you will not be able to
find all independent eigenvectors
1. -E.g. the identity matrix
[ 1 0 ]
[ ]
[ 0 1 ]
which leaves all vectors you put in
unchanged.
To quickly calculate this,
from the trace and determinant rule
(lambda^2 + (trace) . lambda + (determinant)
= 0)
(see video lecture Gilbert Strang)
lambda1 + lambda2 = 1 + 1 = 2
lambda1 . lambda2 = 1 . 1 - (0 . 0) = 1
you get
lambda1 = 1
lambda2 = 1
10. -Once you have found the eigenvalues, you can find the eigenvectors
by for each of the eigenvalues filling in this in the equation,
and solving for the x.
Each solution gives an eigenvector.
1. -So you have to solve this equation
| a11 -lambda a12 a13 ... a1N | [x1]
| | [ ]
| a21 a22 - lambda a23 ... a2N | [x2]
| | . [ ] = 0
| a31 a32 a33 - lambda ... a3N | [x3]
... ...
| aN1 aN2 aN3 ... aNN-lambda | [xN]
11. -The problem is though that if you solve this equation
with the given lambda, it will give you the trivial solution
that all independent variables have to be zero.
Thus you will find that
x1 = 0, x2 = 0, x3 = 0, ...
---
This is because the substitution of an eigenvalue lambda
in the eigenvector equation always gives a singular
homogeneous linear system, and among its infinity of
solutions you generally have to seek a 'simple' solution
(with small integer values (if this is possible)).
[book: Edwards, Charles Henry / Penney, David E. - differential
equations and boundary value problems / Computing and modeling - p.
279 'Distinct real eigenvalues']
12. To circumvent this problem, you can follow the following
procedure to find the eigenvectors
---
[book: Cannon, Robert H. - dynamics of physical systems - p.
509 'Matrix description of eigenvector calculation']
---
[book: Slater, John C. / Frank, Nathaniel H. - mechanics - p.
272 'Tensors']
---
Steps: Overview:
1. -Given the original system of linear equations
2. -Remove (arbitrarily) one of the equations (say the first
equation)
3. -Select one of the variables (say the first variable, x1)
4. -Divide by that variable (say that that variable x1 is not zero)
5. -Solve this linear equations for this new variables
x2/x1
x3/x1
x4/x1
...
xN/x1
6. -Possibly multiply nominator and denominator of
the solution quotient with some multiple m
(because a scaled version of the eigenvector is also
a solution). This will give the eigenspace for that
specific eigenvalue.
7. -Possibly normalize the magnitudes of the eigenvectors, by
making it a unit vector, as usual by dividing by its length
(where mi are multiples)
2 2 2 2
sqrt( x1 + (m1 . x1) + (m2 . x1) + ... + (mN . x1) )
===
Steps: Worked out:
1. -Given the original system of linear equations
| a11 -lambda a12 a13 ... a1N | [x1]
| | [ ]
| a21 a22 - lambda a23 ... a2N | [x2]
| | . [ ] = 0
| a31 a32 a33 - lambda ... a3N | [x3]
... ...
| aN1 aN2 aN3 aNN-lambda | [xN]
-You can write this also as
(a11-lambda ).x1+ a12 .x2+ a13 .x3... a1N .xN=0
a21 .x1+(a22-lambda ).x2+ a23 .x3... a2N .xN=0
a31 .x1+ a32 .x2+(a33-lambda ).x3... a3N .xN=0
...
aN1 .x1+ aN2 .x2+ aN3 .x3...(aNN-lambda ).xN=0
2. Remove (arbitrarily) one of the equations (say the first
equation)
| a11 -lambda a12 a13 ... a1N | [x1]
| | [ ]
| a21 a22 - lambda a23 ... a2N | [x2]
| | . [ ] = 0
| a31 a32 a33 - lambda ... a3N | [x3]
... ...
| aN1 aN2 aN3 aNN-lambda | [xN]
-You can write this also as
a21 .x1+(a22-lambda ).x2+ a23 .x3... a2N .xN=0
a31 .x1+ a32 .x2+(a33-lambda ).x3... a3N .xN=0
...
aN1 .x1+ aN2 .x2+ aN3 .x3...(aNN-lambda ).xN=0
3. Select one of the variables (say the first variable, x1)
4. -Divide by that variable (say that that variable x1 is not zero)
(a22-lambda ).x2+ a23 .x3... a2N .xN= - a21 . x1
-- -- -- --
x1 x1 x1 x1
a32 .x2+(a33-lambda ).x3... a3N .xN= - a31 . x1
-- -- -- --
x1 x1 x1 x1
...
aN2 .x2+ aN3 .x3...(aNN-lambda ).xN= - aN1 . x1
-- -- -- --
x1 x1 x1 x1
5. -Solve this linear equations for this new variables
x2/x1
x3/x1
x4/x1
...
xN/x1
6. -Possibly multiply nominator and denominator of
the solution quotient with some multiple m
(because a scaled version of the eigenvector is also
a solution). This will give the eigenspace for that
specific eigenvalue.
10. -Possibly normalize the magnitudes of the eigenvectors, by
making it a unit vector, as usual by dividing by its length
(where mi are multiples)
2 2 2 2
sqrt( x1 + (m1 . x1) + (m2 . x1) + ... + (mN . x1) )
13. -Apply this procedure
1. -Fill in each of the found lambdas and solve for the x
1. -Fill in lambda1 and solve that equations. That gives the
1st eigenvector.
2. -Fill in lambda2 and solve that equations. That gives the
2nd eigenvector.
...
3. -Fill in lambdaN and solve that equations. That gives the
Nth eigenvector.
14. -If (some of) the eigenvalues are equal, then you form a sum of
multiples of the corresponding eigenvectors
m1 . x1 + m2 . x2 + ... + mN . xN
[book: Bronshtein, I. N. / Semendyayev, K. A. - handbook of
mathematics - page 151 'Eigenvalues and eigenvectors' - in example
of 'transformation of matrix to diagonal form']
---
---
Internet: see also:
---
Math: Transformation: Eigenvector: Eigenvalue: Link: Can you give an
overview of links?
http://www.faqts.com/knowledge_base/view.phtml/aid/39001/fid/1856
----------------------------------------------------------------------
