COS2633
NOTES
,1 Error and Related Topics
1.1 Objectives
The objectives of this lesson are:
• To understand the process involved in studying the various numerical methods;
• To understand the notion of error, the types of error and their effect when carrying out computations;
• To understand the notion of convergence as it relates to general numerical analysis, as well as to the
discussion of the various techniques and their algorithms.
1.2 Introduction
Many of the problems used to model different situations arising in real life cannot be solved by analytic
techniques. Numerical techniques are available as an alternative to solving many practical problems. An un-
derstanding of how the various numerical schemes work is inevitable and brings about the need for analysis
of the various techniques and aspects of their ’goodness’.
Many numerical algorithms are too tedious to be implemented by a hand calculator because of the volume
of computations involved. The increased use of computers have made them an indispensable tool for many
computational processes. An understanding of how a computer handles numbers and computations is neces-
sary in knowing what results to expect from an algorithm.
Many of the numerical schemes are used to approximate the exact solutions of certain problems. Thus any
numerical approximation is done with an error which may be sometimes large depending on how good the
particular scheme is. The superiority of a numerical scheme is at times judged based on the computational
error incurred when using it.
The process of numerical analysis involves three components:
(i) Formulation of the method;
(ii) Execution or implementation of the method;
(iii) Validation of the method - which involves analysis of convergence of the method and error thereof.
This tutorial letter supplements Tutorial Letter 102 and Chapter 1 of the textbook [1] (section 1.2) in which
a brief discussion of Error and Computer Arithmetic is given.
1.3 Error
Since in computations we work with a finite number of digits and carry out finitely many steps, the numerical
methods of computation are finite processes. A numerical result is an approximate value of the unknown
exact result, except for rare cases where the exact answer is sufficiently simple rational number and we can
use a numerical method which gives the exact result.
2
,1.3.1 Notation
The following notation shall be adopted for this lesson (or discussion) and subsequent ones. If x is an
approximate value of the quantity whose exact value is X, then the difference
e=X −x
is the (actual) error of X. Hence
X = x + e; i.e. Approximation = True value + Error.
The absolute error is defined as
e = |X − x|
The relative error
e X −x error
er = = =
X X true value
If the absolute value of the error is sufficiently small (i.e. much less than |x|), then
e
er ≈ .
X
Sometimes it is useful to introduce the quantity γ − −e = x − X and call it the correction. That is
x=X +γ True value = Approximation + Correction
Finally, an error bound for x is the number β such that
|X − x| ≤ β
That is |e| ≤ β.
1.3.2 Types and Sources of Error
We distinguish between several types of error associated with differnt sources as follows:
1. Experimental error is error in given data, usually due to the emprecision of the given data. This
probably arises from measurement. this type of error limits the accuracy of a result in subsequent
calculations.
2. Truncation error arises from replacing an exact method with an approximate one (often resulting
from cutting off some terms of an expression). Thus truncation error is due to the method used for
the computation. it is associated with the fact that a finite or infinite sequence of computational steps
necessary to produce an exact result is ”truncated” prematurely after a certain number of steps. This
error depends on the computational method and has to be discussed individually with each method.
3. Round-off error is caused by the physical limit to the number of digits which can be retained in a
number (by choice or machine/device limitation). It occurs in the process of rounding off during a
computation.
4. Machine error is due to data representation and manipulation in computer memory (section 1.2 of
the textbook focuses on this type of error).
Section 1.2 of the textbook discusses in depth the subject of Round-off error and Computer Arithmetic,
thus giving a glimpse into how error due to device limitations occur.
In what follows is an introduction to how error propagates as the various arithmetic operations are carried
out in calculations.
3
, 1.3.3 Propagation of Error
The question may be asked whether it is possible to ”measure” the error in the final answer that results from a
sequence of computations. An answer to this question is that it is not always possible to analyse the problem
by taking each step of the computation. It is not practical. However, it is possible to analyse the effect of
errors on each basic arithmetic operation (+, −, ×, ÷).
It is usually assumed that whenever unspecified values are used, the errors which occur in the numbers used
(in basic operations) are small.
NB: An approximate number given to k decimal places has 21 10−k uncertainty in the last digit. Hence the
exact number is x + e where e = ± 21 10−k and x − 21 10−k < X < x + 12 10−k .
Let x1 and x2 be approximations to the true values of X1 and X2 , respectively. Let also e1 and e2 be the
corresponding errors in these approximations. i.e. Let X1 = x1 + e1 and X2 = x2 + e2 . Then the propagation
of error when carying out the various arithmetic operations is discussed below for each one.
Error in Addition and Subtraction
We analyse the approximation to the sum X1 + X2 as follows:
X1 + X2 = (x1 + e1 ) + (x2 + e2 ) (1)
= (x1 + x2 ) + (e1 + e2 ) (2)
So that the error is
(X1 + X2 ) − (x1 + x2 ) = e1 + e2
i.e. the approximate sum is in error by the sum of the individual errors.
Example 1.1 Two lengths measured correct to the nearest 0.1 mm are 3.2 mm and 1.6 mm. What
is the best estimate we can obtain from these two measurements of the sum of the two (exact)
lengths? Discuss the error in the estimate.
Solution:
First length is 3.2 ± 0.05 mm i.e. between 3.15 mm and 3.25 mm
Second length is 1.6 ± 0.05 mm i.e. between 1.55 mm and 1.65 mm.
Hence the minimum value of the sum is
(3.15 + 1.55) mm = 4.70 mm
and the maximum value is
(3.25 + 1.65) mm = 4.90 mm
The approximation
(3.2 + 1.6) mm = 4.8 mm
has the possible error of 0.1 mm, noting that 0.1 mm is the sum of the maximum individual errors
in the original measurements since each is 0.05 mm.
Thus the estimate of the sum can be given as 4.8 ± 0.1 mm.
4
NOTES
,1 Error and Related Topics
1.1 Objectives
The objectives of this lesson are:
• To understand the process involved in studying the various numerical methods;
• To understand the notion of error, the types of error and their effect when carrying out computations;
• To understand the notion of convergence as it relates to general numerical analysis, as well as to the
discussion of the various techniques and their algorithms.
1.2 Introduction
Many of the problems used to model different situations arising in real life cannot be solved by analytic
techniques. Numerical techniques are available as an alternative to solving many practical problems. An un-
derstanding of how the various numerical schemes work is inevitable and brings about the need for analysis
of the various techniques and aspects of their ’goodness’.
Many numerical algorithms are too tedious to be implemented by a hand calculator because of the volume
of computations involved. The increased use of computers have made them an indispensable tool for many
computational processes. An understanding of how a computer handles numbers and computations is neces-
sary in knowing what results to expect from an algorithm.
Many of the numerical schemes are used to approximate the exact solutions of certain problems. Thus any
numerical approximation is done with an error which may be sometimes large depending on how good the
particular scheme is. The superiority of a numerical scheme is at times judged based on the computational
error incurred when using it.
The process of numerical analysis involves three components:
(i) Formulation of the method;
(ii) Execution or implementation of the method;
(iii) Validation of the method - which involves analysis of convergence of the method and error thereof.
This tutorial letter supplements Tutorial Letter 102 and Chapter 1 of the textbook [1] (section 1.2) in which
a brief discussion of Error and Computer Arithmetic is given.
1.3 Error
Since in computations we work with a finite number of digits and carry out finitely many steps, the numerical
methods of computation are finite processes. A numerical result is an approximate value of the unknown
exact result, except for rare cases where the exact answer is sufficiently simple rational number and we can
use a numerical method which gives the exact result.
2
,1.3.1 Notation
The following notation shall be adopted for this lesson (or discussion) and subsequent ones. If x is an
approximate value of the quantity whose exact value is X, then the difference
e=X −x
is the (actual) error of X. Hence
X = x + e; i.e. Approximation = True value + Error.
The absolute error is defined as
e = |X − x|
The relative error
e X −x error
er = = =
X X true value
If the absolute value of the error is sufficiently small (i.e. much less than |x|), then
e
er ≈ .
X
Sometimes it is useful to introduce the quantity γ − −e = x − X and call it the correction. That is
x=X +γ True value = Approximation + Correction
Finally, an error bound for x is the number β such that
|X − x| ≤ β
That is |e| ≤ β.
1.3.2 Types and Sources of Error
We distinguish between several types of error associated with differnt sources as follows:
1. Experimental error is error in given data, usually due to the emprecision of the given data. This
probably arises from measurement. this type of error limits the accuracy of a result in subsequent
calculations.
2. Truncation error arises from replacing an exact method with an approximate one (often resulting
from cutting off some terms of an expression). Thus truncation error is due to the method used for
the computation. it is associated with the fact that a finite or infinite sequence of computational steps
necessary to produce an exact result is ”truncated” prematurely after a certain number of steps. This
error depends on the computational method and has to be discussed individually with each method.
3. Round-off error is caused by the physical limit to the number of digits which can be retained in a
number (by choice or machine/device limitation). It occurs in the process of rounding off during a
computation.
4. Machine error is due to data representation and manipulation in computer memory (section 1.2 of
the textbook focuses on this type of error).
Section 1.2 of the textbook discusses in depth the subject of Round-off error and Computer Arithmetic,
thus giving a glimpse into how error due to device limitations occur.
In what follows is an introduction to how error propagates as the various arithmetic operations are carried
out in calculations.
3
, 1.3.3 Propagation of Error
The question may be asked whether it is possible to ”measure” the error in the final answer that results from a
sequence of computations. An answer to this question is that it is not always possible to analyse the problem
by taking each step of the computation. It is not practical. However, it is possible to analyse the effect of
errors on each basic arithmetic operation (+, −, ×, ÷).
It is usually assumed that whenever unspecified values are used, the errors which occur in the numbers used
(in basic operations) are small.
NB: An approximate number given to k decimal places has 21 10−k uncertainty in the last digit. Hence the
exact number is x + e where e = ± 21 10−k and x − 21 10−k < X < x + 12 10−k .
Let x1 and x2 be approximations to the true values of X1 and X2 , respectively. Let also e1 and e2 be the
corresponding errors in these approximations. i.e. Let X1 = x1 + e1 and X2 = x2 + e2 . Then the propagation
of error when carying out the various arithmetic operations is discussed below for each one.
Error in Addition and Subtraction
We analyse the approximation to the sum X1 + X2 as follows:
X1 + X2 = (x1 + e1 ) + (x2 + e2 ) (1)
= (x1 + x2 ) + (e1 + e2 ) (2)
So that the error is
(X1 + X2 ) − (x1 + x2 ) = e1 + e2
i.e. the approximate sum is in error by the sum of the individual errors.
Example 1.1 Two lengths measured correct to the nearest 0.1 mm are 3.2 mm and 1.6 mm. What
is the best estimate we can obtain from these two measurements of the sum of the two (exact)
lengths? Discuss the error in the estimate.
Solution:
First length is 3.2 ± 0.05 mm i.e. between 3.15 mm and 3.25 mm
Second length is 1.6 ± 0.05 mm i.e. between 1.55 mm and 1.65 mm.
Hence the minimum value of the sum is
(3.15 + 1.55) mm = 4.70 mm
and the maximum value is
(3.25 + 1.65) mm = 4.90 mm
The approximation
(3.2 + 1.6) mm = 4.8 mm
has the possible error of 0.1 mm, noting that 0.1 mm is the sum of the maximum individual errors
in the original measurements since each is 0.05 mm.
Thus the estimate of the sum can be given as 4.8 ± 0.1 mm.
4
