Where do errors originate from?

1. Significant figures: Measurement errors, Imperfect input data

Where do errors originate from?

1. Significant figures: Measurement errors, Imperfect input data
2. Machine errors or round-off Errors: floating point representation
3. Truncation errors from Numerical approximations

Significant figures

• We only know some number up to certain accuracy
• Scientific notation indicates accuracy

Relative error

• We only know some number up to certain accuracy
• Compare the error to the actual value
• Relative errors are important!
• How much accuracy do you need?

Approximating errors

\[ \textrm{relative error }\varepsilon_t = \frac{\textrm{true error}}{\textrm{true value}} \]

• Often we don’t know the true value
• So … estimate the true value: approximate value \(\longrightarrow\) estimation of the error

Floating point numbers (single & double)

\(S\) = sign-bit (1 bit), \(E\) = exponent (8 bits), \(M\) = mantissa (23 bits)

To calculate real number \(x\): \[ x = (−1)^S \times M \times 2^{(E-127)} \]

Floating point numbers (single & double)

To calculate real number \(x\) (single precision): \[ x = (−1)^S \times M \times 2^{(E-127)} \]

• Real numbers are finite in computers
• Finite number of significant figures (binary)
• Maximum number?
• Minimum nonzero number?

0.3/0.1 - 3

ans =

4.4409e-16

Floating point numbers (single & double)

To calculate real number \(x\) (single precision): \[ x = (−1)^S \times M \times 2^{(E-127)} \]

realmin("double")

ans =

2.2251e-308

Cumulation of Round-off errors

\[ \begin{aligned} A & = \pmatrix{ a_{11} & a_{12} & a_{13}\\ a_{21} & a_{22} & a_{23}\\ a_{31} & a_{32} & a_{33}\\ } = \pmatrix{ 1/10 & -1/5 & 3/10\\ 0 & 2 & 4\\ -1 & 1 & 0\\ }\\ & \\ \det(A) & = a_{11} \, \begin{vmatrix} a_{22} & a_{23}\\ a_{32} & a_{33}\\ \end{vmatrix} - a_{12} \, \begin{vmatrix} a_{21} & a_{23}\\ a_{31} & a_{33}\\ \end{vmatrix} + a_{13} \, \begin{vmatrix} a_{21} & a_{22}\\ a_{31} & a_{32}\\ \end{vmatrix} \\ & = 1\\ & \\ \Rightarrow \det(A^n) & = \det(A\cdot A\cdot A\cdot A \dots) = 1 \end{aligned} \]

Cumulation of machine errors

Numerical result for increasing power \(n\):

     det(A^1) = 1.0
     det(A^2) = 1.0000000000000009
     det(A^3) = 0.9999999999999853
     det(A^4) = 0.9999999999999742
     det(A^5) = 0.9999999999996483
     det(A^6) = 0.9999999999888063
     det(A^7) = 0.9999999997708064
     det(A^8) = 1.0000000024722748
     det(A^9) = 1.0000000062737353
     det(A^10) = 0.9999999184512143
     det(A^11) = 0.9999991943727314
     det(A^12) = 0.9999979345136666
     det(A^13) = 1.0000381758582473
     det(A^14) = 1.0002862190114559
     det(A^15) = 1.0003816176433191
     det(A^16) = 1.025266132959692
     det(A^17) = 0.9320746727642004
     det(A^18) = 1.1651039421406175
     det(A^19) = -1.9418398011213938
     det(A^20) = 0.0
     det(A^21) = 194.18682395190248
     det(A^22) = 12629.065200053725
     det(A^23) = 0.0
     det(A^24) = 48526.701758954136

Errors due to numerical approximations

• Depend on numerical method and its implementation
• These errors are independent of round-off errors
• There are also physical model approximations: these are sometimes intertwined with the numerical method. Some Examples:
  • “Raycasting” (e.g. Zemax) uses geometric optics, very small optical systems cannot be modeled.
  • Finite difference methods can work in the time domain (FDTD) or frequency domain. Their errors depend on the problem AND the numerical approximations within the method.

The trajectory of a ball

• Gravitation: \(g=9.81\) m/s\(^2\)
• Air resistance: ignore for a slow heavy ball
• Horizontal velocity is constant

Trajectory of a ball: Numerically

   t = 0.10: (0.38,1.57)
   t = 0.20: (0.76,1.84)
   t = 0.30: (1.13,2.01)
   t = 0.40: (1.51,2.08)
   t = 0.50: (1.89,2.06)
   t = 0.60: (2.27,1.94)
   t = 0.70: (2.64,1.72)
   t = 0.80: (3.02,1.40)
   t = 0.90: (3.40,0.98)
   t = 1.00: (3.78,0.47)
   t = 1.10: (4.15,-0.15)

Trajectory of a ball: Errors

Mismatch with exact curve due to approximation

Propagating errors

Performing mathematical operations changes the error

\[ \begin{aligned} (1.31 \,\pm\, 0.04) \,\,+\,\, (2.5 \,\pm\, 0.1) & \quad=\quad 3.81 \,\, \pm \,\, ?\\ \end{aligned} \]

Propagating errors: Taylor expansions

Suppose a function \(f(x)\):

• true value is \(x\)
• true error is \(\Delta x\)

Propagating errors: Taylor expansions

\[ f(x + \Delta x ) = f(x) + f'(x) \, \Delta x + R_3 \]

Propagating errors: Taylor expansions

Error propagation formula:

\[ \Delta f = \frac{\partial f}{\partial x} \, \Delta x + \frac{\partial f}{\partial y} \, \Delta y + \frac{\partial f}{\partial z} \, \Delta z + \dots \]

Propagating uncertainty vs. true errors

• Often we don’t know the true error
• We can use uncertainty of our value: estimated error \(\sigma\)

Propagating uncertainty

Propagation formula:

\[ \sigma_f = \sqrt{\left(\sigma_x \,\frac{\partial f}{\partial x}\right)^2 \, + \, \left(\sigma_y \,\frac{\partial f}{\partial y}\right)^2 \, + \, \left(\sigma_z \,\frac{\partial f}{\partial z}\right)^2 + \dots} \]

Linear equations

A system of linear equations with

• Unknown variables \(x_i\)
• Constant coefficients \(a_{ij}\)
• Constants \(b_i\)

\[ \left\{ \begin{aligned} a_{11}x_1 + a_{12}\,x_1 + \dots + a_{1n}\,x_n & = b_1\\ a_{21}x_1 + a_{22}\,x_2 + \dots + a_{2n}\,x_n & = b_2\\ \vdots\qquad & \\ a_{m1}x_1 + a_{m2}x_2 + \dots + a_{mn}x_n & = b_m\\ \end{aligned} \right. \]

Solving small systems

A system of linear equations with

• Unknown variables \(x_i\)
• Constant coefficients \(a_{ij}\)
• Constants \(b_i\)

\[ \left\{ \begin{aligned} a_{11}\, x_1 + a_{12}\,x_2 & = b_1\\ a_{21}x_1 + a_{22}\,x_2 & = b_2\\ \end{aligned} \right. \]

Matrix equations

• Systems of linear equations in matrix-form

\[ \begin{pmatrix} a_{11} & a_{12} & \dots & a_{1n}\\ a_{21} & a_{22} & \dots & a_{2n}\\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \dots & a_{mn}\\ \end{pmatrix} \begin{pmatrix} x_1\\ x_2\\ \vdots\\ x_m\\ \end{pmatrix} = \begin{pmatrix} b_1\\ b_2\\ \vdots\\ b_m\\ \end{pmatrix} \]

Discretizing functions

• simplest: finite number of \(x_i\) and \(y_i = f(x_i)\) (with \(i = 1, \dots , N\))
• Points \(x_i\) can be regularly spaced
• Accuracy depends on:
  • how smooth the real function is
  • how close the points are taken But more points \(\longrightarrow\) more data, slower

Discretizing functions

Discretizing functions

Discretizing functions