Revision of Concepts

2.1. Revision of Concepts#

Classification of Differential Equations#

Note

Important things to retain from this block:

  • Identify characteristics of differential equations

  • Understand how numerical and analytical solutions might differ as we get to more complex problems which need simplification

  • Remember that analytical solutions are not always possible

Differential equations can be classified as Ordinary Differential Equations (ODEs) and Partial Differential Equations (PDEs). ODEs have derivatives with respect to a single independent variable (either time or space), for example:

\[ \frac{d x(t)}{d t} = \cos t \]

describes the rate of change of the variable \(x\) in time \(t\).

PDEs have derivatives with respect to multiple independent variables (often time and space), for example:

\[ \frac{\partial c(x,t)}{\partial t} + u\frac{\partial c(x,t)}{\partial x} = 0 \]

describes the propagation of the concentration \(c\) in time \(t\) along dimension \(x\).

The classification can be done more precisely by the equation’s order and linearity (linear or non-linear). The order refers to the highest derivative while non-linear equations are those in which the dependent variable or its derivative(s) are non-linear (e.g., exponential, sine or with a power different than 1). See the following examples:

\[ \frac{d^3y}{dx^3} - x\frac{d^2y}{dx^2} + y = 0 \qquad \text{third-order linear ODE} \]
\[ \frac{dy}{dx} = y^2 + x \qquad \text{first-order non-linear ODE} \]
\[ \frac{d^2y}{dx^2} + y\left(\frac{dy}{dx}\right)^2 = \sin(y) \qquad \text{second-order non-linear ODE} \]

Analytical vs Numerical Solutions#

Equations can be solved in two ways: analytically and numerically. The analytical solution is exact while a numerical solution requires computational methods to approximate the solution. So why would we use anything other than analytical solutions? Well, analytical solutions are difficult or impossible to find for complex equations, especially when the problem involves a complex geometry. This complexity will be treated later in the book. For now, let’s illustrate analytical and numerical solutions considering a simple problem.

Problem

Find the value \(x\) where \(f(x) = 0\), for \(f(x) = 3x-2\) and \(x\) in the interval \([0,1]\).

Analytical Solution#

\[f(x) = 0 \Leftrightarrow 3x-2=0 \Leftrightarrow x = \frac{2}{3} = 0.66666\ldots\]

Note that there is no need for any extra computation. You can say that there is only one computation needed: the assignment of the \(x\) value.

Numerical Solution#

The code below shows an iterative method to find the x value. We step through the interval \([0, 1]\) in small increments \(\Delta x\) and evaluate the function \(f(x)\). Then we check if two consecutive values \(f(x_i)\) and \(f(x_{i+1})\) have the same sign. If they have different signs, it means that we crossed the point where \(f(x)=0\), so we can stop.

def f(x):
    return 3 * x - 2


# Step through the interval [0, 1]
dx = 0.001
for i in range(1000):
    x = dx * i
    # Check if the sign changed
    sign_changed = f(x) * f(x + dx) < 0
    if sign_changed:
        print(f"Number of computations needed to find a solution: {i * 4}")
        break

print(f"Answer: x = {x + dx}")

Number of computations needed to find a solution: 2664
Answer: x = 0.667

Note that the number of computations needed is highly dependent on the method (this one is not very efficient). Here the search starts at x=0 and increases in steps of 0.001, which also limits the accuracy of the solution.

Exercise

Let us now look to another simple example:

Find the value \(x\) where \(f(x) = 0\) for \(f(x) = 3\sin(x)-2\), for \(x\) in the interval \([0,1]\).

Try an analytical and a numerical approach.

Solution

Analytical Solution

\[ f(x) = 0 \Leftrightarrow 3\sin(x)-2=0 \Leftrightarrow x = \arcsin\left(\frac{2}{3}\right) = 0.7297276562269663..\]

Numerical Solution

import math


def f(x):
    return 3 * math.sin(x) - 2


dx = 0.001
for i in range(1000):
    x = dx * i
    if f(x) * f(x + dx) < 0:
        print(f"Number of computations needed to find a solution: {i * 4}")
        break

print(f"Answer: x = {x + dx}")

Number of computations needed to find a solution: 2916
Answer: x = 0.73

Note that there is no attempt to solve \(f(x)=0\) directly. Only the function is defined and the exact same steps of the previous problem are followed.

Extra: More efficient numerical solution approach

A better (more efficient) solution would be to use Newton’s method, which would look like:

def df(x):
    """Numerically evaluate the derivate of f(x)"""
    dx = 1e-5
    return (f(x + dx) - f(x)) / dx


x = 0.5  # start halfway the search interval
i = 0
error = 1
while error > 1e-10:
    x_new = x - f(x) / df(x)
    error = abs(x_new - x)
    x = x_new
    i += 1
print(f"Answer x = {x}")
print(f"Number of iterations: {i}")

Answer x = 0.7297276562269663
Number of iterations: 5

The result is a significant reduction of the number of computations needed to get a solution that is also more accurate than the first algorithm.

This example uses two concepts that we will address in more detail later:

  1. The algorithm requires the derivative of the function, which is approximated by considering a small interval dx around point x. The following sections will show where this approximation comes from and discuss how accurate it is.

  2. We will also have a closer look at Newton’s method in Section 2.7.

Attribution

This chapter is written by Jaime Arriaga Garcia, Anna Störiko, Justin Pittman and Robert Lanzafame. Find out more here.

Contents