# Implementing the derivative in C/C++

### Question

How is the derivative of a `f(x)` typically calculated programmatically to ensure maximum accuracy?

I am implementing the Newton-Raphson method, and it requires taking of the derivative of a function.

1
21
10/17/2017 7:08:05 PM

I agree with @erikkallen that `(f(x + h) - f(x - h)) / 2 * h` is the usual approach for numerically approximating derivatives. However, getting the right step size h is a little subtle.

The approximation error in (`f(x + h) - f(x - h)) / 2 * h` decreases as `h` gets smaller, which says you should take `h` as small as possible. But as `h` gets smaller, the error from floating point subtraction increases since the numerator requires subtracting nearly equal numbers. If `h` is too small, you can loose a lot of precision in the subtraction. So in practice you have to pick a not-too-small value of `h` that minimizes the combination of approximation error and numerical error.

As a rule of thumb, you can try `h = SQRT(DBL_EPSILON)` where `DBL_EPSILON` is the smallest double precision number `e` such that `1 + e != 1` in machine precision. `DBL_EPSILON` is about `10^-15` so you could use `h = 10^-7` or `10^-8`.

For more details, see these notes on picking the step size for differential equations.

59
6/20/2018 7:40:01 PM

Newton_Raphson assumes that you can have two functions f(x) and its derivative f'(x). If you do not have the derivative available as a function and have to estimate the derivative from the original function then you should use another root finding algorithm.

Wikipedia root finding gives several suggestions as would any numerical analysis text.