# 2.3 Gradient and Gradient-Hessian Approximations

Polynomials are frequently used to locally approximate functions. There are various ways this may be done. We consider here several forms of differential approximation.

###### 2.3.1 Univariate Approximations

Consider a function *f* : → that is differentiable in an open interval about some point *x*^{[0]} . The linear polynomial

[2.16]

provides a good approximation for *f*, at least in a small interval about *x*^{[0]}. This is because:

*p*_{1}equals*f*at*x*^{[0]}, and*p*_{1}has the same first derivative as*f*at*x*^{[0]}.

If *f* is twice differentiable in an open interval about *x*^{[0]}, we can improve the approximation with a quadratic polynomial

[2.17]

Consider the function

[2.18]

which has first and second derivatives

[2.19]

[2.20]

on . Let’s construct a linear polynomial approximation for *f* about the point *x*^{[0]} = 0. Applying [2.16], we obtain

[2.21]

[2.22]

This is graphed in Exhibit 2.2.

*f*and linear polynomial

*p*

_{1}.

We can improve the approximation, at least for values of *x* close to 0, with a quadratic polynomial. Applying [2.17] at *x*^{[0]} = 0, we obtain

[2.23]

[2.24]

This is graphed in Exhibit 2.3.

*f*and linear polynomial

*p*

_{2}.

###### 2.3.2 Multivariate Approximations

Polynomial approximations [2.16] and [2.17] generalize to multiple dimensions. For *f* : ^{n} → , gradients replace first derivatives and Hessians replace second derivatives, so linear polynomial [2.16] and quadratic polynomial [2.17] become

[2.25]

[2.26]

We call these gradient approximations and gradient-Hessian approximations, respectively.

Consider the function

[2.27]

which has gradient and Hessian

[2.28]

[2.29]

Let’s construct a gradient-Hessian approximation about the point (0, 1). Applying [2.26], we obtain

[2.30]

[2.31]

###### 2.3.3 Taylor Polynomials

The linear and quadratic polynomial approximations discussed in this section are examples of a more general concept called Taylor polynomials. Consider a function *f* : → whose first *m* derivatives exist in an open interval about a point *x*^{[0]} . The polynomial

[2.32]

is called the *m ^{th}*-order

**Taylor polynomial**of

*f*about the point

*x*

^{[0]}. It provides a good approximation for

*f*, at least in a small interval about

*x*

^{[0]}. If all derivatives exist for

*f*in an open interval about

*x*

^{[0]}, we may consider the limiting polynomial as

*m*approaches infinity. This is called the

**Taylor series expansion**of

*f*about the point

*x*

^{[0]}. In some cases—but not all!—a function equals its Taylor series expansions on . For example, functions

*e*and

^{x}*sin*(

*x*) both equal their Taylor series expansions about the point

*x*

^{[0]}= 0:

[2.33]

[2.34]

Taylor polynomials and Taylor series generalize to higher dimensions.

###### Exercises

Apply [2.17] to construct a quadratic polynomial approximation for the function *f*(*x*) = *xe ^{x}* about the point

*x*

^{[0]}= 0.

Solution

Apply [2.26] to construct a quadratic polynomial approximation for the function *f*(*x*_{1}, *x*_{2}) = *x*_{1}*exp*(*x*_{2}) about the point **x**^{[0]} = (0, 0).

Solution

Construct the Taylor series expansion for the function *log*(1 + *x*) about the point *x*^{[0]} = 0.

Solution