# Optimization

Optimization techniques tend to involve a large portion of calculus. The core idea is what we can say about a function and its derivative where it takes on its maximum value. Since it will not increase regardless of what direction we change $x$ in, we find this point where the derivative is equal to zero.

## Intro

The field of medicine have made a lot of progress in recent years when it comes to cancer treatment. Although not yet perfect, the process of curing patients from the disease have in many ways been optimized.

With modern machine learning techniques, medical teams can use various types of imaging to scan a patients tissue to detect tumors.

A crucial part of the algorithms for computer vision used to diagnose patients from images is to maximizing the program's probability of finding cancer cells, while minimizing the risks of making erroneous predictions.

After diagnosis, another type of optimization comes into play as it is time to get rid of the tumor.

For successful radiation therapy, it is important to balance the amount of radiation to be effective for killing the malignant cells, while not exceeding an overall unhealthy level.

## Concept

Functions are used everywhere. They're used within economics, physics, social sciences, etc. But functions aren't that special per se. They're just rules assigning output values to input values.

But if we know how a function looks like, and we know our basic calculus, we might be able to find the maximum and the minimum of a function. Now that's really useful. A business owner wants to maximise revenues and decrease costs. A physicist wants to maximise speed and minimise energy loss. A computer scientist wants to maximise the accuracy of a program and minimise the runtime.

Calculus allows us to find minima and maxima, and determine whether they're local, like , or global, like . Is this minima the smallest possible minima? If yes, then it's global.

## Math

We will illustrate the process of solving optimization problems by finding the maximum of the function on the interval .

We first look for critical points that satisfies:

They are:

Next, we examine which of these are local maxima, which we can do by looking at the sign table of the derivative or by studying the second derivative.

Only is a local maximum.

Lastly, since is defined on a closed interval, we have to examine the endpoints. Thus, the global maximum is either , or .

In this case, is the global maximum.

## Optimization

### Optimization problems

One of the useful applications of calculus is for finding the best possible solution to some problem, often subject to certain constraints. This is what optimization is all about.

From maximizing a company's profit, to minimizing the risk of cancer from exposure to radiation, optimization is a useful tool for improvement on various fronts.

To optimize is to reduce the cost, and an optimization problem is solved by defining and minimizing a cost function

An optimization problem tends to consist of two steps:

1. Setting up a cost function based on the goal and constrains.

2. Solving the problem by minimizing the cost function.

Often times, the most challenging part of an optimization problem is to define it correctly. It is important to consider all available information, and to use it the right way when constructing the cost function. It does not matter how good we execute step two if the function does not apply to the problem.

The cost function generally result from the causal relationship of an independent variable on a quantity of interest. Constraints resulting from some type of trade-off, where we want to make the most out of the limited resources available, can sometimes be baked into the cost function. Other times, constraints in terms of disallowed values define an interval of the independent variable.

Solving the problem is analogous to finding an extrema, namely the global minimum of the cost function on the interval. This is where the concept of calculus come into play, where we use the derivative of the cost function to find its critical points.

There are two options when it comes to localizing the global minimum: it can be at a critical point, or at one of the end points.

The cost function does not have to represent an actual cost in terms of money, but the name is accurate in that we are after its lowest value on some interval.

Even in the case of maximizing a quantity, we usually make it a cost function through multiplying it by . What was earlier the highest value now becomes the lowest, and our hunt for a minimum begins.

### Example 1

A classic type of optimization problems is to construct the region that give rise to the largest area for some predetermined shape with a limited circumference. Consider the following case:

A farmer wants to build the largest possible paddock for her horse. For simplicity, she decides to make it rectangular. Further, due to budget constraints, only 400 meters of fence can be used. What should be the dimensions of the paddock?

Solution:

First, we consider the area of a rectangle:

where is the width, and the height.

Next, since the circumference cannot exceed the total length of the fence, we can write in terms of as:

The area function then becomes:

This being a case of maximization, we invert the area function to get the cost function:

We take the derivative of the cost function, and equates it to zero to find any critical points:

and we realize that is the only critical point of the cost function.

Now we are left with contestants for the width that minimize the cost, one of them being the critical point we just found. The last two are the end points. Since the circumference is and the paddock needs to be rectangular, the width must be between and , which will be our end points.

The lowest cost is achieved at our critical point , which will maximize the area of the paddock.

That takes care of the width, now the last step is to find the height of the rectangle:

In conclusion, the optimal dimensions will be a square paddock with a side of meters.

### Example 2

Let us make the problem a bit more challenging. The task is largely the same, but this time, the farmer adds one more condition.

The horse needs access to water, and so the farmer want to make clever use of a river on the property and put the top corners of the paddock a bit out in the water. The river has the same shape as the curve of the function , where the -axis is taken as a road that cannot be included in the paddock. What should be the dimensions of the paddock to achieve the maximal area in this case?

Solution:

We want to find an expression of the area enclosed by the fence. First, note that the area in the first quadrant is given by

Since the area consists of two of these parts, the total area of the paddock is given by:

Now we can define our cost function as:

Next, we find all the critical points by taking the derivative and setting it to 0.

Since negative distances does not make any sense, we are only interested in the positive solution. Thus, the base of the paddock should be meters long, and the height becomes meters.

The last thing we need to check is that the budget, allowing for no more than 400 meters of fencing, is not exceeded:

Hence, these dimensions are allowed, and we have optimized the area of the paddock, based on the given constraints.