Linear transformations

All matrix multiplications are linear transformations. For the general case, a transformation could be seen as a function, or a black box, that for any given input has an output. The definition for the transform to be linear is that the operation is consistent for two input elements, in our case, two vectors. Let L be a transformation, x and y be vectors, and c and k be scalars. Then L is a linear transformation if, and only if, $$L(cx + ky) = cL(x) + kL(y)$$

What are linear transformations used for in practise?

Just like human drivers, self-driving cars must constantly scan the roads for obstacles and road signs to safely operate our streets.

To be able to do so, the car is equipped with cameras taking snap shots of its surroundings with very short intervals.

But how does the car know whether the Volvo in front of it is casually cruising down the road, or has come to a sudden stop as a result of an accident?

The answer is linear transformations. A picture of a car far away has a very different pixel representation compared to a close-up picture of the same car. However, there is a linear relationship between the pictures, since the car itself does not change its appearance.

Through linear transformations that zoom and rotate the sequence of pictures, the self-driving algorithm can determine the behavior of the car in front and act accordingly.

What does linear transformations mean?

We have previously seen that matrices provide a useful way of storing coefficients of the variables involved in a linear system of equations. This chapter will introduce a new but related way of interpreting matrices and the familiar equation:

Algebraically speaking, the matrix multiplies a vector to produce a new vector . We can think of as acting on vectors that map them to other vectors.

Matrices transform vectors into new vectors

Viewed this way, matrices are functions with vectors as their input and outputs. This is the key concept of linear transformations.

What are the requirements for linear transformations?

For the following vector function to qualify as a linear transformation:

then the following criteria must apply:

This is the case for the multiplication of any vector by a matrix in the equation:

As with any other mathematical function, has a specific domain and range. Its domain is all vectors of length , and the range is the set of -dimensional vectors that are possible outputs.

The domain and range of a linear transformation are vector spaces, and constitute a mapping between them. We will cover this in more detail in later chapters.

What is a linear transformation?

Welcome to the most fun part of the linear algebra course! This section can be somewhat demanding for the beginner, but with the right introduction and the right tempo, this section can also be made really enjoyable. The course in linear algebra is almost exclusively about the following equation:

So far, we have considered the equation as a linear system of equations and determined the solution sets for the three cases: a unique solution, infinite solutions and no solution. Now we will look at the matrix from another perspective, namely as a linear transformation. We will focus on the properties of the matrix from the perspective of seeing as a function. Let us again take the same equation, but instead name the vectors and .

Now we have no unknowns and can instead see as a function that transforms the vector to the vector . This section is about what can look like, how the transformation takes place and what geometric interpretations can be made. We start by going through what generally applies to the definition of functions, that is, something that also applies outside this course.

Function

This section is a bit difficult to digest for some, so it is okay if it doesn't come naturally at first. But it is important for the beginner to practice abstract thinking, so we start with an explanation of the analogy between linear transformations and functions that one has learned in courses in calculus.

Look at the picture! We have two quantities, one on the left and one on the right. Let be a function that relates elements from the left set to the right set. Then the left set is called the domain of , while the right set is called the codomain of . Let's see the domain as an input to . We then have a set of elements on the right that we can see as an output of . If we note the input to as , then the output of is called , which is the value of or the transformation of . Furthermore, we say that maps to . It is common to note outputs with a single variable , as in . However, does not have to be the entire target set, and it is common for to cover a subset of the target set, called range or map.

Three classes of functions (and linear transformations for that matter) are injection, surjection and bijection. We summarize with:

Let:

be a linear transformation. Then the definitions of the following classes apply:

• Injective (one-to-one): have for each at most one solution .

• Surjective (onto): have for each at least one solution

• Bijective (one-to-one and onto): have for each exactly one solution

A nice observation that can be made by the beginner is that we can define a bijective mapping as both injective and surjective.

Example 1

Let be . Then we have the following:

• The function is

• The domain is the real number plane

• The target set is the real number plane

• The range is the real number plane because all possible outputs of cover the whole

Example 2

Let be . Then we have the following:

• The function is

• The domain is , that is, but not because is not defined for

• The codomain is

Example 3

Let be . Then we have the following:

• The function is

• The domain is because both and are inputs

• The codomain is because the output is the sum of the squares of and

• The range is because squaring gives only positive numbers as outputs.

Mapping

Let's again revisit the equation:

and do not focus too much on the fact that and are usually noted as unknown vectors, since it is common to use the same notation in these contexts. Now remember that the matrix maps to the vector . But what then is a mapping? A mapping is a function whose input and output are vectors and can also be called a transformation. A mapping is usually noted with a capital letter such as , or . If is a mapping that transforms the vector from the subspace (noted as and read as " in ") onto from the subspace , this relation is noted or sometimes as:

which reads as " maps onto ".
In a linear algebra course, the spaces for the domain and codomain are usually defined, so let be and be , such that:

which is usually a more familiar notation in this course.

Linear transformation

A linearity or linear system outside of mathematics is usually defined as something that has a proportional relationship between input and output. A system meets the condition of linearity, if, when you modify its input, it returns a correspondingly modified output. For a real-world example, we can look to personal finance. Imagine a savings account without interest. Let's assume that we have a monthly savings of 10 USD. This means that we have saved 120 USD in one year and 1,200 USD in ten years. If we were to save double, i.e increase our savings rate by a factor of 2, we will have saved 2,400 USD in ten years. Should we instead save in two savings accounts separately with 10 USD per month each, we will then also receive the same total amount. These are examples of linear systems, and as well, an introduction to the definition of a linear transformation.

A function is called a linear transformation if it takes a vector from to and satisfies the following two properties for all vectors and in and for all scalars :

• Homogeneity

For the special case , the linear transformation is called a linear operator of .

This definition leads to the following properties.
If:

is a linear mapping, then it applies that:

What is a standard matrix?

Before we go into what a standard matrix is, we must first determine that all linear transformations, or mappings, from to are matrix transformations. First we show that a matrix transformation is a linear transformation, then we show that a linear transformation must be a matrix transformation.

A matrix transformation is a transformation that can be written as:

In addition, the following applies to all linear transformations:

is a matrix transformation.

We begin by showing that a matrix transformation is a linear transformation by testing its homogeneity and additivity. Let be an -matrix, and be -vectors and a scalar.

which shows that the matrix transformation is a linear transformation by definition. Now we assume that we have a linear transformation:

and write the vector as a linear combination of the standard vectors :

We now use to develop the following:

which shows that for each linear transformation , we can create a matrix transformation whose columns are mappings of the standard vectors .

The -matrix noted above as is called the standard matrix. It is often noted in the literature as or for the respective linear transformations and . It can also be easily noted as . The two previously mentioned notes are written:

and can appear in a sentences as the standard matrix for , is the transformation of or is the transformation represented by .
The insight above is so interesting, that we summarize it again here. The proof is already clear from the previous statement.

Let be a linear transformation. If are the standard vectors in , and is any vector in , then can be expressed as:

where:

Example 1

Let be a transformation that maps to with the help of the matrix :

We then have:

Then we can express the transformation as:

with mapping to :

Summarizing the previous section, we have:

• The transformation is

• The domain is because the input is:

• The codomain is since the output is:

• The range is the plane in with the directional vectors:

The fact that the range is a plane usually needs a further explanation, but looks like the following:

Note the last line! Surely it can be compared to the plane's parametric form? Namely:

This is usually a point of further confusion for the beginner, but we introduce this aspect early, so the knowledge has time to mature to insight before the exam. All content in linear algebra is interconnected, which makes the course conceptually difficult. When you see the context for what it is without being confused, then you know that the insight is in place!

Example 2

Let be a transformation via the matrix:

We then have:

• The transformation is

• The domain is

• The codomain is

• The range is all because all points are possible based on the choice of

We say that collapses onto . In this particular case, any and in the point in collapse onto the point in .

Three types of linear transformations everyone must handle, understand and be able to derive are rotation, projection, and reflection.

What is a rotation?

By rotation as a linear transformation, we mean a matrix , called a rotation matrix, which maps each vector to a rotation around the origin at a given angle . The beginner is expected to learn this for both and , and as usual it is recommended to first try to understand instead of first learning formulas by heart.

Rotations in 2 dimensions

Let be the result vector of an arbitrary vector that has been rotated counterclockwise around the origin by the angle . Then we have that the standard matrix returns as follows:

According to the general formula of the standard matrix for a linear transformation, we can express for:

as:

Thus, the transformed vector can be expressed as:

The last row is a linear combination of two vectors with the scalars and . Graphically, this can be deduced with the help of the unit circle. The beginner is expected to know both it and the values of cosine, sine and tangent for the standard angles:

Rotations in 3 dimensions

Rotation around the origin in 3 dimensions raises the follow-up questions: around which axis? and with which orientation? We get the answer to the second question with the help of the right-hand rule that defines the direction of a selected angle (in two dimensions it becomes as simple as counterclockwise or clockwise). The vector can be rotated about the -axis, -axis or -axis. If it is rotated around several, it is easiest to produce the rotation matrix for each of the axes and then multiply them together (called composite transformation). The rotation matrix in is derived using the rotation matrix in that we derived in the previous section. Let's start with an example where the vector is rotated about the -axis. This means that we lock the -coordinates for and apply the rotation matrix only for the - and -coordinates. We start from scratch and have:

where the marked elements can be recognized as the -rotation matrix in a part of the -rotation matrix that we note . Note that in the last row of the result vector, the -coordinate is locked with and does not affect the - and -coordinates. There we find instead the recognized expression for the mapping of the -rotation matrix. Analogously, we can arrive at the corresponding -rotation matrices and with rotation around the - and - axes, respectively. We define all three:

Note, however, how the minus sign changes places for . This is associated with being consistent with the orientation of the three axes and is linked to the right-hand rule. For now, the recommendation is to buy into it, as complete understanding may require quite a lot of time that can be spent on other parts of the material in the course.

What is a projection?

The projection matrix takes a vector and orthogonally projects it on a line in , or on a line or plane in , using the projection theorem. We go through both standard matrices here.

Projection matrices in 2 dimensions

Let be the projection of the vector onto the vector . Then it is true that the standard matrix is:

To derive the standard matrix, we remember the projection theorem for projected onto as:

but where we instead let the vectors:

Then we can derive the standard matrix as:

where the last line shows the formula for the standard matrix .

Projection matrices in 3 dimensions

Projection onto vectors

Let's take the linear transformation:

which projects the vector onto the vector:

From the derivation of the -projection matrix, we have that the standard matrix is:

Thus, the standard matrix for the linear transformation of the projection onto a vector in is:

Projection onto an arbitrary plane

Let:

be the linear transformation that projects a vector onto the plane with the normal vector . Then the standard matrix is :

where is the identity matrix in , and the right-most matrix we recognize as the standard matrix for vector projection in . Unfortunately, the standard matrix cannot be noted more beautifully, but the easiest way is to insert numbers from the given normal vector first and then do the subtraction. The derivation is quite simple, so much so that we learned this already at the beginning of the course, namely the summation of two vectors. See the picture! Where we define the projected vector onto the plane and as the sum:

where is the projection of on the plane's normal vector .

where is the standard matrix for projection onto the vector . Thus, we have derived the definition of the standard matrix above.

Projection onto the coordinate plane

Once we have understood the projection onto an arbitrary plane, we can reuse the formula for projection onto the coordinate plane. We have three coordinate planes in , namely, the -plane, the -plane and the -plane. We take the example of the -plane. Let:

be a linear transformation, with standard matrix projecting any vector onto the -plane. This means that:

While the coordinates and exist, the coordinate becomes 0, since the vector is projected onto the -plane. We can both algebraically and geometrically derive the standard matrix to:

and we can also confirm using the formula for the standard matrix for a projection onto an arbitrary plane. The same applies to the other two standard matrices for projection onto the coordinate plane. We summarize all three:

What is reflection?

Reflection of a vector always refers to the transformation of a line or plane whose result vector is a mirror image on the other side.

We have the linear transformation:

which reflects the vector with respect to the line with the direction vector . Its standard matrix is:

where is the -projection matrix that projects an arbitrary vector onto the line of the direction vector . We name the result vector of the projection . (Remember that the projection formula requires two vectors, so even if we project onto a line, we need a direction vector). According to the transformation, we can write the result vector , which is the reflection vector of around the line as the sum:

which connects to the definition of the standard matrix above. The reflection matrix for a line in and for a plane in can be derived analogously.

What is a linear operator?

This is a very technical section, and therefore we have placed extra emphasis on keeping it short. It is a concise section, but it may appear on the vocabulary section that each exam in linear algebra offers.

A linear operator can refer to different definitions. In the basic course in linear algebra, it is fairly conventional that a linear operator is referred to as a linear transformation with the special case that the dimensions between the domain and the codomain are the same. Namely that:

Quite simply, we have dimensions both to the left and right of the arrow. If this were not the case, we would not have a linear operator, only a linear transformation (provided that the definition of linear transformation is met). Examples of linear operators are projection, reflection and rotation. The latter two are also called linear isomorphism, which means that the image is invertible, which also means that it is a bijection.
If a linear operator has a length-preserving property, that is:

the operator is called an orthogonal operator. The following theorem applies to these:

If is a linear operator, the following two statements are always true:

An orthogonal operator gives rise to the definition of an orthogonal matrix.

A square matrix is said to be orthogonal if .

Which in turn leads to the following theorem:

• The transpose of an orthogonal matrix is orthogonal.

• The inverse of an orthogonal matrix is orthogonal.

• The product of an orthogonal matrix is orthogonal.

• If is orthogonal, then or .

And finally, we connect all the theorems and definitions with the following:

A linear operator:

is orthogonal if, and only if, its standard matrix is orthogonal.