Solving Linear Equations

Michael Friendly and John Fox

2020-10-29

This vignette illustrates the ideas behind solving systems of linear equations of the form \(\mathbf{A x = b}\) where

The general conditions for solutions are:

We use c( R(A), R(cbind(A,b)) ) to show the ranks, and all.equal( R(A), R(cbind(A,b)) ) to test for consistency.

library(matlib)   # use the package

Equations in two unknowns

Each equation in two unknowns corresponds to a line in 2D space. The equations have a unique solution if all lines intersect in a point.

Two consistent equations

## 1*x1 - 1*x2  =  2 
## 2*x1 + 2*x2  =  1
## [1] 2 2
## [1] TRUE

Plot the equations:

##   x[1] - 1*x[2]  =  2 
## 2*x[1] + 2*x[2]  =  1

Solve() is a convenience function that shows the solution in a more comprehensible form:

## x1    =   5/4 
##   x2  =  -3/4

Three consistent equations

For three (or more) equations in two unknowns, \(r(\mathbf{A}) \le 2\), because \(r(\mathbf{A}) \le \min(m,n)\). The equations will be consistent if \(r(\mathbf{A}) = r(\mathbf{A | b})\). This means that whatever linear relations exist among the rows of \(\mathbf{A}\) are the same as those among the elements of \(\mathbf{b}\).

Geometrically, this means that all three lines intersect in a point.

## 1*x1 - 1*x2  =  2 
## 2*x1 + 2*x2  =  1 
## 3*x1 + 1*x2  =  3
## [1] 2 2
## [1] TRUE
## x1    =   5/4 
##   x2  =  -3/4 
##    0  =     0

Plot the equations:

##   x[1] - 1*x[2]  =  2 
## 2*x[1] + 2*x[2]  =  1 
## 3*x[1]   + x[2]  =  3

Three inconsistent equations

Three equations in two unknowns are inconsistent when \(r(\mathbf{A}) < r(\mathbf{A | b})\).

## 1*x1 - 1*x2  =  2 
## 2*x1 + 2*x2  =  1 
## 3*x1 + 1*x2  =  6
## [1] 2 3
## [1] "Mean relative difference: 0.5"

You can see this in the result of reducing \(\mathbf{A} | \mathbf{b}\) to echelon form, where the last row indicates the inconsistency.

##      [,1] [,2]  [,3]
## [1,]    1    0  2.75
## [2,]    0    1 -2.25
## [3,]    0    0 -3.00

Solve() shows this more explicitly:

## x1    =  11/4 
##   x2  =  -9/4 
##    0  =    -3

An approximate solution is sometimes available using a generalized inverse.

##      [,1]
## [1,]    2
## [2,]   -1

Plot the equations. You can see that each pair of equations has a solution, but all three do not have a common, consistent solution.

##   x[1] - 1*x[2]  =  2 
## 2*x[1] + 2*x[2]  =  1 
## 3*x[1]   + x[2]  =  6

Equations in three unknowns

Each equation in three unknowns corresponds to a plane in 3D space. The equations have a unique solution if all planes intersect in a point.

Three consistent equations

##  2*x1 + 1*x2 - 1*x3  =    8 
## -3*x1 - 1*x2 + 2*x3  =  -11 
## -2*x1 + 1*x2 + 2*x3  =   -3

Are the equations consistent?

## [1] 3 3
## [1] TRUE

Solve for \(\mathbf{x}\).

## x1 x2 x3 
##  2  3 -1
##    [,1]
## x1    2
## x2    3
## x3   -1
##      [,1]
## [1,]    2
## [2,]    3
## [3,]   -1

Another way to see the solution is to reduce \(\mathbf{A | b}\) to echelon form. The result is \(\mathbf{I | A^{-1}b}\), with the solution in the last column.

##      x1 x2 x3   
## [1,]  1  0  0  2
## [2,]  0  1  0  3
## [3,]  0  0  1 -1
## 
## Initial matrix:
##      x1  x2  x3     
## [1,]   2   1  -1   8
## [2,]  -3  -1   2 -11
## [3,]  -2   1   2  -3
## 
## row: 1 
## 
##  exchange rows 1 and 2 
##      x1  x2  x3     
## [1,]  -3  -1   2 -11
## [2,]   2   1  -1   8
## [3,]  -2   1   2  -3
## 
##  multiply row 1 by -1/3 
##      x1   x2   x3       
## [1,]    1  1/3 -2/3 11/3
## [2,]    2    1   -1    8
## [3,]   -2    1    2   -3
## 
##  multiply row 1 by 2 and subtract from row 2 
##      x1   x2   x3       
## [1,]    1  1/3 -2/3 11/3
## [2,]    0  1/3  1/3  2/3
## [3,]   -2    1    2   -3
## 
##  multiply row 1 by 2 and add to row 3 
##      x1   x2   x3       
## [1,]    1  1/3 -2/3 11/3
## [2,]    0  1/3  1/3  2/3
## [3,]    0  5/3  2/3 13/3
## 
## row: 2 
## 
##  exchange rows 2 and 3 
##      x1   x2   x3       
## [1,]    1  1/3 -2/3 11/3
## [2,]    0  5/3  2/3 13/3
## [3,]    0  1/3  1/3  2/3
## 
##  multiply row 2 by 3/5 
##      x1   x2   x3       
## [1,]    1  1/3 -2/3 11/3
## [2,]    0    1  2/5 13/5
## [3,]    0  1/3  1/3  2/3
## 
##  multiply row 2 by 1/3 and subtract from row 1 
##      x1   x2   x3       
## [1,]    1    0 -4/5 14/5
## [2,]    0    1  2/5 13/5
## [3,]    0  1/3  1/3  2/3
## 
##  multiply row 2 by 1/3 and subtract from row 3 
##      x1   x2   x3       
## [1,]    1    0 -4/5 14/5
## [2,]    0    1  2/5 13/5
## [3,]    0    0  1/5 -1/5
## 
## row: 3 
## 
##  multiply row 3 by 5 
##      x1   x2   x3       
## [1,]    1    0 -4/5 14/5
## [2,]    0    1  2/5 13/5
## [3,]    0    0    1   -1
## 
##  multiply row 3 by 4/5 and add to row 1 
##      x1   x2   x3       
## [1,]    1    0    0    2
## [2,]    0    1  2/5 13/5
## [3,]    0    0    1   -1
## 
##  multiply row 3 by 2/5 and subtract from row 2 
##      x1 x2 x3   
## [1,]  1  0  0  2
## [2,]  0  1  0  3
## [3,]  0  0  1 -1

Plot them. plotEqn3d uses rgl for 3D graphics. If you rotate the figure, you’ll see an orientation where all three planes intersect at the solution point, \(\mathbf{x} = (2, 3, -1)\)

You must enable Javascript to view this page properly.

Three inconsistent equations

## 1*x1 + 3*x2 + 1*x3  =  2 
## 1*x1 - 2*x2 - 2*x3  =  3 
## 2*x1 + 1*x2 - 1*x3  =  6

Are the equations consistent? No.

## [1] 2 3
## [1] "Mean relative difference: 0.5"