数值线性代数基础（一：矩阵）

基础表示

矩阵

the set of real numbers：$\mathbb{R}$

vector space of all $m \times n$ matrice： $\mathbb{R}^{m \times n}$

\[ A \in \mathbb{R}^{m \times n} \Leftrightarrow A=\left(a_{i j}\right)=\left[\begin{array}{ccc} a_{11} & \cdots & a_{1 n} \\ \vdots & & \vdots \\ a_{m 1} & \cdots & a_{m n} \end{array}\right], a_{i j} \in \mathbb{R} . \]

矩阵操作

转置 $\left(\mathbb{R}^{m \times n} \rightarrow \mathbb{R}^{n \times m}\right)$ $$ C=A^{T} \Rightarrow c_{i j}=a_{j i} $$
相加 $\left(\mathbb{R}^{m \times n} \times \mathbb{R}^{m \times n} \rightarrow \mathbb{R}^{m \times n}\right)$ $$ C=A+B \Rightarrow c_{i j}=a_{i j}+b_{i j} $$
标量乘法 $\left(\mathbb{R} \times \mathbb{R}^{m \times n} \rightarrow \mathbb{R}^{m \times n}\right)$ $$ C=\alpha A \Rightarrow c_{i j}=\alpha a_{i j} $$
矩阵乘法 $\left(\mathbb{R}^{m \times p} \times \mathbb{R}^{p \times n} \rightarrow \mathbb{R}^{m \times n}\right)$ $$ C=A B \Rightarrow c_{i j}=\sum_{k=1}^{p} a_{i k} b_{k j} $$

向量

vector space of real $n$-vectors: $\mathbb{R}^{n}$

\[ x \in \mathbb{R}^{n} \Leftrightarrow x=\left[\begin{array}{c} x_{1} \\ \vdots \\ x_{n} \end{array}\right], x_{i} \in \mathbb{R} . \]

We refer to $x_{i}$ as the $i$ th component of $x$. We identify $\mathbb{R}^{n}$ with $\mathbb{R}^{n \times 1}$ so that the members of $\mathbb{R}^{n}$ are column vectors. On the other hand, the elements of $\mathbb{R}^{1 \times n}$ are row vectors: $$ y \in \mathbb{R}^{1 \times n} \Leftrightarrow y=\left(y_{1}, \cdots, y_{n}\right) . $$ If $x \in \mathbb{R}^{n}$, then $y=x^{T}$ is a row vector.

默认情况下，Vector是列向量，$y=x^{T}$是一个行向量。

向量操作

Assume $a \in \mathbb{R}$ and $x, y \in \mathbb{R}^{n}$.

标量乘法 scalar-vector multiplication

\[ z=a x \Rightarrow z_{i}=a x_{i} \]

向量加法 vector addition

\[ z=x+y \Rightarrow z_{i}=x_{i}+y_{i} \]

点乘/内积 dot product (or, inner product)

\[ c=x^{T} y(=x \cdot y) \Rightarrow c=\sum_{i=1}^{n} x_{i} y_{i} \]

向量乘法/哈达玛积 vector multiply (or, Hadamard product)

\[ z=x . * y \Rightarrow z_{i}=x_{i} y_{i} \]

saxpy (“scalar a $x$ plus $y$ “): a LAPACK routine

\[ z=a x+y \Rightarrow z_{i}=a x_{i}+y_{i} \]

矩阵向量乘法

gaxpy (“generalized saxpy”): a LAPACK routine

\[ z=A x+y \Rightarrow z_{i}=\sum_{j=1}^{n} a_{i j} x_{j}+y_{i} \]

冒号标记

A handy way to specify a column or row of a matrix is with the “colon” notation. Let $A \in \mathbb{R}^{m \times n}$. Then, $$ A(i,:)=\left[a_{i 1}, \cdots, a_{i n}\right], \quad A(:, j)=\left[\begin{array}{c} a_{1 j} \\ \vdots \\ a_{m j} \end{array}\right] $$

浮点运算数

A flop (floating point operation) is any mathematical operation (such as $+,-$, *, /) or assignment that involves floating-point numbers Thus, the gaxpy (1.3) or (1.4) requires $2 m n$ flops. The following will be frequently utilized in counting flops: $$ \begin{aligned} &\sum_{k=1}^{n} k=\frac{n(n+1)}{2}, \quad \sum_{k=1}^{n} k^{2}=\frac{n(n+1)(2 n+1)}{6}, \\ &\sum_{k=1}^{n} k^{3}=\left(\sum_{k=1}^{n} k\right)^{2}=\frac{n^{2}(n+1)^{2}}{4} \end{aligned} $$

范数

向量范数

A norm (or, vector norm) on $\mathbb{R}^{n}$ is a function that assigns to each $x \in \mathbb{R}^{n}$ a nonnegative real number $\|x\|$, called the norm of $x$, such that the following three properties are satisfied: for all $x, y \in \mathbb{R}^{n}$ and $\lambda \in \mathbb{R}$,

\[ \begin{array}{ll} \|x\|>0 \text { if } x \neq 0 & \text { (positive definiteness) } \\ \|\lambda x\|=|\lambda|\|x\| & \text { (homogeneity) } \\ \|x+y\| \leq\|x\|+\|y\| & \text { (triangle inequality) } \end{array} \]

p-范数 p-norms：

\[ \|x\|_{p}=\left(\sum_{i}\left|x_{i}\right|^{p}\right)^{1 / p}, \quad 1 \leq p<\infty \]

无穷范数 infinity-norm/maximum-norm：

\[ \|x\|_{\infty}=\max _{i}\left|x_{i}\right|, \]

Two of frequently used $p$-norms are $$ \|x\|_{1}=\sum_{i}\left|x_{i}\right|, \quad\|x\|_{2}=\left(\sum_{i}\left|x_{i}\right|^{2}\right)^{1 / 2} $$

2-norm（Euclidean norm）通常使用 $\| \cdot \|$来表示。

Cauchy-Schwarz inequality

对于所有$x, y \in \mathbb{R}^{n}$,

\[ \left|\sum_{i=1}^{n} x_{i} y_{i}\right| \leq\left(\sum_{i=1}^{n} x_{i}^{2}\right)^{1 / 2}\left(\sum_{i=1}^{n} y_{i}^{2}\right)^{1 / 2} \]

也可以写成 $$ |x \cdot y| \leq\|x\|\|y\| $$

Lemma 1.8. All $p$-norms on $\mathbb{R}^{n}$ are equivalent to each other. In particular, $$ \begin{aligned} \|x\|_{2} & \leq\|x\|_{1} \leq \sqrt{n}\|x\|_{2} \\ \|x\|_{\infty} & \leq\|x\|_{2} \leq \sqrt{n}\|x\|_{\infty} \\ \|x\|_{\infty} & \leq\|x\|_{1} \leq n\|x\|_{\infty} \end{aligned} $$ Note: For all $x \in \mathbb{R}^{n}$, $$ \|x\|_{\infty} \leq\|x\|_{2} \leq\|x\|_{1} \leq \sqrt{n}\|x\|_{2} \leq n\|x\|_{\infty} $$

矩阵范数

Definition 1.9. A matrix norm on $m \times n$ matrices is a vector norm on the $m n$-dimensional space, satisfying $$ \begin{array}{ll} \|A\| \geq 0, \text { and }\|A\|=0 \Leftrightarrow A=0 & \text { (positive definiteness) } \\ \|\lambda A\|=|\lambda|\|A\| & \text { (homogeneity) } \\ \|A+B\| \leq\|A\|+\|B\| & \text { (triangle inequality) } \end{array} $$

Example $1.10$ - $\max_{i, j}\left|a_{i j}\right|$ is called the maximum norm. - $\|A\|_{F} \equiv\left(\sum_{i, j}\left|a_{i j}\right|^{2}\right)^{1 / 2}$ is called the Frobenius norm.

参考

Seongjai Kim Fundamentals of Numerical Linear Algebra