Vector space

A vector space $V$ over a field $F$ has two operations:

  • Vector addition $\mathbf{v} + \mathbf{w}$
  • Scalar multiplication $\alpha \mathbf{v}$

To have a vector space, the following axioms must be satisfied for every $\mathbf{v}, \mathbf{w} \in V$ and $a, b \in F$.

  • Associativity of vector addition: $u + (v + w) = (u + v) + w$
  • Commutativity of vector addition: $u + v = v + u$
  • Identity element of vector addition: There exists an element $0 \in V$, such that $v + 0 = v$ for all $v \in V$.
  • Inverse elements of vector addition: For every $v \in V$, there exists an element $−v \in V$, such that $v + (−v) = 0$.
  • Compatibility of scalar multiplication with field multiplication: $a(bv) = (ab)v$
  • Identity element of scalar multiplication: $1v = v$, where 1 denotes the multiplicative identity in $F$.
  • Distributivity of scalar multiplication with respect to vector addition:   $a(u + v) = au + av$
  • Distributivity of scalar multiplication with respect to field addition: $(a + b)v = av + bv$

Operations between two vectors

Dot product

For $\mathbf{x} = [x_1, \cdots, x_d]$ and $\mathbf{y} = [y_1, \cdots, y_d]$, the dot product of $\mathbf{x}$ and $\mathbf{y}$ is denoted by $\mathbf{x} \cdot \mathbf{y}$ and defined as follows:

\[\mathbf{x} \cdot \mathbf{y} = \sum_{i=1}^{d} x_i y_i\]

Here are some major characteristics about the dot product.

Dot product is a special case of inner product, which allows formal definitions of intuitive geometric notions, such as lengths, angles, and orthogonality.

  • Dot product is commutative: $\mathbf{x} \cdot \mathbf{y} = \mathbf{x} \cdot \mathbf{y}$
  • Dot product is distributive over multiplication: $\mathbf{x} \cdot (\mathbf{y} + \mathbf{z}) = \mathbf{x} \cdot \mathbf{y} + \mathbf{x} \cdot \mathbf{z}$
  • The dot product of a vector with itself is referred to as squared Euclidean norm. The norm defines the length of vector.
\[|\mathbf{x}|^2 = \mathbf{x} \cdot \mathbf{x}\]
  • By dividing a vector with its norm, we can normalize the vector; normalized vector only contains the information about the direction (unit vector).
\[\mathbf{x}^\prime = \frac{\mathbf{x}}{|\mathbf{x}|}\]