Linear algebra utility functions

Linear algebra utility functions.

mklaren.util.la.cosine_similarity(A, B)

Cosine similarity between matrices \(\mathbf{A}\) and \(\mathbf{B}\) defined in terms of the Frobenius product

\[\dfrac{<\mathbf{A}, \mathbf{B}>_F} {<\mathbf{A}, \mathbf{A}>_F <\mathbf{B}, \mathbf{B}>_F}\]
Parameters:
  • A – (numpy.ndarray) a matrix.
  • B – (numpy.ndarray) a matrix.
Returns:

(float) Cosine similarity value.
mklaren.util.la.cosine_similarity_low_rank(a, b)

Cosine similarity between matrices from outer products of vectors \(\mathbf{a}\) and \(\mathbf{b}\):

\[\mathbf{A} = \mathbf{aa}^T\]
\[\mathbf{B} = \mathbf{bb}^T\]
\[\dfrac{<\mathbf{A}, \mathbf{B}>_F} {<\mathbf{A}, \mathbf{A}>_F <\mathbf{B}, \mathbf{B}>_F}\]
Returns:(float) Cosine similarity value.
mklaren.util.la.cosine_similarity_low_rank_multi(G, y)

Cosine similarity between matrices from outer products of matrix \(\mathbf{G}\) and vector \(\mathbf{y}\).

Parameters:
  • G – (numpy.ndarray) Low-rank matrix.
  • y – (numpy.ndarray) Column vector.
Returns:

(float) Cosine similarity (kernel alignment).
mklaren.util.la.covariance_full_rank(K, sigma2)

Complete the kernel/covariance matrix to full rank by adding an indetity matrix.

\[(\mathbf{K} + \sigma^2 \mathbf{I})^{-1}\]
Parameters:
  • K – (numpy.ndarray) Kernel matrix of shape (n, n).
  • sigma2 – (float) Added covariance.
Returns:

(numpy.ndarray) Modified kernel matrix.
mklaren.util.la.ensure_symmetric(K)

Ensure a symmetric matrix. Small deviations in symmetry can present obstacles to numpy.svd used by samplers.

Parameters:K – (numpy.ndarray) Kernel matrix of shape (n, n).
Returns:(numpy.ndarray) Correction to C up to symmetry.
mklaren.util.la.fro_prod(A, B)

The Frobenius product is an inner product between matrices \(\mathbf{A}\) and \(\mathbf{B}\) of same shape.

\[<\mathbf{A}, \mathbf{B}>_F = \sum_{i, j} \mathbf{A}_{ij} \mathbf{B}_{ij}\]
Parameters:
  • A – (numpy.ndarray) a matrix.
  • B – (numpy.ndarray) a matrix.
Returns:

(float) Frobenius product value.
mklaren.util.la.fro_prod_low_rank(A, B)

The Frobenius product of kernel matrices induced by linear kernels on \(\mathbf{A}\) and \(\mathbf{B}\):

\[<\mathbf{AA}^T, \mathbf{BB}^T>_F\]

Note \(A\) and \(B\) need not be of the same shape.

Parameters:
  • A – (numpy.ndarray) a matrix.
  • B – (numpy.ndarray) a matrix.
Returns:

(float) Frobenius product value.
mklaren.util.la.normalize(X)

Normalize \(\mathbf{X}\) to have values between 0 and 1.

Parameters:X – (numpy.ndarray) Data matrix.
Returns:(numpy.ndarray) Normalize matrix.
mklaren.util.la.outer_product(a, b)

Outer product between vectors \(\mathbf{a}\) and \(\mathbf{b}\):

Parameters:
  • a – (numpy.ndarray) column vector.
  • b – (numpy.ndarray) column vector.
Returns:

(float) Outer product of the appropriate dimension.
mklaren.util.la.qr(A)

Fast QR decomposition.

Parameters:A – (numpy.ndarray) A matrix of shape (m, n).
Returns:(tuple) of (numpy.ndarray) orthonormal vectors (Q) and (numpy.ndarray) upper triangular matrix (R).
mklaren.util.la.safe_divide(A, b)

Perform a safe float division between \(\mathbf{A}\) and \(\mathbf{b}\).

Parameters:
  • A – (numpy.ndarray) Scalar / Matrix.
  • b – (numpy.ndarray) Scalar / Matrix.
Returns:

(float) Matrix divided by b if b not zero else 0.
mklaren.util.la.safe_func(x, f, val=0)

Evaluate a function defined on positive reals.

Parameters:
  • x – (float) a value.
  • f – (callable) Function to evaluate.
  • val – (float) Return value when x is out of domain.
Returns:

(float) Function value or out of domain.
mklaren.util.la.woodbury_inverse(G, sigma2)

Matrix inversion using the Woodbury-Sherman-Morrison lemma.

\[(\mathbf{GG}^T + \dfrac{1}{\sigma^2} \mathbf{I})^{-1}\]

Order of operations is important.

Parameters:
  • G – (numpy.ndarray) Low-rank matrix.
  • sigma2 – (float) Noise / nugget term / sigma squared.
Returns:

(numpy.ndarray) The solution to the above equation by inverting a shape (k, k) matrix.
mklaren.util.la.woodbury_inverse_full(Ai, U, Ci, V)

Matrix inversion lemma Woodbury-Sherman-Morrison in its general form.

\[(\mathbf{A} + \mathbf{UCV})^{-1}\]
Parameters:
  • Ai – (numpy.ndarray) Inverse of a square matrix of shape (n, n).
  • U – (numpy.ndarray) Matrix of shape (n, k).
  • Ci – (numpy.ndarray) Inverse of square matrix of shape (k, k).
  • V – (numpy.ndarray) Matrix of shape (k, n).
Returns:

(numpy.ndarray) The solution to the above equation by inverting A and C. The solution is of shape (n, n).