There is a well known vectorization operator $\mbox{vec}$ in matrix analysis.
I've vectorized my matrix equations, did some transformation of vectorized equations and now I want to get back to the matrix form. Is there special operator for it?
$\endgroup$ 62 Answers
$\begingroup$Adding to the excellent answer by Rodrigo de Azevedo, I would like to point out that there is an explicit formula for the inverse $\operatorname{vec}_{m\times n}^{-1}$, given by$$ \mathbb{R}^{mn}\ni x \mapsto \operatorname{vec}_{m\times n}^{-1}(x) = \big((\operatorname{vec} I_n)^\top \otimes I_m\big)(I_n \otimes x) \in \mathbb{R}^{m\times n}, $$where $I_n$ denotes the $n\times n$ identity matrix and $\otimes$ denotes the Kronecker product.
The formula above can be verified in the following way:
Let $X\in\mathbb{R}^{m\times n}$ be such that $\operatorname{vec}{X}=x\in\mathbb{R}^{mn}$, and let $X_k$ denote the $k$-th column of $X$. Additionally, let $M=I_n\otimes x \in\mathbb{R}^{mn^2\times n}$ and let $M_k$ denote the $k$-th column of $M$. Finally, we let $e_k\in\mathbb{R}^n$ be the $k$-th column of $I_n$. Note that $M_k=e_k\otimes\operatorname{vec}{X}$.
Recall the identity $\operatorname{vec}(ABC) = (C^\top\otimes A)\operatorname{vec}{B}$, which we shall make heavy use of.
Observing that $\operatorname{vec}(e_k^\top\otimes X)=M_k$, we see that$$ \big((\operatorname{vec} I_n)^\top \otimes I_m\big)M_k = \big((\operatorname{vec} I_n)^\top \otimes I_m\big)\operatorname{vec}(e_k^\top\otimes X) = \operatorname{vec}\big(I_m(e_k^\top\otimes X)\operatorname{vec}{I_n}\big) = \operatorname{vec}\big((e_k^\top\otimes X)\operatorname{vec}{I_n}\big) = \operatorname{vec}\big(\operatorname{vec}(XI_ne_k)\big) = \operatorname{vec}(Xe_k) = X_k, $$and thus$$ \big((\operatorname{vec} I_n)^\top \otimes I_m\big)M = \big((\operatorname{vec} I_n)^\top \otimes I_m\big) \begin{bmatrix} M_1 & \ldots & M_n \end{bmatrix} = \begin{bmatrix} X_1 & \ldots & X_n \end{bmatrix} = X. $$
$\endgroup$ $\begingroup$The inverse of the vectorization operator
$$\mbox{vec} : \mathbb{R}^{m \times n} \to \mathbb{R}^{mn}$$
is the operator
$$\mbox{vec}^{-1} : \mathbb{R}^{mn} \to \mathbb{R}^{m \times n}$$
such that
$\mbox{vec}^{-1} (\mbox{vec} (X)) = X$ for all $X \in \mathbb{R}^{m \times n}$.
$\mbox{vec} (\mbox{vec}^{-1} (x)) = x$ for all $x \in \mathbb{R}^{m n}$.
Once a matrix is vectorized, the original dimensions of the matrix are "forgotten". Hence, it would be wise to write the dimensions of the inputs of $\mbox{vec}$ and of the outputs of $\mbox{vec}^{-1}$ in subscripts, e.g., $\mbox{vec}_{m,n}$ and $\mbox{vec}_{m,n}^{-1}$. Note that $\mbox{vec}_{3,2}^{-1}$ is the inverse of $\mbox{vec}_{3,2}$, but not of $\mbox{vec}_{2,3}$.
$\endgroup$
