On spectral and nuclear norms of order three tensors with one fixed dimension

The recent decade has witnessed a surge of research in modelling and computing from two-way data (matrices) to multiway data (tensors). However, there is a drastic phase transition for most tensor computation problems when the order of a tensor increases from two to three: Most tensor problems are NP-hard while that for matrices are easy. It triggers a question on where exactly the transition occurs. The paper aims to study the kind of question for the spectral norm and the nuclear norm. Although computing the spectral norm for a general ℓ×m×n tensor is NP-hard, we show that it can be computed using a number of arithmetic operations which is polynomial in m and n if ℓ is fixed. In another word, the best rank-one approximation of a general ℓ×m×n tensor can be computed in polynomial time for fixed ℓ. Under an assumption of polynomial-time algorithm for quadratic feasibility problem in the bit complexity, both the spectral norm and the nuclear norm of a general ℓ×m×n tensor can be computed in polynomial time in the bit complexity for fixed ℓ. While these polynomial-time methods are not currently implementable in practice, we propose fully polynomial-time approximation schemes (FPTAS) for the spectral norm based on polytope approximation and for the nuclear norm with further help of duality theory and semidefinite optimization. Numerical experiments show that our FPTAS can compute these tensor norms for small ℓ but large m and n. To the best of our knowledge, this is the first method to compute the nuclear norm of general asymmetric tensors. Both polynomial-time algorithms and FPTAS can be extended to higher-order tensors as well.