In this paper a twofold inverse problem for orthogonal matrix functions in the Wiener class is considered. The scalar-valued version of this problem was solved by Ellis and Gohberg in 1992. Under reasonable conditions, the problem is reduced to an invertibility condition on an operator that is defined using the Hankel and Toeplitz operators associated to the Wiener class functions that comprise the data set of the inverse problem. It is also shown that in this case the solution is unique. Special attention is given to the case that the Hankel operator of the solution is a strict contraction and the case where the functions are matrix polynomials.