Semi-parametric Order-based Generalized Multivariate Regression

1. Problem Setup In linear multivariate regression, we have the following model: y T i = x T i B + ǫ, i = 1, . . . , n, (1) where yi ∈ R q×1 is the response vector (q 1), xi ∈ R p×1 is the predictor vector, B ∈ R p×q is the coefficient matrix, and ǫi ∈ R q×1 represents the noise with i.i.d. elements that are independent of xi . In this paper, we consider the following extension of this problem: y T i = Ui(x T i B + ǫ T i ), i = 1, . . . , n, (2) where Ui : R → R is a non-degenerate monotonic function called the utility or link function. When the input of Ui is a vector or a matrix, it is implied that Ui is applied separately on each individual element to give the output, which is a vector or matrix of the same size as the input. Without loss of generality, we assume that Ui is an increasing function. We propose a semi-parametric, rank-based approach to estimate B which is invariant with respect to the functional form of Ui functions. Our approach only uses the ordering of the elements of yi , which makes it more robust to outliers and heavy-tailed noise compared to traditional regression algorithms. This also makes our approach applicable to cases where the numeric values of yi are not available, and only their ordering is known. We show that it is possible to consistently estimate B solely based on the ordering of the elements of yi . Our approach to estimating B is based on maximizing Kendall’s rank correlation of y T i and x T i B. For notational simplicity, we assume that all the link functions are equal and denote them by arXiv:1602.06276v1 [stat.ML] 19 Feb 2016 U; however, all the results presented in this paper hold for the case where there is a separate link function, Ui , for each observation. Let us rewrite (2) in matrix form: