Quick Constructions of Non-Trivial Real Symmetric Idempotent Matrices

In the theory and applications of linear models, idempotent matrices appear frequently. It remains important to construct non-trivial quadratic forms distributed as chi-squares. However, quick and easy methods to explicitly construct such matrices with all non-zero entries are hard to come by. We include two direct methods to explicitly exhibit n × n real symmetric idempotent matrices of rank r with all non-zero entries. Then, we exhibit explicit non-trivial real symmetric idempotent matrices A, B such that AB = O.


Introduction
An n×n matrix A is called idempotent if A = A 2 .Idempotent matrices are discussed in many books including Rao (1973), Cook and Weisberg (1982), Chatterjee and Hadi (1988), Harville (2001, chapter 10), Rao and Bhimasankaram (2000), and Ravishanker and Dey (2002).An n × n idempotent matrix A of real numbers with its rank ρ(A) = n is necessarily I n×n .Hence, non-trivial idempotent matrices of less than full rank play key roles in much of the theory and practice of linear models.
We explicitly exhibit an n × n real symmetric idempotent matrix A with ρ(A) = r < n for r and n held fixed.Initially, one may propose a partitioned matrix as follows: which is idempotent.But, in principle, this idempotent matrix has not gone very far from I n×n .We may require, for example, an n × n symmetric idempotent matrix A of real numbers with ρ(A) = r < n with all non-zero entries when (i) n = 2, r = 1, (ii) n = 3, r = 1, and (iii) n = 3, r = 2. Two simple examples for case (i) follow: 2) The next cases (ii)-(iii) are certainly more demanding.Some examples follow: Case (i): A = We include case (i) again in (1.3) simply because this A matrix is drastically different from those in (1.2).One may ask: Where do these examples come from?Can such examples be found quickly and easily?At this juncture, we may turn back and think of two specific results: (a) Every real idempotent matrix has the form PQ for some P and Q where Q P = I.This characterizes all idempotent matrices and it also means that P and Q have full column ranks; and (b) Every real symmetric idempotent matrix is of the form PP for some P where P P = I.This characterizes all symmetric idempotent matrices and it also means that P is of full column rank.Khatri (1985), on the other hand, included the following result which he compared with his previous result from Khatri (1983): Let A be an n × n matrix, and let the trace, the conjugate transpose, and the Moore-Penrose inverse of A be respectively denoted by trA, A * , and A † .Suppose that ρ(A) = trA.Then, tr A 2 A † A * ≥ ρ(A), and the equality holds if and only if A is idempotent.These rather well-known results, however, do not render immediate help in exhibiting concrete but non-trivial idempotent matrices quickly.Instead, they assure us to be able to claim that there exist ways to find non-trivial idempotent matrices.But, the existence and finding concrete and quick examples are not the same.We provide cute ideas to come up with non-trivial and concrete idempotent matrices quickly and efficiently.This paper describes two such instructive techniques explicitly.One may argue in the context of a linear model that a projection matrix A n×n (equivalently, I n×n − A) is idempotent.While that is true, the point is that one must then exhibit some appropriate projection matrix A n×n with a specified rank r(< n), both quickly and explicitly.This idea makes up Section 2. Next, we provide an alternative method involving chi-square distributions of quadratic forms in the first part of Section 3. In Section 3, we also illustrate explicit pairs of non-trivial, symmetric, real n × n idempotent matrices A, B such that AB = O n×n with ρ(A) = r(< n), ρ(B) = s(< n) where n, r, and s are arbitrary positive integers.Section 4 adds some concluding thoughts.

The First Idea: Projection Matrix
We begin with a customary linear model Y = Xβ + ε where Y n×1 is a response vector, X n×r is a known design matrix of real numbers, β r×1 is an unknown vector of regression parameters, and ε n×1 is the error vector with E[ε] = 0 and V[ε] = σ 2 I n×n .Let us assume that ρ(X) = r.One may refer to Rao (1973, chapter 4), Ravishanker and Dey (2002, chapter 4) or another appropriate source.In the theory and practice of linear models, a projection matrix appears very frequently.A key role of a projection matrix is especially played out in regression diagnostics and sensitivity analysis.One may refer to Cook and Weisberg (1982), Chatterjee and Hadi (1988, chapters 2-4), Rao (1973), and Ravishanker and Dey (2002, pp.320-336) for important discourses.In this section, an explicit construction depends heavily upon the specific form of a projection matrix: It is clear that the real matrix A from (2.1) is symmetric and idempotent.We first propose a construction in the light of (2.1) where X is an arbitrary real matrix of rank r.But, if X X is a general r × r matrix, finding its inverse explicitly may be highly time consuming and tedious especially when r is moderate or large.
Our point is to make sure to explicitly exhibit all entries of A which necessarily demands that we may write down each entry of (X X) −1 fast.In other words, we must specify the X matrix in such a way that the inverse of X X may be determined effortlessly.

A Method of Construction
Let us begin with r pairwise orthogonal non-zero column vectors p 1 , ..., p r from the n-dimensional Euclidean space, n .We define: that is where c i = p i p i (> 0), i = 1, ..., r.Obviously, one has: Then, we rewrite the idempotent matrix from (2.1) as follows:

Concrete Illustrations of Idempotent Matrices
Let us first point out how easily and quickly the examples in (1.3) were found.
Case (iv): n = 6 and r = 3 We begin with p 1 , p 2 , p 3 where That is, we have: Hence, (2.5) implies: , which reduces to: (2.9) One may obviously check that this A 6×6 matrix from (2.9) is idempotent with rank 3.
Recall that our general approach is found in (2.1)-(2.5)which puts great emphasis upon finding (X X) −1 quickly, painlessly, and explicitly.

The Second Idea: Quadratic Forms
Consider the following two quadratic forms: ) as illustrations, both constructed from U 1 , ..., U 10 which are assumed independent N(0, 1) random variables.With regard to the quadratic forms Q 1 and Q 2 from (3.1)-(3.2),one may ask the following questions: (i) Are Q 1 and Q 2 distributed as central chi-squares?If so, can we specify the respective degrees of freedom? and (ii) Are Q 1 , Q 2 distributed independently of each other?Why or why not?But, before we address these questions, we may indeed raise the following more pertinent question: How could we envision such formidable looking quadratic forms Q 1 , Q 2 in the first place?We give some directions and the main motivation before proposing a systematic method of construction.Let us begin with a set of n independent standard normal random variables U 1 , ..., U n with n ≥ 2. Next, let us define a new set of n random variables Y 1 , ..., Y n as follows: These are the customary Helmert's (1876) orthogonal variables.It is well-known that Y 1 , ..., Y n are distributed as independent standard normal random variables (Mukhopadhyay 2000, Example 4.4.9, pp. 197-201).Observe that we may add the squares of any suitable subset of the Y variables and accordingly claim the following: which follows from the reproductive property of independent central chi-square random variables, r < n (Mukhopadhyay 2000, p. 192).
Next, we may equivalently express W as a quadratic form, namely, U AU corresponding to the original random vector U where U = (U 1 , ..., U n ).U AU would then necessarily have a central χ 2 r distribution with an associated real symmetric matrix A n×n such that ρ(A) = r.This matrix A would then be idempotent with its rank r.

Concrete Illustrations of Idempotent Matrices
For the illustrations included in this section, we will fix moderately large n and r and exhibit how smoothly and efficiently the present approach works that builds upon distributions of quadratic forms. .
Hence, we have: which is a 10 × 10 real symmetric idempotent matrix with its rank 6.
Case (vi): n = 10 and r = 6 In this illustration, we may begin in a slightly different way than we did in Case (v).We now work with Y 1 , Y 3 , Y 4 , Y 6 , Y 7 , Y 9 from (3.3) and alternately express W from (3.4) as follows: where .
Hence, we have: ) again a 10 × 10 real symmetric idempotent matrix with its 6.The matrix A from (3.7) is clearly different from the one shown in (3.5).It is our hope that the inherent beauty and simplicity of this approach is now apparent.

Concrete Illustrations of Idempotent Matrices A and B with AB = O
Now, we want to construct two non-trivial real symmetric n×n idempotent matrices A and B where ρ(A) = r(< n) and ρ(B) = s(< n), and AB = O.Here, we will explain our basic approach with the help of an illustration only.
Case (vii): n = 10, r = 6, and s = 3 Let us first recall A from (3.7) and that it was derived from the quadratic form defined in (3.6).In this illustration, let us begin with Y 2 , Y 5 , Y 9 from (3.3) and express W from (3.4) as: where .
Quick Constructions of Non-Trivial Real Hence, we have .9) Obviously, the two quadratic forms from (3.6) and (3.8) are distributed independently and hence AB must be the zero matrix, O 10×10 .One may directly verify that AB = O and that ρ(B) = 3.
One may certainly construct non-trivial 10 × 10 idempotent matrices A, B such that AB O 10×10 .For example, consider the quadratic forms: (3.10) Obviously, the two matrices A, B associated with (3.10) will both be 10 × 10 idempotent with respective ranks 6 and 4 such that AB O 10×10 since the two associated quadratic forms from (3.8) are surely dependent.One may alternatively check:

Concluding Thoughts
The approach given in Section 2 works well with any set of orthogonal non-zero column vectors p 1 , ..., p r with r < n, however its implementation is quick and simple when n, r are both small or moderate.One is expected to begin with an arbitrary non-zero vector p 1 , and then write down a suitable non-zero vector p 2 that is orthogonal to p 1 .But, continuing too much further along may not necessarily be a simple task.In this regard, the orthogonal set of vectors p 1 , ..., p n in n where p 1 = (1, 1, 1, ..., 1, 1, 1), p 2 = (1, −1, 0, ..., 0, 0, 0), p 3 = (1, 1, −2, ..., 0, 0, 0), ..., p n−1 = (1, 1, 1, ..., 1, −n + 2, 0), p n = (1, 1, 1, ..., 1, 1, −n + 1), (4.1) are helpful.One will note that Helmert's orthogonal vectors utilized in (3.3) are normalized versions of the vectors from (4.1).Another idea will prove to be useful in locating numerous idempotent n × n real matrices with rank r(< n) once we find one such matrix A. One may construct a new matrix BAB −1 where B n×n is arbitrary but non-singular.The matrix BAB −1 is also an idempotent n × n real matrix with rank r(< n) (Harville 200, Exercise 1, p. 49).But, then a major obstacle is that a non-trivial and non-singular matrix B along with its inverse ought to be exhibited quickly and painlessly.However, we will encounter a rather painless resolution if we use B ≡ (b , for all i j. (4.2) In case (iii), that is when n = 3 and r = 2, we showed one matrix A in (2.8).Now, let us employ the technique of pre-and post-multiplying A with B and B −1 where B = 3I 3×3 + 1 3×3 1 3×3 .Then, we will have: , which itself is a real idempotent matrix with rank 2, but it is not a symmetric matrix.In Section 3, we exploited the Helmert orthogonal matrix.One could alternatively exploit Householder's symmetric orthogonal matrix.One may refer to Rao and Bhimasankaram (2000, pp. 269-270).
Case (v): n = 10 and r = 6 Let us begin with Y 1 , ..., Y 6 from (3.3) and express W from (3.4) as i j ) = (c − d)I n×n + d1 n×n 1 n×n where c, d are different but arbitrary non-zero real numbers, c −(n−1)d.Then, the matrix B −1 ≡ (b i j ) is known to have exactly the same form as that of B. Indeed, it is well-known (Rao 1973, Exercise 1.1, p. 67) that b ii = c + (n − 2)d [c + (n − 1)d](c − d) and b i j = −d [c + (n − 1)d](c − d)