# Rao-Blackwell Theorem and it’s applications 2

Suppose:

Data : , i.e. we assume the data comes from some probability distribution which can be defined using the parameters . Now we want to make inference about the unknown parameter .

## Some Definitions:

• An estimator T(X) is a statistic (or a function of X).
• For a given observed value of x of X, T(x) is an estimate of .
• Mean Squared Error:  .

We may define an estimator  to be the estimator of  if .

Unfortunately, such an estimator does not exist (Think Why!!!). Hence we have to approach from another direction. One way is to restrict attention to a class of estimators showing some degree of impartiality with a hope that we can find a best estimate in the restricted class. One such restriction is Unbiasedness.

• A statistic T(X) is said to be unbiased for estimating a parametric function  if . We say that  is unbiasedly estimable if such an estimator exists.

It can be easily seen that  where  is called the bias of T. (Check!!!)

• An unbiased estimator  of  is uniformly minimum variance unbiased estimator (UMVUE) if : and for any unbiased estimator T of .

An Inequality: For a random variable Z,

where c is a fixed real number. (Check!!!)

Consider now conditional distribution of a random variable U given another random variable T. Then we can see:

, thereby implying:

## Rao-Blackwell Theorem:

Let U be an unbiased estimator of . Consider sufficient statistics T for  and let . Then  is an unbiased estimator of  and .

Proof: First note that  is free of  as T is sufficient. Then  can be used as an estimator of  Also,

Thus given any unbiased estimator, there exists an unbiased estimator  based on T, which is “as good as” U in the sense that: .

Lemma 1Let , and T(X) be a complete sufficient statistic for . Then every U-estimable parametric function  has one and only one unbiased estimator based on T in the sense that if  and  are two unbiased estimators based on T, then:

Lemma 2Let T be a complete sufficient statistic. An Unbiased Estimator  of based on T, is also an UMVUE.

(Try to Prove the above lemma.)

## Some Examples:

Example 1Let’s first think about the Bernoulli case.  iid Ber(p).

Here T=T(X)=  is a complete sufficient statistic.

• UMVUE of p:

, i.e. T/n is an unbiased estimator based on the complete sufficient statistics T , implying T/n is UMVUE.

• UMVUE of :

Check that: . Thus UMVUE of p is .

• UMVUE of :

UMVUE of  = .

It can be easily seen that any U-estimable function is a polynomial in p of degree .

Example 2: Let’s now think about Poisson case. .Then, we know,  is complete sufficient.

It is very easy to get UMVUE of  for r integer.

Let’s try to find UMVUE of P[X=k].

• .

Exercise: . Find UMVUE of  .                                               [Hint: ]

Example 3: . Here, =sample mean is complete sufficient.

• Case 1: ( is known)  So,  is UMVUE of . Similarly, UMVUE of  is

Suppose  =1. Want to estimate: .

, using the fact that  and  are independent (by Basu’s Theorem).

Hence, UMVUE: , where  is cdf of N(0,1). (Check!!!)

• Case 2: ( known) Want UMVUE of

is complete sufficient statistic for .

, thereby giving , i.e T/n is UMVUE of .

, if .

Replacing r by r/2, we get the required UMVUE of  as:  if .

• Case 3: (both  and  unknown)

Now,  is complete sufficient for , where .

Now, , , which gives UMVUE of  as:  (Chcek!!!)

Similarly, UMVUE of  is:   (Since  and T are independent) (Check!!!)

Also, UMVUE of  is , the small change from the previous version is due to change in degrees of freedom.

Thus, now we can get UMVUE of , which turns out to be: .

Now, to estimate quantiles: let: , we get the UMVUE of u, the p-th quantile, as: , where  is the UMVUE of .

Example 4  iid Exp(), i.e. with density  .Want UMVUE of .

Let . A complete sufficient statistics is .

, since  is ancillary.

Now we know, if  and , then

•

Therefore:

Conclusion: Thus, in this article, we have gathered some basic knowledge about parametric inference, and a detailed discussion on UMVUE and some examples. Let me finish this topic with an example, which would illustrate why UMVUE does not guarantee minimum Mean Square Error.

Concluding Example:  , where . We have learnt that the UMVUE of  is .  Now let us consider the statistic , which takes  if , otherwise take – if  and    if  .

Then, we can see that , even though  is UMVUE.

So, let’s bid adieu for now. I will be back with some other interesting topic very soon.

## 2 thoughts on “Rao-Blackwell Theorem and it’s applications”

• Debarghya Mukherjee

Just try to motivate a little bit why one needs the methods of Parametric Estimation.

• djghosh Post author

Thanks for the comment…This was mainly on Rao-Blackwell Theorem…Will try to post an introductory post on Parametric Inference with motivations later.