Suppose:

Data : , i.e. we assume the data comes from some probability distribution which can be defined using the parameters . Now we want to make inference about the unknown parameter .

## Some Definitions:

- An estimator T(X) is a statistic (or a function of X).
- For a given observed value of x of X, T(x) is an estimate of .
- Mean Squared Error: .

We may define an estimator to be the estimator of if .

Unfortunately, such an estimator does not exist (Think Why!!!). Hence we have to approach from another direction. One way is to restrict attention to a class of estimators showing some degree of impartiality with a hope that we can find a best estimate in the restricted class. One such restriction is **Unbiasedness.**

- A statistic T(X) is said to be unbiased for estimating a parametric function if . We say that is unbiasedly estimable if such an estimator exists.

It can be easily seen that where is called the bias of T. (Check!!!)

- An unbiased estimator of is uniformly minimum variance unbiased estimator (UMVUE) if : and for any unbiased estimator T of .

**An Inequality: **For a random variable Z,

where c is a fixed real number. (Check!!!)

Consider now conditional distribution of a random variable U given another random variable T. Then we can see:

, thereby implying:

## Rao-Blackwell Theorem:

**Let U be an unbiased estimator of . Consider sufficient statistics T for and let . Then is an unbiased estimator of and .**

**Proof: **First note that ** **is free of as T is sufficient. Then ** **can be used as an estimator of ** ****. **Also,

Thus given any unbiased estimator, there exists an unbiased estimator based on T, which is “as good as” U in the sense that: .

**Lemma 1: **Let , and T(X) be a complete sufficient statistic for . Then every U-estimable parametric function ** **has one and only one unbiased estimator based on T in the sense that if and are two unbiased estimators based on T, then:

**Lemma 2: **Let T be a complete sufficient statistic. An Unbiased Estimator ** **of **, **based on T, is also an UMVUE.

(Try to Prove the above lemma.)

## Some Examples:

**Example 1: **Let’s first think about the Bernoulli case. iid Ber(p).

Here T=T(X)= is a complete sufficient statistic.

**UMVUE of p:**

, i.e. T/n is an unbiased estimator based on the complete sufficient statistics T , implying T/n is UMVUE.

**UMVUE of :**

Check that: . Thus UMVUE of p is .

**UMVUE of :**

UMVUE of = .

It can be easily seen that any U-estimable function is a polynomial in p of degree .

**Example 2: **Let’s now think about Poisson case. .Then, we know, is complete sufficient.

It is very easy to get UMVUE of for r integer.

**Let’s try to find UMVUE of P[X=k].**

- .

**Exercise: . Find UMVUE of .** [Hint: ]

**Example 3:** . Here, =sample mean is complete sufficient.

**Case 1: ( is known)**So, is UMVUE of . Similarly, UMVUE of is .

Suppose ** =1. **Want to estimate: .

Start with , then condition by .

, using the fact that and are independent (by Basu’s Theorem).

Hence, UMVUE: , where is cdf of N(0,1). (Check!!!)

**Case 2: ( known) Want UMVUE of .**

is complete sufficient statistic for .

, thereby giving , i.e T/n is UMVUE of .

, if .

Replacing r by r/2, we get the required UMVUE of ** **as: if .

**Case 3: (both and unknown)**

Now, is complete sufficient for , where .

Now, , , which gives UMVUE of as: (Chcek!!!)

Similarly, UMVUE of is: (Since and T are independent) (Check!!!)

Also, UMVUE of is , the small change from the previous version is due to change in degrees of freedom.

Thus, now we can get UMVUE of , which turns out to be: .

Now, to estimate quantiles: let: , we get the UMVUE of u, the p-th quantile, as: , where is the UMVUE of .

**Example 4: iid Exp(), i.e. with density **.Want UMVUE of .

Let . A complete sufficient statistics is .

, since is ancillary.

Now we know, if and , then

Therefore:

**Conclusion: Thus, in this article, we have gathered some basic knowledge about parametric inference, and a detailed discussion on UMVUE and some examples. Let me finish this topic with an example, which would illustrate why UMVUE does not guarantee minimum Mean Square Error.**

**Concluding Example: ** , where . We have learnt that the UMVUE of is . Now let us consider the statistic , which takes if , otherwise take – if and if .

Then, we can see that , even though is UMVUE.

So, let’s bid adieu for now. I will be back with some other interesting topic very soon.

Just try to motivate a little bit why one needs the methods of Parametric Estimation.

Thanks for the comment…This was mainly on Rao-Blackwell Theorem…Will try to post an introductory post on Parametric Inference with motivations later.