Lecture 22: Random Walks and Thresholds

• Download English-US transcript (PDF)

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu

PROFESSOR: OK, we have a busy day today, so let's get started. Want to go through Chernoff bounds and the Wald identity, which are closely related, as you'll see, and that involves coming back to the TG1Q a little bit and making use of what we did for that. It also means coming back to hypothesis testing and using that. Would probably have been better to start out with Wald's identity and the Chernoff bound and then do the applications when it was at the natural time for them. But anyway, this is the way it is this time, and next time we'll probably do it differently.

Suppose you have a random variable z. It has a moment generating function. Remember, not all random variables have moment generating functions. It's a pretty strong restriction.

You need a variance. You need moments of all orders. You need all sorts of things, s but we'll assume it exists in some region between r and r plus.

There's always a question, with moment generating functions, if they exist up to some maximum value of r because some of them exist at that value of r and then disappear immediately after that, and others just sort of peter away as r approaches r plus from below. I think in the homework this week, you have an example of both of those. I mean, it's a very simple issue. If you have an exponential distribution, then as r approaches, the rate of that exponential distribution, obviously, the moment generating function blows up because you're taking e to the minus lambda x, and you're multiplying it by a the r x. And when r is equal to lambda, bingo, you're integrating 1 over an infinite range, so you've got infinity. If you multiply that exponential by something which makes the integral finite when you set r equal to lambda, then of course, you have something which is finite at r star.

That is a big pain in the neck. It's usually not important. The notes deal with it very carefully, so we're not going to deal with it here. We will just assume here that we're talking about r less than r plus and not worry about that special case, which usually is not all that important. But sometimes you have to worry about it.

OK, the Chernoff bound says that the probability that random variable is greater than or equal to alpha is less than or equal to the moment generating function evaluated at some arbitrary value r times e to the minus r alpha. And if you put it in terms of the semi invariant moment generating function, the log of the moment generating function, then the bound is e to the gamma z of r minus alpha r. When you see something like that, you ought to look at it and say, gee, that looks funny because here, we're taking an arbitrary random variable and saying the tails of it have to go down exponentially.

That's exactly what this says. It says that a z takes on very large values. This is a fixed quantity here for a given value of r, and it's going down as e to the minus r times alpha. As you make alpha larger and larger, this goes down faster and faster.

So what's going on? How do you take an arbitrary random variable and say the tails of it is exponentially decreasing? That's why you have to insist that the moment generating function exists because when the moment generating function exists for some r, it means that the tail of that distribution is, in fact, going down at least that fast, so you get something that exists. So the question is what's the best bound of this sort of when you optimize o for r?

Then the next thing we did is we said that z is a sum of IID, then the semi invariant moment generating function for that sum is equal to n times the semi invariant moment generating function for the underlying random variable x. S of n is n of these IID random variable. So one thing you see immediately, and ought to be second nature to you now, is that if a random variable has a moment generating function over some range, the sum of a bunch of those IID random variables also has a moment generating function over that same range. You can just count on that because the semi invariant moment generating function is just n times this b.

OK, so then what we've said is the probability that sn is greater than or equal to na, where na is playing the role of alpha and sn is playing the role of z, is just a minimum over r of e to the n times gamma x of r minus ra, and the n is multiplying the ra as well as the n. OK, this is exponential n for a fixed a. In other words, what you do in this minimization, if you don't worry about the special cases or anything, how do you minimize something? Well, obviously, you want to minimize the exponent here, so you take the derivative of this gamma prime of r has to be equal to a. Then n can be whatever it wants to be when you find that optimum r, which is where gamma prime of r equals a, then this goes down exponentially with a.

Now, however, we're interested in something else. We're interested in threshold crossings. We're not interested in picking a particular value of a and asking, as n gets very, very big, what's the probability that the sum of random variable is greater than or equal to n times a. That is exponential in n, but what we're interested in is the probability that s of n is greater than or equal to just some constant alpha, and what we're doing, now, is instead of varying n and varying this with n also, we're holding the stick.

So we're asking as n gets very, very large, but you hold this alpha fixed, what happens on this bound over here? Well, when you minimize this, taking the same simple-minded view, now the n is not multiplied by the ra. It's just multiplied by the gamma x. You get n times gamma prime of r is equal to alpha s where the minimum is so that it says gamma prime of r is optimized when you pick gamma prime of r equal to alpha over and n. This quantity is minimized when you pick gamma prime of r equal to alpha over n.

So if you look at this bound as n changes, what's happening is, as n changes, r is changing also, so this is a harder thing to deal with for variable n. But graphically, it's quite easy to deal with. I'm not sure you all got the graphical argument last time when we went through it, so I want to go through it again. Let's look at this exponent r minus n over alpha times gamma of r, and see what it looks like.

We'll take r, pick any old r, there. What we want to do is show that this, if you take a slope of alpha over n, and take an arbitrary r, come down to gamma of x of r, draw a line in this slope, and look at where it hits the horizontal axis here, that point is r plus the length of this line here. The length of this line here is gamma of r, that's a negative value, times 1 over that slope of this line. And 1 over the slope of this line is n over alpha, so when I pick a particular value of r, the value of the experiment I have is this value here.

How do I optimize this over r? How do I get the largest exponent here? Well, I think of varying r, as I vary r from 0, and each time, I take this straight line here. And I start here, draw a straight line over there, start here, draw a straight line over, start at this tangent here, draw a straight line over.

And what happens when I come to larger values of r? Just because gamma of s of r is convex, what happens is I start taking these slope lines, slope alpha over n, and they intercept the horizontal axis at a smaller value. So this is optimized over r at the value of r0, which satisfies alpha over n equals gamma prime of r0.

That's the same answer we got before when we just used elementary calculus. Here, we're using a more sophisticated argument, which you learned about probably in 10th grade. I would argue that you learned mostly really sophisticated things when you're in high school, and then when you get to study engineering in college, somehow you always study these mundane things.

But anyway, aside from that, why is this geometric argument better? Well, when you look at these special cases of what happens when gamma of r comes around like this, and then suddenly it stops in midair and just doesn't exist anymore? So it comes around here, it's still convex, but then suddenly it goes off to infinity. How do you do that optimization then? Well, the graphical argument makes it clear how you do it, and makes it perfectly rigorous how to do it, whereas if you're doing it by calculus, you've got a really think it through, and it becomes fairly tricky.

OK, so anyway, now, the next question we want to ask-- I mean, at this point, we've seen how to minimize this quantity over r, so we know what this exponent is for a particular value of n. Now, what happens when we vary n? As you vary n, the thing that happens is we have this tangent line here, a slope alpha over n. When you start making n larger, alpha over n becomes smaller, so the slope becomes smaller. And as n approaches infinity, you wind up going way, way the heck out.

As n gets smaller, you come in again. You keep coming in until you get to this point here. And what happens then? We're talking about a line of-- maybe I ought to draw it on the board. It would be clearer, I think.

As n gets smaller, you get a point which is tangent here, this here. When you're here, the tangent gets right here, so we've moved all the way into this quantity we call r star, which is the root of the equation gamma of r equals 0. Gamma of r equals 0 typically has two roots, one here, and one at 0. It always has a root at 0 because moment generating function evaluated is 0 is always 1, so the log of it is always 0. There should be another root because this is convex, unless it drops off suddenly, and even if it drops off suddenly, you can visualize it as a straight line going off to infinity.

So when you get down to this point, what happens? Well, we just keep moving along. So as n increases, we start out very large. We come in. We hit this point, and then we start coming out again.

I mean, if you think about it, that makes perfect sense because what we're doing here is we're imagining experiment where this random variable has a negative expected value. That's what's indicated by this quantity there. We're asking what's the probability that the sum of a large number of IID random variables with a negative expected value ever rises above some positive threshold?

Well, the law of large numbers says it's not going to do that when n is very, very large, and this says that, too. It says the probability of it for n very large is extraordinarily small. It's e to the minus 10 times an exponent, which is very, very large. So as n gets very small, it's not going to happen either because it doesn't have time to get to the threshold.

So there's some intermediate value at which it's most likely to cross the threshold, if you're going to cross the threshold, and that intermediate value is just that value at which gamma of r star is equal to zero. So the probability this union of terms, namely the probability you ever cross alpha, is going to be, in some sense, approximately a to the minus alpha r star because that's where the dominant term is. The dominant term is where alpha over n is equal to gamma prime.

Blah, blah, blah, blah, blah, where'd I put that? r star satisfies gamma of r star equals 0. When you look at the line of slope, gamma prime of r plus of r star, that's where you get this critical value of n where it's most likely the cross the threshold.

OK, I put that somewhere. I thought it was on this slide, but it's the n, the critical n, let's call it n crit, is equal to gamma prime. Is at right? Alpha over m is the gamma prime. Alpha over n, 1 over n crit. n crit, this says, is alpha over gamma prime of r star.

OK, so that sort of nails down everything you want to know about the Chernoff bound accept for the fact that it is exponentially tight. The text proves that. I'm not going to go through that here. Exponentially tight means, if you take an exponent which is just a little bit larger than the one you found here, and look what happens as alpha gets very, very large, then you lose.

OK, let's go on, and at this point, we're ready to talk about Wald's identity. And we'll prove Wald's identity at the end of the lecture today. Turns out there's a very, very simple proof of it. There's hardly anything to it, but it seems more important to use it in several ways first so that you get a sense that it, in fact, is sort of important. OK, so we want to think about a random walk, s sub n and greater than or equal to n, so it's a sequence of sums of random variable, s sub n is equal to x1 plus up to x sub n. The x's are all IID.

This is the thing we've been talking about all term. We have a bunch of IID random variables. We look at the partial sums of them. We're interested in what happens to that sequence of partial sums.

The question we're asking here is does that sequence of partial sums ever cross a positive threshold? And now we're asking does it ever cross a positive threshold, or does it cross a negative threshold, and which does it cross first? So the probability that it crosses this threshold is the probability that it goes up, first. The probability that it crosses this threshold is the probability that it goes down, first.

Now, what Wald's identity says is the following thing. We're going to assume that x is not identically 0. If x is identically 0, then it's never going to go any place. We're going to assume that it has a semi invariant moment generating function in some region, r minus to r plus. That's the same as assuming that it has a generating function in that region, so it exists from some value less than zero to some value greater than zero.

And we picked two thresholds, one of them positive, one of them negative, and we let j be the smallest value of n. j is a random variable, now, because we've start to run this random walk. We run it until it crosses one of these thresholds, and if it crosses the positive threshold, j is the time at which it crosses the positive threshold. If it crosses the negative threshold, j is the time that it crosses the negative threshold. We're only looking at the first threshold that it crosses.

Now, notice that j is a stopping trial. In other words, what that means is you can determine whether you've crossed a threshold at time n solely in terms of s1 up to s sub n. If you see all these sums, then you know that you haven't crossed a threshold up until time n. You know you have crossed it at time n. Doesn't make any difference what happens at times greater than n.

OK, so it's a stopping trial in the same sense as the stopping trials we talked about before. You get the sense that Wald's identity, which we're talking about here, is sort of like Wald's equality, which we talked about before. Both of them have to do with these stopping trials. Both of them have everything to do with stopping trials.

Wald was a famous statistician, not all that much before your era. He didn't die too long ago. I forget when, but he was one of the good statisticians. See, he was a statistician who recognized that you wanted to look at lots of different models to understand the problem, rather than a statistician who only wanted to take data and think that he wasn't assuming anything. So Wald was a good guy.

And what his identity says is, and the trouble with his identity, is you look at it, and you blink. The expected value of e to the r s sub j. s sub j is the value of the random walk at the time when you cross a threshold minus the time at which you've crossed a threshold times gamma of r. So when you take the expected value of e to the this, you're averaging over j over the time that you crossed the threshold, and also at the value at which you crossed the threshold, so you're averaging over both of these things.

And Wald says this expectation not is less than or equal to 1, but it's exactly 1, and it's exactly 1 for every r between r minus and r plus. So it's a very surprising result. Yes?

AUDIENCE: Can you please explain why j cannot be defective? I don't really see it.

PROFESSOR: Oh, it's because we were looking at two thresholds. If we only had one threshold, then it could be defective. Since we're looking at two thresholds, you keep adding random variables in, and the sum starts to have a larger and larger variance. Now, even with a large variance, you're not sure that you crossed a threshold, but you see why you must cross a threshold. Yes?

AUDIENCE: If the MGF is defined at r minus, r plus, then is that also [INAUDIBLE] quality?

PROFESSOR: Yes. Oh, if it's defined at r plus. I don't know. I don't remember, and I would have to think about it hard. Funny things happen right at the ends of where these moment generating functions are defined, and you'll see why when we prove it.

I can give you a clue as to how we're going to prove it. What we're going to do is, for this random variable x, we're going to define another random variable which has the same distribution as x except it's tilted. For large values of x, you multiply it by e to the rx. For small values of x, you multiply it by e to the rx also.

But if r is positive, that means the positive values could shifted up, and the small values get shifted down. So you're taking some of the density that looks like this, and when you shift it to this tilted value, you're shifting the whole thing upward. When r is negative, you're shifting the whole thing downward. Now, what this says is that tilted random variable, when it crosses the threshold, the time of crossing the threshold is still a random variable.

You will see that this simply says that the expected value of that tilted random variable is equal to-- it says that tilted random variable is, in fact, the random variable. It's not defective. And it's the same argument as before, that has a finite variance, and therefore, since it has a finite variance, it keeps expanding. It will cross one of the thresholds eventually.

OK, so the other thing you can do here is to say, suppose instead of crossing a threshold, you just fix this stopping rule to say we'll stop at time 100. If you stop at time 100, then what this says is expected value of e to the r100 minus 100 times gamma of r is equal to 1. But that's obvious because the expected value of e to the r is j is, in fact-- it's j times the expected value of rx, so then you're subtracting off j times the log of the expected value of rx. So it's a trivial identity if x is fixed.

OK, so Wald's identity says this. Let's see what it means in terms of crossing a threshold. We'll assume both thresholds are there. Incidentally, Wald's identity is valid in a much broader range of circumstances than just where you have two thresholds, and you're looking at a threshold crossing. It's just that's a particularly valuable form of the Wald identity.

So that's the only thing we're going to use. But now, if we assume further that this random variable x has a negative expectation, when x has a negative expectation, gamma of r starts off going down. Usually, it comes back up again. We're going to assume that this quantity r star here, where it crosses 0 again, we're going to assume there is some value of r, for which gamma of r star equals 0.

Mainly, we're going to assume this typical case in which it comes back up and crosses the 0 point. And in that case, what it says is the probability that sj is greater than or equal to alpha is just less than or equal to e to the minus r star times alpha. Very, very simple bound at this point. You look at this, and you sort of see why we're looking, now, not at r in general, but just r star.

At r star, gamma of r star is equal to 0. So this term goes away, so we're only talking about the expected value of e to the r sj, e to the r star sj is equal to 1. So let's see what happens. We know that e to the r star sj is greater than or equal to 0 for all values of sj because e to the anything real is going to be positive.

OK, since e to the r star sj is greater than or equal to 0, what we can do is break this expected value here, this term is 0, now, remember, break it into two terms. Break it into the term where s sub j is bigger than alpha, and break it into the term where s sub j is less than beta. So I'm just going to ignore the case where it's less than or equal to beta.

I'm going to take this expected value. I'm going to write it as the probability that s sub j is greater than or equal to alpha times e times the expected value of either the r star s sub j given s sub j greater than or equal to alpha. There should be another term in here to make this equal, and that's the probability that s sub j is less than or equal to beta times e to the r star s sub j, given that s sub j is less than or equal to beta. We're going to ignore that, and that's why we get the less than or equal to 1 here.

Now, you can lower bound e to the r star sj under this condition. What's a lower bound to s sub j given that s sub j is greater than or equal to alpha? Alpha.

OK, we're looking at all cases where s sub j is greater than or equal to alpha, and we're going to stop this experiment at the point where it first exceeds alpha. So we're going to lower bound the point where it first exceeds alpha by alpha itself, so this quantity is lower bounded, again, by taking the probability that s sub j greater than or equal to alpha times e to the r star alpha, and that whole thing is less than or equal to 1. That says the probability that sj is greater than or equal to alpha is less than or equal to e to the minus r star alpha, which is what this inequality says here. OK, so this is not rocket science. This is a fairly simple result if you believe in Wald's identity, which we'll prove later.

OK, so it's valid for all choices of this lower threshold. And remember, this probability here, it doesn't look like it's a function of both alpha and beta, but it is because you're asking what's the probability that you cross the threshold alpha before you cross the threshold beta. And if you make beta very, very large, it makes it more likely that you're going to cross the threshold. If you make beta very close to 0, then you're probably going to cross beta first, so this inequality here, this quantity here, depends on beta also. But we know that this inequality is valid no matter what beta is, so we can let beta approach minus infinity, and we can still have this inequality.

There's a little bit tricky math involved in that. There's an exercise in the text which goes through that slightly tricky math, but what you find is that this bound is valid with only one threshold, as well as with two thresholds. But this proof here that we've given depends on a lower threshold, which is somewhere. We don't care where.

Valid for all choices of beta, so it's valid without a lower threshold. The probability that the union overall n of sn less than or equal to alpha. In other words, the probability that we ever crossed a threshold alpha--

AUDIENCE: It's not true equal.

PROFESSOR: What?

AUDIENCE: It's supposed to be sn larger [INAUDIBLE] as the last time?

PROFESSOR: It's less than or equal to e to the minus r star alpha, which is--

AUDIENCE: Oh, sn? sn?

PROFESSOR: n.

AUDIENCE: You just [INAUDIBLE] [? the quantity? ?]

PROFESSOR: Oh, it's a union overall n greater than or equal to 1. OK, in other words, this quantity we're dealing with here is the probability that sn--- oh, I see what you're saying. This quantity here should be greater than or equal to alpha. You're right.

Sorry about that. I think it's right most places. Yes, it's right. We have it right here. The probability of this union is really the same as the probability that the value of it, after it crosses the threshold, is greater than or equal to alpha.

OK, now, we saw before that the probability that s sub n is greater than or equal to alpha. Excuse me, that's the same. When you're writing things in LaTeX, the symbol for less than or equal to is so similar to that for greater than or equal to that's hard to keep them straight. That quantity there is a greater than or equal to sign, if you're going from right to left instead of right to left. So all we're doing here is simply using this, well, greater than or equal to.

OK, the corollary makes a stronger and cleaner statement that the probability that you ever cross alpha is less than or equal to-- my heavens, my evil twin got a hold of these slides. And let me rewrite that one. The probability that the union overall n of the event s sub n greater than or equal to alpha is less than or equal to e to the minus r star alpha.

OK, so we've seen from the Chernoff bound that for every n this bound has satisfied, this says that it's not only satisfied for each n, but it says it's satisfied overall n collectively. Otherwise, if we were using the Chernoff bound, what would we have to do to get a handle on this quantity? We'd have to use the union bound, and then when we use the union bound, we can show that for every n, the probability that sn is greater than or equal to alpha is less than or equal to this quantity.

But then we'd have to add all those terms, and we would have to somehow diddle around with them to show that there are only a few of them which are close to this value, and all the rest are negligible. And the number that are close to that value is only growing with n and goes through a lot of headache. Here, we don't have to do this anymore because the Wald identity has saved is from all that difficulty.

OK, we talked about the G/G/1 queue. We're going to apply this corollary to the G/G/1 queue to the queueing time, namely to the time w sub i that the i's arrival spends in the queue before starting to be served. You remember, when we looked at that, we found that if we define u sub i to be equal to the ith interarrival time minus the i minus first service time, those are independent of each other, so this is the difference between those. So ui is the difference between the i's arrival time and the previous service time.

What we showed was that this sequence, u sub i, the sequence of the sums of u sub i as a modification of a random walk. In other words, the sums of the u sub i behave exactly like a random walk does, but every time it gets down to 0, if it crosses 0, it resets to 0 again. So it keeps bouncing up again.

If you look in the text, what it shows is that if you look at this sequence of u sub i's, and you look at the sum of them, and you look at them backward, if you look at the sum of u sub i plus u sub i minus 1 plus u sub i minus 2, and so forth, when you look at the sum that way, it actually becomes a random walk. Therefore, we can apply this bound to the random walk, and what we find is that the probability that the waiting time of n queue, of the nth customer, is probability that it's greater than or equal to an arbitrary number alpha is less than or equal to the probability that w sub infinity is greater than or equal to alpha, and it's less than e to the minus r star alpha. So again, all you have to do is you have this inner arrival time x, you have this service time y, you take the difference of the two, that's a random variable, you find a moment generating function of that random variable, you find the point of r star at which that moment generating function equals 1, and then the bound says that the probability that the queueing time that you're going to be dealing with is less than or equal to this quantity alpha here. Yes?

AUDIENCE: What do you work with when you have the gamma function go like this, and thus have infinity, and you cross it there. [INAUDIBLE] points that we're looking for?

PROFESSOR: For that, you have to read the text. I mean, effectively, you can think of it just as if gamma of r is a convex function like anything else. It just has a discontinuity in it, and bingo, it shoots off to infinity. So when you take these slope arguments, what happens is that for all slopes beyond that point, they just seesaw around at one point. But the same bound holds.

OK, so that's the Kingman bound. Then we talked about large deviations for hypothesis test. Well, actually we just talked about hypothesis test, but not large deviation for them. Let's review where we were on that.

Let's let the vector y be an n tuple of IID random variables, y1 up to y sub n. They're IID conditional on hypothesis 0. They're also IID conditional on hypothesis 1, so the game is nature chooses either hypothesis 0 or hypothesis 1. You take n samples of some IID random variable, and those n samples are IID conditional on either nature choosing 0 or nature choosing 1. At the end of choosing those n samples, you're supposed to guess whether h0 is the right hypothesis or 1 is a right hypothesis.

Invest in Apple stock 10 years ago, and one hypothesis is it's going to go broke. The other hypothesis is it's going to invent marvelous things, and your stock will go up by a factor of 50. You take some samples, you make your decision on that. Fortunately, with that, you can make a separate decision each year, but that's the kind of thing that we're talking about. We're just restricting it to this case where you have n sample values that you're taking one after the other, and they're all IID when the particular value of the hypothesis that happens to be there.

OK, so we said there is something called a likelihood ratio. The likelihood ratio for a particular sequence y is lambda of y is equal to the density of y given h1 divided by the density of y given h0. Why is it h1 on the top and h0 on the bottom? Purely convention, nothing else.

The only thing that distinguishes hypothesis 1 from hypothesis 0 is you choose one and call it 1, and you choose the other and call it 0. Doesn't make any difference how you do it. So after we make that choice, the likelihood ratio is that ratio.

Now, the reason for using semi invariant moment generating functions is that this density here is a product of densities. This density is a product of densities, and therefore when you take the log of this ratio of products, you get the sum from i equals 1 to n of this log likelihood ratio for just a single experiment. It's a single experiment that you're taking based on the fact that all n experiments are based on the same hypothesis, either h0 or h1. So the game that you're playing, and please remember what the game is if you forget everything else about this game, is the hypothesis gets chosen, and at the same time, you take n sample values. All n sample values correspond to the same value of the hypothesis.

OK, so when you do that, we're going to call z sub i, this logarithm here, this log likelihood ratio. And then we showed last time that a threshold test is-- well, we define the threshold test as comparing the sum with the logarithm of a threshold. And the threshold is equal to p0 over p sub 1, if in fact you're doing a maximum a posteriori probability test, and p0 and p1 are the probabilities of hypothesis.

Remember how we did that. It was a very simple thing. You just write out what the probability is of hypothesis 0 and a sequence of n values of y. You write out what the probability is of hypotheses 1 and that same sequence of values with the appropriate probability on that sequence for h equals 1 and h equals 0. And what you get out of that is that the threshold test sums up all the z sub i's, compares it with the threshold, and makes a choice, and that is the map choice.

OK, so conditional on h0, you're going to make an error if the sum of the z sub i's is greater than the logarithm of eta. And conditional on h1, you're going to make an error if the sum is less than or equal to log eta. I denote these as the random variable z sub i 0 to make sure that you recognize that this random variable here is conditional on h0 in this case, and it's conditional on h1 in the opposite case.

OK, so the exponential bound for z sub i sub 0-- OK, so what we're doing now is we're saying, OK, suppose that 0 is the actual value of this hypothesis. 0 is the value of the hypothesis. The experimenter doesn't know this. What the experimenter does is does what the experimenter has been told to do, namely the experimenter take these n values, y1 up to y sub n, finds the likelihood ratio, compares that likelihood ratio with the threshold, and if the threshold is larger than the threshold, it decides 1. If it's smaller than the threshold, that decides opposite thing. It decides 1 if it's above the threshold, 0 if it's below the threshold.

Well, first thing we want to do, then, is to find the log likelihood ratio under the assumption that 0 is the correct hypothesis, and something very remarkable happens here. Gamma sub 0 of r is now the logarithm because it's a semi invariant moment generating function of the expected value of this quantity of e to the r times z sub i. When we take the expected value, we integrate over f of y given h0 times e to the r times log of f of y given h1 over f of y given h0. You look at this, and what do you get? This quantity here is e to the r times log of f of y given h1.

That whole quantity in there is just f of y given h1 to the rth power. So what we have is, in this quantity here, is f of y given h0 to the minus r power. So this term combined with this term gives us f of 1 minus r of y given h0, and this quantity here is f to the r of y given h1 dy. So the semi invariant moment generating function is this quantity here. At r equals 1, this is just f of y given h1, so the log of it is equal to 0.

So what we're saying is that, for any old detection problem in the world, so long as this moment generating function exists, what happens is it starts at 0, it comes down, comes back up again, and r star is equal to 1. That's what we've just shown. When r is equal to 1, this whole thing is equal to 1, so the log of 1 is equal to 0. For every one of these problems, you know where this intercept is, you know where this intercept is, one is at 0, one is at 1.

What we're going to do now is try to find out what the probability of error is given that h is 0, h equals 0, is the correct hypothesis. So we're assuming that the probabilities are actually f of y given h0. We calculate this quantity that looks like this, and we ask what is the probability that this sum of random variables exceeds the threshold, exceeds the threshold eta. So the thing that we do is we draw a line, a slope, natural log of eta divided by eta. We draw that slope along here, and we find that the probability of error is upper bounded by gamma 0 of this quantity, defined by the slope, minus r0 times log of eta divided by eta.

That's all there is to it. Any questions about that? Seem obvious? Seem strange?

OK, so the probability of r conditional on h equals 0 is e to the n times gamma 0 of r0 minus r0, natural log of eta over eta. And ql of eta is the probability of error given that h is equal to l. OK, we can do the same thing for hypothesis 1. We're asking what's the probability of error given that h equals 1 is the correct hypothesis, and given that we choose a threshold, say we know the a priori probabilities, so we choose a threshold that way.

OK, we go through the same argument, z1 of s is the natural log of f of y given F1 times e to the s, we're using s in place of r here, times the natural log of f of y given h1 over f of y given h0. And this quantity, now, f of y given h1, the f of y given h1 is upstairs, so we have f of 1 plus s of y given h1. This quantity is down here, so we have f of minus s of y given h0. And we notice that when s is equal to minus 1, this is again equal to 0, and we notice also, if you compare this, gamma 1 of s is equal to gamma 0 of r minus 1. These two functions are the same, just shifts it by one.

OK, so this one of the very strange things about hypothesis testing, namely you are calculating these expected values, but you're calculating the expected value of a likelihood ratio. And the likelihood ratio involves the probabilities of the hypotheses also, so when you calculate that ratio, what you get this is funny quantity here, which is related to what you get when you calculate the semi invariant moment generating function given the other hypothesis. So that now, what we wind up with is a gamma 1 of the eta, is e to the n times gamma 0 of r0.

I'm using the fact that gamma 1 of s is equal to gamma 0 of r minus 1, s is just r shifted over by 1, so I can do the same optimization for each. So what I wind up with is the probability of error conditional on hypothesis 0, is this quantity down here. That's this one, and the probability of error conditional on the other hypothesis, the exponent is equal to this quantity here.

OK, so what that says is that as you shift the threshold-- in other words, suppose instead of using a map test, you say, well, I want the probability of error to be small when hypothesis 0 correct. I want it to be small when hypothesis 1 is correct. I have a trade off between those two. How do I choose my threshold in order to get the smallest value overall?

So you say, well, you're stuck. You have one exponent under hypothesis 0. You have another exponent under hypothesis 1. You have this curve here. You can take whatever value you want over here, and that sticks you with a value here.

You can rock things around this inverted seesaw, and you can make one probability of error bigger by making the other one smaller, or you make the other one bigger by making the other one smaller. Namely, what you're doing is changing the threshold, and as you change the threshold, as you make the threshold positive, what you're doing is making it harder to accept h1, h equals 1, and easier to accept h equals 0. When you move the threshold the other way, you're making it easier the other way.

This, in fact, gives you the choice between the two. You decide you're going to take n tests. You can make both of these smaller by making n bigger. But there's a trade off between the two, and the trade off is given by this tangent line to this curve here. And you're always stuck with r star equals 1 and all of these problems.

So the only question is what does this curve look like? Notice that the expected value of the likelihood ratio given h equals 0 is negative. The expected value given h equals 1 is positive, and that's just because of the form of the likelihood ratio.

OK, so this actually shows these two exponents. These are the exponents for the two kinds of errors. You can view this as a large deviation form of the Neyman Pearson test. In the Neyman Pearson test, you're doing things in a very detailed way, and you're taking a choice between choosing different thresholds to make the probability of error of one type bigger or less than the other one, just the other way.

Here, we're looking at the large deviation form of it that becomes an upper bound rather than an exact calculation, but it tells you much, much more because for most of these threshold tests, you're going to do enough experiments that your probability of error is going to be very small. So the only question is where do you really want the error probability to be small? You can make it very small one way by shifting the curve this way, and make it very small the other way by shifting the curve the other way. And you take your choice of which you want.

OK, the a priori probabilities are usually not the essential characteristic when you're dealing with this large deviation kind of result because, when you take a large number of tests, this threshold, log eta over eta over n, when n becomes very large, when you have a large number of experiments, log eta over n becomes relatively small. So that's not the thing you're usually concerned with. What you're concerned with is whether one test, the patient dies, and the other tests costs a lot of money; or one test, the nuclear plant blows up, and the other test, you waste a lot of money, which you wouldn't have had to pay otherwise.

OK, now, here's the important part of all of this. So far, it looked like there wasn't any way to get out of this trade off between choosing a threshold to make the error probability small one way, or making the error probability small the other way. And you think, well, yes, there is a way to get around it. What I should do is what I do in real life, namely if I'm trying to decide about something, what I'm normally going to do, I don't like to waste my time deciding about it, so as soon as the decision becomes relatively straightforward, I make up my mind. If the decision is not straightforward, if I don't have enough evidence, I keep doing more tests, so sequential tests are an obvious thing to try to do if you can do it.

What we have here, what we've shown, is we have two coupled random walks. Given hypothesis h equals 0, we have one random walk, and that random walk is typically going to go down. Given h equals 1, we have another random walk. That random walk is typically going to go up. And one is going to go down, one is going to go up, because we've defined the random variable involved is a log of f of y given h1 divided by f of y given h0, which is why the 1 walk goes up, and the 0 walk goes down.

Now, the thing we're going to do is do a sequential test. We're going to keep doing experiments until we cross a threshold. We're going to decide what threshold is going to give us a small enough probability of error under each condition, and then we choose that threshold. And we continue to test until we get there. So we want to find out whether we've gained anything by that, how much we've gained if we gain something by it, and so forth.

OK, when you use two thresholds, alpha's going to be bigger than 0. Beta's going to be less than 0. The expected value of z given h0 is less than 0, but the value of z given h1 is greater than 0. That's why the walks are coupled, so we can handle each of them separately until we can get the answers for one from the answers for the other. Crossing alpha is a rare event for the random walk with h0 because a random walk with h0, you're going to go down typically. You hardly ever go up. Yes?

AUDIENCE: Can you please explain again sign of expectations?

PROFESSOR: The sign of the expectations? Yes, z is the log, so that when we actually have h equals 1, the expected value of this is going to be lined up with this term on top. We have f of y. When we have h equals 0, this lined up with the term on the bottom.

I mean, actually, you have to go through and actually show that the integral of f of y given h1 of this quantity is greater than 0, and the other one is less than 0. We don't really have to do that because, if we calculate this moment generating function, we can pick it off of there. When we look at this moment generating function, that slope there is the expected value of z conditional on h equals 0, and because of the shifting property, this slope here is the expected value of z given h equals 1, just because the 1 curve is shifted from the other by one unit. It's really because of that ratio. If you defined it the other way, you just changed the sign, so nothing important would happen.

OK, so r start equals 1 for the h0 walk, so the probability of error, given h0, is less than or equal to e to the minus alpha. Well, that's a nice simple result, isn't it? In fact, that's really beautifully. You just calculate this moment generating function, you find the root of it, and you're done. You have a nice bound, and in fact, it's an exponentially tight bound.

And on the other hand, when you deal with the probability of error given h1 by symmetry, it's less than or equal to e to the beta. Beta is a negative number, remember, so this is exponentially going down as you choose beta, smaller and smaller. So the thing that we're getting is we can make each of these error probabilities as small as we want, this one, by making alpha big. We can make this one as small as we want by making beta big negative.

There must be a cost to this. OK, but what's the cost? What happens when you make alpha big?

When hypothesis 1 is the correct hypothesis, what normally happens is that this random walk is going to go up roughly at a slope of the expected value of z given h equals 0. So when you make alpha very, very large, you're forced to make a very large number of tests when h is equal to 1. When you make beta very, very large, you're forced to take a large number of tests when h is equal to 0.

So the trade off here is a little bit funny. You make your error probability for h equals 0 very, very small by costing more money when hypotheses 1 is the correct hypothesis because you don't make a decision until you've really climb way up on this random walk. And that means it takes a long time when you have h equals 1. Since when h is equal to 1, the probability of crossing this lower threshold is it is almost negligible, this expected time that it takes is really just a function of h equals 1. I'm going to show that in the next slide.

When you increase alpha, it lowers the probability of error given h equals 0. Excuse me, I should have h equals 0 instead of h sub 0. Exponentially, it increases the expected number of steps until you make a decision given h1. Expected value of j given h1 is effectively equal to alpha divided by expected value of z given h1.

Why is that? That's essentially Wald's equality. Not Wald's identity, but Wald's equality because-- Yes, it says from Wald's equality, since alpha is essentially equal to the expected value of s of j given h equals 1, the number of testing you have to take when h is equal to 1, when alpha is very, very large, is effectively the amount of time that it takes you to get up to the point alpha. That expected amount of time is typically pretty close to the mean value.

So alpha there is close to the expected value of s of j given h equals 1. So Wald's equality, given h equals 1, says the expected value of j given h1 is equal to the expected value of sj given h equals 1, that's alpha, divided by the expected value of z given h1, which is just the underlying likelihood ratio. So to get this result, we just substitute alpha for the expected value. And then the probability of error, given h equals 0, if we write it this way, we see the cost immediately. That's the expected value of j given h equal to 1.

In other words, the expected number of tests given h equals 1 times the expected value of the log likelihood ratio given h equals 1. When you decrease beta, that lowers the probability of error given h1 exponentially, but it increases the number of tests when h0 is the correct hypothesis. So in that case, you get the probability of error given h equals 1 is effectively equal to the expected value e to the expected value of j equals j equals 0. This is just the number of tests you have to do when h is equal to 0. This is the expected value of the log likelihood ratio when h is equal to 0.

This is very approximate, but this is how you would actually choose how big you make alpha, how big do you make beta if you want to do a test between these two hypotheses. Now, this shows what you're gaining by the sequential test over what you're gaining by the non-sequential test. You don't have this in your notes, so you might just jot it down quickly.

The expected value of z, conditional on h equals 0, is this slope here, the slope of the moment generating function is z equals 0. That's the slope of the underlying random variable. Since this point is r equal to 1, this point down here is the expected value of z given h equals 0. That's the exponents that you get when h equals 0 is, in fact, the correct exponent. When given the probability of error given that h is equal to 0, namely the probability that you choose hypothesis 1.

Same way over here. This slope here is the expected value of the log likelihood ratio given h equals 1. This hits down here at minus expected value of z given h equals 1. So you have this exponent going one way, you have this exponent going the other way when the thing multiplying the exponent is not an absolute value but is, in fact, the number of tests you have to do than the other test.

Now, if we do the fix test, what we're fixed with is a test where you take a line tangent to this curve, which goes from here across here to there. We can see-saw it around. When we see-saw it all the way in the limit, we can get this result here. But we get this result here at the cost of an error, which is almost one in the other case, so that's not a very good deal.

This says that sequential testing, well, it shows you how much you gain by doing a sequential test. I mean, it might not be intuitively obvious why this is happening. I mean, really the reason it's happening is that the times when you want to make the test very long are those times when if h is equal to 0, you normally go down.

The next most normal thing is you wobble around without doing anything for a long time, in which case you want to keep doing additional tests until finally it falls down, or finally it goes up. But by taking additional tests, you make it very unlikely that you're ever going to cross that threshold. So that's the thing you're gaining. You are gaining the fact that the error is small in those situations where the sum of these random variables stays close to 0 for a long time, and then you don't make errors in those cases.

We now have just a little bit of time to prove Wald's identity. I don't want to have a lot of time to prove it because proofs of theorems are things you really have to look at yourselves. This one, you almost don't have to look at it. This one is almost obvious as soon as you understand what a tilted probability is.

So let's suppose that x sub n is a sequence of IID discrete random variables. It has a moment generating function for some given r. We're going to assume that these random variables are discrete now to make this argument simple. If they're not discrete, this whole argument has to be replaced with all sorts of [INAUDIBLE] integrals and all of that stuff. It's exactly the same idea, but it just is messy mathematically.

So what we're going to do is we're going to define a tilted random variable. A tilted random variable is a random variable in a different probability space. OK, we start out with this probability space that we're interested in, and then we say, OK, suppose that we, just to satisfy our imaginations, we suppose the probabilities are different. We assume that the probabilities for a given r is the probability that the random variable X is equal to little x, namely this quantity here, is equal to the original probability that X is equal to little x.

All the sample values are the same, it's just the probability's changed, times e to the rx minus gamma of r. So we're taking these probabilities when X is large. We're magnifying them when x is small. We're knocking them down.

What's the purpose of this? It's just a normalization factor. e to the minus gamma of r is 1 over the moment generating function of r, so you take p of x, e to the rx, divide it by g of r. So this is a probability mass function, as well as this. This is the correct probability mass function for the model you're looking at.

This is an imaginary one, but you can always imagine. You can say let's suppose that we had this model instead of the other model. All the sample values are the same, but the probabilities are different. So we want to see what we can find out from these different probabilities in this different probability model.

If you sum over x here, this sum is equal to 1, as we just said. So we'll view q sub xr of x as the probability mass function on x in a new probability space. We can use all the laws of probability in this new space, and that's exactly what we're going to do. And we're going to say things about the new space, but then we can always come back to the old space from this formula here because whatever we find out in the new space will work in the old space.

One thing we'd like to do is to be able to find the expected value of the random variable x in this new probability space, so this isn't the expected value in the old space. It's a probability in the new space. It's the sum over x of x times q sub xr of x. That's what the expected value is.

X is the same in both spaces. That's just the probabilities that have changed. These are p of x times z to the rx minus gamma of r, so when you sum this, what you get is 1 over g of xr, which is that term, times the derivative of p sub x of x, e to the rx. When you take this derivative, then you get an x in front, which is that x there.

So you get g prime of xr over gx of r, which is gamma prime of r. OK, so in terms of that graph we've drawn, when you take these tilted probabilities, you move that slope, that r equals 0, and now you're looking at a slope at whatever r you're looking at. And that gives you the expected value there.

OK, if you have a joint tilted probability mass function-- and don't think it gets any more complicated. It doesn't. I mean, you've already gone through the major complication of this argument.

The joint tilted PMF is the probability of x1 to xn is the old probability of x1 to xn times all of these tilted factors here. If you let a of sn be the set of n tuples which have the same sum, then all these terms become r times s sub n. So what you get is that for each xn for which the sum is sn, this tilted probability becomes the old probability times e to the r sn minus n gamma of r, which says that when we look at the tilted probability of the sum, namely we said that when we tilt these probabilities, we can do everything in a new space that we could do in the old space. We can do everything that probability theory allows us to do, so we can look at the probability of s sub n in the new space also. The probability of sn in the old space, namely we're summing this quantity, overall xn in a of sn, so we sum up all of those as the probability sub s sub n at sn times this quantity, which is fixed.

So this is the key to a lot of large deviation theory. Any time you're dealing with a difficult problem, and you want to see what's happening way, way away from the mean, you want to see what these sums look like for these exceptional cases, what we do is we look at a new model where we tilt the probability so that the region of concern becomes the main region for that tilted model. So for r equals 0, we're tilting the probability towards large values, and you can use the law of large numbers, essential limit theorem, whatever you want to, in that new space, then.

Now, we can prove Wald's equality. What Wald's identity is is the statement that when you tilt these probabilities, a stopping rule in this tilted world is still the stopping time is still a random variable, namely you still stop with probability 1. Somebody questioned whether you stop with probability 1 in the old world. Like I said, you do because you have this positive variance, and the thing with two thresholds keeps growing and growing. Here, you have the same thing.

I mean, the mean doesn't make any difference at all. I mean, you're looking at trying to exceed one of two different thresholds, and eventually, you exceed one of them no matter where you set r. So what this is saying is the probability that j is equal to n in this tilted space is equal to the probability that j is equal to n in the old space times z to the r sn minus gamma of r. So this quantity is equal to the expected value of e to the r sn minus gamma of r. Given j equals n times the probability that j is equal to n, you sum this over n and, bingo, you're back at the Wald identity.

So that's all the Wald identity is, is just a statement that when you tilt a probability, and you have a stopping rule on the original probabilities, you then have a stopping rule on the new probabilities. And Wald's identity says-- well, Wald's identity holds whenever that tilted stopping rule is a random variable. OK, that's it for today. We will do martingales on Wednesday.