Lecture 16: Review; Introduction to Detection

• Download English-US transcript (PDF)

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.

PROFESSOR: OK. I want to review zero-mean jointly Gaussian random variables. And I want to review a couple of the other things I did last time. Because when I get into questions of stationarity and things like that today, I think it will be helpful to have a little better sense of what all that was about, or it will just look like a big mess. And there are sort of a small number of critical things there that we have to understand.

One of them is that if you have a non-singular covariance matrix, OK, you have a bunch of random variables. This bunch of random variables, each one of them has a has a covariance. Namely, z sub i and z sub j have an expected value of z sub i and z sub j. I'm just talking about zero mean here. Anytime I don't say whether there's a mean or not, I mean there isn't a mean. But I think the notes are usually pretty careful.

If the covariance matrix is non-singular, then the following things happen. The vector, z, is jointly Gaussian. And our first definition was, if you can represent it as a -- all the components of it each as linear combinations of IID normal Gaussian random variables, namely independent Gaussian random variables -- so they're all just built up from this common set of independent Gaussian random variables.

Which sort of matches our idea of why noise should be Gaussian. Namely, it comes from a collection of a whole bunch of independent things, which all get added up in various ways. And we're trying to look at processes, noise processes. The thing that's going to happen then is that, at each epoch in time, the noise is going to be some linear combination of all of these noise effects, but is going to be filtered a little bit. And because it's being filtered a little bit, the noise at one time and the noise at another time are all going to depend on common sets of variables. So that's the intuitive idea here.

Starting with that idea, last time we derived what the probability density had to be. And if you remember, this probability density came out rather simply. The only thing that was involved here was this idea of going from little cubes in the IID noise domain into parallelograms in the z domain. Which is what happens when you go through a linear transformation. And because of that, this is the density that has to come out of there. By knowing a little bit about matrices -- which is covered in the appendix, or you can just take it on faith if you want to -- you can break up this matrix into eigenvalues and eigenvectors.

If the matrix is nonsingular -- well, if the matrix is singular or nonsingular, if it's a covariance matrix, or it's -- a non-negative definite matrix, is what these matrices are -- then you can always break this matrix up into eigenvalues and eigenvectors. The eigenvectors span the space. The eigenvectors can be taken to be orthonormal to each other.

And then when you express this in those terms, this probability density becomes just a product of normal Gaussian random variable densities, where the particular Gaussian random variables are these inner products of the noise vector with these various eigenvectors. In other words, you're taking this set of random variables which you're expressing in some reference system, where z1 up to z sub k are the different components to the vector. When you express this in a different reference system, you're rotating that space around. What this says is, you can always rotate the space in such a way that what you will find is orthogonal random variables, which are independent of each other. So that this is the density that you wind up having.

And what this means is, if you look at the region of equal probability density over this set of random variables, what you find is it's a bunch of ellipsoids. And the axes of the ellipsoids are simply these eigenvectors here. OK, so this is the third way. If you can represent the noise in this way, again, it has to be jointly Gaussian.

Finally, if all linear combinations of this random vector are Gaussian, that's probably the simplest one. But it's the hardest one to verify, in a sense. And it's the hardest one to get all these other results from. But if all linear combinations of these random variables are all Gaussian, then in fact, again, the variables have to be jointly Gaussian.

Again, it's important to understand that jointly Gaussian means more than just individually Gaussian. It doesn't mean what you would think from the words jointly Gaussian, as saying that each of the variables are Gaussian. It means a whole lot more than that. In the problem set, what you've done in one of the problems is to create a couple of examples of two random variables which are each Gaussian random variables, but which jointly are not jointly Gaussian.

OK, finally, if you have a singular covariance matrix, z is jointly Gaussian if a basis of these zi's are jointly Gaussian. OK? In other words, you throw out the random variables which are just linear combinations of the others, because you don't even want to think of those as random variables in a sense. I mean, technically they are random variables, because they're defined in the sample space. But they're all just defined in terms of other things, so who cares about them? So after you throw those out, the others have to be jointly Gaussian.

So in fact, if you have two random variables, z1 and z2, and z2 equals z1 -- OK, in other words the probability density is on a straight diagonal line -- and z1 is Gaussian, z1 and z2 are jointly Gaussian in that case. This is not an example like the ones you did in this problem set that you're handing in today, where you in fact have things like a probability density which doesn't exist because it's impulsive on this line, and also impulsive on this line. Or something which looks Gaussian in two of the quadrants and is zero in both the other quadrants, and really bizarre things like that. OK?

So please try to understand what jointly Gaussian means. Because everything about noise that we do is based on that. It really is. And if you don't know what jointly Gaussian means, you're not going to understand anything about noise detection, or anything from this point on. I know last year I kept telling people that, and in the final exam there were still four or five people who didn't have the foggiest idea what jointly Gaussian meant. And you know, you're not going to understand the rest of this, you're not going to understand why these things are happening, if you don't understand that. OK.

So the next thing we said is Z of t -- I use those little curly brackets around something just as a shorthand way of saying that what I'm interested in now is the random process, not the random variable at a particular value of t. In other words, if I say Z of t, what I'm usually talking about is a random variable, which is the random variable corresponding to one particular epoch of this random process. Here I'm talking about the whole process. And it's a Gaussian process. If Z of t1 to Z of tk are jointly Gaussian for all k and for all sets of t sub i.

And if the process can be represented as a linear combination of just ordinary random variables multiplied by some set of orthonormal functions, and these Z sub i are independent, and if the sum of the variances of these random variables sum up to something less than infinity. So you have finite energy in these sample functions, effectively is what this says. Then it says the sample functions of Z of t are L2 with probability one.

OK. You have almost proven that in the problem set. If you don't believe that you've almost proven it, you really have. And anyway, it's true. One of the things that we're trying to do when we're trying to deal with these wave forms that we transmit, the only way we can deal with them very carefully is to know that they're L2 functions. Which means they have Fourier transforms. We can do all this stuff we've been doing. And you don't need anything more mathematically than that. So it's important to be dealing with L2 functions.

We're now getting into random processes, and it's important to know that the sample functions are L2. Because so long as the sample functions are L2, then you can do all of these things we've done before and just put it together and say well, you take a big ensemble of these things. They're all well defined, they're all L2. We can take Fourier transforms, we can do everything else we want to do. OK, so that's the game we're playing.

OK, I'm just going to assume that sample functions are L2 from now on, except in a couple of bizarre cases that we look at. Then a linear functional is a random variable given by the random variable is equal to the integral of the noise process z of t, multiplied by an ordinary function, g of t, dt. And we talked about that a lot last time. This is the convolution of a process -- well, it's not the convolution. It's just the inner product of a process with a function. And we interpret this last time in terms of the sample functions of the process and the sample values of the random variable. And then since we could do that, we could really talk about the random variable here.

This means that for all of the sample values in the sample space with probability one, the sample values of the random variable v are equal to the integral of the sample values of the process times g of t dt. OK, in other words, this isn't really something unusual and new.

I mean, students have the capacity of looking at this in two ways, and I've seen it happen for years. The first time you see this, you say this is trivial. Because it just looks like the kind of integration you're doing all your life. You work with it for a while, and then at some point you wake up and you say, oh, but my god, this is not a function here. This is a random process. And you say, what the heck does this mean? And suddenly you're way out in left field. Well, this says what it means. Once you know what it means, we go back to this and we use this from now on. OK?

If we have a zero-mean Gaussian process z of t, if you have L2 sample functions, like you had when you take a process and make it up as a linear combination of random variables times orthonormal functions, and if you have a bunch of different L2 functions, g1 up to g sub j 0, then each of these random variables, each of these linear functionals, are going to be Gaussian. And in fact the whole set of them together are jointly Gaussian. And we showed that last time. I'm not going to bother to show it again.

And we also found what the covariance was between these random variables. And it was just this expression here. OK. And this follows just by taking vi, which is this kind of integral, times vj, and interchanging the order of integration, taking the expected value of z of t times z of tau, which gives you this quantity. And you have the g i of t and the g j of t stuck in there with an integral around it. And it's hard to understand what those integrals mean and whether they really exist or not, but we'll get to that a little later.

We then talked about linear filtering of processes. You have a process coming into a filter, and the output is some other stochastic process. Or at least we hope it's a well defined stochastic process. And we talked a little bit about this last time. And for every value of tau, namely for each random variable in this output random process, V of tau is just a linear functional. It's a linear functional corresponding to the particular function h of that tau minus t. So the linear functional is a function of t here for the particular value of tau that you have over here. So this is just a linear functional like the ones we've been talking about.

OK, if z of t is a zero-mean Gaussian process, then you have a bunch of different linear functionals here for any set of times tau 1 up to tau sub k. And those are jointly Gaussian from what we just said. And if all of these sets of k sample -- If based a random process, v of tau, is a Gaussian random process, if for all k sets of epochs, v, tau 1, tau 2, up to tau sub k, if this set of random variables are all jointly Gaussian. So that's what we have here. So the V of tau is actually a Gaussian process if Z of t is a Gaussian random process to start with. And the covariance function is just this quantity that we talked about before.

OK, so we have a covariance function. We also have a covariance function for the process we started with. And if it's Gaussian, all you need to know is what the covariance function is. So that's all rather nice.

OK, as we said, we're going to start talking about stationarity today. I really want to talk about two ideas of stationarity. One is the idea that you have probably seen as undergraduates one place or another, which is simple computationally, but is almost impossible to understand when you try to say something precise about it. And the other is something called effectively stationary, which we're going to talk about today. And I'll show you why that makes sense.

So we say that a process Z of t is stationary if, for all integers k, all shifts tau, all epochs t1 to t sub k, and all of values of z1 to zk, this joint density here is equal to the joint density shifted. In other words, what we're doing is we're taking a set of times over here, t1, t2, t3, t4, t5. We're looking at the joint distribution function for those random variables here, and then we're shifting it by tau. And we're saying if the process is stationery, the joint distribution function here has to be the same as the joint distribution function there.

You might think that all you need is the distribution function here has to be the same as the distribution function there. But you really want the same relationship to hold through here as there if you want to call process stationary. OK. If we have zero-mean Gauss, that's just equivalent to saying that the covariance function at ti and t sub j is the same as the covariance function at ti plus tau and tj plus tau. And that's true for t1 up to t sub k. Which really just means this has to be true for all tau, for all t sub i and for all t sub j. So you don't need to worry about the k at all once you're dealing with a Gaussian process. Because all you need to worry about is the covariance function. And the covariance function is only a function of two variables, the process at one epoch and the process at another epoch.

And this is equivalent, even more simply, to saying that the covariance function at t1 and t2 has to be equal to the covariance function at t1 minus t2 and zero. Can you see why that is? If I start out with this, and I know that this is true, then the thing that I can do is take this and I can shift it by any amount that I want to. So if I shift this by adding tau here, then what I wind up with is kz of t1 minus t2 plus tau, comma tau, is equal to kz of t1 plus tau, t2 plus tau. And if I change variables around a little bit, I come to this.

So this is the condition we need for a Gaussian process to be stationary. I would defy any of you to ever show in any way that a process satisfies all of these conditions. I mean, if you don't have nice structural properties in the process, like a Gaussian process, which says that all you need to define it is this, this is something that you just can't deal with very well.

So we have this. And then we say, OK, this covariance is so easy to work with -- I mean no, it's not easy to work with, but it's one hell of a lot easier to work with than that. And therefore if you want to start talking about processes, and you don't really want to go into all the detail of these joint distribution functions, you will say to yourselves that one thing I might ask for in a process is the question of whether the covariance function satisfies this property. I mean, for a Gaussian process this is all you need to make the process stationary. For other processes you need more, but at least it's an interesting question to ask. Is the process partly stationary, in the sense that the covariance function at least is stationary? Namely, the covariance function here looks the same as the covariance function there.

So we say that a zero-mean process is wide-sense stationary if it satisfies this condition. So a Gaussian process then is stationary if and only if it's wide-sense stationary.

And a random process with a mean, or with a mean that isn't necessarily zero, is going to be stationary or wide-sense stationary if the mean is constant and the fluctuation is stationary. Or wide-sense stationary, as the case may be. So you want both of these properties there. So as before, we're just going to throw out the mean and not worry about it. Because if we're thinking of it as noise, it's not going to have a mean. Because of it has a mean, it's not part of the noise. It's just something we know, and we might as well remove it.

OK, interesting example here. Let's look at a process, V of t, which is defined in this way here. It's the sum of a set of random variables, V sub k, times the sinc function for some sampling interval, capital T. And I'm assuming that the v sub k are zero-mean, and that, at least as far as second moments are concerned, they have this stationarity property between them. Namely, they are uncorrelated from one j to another k. Expected value of vj squared is sigma squared. Expected value of vj vk for k unequal to j is zero. So they're uncorrelated; they all have the same variance.

Then I claim that the process V of t is wide-sense stationary. And I claim that this covariance function is going to be sigma squared times sinc of t minus tau over t. Now for how many of you is this an obvious consequence of that? How many of you would guess this if you had to guess something? Well, if you wouldn't guess it, you should. Any time you don't know something, the best thing to do is to make a guess and try to see whether your guess is right or not. Because otherwise you never know what to do.

So if you have a function which is defined in this way -- think of it this way. Suppose these V sub k's were not random variables, but instead suppose they were all just constants. Suppose they were all 1. Suppose you're looking at the function V of t, which is the sum of all of the sinc functions. And think of what you'd get when you add up a bunch of sinc functions which are all exactly the same. What do you get? I mean, you have to get a constant, right? You're just interpolating between all these points, and all the points are the same. So it'd be very bizarre if you didn't get something which was constant.

Well the same thing happens here. The derivation of this is one of those derivations which is very, very simple and very, very slick and very elegant. And I was going to go through it in class and I thought, no. You just can't follow this in real time. It's something you have to sit down and think about for five minutes. So I urge you all to read this, because this is a trick that you need to use all the time. And you should understand it, because various problems as we go along are going to use this idea in various ways.

But anyway, when you have a process defined this way, it has to be wide-sense stationary. And the covariance function is just this function here. Now if these V sub k up here are jointly Gaussian, and they all have the same variance, so they're all IID, then what this means is you have a Gaussian random process. And it's a stationary Gaussian random process. In other words, it's a process where, if you look at the random variable v of tau at any given tau, you get the same thing for all tau.

In other words, it's a little peculiar, in the sense that when you look at this, it looks like time 0 was a little special. Because time 0 is where you specified the process. Time t you specified the process. Time 2t you specified the process. But in fact this is saying, no. Time 0 is not special at all. Which is why we say that the process is wide-sense stationary. All times look the same.

On the other hand, if you take these V sub k here to be binary random variables which take one the value plus 1 or minus 1 with equal probability, what do you have then? You look at the sum here, and at the sample points -- namely at 0, at t, at 2t, and so forth -- what values can the process take away? At time 0? If v sub 0 is either plus 1 or minus 1, what is v of 0 over here? It's plus 1 or minus 1. It can only be those two things.

If you look at V of capital T over 2, namely if you look halfway between these two sample points, you then ask, what are the different values that v of T over 2 can take on? It's an awful mess. It's a very bizarre random variable. But anyway, it's not a random variable which is plus or minus 1. Because it's really a sum of an infinite number of terms. So you're taking an infinite number of binary random variables, each with arbitrary multipliers. So you're adding them all up. So you get something that's not differentiable. It's not anything nice.

I don't care about that. The only thing that I care about is that v of capital T over 2 is not a binary random variable anymore. And therefore this process is not stationary anymore. Here's an example of a wide-sense stationary process which is not stationary. So it's a nice example of where you have wide-sense stationarity, but you don't have stationarity.

So all kinds of questions about power and things like that, this process works very, very well. Because questions about power you answer only in terms of covariance functions. Questions of individual possible values and probability distributions you can't answer the very well.

One more thing. If these variables are Gaussian, and if you actually believe me that this is a stationary Gaussian process, and it really is a stationary Gaussian process, what we have is a way of creating a broad category of random processes. Because if I look at the sample functions here of this process, each sample function is bandlimited. Baseband limited. And it's baseband limited to 1 over 2 times capital T. 1 over the quantity 2 times capital T. So by choosing capital T to be different things, I can make these sample functions have large bandwidth or small bandwidth so I can look at a large variety of different things. All of them have Fourier transforms, which look sort of flat in magnitude. And we'll talk about that as we go on.

And when I pass these things through linear filters, what we're going to find is we can create any old Gaussian random process we want to. So that's why they're nice. Yes?

AUDIENCE: [INAUDIBLE]

PROFESSOR: Can we make V of t white noise? In a practical sense, yes. In a mathematical sense, no. In a mathematical sense, you can't make anything white noise. In a mathematical sense, white noise is something which does not exist. I'm going to get to that later today. And it's a good question. But the answer is yes and no.

OK, the trouble with stationary processes is that the sample functions aren't L2. That's not the serious problem with them, because all of us as engineers are willing to say, well, whether it's L2 or not I don't care. I'm just going to use it because it looks like a nice random process. It's not going to burn anything else or anything. So, so what? But the serious problem here is that it's very difficult to view stationary processes as approximations to real processes. I mean, we've already said that you can't have a random process which is running merrily away before the Big Bang. And you can't have something that's going to keep on running along merrily after we destroy ourselves, which is probably sooner in the future than the Big Bang was in the past.

But anyway, this definition of stationarity does not give us any way to approximate this. Namely, if a process is stationary, it is either identically zero, or every sample function with probability one has infinite energy in it. In other words, they keep running on forever. They keep building up energy as they go. And as far as the definitions current is concerned, there's no way to say large negative times and large positive times are unimportant. If you're going to use mathematical things, though you're using them as approximations, you really have to consider what they're approximations to. So that's why I'm going to develop this idea of effectively wide-sense stationary or effectively stationary.

OK. So a zero-mean process is effectively stationary or effectively wide-sense stationary, which is what I'm primarily interested in, within two intervals, within some wide interval of time, minus T0 to plus T0. I'm thinking here of choosing T0 to be enormous. Namely if you build a piece of equipment, you would have minus T0 to plus T0 include the amount of time that the equipment was running.

So we'll say it's effectively stationary within these limits if the joint probability assignment, or the covariance matrix if we're talking about effectively wide-sense stationary, for t1 up to t sub k is the same as that for t1 plus tau up to tk plus tau. In other words I have this big interval, and I have a bunch of times here. t sub k. And I have a bunch of times over here, t1 plus tau up to tk plus tau. And this set of times don't have to be disjoint from that set of times. And I have this time over here, minus t0, and I have this time way out here which is plus t0. And what I'm saying is, I'm going to call this function wide-sense stationery if, when we truncate it to minus t0 to plus t0, it is stationary as far as all those times are concerned. In other words, we just want to ignore what happens before minus t0 and what happens after plus t0.

You have to be able to do that. Because if you can't talk about the process in that way, you can't talk about the process at all. The only thing you can do is view the noise over some finite interval.

OK. And as far as covariance matrix for wide-sense and stationary, well it's the same definition. So we're going to truncate the process and deal with that.

For effectively stationary or effectively wide-sense stationary, I want to view the process as being truncated to minus t0 to plus t0. We have this process which might or might not be stationary. I just truncate it and I say I'm only going to look at this finite segment of it. I'm going to define this one variable covariance function as kz of t1 and t2. And kz of t1 and t2 is going to be -- blah. The single variable covariance function is defined as kz if t1 minus t2 is equal to the actual covariance function evaluated with one argument at t1 and the other argument at t2. And I want this to hold true for all t1 and all t2 in this interval.

And this square here gives a picture of what we're talking about. If you look at the square in the notes, unfortunately all the diagonal lines do not appear because of the bizarre characteristics of LaTeX. LaTeX is better than most programs, but the graphics in it are awful. But anyway, here it is drawn properly. And along this line, t minus tau is constant. So this is the line over which we're insisting that kz of t1 and t2 be constant. So for a random process to be wide-sense stationary, what were insisting is that within these limits, minus t0 to plus t0, the covariance function is constant along each of these lines along here. So it doesn't depend on both t1 and t2. It only depends on the difference between them.

Which is what we require for stationary to start with. If you don't like the idea of effectively wide-sense stationary, just truncate the process and say, well if it's stationary, this has to be satisfied. It has to be constant along these lines.

The peculiar thing about this is that the single variable covariance function does not run from minus t0 to plus t0. It runs from minus 2T0 to plus 2T0. So that's a little bizarre and a little unpleasant. And it's unfortunately the way things are. And it also says that this one variable covariance function is not always the same as the covariance evaluated at t minus tau and 0. Namely, it's not the same because t minus tau might be considerably larger than t0. And therefore, this covariance is zero, if we truncated the process, and this covariance is not zero.

In other words, for these points up here, and in fact this point in particular -- well, I guess the best thing to look at is points along here, for example. Points along here, what we insist on is that t minus tau, k sub z of t and tau is constant along this line. But we don't insist on kz of t minus tau, which is some quantity bigger than t0 along this line, being the same as this quantity here.

OK, so aside from that little peculiarity, this is the same thing that you would expect from just taking a function and truncating it. There's nothing else that's peculiar going on there.

OK, so let's see if we can do anything with this idea. And the first thing we want to do is to say, how about these linear functionals we've been talking about? Why do I keep talking about linear functionals and filtering? Because any time you take a noise process and you receive it, you're going to start filtering it. You're going to take start taking linear functionals of it. Namely all the processing you do is going to start with finding random variables in this particular way. And when you look at the covariance between two of them, what you find is the same thing we found last time. Expected value of Vi times Vj is equal to the integral of gi of t times the covariance function evaluated at t and tau, times gj of tau.

All of this is for real processes and for real functions. Because noise random processes are in fact real. I keep trying to alternate between making all of this complex and making it real. Both have their advantages. But at least in the present version of the notes, everything is real, concerned with random processes.

So I have this function here. If gj of t is zero for t greater than T0, what's going to happen here then? I'm evaluating this where this is a double integral over what region? It's a double integral over the region where t is constrained to minus T0 to plus T0, and tau is constrained to the interval minus t0 to plus T0. In other words, I can evaluate this expected value, this covariance, simply by looking at this box here. If I know what the random process is doing within this box, I can evaluate this thing. So that if the process is effectively stationary within minus t0 to plus t0, I don't need to know anything else to evaluate all of these linear functionals.

But that's exactly the way that were choosing this quantity, T0. Namely, we're choosing T0 to be so large that everything we're interested in happens inside of there. So we're saying that all of these linear functionals for effective stationarity are just defined for what happens inside this interval.

So if Z of t is effectively wide-sense stationary within this interval, you make the intervals large as you want or as small as you want. The real quantity is that these functions g have to be constrained within minus T0 to plus T0. Then Vi, Vj, are jointly Gaussian. In other words, you can talk about jointly Gaussian without talking at all about whether the process is really stationary or not. You can evaluate this for everything within these limits, strictly in terms of what's going on within those limits.

That one was easy. The next one is a little harder. We have our linear filter now. We have the noise process going into a linear filter. And we have some kind of process coming out of the filter. Again, we would like to say we couldn't care less about what this process is doing outside of these humongous limits. And the way we do that is to say -- well, let's forget about that for the time being. Let us do what you did as an undergraduate, before you acquired wisdom, and say let's just integrate and not worry at all about changing orders of integration or anything else. Let's just run along and do everything as carelessly as we want to.

The covariance function for what comes out of this filter is something we already evaluated. Namely, what you want to look at is v of t and v of tau. v of t is this integral here. v of tau is this integral here with tau substituted for t. So when you put both of these together and you take the expected value, and then you bring the expected value in through the integrals -- who knows whether you can do that or not, but we'll do it anyway -- what we're going to get is a double integral of the impulse response, evaluated at t minus t1, times the covariance function evaluated at t1 and t2, times the function that we're dealing with, h, evaluated at tau minus t2.

This is what you got just by taking the expected value of v of t times v of tau. And you just write it out. You write out the expected value of these two integrals. It's what we did last time. It's what's in the notes if you don't see how to do it. You take the expected value inside, and this is what you get.

Well now we start playing these games. And the first game to play is, we would like to use our notion of stationarity. So in place of kz of t1 and t2, we want to substitute the single variable form, kz tilde of t1 minus t2. But we don't like the t1 minus t2 in there, so we substitute phi for t1 and t2. And then what we get is the double integral of kz of phi now. And we have gotten rid of t1 by this substitution, so we wind up with the integral over phi and over t2.

Next thing we want to do, we want to get rid of the t2 also. So we're going to let mu equal t2 minus tau. When we were starting here, we had four variables, t and tau and t1 and t2. We're not getting rid of the t and we're not getting rid of the tau, because that's what we're trying to calculate. But we can play around with the t1 and t2 as much as we want, and we can substitute variables of integration here. So mu is going to be t2 minus tau. That's going to let us get rid of the t2. And we wind up with this form here when we do that substitution. It's just that h of tau minus t2 becomes h of minus mu instead of h of plus mu.

I don't know what that means. I don't care what it means. The only thing I'm interested in now is, you look at where the t's and the taus are, and the only place that t's and taus occur are here. And they occur together. And the only thing this is a function of is t minus tau. I don't know what kind of function it is, but the only thing we're dealing with is t minus tau. I don't know whether these intervals exists or not. I can't make any very good argument that they can. But if they do exist, it's a function only of t minus tau. So aside from the pseudo-mathematics we've been using, V of t is wide-sense stationary. OK?

Now if you didn't follow all this integration, don't worry about it. It's all just substituting integrals and integrating away. And it's something you can do. Some people just take five minutes to do it, some people 10 minutes to do it.

Now we want to put the effective stationarity in it. Because again, we don't know what this means. We can't interpret what it means in terms of stationarity. So we're going to assume that our filter, h of t, has a finite duration impulse response. So I'm going to assume that it's zero when the magnitude of t is greater than A. Again, I'm assuming a filter which starts before zero and runs until after zero. Because again, what I'm assuming is the receiver timing is different from the transmitter timing. And I've set the receiver timing at some convenient place. So my filter starts doing things before anything hits it, just because of this change of time.

So v of t, then, is equal to the integral of the process Z of t1 times h of t minus t1. This is a linear functional again. This depends only on what Z of t1 is for t minus A less than or equal to t1, less than or equal to t plus A. Because I'm doing this integration here, this function here is 0 outside of that interval. I'm assuming that h of t is equal to 0 outside of this finite interval. And this is just a shift of t1 on it. So this integral is going to be equal to zero from when t is equal to t1 minus A to when t is equal to t1 plus A. Every place else, this is equal to zero.

So V of t, the whole process depends -- and if we only want to evaluate it within minus T0 plus A to T0 minus A -- in other words we look at 10 to the eighth years and we subtract off a microsecond from that region. So now we're saying we want to see whether this is wide-sense stationary within 10 to the eighth years minus a microsecond, minus that to plus that. So that's what we're dealing with here.

V of t in this region depends only on Z of t for t in the interval minus T0 to plus T0. In other words, when you're calculating V of t for any time in this interval, you're only interested in Z of t, which is diddling within plus or minus A of that. Because that's the only place where the filter is doing anything.

So V of t depends only on Z of t. And here, I'm going to assume that Z of t is wide-sense stationary within those limits. And therefore I don't have to worry about what Z of t is doing outside of those limits. So if the sample functions of Z of t are L2 within minus T0 to plus T0, then the sample functions of V of t are L2 within minus T0 plus A to T0 minus A.

Now this is not obvious. And there's a proof in the notes of this, which goes through all this L1 stuff and L2 stuff that you've been struggling with. It's not a difficult proof, but if you don't dig L2 theory, be my guest and don't worry about it. If you do want to really understand exactly what's going on here, be my guest and do worry about it, and go through it. But it's not really an awful argument. But this is what happens.

So what this says is we can now view wide-sense stationarity and stationarity, at least for Gaussian processes, as a limit of effectively wide-sense stationary processes. In other words, so long as I deal with filters whose impulse response is constrained in time, the only effect of that filtering is to reduce the interval over which the process is wide-sense stationary by the quantity A. And the quantity A is going to be very much smaller than T0, because we're going to choose T0 to be as large as we want it to be. Namely, that's always the thing that we do when we try to go through limiting arguments.

So what we're saying here is, by using effective stationarity, we have managed to find a tool that lets us look at stationary as a limit of effectively stationary processes as you let T0 become larger and larger. And we have our cake and we eat it too, here. Because at this point, all these functions we're dealing with, we can assume that they're L2. Because they're L2 for every T0 that we want to look at. So we don't have to worry about that anymore.

So we have these processes. Now we will just call them stationary or wide-sense stationary. Because we know that what we're really talking about is a limit as T0 goes to infinity of things that make perfect sense. So suddenly the mystery, or the pseudo-mystery has been taken out of this, I hope.

So you have a wide-sense stationary process with a covariance now of k sub z of tau. In other words, it's effectively stationary over some very large interval, and I now have this covariance function, which is a function of one variable. I want to define the spectral density of this process as a Fourier transform of k sub z. In other words, as soon as I get a one variable covariance function, I can talk about its Fourier transform. At least assuming that it's L2 or something. And let's forget about that for the time being, because that all works.

We want to take the Fourier transform of that. I'm going to call that spectral density. Now if I call that spectral density, the thing you ought to want to know is, why am I calling that spectral density? What does it have to do with anything you might be interested in? Well, we look at it and we say well, what might it have to do with anything?

Well, the first thing we're going to try is to say, what happens to these linear functionals we've been talking about? We have a linear functional, V. It's an integral of g of t times Z of t dt. We would like to be able to talk about the expected value of V squared. Talk about zero-mean things, so we don't care about the mean at zero. So we'll just talk about the variance of any linear functional of this form. If I can talk about the variance of any linear functional for any g of t that I'm interested in, I can certainly say a lot about what this process is doing.

So I want to find that variance. If I write this out, it's same thing we've been writing out all along. It's the integral of g of t times this one variable covariance function now, times g of tau d tau dt. When you look at this, and you say OK, what is this? It's an integral of g of t times some function of t. And that function of t is really the convolution of k sub z. This is a function now. It's not a random process anymore; it's just a function. That's a convolution of the function k tilde with g.

So I can rewrite this as the integral of g of t times this convolution of t. And then I'm going to call the convolution theta of t, just to give it a name. So this variance is equal to g of t times this function theta of t, which is just the convolution of k with g.

Well now I say, OK, I can use I can use Parseval's relation on that. And what I get is this integral, if I look in the frequency domain now, as the integral of the Fourier transform of g of t, times the complex conjugate of theta star of f times df. I've cheated it you just a little bit there because what I really want to talk about is -- I mean, Parseval's theorem relates this quantity to this quantity, where theta is theta complex conjugate.

But fortunately, theta is real. Because g is real and k is real. Because we're only dealing with real processes here, and we're only dealing with real filters. So this is real, this is real. So this integral is equal to this in the frequency domain. And now, what is the Fourier transform of -- OK, theta of t is this convolution. When I take the Fourier transform of theta, what I get is the product of the Fourier transform of k times the Fourier transform of g. So in fact what I get is g hat of f times theta star of f. Which is g complex conjugate of that times k complex conjugate -- times -- it doesn't make any difference whether it's a complex conjugate or not, because it's real.

So I wind up with the integral of the magnitude squared of g times this spectral density. Well at this point, we can interpret all sorts of things from this. And we now know that we can interpret this also in terms of effectively stationary things. So we don't have any problem with things going to infinity or anything. I mean, you can go through this argument carefully for effectively stationary things, and everything works out fine. So long as g of t is constrained in time.

Well now, I can choose this function any way that I want to. In other words, I can choose my function g of t in any way I want to. I can choose this function in any way I want to. So long as its inverse transform is real. Which means g hat of f has to be equal to g hat of minus f complex conjugate. But aside from that, I can choose this to be anything. And if I choose this to be very, very narrow band, what is this doing? It's saying that if I take a very narrow band g of t, like a sinusoid truncated out to some very wide region, I multiply that by this process, I integrate it out, and I look at the variance. What am I doing when I'm doing that?

I'm effectively filtering my process with a very, very narrow band filter, and I'm asking, what's the variance of the output? So the variance of the output really is related to the amount of energy in this process at that frequency. That's exactly what the mathematics says here. It says that s sub z of f is in fact the amount of noise power per unit bandwidth at this frequency. It's the only way you can interpret it. Because I can make this as narrow as I want to, so that if there is any interpretation of something in this process as being at one frequency rather than another frequency and the only way I can interpret that is by filtering the process. This is saying that when you filter the process, what you see at that bandwidth is how much power there is in the process at that bandwidth. So this is simply giving us an interpretation of spectral density as power per unit bandwidth.

OK, let's go on with this. If this spectral density is constant over all frequencies of interest, we say that it's white. Here's the answer to your question: what is white Gaussian noise? White Gaussian noise is a Gaussian process which has a constant spectral density over the frequencies that we're interested in. So we now have this looking at it in a certain band of frequencies.

You know, if you think about this in more or less practical terms, suppose you're building a wireless network, and it's going to operate, say, at five or six gigahertz. And suppose your bandwidth is maybe 100 megahertz or something. Or maybe it's 10 megahertz, or maybe it's 1 megahertz. In terms of this frequency of many gigahertz, you're talking about very narrowband communication. People might call it wideband communication, but it's really pretty narrowband.

Now suppose you have Gaussian noise, which is caused by all of these small tiny noise affects all over the place. And you ask, is this going to be flat or is it not going to be flat? Well, you might look at it and say, well, if there's no noise in this little band and there's a lot of noise in this band, I'm going to use this little band here. But after you get all done playing those games, what you're saying is, well, this noise is sort of uniform over this band. And therefore if I'm only transmitting in that band, if I never transmit anything outside of that band, there's no way you can tell what the noise is outside of that band. You all know that the noise you experience if you're dealing with a carrier frequency in the kilohertz band is very different than what you see in the megahertz band, is very different from what you see at 100 megahertz, is very different from what you see in the gigahertz band, is very different from what you see in the optical bands. And all of that stuff. So it doesn't make any sense to model the noise as being uniform spectral density over all of that region.

So the only thing we would ever want to do is to model the noise as having a uniform density over this narrowband that we're interested in. And that's how we define white Gaussian noise. We say the noise is white Gaussian noise if, in fact, when we go out and measure it we can measure it by passing it through a filter by finding this variance, which is exactly what we're talking about here, if what we get at one frequency is the same as what we get at another frequency. So long as we're talking about the frequencies of interest, then we say it's white.

Now how long do we have to make this measurement? Well, we don't make it forever, because all the filters we're using and everything else are filters which only have a duration over a certain amount of time. We only use our device over a certain amount of time. So what we're interested in is looking at the noise over some large effective time from minus T0 to plus T0. We want the noise to be effectively stationary within minus T0 to plus T0.

And then what's the next step in the argument? You want the noise to be effectively stationary between these very broad limits. And then we think about it for a little bit, and we say, but listen. I'm only using this thing in a very small fraction of that time region. And therefore as far as my model is concerned, I shouldn't be bothered with T0 at all. I should just say mathematically, this process is going to be stationary, and I forget about the T0. And I look at it in frequency, and I say I'm going to use this over my 10 megahertz or 100 megahertz, or whatever frequency band I'm interested in. And I'm only interested in what the noise is in that band. I don't want to specify what the bandwidth is. And therefore I say, I will just model it as being uniform over all frequencies.

So what white noise is, is you have effectively gotten rid of the T0. You've effectively gotten rid of the w0. And after you've gotten rid of both of these things, you have noise which has constant spectral density over all frequencies, and noise way which has constant power over all time. And you look at it, and what happens?

If you take the inverse Fourier transform of sz of f, and you assume that sz of f is just non-zero within a certain frequency band, what you get when you take the Fourier transform is a little kind of wiggle around zero. And that's all very interesting. If you then say well, I don't care about that. I just like to assume that it's uniform over all frequencies. You then take the Fourier transform, and what you've got is an impulse function.

And what the impulse function tells you is, when you pass the noise through a filter, what comes out of the filter, just like any time you deal with impulse functions -- I mean, the impulse response is in fact the response to an impulse. So that as far as the output from the filter goes, it's all very fine. It only cares about what the integral is of that pulse. And the integral of the pulse doesn't depend very much on what happens at enormously large frequencies or anything. So all of that's well behaved, so long as you go through some kind of filtering first. Unfortunately, as soon as you start talking about a covariance function which is an impulse, you're in real trouble. Because the covariance function evaluated at zero is the power in the process. And the power in the process is then infinite. So you wind up with this process which is easy to work with any time you filter it. It's easy to work with because you don't have these constants capital T0 and capital W0 stuck in them. Which you don't really care about, because what you're assuming is you can wander around as much as you want in frequency, subject to the antennas and so on that you have. And you want to be able to wander around as much in time as you want to and assume that things are uniform over all that region.

But you then have this problem that you have a noise process which just doesn't make any sense at all. Because it's infinite everywhere. You look at any little frequency band of it and it has infinite energy if you integrate over all time. So you really want to somehow use these ideas of being effectively stationary and of being effectively bandlimited, and say what I want is noise which is flat over those regions.

Now what we're going to do after the quiz, which is on Wednesday, is we're going to start talking about how you actually detect signals in the presence of noise. And what we're going to find out is, when the noise is white in the sense -- namely, when it behaves the same over all the degrees of freedom that we're looking at -- then it doesn't matter where you put your signal. You can put your signal anywhere in this huge time space, in this huge bandwidth that we're talking about. And we somehow want to find out how to detect signals there. And we find out that the detection process is independent of what time we're looking at and what frequency we're looking at. So what we have to focus on is this relatively small interval of time and relatively small interval of bandwidth.

So all of this works well. Which is why we assume white Gaussian noise. But the white Gaussian noise assumption really makes sense when you're looking at what the noise looks like in these various degrees of freedom. Namely, what the noise looks like when you pass the noise through a filter and look at the output at specific instance of time. And that's where the modeling assumption is come in.

So this is really a very sophisticated use of modeling. Did the engineers who created this sense of modeling have any idea of what they were doing? No. They didn't have the foggiest idea of what they were doing, except they had common sense. And they had enough common sense to realize that no matter where they put their signals, this same noise was going to be affecting them. And because of that, what they did is they created some kind of pseudo-theory, which said we have noise which looks the same wherever it is. Mathematicians got a hold of it, went through all this theory of generalized functions, came back to the engineers. The engineers couldn't understand any of that, and it's been going back and forth forever.

Where we are now, I think, is we have a theory where we can actually look at finite time intervals, finite frequency intervals, see what's going on there, make mathematical sense out of it, and then say that the results don't depend on what t0 is or w0 is, so we can leave it out. And at that point we really have our cake and we can eat it too. So we can do what the engineers I've always been doing, but we really understand it at this point.

OK. I think I will stop at that point.