The following content is provided under a Creative
Commons license.
Your support will help MIT OpenCourseWare continue to
offer high quality educational resources for free.
To make a donation or to view additional materials from
hundreds of MIT courses, visit MIT OpenCourseWare at
ocw.mit.edu.
PROFESSOR: We started to talk about the discrete time
Fourier transform last time.
This is probably something you've been exposed to before,
and as in many cases, we're looking at it in a very
different way than what you probably looked at it before.
The discrete time Fourier transform is simply the time
frequency dual of the Fourier series.
Nothing more than that.
For any L2 function --
now we're talking about a function of frequency, but a
function is a function.
So this is a complex valued function of frequency.
If it's limited, if it's truncated to the band from
minus w to plus w, then, in fact, it's inverse Fourier
transform is given by this limit in the mean of the sum
of the coefficients times v sub k of f.
This is exactly the same as the Fourier transform
replacing times with frequencies, replacing w for t
over 2, and in the complex frequency you're replacing a
to the plus 2 pi i here with e to the minus 2 pi i in the
Fourier series.
So those are the only differences, it's just
notational differences, and aside from that, it works
exactly the same way.
The coefficients are given by this.
We showed, when we were talking about the Fourier
series, that these coefficients exist as complex
numbers, they're always finite.
You can calculate them if you want to.
This quantity here can be rather fishy.
This is this limit in the mean which says that you have to
calculate this by looking at the sum here over a finite
sum, over a finite sum this is well-defined and
behaves very nicely.
As you go to the limit funny things can happen, but the
thing that we showed is in terms of energy, nothing funny
can happen.
I'm going to give you an example of this kind of funny
business as we talk about the sampling theorem in just a
couple of minutes.
So the u hat of f has to be L1 since it is
limited in this way.
It has a continuous inverse transform, which is this.
OK, so you can go from --
blah blah blah blah blah.
The discrete Fourier transform is simply a transform between
a function and a sequence of terms.
Now this bit here is something we haven't
talked about before.
Because now we've talked about Fourier transforms also, and
if you have an L2 function, u hat of f, that L2 function has
a Fourier transform and an inverse Fourier transform in
this case, which is u of t, which is given by this
expression here, the usual expression
for the Fourier transform.
Now, the thing that's peculiar about this Fourier transform
is that since u hat of f is limited, is truncated, this
transform here exists everywhere.
We don't need a limit in the mean here.
You can calculate this for every t this exists.
This is a point-wise convergent thing.
This is the thing that you're used to when you think of
functions, because the function is defined everywhere
in this case.
And since we're going to go into the sampling theorem, it
had damned well better be defined everywhere because
otherwise the sampling theorem wouldn't make
any sense at all.
So, the inverse Fourier transform of the discrete time
Fourier transform, as we said, is this, it's a
limit in the mean.
Namely, you take more and more terms here.
You get closer and closer to this in terms of energy.
It doesn't say anything about what happens for particular
values of f.
This is a sampling expansion with t equals 1 over 2w.
OK, let's go back here.
We're talking about some function of frequency which
has a Fourier series, it also has a Fourier transform.
What we're interested in now is what is the relationship
between these coefficients here and a discrete time
Fourier transform and this function here.
What I want to show you is that, in fact, these two
things are very closely related.
You can now go back to the Fourier series itself and
relate the Fourier series coefficients to the Fourier
transform and you'll get the same kind of sampling
representation that we're getting right now.
So, if we -- there's something missing in what I'm
trying to say here.
Oh, I see what I'm trying to do, sorry.
What I'm trying to do is to take the inverse Fourier
transform of u hat of f, which is given by
this expression here.
Temporarily I'm going to forget about the fact that
this is a limit in the mean, throw mathematics to the
winds, and simply take this inverse transform.
Take the inverse transform of this also.
So, when I take the inverse transform of the sum, what I'm
going to get is the sum over k of u sub k, and then in place
of this frequency function -- these are orthogonal functions
here, the things I listed on the previous page --
I got u of t is the sum of u sub k times these time
functions now.
These time functions are just the Fourier transforms of
these frequency functions here.
This is a set of orthogonal wave forms here, which are
truncated sinusoids.
I want to take the Fourier transform of these and the
Fourier transforms of these --
I should have put them both on the same slide so you could
see what they are --
but in fact, in your homework you're going to take the
inverse Fourier transform of that and show that it is, in
fact, this.
That's just a nice exercise in taking rectangular functions
and sinc functions and applying shifts in both time
and frequency.
When you do this, this turns out to have the inverse
transform of this.
So this is u of t is just this, where this comes from
here, and then this turns out to be the inverse transform of
this function here, which, in fact, is just this.
Sorry for all of that.
So now if we want to try to understand what this means,
suppose that you take a function u of t, which is, in
fact, the inverse transform of this u hat of f, which is
truncated to a finite band limit.
If we then take v of t equal to u of t everywhere except on
the sample points, and on every sample point we simply
add 1 to v -- in other words, v of kt is equal to
u of kT plus 1.
So we take this nice smooth function that we have here,
and at every sample point we simply add 1.
So now the question is, is this new function I have still
baseband limited or isn't it?
You see you can't answer that question because we weren't
careful enough to say what we meant by a
baseband limited function.
Usually when you talk about a baseband limited function,
you're talking about a function whose Fourier
transform is zero, except in the range minus w to plus w.
Well, this new function v of t here has the property that its
Fourier transform is zero outside of minus w to plus w.
And therefore if you define baseband limited as functions
whose Fourier transform is zero outside of limits, then
the sampling theorem doesn't hold.
So what do we do about this?
Well, the easiest thing to do about it is to change what we
mean by baseband limited to what you would have meant if
we hadn't going through all of this mathematics.
In other words, the Fourier transform cuts both ways -- u
of t has a Fourier transform, u hat of f, u hat of f has an
inverse transform, u of t.
What we mean now by baseband limited is that u of t is the
inverse transform of a frequency function, which is
limited to minus w over plus w.
In other words, we will regard u of t here as being baseband
limited, we will not regard v of t as
being baseband limited.
Because we have these two functions, u of t and v of t,
which both have the same Fourier transforms, but which
are not equal to each other.
We want to, at this point, say that the only one which is
really baseband limited is the function which is the inverse
transform of u of t.
If you read the sampling theorem as it's written in the
notes, that's exactly what it says.
It defines this function u of t to which the sampling
theorem applies in that way.
Well anyway, this is the sampling
theorem at this point.
It says you can take u of t, you can
express it in this way.
If I go back to the previous slide, what we did here by
comparing u of t with this expression for u sub k. u sub
k are the coefficients in the discrete time Fourier
transform. u of t is this inverse transform.
These quantities here are almost the same.
This quantity here is just evaluating this at particular
frequencies.
Namely, if for frequency f I substitute in, if we're time
t, I substitute in k over 2w, then this formula just becomes
that formula.
In other words, 2w times u sub k is equal to u
evaluated at k over 2w.
You already saw that when you looked at the Fourier series.
When you looked at the Fourier series, you saw these
coefficients, which, in fact, look like the Fourier
transform terms.
And which, in fact, were the same as the Fourier transform
except for a scale factor at some particular frequency.
Here we have these coefficients, which are, in
fact, scaled versions of the inverse transform of
particular times.
So the conclusion from that is that, in fact, what the
discrete time Fourier transform is is it's simply
the Fourier transform of the sampling theorem expansion.
The two of them are duals in a very different way than the
Fourier series and the dtft are duals.
The dtft and the Fourier series are duals in the sense
that if you take the expressions for the Fourier
series and change frequency for time and time for
frequency, you get to the dtft.
Here what we're doing is taking the dtft and simply
taking the inverse Fourier transform of it, so that the
sampling theorem is, in fact, the Fourier
transform of the dtft.
It's not the dual, it's the Fourier transform itself.
Well, the discrete time Fourier transform generalizes
to arbitrary frequency intervals just as well as to a
baseband interval.
Namely, you can do exactly the same thing as what we've just
done if you're looking not at the frequency at the range of
frequencies from minus w to plus w, but you shift that up
to any old place you want to and look at delta minus w to
delta plus w.
And the fact the discrete time Fourier transform, if you
don't put the rectangular function in it is going to be
periodic anyway.
It's exactly the same thing that we had
with a Fourier series.
With a Fourier series, we could find the Fourier series
for a function limited between minus t over 2
to plus t over 2.
Now by duality, we can find the dtft for a function
limited between minus w plus delta and plus w plus delta.
That's what we're doing here.
So the dtft in generalized form is now this, and v sub k
is now the integral from delta minus w to delta plus w of the
same old thing as before.
This is equal to this, which is the same old thing as
before, except we now have this shifted frequency in the
rectangular function.
So if we take the inverse Fourier transform of this,
again, using the same duality we had before, we get v of t
is equal to the sum times the sinc function.
And the only difference now that we're expanding this
given frequency band not centered around zero but
centered around something else, the only difference in
the sampling theorem is now we have this rotating term
gyrating around up at this frequency, k over 2w.
That's the only way in which this is different from the
sampling theorem that we had before.
I wish I could put more things on one slide but you wouldn't
see them if I could.
So here's a Fourier transform of the sum 1 over t, u of kt
time the sinc function.
Here it's the same thing.
That's one reason for comparing
these things sometimes.
1 over t in here.
OK, times, so it's the same thing with this rotating term
which is the only difference.
Now, how many of you can see that the Fourier transform of
this quantity is equal to this sinc function?
I can't do that.
It's one of the things you're going to do in your homework.
I get confused every time I do it, and I got confused enough
to leave out the 1 over t this time when I did it.
You just have to be patient with that and do it a few
times and you'll find that it's not --
well, it becomes automatic after awhile.
It's one place where you need plug and chug.
So now that we've generalized the dtft to look at any old
frequency band instead of just the frequency band around
zero, we can do the same thing that we
did with time functions.
Namely, with a time function --
Yes?
AUDIENCE: [INAUDIBLE].
PROFESSOR: You don't think it should be a 1 over t?
Well, you very well might be right because I didn't think
it should have been either when I wrote it down.
But I don't see --
AUDIENCE: [INAUDIBLE].
PROFESSOR: It was the--.
AUDIENCE: [INAUDIBLE].
PROFESSOR: Oh, that's the difference, yes, of course.
It's not the coefficient that I have here,
it's the actual --
yeah.
Yes.
So it should be the same as the--
Oh, I see the problem.
I shouldn't have had the 1 over t here, should I?
No.
I know one of them couldn't be right.
It is right in the notes, so you can sort it out there.
So, the thing we did when we were dealing with a Fourier
series is we took an arbitrary function of time, we segmented
it into time intervals and then we expanded each one of
those time intervals into a Fourier series.
By doing that we could take an arbitrary L2 function and
represent it as an orthogonal expansion over this double sum
of time shifts and frequency terms.
We call that the truncated sinusoidal expansion, the t
spaced truncated sinusoids and we made an expansion out of
that that would allow us to express any old L2 function in
terms of that.
Here we're going to do the same thing.
We can take an arbitrary frequency function, separate
into bands of frequencies.
You often want to do this in digital communication when
you're looking at transmitting information in different
bands, which you do all the time in radio.
Somebody has a certain part of the spectrum, they transmit a
signal there, somebody else has another part of the
spectrum, they transmit a signal there.
You can look at those different signals which are in
different frequency bands, they're all orthogonal to each
other, they don't interfere with each other at all.
We're doing the same thing here.
We're just saying an arbitrary function can be split into
different frequency bands, each one of those frequency
bands can be represented both by a dtft, which is the thing
we just did on the last slide, and by sampling theorem
expression, which is what we get when we take the inverse
Fourier transform of the dtft.
So when we do that what we get is a perfectly arbitrary
frequency function which exists from minus infinity to
plus infinity.
We can represent it as the sum of all these separate
frequency functions.
I just threw a limit in the mean here because I'm not
being careful about what happens where we separate from
one frequency function to the next.
Namely, at frequency w do I use one term or do I use the
other term or do I use the sum.
But we don't want to worry about that.
We don't want to even think about it.
So we put a limit in the mean here.
So the v sub hat m of f then is going to be the part of u
of f which is in this particular frequency range.
That's completely the analogy of taking a time function,
looking at that time function over a particular range of
time, and here what we're doing is taking a frequency
function, segementing it into different frequency intervals
so that the end frequency interval is then just this
with a rectangular function to truncate it.
If I take the inverse Fourier transform of this what I'm
going to get is u of t, take the inverse transform of all
of these terms, so I'll get the sum of vm of t.
Now, vm of t is the inverse Fourier transform of vm of f.
You take that quantity, take the inverse transform of it,
and sure enough you get this kind of expression here.
It's a sampling theorem in v sub m of f with this rotating
frequency term here, which is just the
thing that we had before.
So, all we're doing here is starting out with some
arbitrary frequency function, we're segementing it in
frequencies.
Each frequency band then has a dtft associated with it.
When we take the inverse Fourier transform of that
dtft, what we get is a sampling expansion for that
particular frequency band.
So what this is doing here, finally when we get all done
with this, is I'm just combining the sampling theorem
expansion in each frequency range, which
is what u of t is.
So I've taken u of t, I split it up into different frequency
ranges, I've expressed what's in each frequency range in
terms of a sampling theorem.
The sampling theorem terms are these with these rotating
terms in them corresponding to the mth frequency range.
This is completely analogous then to the truncated sinc
function expansion we had before.
This becomes a little more sensible if we substitute a
sampling time, t, for 1 over 2w.
Namely, all of these expressions here are talking
about what happens when you sample these individual
frequency bands at intervals 1 over 2w.
So we'll just call that capital T to make the formula
look a little simpler.
Then we get u of t is this limit in the mean of this
whole expression there.
So it's a double sum, it's a sum over time, over the
samples, so there's one term for each time, kT, and there's
one term for each frequency interval.
Frequencies are indexed by m, time is indexed by k.
So the thing we have here is an expansion now in terms of
coefficients -- these are just called coefficients again.
This expansion which looks suspiciously like the t space
truncated sinusoids that we had before.
The only difference is that, the terms were truncated in
time; here, the terms are truncated in frequency.
So the different terms making up this expansion, these
orthogonal terms here, in one case what we have is a sinc
function which is translated in time and then
translated in frequency.
In another case we have the rectangular function, which is
translated in time and then translated in frequency.
So in that sense, these two expansions are almost the
same, and you can think of doing expansions perhaps in
other things also and we'll talk more about that later.
So, this then is just this thing we're going to call a t
spaced sinc weighted sinusoid expansion.
So the only thing we have is this one sinc function which
is this hat sort of function.
The terms in here are those functions shifted in time by
some number of sampling intervals, t, and then shifted
in frequency by some number of frequency bands, 2w.
See, the original frequency bands that we had went from
minus w to plus w.
The next one goes from w to 3w, the next one
goes from 3w to 5w.
So the frequency bands we're talking about here
are of width 2w.
The time intervals we're talking about are of width t.
Why do people confuse you that way?
Well, because all of this happened a long time before
people realized how closely the duality relationship
between time and frequency was.
So people wanted to talk about frequencies, baseband
frequencies.
You talk about a baseband limited to w and you're
talking about positive frequencies, because engineers
used to deal with cosines and sine, and there weren't any
such thing as negative frequencies.
Then they decided everything was easier when they dealt
with complex sinusoids, negative frequencies reared
their ugly head, but people didn't want to change their
notation for what a frequency band was,
which is probably good.
So we're simply stuck with this incompatibility of
dealing with frequencies one way and dealing with
time the other way.
We can look at that as increments of time t, and
increments of frequency, 1 over t, but, in fact, 1 over t
is 2w, so the increments in time we're using in both of
these expansions, are t, the increments in frequency we're
using are 2w.
Now, there's a relatively long section in the notes talking
about degrees of freedom, which is a
pretty important topic.
It's a little bit fishy mathematically, but it really
makes good engineering sense.
It's an idea which is important both in terms of
taking source wave forms and representing them in
orthogonal expansions, and in taking frequency functions and
representing them --
well, it is also important in terms of taking things that we
transmit where you have bits coming into an encoder.
We're going to turn those bits into signals, we're going to
turn those signals into wave forms.
Those wave forms will usually be thought of as things that
we transmit in time, and we also transmit them in
frequency, because we often use some kind of multiplexing
between different frequency bands.
We want to have a common way of thinking about all of these
things, and this is the way that we're going to do it.
Namely, if we're thinking in terms of a particular sampling
time, t, and we want to look at a very, very large
frequency band, and therefore, look at many multiplex
frequency bands, we can say how many coefficients can we
send on this channel?
When we look at it in these terms of different frequency
bands, the number of coefficients we can send is
over a period of time, t zero.
We can send t zero over t different
coefficients in time.
We now look at frequency.
We have some very broad frequency band, w zero.
The number of different bands that we have
is w zero over 2w.
As a result of all of this when you add everything up,
you get 2t zero w zero degrees of freedom over this overall
bandwidth of w zero.
Now remember, an overall bandwidth of w zero in terms
of these complex frequencies goes from minus w to plus w.
Minus w zero to plus w zero.
The time interval goes from minus t zero over 2 to plus t
zero over 2.
So this factor 2 here, which we always talk about in terms
of number of degrees of freedom, is really a
consequence of the fact that we measure time intervals and
frequency intervals in a slightly different way.
But anyway, whether we look at it in terms of one expansion
or the other expansion the answer we get is the same.
If you take some large time interval, some large frequency
interval, tuck as many numbers as you can in that interval,
this is what you come up with.
Now, why did I say that this is just slightly fishy?
Well, it's slightly fishy because if you take a function
and you truncate it in time --
if we take this function and truncate it to minus t zero
over 2 and plus t zero over 2, even though that might be ten
years, how can we limit the frequency?
Well, we can't.
Because when we take the Fourier transform of a time
limited function, it exists for all frequencies.
The same thing happens if we try to limit it in frequency,
it squirts out forever in time.
So you can't get around that.
The thing that saves us is that if t zero and w zero are
both large enough, these functions all dribble away
quickly enough that it doesn't make any difference.
You know it has to.
If you think in terms of the sampling theorem, and you try
to think about it carefully in mathematics, what does it say?
If you want to transmit a function by putting these
little sine x over x hats around each of the
coefficients in the function, when do you have to start
transmitting those wave forms?
You have to start transmitting them at minus infinity.
I mean we turn on our transmitter and we somehow
have to have been transmitting for an infinite amount of time
before we send the first symbol.
Well that's ridiculous, of course.
So that we always approximate these sinusoids by sinusoids
which are truncated, and we always have some engineering
faith that what we're throwing away is not important.
The only place that it's important is when you start
talking about things which are really zero everywhere and
then it becomes important.
But the idea of degrees of freedom is a very sensible
idea until you try to express it precisely.
Let's get on to something called aliasing.
We're going to spend most of the rest of today talking
about aliasing.
I want to try to explain why it is that we want to spend
time on this, because there are really two
things going on here.
One of the things that are going on is that if you want
to look at a wave form and do some processing on it, the
usual way to do it with the digital technology we have
today, is to take that wave form and sample it very, very
rapidly in time.
Then process the hell out of all those samples.
We do that hardly caring whether it's band limited,
hardly caring about the information in it, hardly
caring about anything, we just want to sample it so fast that
we essentially approximate the function very well.
Now when we do that something's going to get lost,
because when we take those samples we're ignoring what
happens between the samples.
In some approximate sense, wave forms are always smooth
because they always get filtered by something before
anybody looks at them.
And because they're smooth, if you'd sample it fast enough,
your fast enough samples are going to look
like the wave form.
If you just connect them with straight lines you're going to
get a very good approximation of what the wave form is.
But then you stop and ask, and when you stop and ask you're
in trouble because you say well, how fast do I have to
sample, and if I sample that fast, how much error am I
going to make?
That's the question for which aliasing gives you the answer.
So we want to explore it for that reason.
The other reason that we want to explore it is when we start
talking about modulation, we'll start talking about
something called Nyquist criterion, and that's best
looked at in terms of aliasing again, and we'll see why that
is when we get there.
So for both of those reasons, and also for the reason of
trying to understand these expansions in terms of source
wave forms, we want to understand what this
relationship is between the samples of a function that
isn't quite band limited and the function itself.
So the thing we're going to do to try to understand that is
instead of studying just the samples, which is what you
usually do, if you're just looking at the samples and
you're trying to say how much error do I incur by doing
that, and we're looking at mean square error, somehow or
other we have to get back to wave forms and compare the
resulting wave form with the wave form we started with.
So the thing that we're going to do is to take our function
u of t, we're going to sample it at some very rapid speed,
and then we're going to take those samples and recreate a
wave form by the sampling theorem.
So we're going to call the approximation to u of t some
approximation s of t -- s of t is, in fact, baseband limited
at this point to a frequency w where t is equal to 1 over 2w,
and we simply have this sampling theorem applied to u
of kt, as if u of kt came from a band limited wave form.
So this is, in a sense, an interpolation formula.
It's a little better than taking these samples and
joining them by straight lines, because we're, in fact,
joining them by these smoother sinc functions.
The question is how close is s of t to u of t and how do we
look at that question?
Well, we now have a nice way of looking at it because after
going through this t spaced sinc weighted sinusoidal
expansion, we have an exact expression for u of t, which
is the limit in the mean of these frequency
terms in u of t.
Namely, to get this expression, remember the thing
we did was to take arbitrary u of t, split it up into little
frequency bands, each of width w, then apply the sampling
theorem to each of those frequency terms, which we can
do exactly.
So this, in fact, is an exact expansion -- don't worry about
the limit in the mean right now, we're going to get rid of
that in a while.
So we have s of t, which is this.
We have u of t, which is this.
Now, we look at this and we say OK, s of kt, namely, the
case sample of s of t, is simply u of kt.
That's what happens here.
You put in any old time which is some integer j times
capital T and you look at this expression here, and the sinc
function is only non-zero when j capital T over t, when if
you take the sinc function of an integer it's zero unless
the integer is zero.
The sinc function goes through zero at every integer point
except for zero itself where it's equal to 1.
That's the thing which makes the sync function nice.
It dribbles away --
2, 3 and so forth minus 1.
So it's zero at all those sample times, so that the case
sample of s of ts of kt is just equal to u of kt.
Namely, what we're doing in this approximation, which is
what one usually does if one samples something, is we're
assuming that the approximation is correct at
the sample points, and we're arranging it so it's correct
at the sample points.
So, s of kt then is equal to u of kt from this.
u of kt from this --
OK, now take t and substitute k times capital T in here.
And if we substitute k times capital T in here -- let's
not, let's substitute j times capital T to avoid a conflict
with notation here.
Then this T here becomes jT and what we have is
sinc of j minus k.
So we have a sum here over all k of sync of j minus k.
Sinc of j minus k is zero for all integers k, except for j,
therefore, this quantity is zero every time k is
not equal to j.
Therefore, we just have the sum over m.
So, s of kt is equal to u of kt, which is equal to the sum
over all frequency bands, m, of v sub m of kt.
Now what is this saying?
The thing this is saying is if you take this function u of t,
which has a bunch of different frequency bands in it, each of
those frequency bands has a sampling theorem
associated with it.
Each one of those frequency bands is represented by its
samples at periods of time, k.
But as soon as we look at u of kt, if we only have the
samples u of kt, there's no way to tell which frequency
band it's coming from.
So all of these different frequency bands
all get alias together.
If I just look at u of kt, what it is is the sum over all
these frequency bands of the samples of the individual
frequency functions.
So that if you tell me what u of kt is, I can't tell you
what these samples are.
All I know is what the sum of them is.
So, in fact, if you start out with a function which instead
of being baseband limited to w, in fact, is sitting between
w and 3w, and there's nothing in this baseband, and I look
at these samples and then I recreate things this way, what
am I doing?
I'm just taking that function at w to 3w and translating
down in frequency to minus w to w.
You can't tell.
There's no way to tell just from the samples which
frequency band we're looking at.
So all these things get mixed together.
So, s of t then, since s of t is this times these sinc
functions, and this is the sum of all the vm's, s of t is
just this double sum now where all of these are now down at
this baseband frequency interval.
There's no way to tell them apart.
We have this double sum here so I'm adding up all of these
different coefficients and they're all mixed together all
down in this one frequency band.
So, u of t is represented this way, double sum, vm of kt, vm
of kt, sinc of t over t minus k, sinc of t over t minus k.
This rotating term up here, an s of t, what I've done
effectively is to get rid of all these rotating terms.
It simplifies it enormously, but I changed the function.
In fact, this function is low path limited to w and this
function has all of the glory of an arbitrary set of
frequencies in it.
By just looking at these samples, I've lost all of this
stuff and it's just back down to a baseband limited function
at this point.
So, if I look at the difference between u of t and
s of t, what I'm going to get -- and this is expressed a
little differently than the way it is in the notes but
it's the same thing -- just the sum over k and m, and it
has these sinc functions in it, which both
of these terms have.
Then u of t has these rotating terms and s of t doesn't, so s
of t just has one in place of the rotating term.
So the difference between u of t and s of t is just this big
monster sum here, which is looking at all the different
frequency bands at all the different sampling times.
If I now try to look at the energy difference between u of
t and s of t, because that's what I'm interested in -- how
much error have I accumulated, how much mean square error
have I gotten?
By taking u of t and sampling it and then viewing it as a
low pass function.
This energy difference, well, we have a set of coefficients
here, we have a set of functions here.
These functions are all orthogonal to each other.
Why are they orthogonal?
Well, because sinc functions are orthogonal, which I think
we've shown on our homework, and space time functions are
orthogonal.
So, all of these functions are orthogonal to --
excuse me --
spaced frequency functions are orthogonal to each other, and
the sinc weighted functions are orthogonal to each other.
So all I'm doing here is expanding all of this in terms
of these orthogonal functions.
I have to do this separately for these terms --
I know what happens.
This works when it's cold and it doesn't work when it's hot.
Interesting.
So I'm separating this into this term and this term.
Now, look at what this is when m is equal to zero.
When m is equal to zero, e to the 2 pi i, zero t over t
minus 1 is zero.
So, all of the m terms here are collapsed into zero, so
there isn't any error down at baseband in a sense.
All of the error occurs due to these frequency terms larger
than minus w to w.
Well, that's the way it should be.
Because we know that if u of t didn't have any terms outside
of minus w to plus w, the sampling theorem would be
absolutely rock solid with no error.
So the errors are due to two kinds of terms.
One, they're due to these terms, which is this thing.
Two, they're due to these terms, which is this.
The only difference between this is that this is a square
of a sum and this is a sum of squares.
Sum of squares are nicer, we can deal with them more
nicely, and we understand what that is better.
A sum and then taking the square of it after we take the
sum is a good deal dirtier.
The trouble is that this difference
doesn't have to be L2.
In other words, the mean square error we got from doing
this can be infinite.
One of the problems in the current problem set is a nice
simple example of this.
So, s of t need not have finite energy.
That's a real kicker, because all of this theory is very
nice until this point.
I mean the sampling theorem works perfectly because when
you're dealing with baseband limited functions --
for up baseband limited function, the sinc functions
give you a perfect approximation.
Namely, the sampling theorem works with no error at all.
As soon as you get a function which spills out into higher
frequencies, you can, in fact, have these samples coming up
with infinite energy in them, and at that point this
difference here can have infinite energy.
In fact, you would do much, much better if you simply
represented u of t by zero.
You'd do infinitely better in terms of mean square error
just by throwing it away and saying I'm not going even
bother to approximate it.
I'll just call it zero and nothing else.
If you did that, you would only have the error in u of t
and you wouldn't have all this generated stuff from all these
terms that you're throwing away.
When we start looking at random processes, and really
the only thing that we're interested in here is random
processes because noise is a random
process, signals are random.
So when we start looking at random processes, the thing
that we're going to find is that this square of a sum in
terms of expected value is going to be approximately the
same as this sum of squares.
So that this term and this term are going to be roughly
equal for most of the stochastic processes that we
deal with, so this problem doesn't occur of s of t having
infinite energy.
What that means is that these two terms are of
roughly equal magnitude.
Now, we can understand something more
about these two terms.
The term here is what you get, namely, these are the parts of
u of t which are outside of the frequency range
from minus w to w.
A reasonable alternative to simply sampling this function
would be to filter the function first.
If you filter the function first what's going to happen?
These terms will go away.
These terms will stay.
You still have the aliasing --
you can't get rid of that.
But you can get rid of these terms.
Excuse me.
I take that back.
I seem to have a binary problem today, whatever.
I seem to be mixing up proof and falsehood.
If you filter a function you're clearly going to get
rid of all of the aliasing, because after you filtered the
function you will have a band limited
function sitting there.
So the only error you're going to get is the error in what
you've thrown away.
What you've thrown away is, in fact, this quantity here.
Slow down, say it right.
What you have thrown away is all of the
extra frequency terms.
So the error is all of those frequency terms that you've
thrown away.
These are the frequency terms that you've thrown away.
This is the error that you wind up with after you filter.
These terms here are the aliasing terms.
These are the things which, in fact, could be infinite if
you're unlucky enough.
These you can get rid of by filtering.
So now the question is should you filter first or should you
just sample and say to hell with it?
Well, if you think about why you want to sample and use
digital signal processing, what you're trying to do is to
avoid building very complex analog filters.
So if you try to build a very, very sharp filter which is
getting rid of all of those out of band terms, you're sort
of throwing away a lot of the reason for trying to sample to
start with.
So the usual conclusion is no, we're not going to filter
first, or we're only going to have a very crude filtering
operation first, and we'll just sample a little faster if
we have to, because sampling faster is easier.
So we usually just sample faster and avoid these terms.
Now we want to look at aliasing viewed in frequency
terms, and we've almost been doing this.
We've been sort of talking around it a little bit.
We're viewing an arbitrary function, u of t, in terms of
a sum of these frequency limited functions.
We're just arbitrarily taking a function, splitting it up
into these frequency bands.
In terms of frequency what we're doing is just
segementing the Fourier transform u of f.
As far as the samples are concerned, uf of kt is just
equal to the sum of the samples in each of these
frequency bands -- that's what we've been saying.
That's really what aliasing is at a fundamental level.
So when you sample you can't tell which frequency band each
of these samples come from, and they usually come a little
bit from each one.
So, s of t now, I'd like to split up s of t in the same
way that I split up u of t.
I'd like to split up s of t, even though I can't do this
from looking at s of t, I can do it mathematically.
I want to split up s of t into the contributions to s of t
from all of these different bands.
So I want to view the mth frequency band as the sum of
vm of kt times sinc of t over t minus k.
Namely, look at what s of t was to start with, somewhere
back here a long time ago.
What we found was that s of t is equal to this quantity
here, which is the sum over time terms, the sampling
terms, and a sum over frequency term.
What I'm doing now is I'm just defining s sub m of t to be
this sum in here for one particular value of m.
I'm just splitting this double sum into a
number of separate terms.
So this is the contribution to s of t from the
mth frequency band.
So, s of t is the sum of all of these different frequency
contributions in this quantity. vm of t, which came
from u of t, same quantity here except for the complex
exponential here. vm of t is the same quantity with the
rotating term at the end of it.
Namely, this is the actual signal at this
mth frequency band.
So the contribution to it and the sampling approximation is
this, the actual term is this.
So we look at these two and we say gee, this looks like -- if
we look at this in the frequency domain this looks
like just a frequency shift.
So the thing we can then say is that v sub m of f, namely
the Fourier transform of this, differs from the Fourier
transform of this just by a frequency shift.
If we take the frequency shift formula and frequency and look
at it in time it just becomes that rotating term there.
Namely, frequency shifts look like complex exponential
multipliers.
We already know that d sub m of f is the mth frequency band
and u of t.
So in fact, it's u of f truncated to the
mth frequency band.
That's what this said .
So, u hat of f rect ft minus m is just equal
to this term here.
So this says that s hat of f, which is the sum of all these
terms, is just the sum over m, sum over all the frequency
terms of u hat of f plus m over t times a rectangular
function of ft.
All this is saying is that s of f in frequency, it's all
down at baseband now.
Each of these frequency terms in u of t, up at some band and
frequency, when we sample it, we're effectively bringing it
down to base then.
When we sample it and multiply by the sinc functions we're
bringing it down to baseband.
So this is saying when you look at this approximation,
the space band approximation, what we wind up with is this
frequency function evaluated at all of these different --
well, just evaluated at different times.
So, s hat of f is, in fact, frequency limited to minus w
to plus w, and it has all of these different terms
contributing to it.
It's just looking at aliasing and frequency instead of
looking at aliasing and time.
In both cases it's the same thing.
In one case, all of the samples get mixed together.
In the other case, all of the frequency
bands get mixed together.
I hope this will be clearer in terms of this.
Let's take some arbitrary function here -- a most
amazing thing, these things all get.
It seems as if latex -- a thing for drawing pictures is
fine on my screen but not fine when I print things.
But anyway, this is fine as it is.
It's slightly different from the one in the notes, which is
also goofed up in the same way.
Let's suppose we start out with a frequency function
which looks like this.
Actually, it should have something else
there but that's OK.
This is minus 1 over 2t, this is plus 1 over 2t.
In other words, minus w to w.
Here we have another frequency band from w to 3w, another
frequency band from 3w to 5w.
What that formula said is that each frequency band gets
picked up and stuck down in the baseband area.
So, in fact, what's happening is that this part of the
frequency function is going to be picked
up, stuck down there.
So this goes over there.
This quantity here is in the band from 3w to 5w.
It gets picked up and put in here also.
It's gotten picked up and put down there.
So I have this part, this part, and this part just stays
where it is -- this is the part that's
actually band limited.
So that s hat of f now is going to be the sum of this
and this and this, and one disappeared term which was
supposed to be here, and that's going to come in there.
When you add up all of these -- this goes over to there.
When you add up all of these you get the total frequency
function s hat of f.
All of you understand the mechanics of this?
I mean graphically you can just find s hat of f from
taking all of these things and folding them all into this
baseband approximation.
So that you can look at the error that you get in terms of
sampling a function which is not quite band limited in
these terms, also.
The aliasing then looks like the translation from something
which is in this band stuck into the main band between
minus w and w.
Translation of this thing stuck into the main band.
Translation of this stuck into the main band.
They're all added together in such a way that you can't tell
which one came from where.
So once you look at this, you can't go back to here, which
is why people call it aliasing.
These things are all just mixed together and there's no
way to get back.
The theorem that corresponds to all of this, and the
theorem is pretty much proven in the appendix if you
want to read it.
If you're not interested in those mathematical details
you're certainly welcome not to read it.
I'm not going to have any problems on it.
I'm not going to have any quiz problems on it, we might have
a problem on it.
It turns out that just L1 and L2 is not enough when you're
dealing with aliasing.
Aliasing, since you're both sampling and looking at things
at arbitrarily large frequencies, the mathematics
just gets messy.
So the condition that you need is that the frequency function
you're dealing with in order for all of these aliasing
results to hold true, is that the limit as f goes to
infinity of u hat of f times some function, f to the 1 plus
epsilon has to go to zero.
In other words, this is saying that u hat of f has to go to
zero with increasing frequency fast enough.
It has to go to zero a little faster than 1 over f.
If it went to zero, it was 1 over f, that would be
guaranteed by a thing, L2.
That's not enough here.
You need the stronger condition.
So if it goes to zero fast enough as f gets very, very
large -- you know any function you're going to deal with you
can always model it so that it does this.
Because you simply can't transmit wave forms that have
arbitrarily high frequencies in them.
I mean no matter what kind of antenna you use to transmit
them, if it's an optical antenna you can get to much
higher frequencies, but no matter what kind of antenna
you use you're going to be limited somehow in frequency,
and it's going to drop off much faster with increasing
frequency than this.
So this isn't any sort of practical limitation on
aliasing, it's just that it's there and it limits the models
you can create somewhat.
If you have this condition then it says that the Fourier
transform of u of t has to be an L1 function.
The inverse transform, u of t, has to be
continuous and bounded.
In other words, when we go from u hat of f to u of t we
get a bonafide function there.
It's not something which is a limit in the mean.
Because as soon as we're dealing with samples of
something you really need a function to talk about.
You can't have something which is --
well, you can't have something which has these little extra
things on it.
You can't live with that.
So the theorem says that this frequency
function has to be L1.
It says the inverse transform has to be
continuous and bounded.
We've already seen that L1 functions have continuous and
bounded Fourier transforms.
For any t greater than zero, the sampling approximation,
namely, s of t, which is this, is going to be bounded and
continuous.
And s hat of f satisfies this relationship here.
So this is the frequency version of
the aliasing formula.
Here you still have the limit in the mean here.
As soon as you go back to frequencies you can't say
anything precise anymore.
In time, everything is precise.
These functions are bonafide functions.
In terms of frequencies they're not quite.
I want to just start talking about these L2 functions.
As you realize, we've not only been talking about L2
functions, we've been talking about L1 functions also, and
now these crazy functions which go off
to zero fast enough.
The thing we're basically interested in, aside from
sampling, is the L2 functions because they're the ones where
you can go from function to Fourier
transform and back again.
They're the ones which work or these orthogonal expansions,
and they're the main things we're going to
be interested in.
When we try to go further talking about these functions,
it turns out that it's much easier to think about them in
terms of vector spaces.
Some of you probably have thought about wave forms
before as vectors, some of you probably haven't.
I'm sure all of you are familiar with vectors at least
in terms of a notational convenience as a way of
representing n tuples of numbers.
You can always take an n tuple of numbers and instead of
writing out u1, u2, u3, up to u sub n, you can say vector u,
and you can manipulate those vectors, you can add vectors,
you can multiply them by scalers, you can do all of the
neat things that people do, even without knowing anything
about vector spaces.
It's not much of an extension on that to take accountably
infinite sequence of numbers and represent it as a vector,
especially if you've never done it before.
If you've done it before and if you've thought about it
before, you will realize there are some problems there.
But at least conceptually there's no problem.
You can think about a sequence of numbers as being a vector,
you can add those sequences, you can multiply them by
scalers, you can do all sorts of neat things with them.
Since we've shown how to represent wave forms as
sequences of coefficients in orthogonal expansions, it then
isn't much of an extension to think about wave forms as
being vectors.
But the nice thing about doing this is that you're not really
stuck to a particular orthogonal
expansion when you do this.
You can think of wave forms as being vectors in just as nice
a sense as you can think about these n tuples as being
vectors, and that's what we're trying to get at.
These orthogonal expansions we're going to be looking at
are really viewed most easily in terms of vector space.
All of the questions about convergence of orthogonal
expansions and things like that, limits and all of that,
all of these are very natural in vector spaces.
They're so natural that people often forget about the fact
that they're taking limits even.
So it just looks nice there.
But as soon as we do this, what we're going to be doing
constantly from now on is trying to say what we know
when we're looking at two-dimensional vectors, we
have all sorts of pretty pictures
about how they behave.
We are going to be trying to use those pretty pictures in
two-dimensional space to understand what happens with
wave forms, and in order to do that we have to know a little
bit more about vectors than just vectors are things you
can add and multiply scalers by.
So I will bore you for two minutes by quickly going
through the axioms of a vector space.
My reason for going through this is when you define a
vector space axiomatically, then you can, in fact, prove
that wave forms satisfy all those axioms.
Once you prove that wave forms satisfy all those axioms, then
everything you know about two-dimensional and
three-dimensional vectors and all of that stuff all applies.
Namely, everything which is general for vector spaces
applies to these vectors.
So the axioms are the following.
You start out with a set of elements
which you call vectors.
In terms of n tuples, the set of elements is just the set of
different n tuples.
You also start out -- well, for here we don't have to
worry about scalers.
So we have the set of n tuples now, or perhaps it might be a
set of sequences or a set of wave forms.
The things we insist on before we will call this set a vector
space are that we have an operation
which we call addition.
Fortunately for n tuples, addition is what you would
think it would be.
It's element-wise addition.
For sequences, it's element-wise addition again.
For wave forms, it's function addition.
You have communitivity --
you can add things in either order.
I mean look, if nobody pointed this out to you, you'd do it
anyway, right?
So that it looks like nothing.
But in fact, you can invent things, mathematical objects,
which you can deal with which are not communitive.
So this is saying to be a vector space it has to have
this communitivity operation.
Associativity --
again, it's hard to see why you would say something like
this, but all we're defining is this addition operation,
namely, there's a definition of addition, and we know by
axiom that the sum has to be in this vector space.
As soon as we assume this associativity axiom, it says
OK, this is a vector in the space, this and this are both
vectors in the space.
Therefore, this sum has to be in the space.
Therefore this plus the sum has to be in the space.
What associativity says is that that's the
same element as this.
Once you see this you leave out the parentheses because
you can add as many things as you want to.
Same thing -- well, here there has to be a unique
vector, zero --
v plus zero is equal to v for all v, and there's only one
vector that has that property.
We're going to see some problems there in a little bit
when we start studying these L2 functions, but we'll do
that next time.
Finally, for every v there's a unique negative vector so that
the sum is equal to zero.
All the operations that you're used to.
We'll go through the other things next time.