1
00:00:00,500 --> 00:00:03,020
In order to build
circuits to manipulate,

2
00:00:03,020 --> 00:00:06,030
transmit or store information,
we are going to need some

3
00:00:06,030 --> 00:00:08,700
engineering tools to see
if we're choosing a good

4
00:00:08,700 --> 00:00:10,730
representation for
the information --

5
00:00:10,730 --> 00:00:13,660
that's the subject
of this chapter.

6
00:00:13,660 --> 00:00:16,520
We'll study different ways of
encoding information as bits

7
00:00:16,520 --> 00:00:18,780
and learn the mathematics
that help us determine

8
00:00:18,780 --> 00:00:20,970
if our encoding is a good one.

9
00:00:20,970 --> 00:00:24,800
We'll also look into what we can
do if our representation gets

10
00:00:24,800 --> 00:00:26,380
corrupted by errors.

11
00:00:26,380 --> 00:00:28,890
It would be nice to detect
that something bad has happened

12
00:00:28,890 --> 00:00:32,659
and possibly even
correct the problem.

13
00:00:32,659 --> 00:00:35,860
Let's start by asking
"what is information?"

14
00:00:35,860 --> 00:00:37,370
From our engineering
perspective,

15
00:00:37,370 --> 00:00:40,590
we'll define information as
data communicated or received

16
00:00:40,590 --> 00:00:43,230
that resolves uncertainty
about a particular fact

17
00:00:43,230 --> 00:00:44,660
or circumstance.

18
00:00:44,660 --> 00:00:47,060
In other words, after
receiving the data

19
00:00:47,060 --> 00:00:49,190
we'll know more about
that particular fact

20
00:00:49,190 --> 00:00:50,900
or circumstance.

21
00:00:50,900 --> 00:00:53,380
The greater the uncertainty
resolved by the data,

22
00:00:53,380 --> 00:00:55,534
the more information
the data has conveyed.

23
00:00:57,150 --> 00:00:59,040
LET'S LOOK AT AN
EXAMPLE: a card has

24
00:00:59,040 --> 00:01:01,340
been chosen at random
from a normal deck of 52

25
00:01:01,340 --> 00:01:02,940
playing cards.

26
00:01:02,940 --> 00:01:05,050
Without any data
about the chosen card,

27
00:01:05,050 --> 00:01:08,450
there are 52 possibilities
for the type of the card.

28
00:01:08,450 --> 00:01:12,220
Now suppose you receive one of
the following pieces of data

29
00:01:12,220 --> 00:01:13,720
about the choice.

30
00:01:13,720 --> 00:01:17,470
A. You learn the suit
of the card is "heart".

31
00:01:17,470 --> 00:01:21,440
This narrows the choice
to down one of 13 cards.

32
00:01:21,440 --> 00:01:25,610
B. You learn instead the card
is NOT the Ace of Spades.

33
00:01:25,610 --> 00:01:29,080
This still leaves 51
cards that it might be.

34
00:01:29,080 --> 00:01:32,740
C. You learn instead that
the card is a face card, that

35
00:01:32,740 --> 00:01:35,140
is, a Jack, Queen or King.

36
00:01:35,140 --> 00:01:38,050
So the choice is
one of 12 cards.

37
00:01:38,050 --> 00:01:42,190
D. You learn instead that the
card is the "suicide king."

38
00:01:42,190 --> 00:01:44,370
Our little blue friend is
showing us that this is

39
00:01:44,370 --> 00:01:47,420
actually a particular card – the
King of Hearts where the king

40
00:01:47,420 --> 00:01:49,910
is sticking the sword
through his head!

41
00:01:49,910 --> 00:01:54,690
No uncertainty here, we know
exactly what the choice was.

42
00:01:54,690 --> 00:01:56,650
Which of the possible
pieces of data

43
00:01:56,650 --> 00:01:58,900
conveys the most information?

44
00:01:58,900 --> 00:02:02,070
In other words, which data
resolves the most uncertainty

45
00:02:02,070 --> 00:02:04,110
about the chosen card?

46
00:02:04,110 --> 00:02:06,930
Similarly, which data
conveys the least

47
00:02:06,930 --> 00:02:08,600
amount of information?

48
00:02:08,600 --> 00:02:10,280
Before we talk about
the mathematics

49
00:02:10,280 --> 00:02:12,690
behind the correct answers
to these questions,

50
00:02:12,690 --> 00:02:15,910
vote for your answers
in the following poll.