1
00:00:00,330 --> 00:00:01,080
Hey, welcome back.

2
00:00:01,320 --> 00:00:05,970
In the previous video, I gave you some clue on what you be doing in this section.

3
00:00:06,420 --> 00:00:14,220
And we talked about comparing words, which is one of the main activities you can do in natural language

4
00:00:14,220 --> 00:00:15,010
processing.

5
00:00:15,030 --> 00:00:23,310
You compare if this text is that text and you're trying to find out if they mean the same thing.

6
00:00:23,790 --> 00:00:26,550
So this is what I'll teach you in this lecture.

7
00:00:26,550 --> 00:00:27,420
How to do that?

8
00:00:28,380 --> 00:00:29,780
So let's take an example.

9
00:00:29,790 --> 00:00:31,920
Let's say we have these two words.

10
00:00:32,310 --> 00:00:38,640
They have the same meaning they derive from the word be the verb being.

11
00:00:38,880 --> 00:00:46,200
So how can we find out that's how can we tell that these are the same verb?

12
00:00:47,100 --> 00:00:51,450
Of course, if you just do, X is equal to why you get false.

13
00:00:53,350 --> 00:00:55,570
So how can we get true for this?

14
00:00:56,590 --> 00:00:59,470
This is where natural language processing comes in.

15
00:00:59,860 --> 00:01:05,170
And we do natural language processing using a few libraries.

16
00:01:05,530 --> 00:01:08,860
The main one is called an AV Tiki.

17
00:01:10,770 --> 00:01:17,670
And Typekit, which stands for Natural Language Tool Kit, it's a third party library which you need

18
00:01:17,670 --> 00:01:19,830
to install, both on deep notes.

19
00:01:19,830 --> 00:01:22,140
You don't have to do that because it's already installed.

20
00:01:22,860 --> 00:01:25,500
I'm using deep notes as a notebook.

21
00:01:26,790 --> 00:01:37,850
No analytic provides a limit ization function, which allows you to find out the roots of a word.

22
00:01:38,130 --> 00:01:43,950
So in this case, we want to find all the roots of woods and the root of is, which is B.

23
00:01:44,130 --> 00:01:45,780
So the root of words is b.

24
00:01:45,780 --> 00:01:53,370
The root of is is B, and then we want to compare if the root of that word is the same with the root

25
00:01:53,370 --> 00:01:54,090
of vets words.

26
00:01:54,480 --> 00:01:58,560
So B is the same with B, therefore we will get through.

27
00:01:58,980 --> 00:02:07,360
So if the idea is to find the roots of the words, you do that by importing from actually from and nail

28
00:02:07,410 --> 00:02:16,110
t k that stem, you import the words and its limits either class.

29
00:02:16,530 --> 00:02:21,750
This is a class, so you want to create a new sense of that's class first.

30
00:02:22,080 --> 00:02:31,440
So I'm going to execute this successful creates a new code cell with Control J or Command G and.

31
00:02:34,860 --> 00:02:42,360
I'm going to call this valuable limited Pfizer, which is equal to world's net limited size with parentheses.

32
00:02:42,540 --> 00:02:52,500
So now we have a limited Pfizer instance, which we can use in this other sale together route of a verb.

33
00:02:53,370 --> 00:02:57,930
The root is also known as the lemmer of the words.

34
00:02:58,710 --> 00:03:09,310
So I'm going to have a variable lemon, a sequel to limit either the object's instance that will help

35
00:03:09,330 --> 00:03:15,540
us through the limitées methods to get the lemma of the words.

36
00:03:15,810 --> 00:03:22,410
So the first argument is the words, and in this case, it would be XOR.

37
00:03:22,410 --> 00:03:25,620
The words words x contains that string.

38
00:03:25,920 --> 00:03:29,640
So X and the second argument, as you see here.

39
00:03:29,730 --> 00:03:31,080
So the first was the words.

40
00:03:31,320 --> 00:03:37,290
The second is the part of speech tag, also known as the Pulse.

41
00:03:37,870 --> 00:03:42,930
And so this is where you define what type of world this is.

42
00:03:43,410 --> 00:03:47,220
So you need to see that this is a verb.

43
00:03:48,400 --> 00:03:55,410
Now this can also be automatic, so we can let Python understand the roots of the words, but we will

44
00:03:55,410 --> 00:03:56,280
do that later.

45
00:03:56,790 --> 00:03:58,800
So let's take it step by step.

46
00:03:58,800 --> 00:04:05,820
For now, let's tell Python that this is a verb, so sort of a limitless method knows how to get the

47
00:04:05,820 --> 00:04:08,010
lemma of that word.

48
00:04:08,400 --> 00:04:18,480
So V for verbs, a four adjectives or four adverbs and there's four sets of lights, adjectives and

49
00:04:18,510 --> 00:04:20,440
of course, is the default one.

50
00:04:20,459 --> 00:04:25,200
If you don't want to place an argument here, then it would be a noun.

51
00:04:25,650 --> 00:04:26,640
So a verb.

52
00:04:26,640 --> 00:04:28,350
Is this in our case?

53
00:04:28,830 --> 00:04:33,450
And let's call Lemma here to see the version of words.

54
00:04:33,870 --> 00:04:40,530
Now we get this of error the first time when you execute this, and that is because you first need to

55
00:04:40,530 --> 00:04:49,260
download the words net dictionary, which contains all the English words because you are trying to get

56
00:04:49,270 --> 00:04:54,570
the lemma of this word, but you need a database to do to get that information right.

57
00:04:54,630 --> 00:05:02,880
And this database should be an English dictionary for this method to look up the lemma of the words.

58
00:05:03,210 --> 00:05:10,580
So before you do that, you want to create a quote, sell and do any of the kids or the downloads.

59
00:05:10,590 --> 00:05:12,330
Wouldn't it execute that?

60
00:05:12,710 --> 00:05:17,640
Oh, we need to import and ldk, so execute again.

61
00:05:18,030 --> 00:05:18,300
Yeah.

62
00:05:18,660 --> 00:05:22,410
So it downloaded and unzipped this word and its file.

63
00:05:22,740 --> 00:05:23,940
So let's execute again.

64
00:05:26,710 --> 00:05:30,640
And this is the ultimate so be a piece of the limb of Edwards.

65
00:05:31,090 --> 00:05:36,760
Similarly, we can get the limo of the otherwords, so let's call this limo one.

66
00:05:37,360 --> 00:05:45,520
And the other words, would be limited to a sequel to their ties or limited size of the same instance

67
00:05:45,700 --> 00:05:46,480
that we got here.

68
00:05:46,840 --> 00:05:50,080
That's where the ties this time is.

69
00:05:50,080 --> 00:05:56,200
Why that's worth so is, and this is also a verb.

70
00:05:58,570 --> 00:06:00,670
Lemma two is again, the.

71
00:06:01,180 --> 00:06:06,010
So then we can compare Lemma one, a single to Lemma to.

72
00:06:06,250 --> 00:06:13,320
But let me execute the Lemma one variable first because we renamed it Lemma two executes.

73
00:06:14,230 --> 00:06:14,830
There we go.

74
00:06:14,890 --> 00:06:18,880
So now we are seeing that these two words are the same.

75
00:06:19,940 --> 00:06:21,130
That is how you do that.

76
00:06:22,630 --> 00:06:27,290
So let me put all the codes in one single cell to make it look better.

77
00:06:27,770 --> 00:06:30,790
This maybe it's better to have it on top here.

78
00:06:33,530 --> 00:06:40,280
Or even in a separate cell, because this should be only executed once you only have to download, that

79
00:06:40,280 --> 00:06:42,050
wants of a dictionary.

80
00:06:43,730 --> 00:06:49,880
And then we created the limbo of the first words, and then we created the limo of the second words.

81
00:06:51,710 --> 00:06:58,490
And then we compared a landmark on a single to limo two and we got true.

82
00:06:59,540 --> 00:07:00,560
So that's how.

83
00:07:03,440 --> 00:07:08,690
You compare words by taking the meaning of the words into consideration.

84
00:07:09,590 --> 00:07:18,620
This, of course, has a problem because if you try to get another word such as vegetable and compare

85
00:07:18,620 --> 00:07:28,160
it with Virgin Vegetable, so the plural form and the singular form of these two words, you will get

86
00:07:28,160 --> 00:07:32,930
falls because you see that Lemma one is vegetables.

87
00:07:33,290 --> 00:07:41,150
So when you have verb as the second argument, this lemma size function will not know how to get the

88
00:07:41,390 --> 00:07:42,650
lemov into words.

89
00:07:42,950 --> 00:07:46,370
But if you say known, then you get vegetable.

90
00:07:46,820 --> 00:07:52,370
So you say that this is a noun, and then the method knows how to get the singular form of vatanen.

91
00:07:52,940 --> 00:08:00,350
So that is a drawback that doesn't lets you automatically gets dilemma of all the words without knowing

92
00:08:00,350 --> 00:08:00,890
what they are.

93
00:08:00,890 --> 00:08:02,660
But it has a simple fix.

94
00:08:02,960 --> 00:08:05,900
We can simply get the information.

95
00:08:05,990 --> 00:08:13,370
If this is a noun or a verb, then provide that information dynamically in here so that we don't have

96
00:08:13,370 --> 00:08:18,650
to know beforehand if this is a verb or noun, an objective or an adverb.

97
00:08:19,280 --> 00:08:21,230
So we'll do that in the next video.

98
00:08:21,560 --> 00:08:22,010
See you there!