Table of Contents
This is the first module of our Natural Language Processing course. In this article we will discuss the following topics:
- What is Natural Language Processing?
- Types of Natural Language Processing
- Phases of Natural Language Processing
- Applications of Natural Language Processing
1. What is Natural Language Processing
Natural language processing (NLP) also referred to as Text Analytics is the capability of the machine to understand the contextual meaning of textual data and speech in the much same way as a human does.
NLP combines the power of computational linguistics i.e., rule-based modeling with machine learning and deep learning based modeling to understand the contextual meaning of human language and speech.
There are many tasks which comes under NLP such as speech recognition, sentiment analysis, question answering, contextual summary of large text, text classification, named entity recognition, part of speech tagging, machine translation.
2. Types of Natural Language Processing
There are broadly two types of NLP: Rule-based and Artificial Intelligence based.
- Rule-based NLP: This type of NLP is a very older approach of handling textual data where linguistic rules are applied to understand the meaning of the written text. In this type of NLP approach, customized rules are designed with the help of linguistic experts or knowledge engineers to manually encode rules for the NLP tasks. The major disadvantage of this approach is that they have to constantly enhance the rules all the time which sometimes results in more complex rules contradicting each other.
- AI-based NLP: In this type of NLP, machine learning algorithms, deep learning methods, neural networks, and other statistical methods are used to understand the meaning of the text. This textual data is fed to AI-based algorithms in the form of training data where the system can understand the patterns and build its own rule through repeated processing and retraining.
3. Phases of Natural Language Processing
There are basically three main phases of NLP which includes the following:
1. Lexical Analysis: It involves analyzing the structure of the text by converting it into a paragraph, sentence, or words (lexemes). It is the process of understanding the meaning of words, their context in a sentence, and noting its relationship with other words.
2. Syntactic Analysis: It involves analysis of the syntactic structure of the sentence in order to infer the meaning of the sentence. The syntactic structure of a sentence is basically a logical arrangement of words in a sentence based on grammatical rules which convey their meaning in order to make sense. Following techniques are generally used in the syntactic analysis:
- Stemming: It is a text pre-processing technique that ensures that different variations of a word, say ‘hard’, ‘harder’, and ‘hardest’ are represented by a single token – ‘hard’, because they all represent the same meaning.
- Lemmatization: It is a more refined and improved form of stemming in which different variations of a word are reduced to a base form i.e., lemma which is actually present in the dictionary
- Parts-of-Speech (POS) Tagging: POS tagging involves tagging a word as a noun, verb, adjective, adverb, preposition, etc. helps us better understand the contextual meaning of a phrase or a sentence and hence a crucial part of syntactic processing.
3. Semantic Analysis: It is a process to infer the meaning of a given piece of text. This is the most challenging phase of NLP as it requires the system to understand the meaning of a particular text in the same way as a human does. It involves defining terms, concepts, entities, arity, the relation between entities, etc. The following techniques are generally come under semantic analysis:
- Semantic association: It is a technique of analyzing the association or relation of different words in a sentence in order to understand the meaning of natural language.
- Word sense disambiguation: Word sense disambiguation (WSD) is the task of identifying the correct sense of an ambiguous word such as ‘bank‘ has different meanings in these two sentences: “A bank of clouds was building to the northeast” and “Her bank account was rarely over two hundred“
- Word embeddings: Word embeddings is a process of mapping words to a typically smaller dimension vector space that preserves their distributional semantics.
- Topic modeling: Topic modeling is the art and science of identifying ‘latent topics’ in text. It helps us in automating the task of identifying the actual topic of discussion in the sentence.
4. Applications of Natural Language Processing
Over the last couple of years, users are generating a huge volume of textual data around the words through different digital channels such as blogs, social media websites, e-commerce websites, etc., which increases the scope of applying NLP in varied domains.
Some of the important applications of NLP in the industry are as follows:
- Sentiment Analysis: Nowadays companies are focusing on analyzing the customer’s feedback about their product which helps them in aligning their product best fit to the customer. This is achieved using sentiment analysis by which they can understand the basic sentiment of customer’s feedback i.e., whether it’s a positive, negative or neutral sentiment.
- Chatbots: As we visit any product-based website a chatbot usually appears which usually helps us in answering basic frequent questions about the product and gives us the feeling that we are talking to a human-based agent. Chatbot primarily helps in automatically attending to the customers visiting the website and helps reduce the workload on human-based agents.
- Language Translation: Automatic language translator helps to translate one language to another which in turn removes the barrier of communication in a different language. Google translate is a real-world example of a language translator which provides real-time translation of 107 languages all around the world. The translation may not be 100% accurate all the time but it helps in understanding the context of the text.
- Autocomplete in search engine: When we want to search something and type our query on a search engine, it suggests us most searched or related terms to help the user actually find the most appropriate related query which people already searched for. Search engines analyze their enormous datasets to give real-time suggestions and suggest the most probable possibilities. They use natural language processing to understand the sense of the phrase so that they can suggest the most interconnected sentences in the form of suggestions
- Spell corrector: Spell corrector is a very useful utility in daily life. It helps in correcting the spellings while we are drafting important documents. The spell checker is usually present in MS Word under Review Tab.
- Grammar checkers: When we are drafting business reports it’s very important that we use correct spelling and grammar in the sentences. Failure to do so may lead you to get fired from the job. That’s the reason grammar checkers are a very important utility for professional writers, and managers in writing business reports, lecture presentations, etc. Natural language processing is basically used in building automated grammar checkers which are trained on millions of documents in order to point out grammatical mistakes and help improve the overall readability of the content by suggesting suitable synonyms and sentence rephrasing. Grammarly, Ginger, Hemmingway, Typely, etc. are some of the most used grammar checker tools online.
- Voice Assistants: Siri, Alexa, and Google Assistant are real-world examples of smart voice assistants which help us in scheduling an appointment, making a call, placing a reminder, setting an alarm, etc. They have made our life easier by automating most of our recurring tasks. They are one of the most advanced combinations of speech recognition, natural language understanding, and natural language processing in order to understand the contextual meaning of human speech and take actions accordingly.
Apart from the above-discussed applications, there are many more applications of NLP which are actively used in different domains in the industries.
Conclusion
So, in this module, we get the overall basic understanding of natural language processing, different types and phases of natural language processing, and at last, and we also discussed some of the important real-world applications of natural language processing. In the next lesson, we will discuss the Lexical Processing technique i.e., Regular Expressions.