Natural Language Processing

Giving computers the ability to interpret sentences and text as well as humans do.

Introduction

Natural Language Processing (NLP) is studying algorithms and methods of giving computers the ability to interpret sentences and text as well as humans do. This is similar to Computer Vision, except rather than images and videos, the goal is to understand text. Examples include sentiment analysis and topic modelling. Popular applications include chatbots or conversational agents.

Common techniques in NLP pre-processing include:

• Tokenisation: Breaking up a paragraph into a list of words for future processing.

• Stop words: Commonly used words such as "the", "a" and "for", which usually are removed during text pre-processing as they have no significant meaning in use-cases such as sentiment analysis.

• Lemmatisation: Grouping different forms of the same word into one category. For example, organised, organises, and organising all refer to the base word organise, and have the same meaning.

Resources

• Popular open source libraries for NLP include:

  • spaCy

  • NLTK

  • Stanford CoreNLP

  • TextBlob

• AWS technologies include:

  • Amazon Comprehend: Find insights and relationships in text.

  • Amazon Lex: Building conversational interfaces using voice and text.

• Google Cloud technologies include:

  • Natural Language API

  • Dialogflow is a Google tool aimed to quickly create conversational agents which can easily interface with websites such as Facebook, Skype.

• Microsoft Azure technologies include:

  • Text Analytics