Powering a non-structured text search application with natural language processing

Abstract

Many organizations find data reduction and analysis complex and costly. The Teacher Performance Assessment (VDD) process evaluates student satisfaction through open-ended questionnaires, using an algorithm with regular expressions for searching. However, this method may be outdated due to recent advancements in natural language processing (NLP) and artificial intelligence. These technologies can improve the analysis of unstructured text in the VDD by applying computational hermeneutics and NLP techniques. Tests indicate that NLP enhances the contextual search of relevant terms, yielding more accurate teacher evaluations while minimizing false positives and negatives. However, NLP implementation is more costly and time-consuming, making it suitable only for larger datasets and complex grammatical structures. Traditional algorithms remain effective for smaller datasets and simpler structures with limited computational resources.