Master Thesis Topic 3: Metrics for quality of Topic Modeling

Evaluation of methods and metrics for quality assessment of topic modelling algorithms

RQ: What are the state-of-the-art metrics for evaluation of topic modelling algorithm? And how well do they capture to what extend the topic representative words manage to successfully represent the topic documents?

In other words how to quantitatively measure if the set of topic representative words are from one consistent topic and that this topic expresses a common similarity in the underlying documents. Also what are the advantages and disadvantages of the currently available metrics?

Sub RQs:

  • What methods exist currently for measuring quality of topic modelling?
  • Which ones are comprehensible to humans and which are hard to interpret?
  • When given a topic (representative words + documents set) with what confidence level we can confirm that this topic is coherent and expresses one clear similarity between the underlying documents?

Students are also encouraged to propose new methods on top of what they find in their study.

The work requires a literature review on what metrics exists and then analysis on the Iris AI topic modelling algorithm. Expected output is assessment of the metrics, summarization of the results and proposition either an existing metric, a new metric, or a combination of existing metric and suggested improvements, to be used for automatic quality metric for current and future algorithms.

Interested? Get in touch!