coherence score

[RaymondLi1101]

Hi Ruidan,
First of all, really great work. I tried it on some really noisy data set and some really cohesive aspects popped out.
I do find the coherence score as a very interesting way to evaluate this unsupervised model. But I did not find the corresponding code in your evaluation(Or I missed it?). So I am wondering how did you compute the coherence score.

[ruidan]

Hi,
I did not include the coherence score computation in the code, you can find how it could be computed in Section 5.1.1 in the paper. However I do not think it is a good evaluation metric, as I found it does not well reflect the coherence in certain cases according to my personal experience.

[RaymondLi1101]

thanks for the reply. Would you mind share some of your experience with coherence score and how it can't reflect the coherence?