Accelerated High-Quality Mutual-Information Based Word Clustering

TY - GEN

T1 - Accelerated High-Quality Mutual-Information Based Word Clustering

AU - Ciosici, Manuel R.

AU - Assent, Ira

AU - Derczynski, Leon

PY - 2020

Y1 - 2020

N2 - Word clustering groups words that exhibit similar properties. One popular method for this is Brown clustering, which uses short-range distributional information to construct clusters. Specifically, this is a hard hierarchical clustering with a fixed-width beam that employs bi-grams and greedily minimizes global mutual information loss. The result is word clusters that tend to outperform or complement other word representations, especially when constrained by small datasets. However, Brown clustering has high computational complexity and does not lend itself to parallel computation. This, together with the lack of efficient implementations, limits their applicability in NLP. We present efficient implementations of Brown clustering and the alternative Exchange clustering as well as a number of methods to accelerate the computation of both hierarchical and flat clusters. We show empirically that clusters obtained with the accelerated method match the performance of clusters computed using the original methods.

AB - Word clustering groups words that exhibit similar properties. One popular method for this is Brown clustering, which uses short-range distributional information to construct clusters. Specifically, this is a hard hierarchical clustering with a fixed-width beam that employs bi-grams and greedily minimizes global mutual information loss. The result is word clusters that tend to outperform or complement other word representations, especially when constrained by small datasets. However, Brown clustering has high computational complexity and does not lend itself to parallel computation. This, together with the lack of efficient implementations, limits their applicability in NLP. We present efficient implementations of Brown clustering and the alternative Exchange clustering as well as a number of methods to accelerate the computation of both hierarchical and flat clusters. We show empirically that clusters obtained with the accelerated method match the performance of clusters computed using the original methods.

KW - Efficient computation

KW - Word clusters

KW - Word representations

UR - http://www.scopus.com/inward/record.url?scp=85096595225&partnerID=8YFLogxK

M3 - Article in proceedings

SN - 979-10-95546-34-4

SP - 2491

EP - 2496

BT - LREC 2020 - 12th International Conference on Language Resources and Evaluation, Conference Proceedings

PB - European Language Resources Association

CY - Marseille

T2 - 12th Conference on Language Resources and Evaluation

Y2 - 11 May 2020 through 16 May 2020

ER -