Generalised Brown Clustering and Roll-up Feature Generation

TY - GEN

T1 - Generalised Brown Clustering and Roll-up Feature Generation

AU - Derczynski, Leon

AU - Chester, Sean

PY - 2016/2/21

Y1 - 2016/2/21

N2 - Brown clustering is an established technique, used in hundreds of computational linguistics papers each year, to group word types that have similar distributional information. It is unsupervised and can be used to create powerful word representations for machine learning. Despite its improbable success relative to more complex methods, few have investigated whether Brown clustering has really been applied optimally. In this paper, we present a subtle but profound generalisation of Brown clustering to improve the overall quality by decoupling the number of output classes from the computational active set size. Moreover, the generalisation permits a novel approach to feature selection from Brown clusters: We show that the standard approach of shearing the Brown clustering output tree at arbitrary bitlengths is lossy and that features should be chosen insead by rolling up Generalised Brown hierarchies. The generalisation and corresponding feature generation is more principled, challenging the way Brown clustering is currently understood and applied.

AB - Brown clustering is an established technique, used in hundreds of computational linguistics papers each year, to group word types that have similar distributional information. It is unsupervised and can be used to create powerful word representations for machine learning. Despite its improbable success relative to more complex methods, few have investigated whether Brown clustering has really been applied optimally. In this paper, we present a subtle but profound generalisation of Brown clustering to improve the overall quality by decoupling the number of output classes from the computational active set size. Moreover, the generalisation permits a novel approach to feature selection from Brown clusters: We show that the standard approach of shearing the Brown clustering output tree at arbitrary bitlengths is lossy and that features should be chosen insead by rolling up Generalised Brown hierarchies. The generalisation and corresponding feature generation is more principled, challenging the way Brown clustering is currently understood and applied.

KW - clustering

KW - natural language processing

KW - feature generation

UR - http://www.scopus.com/inward/record.url?scp=85007198068&partnerID=8YFLogxK

M3 - Article in proceedings

SN - 978-1-57735-700-1

SP - 1533

EP - 1539

BT - 30th AAAI Conference on Artificial Intelligence, AAAI 2016

PB - AAAI Press

T2 - The Thirtieth AAAI Conference on Artificial Intelligence

Y2 - 12 February 2016 through 17 February 2017

ER -