Collective classification

Traditionally, a major focus of machine learning is to solve classification problems. (For example, given a collection of e-mails, we wish to determine which are spam, and which are not.) Many machine learning models for performing this task will try to categorize each item independently, and focus on predicting the class labels separately. However, the prediction accuracy for the labels whose values must be inferred can be improved with knowledge of the correct class labels for related items. For example, it is easier to predict the topic of a webpage if we know the topics of the webpages that link to it. Similarly, the chance of a particular word being a verb increases if we know that the previous word in the sentence is a noun; knowing the first few characters in a word can make it much easier to identify the remaining characters. Many researchers have proposed techniques that attempt to classify samples in a joint or collective manner, instead of treating each sample in isolation; these techniques have enabled significant gains in classification accuracy.[1][2]

Consider the task of inferring the political affiliation of users in a social network, where some portion of these affiliations are observed, and the remainder are unobserved. Each user has local features, such as their profile information, and links exist between users who are friends in this social network. An approach that does not collectively classify users will consider each user in the network independently and use their local features to infer party affiliations. An approach which performs collective classification might assume that users who are friends tend to have similar political views, and could then jointly infer all unobserved party affiliations while making use of the rich relational structure of the social network.

Consider the semi supervised learning problem of assigning labels to nodes in a network by using knowledge of a subset of the nodes' labels. Specifically, we are given a network represented by a graph ${\displaystyle G}$ with a set of nodes ${\displaystyle V}$ and an edge set ${\displaystyle E}$ representing relationships among nodes. Each node ${\displaystyle v_{i}\in V}$ is described by its attributes: a feature vector ${\displaystyle x_{i}\in X}$ and its label (or class) ${\displaystyle y_{i}\in Y}$.

${\displaystyle V}$ can further be divided into two sets of nodes: ${\displaystyle L}$, the set of nodes for which we know the correct label values (observed variables), and ${\displaystyle U}$, the nodes whose labels must be inferred. The collective classification task is to label the nodes in ${\displaystyle U}$ with a label from a label set ${\displaystyle L=\{L_{1},\cdots L_{q}\}}$.

In such settings, traditional classification algorithms assume that the data is drawn independently and identically from some distribution (iid). This means that the labels inferred for nodes whose label is unobserved are independent of each other. One does not make this assumption when performing collective classification. Instead, there are three distinct types of correlations that can be utilized to determine the classification or label of ${\displaystyle v}$:

1. The correlations between the label of ${\displaystyle v}$ and the observed attributes of ${\displaystyle v}$. Traditional iid classifiers which make use of feature vectors are an example of approaches that use this correlation.
2. The correlations between the label of ${\displaystyle v}$ and the observed attributes (including observed labels) of nodes in the neighborhood of ${\displaystyle v}$.
3. The correlations between the label of ${\displaystyle v}$ and the unobserved labels of objects in the neighborhood of ${\displaystyle v}$.

Collective classification refers to the combined classification of a set of interlinked objects using the three above types of information.

There are several existing approaches to collective classification. The two major methods are iterative methods and methods based on probabilistic graphical models.[3]

Collective classification is applied in many domains which exhibit relational structure, such as:

• Social network analysis, where collective approaches to node classification tasks such as detecting malicious users can utilize information about relationships between nodes.[15][16]
• Entity resolution, where one can make use of co-authorship relationships to identify authors of papers.[17]
• Named entity recognition, where some approaches treat this as a text sequence labeling problem and jointly infer the labels of every word in a sentence, typically by using a conditional random field which models a linear chain of dependencies between the labels of adjacent words in the sentence.[18]
• Document classification, where for example inter-document semantic similarities can be collectively utilized as signals that certain documents belong to the same class.[19]
• Computational biology, where graphical models such as Markov random fields are utilized to jointly infer relations between biological entities such as genes.[20]
• Computer vision, where for example collective classification can be applied to recognizing multiple objects simultaneously.[21]

• Machine learning
• Classification
• Similarity (network science)
• Graph (discrete mathematics)
• Statistical relational learning
• Bayesian Networks
• Markov Random Field