High influence data samples: some data points that dramatically alter the prediction of the model (confusing to the model, leading it to directly memorize the data point rather than learning useful information from the data)
Ambiguous or mislabeled samples: (ie. training image with both fish and dog, but labeled as fish) model has high loss on these data points due to its low confidence, making them alter the predictions drastically
For instance-based interpretability in supervised learning, the various influence methods help answer the question: which training samples are most responsible for the prediction of a text sample? This relies heavily on the label information, ie. the predictive nature in supervised learning such as classification.
Since there is no label in unsupervised learning, this work investigates the application of influence functions on VAE by framing the question as follows: which training samples are most responsible for increasing the likelihood (or reducing the loss) of a test sample?
Influence functions in the unsupervised setting are analyzed through the 3 classical methods: non-parametric k-nearest neighbor (k-NN) density estimator, non-parametric kernel density estimator (KDE), and the parametric Gaussian mixture models (GMM). Let the loss function $l$ be the negative log-likelihood $l(z)=-\log p(z)$.
This work extends the proposed measure of memorization in supervised learning to the unsupervised density estimation problem, and show that it measures a phenomenon not captured by the commonly-used nearest neighbor tests.
When the training data contains numerous highly similar observations, then it is expected that they receive an increased weight in the model and are more likely to be generated. Hence containing duplicates provides a reason for memorization of training observations. However, this study focus on memorization that arises as an increased probability of generating a sample that closely resembles the training data in regions of the input space where the algorithm has not seen sufficient observations to enable generalization. This may include observations that are atypical, or are essential for modeling a particular region of the data manifold.
Note that memorization (blue) and generalization (yellow) can occur simultaneously at distinct regions of the input space, and memorization is not necessarily caused by overtraining (overfitting).
Since models aim to learn the probability density of the training datas and are optimized globally, model has to place some probability mass for regions that are weakly supported by the available data. Moreover, it is shown that the model doesn’t necessarily place low probability on such observations, resulting in observations that are both highly memorized and not significantly less likely under the model than other observations.
Experiments in the study examined how the memorization score evolves during training. It is shown that while the memorization scores are reduced by lowering the learning rate in algorithm, relatively high memorization can still occur. Moreover, memorization starts before the generalization gap appears and it stabilizes as the training epochs increase. This suggest that early stopping doesn’t fully alleviate memorization, since it may occur well before signs of overtraining.
Memorization score measures the impact of removing an observation on the model, quantifying the degree to which the model has memorized it. This score may be used to determine regions of the input space that require additional data collection
To obtain higher diversity in samples synthesized by the generative models, the models will have to learn to generalize to regions of the data manifold that are not well represented in the input space.
Influence functions: estimates the influence of individual training examples by measuring how much they contribute to the likelihood of the generated samples. That is, how much each example contributes to the probability distribution learned by the model, and use this information to identify influential examples that have large impact on model’s behavior.