ICML 2024: Paper Review #4

Julien Ferry, Ricardo Fukasawa, Timothée Pascal, Thibaut Vidal

This paper presents a framework for reconstructing tabular training data from a trained random forest model. The approach is based on solving an optimisation problem where the constraints are derived from the random forest’s structure (such as the number of trees and their depth) and the dataset’s characteristics (such as the number of examples and features).

The success of the reconstruction varies depending on the level of bagging randomisation and the nature of the data. For simpler cases, such as when there is little randomisation and there are only binary features, the method can nearly perfectly reconstruct the training data.

However, in more practical scenarios—such as those involving more extensive randomisation, larger datasets or real-valued features—the reconstruction error increases significantly. Further, API-only attacks, where the attacker has limited visibility into the model, are not feasible under this framework.

Much of the existing literature on data privacy and reconstruction attacks focuses on neural networks, especially in domains like images and text. This paper highlights that training data reconstruction is a more general property of machine learning models. This observation puts the data privacy conversation in a better context – focusing us more on properties of the data and the learning algorithm that determine reconstruction success, rather than just on the model architecture.

Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, Trevor Darrell

Downloading a pretrained neural network and slightly modifying it for a new custom task, all done using industry-standard open source tools is a typical workflow for many researchers and engineers today. Ten years ago, the DeCAF was one of the first works to envisage this reality.

The authors showed that extracting image features from an AlexNet neural network pretrained on one task (ImageNet classification) provides a strong foundation for a wide variety of tasks. These features could be used for a new task typically requiring only a small model trained on top of them. A broad range of experiments, where DeCAF substantially improved on the baselines, confirmed the practical relevance of this idea.

The team behind the paper also released the trained models from their experiments for other people to use. The DeCAF team also played a significant role in the development of the Caffe library—one of the first open-source libraries that supported GPU-based neural networks, facilitating their use in both research and industry.

Many of the components that contributed to DeCAF’s success already existed when it was launched:

• Word2Vec had demonstrated the value of pretraining in NLP and the influence of publicly available models
• AlexNet showed that deep neural networks can be stronger than hand-crafted features in computer vision

The open-source software ML ecosystem was already rapidly developing but DeCAF combined these trends in a coherent demonstration of much of what the next 10 years of machine learning turned out to be.