Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Distribution-Based Invariant Deep Networks for Learning Meta-Features

Abstract : Recent advances in deep learning from probability distributions successfully achieve classification or regression from distribution samples, thus invariant under permutation of the samples. The first contribution of the paper is to extend these neural architectures to achieve invariance under permutation of the features, too. The proposed architecture, called Dida, inherits the NN properties of universal approximation, and its robustness w.r.t. Lipschitz-bounded transformations of the input distribution is established. The second contribution is to empirically and comparatively demonstrate the merits of the approach on two tasks defined at the dataset level. On both tasks, Dida learns meta-features supporting the characterization of a (labelled) dataset. The first task consists of predicting whether two dataset patches are extracted from the same initial dataset. The second task consists of predicting whether the learning performance achieved by a hyper-parameter configuration under a fixed algorithm (ranging in k-NN, SVM, logistic regression and linear classifier with SGD) dominates that of another configuration, for a dataset extracted from the OpenML benchmarking suite. On both tasks, Dida outperforms the state of the art: DSS (Maron et al., 2020) and Dataset2Vec (Jomaa et al., 2019) architectures, as well as the models based on the hand-crafted meta-features of the literature.
Document type :
Preprints, Working Papers, ...
Complete list of metadata

https://hal.inria.fr/hal-03153200
Contributor : Herilalaina Rakotoarison <>
Submitted on : Friday, February 26, 2021 - 10:57:01 AM
Last modification on : Wednesday, April 14, 2021 - 3:40:01 AM

File

7377-32661-1-PB.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03153200, version 1
  • ARXIV : 2006.13708

Citation

Gwendoline de Bie, Herilalaina Rakotoarison, Gabriel Peyré, Michèle Sebag. Distribution-Based Invariant Deep Networks for Learning Meta-Features. 2021. ⟨hal-03153200⟩

Share

Metrics

Record views

64

Files downloads

379