Valid interpretation of feature relevance for linear data mappings

Benoît Frenay, Daniela Hofmann, Alexander Schulz, Michael Biehl, Barbara Hammer

Research output: Contribution in Book/Catalog/Report/Conference proceedingChapter (peer-reviewed)

Abstract

Linear data transformations constitute essential operations in various machine learning algorithms, ranging from linear regression up to adaptive metric transformation. Often, linear scalings are not only used to improve the model accuracy, rather feature coefficients as provided by the mapping are interpreted as an indicator for the relevance of the feature for the task at hand. This principle, however, can be misleading in particular for high-dimensional or correlated features, since it easily marks irrelevant features as relevant or vice versa. In this contribution, we propose a mathematical formalisation of the minimum and maximum feature relevance for a given linear transformation which can efficiently be solved by means of linear programming. We evaluate the method in several benchmarks, where it becomes apparent that the minimum and maximum relevance closely resembles what is often referred to as weak and strong relevance of the features; hence unlike the mere scaling provided by the linear mapping, it ensures valid interpretability.

Original languageEnglish
Title of host publicationIEEE SSCI 2014 - 2014 IEEE Symposium Series on Computational Intelligence - CIDM 2014: 2014 IEEE Symposium on Computational Intelligence and Data Mining, Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages149-156
Number of pages8
ISBN (Print)9781479945191
DOIs
Publication statusPublished - 13 Jan 2015
Event5th IEEE Symposium on Computational Intelligence and Data Mining, CIDM 2014 - Orlando, United States
Duration: 9 Dec 201412 Dec 2014

Conference

Conference5th IEEE Symposium on Computational Intelligence and Data Mining, CIDM 2014
CountryUnited States
CityOrlando
Period9/12/1412/12/14

Fingerprint

Linear transformations
Linear regression
Linear programming
Learning algorithms
Learning systems

Cite this

Frenay, B., Hofmann, D., Schulz, A., Biehl, M., & Hammer, B. (2015). Valid interpretation of feature relevance for linear data mappings. In IEEE SSCI 2014 - 2014 IEEE Symposium Series on Computational Intelligence - CIDM 2014: 2014 IEEE Symposium on Computational Intelligence and Data Mining, Proceedings (pp. 149-156). [7008661] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CIDM.2014.7008661
Frenay, Benoît ; Hofmann, Daniela ; Schulz, Alexander ; Biehl, Michael ; Hammer, Barbara. / Valid interpretation of feature relevance for linear data mappings. IEEE SSCI 2014 - 2014 IEEE Symposium Series on Computational Intelligence - CIDM 2014: 2014 IEEE Symposium on Computational Intelligence and Data Mining, Proceedings. Institute of Electrical and Electronics Engineers Inc., 2015. pp. 149-156
@inbook{64aa7c43da93409686581e5209d10fe7,
title = "Valid interpretation of feature relevance for linear data mappings",
abstract = "Linear data transformations constitute essential operations in various machine learning algorithms, ranging from linear regression up to adaptive metric transformation. Often, linear scalings are not only used to improve the model accuracy, rather feature coefficients as provided by the mapping are interpreted as an indicator for the relevance of the feature for the task at hand. This principle, however, can be misleading in particular for high-dimensional or correlated features, since it easily marks irrelevant features as relevant or vice versa. In this contribution, we propose a mathematical formalisation of the minimum and maximum feature relevance for a given linear transformation which can efficiently be solved by means of linear programming. We evaluate the method in several benchmarks, where it becomes apparent that the minimum and maximum relevance closely resembles what is often referred to as weak and strong relevance of the features; hence unlike the mere scaling provided by the linear mapping, it ensures valid interpretability.",
author = "Beno{\^i}t Frenay and Daniela Hofmann and Alexander Schulz and Michael Biehl and Barbara Hammer",
year = "2015",
month = "1",
day = "13",
doi = "10.1109/CIDM.2014.7008661",
language = "English",
isbn = "9781479945191",
pages = "149--156",
booktitle = "IEEE SSCI 2014 - 2014 IEEE Symposium Series on Computational Intelligence - CIDM 2014: 2014 IEEE Symposium on Computational Intelligence and Data Mining, Proceedings",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

Frenay, B, Hofmann, D, Schulz, A, Biehl, M & Hammer, B 2015, Valid interpretation of feature relevance for linear data mappings. in IEEE SSCI 2014 - 2014 IEEE Symposium Series on Computational Intelligence - CIDM 2014: 2014 IEEE Symposium on Computational Intelligence and Data Mining, Proceedings., 7008661, Institute of Electrical and Electronics Engineers Inc., pp. 149-156, 5th IEEE Symposium on Computational Intelligence and Data Mining, CIDM 2014, Orlando, United States, 9/12/14. https://doi.org/10.1109/CIDM.2014.7008661

Valid interpretation of feature relevance for linear data mappings. / Frenay, Benoît; Hofmann, Daniela; Schulz, Alexander; Biehl, Michael; Hammer, Barbara.

IEEE SSCI 2014 - 2014 IEEE Symposium Series on Computational Intelligence - CIDM 2014: 2014 IEEE Symposium on Computational Intelligence and Data Mining, Proceedings. Institute of Electrical and Electronics Engineers Inc., 2015. p. 149-156 7008661.

Research output: Contribution in Book/Catalog/Report/Conference proceedingChapter (peer-reviewed)

TY - CHAP

T1 - Valid interpretation of feature relevance for linear data mappings

AU - Frenay, Benoît

AU - Hofmann, Daniela

AU - Schulz, Alexander

AU - Biehl, Michael

AU - Hammer, Barbara

PY - 2015/1/13

Y1 - 2015/1/13

N2 - Linear data transformations constitute essential operations in various machine learning algorithms, ranging from linear regression up to adaptive metric transformation. Often, linear scalings are not only used to improve the model accuracy, rather feature coefficients as provided by the mapping are interpreted as an indicator for the relevance of the feature for the task at hand. This principle, however, can be misleading in particular for high-dimensional or correlated features, since it easily marks irrelevant features as relevant or vice versa. In this contribution, we propose a mathematical formalisation of the minimum and maximum feature relevance for a given linear transformation which can efficiently be solved by means of linear programming. We evaluate the method in several benchmarks, where it becomes apparent that the minimum and maximum relevance closely resembles what is often referred to as weak and strong relevance of the features; hence unlike the mere scaling provided by the linear mapping, it ensures valid interpretability.

AB - Linear data transformations constitute essential operations in various machine learning algorithms, ranging from linear regression up to adaptive metric transformation. Often, linear scalings are not only used to improve the model accuracy, rather feature coefficients as provided by the mapping are interpreted as an indicator for the relevance of the feature for the task at hand. This principle, however, can be misleading in particular for high-dimensional or correlated features, since it easily marks irrelevant features as relevant or vice versa. In this contribution, we propose a mathematical formalisation of the minimum and maximum feature relevance for a given linear transformation which can efficiently be solved by means of linear programming. We evaluate the method in several benchmarks, where it becomes apparent that the minimum and maximum relevance closely resembles what is often referred to as weak and strong relevance of the features; hence unlike the mere scaling provided by the linear mapping, it ensures valid interpretability.

UR - http://www.scopus.com/inward/record.url?scp=84925035581&partnerID=8YFLogxK

U2 - 10.1109/CIDM.2014.7008661

DO - 10.1109/CIDM.2014.7008661

M3 - Chapter (peer-reviewed)

SN - 9781479945191

SP - 149

EP - 156

BT - IEEE SSCI 2014 - 2014 IEEE Symposium Series on Computational Intelligence - CIDM 2014: 2014 IEEE Symposium on Computational Intelligence and Data Mining, Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Frenay B, Hofmann D, Schulz A, Biehl M, Hammer B. Valid interpretation of feature relevance for linear data mappings. In IEEE SSCI 2014 - 2014 IEEE Symposium Series on Computational Intelligence - CIDM 2014: 2014 IEEE Symposium on Computational Intelligence and Data Mining, Proceedings. Institute of Electrical and Electronics Engineers Inc. 2015. p. 149-156. 7008661 https://doi.org/10.1109/CIDM.2014.7008661