The Analysis of Deep Neural Networks by Information Theory: From Explainability to Generalization

Shujian Yu

doi:10.1609/aaai.v37i13.26829

The Analysis of Deep Neural Networks by Information Theory: From Explainability to Generalization

Shujian Yu^*

^*Corresponding author for this work

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

Abstract

Despite their great success in many artificial intelligence tasks, deep neural networks (DNNs) still suffer from a few limitations, such as poor generalization behavior for out-of-distribution (OOD) data and the “black-box” nature. Information theory offers fresh insights to solve these challenges. In this short paper, we briefly review the recent developments in this area, and highlight our contributions.

Original language	English
Title of host publication	AAAI-23 Special Programs, IAAI-23, EAAI-23, Student Papers and Demonstrations
Subtitle of host publication	Thirty-Seventh AAAI Conference on Artificial Intelligence Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence Thirteenth Symposium on Educational Advances in Artificial Intelligence
Editors	Brian Williams, Yiling Chen, Jennifer Neville
Publisher	AAAI Press
Pages	15462-15462
Number of pages	1
Volume	13
ISBN (Electronic)	9781577358800
DOIs	https://doi.org/10.1609/aaai.v37i13.26829
Publication status	Published - 2023
Event	37th AAAI Conference on Artificial Intelligence, AAAI 2023 - Washington, United States Duration: 7 Feb 2023 → 14 Feb 2023

Publication series

Name	Proceedings of the xxth AAAI Conference on Artificial Intelligence
Number	13
Volume	37

Conference

Conference	37th AAAI Conference on Artificial Intelligence, AAAI 2023
Country/Territory	United States
City	Washington
Period	7/02/23 → 14/02/23

Bibliographical note

Publisher Copyright:
Copyright © 2023, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

Access to Document

10.1609/aaai.v37i13.26829

Persistent URL (handle)

Copy link address

Cite this

Yu, S. (2023). The Analysis of Deep Neural Networks by Information Theory: From Explainability to Generalization. In B. Williams, Y. Chen, & J. Neville (Eds.), AAAI-23 Special Programs, IAAI-23, EAAI-23, Student Papers and Demonstrations: Thirty-Seventh AAAI Conference on Artificial Intelligence Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence Thirteenth Symposium on Educational Advances in Artificial Intelligence (Vol. 13, pp. 15462-15462). (Proceedings of the xxth AAAI Conference on Artificial Intelligence; Vol. 37, No. 13). AAAI Press. https://doi.org/10.1609/aaai.v37i13.26829

Yu, Shujian. / The Analysis of Deep Neural Networks by Information Theory : From Explainability to Generalization. AAAI-23 Special Programs, IAAI-23, EAAI-23, Student Papers and Demonstrations: Thirty-Seventh AAAI Conference on Artificial Intelligence Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence Thirteenth Symposium on Educational Advances in Artificial Intelligence. editor / Brian Williams ; Yiling Chen ; Jennifer Neville. Vol. 13 AAAI Press, 2023. pp. 15462-15462 (Proceedings of the xxth AAAI Conference on Artificial Intelligence; 13).

@inproceedings{53ac0579447c4e309fefe1391bdedb46,

title = "The Analysis of Deep Neural Networks by Information Theory: From Explainability to Generalization",

abstract = "Despite their great success in many artificial intelligence tasks, deep neural networks (DNNs) still suffer from a few limitations, such as poor generalization behavior for out-of-distribution (OOD) data and the “black-box” nature. Information theory offers fresh insights to solve these challenges. In this short paper, we briefly review the recent developments in this area, and highlight our contributions.",

author = "Shujian Yu",

note = "Publisher Copyright: Copyright {\textcopyright} 2023, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.; 37th AAAI Conference on Artificial Intelligence, AAAI 2023 ; Conference date: 07-02-2023 Through 14-02-2023",

year = "2023",

doi = "10.1609/aaai.v37i13.26829",

language = "English",

volume = "13",

series = "Proceedings of the xxth AAAI Conference on Artificial Intelligence",

publisher = "AAAI Press",

number = "13",

pages = "15462--15462",

editor = "Brian Williams and Yiling Chen and Jennifer Neville",

booktitle = "AAAI-23 Special Programs, IAAI-23, EAAI-23, Student Papers and Demonstrations",

}

Yu, S 2023, The Analysis of Deep Neural Networks by Information Theory: From Explainability to Generalization. in B Williams, Y Chen & J Neville (eds), AAAI-23 Special Programs, IAAI-23, EAAI-23, Student Papers and Demonstrations: Thirty-Seventh AAAI Conference on Artificial Intelligence Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence Thirteenth Symposium on Educational Advances in Artificial Intelligence. vol. 13, Proceedings of the xxth AAAI Conference on Artificial Intelligence, no. 13, vol. 37, AAAI Press, pp. 15462-15462, 37th AAAI Conference on Artificial Intelligence, AAAI 2023, Washington, United States, 7/02/23. https://doi.org/10.1609/aaai.v37i13.26829

The Analysis of Deep Neural Networks by Information Theory: From Explainability to Generalization. / Yu, Shujian.
AAAI-23 Special Programs, IAAI-23, EAAI-23, Student Papers and Demonstrations: Thirty-Seventh AAAI Conference on Artificial Intelligence Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence Thirteenth Symposium on Educational Advances in Artificial Intelligence. ed. / Brian Williams; Yiling Chen; Jennifer Neville. Vol. 13 AAAI Press, 2023. p. 15462-15462 (Proceedings of the xxth AAAI Conference on Artificial Intelligence; Vol. 37, No. 13).

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - The Analysis of Deep Neural Networks by Information Theory

T2 - 37th AAAI Conference on Artificial Intelligence, AAAI 2023

AU - Yu, Shujian

PY - 2023

Y1 - 2023

N2 - Despite their great success in many artificial intelligence tasks, deep neural networks (DNNs) still suffer from a few limitations, such as poor generalization behavior for out-of-distribution (OOD) data and the “black-box” nature. Information theory offers fresh insights to solve these challenges. In this short paper, we briefly review the recent developments in this area, and highlight our contributions.

AB - Despite their great success in many artificial intelligence tasks, deep neural networks (DNNs) still suffer from a few limitations, such as poor generalization behavior for out-of-distribution (OOD) data and the “black-box” nature. Information theory offers fresh insights to solve these challenges. In this short paper, we briefly review the recent developments in this area, and highlight our contributions.

UR - http://www.scopus.com/inward/record.url?scp=85168247018&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85168247018&partnerID=8YFLogxK

U2 - 10.1609/aaai.v37i13.26829

DO - 10.1609/aaai.v37i13.26829

M3 - Conference contribution

AN - SCOPUS:85168247018

VL - 13

T3 - Proceedings of the xxth AAAI Conference on Artificial Intelligence

SP - 15462

EP - 15462

BT - AAAI-23 Special Programs, IAAI-23, EAAI-23, Student Papers and Demonstrations

A2 - Williams, Brian

A2 - Chen, Yiling

A2 - Neville, Jennifer

PB - AAAI Press

Y2 - 7 February 2023 through 14 February 2023

ER -

Yu S. The Analysis of Deep Neural Networks by Information Theory: From Explainability to Generalization. In Williams B, Chen Y, Neville J, editors, AAAI-23 Special Programs, IAAI-23, EAAI-23, Student Papers and Demonstrations: Thirty-Seventh AAAI Conference on Artificial Intelligence Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence Thirteenth Symposium on Educational Advances in Artificial Intelligence. Vol. 13. AAAI Press. 2023. p. 15462-15462. (Proceedings of the xxth AAAI Conference on Artificial Intelligence; 13). Epub 2023 Sept 16. doi: 10.1609/aaai.v37i13.26829

The Analysis of Deep Neural Networks by Information Theory: From Explainability to Generalization

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Persistent URL (handle)

Other files and links

Fingerprint

Cite this