Can a Transformer Assist in Scientific Writing? Generating Semantic Web Paper Snippets with GPT-2

GPT-2

doi:10.1007/978-3-030-62327-2_27

Can a Transformer Assist in Scientific Writing? Generating Semantic Web Paper Snippets with GPT-2

GPT-2

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

234 Downloads (Pure)

Abstract

The Semantic Web community has produced a large body of literature that is becoming increasingly difficult to manage, browse, and use. Recent work on attention-based, sequence-to-sequence Transformer neural architecture has produced language models that generate surprisingly convincing synthetic conditional text samples. In this demonstration, we re-train the GPT-2 architecture using the complete corpus of proceedings of the International Semantic Web Conference since 2002 until 2019. We use user-provided sentences to conditionally sample paper snippets, therefore illustrating cases where this model can help at addressing challenges in scientific paper writing, such as navigating extensive literature, explaining the Semantic Web core concepts, providing definitions, and even inspiring new research ideas.

Original language	English
Title of host publication	The Semantic Web
Subtitle of host publication	ESWC 2020 Satellite Events, Heraklion, Crete, Greece, May 31 – June 4, 2020, Revised Selected Papers
Editors	Andreas Harth, Valentina Presutti, Raphaël Troncy, Maribel Acosta, Axel Polleres, Javier D. Fernández, Josiane Xavier Parreira, Olaf Hartig, Katja Hose, Michael Cochez
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	158-163
Number of pages	6
ISBN (Electronic)	9783030623272
ISBN (Print)	9783030623265
DOIs	https://doi.org/10.1007/978-3-030-62327-2_27
Publication status	Published - 2020
Event	17th Extended Semantic Web Conference, ESWC 2020 - Heraklion, Greece Duration: 31 May 2020 → 4 Jun 2020

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12124 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	17th Extended Semantic Web Conference, ESWC 2020
Country/Territory	Greece
City	Heraklion
Period	31/05/20 → 4/06/20

Funding

Acknowledgements. “This paper would not have been possible without the support of several persons and institutions. GPT-2 would like to thank” all of the members of the GPT-1 technical committee. The authors want to thank Frank van Harmelen, Paul Groth, and the anonymous reviewers for their valuable comments. This work is partly supported by the CLARIAH project funded by NWO.

Keywords

Natural language generation
Scholarly communication
Semantic Web papers

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1007/978-3-030-62327-2_27

Can_a_Transformer_Assist_in_Scientific_Writing__Generating_Semantic_Web_Paper_Snippets_with_GPT_2Accepted author manuscript, 299 KBLicence: Article 25fa Dutch Copyright Act
Can a Transformer Assist in Scientific Writing Generating Semantic Web Paper Snippets with GPT-2Final published version, 192 KBLicence: Article 25fa Dutch Copyright Act

Persistent URL (handle)

Persistent link

Cite this

GPT-2 (2020). Can a Transformer Assist in Scientific Writing? Generating Semantic Web Paper Snippets with GPT-2. In A. Harth, V. Presutti, R. Troncy, M. Acosta, A. Polleres, J. D. Fernández, J. Xavier Parreira, O. Hartig, K. Hose, & M. Cochez (Eds.), The Semantic Web: ESWC 2020 Satellite Events, Heraklion, Crete, Greece, May 31 – June 4, 2020, Revised Selected Papers (pp. 158-163). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12124 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-62327-2_27

GPT-2. / Can a Transformer Assist in Scientific Writing? Generating Semantic Web Paper Snippets with GPT-2. The Semantic Web: ESWC 2020 Satellite Events, Heraklion, Crete, Greece, May 31 – June 4, 2020, Revised Selected Papers. editor / Andreas Harth ; Valentina Presutti ; Raphaël Troncy ; Maribel Acosta ; Axel Polleres ; Javier D. Fernández ; Josiane Xavier Parreira ; Olaf Hartig ; Katja Hose ; Michael Cochez. Springer Science and Business Media Deutschland GmbH, 2020. pp. 158-163 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{6ef6f760f50c44c0a66cf27b41f5059e,

title = "Can a Transformer Assist in Scientific Writing? Generating Semantic Web Paper Snippets with GPT-2",

abstract = "The Semantic Web community has produced a large body of literature that is becoming increasingly difficult to manage, browse, and use. Recent work on attention-based, sequence-to-sequence Transformer neural architecture has produced language models that generate surprisingly convincing synthetic conditional text samples. In this demonstration, we re-train the GPT-2 architecture using the complete corpus of proceedings of the International Semantic Web Conference since 2002 until 2019. We use user-provided sentences to conditionally sample paper snippets, therefore illustrating cases where this model can help at addressing challenges in scientific paper writing, such as navigating extensive literature, explaining the Semantic Web core concepts, providing definitions, and even inspiring new research ideas.",

keywords = "Natural language generation, Scholarly communication, Semantic Web papers",

author = "Albert Mero{\~n}o-Pe{\~n}uela and Dayana Spagnuelo and GPT-2",

year = "2020",

doi = "10.1007/978-3-030-62327-2\_27",

language = "English",

isbn = "9783030623265",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "158--163",

editor = "Andreas Harth and Valentina Presutti and Rapha{\"e}l Troncy and Maribel Acosta and Axel Polleres and Fern{\'a}ndez, \{Javier D.\} and \{Xavier Parreira\}, Josiane and Olaf Hartig and Katja Hose and Michael Cochez",

booktitle = "The Semantic Web",

address = "Germany",

note = "17th Extended Semantic Web Conference, ESWC 2020 ; Conference date: 31-05-2020 Through 04-06-2020",

}

GPT-2 2020, Can a Transformer Assist in Scientific Writing? Generating Semantic Web Paper Snippets with GPT-2. in A Harth, V Presutti, R Troncy, M Acosta, A Polleres, JD Fernández, J Xavier Parreira, O Hartig, K Hose & M Cochez (eds), The Semantic Web: ESWC 2020 Satellite Events, Heraklion, Crete, Greece, May 31 – June 4, 2020, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12124 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 158-163, 17th Extended Semantic Web Conference, ESWC 2020, Heraklion, Greece, 31/05/20. https://doi.org/10.1007/978-3-030-62327-2_27

Can a Transformer Assist in Scientific Writing? Generating Semantic Web Paper Snippets with GPT-2. / GPT-2.
The Semantic Web: ESWC 2020 Satellite Events, Heraklion, Crete, Greece, May 31 – June 4, 2020, Revised Selected Papers. ed. / Andreas Harth; Valentina Presutti; Raphaël Troncy; Maribel Acosta; Axel Polleres; Javier D. Fernández; Josiane Xavier Parreira; Olaf Hartig; Katja Hose; Michael Cochez. Springer Science and Business Media Deutschland GmbH, 2020. p. 158-163 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12124 LNCS).

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Can a Transformer Assist in Scientific Writing? Generating Semantic Web Paper Snippets with GPT-2

AU - Meroño-Peñuela, Albert

AU - Spagnuelo, Dayana

AU - GPT-2

PY - 2020

Y1 - 2020

N2 - The Semantic Web community has produced a large body of literature that is becoming increasingly difficult to manage, browse, and use. Recent work on attention-based, sequence-to-sequence Transformer neural architecture has produced language models that generate surprisingly convincing synthetic conditional text samples. In this demonstration, we re-train the GPT-2 architecture using the complete corpus of proceedings of the International Semantic Web Conference since 2002 until 2019. We use user-provided sentences to conditionally sample paper snippets, therefore illustrating cases where this model can help at addressing challenges in scientific paper writing, such as navigating extensive literature, explaining the Semantic Web core concepts, providing definitions, and even inspiring new research ideas.

AB - The Semantic Web community has produced a large body of literature that is becoming increasingly difficult to manage, browse, and use. Recent work on attention-based, sequence-to-sequence Transformer neural architecture has produced language models that generate surprisingly convincing synthetic conditional text samples. In this demonstration, we re-train the GPT-2 architecture using the complete corpus of proceedings of the International Semantic Web Conference since 2002 until 2019. We use user-provided sentences to conditionally sample paper snippets, therefore illustrating cases where this model can help at addressing challenges in scientific paper writing, such as navigating extensive literature, explaining the Semantic Web core concepts, providing definitions, and even inspiring new research ideas.

KW - Natural language generation

KW - Scholarly communication

KW - Semantic Web papers

UR - https://www.scopus.com/pages/publications/85097303310

UR - https://www.scopus.com/inward/citedby.url?scp=85097303310&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-62327-2_27

DO - 10.1007/978-3-030-62327-2_27

M3 - Conference contribution

AN - SCOPUS:85097303310

SN - 9783030623265

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 158

EP - 163

BT - The Semantic Web

A2 - Harth, Andreas

A2 - Presutti, Valentina

A2 - Troncy, Raphaël

A2 - Acosta, Maribel

A2 - Polleres, Axel

A2 - Fernández, Javier D.

A2 - Xavier Parreira, Josiane

A2 - Hartig, Olaf

A2 - Hose, Katja

A2 - Cochez, Michael

PB - Springer Science and Business Media Deutschland GmbH

T2 - 17th Extended Semantic Web Conference, ESWC 2020

Y2 - 31 May 2020 through 4 June 2020

ER -

GPT-2. Can a Transformer Assist in Scientific Writing? Generating Semantic Web Paper Snippets with GPT-2. In Harth A, Presutti V, Troncy R, Acosta M, Polleres A, Fernández JD, Xavier Parreira J, Hartig O, Hose K, Cochez M, editors, The Semantic Web: ESWC 2020 Satellite Events, Heraklion, Crete, Greece, May 31 – June 4, 2020, Revised Selected Papers. Springer Science and Business Media Deutschland GmbH. 2020. p. 158-163. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). Epub 2020 Nov 11. doi: 10.1007/978-3-030-62327-2_27

Can a Transformer Assist in Scientific Writing? Generating Semantic Web Paper Snippets with GPT-2

Abstract

Publication series

Conference

Funding

Keywords

UN SDGs

Access to Document

Persistent URL (handle)

Other files and links

Fingerprint

Cite this