LastPyMile: Identifying the discrepancy between sources and packages

Duc Ly Vu; Fabio Massacci; Ivan Pashchenko; Henrik Plate; Antonino Sabetta

doi:10.1145/3468264.3468592

LastPyMile: Identifying the discrepancy between sources and packages

Duc Ly Vu
, Fabio Massacci
, Ivan Pashchenko
, Henrik Plate
, Antonino Sabetta

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

392 Downloads (Pure)

Abstract

Open source packages have source code available on repositories for inspection (e.g. on GitHub) but developers use pre-built packages directly from the package repositories (such as npm for JavaScript, PyPI for Python, or RubyGems for Ruby). Such convenient practice assumes that there are no discrepancies between source code and packages. These differences pose both operational risks (e.g. making dependent projects unable to compile) and security risks (e.g. deploying malicious code during package installation) in the software supply chain. Our empirical assessment of 2438 popular packages in PyPI with an analysis of around 10M lines of code shows several differences in the wild: modifications cannot be just attributed to malicious injections. Yet, scanning again all and whole most likely good but modified' packages is hard to manage for FOSS downstream users. We propose a methodology, LastPyMile, for identifying the differences between build artifacts of software packages and the respective source code repository. We show how it can be used to extend current package scanning practices for malware injection (which only covers less than 1% of the code of deployed packages).

Original language	English
Title of host publication	ESEC/FSE 2021
Subtitle of host publication	Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering
Editors	Diomidis Spinellis
Publisher	Association for Computing Machinery, Inc
Pages	780-792
Number of pages	13
ISBN (Electronic)	9781450385626
DOIs	https://doi.org/10.1145/3468264.3468592
Publication status	Published - 2021
Event	29th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2021 - Virtual, Online, Greece Duration: 23 Aug 2021 → 28 Aug 2021

Conference

Conference	29th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2021
Country/Territory	Greece
City	Virtual, Online
Period	23/08/21 → 28/08/21

Bibliographical note

Publisher Copyright:
© 2021 Owner/Author.

Funding

This research has been partly funded by the EU H2020 Programs H2020-EU.2.1.1-CyberSec4Europe (Grant No. 830929), AssureMoss (Grant No. 952647) and SPARTA project (Grant No. 830892).

Funders	Funder number
Horizon 2020 Framework Programme	952647, 830929
EU H2020 Programs H2020-EU	830892

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 17 Partnerships for the Goals

Keywords

Open source software
PyPI
Python
software supply chain

Access to Document

10.1145/3468264.3468592

LastPyMileFinal published version, 307 KBLicence: Article 25fa Dutch Copyright Act

Persistent URL (handle)

Persistent link

Cite this

Vu, D. L., Massacci, F., Pashchenko, I., Plate, H., & Sabetta, A. (2021). LastPyMile: Identifying the discrepancy between sources and packages. In D. Spinellis (Ed.), ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (pp. 780-792). Association for Computing Machinery, Inc. https://doi.org/10.1145/3468264.3468592

Vu, Duc Ly ; Massacci, Fabio ; Pashchenko, Ivan et al. / LastPyMile: Identifying the discrepancy between sources and packages. ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. editor / Diomidis Spinellis. Association for Computing Machinery, Inc, 2021. pp. 780-792

@inproceedings{255b4a0098154ec293bee0489641cc14,

title = "LastPyMile: Identifying the discrepancy between sources and packages",

abstract = "Open source packages have source code available on repositories for inspection (e.g. on GitHub) but developers use pre-built packages directly from the package repositories (such as npm for JavaScript, PyPI for Python, or RubyGems for Ruby). Such convenient practice assumes that there are no discrepancies between source code and packages. These differences pose both operational risks (e.g. making dependent projects unable to compile) and security risks (e.g. deploying malicious code during package installation) in the software supply chain. Our empirical assessment of 2438 popular packages in PyPI with an analysis of around 10M lines of code shows several differences in the wild: modifications cannot be just attributed to malicious injections. Yet, scanning again all and whole most likely good but modified' packages is hard to manage for FOSS downstream users. We propose a methodology, LastPyMile, for identifying the differences between build artifacts of software packages and the respective source code repository. We show how it can be used to extend current package scanning practices for malware injection (which only covers less than 1\% of the code of deployed packages). ",

keywords = "Open source software, PyPI, Python, software supply chain",

author = "Vu, \{Duc Ly\} and Fabio Massacci and Ivan Pashchenko and Henrik Plate and Antonino Sabetta",

note = "Publisher Copyright: {\textcopyright} 2021 Owner/Author.; 29th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2021 ; Conference date: 23-08-2021 Through 28-08-2021",

year = "2021",

doi = "10.1145/3468264.3468592",

language = "English",

pages = "780--792",

editor = "Diomidis Spinellis",

booktitle = "ESEC/FSE 2021",

publisher = "Association for Computing Machinery, Inc",

}

Vu, DL, Massacci, F, Pashchenko, I, Plate, H & Sabetta, A 2021, LastPyMile: Identifying the discrepancy between sources and packages. in D Spinellis (ed.), ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. Association for Computing Machinery, Inc, pp. 780-792, 29th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2021, Virtual, Online, Greece, 23/08/21. https://doi.org/10.1145/3468264.3468592

LastPyMile: Identifying the discrepancy between sources and packages. / Vu, Duc Ly; Massacci, Fabio; Pashchenko, Ivan et al.
ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ed. / Diomidis Spinellis. Association for Computing Machinery, Inc, 2021. p. 780-792.

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - LastPyMile: Identifying the discrepancy between sources and packages

AU - Vu, Duc Ly

AU - Massacci, Fabio

AU - Pashchenko, Ivan

AU - Plate, Henrik

AU - Sabetta, Antonino

PY - 2021

Y1 - 2021

N2 - Open source packages have source code available on repositories for inspection (e.g. on GitHub) but developers use pre-built packages directly from the package repositories (such as npm for JavaScript, PyPI for Python, or RubyGems for Ruby). Such convenient practice assumes that there are no discrepancies between source code and packages. These differences pose both operational risks (e.g. making dependent projects unable to compile) and security risks (e.g. deploying malicious code during package installation) in the software supply chain. Our empirical assessment of 2438 popular packages in PyPI with an analysis of around 10M lines of code shows several differences in the wild: modifications cannot be just attributed to malicious injections. Yet, scanning again all and whole most likely good but modified' packages is hard to manage for FOSS downstream users. We propose a methodology, LastPyMile, for identifying the differences between build artifacts of software packages and the respective source code repository. We show how it can be used to extend current package scanning practices for malware injection (which only covers less than 1% of the code of deployed packages).

AB - Open source packages have source code available on repositories for inspection (e.g. on GitHub) but developers use pre-built packages directly from the package repositories (such as npm for JavaScript, PyPI for Python, or RubyGems for Ruby). Such convenient practice assumes that there are no discrepancies between source code and packages. These differences pose both operational risks (e.g. making dependent projects unable to compile) and security risks (e.g. deploying malicious code during package installation) in the software supply chain. Our empirical assessment of 2438 popular packages in PyPI with an analysis of around 10M lines of code shows several differences in the wild: modifications cannot be just attributed to malicious injections. Yet, scanning again all and whole most likely good but modified' packages is hard to manage for FOSS downstream users. We propose a methodology, LastPyMile, for identifying the differences between build artifacts of software packages and the respective source code repository. We show how it can be used to extend current package scanning practices for malware injection (which only covers less than 1% of the code of deployed packages).

KW - Open source software

KW - PyPI

KW - Python

KW - software supply chain

UR - https://www.scopus.com/pages/publications/85116245822

UR - https://www.scopus.com/pages/publications/85116245822#tab=citedBy

U2 - 10.1145/3468264.3468592

DO - 10.1145/3468264.3468592

M3 - Conference contribution

AN - SCOPUS:85116245822

SP - 780

EP - 792

BT - ESEC/FSE 2021

A2 - Spinellis, Diomidis

PB - Association for Computing Machinery, Inc

T2 - 29th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2021

Y2 - 23 August 2021 through 28 August 2021

ER -

Vu DL, Massacci F, Pashchenko I, Plate H, Sabetta A. LastPyMile: Identifying the discrepancy between sources and packages. In Spinellis D, editor, ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. Association for Computing Machinery, Inc. 2021. p. 780-792 Epub 2021 Aug 18. doi: 10.1145/3468264.3468592

LastPyMile: Identifying the discrepancy between sources and packages

Abstract

Conference

Bibliographical note

Funding

UN SDGs

Keywords

Access to Document

Persistent URL (handle)

Other files and links

Fingerprint

Cite this