A unified race algorithm for offline parameter tuning

Tim Van Dijk, Martijn Mes, Marco Schutten, Joaquim Antonio Gromicho Dos Santos

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

This paper proposes uRace, a unified race algorithm for efficient offline parameter tuning of deterministic algorithms. We build on the similarity between a stochastic simulation environment and offline tuning of deterministic algorithms, where the stochastic element in the latter is the unknown problem instance given to the algorithm. Inspired by techniques from the simulation optimization literature, uRace enforces fair comparisons among parameter configurations by evaluating their performance on the same training instances. It relies on rapid statistical elimination of inferior parameter configurations and an increasingly localized search of the parameter space to quickly identify good parameter settings. We empirically evaluate uRace by applying it to a parameterized algorithmic framework for loading problems at ORTEC, a global provider of software solutions for complex decision-making problems, and obtain competitive results on a set of practical problem instances from one of the world's largest multinationals in consumer packaged goods.

Original languageEnglish
Title of host publicationProceedings of the 2014 Winter Simulation Conference, WSC 2014
PublisherInstitute of Electrical and Electronics Engineers, Inc.
Pages3971-3982
Number of pages12
Volume2015-January
ISBN (Electronic)9781479974863
DOIs
Publication statusPublished - 2015
Event2014 Winter Simulation Conference, WSC 2014 - Savannah, United States
Duration: 7 Dec 201410 Dec 2014

Conference

Conference2014 Winter Simulation Conference, WSC 2014
CountryUnited States
CitySavannah
Period7/12/1410/12/14

Fingerprint

Parameter Tuning
Tuning
Deterministic Algorithm
Simulation Optimization
Configuration
Stochastic Simulation
Simulation Environment
Parameter Space
Elimination
Decision making
Decision Making
Unknown
Software
Evaluate

Cite this

Van Dijk, T., Mes, M., Schutten, M., & Gromicho Dos Santos, J. A. (2015). A unified race algorithm for offline parameter tuning. In Proceedings of the 2014 Winter Simulation Conference, WSC 2014 (Vol. 2015-January, pp. 3971-3982). [7020222] Institute of Electrical and Electronics Engineers, Inc.. https://doi.org/10.1109/WSC.2014.7020222
Van Dijk, Tim ; Mes, Martijn ; Schutten, Marco ; Gromicho Dos Santos, Joaquim Antonio. / A unified race algorithm for offline parameter tuning. Proceedings of the 2014 Winter Simulation Conference, WSC 2014. Vol. 2015-January Institute of Electrical and Electronics Engineers, Inc., 2015. pp. 3971-3982
@inproceedings{1a85caad813242a1a884e0c830b3d7ef,
title = "A unified race algorithm for offline parameter tuning",
abstract = "This paper proposes uRace, a unified race algorithm for efficient offline parameter tuning of deterministic algorithms. We build on the similarity between a stochastic simulation environment and offline tuning of deterministic algorithms, where the stochastic element in the latter is the unknown problem instance given to the algorithm. Inspired by techniques from the simulation optimization literature, uRace enforces fair comparisons among parameter configurations by evaluating their performance on the same training instances. It relies on rapid statistical elimination of inferior parameter configurations and an increasingly localized search of the parameter space to quickly identify good parameter settings. We empirically evaluate uRace by applying it to a parameterized algorithmic framework for loading problems at ORTEC, a global provider of software solutions for complex decision-making problems, and obtain competitive results on a set of practical problem instances from one of the world's largest multinationals in consumer packaged goods.",
author = "{Van Dijk}, Tim and Martijn Mes and Marco Schutten and {Gromicho Dos Santos}, {Joaquim Antonio}",
year = "2015",
doi = "10.1109/WSC.2014.7020222",
language = "English",
volume = "2015-January",
pages = "3971--3982",
booktitle = "Proceedings of the 2014 Winter Simulation Conference, WSC 2014",
publisher = "Institute of Electrical and Electronics Engineers, Inc.",

}

Van Dijk, T, Mes, M, Schutten, M & Gromicho Dos Santos, JA 2015, A unified race algorithm for offline parameter tuning. in Proceedings of the 2014 Winter Simulation Conference, WSC 2014. vol. 2015-January, 7020222, Institute of Electrical and Electronics Engineers, Inc., pp. 3971-3982, 2014 Winter Simulation Conference, WSC 2014, Savannah, United States, 7/12/14. https://doi.org/10.1109/WSC.2014.7020222

A unified race algorithm for offline parameter tuning. / Van Dijk, Tim; Mes, Martijn; Schutten, Marco; Gromicho Dos Santos, Joaquim Antonio.

Proceedings of the 2014 Winter Simulation Conference, WSC 2014. Vol. 2015-January Institute of Electrical and Electronics Engineers, Inc., 2015. p. 3971-3982 7020222.

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - A unified race algorithm for offline parameter tuning

AU - Van Dijk, Tim

AU - Mes, Martijn

AU - Schutten, Marco

AU - Gromicho Dos Santos, Joaquim Antonio

PY - 2015

Y1 - 2015

N2 - This paper proposes uRace, a unified race algorithm for efficient offline parameter tuning of deterministic algorithms. We build on the similarity between a stochastic simulation environment and offline tuning of deterministic algorithms, where the stochastic element in the latter is the unknown problem instance given to the algorithm. Inspired by techniques from the simulation optimization literature, uRace enforces fair comparisons among parameter configurations by evaluating their performance on the same training instances. It relies on rapid statistical elimination of inferior parameter configurations and an increasingly localized search of the parameter space to quickly identify good parameter settings. We empirically evaluate uRace by applying it to a parameterized algorithmic framework for loading problems at ORTEC, a global provider of software solutions for complex decision-making problems, and obtain competitive results on a set of practical problem instances from one of the world's largest multinationals in consumer packaged goods.

AB - This paper proposes uRace, a unified race algorithm for efficient offline parameter tuning of deterministic algorithms. We build on the similarity between a stochastic simulation environment and offline tuning of deterministic algorithms, where the stochastic element in the latter is the unknown problem instance given to the algorithm. Inspired by techniques from the simulation optimization literature, uRace enforces fair comparisons among parameter configurations by evaluating their performance on the same training instances. It relies on rapid statistical elimination of inferior parameter configurations and an increasingly localized search of the parameter space to quickly identify good parameter settings. We empirically evaluate uRace by applying it to a parameterized algorithmic framework for loading problems at ORTEC, a global provider of software solutions for complex decision-making problems, and obtain competitive results on a set of practical problem instances from one of the world's largest multinationals in consumer packaged goods.

UR - http://www.scopus.com/inward/record.url?scp=84940549814&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84940549814&partnerID=8YFLogxK

U2 - 10.1109/WSC.2014.7020222

DO - 10.1109/WSC.2014.7020222

M3 - Conference contribution

VL - 2015-January

SP - 3971

EP - 3982

BT - Proceedings of the 2014 Winter Simulation Conference, WSC 2014

PB - Institute of Electrical and Electronics Engineers, Inc.

ER -

Van Dijk T, Mes M, Schutten M, Gromicho Dos Santos JA. A unified race algorithm for offline parameter tuning. In Proceedings of the 2014 Winter Simulation Conference, WSC 2014. Vol. 2015-January. Institute of Electrical and Electronics Engineers, Inc. 2015. p. 3971-3982. 7020222 https://doi.org/10.1109/WSC.2014.7020222