Best Practices in Software Benchmarking 2016 (#bench16)
Overview
For computer scientists and software engineers, benchmarking (evaluating the running time of a piece of software, or the performance of a piece of hardware) is a common method for evaluating new techniques. However, there is little agreement on how benchmarking should be carried out, how to control for confounding variables, how to analyse latency data, or how to aid the repeatability of experiments. This workshop will be a venue for computer scientists and research software engineers to discuss their current best practices and future directions.
On this page you can find an agenda, directions for travel, availability of wifi access for attendees, and information about our sponsors and organisers.
Registration is now closed.
Agenda
| Time | Session | Speaker |
|---|---|---|
| 09:30 -- 10:00 | Arrival and coffee | |
| 10:00 -- 10:05 | Welcome | Sarah Mount |
| 10:05 -- 11:00 | Keynote talk: Repeatability, Reproducibility and Rigor [slides] | Jan Vitek |
| 11:00 -- 11:30 | Morning break with refreshments | |
| 11:30 -- 12:00 | Why Aren't We Benchmarking Bioinformatics? [slides] | Joe Parker |
| 12:00 -- 12:30 | Introduction to Time Series Data and Analysis [slides] | Simon Taylor |
| 12:30 -- 14:00 | Lunch | |
| 14:00 -- 14:30 | Evaluating Performance using Ratio of Execution Times [slides] | Tomáš Kalibera |
| 14:30 -- 15:00 | The Role of Benchmarking in Buying a High-Performance Computer [slides] | James Davenport |
| 15:00 -- 15:45 | Afternoon break with refreshments | |
| 15:45 -- 16:15 | What Exactly do we Mean by VM Warmup? [slides] | Edd Barrett |
| 16:15 -- 16:45 | Scalability in compiler development: how to get testing and optimization done in a reasonable time [slides] | Jeremy Bennett |
| 16:45 -- 17:00 | Concluding discussion | |
| 18:00 -- late | Dinner at Fire and Stone, 31/32 Maiden Lane, WC2E 7JS |
Travel
King's College London is easily accessible from Central London. Full directions to Strand Campus can be found on the College website.
#bench16 will be held in room K2.31 (King's Building, 2nd floor, room 31) which is also called the Nash Lecture Theatre -- both names are printed on the door itself.
K2.31 will be clearly signposted from the main reception. If you have any problems finding the room, please asking the receptionists, who are friendly and knowledgeable about how to get around the campus.
WiFi access at King's
#bench16 attendees from UK academic institutions can use their Eduroam account during the event (SSID: eduroam). Visitors without an Eduroam account may use the free Cloud service (SSID: The Cloud).
Sponsorship
We have been able to keep #bench16 free to attend, thanks to the generous sponsorship of the Software Sustainability Institute. The Software Sustainability Institute cultivates world-class research with software. The Institute is based at the universities of Edinburgh, Manchester, Southampton and Oxford.
Organisation
If you have any questions at all about #bench16, please don't hesitate to contact us.
| Organisers: | Sarah Mount, Laurence Tratt |
| Email: | info@soft-dev.org |
| Twitter: | @benchmarking16 |
