Support for asynchronous functions, and skipping individual runs#3

jdmarshall wants to merge 2 commits intonowells:masternowells/benchmark-regression:masterfrom

cobblers-children:setupcobblers-children/benchmark-regression:setupCopy head branch name to clipboard