Analyzing Test Completeness for Dynamic Languages

TY - GEN

T1 - Analyzing Test Completeness for Dynamic Languages

AU - Adamsen, Christoffer Quist

AU - Mezzetti, Gianluca

AU - Møller, Anders

N1 - Conference code: 25

PY - 2016/7

Y1 - 2016/7

N2 - In dynamically typed programming languages, type errors can occur at runtime. Executing the test suites that often accompany programs may provide some confidence about absence of such errors, but generally without any guarantee. We present a program analysis that can check whether a test suite has sufficient coverage to prove a given type-related property, which is particularly challenging for program code with overloading and value dependent types. The analysis achieves a synergy between scalable static analysis and dynamic analysis that goes beyond what can be accomplished by the static analysis alone. Additionally, the analysis provides a new coverage adequacy metric for the completeness of a test suite regarding a family of type-related properties. Based on an implementation for Dart, we demonstrate how such a hybrid static/dynamic program analysis can be used for measuring the quality of a test suite with respect to showing absence of type errors and inferring sound call graph information, specifically for program code that is difficult to handle by traditional static analysis techniques.

AB - In dynamically typed programming languages, type errors can occur at runtime. Executing the test suites that often accompany programs may provide some confidence about absence of such errors, but generally without any guarantee. We present a program analysis that can check whether a test suite has sufficient coverage to prove a given type-related property, which is particularly challenging for program code with overloading and value dependent types. The analysis achieves a synergy between scalable static analysis and dynamic analysis that goes beyond what can be accomplished by the static analysis alone. Additionally, the analysis provides a new coverage adequacy metric for the completeness of a test suite regarding a family of type-related properties. Based on an implementation for Dart, we demonstrate how such a hybrid static/dynamic program analysis can be used for measuring the quality of a test suite with respect to showing absence of type errors and inferring sound call graph information, specifically for program code that is difficult to handle by traditional static analysis techniques.

KW - Dart

KW - Program testing

KW - Static analysis

KW - Type checking

UR - http://www.scopus.com/inward/record.url?scp=84984906790&partnerID=8YFLogxK

U2 - 10.1145/2931037.2931059

DO - 10.1145/2931037.2931059

M3 - Article in proceedings

SP - 142

EP - 153

BT - ISSTA 2016 - Proceedings of the 25th International Symposium on Software Testing and Analysis

PB - Association for Computing Machinery

CY - New York, NY, USA

T2 - Proceedings of the 25th International Symposium on Software Testing and Analysis

Y2 - 18 July 2016 through 20 July 2016

ER -