Graph-based Pattern-oriented, Context-sensitive Code Completion

Anh Tuan Nguyen, Tung Thanh Nguyen, Hoan Anh Nguyen, Ahmed Tamrawi, Hung Viet Nguyen, Jafar Al-Kofahi,
Tien N. Nguyen

Video Demo Downloads

Empirical Evaluation

Accuracy in Code Completion -Results

 

We ran our evaluation tool on 24 testing subject systems (Table 1).

Accu Results
Table 1. Code Completion Accuracy Results

Meanings of columns:

  • Methods: the number of test methods,
  • Patterns: the number of recommended API usage patterns,
  • Correct: the numbers of API elements that are correctly recommended,
  • Incorrect: the numbers of API elements that are incorrectly recommended,
  • Missing: the numbers of API elements that are missing,
  • Precision, Recall, and f-score show precision, recall and f-score.

As shown, our experiment was conducted on a very large data set of 24 subject systems, with 15,188 test methods, a total of 18,313 API usage pattern. GraPacc achieves a very high level of accuracy, with up to 95% precision, 92% recall, and 93% f-score. The accumulated result shows that precision, recall, and f-score is 84.6%, 71%, and 77%, respectively.