Graph-based Pattern-oriented, Context-sensitive Code Completion

Anh Tuan Nguyen, Tung Thanh Nguyen, Hoan Anh Nguyen, Ahmed Tamrawi, Hung Viet Nguyen, Jafar Al-Kofahi,
Tien N. Nguyen

Video Demo Downloads

Empirical Evaluation

Usefulness in Code Completion - Results

 

Example 1
Table 1. Controlled Experiment Results

 

Table 1 shows the result. There are four cells for each task in the table because there are two subjects using GraPacc and two subjects using GCE.

Meanings of columns:

  • Ratio of Filled Code: ratio of tokens filled by Grapacc (when using Grapacc) or GCE and total tokens of code.
  • Programming Errors: number of programming errors in the submitted code.
  • Mean and Std Dev: the mean and the standard deviation of the measured data - used to compare the variations of two methods (Grapacc-GCE).

 

Implications:

  • The subjects used GraPacc and submitted code with 2 errors, while code from the subjects using GCE had in total 6 programming errors. We analyzed those for 6 programming errors and found that, two of them are errors due to the subjects' misunderstanding of the requirement, and two others are actually related to the usage of the tools.
  • The result suggests that GraPacc is more useful than GCE in programming assistance. In practice, it could help improve developers' productivity by correctly autofilling in the details of API usages, thus, reducing human errors and saving more time for them to design the algorithms or to test the code. It could also help improve code quality, because the recommended usage patterns could help developers design more efficient/reliable algorithms with better reusing of software libraries. In addition, the usage patterns also reduce the errors due to API mis-usages.
  • Some of the comments from our subjects illustrate those points:

Subject 3: “GraPacc's recommendation reminds me to use many usage patterns that I forgot for a long time”.

Subject 4: “I intended to use FileInputStream, but the tool GraPacc recommended me Scanner, which is easier to use...”.

Subject 4: “... I (would have) missed a close call, but the tool recommended me to do so...”.