We share our experience in developing Code Understanding Linter, an automated code review tool based on language models of code. We introduce several ideas to make the tool be more practical, including combining two different language models, filtering meaningless outputs from the model, and generating developer-friendly diagnosis messages by interpreting the outputs from the model. On top of those ideas, we describe the design and implementation of an automated code review tool to detect variable-misuse defects in Python codes and suggest how to fix them. We evaluated the tool with a set of code repositories in Samsung Electronics, which contains real-world Python codes. Our experiment proves that our tool can discover hidden defects in the real-world codes, but the false positive rate is far higher than we expected. After manually investigating every false positives, we discuss the limitations of the language models and possible solutions.

Current testing practice in industry is often ineffective and slow to detect bugs, since most projects utilize manually generated test cases. Concolic testing alleviates this problem by automatically generating test cases that achieve high coverage. However, specialized execution platforms and resource constraints of embedded software hinder application of concolic testing to embedded software. To overcome these limitations, we have developed CONcrete and symBOLic (CONBOL) testing framework to unit test large size industrial embedded software automatically. To address the aforementioned limitations, CONBOL tests target units on a host PC platform by generating symbolic unit testing drivers/stubs automatically and applying heuristics to reduce false alarms caused by the imprecise drivers/stubs. We have applied CONBOL to four million lines long industrial embedded software and detected 24 new crash bugs. Furthermore, the development team of the target software adopted CONBOL to their development process to apply CONBOL to the revised target software regularly.