The estimate of the value of production of software based on the number of lines of code (LOC or KLOC or SLOC) is as popular as it is controversial. The main criticism is that there are too many factors influencing the final measurement value. Robert E. Park (1992, page 140), software metrics specialist & staunch defender of the method, responded to critics with the following:

“When we hear criticism of SLOC as a software measure, we are reminded of a perhaps apocryphal story about a ditch digger who, when asked one day how he was doing, replied, “Dug seventeen feet of ditch today.” He didn’t bother to say how wide or how deep, how rocky or impenetrable the soil, how obstructed it was with roots, or even how limited he was by the tools he was using. Yet his answer conveyed information to his questioner. It conveyed even more information, we suspect, to his boss, for it gave him a firm measure to use as a basis for estimating time and cost and time to completion.”

Originally, this technique could probably be used in the conditions mentioned by the Park. Later models such as COCOMO (Boehm 1981) also allowed developers take into account a number of parameters whose variability was probably reasonable at the time. But since then, the number of factors affecting the number of lines of code has become so important that it is very unwise to take this action seriously both in the evaluation of the software and the productivity of the design team.  I will try to illustrate the problem using eight arguments.

1. Different languages and different frameworks

Today hundreds of different languages exist (Wikipedia 2013).  For each of these languages there are several frameworks.  For the same functionality, there may be a very different number of lines of code produced depending on which technology is chosen.  In addition, modern architectures use different technologies, which further complicates calculations. Correction factors exist but they hardly seem defensible given the wide variety of types of applications that are being developed today.

2. Experience and competence of the developers

We must also take into account the experience of the developers involved as this may affect the calculation in many ways. A very competent developer often writes fewer lines of code than other less experienced developers because they will use design methods created for the sole purpose of reducing the number of lines and increasing readability and maintainability. In addition, they are more competent with the functionalities offered by tools (technology stack). Indeed, through ignorance of these, many programmers rewrite existing code, greatly increasing the number of lines of code.  In this regard, many experts in the area do not hesitate to speak of “lines of code spent” as opposed to “lines of code produced” (Dijkstra 1983).

3. The practice of refactoring

The fact that the same piece of code can change over time with the refactoring (reworking of code) can skew the results. This practice reworks the source code without adding functionality (Wikipedia 2013) and it is becoming more common because it can increase code quality and reduce technical debt. This can cause unexpected situations: if many developers practice this technique while the lines of code are being measured, the result could give the appearance of a reduction in output (fewer lines of code than in the previous measurement), while it is clear that the opposite occurs.

4. The practice of reuse and / or code generation

The reuse of existing code is very common and highly recommended in DRY (Do not Repeat Yourself). So many parts of the code can be retrieved from a previous project or copied from an open source project, library or another blog post. In addition, modern development tools can automatically generate code for the developer who works with various high level design tools.

5.  Tasks outside development

Activity in the development of software is not limited to writing code on a keyboard. In fact, many other tasks are needed to produce quality code. Here, a high variability can emerge according to the different methods used, the composition of the team or the documentation requirements.

6. The reliability of the measurement tool

A wide variety of measurement tools are available on the market. Given the lack of consensus on the method of counting the amount of lines of code in a source file, the outcome may be materially different depending on the tool used.  In addition, certain technical problems can arise when it comes to identifying what should actually be counted or not. For example, some software has difficulty differentiating comments from instructions when they are mixed (Danial 2013). The efficiency and quality of those source line counter is also very variable.

7. The (potential) manipulations

When a measure may have an impact on one or more person, we need to consider the possibility that some of them try to manipulate it to their advantage. Thus, if the productivity of a developer is measured based on the number of lines of code (or functions), they could very easily manipulate the source code to inflate the results. This problem is very common in companies that use KPIs to conduct assessments of their employees. One can also easily imagine a company trying to maximise the numbers if they know they will be evaluated based on this metric.

8. Time

Almost all the above elements are time sensitive. For example, the competence of a developer does change with practice (this includes the famous learning curve). More features of languages ​​and frameworks are also evolving to increase the productivity of the developers. The longer a project takes, the longer the measurement will be sensitive to this bias

Conclusion

In conclusion we can say that estimating the production effort or value of a program using this software metric is very risky. However, this technique is widely used. Some estimate experts such as Steve McConnell (2006) are very aware of the ineffectiveness of the method but still use it in the absence of anything better. Other methods based on “function point” (business functionality) have attempted to resolve some of the issues addressed above, but the values ​​remain highly correlated with the number of lines of code (Albrecht 1983).  For me, the information obtained by these metrics, and anything based on them, should never be considered as reliable and should be used with great caution in your decision making process.

Note: Some of the information in this text come from the fruit of the research I have done for LIEU (Liaison Entreprises-Universités) a network of valorisation units of Universities and colleges of the Wallonia-Brussels federation.

References

Albrecht, A. (1983). Software Function, Source Lines of Code, and Development Effort Estimation. http://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=&arnumber=1703110&searchWithin%3Dp_Authors%3A.QT.Albrecht%2C+%2FA%2F.J..QT.%26searchWithin%3Dp_Author_Ids%3A37850740200

Boehm, B. W. (1981). Software Engineering Economics.  Englewood Cliffs, NJ. http://userfs.cec.wustl.edu/~cse528/Boehm-SE-Economics.pdf

Danial, A. (2013). CLOC Limitations. Retrieved the 2 august 2013 from http://cloc.sourceforge.net/#Limitations

Dijkstra, E. W. (1983). The fruit of misunderstanding. Retrieved the 2 august 2013 from http://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/EWD854.html

List of programming languages. (2013, July 30). In Wikipedia, The Free Encyclopedia. Retrieved 12:48, August 2, 2013, from http://en.wikipedia.org/w/index.php?title=List_of_programming_languages&oldid=566431816

McConnell, S. (2006). Software Estimation : Demystifying the Black Art.Microsoft Press. http://www.amazon.com/Software-Estimation-Demystifying-Practices-Microsoft/dp/0735605351/

Park, R. E. (1992). Software Size Measurement : A Framework for Counting Source Statements. http://www.sei.cmu.edu/reports/92tr020.pdf

Réusinage de code. (2013, juillet 5). Wikipédia, l’encyclopédie libre. Retrieved the 12:04, august 2, 2013 from  http://fr.wikipedia.org/w/index.php?title=R%C3%A9usinage_de_code&oldid=94719037