Technical feasibility of a software project example PDF

WP

DIGIT B1 - EP Pilot Project 645

Deliverable 12 : Feasibility study and method for doing code reviews of free and open

source projects in European institutions

Specific contract n°226 under Framework Contract n° DI/07172 – ABCIII

July 2016

Software.

Deliverable 12: Feasibility study and method for doing code reviews of free and open source projects in

European institutions

Document elaborated in the specific context of the EU – FOSSA project.

Author:

Disclaimer

The information and views set out in this publication are those of the author(s) and do not necessarily

reflect the official opinion of the Commission. The content, conclusions and recommendations set out in

this publication are elaborated in the specific context of the EU – FOSSA project.

The Commission does not guarantee the accuracy of the data included in this study. All representations,

warranties, undertakings and guarantees relating to the report are excluded, particularly concerning – but

not limited to – the qualities of the assessed projects and products. Neither the Commission nor any person

acting on the Commission’s behalf may be held responsible for the use that may be made of the

information contained herein.

© European Union, 2016.

Reuse is authorised, without prejudice to the rights of the Commission and of the author(s), provided that

the source of the publication is acknowledged. The reuse policy of the European Commission is

implemented by a Decision of 12 December 2011.

Software.

Deliverable 12: Feasibility study and method for doing code reviews of free and open source projects in

European institutions

Document elaborated in the specific context of the EU – FOSSA project.

Software.

Deliverable 12: Feasibility study and method for doing code reviews of free and open source projects in

European institutions

Document elaborated in the specific context of the EU – FOSSA project.

List of Figures

Software.

Deliverable 12: Feasibility study and method for doing code reviews of free and open source projects in

European institutions

Document elaborated in the specific context of the EU – FOSSA project.

Acronyms and Abbreviations

EUI European Institutions

EP European Parliament

DG Directorate General

FOSS Free and Open Source Software

FOSSA Free and Open Source Software Auditing

Software.

Deliverable 12: Feasibility study and method for doing code reviews of free and open source projects in

European institutions

Document elaborated in the specific context of the EU – FOSSA project.

1 Introduction

1. Objective of this Document and Intended Audience

This document represents the deliverable 12 included within TASK-09: ‘ Draft of the Feasibility study’.

The objective of this document is to provide a mechanism to analyse the feasibility of code review projects. This deliverable takes into account the results of TASK-06: ‘ Requirement for the code reviews and their validity for the European Institutions’ , TASK-07: ‘ Analysis of the methods for communicating the results of code reviews, targeting their automated communication’, and especially TASK-08: ‘ Design of the method for performing the code reviews for the European Institutions ’.

1. Scope

To entirely understand the scope of this document, it is necessary to understand the aim of the Work Package (WP) 2. WP2 encompasses four tasks:

 Task 6: Requirements for code reviews that aim to define the list of requirements for proper code reviews and their validity for the European Institutions, as well as to prepare an analysis of how they fit into the working methods of the European Commission and the European Parliament.

 Task 7: Analysis of the methods for communicating the results of code reviews, targeting their automated communication.

Tasks 6 and 7 provide the requirements that the methodology defined in task 8 needs to fulfil. For this reason, deliverables 9 and 10 (output of tasks 6 and 7) are complementary.

 Task 8: Design of the code review process to be used in the European Institutions, taking into account the requirements defined in tasks 6 and 7.

 Task 9: Feasibility study of the method defined to perform code reviews, to be used in the European Institutions.

This document is the deliverable 12, the result of task 9 which covers the analysis of the feasibility study of the code review project, using the code review methodology described in Task 8.

Software.

Deliverable 12: Feasibility study and method for doing code reviews of free and open source projects in

European institutions

Document elaborated in the specific context of the EU – FOSSA project.

1. Deliverables

1 Deliverable 9: List of requirements for code reviews

2 Deliverable 10: List of methods for communicating the results of code reviews

3 Deliverable 11: Design of the method for performing the code reviews for the European Institutions

Software.

Deliverable 12: Feasibility study and method for doing code reviews of free and open source projects in

European institutions

Document elaborated in the specific context of the EU – FOSSA project.

2 Executive Summary

This deliverable addresses the evaluation of the feasibility of code review projects. It provides a set of mechanisms to examine the feasibility, based on the project features and particular context. This will be carried out by analysing the main project requirements (people, knowledge, tools, budget, benefits, legal issues, code for analysis and corresponding documentation, code review methodology, modules for analysis, project duration). Any code review project has limitations, namely its scope, time and cost. These limits make the difference between the project being feasible or not. The requirements are analysed using a set of factors, grouped by project aspects. Aspects are selected using the T.E.L.O approach, which comprises the following: Technical aspects, Economic aspects, Legal aspects, Operational aspects and Scheduling aspects. Technical aspects evaluate the Knowledge Factor, which analyses if the project team has the knowledge required to execute the project; and the Tool Factor, which analyses if the project team has the necessary tools to perform the project. Economic aspects evaluate the Cost Factor, which analyses the budget of the project against the Cost Limit, and the Benefit Factor, which analyses the reasons for carrying out the code review project.

Legal aspects evaluate the Tool License Factor, which analyses if the license of the code review tool adheres to the license terms.

As for the Operational aspects, several factors are evaluated: the Project Staff Factor, which analyses if the project has enough people to execute the project; the Stakeholder Factor, which analyses if the stakeholders are committed to the project; the Code and Code Documentation Factors, which analyses if the code is in the correct format for the analysis, as well as its documentation and The Formal Process Factor, which analyses if the code review project uses a formal methodology, and the Scope Factor analyses if the project scope is respected.

Finally, only one factor is evaluated in terms of Scheduling Aspects: the Time Factor, which analyses the project duration against the Time Limit.

Additionally, two more things are considered in this study: the automation of the code review and the criticality of the Open Source Software in the European Institutions. The automation of the assessment, reporting and results communication is addressed whenever the effects may improve the feasibility by reducing personnel costs, but it adds a cost for maintaining the automatic mechanism.

The Open Source Software code reviews conducted in the European Institutions will help improving the feasibility, as it will increase the benefit factor, by providing reasons to continue this as an ongoing activity

Software.

Deliverable 12: Feasibility study and method for doing code reviews of free and open source projects in

European institutions

Document elaborated in the specific context of the EU – FOSSA project.

3. Definition of the Feasibility Factors to take into Account for the

Analysis

In this section, we will provide a set of factors to assess the feasibility of the project, as well as the limitations. We obtained said factors by means of analysing the common features and needs of any code review project. These features and needs are as follows

 The team that will execute the project.

 The knowledge required to perform the project.

 The tools to use during the code review.

 The project budget and cost limit.

 The benefits of the project (the justification for carrying it out).

 The legal issues that could arise.

 The project input, the code for analysis and its documentation.

 A methodology to conduct the code review in an organised and formal way.

 The time required to execute the project, as well as the time limit.

The feasibility factors are a way to ensure that the code review project has everything that is needed to be feasible, and they are aligned with the needs and features mentioned in this subsection.

Figure 3. Code Review Project

Software.

Deliverable 12: Feasibility study and method for doing code reviews of free and open source projects in

European institutions

Document elaborated in the specific context of the EU – FOSSA project.

3. Case Study for Analysis

The design of the code review has to be coherent and as realistic as possible. The information contained in the case study is related to the needs and features described in the previous subsection. This information comprises the description of a code review project:

 The team that is going to execute the project: the number of the people, their skills and knowledge, and their economic rates.

 The context of the code review project: the tools, the code review methodology, the time, the input of the project (code and its documentation), and the benefits.

 The economic aspects and the budget for the project.

3.3 of the Case Study and Discussion about the Feasibility

Factors

The analysis will determine if the feasibility factors are fulfilled in the case study by analysing the information and context. Furthermore, the impact of these factors on the feasibility of the project is also analysed, as well as plausible solutions to fulfil them if they do not meet the requirements.

The goals of this analysis are:

 To provide an example of a feasibility study.

 To analyse the impact of the factors on the feasibility of the project, as well as their interdependencies.

 To provide a way to make a project feasible by modifying the feasibility factors.

Software.

Deliverable 12: Feasibility study and method for doing code reviews of free and open source projects in

European institutions

Document elaborated in the specific context of the EU – FOSSA project.

4.2 Aspects

Technical factors aim to evaluate the requirements to accomplish the code review from a technical point of view. These technical factors are determined by the technologies and tools necessary to execute the code review. They are as follows:

 Knowledge Factor : This factor analyses two different types of knowledge: the analysed programming languages and the security concepts of the code. This knowledge can be contained in the code review method documents, and/or be held by code review staff. This factor is fulfilled when the code review project team meets the knowledge benchmark desired. This degree of knowledge is required to execute all the project tasks (being able to examine any code and detect security flaws). It also evaluates the project team skills that are required to carry out the project.

 Tool Factor : The tool factor aims to analyse if the code review project team has the required tools to perform the code review, and if they are correctly installed and configured. This factor is satisfied if the code review project has the necessary tools and they are ready to use.

4.3 Aspects

Economic factors evaluate the economic aspect of the project. Costs and benefits are the two factors to consider, and the way of analysing them is different.

While costs result from the sum of all project resources, benefits are quite hard to determine. Making estimations to quantify the benefits of any preventive measure is difficult, so the approach will consist in listing the benefits without a numeric estimation.

 Costs Factor : this factor evaluates the code review project costs, and it is an important element since the feasibility is calculated by adding up the different costs of the project (e. staff, testing tool, logistics, and any other costs that can be generated during the project execution). This factor is fulfilled when the total cost remains below the Cost Limit established.

 Benefits Factor : this factor addresses the justification for the execution of the project, and provides information about the benefits resulting from it. Due to the complexity of measuring the benefits of the project, the approach selected will be qualitative instead of quantitative.

This factor is fulfilled if the project results achieve one or more of the following objectives:

o Ensure that the software developed by third-party organisations (FOSS communities, software providers, contractors) is free of backdoors or malicious code.

o Mitigate the risk of the software components or libraries by detecting vulnerabilities, allowing to choose the most secure option among the software libraries or components available.

o Improve the security of the software applications under development.

Software.

Deliverable 12: Feasibility study and method for doing code reviews of free and open source projects in

European institutions

Document elaborated in the specific context of the EU – FOSSA project.

o Help FOSS communities to improve the security of their software, thus providing more secure FOSS software for EU citizens.

4.4 Aspects

Legal factors aim to analyse the legal aspects of the code review project and their potential effects.

Due to the nature of this kind of projects, legal aspects have a minor effect on the feasibility of the project, because they are related to the software license of the testing tools.

 Tool License Factor: It aims to evaluate the software licenses of the tools. The tool involved in the code review has to adhere to the relevant licensing terms. If it is a proprietary tool, the license cost also has to be considered within the cost factor. This factor is fulfilled when the tool adheres to the license terms.

4.5 Aspects

Operational Factors address the evaluation of the effects that operational aspects could incur in code review projects. Any project needs a set of elements; the project input, the processes to transform said input into the project output, and the people to perform the tasks. This is why these factors are related to people, project inputs and project processes.

The factors related to specific individuals are:

 Project Staff Factor: It analyses whether the project team includes the required number of people to execute the project. This analysis has to take into consideration the number of people, not their skills (which were already analysed in the Knowledge Factor ). This factor is fulfilled when the project team has the required number of headcounts to execute the code review project.

 Stakeholder Factor: It analyses if the project has been able to engage key people in the project (e. software owners, sponsor (DIGIT), project managers), in addition to the code reviewers. They are necessary as they have to provide the software documentation and be available if any other requirements are needed. In the event that a critical vulnerability is detected, they will be contacted to determine how to approach the solution. This factor takes into account the engagement and commitment of the project stakeholders, who are necessary for the success of the project. This factor is fulfilled when those people are actively involved in the code review project.

With regard to the project input there are two factors to consider:

 Code Factor : Since this is a code review project, the code to analyse is the project input. This factor is fulfilled when the code is available in the proper format (source code, etc.) to perform the code review.

 Code Documentation Factor : It evaluates if the documentation of the code is available. Depending on the scope of the code review, the documentation required may be different in terms of module design or in the description of some functionalities. This factor is fulfilled once the code review team has been provided with that documentation.

Software.

Deliverable 12: Feasibility study and method for doing code reviews of free and open source projects in

European institutions

Document elaborated in the specific context of the EU – FOSSA project.

5 Case Study for Analysis

To assess the feasibility of the code review method proposed in Deliverable 11: Design of the method for performing the code reviews for the European Institutions , we will use a case study with non-real data as a guide for future analyses. In order to explain how the study has to be conducted, the use of real data is not relevant because the objective is to describe the evaluation process.

5.1 for the Case Study

In this section we will introduce the case study, as well as the main aspects that impact the feasibility. This information will be analysed according to the factors presented in the previous section.

 Software : The software that is going to be analysed is an Open Source Java application, composed of 4 modules (‘A’, ‘B’, ‘C’, ‘D’).

o Module ‘A’ has 4 Java packages, and 12 Java classes on average per Java package.

o Module ‘B’ has 5 Java packages and 7 Java classes per package.

o Module ‘C’ has 5 Java packages, and an average of 12 Java classes per package.

o Module ‘D’ has 6 Java packages, and 11 Java classes per package.

o For each Java class, the average size of the code is 10 0 lines.

Table 1: Description of the Software Under Test

Software Under Test

Modules ‘A’ ‘B’ ‘C’ ‘D’

Packages 4 5 5 6

Classes per Package 12 7 12 11

Total of Java Classes per Module

####### 48 35 60 66

To sum up, the application has a total amount of 209 classes and 20,900 code lines. The source code will be provided to the code review team, as well as the assembled version of the code.

 Code Review Team : The team is made up of 2 analysts and an external security consultant to provide knowledge in relation to cybersecurity. This team has all the knowledge required about java and secure coding. The team will be managed by a Project Manager, who will dedicate 10% of their time to the project.

Software.

Deliverable 12: Feasibility study and method for doing code reviews of free and open source projects in

European institutions

Document elaborated in the specific context of the EU – FOSSA project.

Due to the diversity of personnel rates in the European Union, we estimate the team member rate at CostDay (variable), regardless of the team member position. This rate also includes indirect costs.

The team will use the methodology defined in Deliverable 11: Design of the method for performing the code reviews for the European institutions , and the open source tool ‘VisualCodeGrepper (VCG)’ [3]. This tool will be correctly installed and configured to execute the code review.

 Stakeholders: The main stakeholders for this code review project will be the FOSS community that owns the application and DIGIT, who manages the code review project. The reason for this twofold engagement is that FOSS software is used as a critical component in some of the European Commission’s applications. Furthermore, the FOSS community is interested in the project and committed to pursuing its success. They provide the documentation about the software under test via the FOSS community website.

 Code Review Project : The objective of the project is to analyse the application modules ‘A’, ‘B’ and ‘D’ using the ‘Defined mode’ of the code review method introduced in Deliverable 11: Design of the method for performing the code reviews for the European Institutions. Model ‘C’ will also be analysed, but using the ‘Optimised mode’.

According to the previous data, 149 classes will be analysed using the ‘Defined mode’, and 60 classes using the ‘Optimised mode’.

For this purpose, the analysis of controls is conducted as follows: when several (e. two or three) pieces of evidence are detected for a specific control, the control fails. The control will also recommend reviewing the rest of the module for that specific error. The reason is to avoid time-consuming tasks that do not provide value. Furthermore, not all the controls are applicable to all the modules and classes.

In this case study, all modules and classes are going to be analysed because it is a small application. Nevertheless, the amount of efforts and resources to analyse the large ones may not be affordable.

In the event that a big application has to be analysed, an efficient option would be to limit the scope by reducing the portion of the code to analyse especially in the manual review (‘Defined mode’ and ‘Optimised mode’). If reducing the scope is required, in our experience, between 5% to 15% of the code suffices (it is a significant amount of code from a security point of view, taking into consideration input / output management, user management, error management, file management, cryptography, etc.) as a trustworthy sample to review the most critical aspects of the application manually.

The estimated time for code analysis depends on several factors, namely:

o The mode selected from the methodology (‘Managed mode’, ‘Defined mode’, or ‘Optimised mode’).

o The documentation of the code (low quality, medium quality, high quality).

o The complexity of the code (low, medium, high).

o The difficulty to analyse the control. Some controls are faster to review than others.

What is technical feasibility of a software project?

What is an example of technical feasibility?

How do you write a technical feasibility project?

What is technical feasibility in SDLC?