Advanced Computing, Mathematics and Data
Research Highlights
August 2009
Survey Reveals the State of Software Architecture Review Practice in Organizations
Results show considerable room for improvement to increase their impact
Results: Confusing. Crashing. Cursing. Three words you wish didn't apply to using software, but often do. The first large-scale research project on how companies ensure the quality of their software indicates that, while many companies review their software, considerable room exists for improving review practices to increase their impact. Overall, the results of the survey clearly show that there is much work to be done in transferring the outcomes of software architecture review research into widespread practice.
The survey, carried out by Ian Gorton, chief architect for PNNL's Data-Intensive Computing Initiative, and Muhammad Ali Babar, a senior researcher at Lero—the Irish Software Engineering Research Center, was published in the July 2009 issue of Computer.
The authors concluded that the software engineering practitioner community has yet to become fully aware of the methods and techniques available to support architecture review and their potential benefits. They urge that the community concentrate on helping practitioners by providing guidelines for justifying and institutionalizing the architecture review processes and associated supporting tools.
Of the survey results they describe, two are particularly significant for the software architecture research community. First, the survey respondents evaluate architectures mostly using informal, experience-based approaches. Second, the survey respondents rarely adopt the techniques that are highly recommended in architecture review research, such as the use of project-independent reviewers.
Why it matters: Contemporary software architecture research increasingly emphasizes the importance of having a defined review process. Architecture reviews are an effective means to ensuring design quality and addressing architectural concerns. The principal objectives of a software architecture review are to assess an architecture's ability to deliver a system capable of fulfilling the quality requirements and to identify potential risks, with the ultimate aim of ensuring a successful outcome. In other words, architecture reviews increase software quality. They help to spot faults before they become real disasters.
Methods: In their cross-sectional study, the researchers administered a questionnaire to 235 participants to obtain self-reported qualitative and quantitative information from software practitioners in a range of organizations. They designed the questionnaire based on an extensive literature review and their previous work on comparing architecture review approaches, their experiences in designing and evaluating architectures for large-scale systems, and in-depth discussions with 20 experienced architects from a range of organizations.
The questionnaire consisted of 20 questions to investigate architecture review practices and 6 questions on the demographics of the respondents and their organizations. The questionnaire underwent a rigorous review process on the format of the questions, suitability of the scales, understandability of the wording, number of questions, and length of time required by experienced software architects to complete.
Reference: Babar MA, and I Gorton. 2009. "Software Architecture Review: The State of Practice." Computer 42(7):26-32.
