|[Home] [Credit Search] [Category Browser] [Staff Roll Call]||The LINUX.COM Article Archive|
|Originally Published: Monday, 20 November 2000||Author: Kevin Cullis|
|Published to: featured_articles/Featured Articles||Page: 1/1 - [Printable]|
Software Quality 101: Essential Concepts
We carry a lot of content on Linux.com, and a large portion of it has to do with software development. While most of it deals with linkers, compilers and programming languages, our weekly feature this week is written from the quality assurance aspect. Kevin Cullis has been in QA for years, and although he comes from the business perspective, he makes a few extremely valid points to the hacker community.
|Page 1 of 1|
Linux and the quality of Linux have become the topic of late in the media, both print and on the Web. As a two-year user, I find that Linux has met most of my computing needs. However, if Linux wants to increase the user base, improvements in the developmental process are needed in the operating system and software. Linux software needs to solve problems by providing solutions not only to the developer, but also to the business and hobby users.
Linux solved my reliability problems, but created other problems in usability and software availability. These should be addressed in the near future. Linux developers and users in the community need to realize that the process developed determines the level of quality it will achieve. Below are some key concepts that may help Linux increase the rate of world domination.
Short History of Quality
Since the first incidence of a broken bow in the pursuit of early Man's next meal, the concept of quality has been important. With the advent of mass production, the level of quality has become increasingly important, especially in the past 30 years. In the 1920's, Dr. Walter Shewhart designed the first Control chart for the manufacturing process, and later with Dr. W. Edwards Deming's and other people's help, a new approach to quality improvement was born. Since then, Total Quality Management, Process Improvement, Quality Assurance, Quality Control, Kaizen, Reengineering, Quality Improvement, Quality is Job One, and Six Sigma Quality are all "programs" started by various organizations to improve products and services for their customers. The key point in each initiative is continuously improving the quality of a product or service over time to achieve reduced costs, increases in sales or adoption, and ultimately, the ability to stay in business. The focus is always the same: getting better at business! The "how to do it" will depend on people and the skills and knowledge they possess and will learn, the trust, understanding, and communication levels of all those involved, and the market. However, in most cases, quality takes a back seat to features. If Linux doesn't keep pace with improvements in all phases of software development, Microsoft or the Mac OS will!
Some organizations in the past have made their process improvement program work. They include: Ford's Quality is Job One, Tata Consultancy Services of India achieving Software Engineering Institute's Capability Maturity Model's Level 5 certification, Toyota's Total Production System, Motorola's Six Sigma, or the Ritz-Carlton winning the Malcolm Baldrige National Quality Award a second time. Others have spent lots of time and money and never got it "right." A recent article in American Society for Quality Software revealed that for every $1 invested in process improvements, there resulted a $4-7 return in savings. Regardless of successes or "failures," the principle of process improvement still applies. When you improve the process, you increase productivity with the result of reducing costs, increasing sales and profits.
Today's software industry attitude as a whole is reminiscent of Detroit during the 80's when Japanese auto quality began eating away at the American automobile market share. While Linux may not have the exact same quality issues as other systems, evidence shows that new skills and habits need to be learned by the IT industry in order to reach the next level of quality.
Why should an organization focus on a Process Improvement Strategy? Where do they focus their effort? What does it take to move forward? First things first:
There are three parts of the "Big Picture" to providing a solution and solving a problem: Product, Process, and People. These comprise an organization's internal customer system and the external system consists of their customers and suppliers. People design and oversee the processes put into place that produce the products or services to be sold. The customer's expectations determine whether the product or service adds value, i.e. solves a problem, which then becomes a sale. It can also be seen in the "small picture" as well: a small applet, which works with GNOME or KDE, such as Xmcd.
Level of Quality and Market
Cost of Quality
A process is a step by step, thought out design and/or course of action to produce an intended result and to ensure consistency of quality. A process can also be considered a mathematical formula, such as 1+1+1+1 = 4. While the formula and the answer are precise, the results will always be the same given the exact same inputs and operators. The expectations are the same within a company. However, when the operators or inputs are changed, much like a person getting sick or the computer goes down, the results will change. As long as the process doesn't change, the results will always give the same answer (see Variation below for more info).
One of the most crucial principles of process design is that in today's business climate, tasks or jobs are no longer the benchmark of business, but process design have taken their place. Smart companies ensure that workers focus on contributing to processes, not doing a job.
Both common and special causes can be eliminated, but special causes are the most difficult because of their unpredictability, such as lightning strike or an earthquake. When considering variations, focus on why it occurred, and what is the root cause (not the symptom) of the variation. Is it predictable and can it be eliminated? Whatever the case, the question is why problems occur; the goal is to find the root cause of the variation and reduce or eventually eliminate it.
An additional point of a process improvement is that it should include traceability. Traceability is the documentation of a process so that assurance of quality is achieved; there is the prevention of errors over time, and to simplify the investigation of product or services failures. Documenting the traceability of a process allows people to discover loose ends of a process and reduce or eliminate non-value-added effort that could affect the outcome of the process. While it might seem that the process moves slower, in the long term it prevents heroics, maintains consistency of production, and causes a ripple affect throughout an organization as to what affect changes have on the rest of the organization, i.e. changes to the system.
Metrics and Statistics
While metrics are the measurement of a process, statistics are the quantification of the measurement data for the basis of inference and induction. Statistics do not have to be ANOVA, Piosson, or Weibull statistics, but can be very simple counts, maximums, averages, or minimums used in a standard checklist. I once talked with a Quality Director and he told me not to talk about statistics because a Big Eight Accounting firm had scared them on the cost of the process improvement effort. Needless to say, if you don't measure anything, how do you know if you're getting better? In addition, when designing metrics, there needs to be an emphasis not only on what the statistics say, but consideration must also be given to what they do not say, such as customer satisfaction example above. The collection of data is for the clarification of a problem or opportunity and insure that it represents typical conditions and reveal the desired information, not what is easy to obtain. Collection of data is easy, sifting through the data to find the gold mine of information that can be translated into a competitive advantage can be time consuming, but well worth the effort once it is known.
Here are some actual numbers of a company's server farm and the metrics, which were used in analysis (while Unix is the actual OS by this company, Linux can be equally substituted for greater potential savings).
Now, most managers would look only at the salary of an individual and say that it's cheaper to run NT than Unix. In Quality Assurance circles, this type of focus is called sub-optimization, only looking at the smallest part and not looking at how it performs within the total system. This is the prevalent thought pattern people who would fight tooth and nail to make sure costs are cheaper and cheaper. In QA circles, this is focusing on COST only, not productivity! Let's look at the REAL numbers and the administrator's salaries.
So, one Unix administrator takes care of 15 machines versus an NT administrator takes care of six machines. Now, let's look at the total cost of the salaries. Now, if you were to take the 1:15 ratio and convert the NT servers to Unix, what would the cost be (180 servers divided by the ratio of 1:15 administrators):
(n) = new
Botton line: The difference (savings) between the NT and Unix (new) is $600,000 in first year savings alone! In addition, that's a 50% increase in a person's salary! This is not taking into consideration that the NT servers may not be able to handle the traffic as the Unix systems can and therefore fewer than 180 may be needed. This does not also consider security issues, etc. but is only used to address metrics and analysis and to show fallacies, which are strictly adhered to (Your mileage may vary!).
In Example 3, the presence of software defects is mostly concentrated in the coding of the development process. For this example, 138 defects were found, while the other processes have less than 40 defects each. Upon investigation, we find that the bottleneck is in the coding. While improvements can be made on other processes within the system, most of the improvements won't directly apply to the overall production of the system, hence the necessity for documenting all of the company's processes and knowing the location of the system's bottleneck. If production is increased in coding with the reduction of defects, then focus can be placed on other areas. However, if the reduction of software defects in coding is reduced to under 30, then the next area to focus on is testing. The most interesting thing is that without the documentation and measurement of a company's processes, the managers of an organization may be making good decisions on bad or incomplete information. For example, if there were no measurements of defects in the compile process and the true measurement of defects were around 60, after the reduction of defects in coding from 138 to 40, then the effort would wrongly focus on testing and would waste valuable time and effort.
Implied in a system are two factors: definitions of the process and the demarcation of the processes. Both are essential in helping to determine the location and corrective action necessary to improve quality.
Effective versus Efficient
Process Improvement Skills and Habits
Ownership of the Process
Agreeably, not everyone wants to be an owner of a process, but if the question is not asked of the person doing the job, the answer will never be known and the opportunity to improve can be lost. Ownership of the process does require responsibility and some may not be ready or willing to accept it, for whatever reason. Given time, most will come around when they're ready.
Levels of Empowerment
During an interview with the Quality Director of the Ritz-Carlton in Aspen, Colorado, for a Master's thesis, the Quality Director described learning about empowerment within their organization. They surveyed all of their front line employees, cooks, cleaning crews, and others and found that 90% of the decisions they would make solving a customer's problem were the same ones they were coming to the supervisor to help solve. So, the hotel chain empowered each employee up to $2,000 to solve a customer's problem and any amount above that would require the supervisor's involvement. With the supervisors now having more free time, they were not fired or let go, but allowed to concentrate on more strategic concerns and quality improvement efforts which added to the esteem of the workers and the supervisors. As one gets more experience, more responsibility and empowerment is allowed.
Process Improvement is not a Witch Hunt. If at anytime a process improvement program becomes a witch-hunt, the efforts toward improvement will be hindered. Why? Because most people don't come to work with the express purpose of failing. Most want to do a good job and just need coaching to improve. Managers should blame the process first before ever blaming the process owners.
The evidence is overwhelming that process improvement is a necessity for organizations, not only to remain in business, but the remain competitive in the marketplace. Training is first, or the understanding of the why of process improvement. People require a complete understanding of a process improvement philosophy and should obtain training in these new skills and habits. Tools and techniques are second, or understanding the how of process improvement. These must be understood to do the right job with the right tools. Time is last, or can be considered the reallocation of "recovered" resources when there is an "investment" of resources up front. It is important that time be applied to these new training and tools to ensure that waste, both in time and money, is eliminated so that resources can be better utilized elsewhere within the organization.
During the thesis process of my masters program for the survey I was completing, I was seated in Denver General Hospital's Chief of Neurology office. I watched as a secretary hit a number of keys on an old text version of WordPerfect. As I watched her hit the 30-40 keys or so to bring up the memo template, I began to watch. I asked here how often she brought up the memo template: up to 30 times a day was her reply. I timed her keystrokes to bring up that memo template and found it took her about 30 seconds to compete that task. So, up to 30 times a day at 30 seconds each amounted to about 15 minutes a day was spent keying the same information. I taught her the keys to the macro recorder and asked her to retype the key sequence again and saved the macro. I asked her to hit the two macros keys at the same time and watched for her reaction as the template immediately popped up. Her smile said everything I knew process improvement was meant to be.
The fundamental principle behind process improvement is not the controlling of the code, which is being written, but the communication of the contribution of the code, whether it is good or bad. A library's card catalog does not limit an author's right to publish, but allows those looking for the author to find their work with greater ease and in context with other works. So to, process improvement does not limit a programmer’s effort, but allows those who can contribute to your project greater access and thus greater participation in improving Linux software development. Point #9 of Deming's Fourteen Points of Management is to Break Down Barriers, and Point #12 is Permit Pride in Workmanship, so, keep on doing this in the Linux community by adopting this new approach to continuously improving Linux.
Think - Understand - Do!
ASQ Glossary of Termshttp://www.asq.org/abtquality/glossary.html
Common causes: causes of variation that are inherent in a process over time. They affect every outcome of the process and everyone working in the process (see also "special causes")." 4
Cost of poor quality: the costs associated with providing poor-quality products or services. There are four categories of costs: internal failure costs (costs associated with defects found before the customer receives the product or service); external failure costs (costs associated with defects found after the customer receives the product or service); appraisal costs (costs incurred to determine the degree of conformance to quality requirements); and prevention costs (costs incurred to keep failure and appraisal costs to a minimum)." 5
Empowerment: a condition whereby employees have the authority to make decisions and take action in their work areas without prior approval. For example, an operator can stop a production process if he detects a problem or a customer service representative can send out a replacement product if a customer calls with a problem." 6
Plan-Do-Check-Act cycle: a four-step process for quality improvement. In the first step (plan), a plan to effect improvement is developed. In the second step (do), the plan is carried out, preferably on a small scale. In the third step (check), the effects of the plan are observed. In the last step (act), the results are studied to determine what was learned and what can be predicted. The plan-do-check-act cycle is sometimes referred to as the Shewhart cycle because Walter A. Shewhart discussed the concept in his book Statistical Method From the Viewpoint of Quality Control and as the Deming cycle because W. Edwards Deming introduced the concept in Japan. The Japanese subsequently called it the Deming cycle." 7 Too often, organizations follow a Do-Act-Plan-Check approach. While an organization is starting out, the larger it gets, the more defined the process becomes.
Process capability: a statistical measure of the inherent process variability for a given characteristic. 8
Red bead experiment: an experiment developed by W. Edwards Deming to illustrate that it is impossible to put employees in rank order of performance for the coming year based on their performance during the past year because performance differences must be attributed to the system, not to employees. Four thousand red and white beads, 20% red, in a jar and six people are needed for the experiment. The participants' goal is to produce white beads, because the customer will not accept red beads. One person begins by stirring the beads and then, blindfolded, selects a sample of 50 beads. That person hands the jar to the next person, who repeats the process, and so on. When everyone has his or her sample, the number of red beads for each is counted. The limits of variation between employees that can be attributed to the system are calculated. Everyone will fall within the calculated limits of variation that could arise from the system. The calculations will show that there is no evidence one person will be a better performer than another in the future. The experiment shows that it would be a waste of management's time to try to find out why, say, John produced four red beads and Jane produced 15; instead, management should improve the system, making it possible for everyone to produce more white beads.
Special causes: causes of variation that arise because of special circumstances. They are not an inherent part of a process. Special causes are also referred to as assignable causes (see also "common causes"). 9
Structural variation: variation caused by regular, systematic changes in output, such as seasonal patterns and long-term trends. 10
Tampering: action taken to compensate for variation within the control limits of a stable system. Tampering increases rather than decreases variation, as evidenced in the funnel experiment. 11
Variation: a change in data, a characteristic, or a function that is caused by one of four factors: special causes, common causes, tampering, or structural variation. 12
|Page 1 of 1|