Friday, March 19, 2010

The true meanings to Priority and Severity

.
I was talking to a friend the other day about this and that going over a few ideas and stuff, when he came out with the best definition of Priority when associated with software defects. The typical definitions go along the following lines.

Priorities and Severities:
Priority is how important something is to fix, while Severity is how severe the impact is on the system as a result of the bug.

Priority:
The priority of a defect is based upon the importance and urgency of resolving the defect. Priorities are usually High (P1), Medium (P2), and Low (P3);

High (P1) – (Associated 1 with Critical (S1) and Serious (S2))
This defect must be fixed as soon as possible. Defect is blocking further progress in this area. It should be resolved immediately. (Must be fix within 24-48 hours. Blocking build test)

Medium (P2) – (Associated 1 with Serious (S2) and Moderate (S3))
This Defect should be fixed prior to the product release. It must be fixed as soon as possible. Bug is blocking further progress (testing) in this area

Low (P3) (Associated 1 with Moderate (S3) and Minor (S4))
This Defect should be fixed if time allows, it can be resolved in a future major system revision or not resolved at all.

Severity:
The severity of a defect is based upon the degree which the defect impacts the application. Severities are Critical (S1), Serious (S2), Moderate (S3) and Minor (S4).

Critical (S1) – (Associated with Priority 1)
The defect causes the system crash or loses data. This defect has no work around. Deviation between the specification and the behavior of the Application (causes the Application to function) No work around is available.

Serious (S2) – (Associated with Priority 1 & 2)
The Application will operate, but its operation is severely restricted. The defect causes major functionality or other severe problems; product crashes in obscure cases. This defect causes the system to produce incorrect, incomplete, inconsistent results or impairs the system usability. There is a work around. Deviation between the Specification and the behavior of the application (causes the Application to function improperly, i.e. incorrect totals, paint types, etc.)

Moderate (S3) – (Associated with Priority 2 & 3)
The Application will operate with limitations that are not critical to its overall operation. The defect causes minor functionality problems, may affect "fit and finish" of the product. The defect does not cause a failure, but impairs usability, and interferes in the fluent work of the program. Violation of standards, conventions, or rules that would not result in a deviation from requirements if not corrected, but could result in difficulties in terms of maintenance or clarity of purpose (workmanship)

Minor (S4) – (Associated with Priority 2 & 3)
Bug contains, formatting, typographical, spelling defect whose correct interpretation is obvious to a knowledgeable reader, or unclear wording of error messages in low visibility fields. The problem does not impact the use of the product in any substantive way. This defect affects nonfunctional aspects of the application. A work around is available. The Application can be used with only slight inconvenience.

This friend of mine put it more simply stating that a High P1 is something we have to fix or lose our customers, a Medium P2 is something that would be nice to fix but it can wait, and a Low P3 is something we will never get to so why bother. As far as Severity goes a Critical S1 is bad enough that it will cause Customer Support headaches with the increase customers calls, Serious S2 is something that the customer won’t like but they have a work around, a Moderate S3 is an issue we are betting the customer will tolerate, and a Minor S4 is something that the customer won’t even notice.

I believe he hit it on the head, how many times has a product been release with no P1s or P2s but a ton of P3s and P4s? The answer is every time a product is released. In the many years, I’ve been in the industry I have seen my friend’s definition put to use more often than not. I was working at a company which had defects 13 years old which have never been touched and the reason given for these defects being left in the product – the customer has not complained about it so it must not be a problem. Yep he has it right.
.

Friday, March 5, 2010

A recipe for disaster

.
I have worked at several companies throughout my career, which wanted to move from a waterfall methodology to agile methodology, and each has had its own unique set of problems. Even though they are different, I have notice similarities as company have tried to move in the direction of agile and the best way to describe them is by using the analogy of remodeling a house. It seems to be a best fit when describing the situation in software development.

In this analogy our faithful customer has a conversation with the architect who writes downs the customer’s ideas and formulates a plan to create a thing of beauty for the customer. When presented to the customer there is the normal back and forth discussion until there is agreement that what the architect has come up with matches the ideas of the customer. The architect takes the customer’s requests to a meeting with the drafters, the construction team, and the inspector and presents his vision of the additions and enhancement the customer is expecting in the near future.

The drafters and the inspectors take time to go over the ideas to achieve clarity on what is really needed to meet the customer’s desires. As the construction team begins this process the superintendant of construction says to the team we don’t have time to go over the architect’s proposal because we have too many other projects that need to be completed, first. So months goes by with no input from the construction team, until one day the architect inquires as to the progress being made on the customer’s request. The superintendant replies since the proposed project has not gone through the proper vetting process that the construction team had not even looked at the request and had no idea as to how long it would take to complete the additions and enhancements.

So after having the new process explained, the architect reworks the customer’s request and submits it into the process expecting a reply in a couple of weeks since that is how long the process is suppose to take. Two months later the architect is given a draft schedule and an indication of how long it would take to deliver the entire project. The end date is unacceptable to the customer, so negotiations on the scope of the project begin with an end date in mind the project’s scope is cut in half, though the architect is not happy, the architect needs to get something completed by the end date, and reluctantly accept the new plan.

The plan is put into action but to ensure the schedule is met, decisions that the team of drafters, construction workers, and inspector usually make on the fly for quick turn around, is now delayed as each decision must be estimated by the construction workers and than vetted by a committee superintendants in a weekly meeting, and can takes weeks to decide if a change can be made and its effect on the schedule.

When the end date arrives, there is much bickering, finger pointing, and blame being tossed around as the incomplete project is delivered to the customer. A root cause analysis is conduct to find out why the project failed, but it is not done using the 5 why of root cause analysis and blame is fixed on individual department and the results of the analysis is never seen by the parties involved. As a result of a superintendant wanting to make a point to another superintendant, the delay in beginning of the original project created a crisis at the end of the project.

The moral to this story is for superintendants to get out of the way and let the work progress, because everyone loses, when superintendants think more of their egos than they did of the project at hand. In an agile environment, egos have no place that the table and when the egos appear a recipe for disaster is beginning to cook.
.

Thursday, January 21, 2010

The Bullpen

.
I have always liked baseball, though I was never much of a player, I enjoyed the game, the skill it takes to hit a fastball, to throw a perfect game, to steal a base or two, or even to complete a triple play. The area that is the most fascinating to me is the bullpen. The area where the pitching staff warms up prior to going into the game, and where they loosen up, it is an area for preparation. You can tell what a team on defense is going to do in the near future by watching the bullpen.

An acquaintance of mine, works as a software tester in a company where there is a different meaning for the bullpen. You see, they used to live in cubical world where one to two people worked in an artificial room with six foot walls in aisles off the main hallway – without these artificial walls it would be a open area where you could see all the desk from one end of the room to the other. You could hold team meetings if everyone stood on their desk at the same time and began to talk. Everyone would look like gophers popping up out of their holes, got the picture. Now for years they worked in this environment, and they were able to get the job done. But for some reason the powers in higher pay grades than this acquaintance of mine decided that all members of each QC teams would have their walls removed, so they could work together as a team. Great idea, the walls disappeared, and the bullpen was born.

How does this bullpen work in an era of trust, independence, and openness? In baseball the bullpen allows the pitcher and catcher to work together creating and special connection, and building trust between the two team members. In this particular bullpen the team lead keeps a watchful eye on the activity of the team, ensuring that not a wasted moment goes by, and ensuring that the team member remain on task. When ever a team member sneezes, coughs, or farts it does not go unnoticed; the team lead knows when each team member leaves and when they return, making mental (if not actual) notes of the departures and returns. Yes, the team led controls of the activities of the team, testing related or not.

This environment has had a side effect which is the opposite to the trend in most software development companies. The environment in the industry is towards more communication and openness between all departments involved in the producing a quality product. The increase in communication and openness between the departments is considered to be the key to their success. Yet in this bullpen conversations are spoken in hushed voices and whispers showing that morale has slipped as the eyes of big brother are constantly upon the team. It is felt that the team has not left kindergartner, where permission is required even to visit the little room down the hall.

I may be exaggerating this a bit but I can guarantee that I would feel this way if my boss was in a position to watch my every move. I am an independent type of worker. I work well once given an assignment and will complete the task without constant checking. To have someone in a physical position to document my every move only brings out the rebel in me.

I believe that the reason for the bullpen is to give developers and testers an opportunity to work closely together, so that development and testing efforts can move along quickly; I have even suggested this where I manage a group of testers, the testers still live in cubical world. But they understand that one day they may be put in the bullpen next to a developer, where they can develop the relationship, and teamwork they will need to deliver a quality product quickly - without the constant eye of big brother monitoring their every move.
.

Thursday, January 14, 2010

WAGILE Methodology

.
Wagile Methodology? I know I can see your expression, what is Wagile Methodology; well it is the latest in a series of testing methodologies. It is the type of testing that is done by most companies and they don’t even recognize it. It has never been made official, and it is quickly forgotten when asked what testing methodology do you follow?

I’ve worked at many companies where Wagile was the methodology of choice. The companies believed that if they were to use Wagile, development would go faster and smoother thus creating customer delight with the speed at which they would get new features to the customer. Yes, Wagile is the methodology of choice.

In all these companies who have chosen Wagile as their methodology, it was usually done after living forever in waterfall. These companies have individuals who have read a few books on a new methodology (agile), usually the reading is skimming, thus becoming the resident expert on this new methodology, who determines that this is the environment they want to move to, so the resident expert gets others to buy in, and why not, anything is better than what we have, which is the common feeling about waterfall. So with great effort to move forward, the institution of daily scrums are begun with teams of 20 to 30 people – so no one feels left out – where everyone tells about what they did yesterday, and what their plans are for today. In reality these comments sound like – I worked on this yesterday and will continue to do the same today, followed by the next individual stating ditto, or same here, until all have thus report their efforts. Yes, Wagile is the methodology of choice.

As the Wagile methodology is used, sprint planning meetings are held for an hour or so, and guessimates are given so project manager can put an air tight schedule together which is never shared. Burn-down charts are created, but seldom use, dashboards are developed but never seen, and as the unknown deadline approaches there is panic among the project management team. Yes, Wagile is the methodology of choice.

Wagile’s benefits just never seem to appear as the proponents of Wagile have articulated. Wagile projects usually end with blame being casted, fingers being pointed, and root-cause analysis being done with an agenda to fix blame. Managers in the Wagile environment feel the need to protect their turf, build up kingdoms, or look for greener pastures elsewhere. Many, however, stay to fight the good fight, in hopes of a better future. Yes, Wagile is the methodology of choice.

So if you are looking to move away from Waterfall to something that will create a world of difference in your company. Come and join us in our Wagile world, it is an experience you will always remember, and regret. Yes, Wagile is the methodology of choice.
.

Wednesday, January 6, 2010

The myth of Software Quality Assurance

.
I have the opportunity to interview every so often, and I like finding out what the candidate knows about the work they do. For this reason I almost always ask the candidate to tell me the difference between Quality Assurance (QA) and Quality Control (QC). At first I was astounded at the responses. I felt as though, I had asked the candidate to list all the heads of states for Zimbabwe since its inception. Now these candidates are usually those with many years of experience, you know 10+ and considered senior testers. I felt safe asking because, I thought if you are working in an industry for more than a couple of years, you should know something basic about your job; sadly I was wrong.

I have a good friend who tells me that I am anal about the difference between QA & QC and she is probably right. But in interviewing these candidates I have found that there are unlimited views on the differences. I have been told that QA – “is following scenarios assuring quality”; “measuring the amount of quality”; “that it is analysis or detailed or in-depth validation of software”; “the job is to find defects”.

For QC it goes something like this – “there’s no difference”; “where faults are found in acceptable ranges”; “it’s an evaluation against standard”; “that QC is more comprehensive than QA”; “it is the job of engineering, development, and design”; “that quality is the composition for a product line”; “that QC is the verification of the end product”; “it is the flow through the application without errors”.

It is amazing that so many of us in the software testing business don’t know what we do. Are we QA or are we QC? Many testers junior or senior do not know. I believe it because the majority of companies we work for do not understand what we do as well as they should. Consequently testers are placed in a department titled QA, and the testers test the software. Go to any Software Quality Assurance (SQA) group and what is the main topic of their conversations it is testing, improving testing, automated testing, manual testing, etc.

Yes, I have a point. My point is that QA is not testing. QA is the assurance of quality and QC is the control of quality. Let me explain, I believe, as many of you, that you cannot test quality into any product. Quality comes when there are processes which are followed and the process are reviewed, followed by some analysis on the process to correct any flaws found in them, and then the flawed process is modified to produce a better product. This is QA; it is a short but simple answer. QA has nothing to do with testing. On the other hand, testing is QC. QC is the validation of the product to a set of specifications, requirements, or standards. The validation is accomplished through the action of testing the product against the requirements. We all work for the QA department yet we do QC work and we perpetuate the myth by calling ourselves “QA”. I have been corrected many times by testers which state very forcefully that “we are QA”.

My stand is that we who test any software product are actually in the business of QC – quality control; in that we focus on controlling the quality of the product, once it is delivered to us. We are not QA, for we assure nothing, we just validate the requirements have been met and report any deviation in the product from them.

Wednesday, December 30, 2009

Working for the Village Idiot


-
In medieval times there were people who were cast out, despised, ridiculed, and mocked; these people were commonly known as the village idiots. Where are these village idiots in our communities today? It is my contention that they can be found in the ranks of the middle management, known as the team lead, playing at shift supervisors, even pretending to be the front-line supervisor, or even the department manager. The pointy haired manager in the Dilbert comic is one image of a modern day village idiot, due to the chaos caused by his fascinating management ability. The modern day village idiot can be defined as a person who is known in their community for their stupidity and ignorant behavior; in this case the community is the office, and everyone usually knows who these people are, and try to avoid working for them as much as possible. But there is always the chance you cannot avoid the privilege of having to work for one or two.

At a company where I was working the following email was sent between a team member and the team lead: team member -“I was just looking at the milestone document and have noticed we only have 13 working days before this application should be completed and we are far from being near completion. I was wondering if we are going to meet this deadline. Or has the deadline been postponed? If the latter, what is the new date? If not what do we need to do to meet the scheduled date? I’m just curious; let me know if there is anything I can do.” The response is classic, a real gem; the team lead, bless their simpleton soul, replied “We can only test what we have been given. Thanks for the concern”. Is there any better answer that could have been given? I think not, the clarity in which each question was handled, the clear direction which was given, and the listing of the abundant opportunities which needed to be handled before the deadline, surpasses anything that could be imagined. Is avoidance to answering questions a typical village idiot reaction? I think so. I believe that the village idiot gets so confused when questioned that they say the first thing which comes to mind.

A friend of mine was called into his boss’s office where his boss informed him that he was being charged a half day of PTO (paid time off) for the previous day. It seems that after working 10 hours my friend left for the day at 2:30 p.m. Since he left before 5 p.m. the boss had to charge him for a half of a day PTO. In defending himself, my friend told his boss that he did work a full day and argued that he should not be charge the PTO. In another classic response this manager, bless their simpleton soul, said “I worked here for four years before I took anytime off.” Boy, what a reason, because I am a village idiot, you must be one too. Oh by the way I’m the boss so I will punish you for being more intelligent than me.

How do these people advance to such levels of responsibilities? I honestly believe that the Peter Principle – where an individual is promoted to their level of incompetence – is alive and well in corporate America.

So what does all this have to do with software quality assurance? It is that we all have known or have work for the village idiot. So how do we deal with the village idiot? Well, you can pick on them behind their back, when their not snooping around trying to figure out whom you’re talking about, which they always think is them. You can involve them in your group and let them be the court jester. You can even take the time to get to understand them – my wife would be proud of me for suggesting that one. You know that rule you were taught as a kid but seem to forget as a teenage and are reminded about it as an adult as you teach your kids, yeah that’s the one “do unto to others as you would have others do unto you."

Since there is only one way to survive the village idiot, sit back, relax, and just laugh. Enjoy the adventure because the village idiot will do something else tomorrow that will make you shake your head and question your sanity.
.