Posted Tuesday, January 29, 2008 by
Ed Frederici
Defining what is or what is not a defect is a lot like defining happiness or love, but in reverse. Happiness and love are fun, defects are not. In two recent posts I discussed metrics and goal setting. For me, and I think for IT in general, metrics are null and void if a company does not have a well defined set of criteria for what’s a defect and what’s not.
A lot of issues can masquerade as a defect and defects can come in many shapes and sizes. Without being specific about what a request type is and means too many issues will get lumped in one bucket and the wrong people and groups will get saddled with the weight of that bucket come metric measuring time.
If you’ve read my posts Metrics 101 and Metrics 102 you know that we’re going to measure IT success (in part) at Achievant based on the flux within our ticketing system. In order to have valid metrics we’ve identified 10 types of tickets:
1. Data Update: This category covers any data transformation, update, whatever. These aren’t defects. They are usually in responses to changes in state or federal codes or a change is business practice by a client. For example, when the fed changed the convention for ethnicity codes that generated a Data Update ticket in our system as we complied with the change.
2. Defect - Business Logic: There are lots of flavors of defects. We wanted to pin our defects down a little so we could better track where we are breaking down. A business logic defect is one where the actual business rules are not followed. For us a ticket is marked as Defect – Business Logic only if the developer coded the business logic incorrectly. If we see a lot of these we know that we’re doing a poor job of communicating business requirements to the product development team and can adjust our requirements gathering and reporting accordingly.
3. Defect - Hard Error: Really you should almost never see these. These are errors that are full on blow-ups. You shouldn’t see them in QA and you shouldn’t seem them in production. A developer doing his job unit testing should uncover any error so egregious prior to releasing his code. If we see a number of these we know we’re either rushing the process or have developers who aren’t doing the due diligence they should.
4. Defect - User Interface: This means the UI has an issue. Maybe the screen does refresh after a drop down list change or a button is in the wrong location on the screen. These are strictly issues relating to the look and behavior of the UI. Too many of these means we are not validating or work against the specs well enough or not following our own design rules.
5. Defect - Validation Failure: this category is exactly what it looks like. The code failed to validate a data entry upfront and bad data has made it into the system or caused an error. To many of this type of issue means we’re not taking the time to bullet proof the application and are rushing.
6. Enhancement: Enhancements are any changes to the app that make it better and which are not a simply a gap in current functionality. These are for brand new functionality only. If our applicant tracking module doesn’t allow for the upload of resumes in Word format that’s a gap, not and enhancement. If we decide to add a module for union labor disputes that’s an enhancement.
7. Gap: Gaps are those things the application should do, but doesn’t. Every application has these kinds of issues. They are like enhancement in that they are new functionality, but unlike enhancements in that we should have thought of them while developing an actual enhancement but didn’t. these are the slap your forehead, :why didn’t we think of that” kind of things
8. Missed Requirement: These are a failure in the requirements definition. If a client MUST have duplicate copies of all emails sent to legal and we fail to note that and don’t develop that it’s a missed requirement. These are things that come up in the sales and discovery process, but which never make it into a requirements document and are then never developed, but should have been.
9. Performance Improvement: These are tweaks to code, SQL, OS configuration, whatever that make the application perform faster. At Achievant we record the load time for every request of every page and regularly review the numbers to make sure we’re not slowing down and to also ensure that we don’t have any dogs out there. Performance Improvement tickets address application slowness.
10. Wish List: We all have this. We might call it a portfolio, a backlog, whatever. It’s those things that one day we’ll get to when all the planets and stars align and we’re not busy doing something else. These are unique tickets because they sit on the shelf for a long time by their very nature. When you’re measuring the closure rate, etc of your tickets this bucket can skew your numbers if not accounted for accordingly.
Just as important as defining what the tickets are is the goal of getting everyone to agree and to follow that convention. We all know there are politics associated with the tickets in queue for product development. If everyone follows the game plan then everyone is measured equally by the metrics.