How do you apply or maintain quality error reporting in your error tracker?

Question

High-quality error reports are essential for effective error tracking - in an ideal world, all error reports will contain important information, such as which version of the software it affects, and a step-by-step description of how to reproduce the error.

In reality, however, error messages can vary greatly in quality. They can be on-liners ("function X does not work, fix!"), Function requests, PEBKAC or unintelligble.

How do you ensure or maintain the quality of bug reports in your bug tracker to stay valid?

Answer 1

I agree with John Limzhap - your QA staff should be competent enough to publish relevant bug reports, given the correct basic training and recommendations. However, there are ways to ensure and encourage improved reporting of errors:

• Most error tracking programs have a way to mark some fields of the error report as mandatory, so the reporter must actually select the appropriate value to successfully create the error.
• Usually it is possible to include a basic template for an error report, something in the lines

Scenario:

Expected results:

Actual Results:

Notes:

• You can (and should) provide an error reporting tool that will be run on the problem machine, collect the relevant information and pack it into an archive file (and possibly put it on the desktop). Then you instruct your employees to run it when they encounter an error that they want to report and attach the archive to the error. This tool should be easy to use (just run the executable file) so that they connect the diagnostic information to any error, without thinking about whether it is relevant or not. This tool is usually also very useful for customers.
• And last but not least, “education, education, education”. People learn best from experience - just make sure that whenever someone opens an error without the right information, the person managing the error will go and talk to the person who discovered the error and explain what is missing and why it is important.

This is a practice that we successfully apply at my workplace, and I believe that they are quite universal for most work environments.

Answer 2

I used to think that the quality of the error report was equivalent. I still think so ... the errors that I report about contain much more useful information in them than those introduced by QA or operations. However, I came to admire the FogBugz model. It is very easy to enter an error. Just knowing that there is an error condition is useful, even if there is not much supporting information. In addition, users feel that something is being done.

Answer 3

Write a good, but not too long guide to using the tracker and what is required for each field. Make a general purpose reference example that others can use if they are stuck.

I have a link copy to edit the pages of the Docbook manual, and using this several times, I already know most of the syntax by heart.

Answer 4

It depends on whether you are talking about a private QA review and public beta.

If this is a public beta, it is not recommended that you allow users to edit your bug list. Someone should be attached to aggregate comments and user reports and recognize which are actual errors and which are duplicates, and which give some clue on how to replicate them.

If this, however, is an element of error that is published by your legitimate QA staff, you have a competency issue with your staff. Correct recommendations should be made on how to report errors, especially if the replication steps are performed correctly.

Answer 5

The tough question. I would try to check if the system has any way to force certain fields you need, try and have some errors that are crucial to your eyes (email, rss) so you can quickly pounce, but basically it is your team that is aware of the quality standard and adheres to it, the guidelines are published and published, something like that.

Assuming your team: If you can have a specific structure that is used every time in the comment field, what is expected when it is introduced, then this will be good too - even better if your software has a default note scheme, where you can define this structure on an empty form.

To some extent, although it depends on the person, they should know that he is part of the communication standards, he is expected as a job requirement and that they are responsible to every other member of the team, because other people should not be able to hunt follow them to find out any details if this can be avoided.

Moreover, the time for fixing errors on elements with a lower priority can be time, and people will definitely forget about the details.

Assuming these are users: you cannot, to a high degree, but I will try, if possible, to ask questions about any form in a way people could understand.

Not quite on this topic, but in the question "how do you ask questions", this blog post is 37 Signals - link text

Even if you have to have a different form asking questions that are visible to your users that only provide basic information to the error program, I would do this to ask the right questions.

Which product? Which version (photographs showing how to find it)? It would be useful to enable a screen dump if they could open the program and press a button to automatically send the log file, if it had stopped them from further work, if it had lost its changes, etc.

For users, this is probably more about how you ask questions, and letting them know that you need specific answers, or which of them you find more useful, then you will probably get better answers.

Answer 6

Use something like UserVoice so that end users report bugs and feature requests. Records of error tracks really should be internal - they are too technical for end users, as well as IMHO, expose too many internal actions.