Enhancement to Mantid Error Reporting #32
Description
Summary
At present the Error Reporter is often used to provide only a stack trace of an error. We rarely get contact information and/or information to replicate the issue. Therefore we would like to create a new error reporting framework to capture essential information about the error itself (specifics like stack trace, screenshots , steps to reproduce) and encourage submission of contact details of the reporter. We also want to provide visibility of incoming error reports for the developers in a way that is easy to analyse, aggregate, prioritize and take action.
Intended outcome
Mantid software becomes more robust since we will be able to get a deeper understanding of the errors and fix them.
Mantid software developers will be seen as reliable and responsive developers who communicate well with the users and scientists who report issues.
Acceptance criteria
The number of crash/bug reports that contain sufficient information to reproduce the error and/or will contact info of the reporter. During mid-September to mid-October 2023 only 3% of error reports contained this information.
The amount of user reported bugs/crashes that gets fixed and released within a release cycle will increase.
How will it work?
Features in the Minimum Viable Product (MVP)
- Build the crash reporting flow in a way that almost all crash reports will contain contact info of the reporter
Features in Scope:
- Error reporting is to be triggered for all crashes and stack trace is always captured
- A user friendly method to report faults in the software(for non-crashing types, like e.g a button not working)
- Process to ensure bug reporters get feedback on the progress of their issue – This will encourage the users to report more bugs with as much details as possible.
Error reporting UX if further enhanced
- Explore avenues such as automatic crash report submissions
- Benefits of submitting error reports with contact info is further emphasized.
- Users can pass on additional comments/relevant screenshots of the error scenarios
- Some form of a visual dashboard where devs/PMs can see the reported bugs
- Process to ensure bug/crash consolidations (i.e what we do when the same bug is reported 100 times)
- Process to determine bug/crash prioritizations (how do we decide which one to fix?)