To bug or not to bug - bug reporting done right
To bug or not to bug - bug reporting done right
In the past 10+ years we have seen and had many a discussion in our community about bug reports and feature requests for CMS Made Simpleâ„¢ that were marked as invalid and then closed by a developer.
Category: Tutorials, Announcements, Community, General
Posted: April 14, 2015 by compufairy
In the past 10+ years we have seen and had many a discussion in our community about bug reports and feature requests for CMS Made Simpleâ„¢ that were marked as invalid and then closed by a developer. Wellâ€¦ a great university professor once told me that statistically 13 people cry â€œmurderâ€ on unhappy matters, and only one will yell â€œhappyâ€ when something works great. I dedicate this article to the people who are in the cry-murder camp.
The natural answer to â€œWhatâ€™s the difference between the green and the red one?â€, asked out of the blue, is usually: â€œSay what??â€. It has no beginning and no end. No explanation, nothing. And still that same kind of question pops up its ugly (and regrettably, often very rude) head in many forms and shapes, such as a BR or an FR.
What the [beep] is a BR and an FR?
See what I mean? It already starts here. Obviously I should have been clear from the beginning by refraining from the use of abbreviations. Assuming that you folks know what FR and BR means because you develop websites running on CMSMS does not seem particularly logical for the team lead of the marketing & information team, now does it? True, it really isnâ€™t very logical. Just as it is totally illogical for me to assume that all of you readers are living in the Forge, in our forum or in our #cms IRC channel all day and are proficient in tech- and geekspeak.
Anywayâ€¦ For those who donâ€™t know these abbreviations, just as I didnâ€™t know them a few years back: An FR is a Feature Request, a BR is a Bug Report. And thus begins the proving of the point. Oh? Has my writing driven you up the wall yet? Yes? Good! Now you begin to remotely feel what many a developer feels when confronted with inaccurate Bug Reports and Feature Requests and rude responses, day in day out.
not reinventing the wheelâ€¦ ehhh, rant(s)
I could tell one of many stories about this subject, but I really donâ€™t need to. One of our core developers, Robert Campbell (a.k.a. Calguy1000), has actually written an epic response on the subject of Bug Reports, and this piece is just as valid for Feature Requests. After having another discussion in our IRC channel today I decided to copy it from our forum, at the end of this blog post. I promise you itâ€™s a long but very good read! Please note I am the one who placed the sometimes vile headings below, to enhance your reading experience.
The epic rant
In the forum someone wrote: â€œI can understand programmers getting fed up with non-programmers trying to use CMSMS - which is not as simple as the name suggests.â€
Hereâ€™s how Robert responded:
We never said it was simple for the site developer. Itâ€™s simple for the end user (your customer) to use.
**BEGIN RANT (aimed at a lot of people (though not all), and nobody in particular)
Nowâ€¦. you as a site developer are supposed to be a professional. That means that you have skills beyond just writing HTML into a WYSIWYG editor, or even basic CSS. Any 10 year old can do that. Also, it means that you have a basic understanding that in order to get assistance you need to provide something whether that be information, or money. Also, if you donâ€™t have all of the information you needâ€¦ as a professional, you expand upon your skills and spend some of your time learning and finding that information so that you can get timely and meaningful assistance without wasting your time, your customers, or anybody elseâ€™s.
You canâ€™t phone up or email your auto mechanic, spend a couple of hours giving him vague descriptions of a problem and expect a free solution. He will need all kinds of information, and probably to take a look at the vehicle. And he will also expect some type of compensation for his time. If you are lucky in that your brother in law is an auto mechanic you may be able to just bring over a case of beer, and buy dinnerâ€¦ but if you take it to a repair shop you will need to fork out cash. The same thing applies with computer systems. Just because the system and modules are open source and free to download does not mean in any way that my time is free.
Bug reports are intended to be reports of reproducible issues that are considered to be software defects. They are not PROBLEM reports or SUPPORT REQUEST formsâ€¦ thatâ€™s what the forum is for. If you can reproduce the issue, and can tell me exactly how to reproduce it, and you think it is a flaw in the software then that is what bug reports are for. Not: â€˜ Iâ€™m having a problem, please help meâ€™ or â€˜I canâ€™t get this thing to workâ€™, or â€˜What does â€œPermission deniedâ€ mean, or a thousand other stupid things. they are specifically for reproducible software defects. Just because you have a problem with one module on one host does not immediately mean that there is a software defect.
"Sushi" is about all the Japanese I know
Additionally, post in ENGLISH, in complete sentences, and try to keep the typos to a minimum (I know this is difficult for people whoâ€™s native language is not englishâ€¦. but you canâ€™t expect me to try to understand farsi either). If the post is gibberish, or in another language it either gets ignored (on the forum) or closed (forge). If you include screenshots etc. make sure they are well illustrated and in Englishâ€¦ Iâ€™m sorry, I also donâ€™t read Japanese.
This is free software, and this software is put out without warranty of any kind. It also does not come with free support. Though I do appreciate that people use my software, and try to help to the best of my ability and patience, I am in no way obliged to do so. If you want assistance from me (or anybody actually) you need to respect that my time is just as valuable as yours. This means that you give me exact instructions as to how to reproduce the software with ALL of the details. and also you have to be patient.
The 5-minute rule
Merely specifying â€˜latest version of CMSMS and all modules up to dateâ€™ is in no way providing enough details. If that little bit of information was enough for you last time consider yourself lucky. Assume when you submit a bug report in the forge, or a problem in the forum that you have at the very most 5 minutes of my time to reproduce an issue, and that my environment may not be exactly the same as yours, or that my environment may be configured to use different software, or use the same software in a different way.
Often you have only a few seconds to get my attention when an issue is reported. It needs to be well researched, well explained, and easily reproducible, or it will get ignored and/or closed. Also assume that I wonâ€™t read it on the day you post it, or remember the brief conversation you may have had with me or somebody else in IRC, the forum or some other chat/messaging protocol. I may not get to your message today, tomorrow, or next week. I specifically want exact details as to how to setup an environment where the problem re-occurs. This includes all and any options and settings that may influence the behaviour. If it works fine for you on two sites, but not on a thirdâ€¦ then you need to do more digging before you submit a bug report. Does it happen on different servers? have you done basic diagnosis issues like enabling/checking your error logs and understanding what they say?
sub rant about error logs
I have â€˜solvedâ€™ many bug reports issued by people without sufficient information just to find out that they â€˜had no access to error logsâ€™ or didnâ€™t know how to find them, or couldnâ€™t read them. And the issue turned out to be a permissions issue or an â€˜out of disk space issueâ€™, or some other system issue, Not issues directly related to my code. Those are a huge waste of time, and a complete lack of professionalism on their part.
back on topic
Additionally, the system info page in CMSMS was written as a convenient means for people needing assistance to provide a starting point for the information we need. Sometimes this is enough information, often it is not. Donâ€™t think that this is ALL of the information you need to provide. More often than not, this isnâ€™t even provided. Often I feel we might as well delete this code from the core.
Polite requests for more information?
I am no longer willing to politely request for days the basic information that should be provided for each and every bug report or forum post. or to teach people how to do it. If you donâ€™t know, find out. It is not my problem (I have enough problems). I am now just ignoring forum posts without a reasonable amount of basic information or an attempt to provide itâ€¦ and I close bug reports. I donâ€™t have the time or patience to deal with half baked crap, or to teach people stuff that they as professionals should either know, or know how to find out on their own.
I am also no longer willing to go on fishing expeditions spending hours of my time trying to reproduce problems just to find out itâ€™s due to some bad configuration option somewhere, a bad upload, an old version, or worse yetâ€¦. because somebody hacked the code and it caused problems down the road that they were not aware of at the time. I have encountered all of those situations, many many times. Also, donâ€™t bother reporting issues where I have to setup some windows or mac environment, specific hosting requirements, or on specific mobile phones or tablets. Though I am willing to fix issues that I can reasonably easily reproduce and identify, and have the time to fix. I am not willing to spend thousands of dollars on hardware to solve issues and Iâ€™m allergic to Microsoft products. Itâ€™s back to spending your time efficiently, and not wasting mine.
It has to be worth it
Due to the countless permutations and combinations of options, configurations, and uses for the various modules it can take a couple of hours (or more) just to setup an environment to try to reproduce an issue. (i.e: to create a new install of CMSMS, extract and install the modules, create categories, create sample data, setup summary, detail templates, setup pretty URLS, upload images) is a considerable investment of time. Your bug report has to be worth it.
Next: I donâ€™t just drop what Iâ€™m doing and put aside a paid project, or stop work on my house or car, or socialising with my family just because somebody submits a bug report. I try to fix bugs at the next time I revisit the module for some reason. It could be months before I revisit the module. And sometimes I donâ€™t fix bugs even then, if I canâ€™t understand them, or canâ€™t reproduce them easily (see the 5 minute rule above).
This is not unique to CMSMSâ„¢
The above issues and complaints apply to all open source packages. Not just content management systems. Also, donâ€™t think for a moment that this is an issue specific to me or CMSMS alone. There are lots of articles on the interweb about people providing crap information when trying to get support. Also, take a look for a second at the number of unanswered posts on various support forums, or stack overflow etc. Sometimes when somebody is experiencing a problem, and wants quick assistance for a critical problem they can get a hold of me via email (if I havenâ€™t responded to you, donâ€™t worry, you arenâ€™t the only one). If the issue is critical enough (to me) and explained well enough (see the 5 minute rule), and I have the cycles to donate to the issue, then I may respond. If you offer to pay, I will most likely respond within a few days.
Want to hire someone other than me for a fix? No problem!
If you hire somebody that is not me (or the appropriate module author) to fix a problem, then you will have to maintain those fixes throughout the lifetime of the site. Unless you send a properly formatted and completely explained patch requestâ€¦.then I (or the module author) have the option to accept that patch into the code stream. If I accept it, I will support it for some time. At least until the module goes through some major change that negates the need for that patch/functionality (in my opinion).
Boooh, mommy, the mean man does not implement my great patch!
A word to the wise, though I personally have in the past accepted patchesâ€¦. I donâ€™t do it very often. It largely depends on the patch and who itâ€™s coming from. I have to trust the developer, and agree with the patch (both its style and function), and whether it is in (more or less) my code style, and a few other factors. Summary: If you expect me to spend my time helping you, and you want it free then you will need to:
- provide sufficient specific instructions as to how to reproduce the issue exactly;
- make sure itâ€™s not a system specific issue (wrong PHP version, out of disk space, out of memory, permissions, the list goes on);
- be patient;
- behave professionally. The more time you spend identifying, reproducing, isolating, and describing the problem increases the likelihood that I will look into it.
If you are willing to pay to expedite the issue, then drop me an email with a description as to how to reproduce the issue on your siteâ€¦. a login/password to a CMSMS admin account (and the appropriate URLS) and SSH access (please no FTP creds) to your environment and I will be all too happy to help you out, and bill you my hourly rate.
END OF RANT**
Thanks for your time.
- Robert Campbell - a.k.a. Calguy1000 | Twitter: @calguy1000
PS In case you think this is a response that is unique to the guy who gave it, youâ€™ll want to read an even more epic piece by a guy called Simon Tatham, professional and free-software programmer, creator of PuTTY. Heâ€™s gone through extensive lengths to prove his point: http://www.chiark.greenend.org.uk/~sgtatham/bugs.htmlÂ in 15 different languages!
PPS There is one category of people whose Bug Reports and Features Requests will always be deemed invalid. It's those people who troll, flame and slander our team or its individual members and/or consciously infringe upon our trademark.
To read the original forum post, click here.