The Object Teams Blog

Adding team spirit to your objects.

Why I’m sometimes a bad bug reporter

with 5 comments

OK, I use Eclipse for getting some work done. Eclipse is software so we know it contains bugs. Given that Eclipse is open source software, we all can only expect it to run smoothly if we diligently report back all errors we encounter (and provide steps on how to reproduce etc.). I know all that and I really want to be a good bug reporter because I really want a good experience using Eclipse because I really want to get the work done (and I may even want to sustain the impression that I’ve got everything under control and thus working with Eclipse is a delight).

A task

The other day, I was a very bad bug reporter, and only today I find some time to reason about what happened. This was my task: In preparing the initial contribution for the Object Teams Project I had to rename a bunch of things from org.objectteams to org.eclipse.objectteams. Simply, huh? Back in the days of emacs+bash I might have chosen to just use one big

find . -exec /bin/sed -i -e "s/org.objectteams/org.eclipse.objectteams/g" {} ;

and voila, if fortuna was with me, I might have been done at the return of that single command. But things were actually just a little bit more challenging, like, a few occurrences would have to remain unchanged plus while touching about every single file in our software I was going to also do some clean-up: rename some packages to foo.internal.bar, fixing some copyright headers, etc. Also I preferred to change one plug-in at a time which would mean that all references to plug-ins not yet processed should stay unchanged, too. Plus a few more little deviations from the grand search-and-replace-globally.

OK, since I like to see myself an Eclipse-wizard this was a nice challenge for its refactoring support. Plug-in by plug-in I renamed, I renamed packages with / or without subpackages, and after each step I wanted to see that compiler and PDE agree with all changes and signal that everything is still (or again) consistent, ready to be built, actually. Perhaps things started the get wrong when I estimated the effort as one or two hours. So, after a day or so, I wasn’t perfectly relaxed any more. My fault, should’ve known better about that estimate. BTW, one of the reasons it took so long was simply the size of my workspace in comparison to the power of my computer / hard-disk: every time I performed a rename with updates in non-Java files, I was nervously looking at the screen: “should I sit and wait for the preview page, or should I go to the kitchen, get a chocolate, coffee, just something?“. I did some emailing in parallel, but let’s just keep this: due to those response times
I wasn’t perfectly relaxed any more.

A story of bug reporting

What I was going to tell here is a story of bug reporting, because as a safe bet doing a real-life stress test to an Eclipse component should give you a good chance to discover and report a few bugs that have not yet been reported by others. And indeed, I was successful in discovering some bugs, in various components actually.

I think one of the first things that occurred was that the svn synchronize view would sometimes fail to open the compare editor, more precisely, I had to explicitly close the compare editor before comparing the next file. At first this really **** me off, because the error dialog was popping up in some kind of infinite loop. Fun!#$ Once I’d figure out how to work around this problem it soon became a habit to just close the compare editor before clicking the next. Next, the svn plugin made a refactoring fail, because it was trying to create a directory which the previous refactoring had already created. The most creative bug having to do with subversive was a chain-reaction of first failing to undo a refactoring and than during reporting this error blocking the UI of Eclipse so I could only kill the Eclipse process, insert a new coin and start a new game.

I don’t intend to blame a particular component. For clean-up of license headers I have a little home-grown plugin that I just wanted to quickly install into the running Eclipse instance, so I went for the cool new feature to export/install into the host. Oops, my plugin depends on another plugin that only exists in the workspace but not in the host, install failed for good reasons. I removed the dependency and tried again. Installation still failed for the same reason: the ghost of this removed dependency prevented installation into the host Eclipse. Oh, I should have incremented the version or let a version qualifier do this automatically, of course. Tried again, still failed. Tried something so slightly different I cannot recall, from there on it worked. Can I reproduce the two or three different levels of failure? I didn’t even take the time to think of it. Well I would’ve been disappointed without a bug from p2 in this list 😉 .

PDE did its share by reporting overlapping text edits in plugin.xml and therefore disabling its refactoring participant. What the **** caused those overlapping text edits, and how do I re-enable the refactoring participant to give it one more chance to behave well?

The list could go on if only I could remember. Instead I was happy to finish this 1.5 hours task after 2.7 days, ready to submit our initial code contribution, wow!

Looking back, I / we missed a great opportunity: we could have identified plenty of bugs in various components of Eclipse. With only a few more days of debugging I might have been able to present reproducing steps for all those bugs. And, if triaged and fixed by the corresponding devs, this might have contributed to M6 containing fewer of those bugs that just only occur in real world, never during testing. I failed, I managed only to submit two bug reports, with very little information on how to reproduce.

Lesson learned

Susan McCourt responded to an earlier bug report of mine in a very descriptive comment:

That is one of those things I’ve been meaning to fix forever, never wrote a
bug, and so keep forgetting to fix. And it seems like if I’m actually
[doing what triggers the bug], it’s because something is wrong, and so I again postpone
writing a bug.

Sure, when we hit a bug (or a bug hits us) we are always in some context of doing something challenging. Something that requires our mind to stay in focus. Something we want to get done.
Well, work isn’t perfectly linear, so we know how to react to interrupts. Bugs are such interrupts. Sometimes I like the challenge of isolating a bug etc. Sometimes I’m sufficiently relaxed when the bug occurs so I actually take the challenge. Sometimes the bug is sufficiently good-natured so making a small note and going back to the bug after the actual work is done is a perfect strategy. Some bugs, however, smell like depending on so many factors from your current context that reproduction an hour later seems extremely unlikely.

I think I have a solution to all this: given we don’t want to be distracted from our actual work, given also that some bugs need immediate care or they will escape our attempts to identify. Given some of the worst moments are when we start to isolate a bug and during that task a second bug stops us from working on the first bug etc. The only way to relentlessly follow all those tasks is to enable branching universes in your working environment. The frequent use of the work “task” may give a hint that I should finally start using Mylyn (I have no excuse for not doing so), but I would need a Mylyn that is able to capture full contexts: the complete state of my machine plus the full state of my brain. As a start I’m dreaming of always working in a virtual machine, and whenever something strange happens, I quickly (!!) create a snapshot of the virtual machine. Then I could first isolate (and fix 🙂 ) the bug that just occurred, and then go back to the exact point where I
interrupted my work and act as if nothing had happened. Branching universes with the ability of back porting fixes between branches is what I need. Of course the clock needs to be reset when returning from a bug reporting / fixing branch.

Yeah, that’s why I can’t quite live up to my dreams of good participation in open source development: I haven’t figured out how to enable branching universes for my character. If anybody has a hint on how to do this or any other strategy to not get overwhelmed between work and bug reporting, I’d really appreciate.

And if I don’t smile while writing a bug report, please excuse, I might just be terribly stressed because your bug interrupted my work on isolating another bug that stopped me from doing …

”Always look at the bright side of life…” 🙂

Advertisements

Written by Stephan Herrmann

February 7, 2010 at 18:31

Posted in Eclipse

Tagged with , , , , ,

5 Responses

Subscribe to comments with RSS.

  1. Hi Stephan

    I know the feeling of not reporting bugs because one would get distracted too much, or a handy workaround already became a habit. Guess everybody experiences this once in a while.
    You have the right idea to at least minimize occurrences of this problem, by using Mylyn. If everything you work on is captured as a task (and that task not being “work”, but a very specific item, preferably finish-able in a day or less), you will find that the context getting saved might just be enough to get you back into the right state after submitting or fixing that bug that just came up.

    Sometimes, when I realize that I will be distracted longer, or I had some things in my mind I wanted to do but can’t right away, I just make some notes in the private section of the Task Editor, to help me remember when I get back to the task.

    If a lot of your tasks are not just developing, but various other things, like researching in the web, or grading papers (in case you are still teaching at the TU Berlin – I had the pleasure attending some courses of yours), you might want to give Tasktop a try as it captures a much broader context.

    So I hope you find some time to try out Mylyn. A colleague of mine, David, wrote some nice articles about best practices with Mylyn (and Tasktop):
    http://tasktop.com/resources/tutorials/

    Thomas Ehrnhoefer

    February 10, 2010 at 19:47

  2. Thomas, thanks for your kind words.

    I’ve indeed followed the Mylar work pretty much from the beginning and I highly appreciate everything about it. Actually I’m still looking for an enthusiast you writes a bridge for OT/J in Mylyn. I already spoke to Mik about this but never found the student who was eager to jump in. Maybe you know somebody for this task? 🙂

    OTOH, even with the best Mylyn the dilemma doesn’t just disappear. As I said, during this big refactoring session I did report a couple of bugs. Unfortunately, after the refactoring was done, one of the bugs was no longer reproduceable. It had to be analyzed right in the second it occurred or never.

    Sometimes I just wonder whether it is normal that right when my work requires ultimate concentration I don’t just hit one bug but they are seeming to have a party on my computer, they just keep coming.

    Put still differently: I’d really like to appreciate each bug that unveils itself, because with the right steps this will mean the software will be improved. Just in some situations that appreciation is hampered a bit by the limitations of my very self in its linear universe 🙂

    stephan

    February 11, 2010 at 01:02

  3. I like the idea of a “save bug context” button which asks for a (short) description and saves the current state of Eclipse (= Java memory dump). I’m not sure it’s possible to return to this state but a screen shot and a way to browse the memory would help a lot.

    I often run in p2 bugs and I can’t produce useful bug reports because the UI just doesn’t give me any. So if I could save the state of the resolver, the involved repositories and a screen shot, that would already help a lot.

    Aaron Digulla

    February 26, 2010 at 17:09

  4. I’ve opened a new bug for this:

    https://bugs.eclipse.org/bugs/show_bug.cgi?id=304544

    Aaron Digulla

    March 3, 2010 at 17:54

  5. […] at 20:17 | In Software | Leave a Comment Tags: Bug Reporting, Bugs, Software, Software Development In his blog, stephan writes about the problems you can have as a bug reporter. Basically, when you encounter a bug, […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: