Bug tracking (no, not *that* one... a different question)

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Bug tracking (no, not *that* one... a different question)

Nyall Dawson
Hey PSC,

I've been thinking more about this topic lately, and I'd like to
re-ignite discussion about whether we should be automatically closing
bug reports after a certain cut off time period.

In the past I've been against this (simply because I think there's
always value in older unresolved tickets), but I've recently changed
my mind and would like to see us adopt an auto-close after ## days of
inactivity for open bug tickets.

Here's my thoughts:

- We've had great success by implementing a 21 day cut off for
auto-closing stale pull requests. After 14 days of inactivity a
warning is automatically added to the PR, and if no further activity
occurs after 7 days the request is closed. This has really pushed back
the effort to original submitters to keep momentum up on THEIR
requests. It's up to them to fix issues, request updated reviews, etc.
This considerably eases the burden on the core development team and
helps share the workload around.

- Because of this, I think the same approach would be valuable for bug
tickets. It pushes the effort back to the original submitter to ensure
that their ticket has sufficient detail to be reproducible, meaning
that the overworked bug triaging team don't always have to be the ones
pushing them for information.

- It helps send the correct message that QGIS is an open source
software project, not a charity service, and not a 24x7 software
service hotline. If they want their bug fixed it's up to them to find
ways to make this easy for us.

- We are drowning in poor value tickets, with limited information, no
test data, etc. Unfortunately, despite TIRELESS efforts by our
fantastic bug triagers it's still very hard to identify good bug
tickets to target. This has gotten more difficult with the addition of
the automated crash reports, and the increase in reporters who are
just pasting the crash report with absolutely no useful other
information.

- People can always reopen tickets if they are auto-closed and they
are still an issue. In this case auto-closing is a good thing, because
it prompts THEM to test latest versions and do the work checking that
the ticket is still valid.

- If a ticket is auto-closed multiple times, it will help convey the
message to the reporter that their bug is low priority for the
project. We could then make the auto-close message hint to the
reporter that they may need to directly engage a QGIS developer or
support contract if they want a speedy resolution to their issue.

- We see roughly 5-6 new tickets opened daily, and maybe a ticket
closed every second day (very rough figures!). The current growth rate
of tickets is making the situation more and more difficult.

Thoughts?

Nyall





- This should only apply to bug tickets, not feature requests. Feature
requests are a different beast and should have different rules.
_______________________________________________
Qgis-psc mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/qgis-psc
Reply | Threaded
Open this post in threaded view
|

Re: Bug tracking (no, not *that* one... a different question)

Alessandro Pasotti-2
Hi Nyall,

thanks for raising this point!

I 100% agree with you proposals, but I have some additional topics that could IMO help a lot to increase the efficiency of bugfixing process, I'm talking about increasing the S/N ratio.
Sorry for the long email, but I think we really need to do something because the task of finding a bug ready to be processed is becoming more and more painful, time consuming and frustrating.

Recently I took part to several bugfixing sprints (but I do this kind of activity also as a volunteer) and this is roughly the process that I have to do several times a week (sometimes more than daily):

1. go through the bug queue (filtered to identify bugs open)
2. click on a ticket link to read the details
3. 95% of the times read some garbage information with some stack-trace
4. 95% of the times ask for some more information and for data/sample project
5. sometimes I just discover that another developer has already answered and he's probably taking care of the bug or willing to do that

6. Another part of my bugfixing time goes with reading email bug notifications (ok this is faster, when the email notification work) but still takes time and most of the times it is wasted time because all what has changed is (for example) a typo in the title or a discussion about if it's a regression or not.

So, here are my proposals to increase the S/N ratio:

a.  immediately close (well, after a few days) the tickets that do not have enough information (example: https://issues.qgis.org/issues/20069 https://issues.qgis.org/issues/19922), a simple criteria for these automatic tickets it to look at the stacktrace and see if is there any Qgs prefixed name there, if not: close because nobody of us will waste time on that ticket, so better spare our attention

b . immediately ask for some more information when it makes sense (and most of the times it does), I think that the triagers should have a way to call the attention of a developer if they have doubts on this point. Note that a sample project/data and/or a screencast almost always helps and greatly speeds up the fixing process.

c. developers should be good citizens and assign bugs to themselves when working on a bug: it's a stupid waste of time when you open a ticket, read it all and the discover that there is a pending PR that addresses it ... ok this just happened one or twice, but unassigned bugs are the rule, not an exception, that means that you need to open every issue if you want to know if it's really unassigned (meaning: nobody is publicly working on it). This is also respectful for other devs time.

d. we should have a clear and documented definition of what criteria are behind the bug priority (and status), is it a logic AND of crash and regression flag? (apparently not). What I'm trying to say here is that I've seen tickets without ANY useful information in them that are high priority, at this time the priority flag does not help at all in finding a bug ready to be processed.

That said, sorry if that sounds like a rant, I really appreciate all the hard work behind the bug queue management  and I wish to thank to all people involved in bug triaging and fixing, we are a team, we can work together and that's what matters!


On Fri, Oct 12, 2018 at 1:56 AM Nyall Dawson <[hidden email]> wrote:

>
> Hey PSC,
>
> I've been thinking more about this topic lately, and I'd like to
> re-ignite discussion about whether we should be automatically closing
> bug reports after a certain cut off time period.
>
> In the past I've been against this (simply because I think there's
> always value in older unresolved tickets), but I've recently changed
> my mind and would like to see us adopt an auto-close after ## days of
> inactivity for open bug tickets.
>
> Here's my thoughts:
>
> - We've had great success by implementing a 21 day cut off for
> auto-closing stale pull requests. After 14 days of inactivity a
> warning is automatically added to the PR, and if no further activity
> occurs after 7 days the request is closed. This has really pushed back
> the effort to original submitters to keep momentum up on THEIR
> requests. It's up to them to fix issues, request updated reviews, etc.
> This considerably eases the burden on the core development team and
> helps share the workload around.
>
> - Because of this, I think the same approach would be valuable for bug
> tickets. It pushes the effort back to the original submitter to ensure
> that their ticket has sufficient detail to be reproducible, meaning
> that the overworked bug triaging team don't always have to be the ones
> pushing them for information.
>
> - It helps send the correct message that QGIS is an open source
> software project, not a charity service, and not a 24x7 software
> service hotline. If they want their bug fixed it's up to them to find
> ways to make this easy for us.
>
> - We are drowning in poor value tickets, with limited information, no
> test data, etc. Unfortunately, despite TIRELESS efforts by our
> fantastic bug triagers it's still very hard to identify good bug
> tickets to target. This has gotten more difficult with the addition of
> the automated crash reports, and the increase in reporters who are
> just pasting the crash report with absolutely no useful other
> information.
>
> - People can always reopen tickets if they are auto-closed and they
> are still an issue. In this case auto-closing is a good thing, because
> it prompts THEM to test latest versions and do the work checking that
> the ticket is still valid.
>
> - If a ticket is auto-closed multiple times, it will help convey the
> message to the reporter that their bug is low priority for the
> project. We could then make the auto-close message hint to the
> reporter that they may need to directly engage a QGIS developer or
> support contract if they want a speedy resolution to their issue.
>
> - We see roughly 5-6 new tickets opened daily, and maybe a ticket
> closed every second day (very rough figures!). The current growth rate
> of tickets is making the situation more and more difficult.
>
> Thoughts?
>
> Nyall
>
>
>
>
>
> - This should only apply to bug tickets, not feature requests. Feature
> requests are a different beast and should have different rules.
> _______________________________________________
> Qgis-psc mailing list
> [hidden email]
> https://lists.osgeo.org/mailman/listinfo/qgis-psc



--
Alessandro Pasotti
w3:   www.itopen.it

_______________________________________________
Qgis-psc mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/qgis-psc
Reply | Threaded
Open this post in threaded view
|

Re: Bug tracking (no, not *that* one... a different question)

pcav
In reply to this post by Nyall Dawson
Hi Nyall,


Il 10/12/2018 01:56 AM, Nyall Dawson ha scritto:
>
> - We see roughly 5-6 new tickets opened daily, and maybe a ticket
> closed every second day (very rough figures!). The current growth rate
> of tickets is making the situation more and more difficult.
>
> Thoughts?
>
Your arguments are convincing to me. However, are you sure about the
figures? I used to monitor the bug queue very regularly, and for what I
remember we are around 4k tickets (more issues than feature requests)
since many years. I do not see a deteriorating situation here.
I also understand that keeping a long list adds a lot of noise and makes
it difficult to spot the workable tickets, and I think this can be
solved by a wise use of tags. I'm still worried about missing important
information a ticket may convey, even if not immediately useful.
All the best.

--
Paolo Cavallini - www.faunalia.eu
QGIS.ORG Chair:
http://planet.qgis.org/planet/user/28/tag/qgis%20board/


_______________________________________________
Qgis-psc mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/qgis-psc
Reply | Threaded
Open this post in threaded view
|

Re: Bug tracking (no, not *that* one... a different question)

Nyall Dawson
On Fri, 12 Oct 2018 at 15:53, Paolo Cavallini <[hidden email]> wrote:

>
> Hi Nyall,
>
>
> Il 10/12/2018 01:56 AM, Nyall Dawson ha scritto:
> >
> > - We see roughly 5-6 new tickets opened daily, and maybe a ticket
> > closed every second day (very rough figures!). The current growth rate
> > of tickets is making the situation more and more difficult.
> >
> > Thoughts?
> >
> Your arguments are convincing to me. However, are you sure about the
> figures?

They aren't scientific at all - just a "gut feeling" based on daily
study of the bug tracker ;)

> I'm still worried about missing important
> information a ticket may convey, even if not immediately useful.

I think it's also important to keep in mind that a closed ticket
doesn't lose any valuable information -- it's not deleted, just
closed. It's always easy enough to re-open (by us, or the original
reporter) if the bug proves to still exist.

Nyall


> All the best.
>
> --
> Paolo Cavallini - www.faunalia.eu
> QGIS.ORG Chair:
> http://planet.qgis.org/planet/user/28/tag/qgis%20board/
>
>
> _______________________________________________
> Qgis-psc mailing list
> [hidden email]
> https://lists.osgeo.org/mailman/listinfo/qgis-psc
_______________________________________________
Qgis-psc mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/qgis-psc
Reply | Threaded
Open this post in threaded view
|

Re: Bug tracking (no, not *that* one... a different question)

pcav
Hi Nyall,


Il 10/12/2018 08:21 AM, Nyall Dawson ha scritto:
> They aren't scientific at all - just a "gut feeling" based on daily
> study of the bug tracker ;)
my feeling is that the number of unresolved tickets is rather stable
across the years which, giving the huge increase of number of users,
speaks very well of the project.
> I think it's also important to keep in mind that a closed ticket
> doesn't lose any valuable information -- it's not deleted, just
> closed. It's always easy enough to re-open (by us, or the original
> reporter) if the bug proves to still exist.

quite right, but in practical terms I do not think anybody will dig into
the closed tickets.
In short, I'm calling for extra caution before closing tickets, but for
the rest I agree with your strategy.
All the best, and thanks.

--
Paolo Cavallini - www.faunalia.eu
QGIS.ORG Chair:
http://planet.qgis.org/planet/user/28/tag/qgis%20board/


_______________________________________________
Qgis-psc mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/qgis-psc
Reply | Threaded
Open this post in threaded view
|

Re: Bug tracking (no, not *that* one... a different question)

Nyall Dawson
On Fri, 12 Oct 2018 at 17:12, Paolo Cavallini <[hidden email]> wrote:

> quite right, but in practical terms I do not think anybody will dig into
> the closed tickets.

Not sure if it's possible in redmine, but maybe we could have a status
for "closed due to inactivity". It's not closed as resolved, but just
closed due to lack of responses. Then it'd still be possible to search
for unfixed issues, but at the same time the only open issues would be
those which the original poster is actively maintaining.

> In short, I'm calling for extra caution before closing tickets, but for
> the rest I agree with your strategy.

I'm a little confused -- my whole strategy is that tickets are
AUTOMATICALLY closed after a period of inactivity, not manually (which
is the current approach).

Nyall

> All the best, and thanks.
>
> --
> Paolo Cavallini - www.faunalia.eu
> QGIS.ORG Chair:
> http://planet.qgis.org/planet/user/28/tag/qgis%20board/
>
>
_______________________________________________
Qgis-psc mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/qgis-psc
Reply | Threaded
Open this post in threaded view
|

Re: Bug tracking (no, not *that* one... a different question)

pcav
Hi Nyall


Il 10/12/2018 09:59 AM, Nyall Dawson ha scritto:
>
> Not sure if it's possible in redmine, but maybe we could have a status
> for "closed due to inactivity". It's not closed as resolved, but just
> closed due to lack of responses. Then it'd still be possible to search
> for unfixed issues, but at the same time the only open issues would be
> those which the original poster is actively maintaining.
this could help
> I'm a little confused -- my whole strategy is that tickets are
> AUTOMATICALLY closed after a period of inactivity, not manually (which
> is the current approach).
>
sure, automatically. the caution I'm talking about would mean having a
not-too-short period before closing, and perhaps having someone
responsible for checking whether an autoclosed issue is it worth
reopening, or possibly better ideas. The rationale is: an user can open
a valid ticket, with proper information, but he's not interested in
following the work further, perhaps because the ticket does not impact
his work directly, or he found a workaround. Closing this will mean a
loss of valid info.

All the best.

--
Paolo Cavallini - www.faunalia.eu
QGIS.ORG Chair:
http://planet.qgis.org/planet/user/28/tag/qgis%20board/

_______________________________________________
Qgis-psc mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/qgis-psc
Reply | Threaded
Open this post in threaded view
|

Re: Bug tracking (no, not *that* one... a different question)

Giovanni Manghi
In reply to this post by Nyall Dawson
thanks Nyall for raising this,

On Fri, Oct 12, 2018 at 12:56 AM Nyall Dawson <[hidden email]> wrote:
>
> Hey PSC,
>
> I've been thinking more about this topic lately, and I'd like to
> re-ignite discussion about whether we should be automatically closing
> bug reports after a certain cut off time period.


maybe is the right occasion to discuss about what to do (if anything)
to 2.* and 1.* tickets the moment that the new LTR will be out.
Personally I'm much in favor of using this occasion (the new LTR
release) for a mass clean up > closing all those tickets asking to
check if they are still valid for the new LTR and eventually reopen
them.

cheers

-- G --
_______________________________________________
Qgis-psc mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/qgis-psc
Reply | Threaded
Open this post in threaded view
|

Re: Bug tracking (no, not *that* one... a different question)

pcav
Hi all,


Il 10/12/2018 11:30 AM, Giovanni Manghi ha scritto:
> maybe is the right occasion to discuss about what to do (if anything)
> to 2.* and 1.* tickets the moment that the new LTR will be out.
> Personally I'm much in favor of using this occasion (the new LTR
> release) for a mass clean up > closing all those tickets asking to
> check if they are still valid for the new LTR and eventually reopen
> them.
>
>
this makes sense to me.
All the best.

--
Paolo Cavallini - www.faunalia.eu
QGIS.ORG Chair:
http://planet.qgis.org/planet/user/28/tag/qgis%20board/

_______________________________________________
Qgis-psc mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/qgis-psc