Dispute Resolution Forum

Daemos

Having seen several threads in the DRF I have to say that, while I think the idea behind the forum is a good one, I think that in practise the forum has been a failure, and should be closed and replaced with the old system.

Before I start I just want to say that I mean no offence to anyone if I cite their posts as examples of problems, they are only examples and not personal attacks.

The problems I see are:

1) Mods don't seem to understand/adhere to the rules of the forum

There are several instances of mods posting on threads, without clear invitation from CMods or Admins to do so, which is one of the rules of the forum. With all due respect to moderators, I fail to see how this procedure can be properly done if moderators can't seem to stick to procedures they are supposed to be following, or don't understand how the procedure works. Examples here, here, and here.

2) The time it takes to process an appeal

Now I understand that Mods, CMods and Admins don't sit at their desks all day processing every single mite of information, however the lenght of time some people have to wait for their appeals to be processed is, quite simply, embarrassing. There are cases where people have had to wait days just to get a reply, and others where there ban has already expired by the time their appeal has finished. Now obviously I can't see what work is being done behind the scenes, but if I thought I was unfairly banned, and then follow an appeals process only to be ignored*, I'd be very annoyed. Again, I don't exactly know what the turnover time was for appeals to CMods in the old system, but taking the current system on it's own merits, I think this fact clearly shows that the process is a failure, and needs to be re-examined. Examples here, here, and here.

* I know I wouldn't really be being ignored, but that's how it would feel

3) "Technicalities"

One of the problems I see with the process is that technicalities seem to be taken advantage of on both sides of the fence. Users try to get out of bans through them, but unfortunately I've seen cases where other points have been ignored or disregarded, while the point where someone is wrong gets focused on. I think this is very unfair, and as an observer looking from the outside-in, seems as though the users mistakes are being focused on, while others are swept under the carpet. I'm hoping that isn't the case, but that is how it looks. Examples here and here.

Again, I don't know what these processes were like under the old system, but I find it very hard to believe that it was better than the one in place now, and think that the whole forum needs to be scraped, or at least given a serious overhaul.

r3nu4l

Thanks for the feedback Daemos, clear, well thought out and not at all offensive so don't fret on that score.

I'll try to deal with your points in turn.

1) Mods not understanding/ignoring the rules.
Yes, we're aware of it and not very happy with that. Sometimes mods are trying to be helpful, other times they're trying to get their word in first, other times they are outraged by what is an obvious lie or twisting of the truth by the poster in question. Doesn't really matter what it is, it shouldn't happen. We've deleted such posts in the past and asked the CMod to deal with it. We've talked about it on the mod forum but just like some users who don't know when to stop or don't read the rules properly before posting, some mods are the same. I think there are fewer mods doing this now than there was a couple of months ago but maybe it's time to revisit this in the mod forum.

2) Time it takes to process.
Yes, again this is being discussed actively in the Admin forum right now. We are frustrated by a number of elements that mean that something like a 3 day ban can take 7 days to be dealt with. The result? Poster stays banned anyway, whether ban would be overturned or not. We (the Admins) are debating how best to deal with this but while we have some ideas we do not (as yet) have a definitive solution.

3) Technicalities.
I understand what you're saying here and it is frustrating, however, even the old system suffered from this. The problem with a public and open system is that users, mods, CMods and Admins all know that people are watching. While we all hope that things will remain unemotive, it doesn't always happen that way and sometimes people start 'point-scoring' which is unproductive.


So we s Admins are aware of these issues, frustrated by some and actively working on solutions to others. However, you should realise that there are people here who have long histories with boards.ie. Some of them come to the DRF with an agenda already set and will do everything they can to sew the seeds of that agenda in the forum to be picked up by others. Sad but true.

It's also fair to say that the vast majority of infractions and bans handed out on boards.ie are either taken on the chin by mature users who realise they deserved it or resolved amicably by PM before they ever even reach the DRF. So those people posting in the DRF have already exhausted a number of avenues for resolution and as such DRF threads are always going to be difficult in many ways.

Thanks again for the feedback.

EDIT: I should also have said that there is a 5-page thread active in the Moderators forum on the DRP and DRF.

EDIT2: Post by Overheal deleted: Quoting from the mod forum, incorrectly attributed to me AND quoted out of context.

metrovelvet

r3nu4l wrote: »

3) Technicalities.

It's also fair to say that the vast majority of infractions and bans handed out on boards.ie are either taken on the chin by mature users who realise they deserved it or resolved amicably by PM before they ever even reach the DRF. So those people posting in the DRF have already exhausted a number of avenues for resolution and as such DRF threads are always going to be difficult in many ways.

Honestly, it looks to me like you have no means by which to let a moderator not lose face and so you are obliged to always back them up because they work for your profit making message boards free of charge.

And with inconsistent rules on abuse this is going to happen over and over and over again, and it really does look like a tokenistic gesture to appear fair without much behind it and just a big time wasting excersize.

Zaph

metrovelvet wrote: »

Honestly, it looks to me like you have no means by which to let a moderator not lose face and so you are obliged to always back them up because they work for your profit making message boards free of charge.

It's probably worth pointing out that r3nu4l, myself and just about all the other admins also do what we do on Boards for free, so we've nothing to gain by backing up a mod, regardless of what the dispute is. Not that this is the case anyway, but people generally don't want to believe that. It's probably also worth pointing out that Boards doesn't actually make a profit, and in fact runs at a considerable loss to its owners.

opinion guy

metrovelvet wrote: »

Honestly, it looks to me like you have no means by which to let a moderator not lose face and so you are obliged to always back them up because they work for your profit making message boards free of charge.

And with inconsistent rules on abuse this is going to happen over and over and over again, and it really does look like a tokenistic gesture to appear fair without much behind it and just a big time wasting excersize.

I agree with metrovelvet here. Couple of disputes I had prior to the dispute resolution forum that I escalated to CMod's I can recall feeling the system was biased against me. The best result I got was kind of a draw where I was told 'yeah basically I can see where you are coming from but these guys mod the forum by invite on their spare time so we have to give them some sway - so you will be unbanned but you can't discuss x,y,z on that forum'. Which by the way is basically censorship (but thats another issue)

Bascially at this point I can't even be bothered disputing anything (not that the need arises often)

Quazzie

Daemos wrote: »

2) The time it takes to process an appeal

Now I understand that Mods, CMods and Admins don't sit at their desks all day processing every single mite of information, however the lenght of time some people have to wait for their appeals to be processed is, quite simply, embarrassing. There are cases where people have had to wait days just to get a reply, and others where there ban has already expired by the time their appeal has finished. Now obviously I can't see what work is being done behind the scenes, but if I thought I was unfairly banned, and then follow an appeals process only to be ignored*, I'd be very annoyed. Again, I don't exactly know what the turnover time was for appeals to CMods in the old system, but taking the current system on it's own merits, I think this fact clearly shows that the process is a failure, and needs to be re-examined. Examples here, here, and here.

* I know I wouldn't really be being ignored, but that's how it would feel

For me this is the biggest failure of the DRP. Over a month later and the ban served and there is still no comment from the CMOD on this thread. For most bans the length of time it takees for the DRP, combined with the fact that the chance of getting a ban overturned are so slim, means that most people who know the system won't bother doing anything about it. The DRP needs a huge review imho and as bad as the old system was, at least stuff got done. The current system just feels like a way to lead the complainant along a merry path until they get bored or the admins can say 'Sure your bans up now so run along'.

RoundyMooney

I don't think that's by design, though.

It seems to me that issues surrounding that particular forum are the bone of contention (not for the first time, it has to be said).

Even so, the chap was perfectly reasonable, and tbh, deserved better. His issue could have been looked at and resolved, and the network brane could then have looked at whatever over riding issues were obviously at play for them.

I don't think the system is wrong, I agree with it actually, but I do think the implementation is piss poor in some cases. It needs more structure at the back end. Even volunteerism can be organised.

Overheal

Quazzie wrote:

For me this is the biggest failure of the DRP. Over a month later and the ban served and there is still no comment from the CMOD on this thread. For most bans the length of time it takees for the DRP, combined with the fact that the chance of getting a ban overturned are so slim, means that most people who know the system won't bother doing anything about it. The DRP needs a huge review imho and as bad as the old system was, at least stuff got done. The current system just feels like a way to lead the complainant along a merry path until they get bored or the admins can say 'Sure your bans up now so run along'.

r3nu4l wrote: »

2) Time it takes to process.
Yes, again this is being discussed actively in the Admin forum right now. We are frustrated by a number of elements that mean that something like a 3 day ban can take 7 days to be dealt with. The result? Poster stays banned anyway, whether ban would be overturned or not. We (the Admins) are debating how best to deal with this but while we have some ideas we do not (as yet) have a definitive solution.

This is the core of the problem imho. It's something that affects mods and users alike and rather than bouncing a pinch of ideas back and forth between admins behind a curtain, it would be my suggestion to fire the whole thing through the FFW mill and get critical thinking from all types on the site and lets get considerably more heads pondering whatever options you are considering or havent thought up yet.

Tom Dunne

By far, the biggest problem we have on this site is the fact that people are volunteers.

Because of this, it is not reasonable to enforce targets or timelines on people. Who know where Real Life will encroach on your Boards time? I know from my own perspective, I have been so busy over the last month or two, I simply have not got the time for the site (on a coffee break right now ).

I don't know what the solution is.

r3nu4l

Just an update for my part. We've recently had a ban overturned completely by CMods and a ban reduced from five months to two weeks in another case. So to those of you who say we're afraid to upset the applecart, there's your answer.
To help speed things up a little we've also added additional CMods to some of the bigger categories too. As Tom Dunne said, we're all volunteers so sometimes when 'real life' intrudes then things don't get the attention that they should. The addition of extra CMods to those categories should help reduce those instances.

We're still not completely happy with certain aspects of the DRP/DRF and we're working on improving those but it will take time. In the meantime, hopefully the changes we've made will improve things. Thanks again Daemos.

NeedaNewName

r3nu4l wrote: »

Just an update for my part. We've recently had a ban overturned completely by CMods and a ban reduced from five months to two weeks in another case. So to those of you who say we're afraid to upset the applecart, there's your answer.
To help speed things up a little we've also added additional CMods to some of the bigger categories too. As Tom Dunne said, we're all volunteers so sometimes when 'real life' intrudes then things don't get the attention that they should. The addition of extra CMods to those categories should help reduce those instances.

We're still not completely happy with certain aspects of the DRP/DRF and we're working on improving those but it will take time. In the meantime, hopefully the changes we've made will improve things. Thanks again Daemos.

Are there links to the ban over turn and the reduction?

Thanks.

r3nu4l

RTE News wrote: »

Are there links to the ban over turn and the reduction?

Thanks.

Why of course RTE News, (glad to see your username isn't impersonating RTÉ News! ).

1. Politics ban overturned.

2. DKIT ban reduced from 5 months to 3 weeks.


In the first instance the problem wasn't dealt with quickly enough but that was acknowledged on thread and apologies made. We've added another CMod to that Category, that may help in the future but there's never any gaurantee as mods, CMods and Admins give their time voluntarily and sometimes that pesky 'real-life' stuff gets in the way

In the second instance the issue was dealt with more quickly and the user complaint upheld.

NeedaNewName

r3nu4l wrote: »

Why of course RTE News, (glad to see your username isn't impersonating RTÉ News! ).

1. Politics ban overturned.

2. DKIT ban reduced from 5 months to 3 weeks.


In the first instance the problem wasn't dealt with quickly enough but that was acknowledged on thread and apologies made. We've added another CMod to that Category, that may help in the future but there's never any gaurantee as mods, CMods and Admins give their time voluntarily and sometimes that pesky 'real-life' stuff gets in the way

In the second instance the issue was dealt with more quickly and the user complaint upheld.

Yeah I thought the second one was going to be the DKIT one and as such I asked.

Welcome to the world of slight of hand. Ban reduced from 5 months to 3 weeks!

Mod made an error in clicking. Hardly a world changing reduction in a ban.

Welcome to the politics.

AnonoBoy

RTE News wrote: »

Yeah I thought the second one was going to be the DKIT one and as such I asked.

Welcome to the world of slight of hand. Ban reduced from 5 months to 3 weeks!

Mod made an error in clicking. Hardly a world changing reduction in a ban.

Welcome to the politics.

Funny, you don't mention then one that was overturned completely.

Is this going to be on Primetime shortly?

r3nu4l

RTE News wrote: »

Mod made an error in clicking. Hardly a world changing reduction in a ban.

Do you know something that I don't? The mod did not make an error in clicking 5 Months. That's what he meant at the time. I personally PM'ed the mod to say that 5 Months was ridiculous and in fairness to him he didn't try to weasel out of it by saying it was an accidental click of a mouse button, he admitted it was too harsh.

Now I accept that there are some boards.ie users who will always look for a reason to have a go at the Mods/CMods or Admins on this site but really you're wrong here. Totally wrong.

IITYWYBMAD

r3nu4l wrote: »

Do you know something that I don't? The mod did not make an error in clicking 5 Months. That's what he meant at the time. I personally PM'ed the mod to say that 5 Months was ridiculous and in fairness to him he didn't try to weasel out of it by saying it was an accidental click of a mouse button, he admitted it was too harsh.

Now I accept that there are some boards.ie users who will always look for a reason to have a go at the Mods/CMods or Admins on this site but really you're wrong here. Totally wrong.

How many complaints have been made since the DRP was setup? How many where dealt with within the time frame of the ban? What is the ratio of overturned/non-overturned appeals? What measurements are being used to deem this "new" procedure a success or not?

Tom Dunne

IITYWYBMAD wrote: »

How many complaints have been made since the DRP was setup? How many where dealt with within the time frame of the ban? What is the ratio of overturned/non-overturned appeals? What measurements are being used to deem this "new" procedure a success or not?

I refer you to my previous post.

We are not employees, we are volunteers. As such, what we do on this site is not subject to measureables and deliverables.

IITYWYBMAD

Tom Dunne wrote: »

I refer you to my previous post.

We are not employees, we are volunteers. As such, what we do on this site is not subject to measureables and deliverables.

Hi Tom,

thanks for that. I understand that you are not an employee per-se, however it should not preclude you from measurables etc..., not that my question was specifically aimed at you. However there are plenty of volunteers out there who "deliver" to a certain measurement and I'm sure the owners of this site have targets for this site.

I would also contend that irrespective of your status on this site "what you do on this site" should be measured in some fashion, as it is the yardstick by which the community decides if it is working or not.

Given that the DRP was introduced relatively recently to address a specific issue, and also given that you feel that you are not in a position to answer this question because of your lack of employment here or whatever, do you not think that somebody should be in a position to address this?

Maybe an employee can address it? I'd presume that there must be some mensurables against it. Or maybe I'm wrong?

Tom Dunne

IITYWYBMAD wrote: »

Hi Tom,

thanks for that. I understand that you are not an employee per-se, however it should not preclude you from measurables etc...,

I am sorry, but I disagree.

The day I start getting "measured" on some sort of metric on this site is the day it turns into an unpaid job. A job I do not want.

But as you say, perhaps it is a question for the site employees.

IITYWYBMAD

Tom Dunne wrote: »

I am sorry, but I disagree.

The day I start getting "measured" on some sort of metric on this site is the day it turns into an unpaid job. A job I do not want.

But as you say, perhaps it is a question for the site employees.

Fair enough, I'll address the metrics question to somebody who feels in a position to answer it. What about the rest of the questions?

i.e. How many complaints have been made since the DRP was setup? How many where dealt with within the time frame of the ban? What is the ratio of overturned/non-overturned appeals? .

r3nu4l

To add to Tom Dunne's point. At what point do you want the employees to stop measuring?

It's unfair to measure all of the variables you mentioned just once, from the beginning of the process. Instead you should measure those variables multiple times.

Firstly as a measure of the forum from the beginning until now, then you should set inclusion criteria such as 'all DRF threads from the point of PROCESS CHANGE 1' where process change 1 is for example, the appointment of a new CMod to a category, or a point such as the beginning of this feedback thread from Daemos.

Then you can truly see if there is any change. Let's say you examine the average turn around time from the first post on a thread to final resolution and you find that timeframe to be 21 days overall. You then need to look at that timeframe after the appointment of a new CMod to that Category. Now let's say you examine that and then find that the average time from first post to response has now dropped to 7 days. It makes it look like the addition of another CMod was responsible for that drop.

However, you then need to look at the timeframes measured. A DRF thread started on December 24th may not get a single response until sometime towards December 27th and as a result may push the timeframe from first response to resolution up to 14 days. Do you exclude that as an outlier? Do you arbitrarily just remove three days over the Christmas period? Or four days? What about August, the traditional 'Holiday month'? What about cases where the Original Poster on the thread doesn't respond for 4 days after the CMod first responded?

Do you examine DRF threads all as one sample or do you examine by Category (Soc, Sci, Rec etc.)? Do you examine resolution by individual CMods?

The kind of analysis you are talking about is not as simple as you seem to think. It would take careful planning and would need a qualified statistical analyst to map out the inclusion and exclusion criterion, definition of outliers and statistical tests to use before any meaningful conclusion to be drawn. Even at that I know from experience that if you put two statisticians in a room together they will agree to a point about how to conduct an analysis and then proceed to murder one another over some obscure probability function that one or other of them wants to use.

To top all of that, I sincerely doubt there is enough data in the DRF to allow the tests to be powered for significance.

The numbers themselves are no good without understanding the data behind them.

It's certainly a potentially useful exercise for boards.ie to conduct but releasing the data to users without fully explaining the methods behind the data would be useless.

The bottom line is that we are continuing to work to improve the process. I feel it's improving but it's still not perfect (if perfection is possible in any dispute resolution procedure). Admins are designating themselves to overseeing particular threads and where necessary, contacting CMods to ensure they are aware of the presence of the thread. We then watch the thread and wait to see if we are needed to make a ruling. Sometimes we are, others happily not

r3nu4l

Okay I've conducted a little exercise.

There are 120 threads in the Dispute Resolution Forum. Please remember that this is 120 complaints that were not settled with the mod. Boards.ie mods, CMods and Admins issue far more than 120 moderator actions, warnings, infractions, bans each day so that 'statistic' is quite heartening! Seems like we have very few complaints at all! (Or that some people don't think it's worth going through the DRP, take your pick )

So I looked at threads on the front page and chose the first five marked 'resolved':

First five resolved threads on Page 1 (view = 20 threads per page )
Ban overturned (belatedly!)
Category ban reduced to a 6 month ban from one forum
5Month ban reduced to 3 weeks
1 week ban overturned
Weird one, no specific actionable complaint, no action taken

What we actually see here is something I wasn't expecting!! Those fncking Admins and CMods keep overturning and reducing Moderator bans!!! It's like their always sticking up for the user, damn CMod/Admin/User clique, always seeking to put the Mods down!! :mad:

So then I went to the last page of the forum and it's a complete mess. It's testimony to the fact that nobody really knew how to use the forum back then, this is before any of the more recent changes were put in and before the big flow-chart procedural diagram at the top of the forum was inserted.

First five resolved threads on Page 1 (view = 20 threads per page )
Mod accepts fault
Spammer complains
Misunderstnaing of boards.ie term 'don't be a dick'
A satanist (no less!) complains about ban, doesn't follow up complaint for too long
Personal abuse ban upheld

What we see here is a mix of bans being upheld, moderators accepting that they could have handled situations better (we're only human!), users dropping complaints and basic misunderstandings.

Now that is in no way a statistical analysis, it's an extremely crude sample. However it shows one very heartening thing.
Since we've implemented changes, things HAVE improved.

I haven't got all day so I haven't looked at the timeframe of resolution I do know some were quick and some took way too long.

There is no dispute here (pardon the pun) in that the DRP and DRF need further improvement. However, what is very clear is that the myth that CMods and Admins always uphold bans is a Complete fallacy. There are improvements to be made, we don't deny that but we are working on it (as I said in the second post in this thread...).

IITYWYBMAD

r3nu4l wrote: »

To add to Tom Dunne's point. At what point do you want the employees to stop measuring?

I think it should be a constant measurement of "where we are at"...for all those stakeholders on the website, irrespective of whether they are employees or volunteers tbh. I don't believe it should ever stop.

It's unfair to measure all of the variables you mentioned just once, from the beginning of the process. Instead you should measure those variables multiple times.

Unfair? Unfair on whom? should it not be the starting point?

Firstly as a measure of the forum from the beginning until now, then you should set inclusion criteria such as 'all DRF threads from the point of PROCESS CHANGE 1' where process change 1 is for example, the appointment of a new CMod to a category, or a point such as the beginning of this feedback thread from Daemos.

I appreciate that the fora/site has a certain fluidity to it, but I totally disagree that this should be an "inclusive" factor when looking at performance analysis. I could be used as a contributory factor or an explanatory note, once the results are known, but fluidity is going to be a big factor across all fora, so that should be a given. I could possibly put a metric in place, but I would need more details.

Then you can truly see if there is any change. Let's say you examine the average turn around time from the first post on a thread to final resolution and you find that timeframe to be 21 days overall. You then need to look at that timeframe after the appointment of a new CMod to that Category. Now let's say you examine that and then find that the average time from first post to response has now dropped to 7 days. It makes it look like the addition of another CMod was responsible for that drop.

Nonsense. TBH. These factors should be taken into consideration when setting up the DRP, not as contributing factors for why it's not working (if that is the case).

However, you then need to look at the timeframes measured. A DRF thread started on December 24th may not get a single response until sometime towards December 27th and as a result may push the timeframe from first response to resolution up to 14 days.

So, therefore, what is the point of the DRP?

Do you exclude that as an outlier? Do you arbitrarily just remove three days over the Christmas period? Or four days? What about August, the traditional 'Holiday month'? What about cases where the Original Poster on the thread doesn't respond for 4 days after the CMod first responded?

I'm not in possession of the data, and therefore I'm not in a position to answer that question, however my "stock" answer would be that if you do not have enough people to enforce/police a procedure, it should not be implemented.

How would you feel about a situation whereby a process was initiated, and when you pointed out the process was not working, you were informed that those charged with implementing that process where away for their August break and will get back to you asap.....It's laughable tbh.

You can also "invent" any number of scenario's to make an argument one way or the other about any process, however what I'm asking for is the data, not the reasons behind the data.

Let's have a look at that, and explore why it is/is not working. Maybe it would be benificial for both sides?

Do you examine DRF threads all as one sample or do you examine by Category (Soc, Sci, Rec etc.)? Do you examine resolution by individual CMods?

again, over analysis would be to the determent of the exercise. I would imagine that most disputes would fall into 2 or 3 simple categories. Perhaps you should start from that point... i.e. Why was I banned? I disagree with this banning! I was banned for x while poster Y did not get banned for the same thing.

The kind of analysis you are talking about is not as simple as you seem to think.

what kind? I'm not asking for a statistical breakdown of who, why and when. I'm looking for an "overall" picture of the effectiveness of the DRP. Would you not think that, for example, an analysis of those disputes resolved (irrespective of what forum, or by whom) before the ban period was complete would be a good indicator of whether the process is working or not?

It would take careful planning and would need a qualified statistical analyst to map out the inclusion and exclusion criterion,

Absolutely not so...

definition of outliers and statistical tests to use before any meaningful conclusion to be drawn. Even at that I know from experience that if you put two statisticians in a room together they will agree to a point about how to conduct an analysis and then proceed to murder one another over some obscure probability function that one or other of them wants to use.

Over analysis would simply kill it. As will obfuscation and the inability to directly answer simple queries. Maybe the data simply does not exist?

To top all of that, I sincerely doubt there is enough data in the DRF to allow the tests to be powered for significance.

Ahhh...

The numbers themselves are no good without understanding the data behind them.

What?

The bottom line is that we are continuing to work to improve the process.

In what way? What is the meat behind the soundbite? What are you and your fellow admins doing to improve this?

r3nu4l

IITYWYBMAD wrote: »

In what way? What is the meat behind the soundbite? What are you and your fellow admins doing to improve this?

Huh? If you are asking me that then you obviously haven't read the thread in detail. Please do so and you'll get a better idea.

I disagree with almost everything you've said above, I won't bother multi-quoting.

Also, you say you want statistics to be provided and yet you don't think a qualified statistical analyst is needed? What utter madness! That's the most fundamental aspect of getting good data. The experimental design and method needs to be correct. To have it correct needs proper analysis, not some half-baked wannabe mathematician throwing together a few numbers and getting an average! :eek:

IITYWYBMAD

r3nu4l wrote: »

Huh? If you are asking me that then you obviously haven't read the thread in detail. Please do so and you'll get a better idea.

I disagree with almost everything you've said above, I won't bother multi-quoting.

I see where you say things are improving, and that things are being addressed...but where is the evidence? I've read this thread in it's entirety.

Also, you say you want statistics to be provided and yet you don't think a qualified statistical analyst is needed? What utter madness!

Utter madness? Apologies, are all stats one this site and it's workings provided by a qualified SA? All of them? Including the job you've done above, and your conclusion? I apologise so...

That's the most fundamental aspect of getting good data. The experimental design and method needs to be correct. To have it correct needs proper analysis, not some half-baked wannabe mathematician throwing together a few numbers and getting an average! :eek:

I'm guessing you don't work with data so, and I've no real desire to get into an analytical argument with you. I'm not looking for degrees of error, nor am I looking for a 40 page report on the who, why and where. If the data is there and accurate, it can be easily analysed.....If the will is not there, that's a different matter.

r3nu4l

IITYWYBMAD wrote: »

I see where you say things are improving, and that things are being addressed...but where is the evidence? I've read this thread in it's entirety.

In the Admin forum but I'm afraid that you won't be getting in there to see it. I know it's not transparent but then again I doubt in your company that Senior Management allow you to see all of their data, long-term strategy etc.

Utter madness? Apologies, are all stats one this site and it's workings provided by a qualified SA? All of them? Including the job you've done above, and your conclusion?

I never said there was an SA working on those, nor did I imply that. You are simply taking an argument about one particular forum on this site, applying those arguments sitewide and making an assumption. Not very progressive or even applicable to this discussion.

I apologise so... I'm guessing you don't work with data so,

Ah, getting personal, very old trick. As it happens I do work with data, most of the reports I look at are compiled by SAs and are over 600 pages in length (:(). Boring, boring medical data but it's my job to interpret that data and draw some conclusions so you're very, very wrong.

and I've no real desire to get into an analytical argument with you. I'm not looking for degrees of error, nor am I looking for a 40 page report on the who, why and where. If the data is there and accurate, it can be easily analysed.....If the will is not there, that's a different matter.

This is where we differ in our points of view. The questions you asked earlier about timeframe to resolution of disputes etc are just the very tip of the iceberg. They would simply provide numbers without any actual meaning.
If the result said that the time to resolution has a mean of 4 days then that doesn't get us any closer to knowing anything about where the process is going wrong. To get that info you need to drill deep, very deep into the data. How long until first CMod response, how long until users next response? Is the delay with the CMods and user, or does it kick in when the Admins get involved? Were the resolutions amicable or was the user bitterly opposed? Did the user stop posting on boards.ie as a result? They're the data you need.

You just want the topline figures, the kind of stuff journalists like to use to write headlines extolling the virtue of eating lots of chocolate or to provide an excuse for drinking more red wine or to scare you off that next cup of coffee because it will kill you etc. Topline figures are merely the start.

I agree with you when you said, you never stop measuring, however I disagree with the level of measurement you seem to want. Those topline figures make for 'interesting' reading but as I say, without all of the underlying data that explains those topline figures then those figures are unexplainable and anyone can be free to draw their own conclusions (usually wrong conclusions) as a result.

Therefore those who would take away from the site or feel that the Admins are doing everything wrong would focus on one set of figures while those who love the way boards.ie runs would highlight the other set of figures. Nether group of people would be 'right' though because those figures haven't had the data behind the initial numbers fully analysed.

If you want an argument on metrics and data analysis I can give you one however, as I said, I don't think we are likely to agree. I am trying to be very unbiased in my agenda but recognise that as an Admin, I cannot be fully unbiased, however hard I try because I will still be coming at this from an Admins viewpoint. Therefore for a true measure of the DRF you would need an Independent Data Monitoring Committee (IDMC). However, who would pay the IDMC? He who pays the piper... (as I've learned in my work).

LoLth

IITYWYBMAD wrote: »

How many complaints have been made since the DRP was setup?

106

How many where dealt with within the time frame of the ban?

dont have time right now to go through it as RL is a tad busy

What is the ratio of overturned/non-overturned appeals?

this one is too black and white. Some appeals were upheld and the ban duration/scope reduced by the mods themselves after coming to an agreement with the user or by the cmod/admin if they felt the ban didnt fit the infraction after examining both sides of the argument. Do these count as overturned?

What measurements are being used to deem this "new" procedure a success or not?

personally, I like trend analysis. The DRP hasnt really been there long enough for a proper trend to be realised but , when I have time, I'm sure some pattern is startign to emerge. not sure what to compare it to though, that would take a trawl through helpdesk threads and, not being an analyst, I'm unsure how to weight the timeframes of the two fora to get an accurate comparison. perhaps you have a suggestion on that?

IITYWYBMAD

r3nu4l wrote: »

In the Admin forum but I'm afraid that you won't be getting in there to see it. I know it's not transparent but then again I doubt in your company that Senior Management allow you to see all of their data, long-term strategy etc.

Hmmmm... your confusing me (and obviously yourself) here. Do you not see where you inferred that the answers where here on this very thread? It's just up there, where you accuse me of not reading this thread, when I asked you where this evidence of improvment was, you said I should read this thread. When I said I had read the thread, you tell me the answer is in the Admin forum???

I never said there was an SA working on those, nor did I imply that. You are simply taking an argument about one particular forum on this site, applying those arguments sitewide and making an assumption. Not very progressive or even applicable to this discussion.

I never said you said anything of the sort. I asked the question, when it did not suit you, you talk about site wide analysis, which is beside the point tbh.

Ah, getting personal, very old trick. As it happens I do work with data, most of the reports I look at are compiled by SAs and are over 600 pages in length (:(). Boring, boring medical data but it's my job to interpret that data and draw some conclusions so you're very, very wrong.

Your mis-interpreting my question and conclusion. Firstly, it's not personal at all, I garnered that information from your answer. Secondly, everybody from the shop assistant to the head of the CSO deal with data on an hourly basis. My conclusion was based on your assumption that an SA is concerned with the "experimental design and method (irrespective of their meaning in his context)" of something, when the field of statistics has a multitude of scopes, the most basic of which is collection and presentation.

What (I presume) you are talking about when you refer to experimental design, is survey analysis, which is about as far from what we are talking about in this instance as you can get on a site like this, as I believe we are not in a democracy. Again, I'm not really willing to get into a "he said" type argument with you on this, but as somebody who has worked, to a number of degrees, in this field for a number of years, I know what I'm talking about.

Thanks Lolth for your concise answer, and I totally agree with your final paragraph.

r3nu4l

IITYWYBMAD wrote: »

Hmmmm... your confusing me (and obviously yourself) here. Do you not see where you inferred that the answers where here on this very thread? It's just up there, where you accuse me of not reading this thread, when I asked you where this evidence of improvment was, you said I should read this thread. When I said I had read the thread, you tell me the answer is in the Admin forum???

Actually you got that wrong
1. You ask what we're doing, I tell you the answers are in the thread and to read the thread

2. You then tell me that you now see where I've said what we're doing but THEN you ask for evidence. That's when I told you the evidence is in the Admin forum. I never offered evidence in this thread.

That was the sequence of events. I'm not confused at all.

The rest of it

I've made my point a number of times, a number of different ways. I'm not going to continue. We obviously disagree and will continue to do so. We would both analyse this from different perspectives. You will say your way is right (trend analysis), I would say it has its uses but I'd want much more than that. Let's agree to disagree.

Thanks Lolth for your concise answer, and I totally agree with your final paragraph.

Fair enough


EDIT: I'm stepping back from this thread now. This is getting us nowhere.

jill_valentine

I'm not overly familiar with the DRF (blessedly), but regarding the timeframe problem, would it make sense to appoint a couple of moderators to that forum - and nothing else, to keep them as impartial as possible - who are tasked with chasing up each inquiry? Kind of an Internal Affairs mods group?

IITYWYBMAD

r3nu4l wrote: »

Actually you got that wrong
1. You ask what we're doing, I tell you the answers are in the thread and to read the thread

Correct

2. You then tell me that you now see where I've said what we're doing but THEN you ask for evidence. That's when I told you the evidence is in the Admin forum. I never offered evidence in this thread.

That was the sequence of events. I'm not confused at all.

I didn't say that, but I get where you would infer that conclusion. You can shout all day that the world is flat tbh....If i ask you to show me how the world is flat, and you tell me you have the evidence but cannot show me, I may think a number of things, one being that you have the evidence and another being that you are just bulls**ting me. I can point out what you told me, but it does not address the question no matter how many times you shout that the world is flat...

I've made my point a number of times, a number of different ways. I'm not going to continue. We obviously disagree and will continue to do so. We would both analyse this from different perspectives. You will say your way is right (trend analysis), I would say it has its uses but I'd want much more than that. Let's agree to disagree.


Fair enough


EDIT: I'm stepping back from this thread now. This is getting us nowhere.

No worries, thanks for your time....