Fiverr Community Forum

STARS WARS: The ARS Menace. A response to Kashmiah

I want to thank you Kashmiah for reaching out to the community.

It wouldn’t have been a task for which I volunteered were I a part of the Fiverr team.

I will assume you sincerely like the ARS or you wouldn’t have taken it up as you have. Especially since there isn’t a single concession in the entire post.

Let me state, I believe we’re in the middle of STARS WARS. There is no element of this system I like, no not one. However, I will reply to your post section by section in a how to vanquish the ARS Menace.

kashmiah said: We made an update to the rating system to ensure a better, more transparent community for buyers and sellers. We believe the advanced rating system allows sellers to get proper feedback for the entire Gig experience - the more accurate the ratings become, the more reliable the marketplace will be, separating the mediocre from the great and benefiting the entire community long-term.

There is not "greater transparency" where there are more things opaque.

Sellers only see a "star summary" and the review. The elements that went into that ranking is withheld. And of course, the worse part is...there's no transparency at all because we do not see the thinking of the buyer at the time of the rating.

We don't know if they were leery to give a high rating simply because of a poorly crafted question.

And of course, it's not "greater anything" to consider a "Satisfactory" rating to be "Negative Feedback".

The flaws in this logic are too numerous to list. However, the biggest presupposition is that you can KNOW good work before you PERSONALLY evaluate it after delivery.

We already have mutual cancellation and request modification to get to the cream and evaluate a provider. Why do we think people need to know "before hand"? It's a fools errand as even a MasterChef can forget an ingredient or serve a dish over or undercooked.

Again, we don't get "greater decisions making ability" by merely having more information...especially poorly generated information.
kashmiah saidIndividual ratings will not have an immediate affect on the overall stars- it's one rating factored into the overall average. As a result this won't change much for sellers who are already featured, in the top results and deliver high quality work.

This seems to be little more than a "Divide and conquer" admission that the system MOST HARMS new sellers.

Well, I for one remain in solidarity with all sellers (and buyers). I for one don't want a buyer who was just being a jerk to prevent a number of other buyers from seeing for themselves.

This is the STARS WARS -- not a civil war among sellers.

We don't want it no matter who it hurts most or first.
kashmiah said We've run tests prior to releasing the advanced rating system and there weren't any changes to the way ratings were left.

The tests are irrelevant because they were just that internal.

Now the incentives have completely changed. But beyond that, if there were tests...and if they were in fact good and compelling tests, you bungled the launch. You should have released the tests, got buy-in, established stakeholders, but NO...

...even TRSs had no early warning.

This is a real chance to learn from the launch error and if you have genuine improvements for buyers and sellers, apply these launch principles in the future. However, now, it's all fruit from the poison tree.

kashmiah said Along with the updates to the advanced rating system, we've also introduced buyer ratings as well. This will let sellers leave ratings for buyers as well, creating greater transparency between buyers and sellers on their overall experience. We monitor these ratings too, so we can weed out buyers who are causing sellers issues by trying to abuse the ratings system.

Again, even if this was a good thing. It's too little, too late on the launch. You lost us already. The STARS WARS are over. However, I'm not convinced it's a good thing either.

I as a SELLER don't know how to rate a buyer.

Overall experience?

The Normal Distribution Curve tells us that 66% of all seller and buyer experiences should be right around 3 stars.
Another 13.6% should be 4 stars and 2 stars each. That leaves only 2.4% for 1 and 5 stars.

But real people are HORRIBLE at dealing with randomness, normal distributions, or ratings. So why would you have us try to do what we're horrible at.

I bet your tests do not have 66% of people with about a 3-star ratings.

And if they don't, why do you think it will identify the "mediocrity". Or even that there is such "Mediocrity" that needs to be found?
kashmiah said We want to reiterate that during our test, we didn't notice any change in how ratings were left for sellers, so it shouldn't change sellers who are already doing great work on Fiverr.

I'm not sure I even understand what you're saying.

However, maybe your test wasn't large enough?

Because we have plenty of folks getting very strange results ... "Love it, excellent seller" in comments and a 1-star review.

I don't understand how you can think the Fiverr system/community is immune to human nature?

ARS is a pain in the ARSe.

Lovely post. I wholeheartedly agree with everything you said, and just wanted to acknowledge that I read it, and agree. It’s really a shame that Fiverr is refusing to change it, because they are going to crash. Hard.

Great post and well replied to CS representative.

I wonder why fiverr is not listening? Why they want to dump their success just because of bad decisions! Clearly this star rating is bad idea, and still they are willing to impose on us.

Clearly fiverr is giving all sellers to think for switching to other similar services launched by other well known freelancing brands. Because they have implemented those features what we were asking for years. Even those site’s product manager’s are reading problems of sellers on fiverr forum and they are implementing in their own system.

I have created support requests regarding analytics page issue, order issue and those are not getting resolved. Fiverr is just keep busy changing the color of site and implementing dumb ideas like “star rating”, “people who bought this gig also bought this” type of crap things.

Reply to @wingle: I don’t want to tell you how many bugs I reported to CS and forum admins too… they are all still here, unsolved!

  • we are asking Fiverr to code some really useful feature for sellers (eg analytics) but they have to play with site color (green, then maroon, then back to green again)

  • we are asking the opportunity to block bad buyers, but they have to code "buyers who bought this gig also bought" feature

  • we asked a way to stop buyers ordering without reading instructions, but they have to send promotional emails with boobs gigs on top (yes, I had them and I was not the only one)

  • we asked a way to stop illegal gigs and prevent new ones, but we have to report them because they don't have time to code filters... they have to code ARS instead

  • they were asked not to introduce new bugs on site with every new feature released, but they are flooding us with problems and they don't have enough workers to solve them... because they need to change Fiverr to let it be like Amazon or Ebay... well, Fiverr was different and this was exactly the best thing, this was the main reason Fiverr had success... and now it's over

  • they were asked to asnwer support tickets as soon as possible, but still there are users complaining they didn't get an answer from weeks! Fiverr CS has too few employees and they are going to be flooded by requests about wrong ARS votes, so CS is going to be bad every day more

In the past Fiverr community played its role into Fiverr policies... now we just see new unwanted feature super-imposed on our heads without being asked or told before; rules are changing day by day without we are warned about them...

If you care your community, ask before and ask for mercy when it's clear you made an epic mistake... don't tell us everything is fine, 'cause we are not stupid.

ARS is the last drop on our full glass... sellers are leaving Fiverr, did you realize this? Old sellers, the ones that helped you to grow, providing their wonderful services for ridicoulus prices, are leaving! We sellers (new ones, Level 1, Level 2, TRS, Super Sellers) and many buyers are telling ARS is stupid and Fiverr answers us everything is ok... I don't understand, do you really think we are brain-damaged?

Please Fiverr, revert to the point we were before, beg our pardon, solve all the problems we have without feeling the need to introduce new ones, listen to your sellers, care your sellers more than you're doing... this is what you have to do.
Remember: buyers bring money, but without good sellers there won't be buyers...


A very well crafted response that I couldn’t have put better myself.

Edit 8th July 2014: I have changed my mind on this, after seeking further information and reviewing the rating system further.

Reply to @alliemadison12: if the best sellers, the ones providing the best service and giving the best customer support too, are going to crash hard, I think that Fiverr is going to crash too.

Having millions of levelled gigs can bring money, sure, but best sellers are the ones giving Fiverr the opportunity to tell the world they are the best site on the market for cheap jobs!

Everyone seems to be so much against the new system.

No one ever said anything positive about it, new possibilities that it brings with new ARS.

I worked on the number of Information Systems and from my experience I can conclude, there is always a lot of potential! Especially in the analysis of collected data (modern term Big Data came a lot later)

For example. Some sellers complain about a buyer who gave 3 star for the delivery time, when the actual gig was delivered even ahead of time! Wow, that sounds bad, but is it really? You don’t think that customers support will not be able to spot it? This one is the easiest to do! You (well not you, Fiverr’s administration or CS) simply make a query to database, showing all buyers who gave that kind of response and within seconds you have a list of “Bad buyers” (who gave low ratings on delivery time). How would they value their ratings in future? click-click and the guy is out of the system, can continue voting but his voice will not count! Really easy to spot and exclude! Why none of you sellers have thought of this new possibility?

Think of it! Of course it will take time to tune the system, assign different weights (priorities) to various parameters and that will affect your ratings (they may begin to jump up and down), but at some points they suppose to balance and begin to show the real picture. Just like Google does various adjustments to his algorithms (like Panda, Penguin, etc) in order to reduce amounts of search-spam, same is for Fiverr!

Reply to @kornilov: You just assume this possibility and actually fiverr team will not do such thing and change your rating and kick that buyer. It is hard to believe.

If they want to implement star rating, at-least make it fair.

>> Make star rating visible on buyer profile which given by sellers.

>> What is the point of “Communication With Seller” factor to rate with stars? Suppose someone order my gig and supply all information i required. And my delivery time is 12 days. Should i tell that buyer a new joke every few hours so he thinks i am responsive with communication? Why on the first place it should be counted against my rating? Why not just this to use for customer support to see and it should have no affect on rating?

>> Let’s talk about “Buy Again or Recommend”. Why on earth this should be calculated and used against seller? If someone gave me full 5 star for work and 5 star for being responsive and do not wanted to recommend me for any reason. Again for what i am being punished?

Basically those 2 factors ( “Communication With Seller” & “Buy Again or Recommend” ) to be counted against rating are UNFAIR.

kornilov said: simply make a query to database, showing all buyers who gave that kind of response and within seconds you have a list of "Bad buyers"

We already met here in forum and I know you're a developer: what you say is true, absolutely true... so sounds easy.
But querying database to find some illegal gig is also easy... but illegal gigs are still there if you don't report them (I've reported last one this morning).
Also querying database to get the correct "earned this month" amount is easy, correct? Anyway it was wrong for many users (and for someone it still is).
Should I go on?
There are loads of mistakes that sound silly, easy to solve but they are not; so, just because it could be easy to find crappy buyers, that doesn't mean it will be done... 'cause remember, most of the times buyers are right for Fiverr...

Anyway, just to show my situation: I deliver my main gig in 6 days max; now, if I deliver it in 4 days, shouldn't I receive automatically 5 stars? Why user should choose the stars for delivery speed if I was within timeframe stated in my gig?
What if the user expected the delivery in 2 days without paying express addon?

I want to argue on this too: if I deliver a "so and so" gig... it becomes subjective (and not objective) to give 2, 3 or 4 stars... so for the very same delivery one buyer could leave 4 stars and another one 3... who is right? Should/could I complain with CS?
Many buyers (not everyone, I had wonderful buyers, lucky me!!) are overdemanding and will be ready to bully you with less stars if you don't overdeliver... should we complain with CS then? CS is already overloaded.... so, what's going to happen in your opinion?

No, ARS is evil, ARS is wrong on Fiverr: thumb up/down is more objective and it's enough for buyers!!
More: why can't we sellers give a thumb up/down for buyers and let our rating be shown on buyers profiles? There is a mismatch, don't you think?

Reply to @mark74: Agreed. There are just too many variables that can come in to play with the star system. With the thumbs up, there were only two variables – thumbs up was good, thumbs down was bad. Easy and simple.

Now with the star system, the buyer may pause and think

– “was his/her work perfect”

– “it would of been nice if I got it faster”

– “I seen other gigs like this I think are better so maybe this is not top notch work”

– “man, I’m having a bad day so I’m in no mood to give this seller 5 stars when my day is like a 3 star day”

– “three things to rate the seller on. Hmmm, let me spread it out a bit and give him 5 stars on this and 4 on this and 3 on this so the rating seems more subjective even though he did everything I ask him/her to do”

– " Fiverr is a pain in the butt for the buyer so I think I will rate the gig only 3 stars and maybe Fiverr will change the way they do things"

– "all my notification messages went to my junk folder. Not sure if that is the sellers fault but I don’t like it and can only give 3 stars for communication"

I’m sure there are a thousand more variables on why the buyer will rate you the way they do with the star system, but the point is, that with thumbs system the buyer thought “Ok, it was not perfect but it was more right than wrong so I will give him/her a thumbs up” The 5 star system just gave the buyer an out for rating you less while thinking they are not causing any harm by doing so. It stinks.

The only way the integrity of ARS will be viable as a stat for Fiverr is that we get smart, honest and reliable buyers rating you. But as we all know, buyers are far from perfect and so Fiverr decided to give imperfect buyers an imperfect system to make seller’s life even more stressful than it already is.

Reply to @steveeyes: Just to use a little math: previously buyers had 2 possibilities (thumb up and down) and now they have 125 different possibilities (5 raised 3) for each gig.

Now, how can CS cope with ARS and complains that are going to flow?

Buyers are humans, so imperfect: give them an almost perfect method and their mistakes will be limited; but give them a deeply subjective and imperfect rating system and they are going to make a real mess…

I wish I had enough eloquence to reply like you did. I’m sure my response would contain a number of profanities which would have been immediately censored.

Great post.

Reply to @mark74: Agreed mark. And add to the formula all the glitches and troubles a buyer may encounter that is not the seller’s fault, but the seller will pay the consequences when rated by the buyer. Communication is biggie and most problems with communication is how clumsy Fiverr laid it out or because of glitches on the site. But the buyer won’t know that and will blame the seller for Fiverr’s Incompetence

Reply to @madmoo: One thing that plays on my mind is the pressure. I think most seller tries their best to meet the buyers demands, even if they are asking for something more than the gig offers or if they modify the gig and ask for something unexpected, sellers either try or they cancel if they can’t do what the buyer wants.

Under the thumbs system, it is a balancing act for me to make money and still manage my burn out. Otherwise, burn out would effect my performance.

For me, the star system makes it tougher to manage and burn out may occur trying to meet buyers demands that the buyer justifies based on their perception of ARS and expectations.

I think all of us will continue to try to give outstanding performance but if ARS adds more pressure and burnout, your performance my be effected even if you are trying your best.

My last point would be that I imagine the burn out under the new ARS may be more or less of a problem depending on the type of gig you have. I’m sure certain gigs (i.e. voice over, writers, etc) may be impacted by ARS than other gigs.

I don’t know. I guess time will tell because no doubt, ARS is coming but I think you will see many complaints on the forum about it.

Reply to @madmoo: exactly. If you did give the buyer less than 5 stars, I’m sure they would of changed their rating also.

I think under the thumbs system sellers had a pretty good idea if they were getting a thumbs up or not. Based on some of the complaints already posted about the ARS by some sellers here on the forum, they were completely shocked when they got a 3 or 4 star. One seller got one star yet the buyer said it was outstanding work and another I just read had to modify the buyers work 24 times and for his efforts he got 4 stars.

I think there will be a lot of unexpected ratings that will shock sellers, thinking they went above and beyond and yet for all their effort, they got 3 or 4 stars or worse.

Ugh! Life goes on :slight_smile:

Reply to @wingle: "“Communication With Seller” & “Buy Again or Recommend”"

I made my assumption, as well as all of you! You all haven’t seen those formulas that are used by Fiverr! How do you know how much “weight” is allocated to these two factors? What if their total counts to 5% of the final grade?

Reply to @mark74: I haven’t seen the sellers panel and all of its features, but as developer I know that most of web-resources have some sort of “user”, “moderator” and “admin” panels. And we all know that admin panel normally has a lot more features than the other two. Most likely tools that buyers and sellers are given (scripts and queries) are very basic compared to what administrations (or at least customer support) has. Most likely they test features first and after release them to public.

I was offered this star rating to feedback on some sellers like half a year ago, and I see that only now it is put into action!

Lets think it is for good! Anyway Fiverr is a business, a source of income to its creators and owners. I would personally give more weight to opinion of buyers who bring the most of money! Like you divide a total amount the person spent on Fiverr on some value and that gives you a weight for his ratings. - Bad buyers don’t buy much, but if they do, probably they are not as bad.

I believe, my assumptions about using the ARS for spotting bad buyers and unusual/suspicious behavior is not much more unrealistic than all of what I heard lately on the forum.

kornilov said: I believe, my assumptions about using the ARS for spotting bad buyers and unusual/suspicious behavior is not much more unrealistic than all of what I heard lately on the forum.

If you believe that, you do so out of your own bias and to the peril of the discussion.

Here's why

Even if you're right, and it's about finding bad buyers (which is a REALLY odd idea for which there is little to no evidence) it still is a problem for two major reasons.

1) A "good buyer" will fit a "bad buyer profile" from time to time when it has nothing to do with the seller. (See Steveeyes comment about "emotions and bad moods" making today a 3 star day. Second a "good buyer" can fit a "bad buyer profile" because they have take things very literally--maybe even have Asperger's syndrom.

2) Even if a "good buyer" doesn't get tagged as a "bad buyer" all that means is the good buyer's random moods are treated as MORE VALID when in fact it was still noise.

mark74 said: we are asking the opportunity to block bad buyers, but they have to code "buyers who bought this gig also bought" feature

Mark, I know I shouldn't laugh as this is not funny what's happening and our struggle is pure.

However, I nearly spat out my beverage when I read this and the rest of your list.

Very nice.