On digital job platforms like Uber, TaskRabbit, and Upwork, this rating system is applied to individual workers—with huge consequences for those being rated.
“It sorely lacks the variety and texture that exists in a work interaction,” she says Hatim Rahman, assistant professor of management and organizations at Kellogg. And employees tend to see the downsides of an overly simplistic appraisal system very quickly, adds Rahman: “They start saying, ‘Hey, this doesn’t make sense. there are so many cases where the rating someone leaves doesn’t accurately reflect what’s going on in the context of my job.”
Perhaps employees would reject low ratings that they perceive to be unfair if it weren’t for the financial implications that may flow directly from those ratings. Uber, TaskRabbit, and Upwork all incorporate customer ratings into their algorithms that determine employee visibility to customers, incentive eligibility, and tenure.
What these platforms have created, in effect, is a system where the traditional role of the administrator has been completely handed over to customers and their star ratings. As Rahman and his colleague, Lindsay Cameron at Wharton, write to a new paper“the rise of this algorithmically mediated customer control to track and rate people has effectively given many workers a new, digital ‘boss.’
Rahman and Cameron investigated how individual workers on digital job platforms chose to resist the power of these ratings, from screening potential clients in advance to quitting work mid-project to avoid a bad rating.
The researchers divided the employee-customer interaction into three stages—before, during, and after the job—and found that each stage produced distinct forms of employee action to regain some of their control over the evaluation process.
With each subsequent stage, employees’ ability to resist unfair evaluations decreases, the researchers found. And efforts to stay alert for a prolonged service encounter and try to maintain high scores contributed to feelings of fatigue among workers.
The researchers stress that these patterns may cause problems across platforms as well. Clients’ interests do not necessarily align with the platforms’ interests, and clients are unlikely to be held accountable for their rating decisions.
“When you delegate that kind of control to people outside of your platform, there are always going to be gaps,” Rahman says. “It opens up areas of mismatch and opportunities to game the system.”
Integration into the Gig Economy
For their analysis of digital work platforms, Rahman focused on a website that connects freelancers with projects, which he nicknamed “FindWork,” while Cameron studied a rideshare platform, which she calls “RideHail.” (The platforms were anonymized to protect the identity of the workers.)
The researchers investigated the dynamics of the platforms by immersing themselves in RideHail and FindWork. Cameron spent the three years between 2016 and 2019 as a RideHail employee and customer. Rahman spent the four years between 2015 and 2019 in the same roles at the FindWork platform.
As part of their research, they conducted interviews with workers and customers and drew on archival sources. This included anonymized data from FindWork that records private communications from 2013 and 2014 between freelancers and clients during projects, and RideHail’s website material, as well as articles, social media posts, YouTube videos, user guides , blogs and discussion boards about the Company.
Across all sources and platforms, the researchers saw clear patterns emerge. As workers progressed through the various stages of a task, they responded with stage-specific resistance measures. And their own power to repel customers and ratings declined as the stages progressed.
During the first stage, employees had the most room to implement covert resistance tactics, as customers have little information about employees at this point and cannot yet evaluate them. Their strategies at this point often involved attempts to screen customers—perhaps calling them with a question to assess their attitude and thus their propensity to score low.
Researchers have learned that drivers who suspect a prospective rider is capable of delivering few stars sometimes cancel a ride as a precaution. “I never start the trip for a passenger with bad behavior because that means a bad rating,” said one RideHail driver. At FindWork, some freelancers contact a client before starting a project to require five stars as a condition of working with them.
“In a grocery store, for example, you don’t really control which customer will come to you in the checkout line,” says Rahman. “But in our context, we’re seeing workers trying to claim more agency because they know how it could go — they’ve taken steps to figure out how to avoid bad customers.”
During the second stage, while employees were in the middle of completing a task, their power to resist a customer’s demand or complaint decreased—as both employees and customers knew that, ultimately, the customer would rate the employee.
Tactics at this stage include, for example, offering a discount for a high score. Alternatively, some freelancers on FindWork have asked clients to spread a single project across multiple contracts so they have the opportunity to get multiple high ratings from a single client they know and trust. Another strategy was to terminate the job early: a worker who suspected a customer interaction was going south simply canceled the job and, with it, the possibility of a low rating—even though that also meant not getting paid.
“In traditional environments, if you have a bad customer, you can at least talk to a manager. But with platforms fully delegating the role of middle manager to the algorithm, we see at this middle stage a greatly increased sensitivity from workers in every interaction,” says Rahman. “They are working to make sure it goes as well as possible and trying to mitigate the risks of even a low rating.”
In the third and final stage, when employees have the least ability to push back against low ratings, they are left to resort to what the researchers called “Hail Mary” strategies of either filing a dispute with the platform or giving a customer a low rating. rate when they suspect the customer has done the same for them. These tactics are unlikely to be successful in removing or changing a low rating, but employees still try them given the importance of ratings to the success of their platform.
Collective search control
Rahman believes that the limitations of job platforms’ rating systems and the frustration and fatigue they cause workers are already starting to manifest as problems for the platforms themselves.
“Around the world, including in the US, we’re seeing some questioning about how the platforms treat workers,” Rahman says, adding that former Uber and Lyft drivers establish worker cooperatives in New York that offer similar services—while allowing employees to retain more ownership in how their workplaces operate. “We’re definitely seeing some momentum towards the model.”
But the study’s implications extend far beyond the usual gig-platform suspects, Rahman says. In the paper, he and Cameron explain how even in traditional service settings, employers are incorporating more technology to solicit customer reviews at every stage of their experience. They note that hospitals are asking for real-time feedback from patients about the care they receive, the scheduling process, parking and dining, while airlines are looking for customer reviews about their ticketing experience, baggage check-in and boarding.
“One thing our paper highlights is the need to re-examine whether this model of outsourcing control to a client and especially using a five-star rating system for all types of work contexts really makes sense,” says Rahman.
