BLACK PANTHER And More Audience Score Problems For Rotten Tomatoes And Metacritic

One’s a Dot, two’s a line and three’s a trend.

By Feb. 18, 2018

Over The Last Jedi’s opening weekend, I wrote about how websites that aggregate both critic and user movie reviews were being reviewed-bombed in an apparent attack to lower its audience score. Over time, Rotten Tomatoes ended up with a 48 score and Metacritic ended up with a 4.5 score for The Last Jedi. My issue was not with the fact that some folks had a negative opinion of the film, it was in the methodology of how these sites polled their voters because it allowed for a self-selecting negative bias that prevented their results from being an accurate representation of the film's larger audience. Most importantly, I was able to publish some data from comScore, a service that polls audience members immediately after they screen a film and collects their thoughts as well as their demographic information which allows them to adjust the data to be as reflective of the larger audience despite only interviewing a small percentage of that audience. This data showed that not only did most people love the movie and wanted to recommend it to others, but the audience that hated the film was the smallest percentage of the audience possible.

Unfortunately, the larger narrative around the film and its audience score accepted these biased polls as accurate. Despite comScore and Cinemascore indicating differently, the conversation focused the small group of the very vocal and frustrated fans. Forbes even went as far as to publish a piece that stated that views to the contrary were the work of “conspiracy theorists” who “ignored the overwhelming evidence that many moviegoers disliked the film, and circulated accusations of organized vote campaigns designed to drive down the film’s scores, despite the absence of any credible evidence to support this notion”.

Rotten Tomatoes was certainly questioned about their audience scores in the wake of what was likely very fake claims that individuals created bots to review-bomb their site. They stated:

"The authenticity of our critic and user scores is very important to Rotten Tomatoes and as a course of regular business, we have a team of security, network, social and database experts who closely monitor our platforms,” the rep added.

However, Rotten Tomatoes didn’t view the controversy as a negative. Using the old adage that any publicity is good publicity, they promoted the audience score in a promotional campaign designed to use the dissent as a means to drive traffic to their website.

I state that the claims made about creating bots are fake as an opinion, but it's based on reviewing the users who left comments on both Rotten Tomatoes and Metacritic. The comments left at both sites appeared, again in my educated opinion, to be organic responses based on the same theme: Disney had made a movie that violated the core tenants and values that Star Wars represented to them. The boasts about creating these bots only applied to Rotten Tomatoes and did not account for the fact that the same exact phenomenon occurred to Metacritic with the same type of comments left there.

One of these values that the vocal minority felt was betrayed in The Last Jedi is the concept of a strong, central male hero. In an interview with the Huffington Post, the moderator of the website that claimed to create the bots engaged as follows:

He explained that he’s upset with “Star Wars” producers for, “among other things,” introducing more female characters into the franchise’s universe.

Throughout the course of our conversation, the self-identified member of the “alt-right” claimed that Poe Dameron (played by Oscar Isaac) is a “victim of the anti-mansplaining movement,” that Poe and Luke Skywalker (Mark Hamill) are in danger of being “turn[ed]” gay, and that men should be reinstated as rulers of “society.”

This is incredibly insensitive rhetoric for sure. I don’t want to include that level of hate here as a means to give it a platform, but to demonstrate that the act of review-bombing Rotten Tomatoes and Metacritic was a means to normalize extreme ideology. Their hope was that they could control a unified voice to Disney through these polls and send a message that the product Disney is creating did not meet with the approval of the masses. Because their narrative became a central one in the discussion around The Last Jedi, they likely succeeded despite the film becoming the highest grossing film of 2017 and one of the ten highest grossing films worldwide in history. Rotten Tomatoes is an established measurement of a film’s quality with its critic score used in transactional sites like iTunes and cable video on demand platforms that use the critics score to help drive sales to critically well-regarded films. Ergo, being able to control the user score on this platform will validate these voices.

This leads us to Disney's next release, Marvel’s Black Panther. Riding on the success they obtained previously, the group who claimed to organize the effort behind The Last Jedi’s review-bombing announced they would replicate their efforts for Black Panther showing that racism was just as important to them as sexism and homophobia. Why Black Panther? Because it’s a huge part of Disney’s effort to create diverse storytelling and to fill these stories with diverse actors and actresses that provide a more accurate reflection of the world we live in. I can’t link to their Facebook page because Facebook, still reeling from being a significant vehicle in Russian efforts to undermine our democracy, quickly moved to prevent them from organizing on their platform and banned them. Rotten Tomatoes, not wanting to be associated with obvious racist actions, made the following public statement about efforts to undermine and skew their user score polling:

We at Rotten Tomatoes are proud to have become a platform for passionate fans to debate and discuss entertainment and we take that responsibility seriously. While we respect our fans’ diverse opinions, we do not condone hate speech. Our team of security, network and social experts continue to closely monitor our platforms and any users who engage in such activities will be blocked from our site and their comments removed as quickly as possible.

In order to establish how successful Rotten Tomatoes efforts to curb hateful thinking were, let’s first review audience polls that we know are more scientifically valid because they adjust their results to have a small population reflect a larger population. Cinemascore rated the movie an A+ and comScore has also revealed Black Panther “drew almost perfect scores during exit interviews". So, on one hand, you have the two services the studios themselves use to measure how an audience views a film and they both concluded that audiences loved Black Panther as much as an audience can love a movie. Rotten Tomatoes and Metacritc? It’s a different story there. It appears that both sites are again victims of a focused review-bombing effort to control their polling. As of today, Rotten Tomatoes has its audience score at 74% and Metacritic has its audience score as 6.4.

It's worthwhile to state the obvious – not every negative user review on these websites is reflective of racist ideology. What I’m trying to demonstrate is that the user scores on Rotten Tomatoes and Metacritic are not a reflection of the actual audience seeing Black Panther. The audience for these films loves it so much some of them burst into spontaneous singing and dancing after seeing the film. So while efforts to weed out overt racist comments are in effect, the effort to weed out the overall agenda to send Disney a message that the movie is disliked by the masses is still affecting both sites.

Knowing that Rotten Tomatoes is making an effort to eliminate the obvious racist comments opens up the debate whether the negative ones that remain are coming from a racist perspective. Maybe the answer is even more insidious knowing that as a society, overt racism and sexism is mostly (sigh) marginalized but institutional and normalized racism still negatively impacts our culture every day. Once we determine, as we have by comparing these user polls to comScore and Cinemascore, that the Rotten Tomatoes and Metacritic user scores are skewed, what other motivation do we have that explains the bias? It’s a sincere question because my conclusion here is not easily provable when one reads all the negative user comments but contending that the bias is coming from a place of racism – whether it's institutional or overt – seems to fit all the facts. Until information that contradicts these facts emerges, my opinion remains that the review-bombing effort is a means to send a normalized but extremist message to Disney that women and persons of color should not be given central roles and stories in their movies. While genuine negative opinions of the movies exist, they are not the consensus and consensus ultimately matters most.

Given the effort to control user polling around these two films – one which empowers its female characters and includes casting from a spectrum of cultures and the other being a film with a central black hero and a message about the effects of living in a diaspora compared to a home and history which enriches and advances its people on their own terms – we are one more film short of seeing a trend where Disney tentpole films will always be reviewed-bombed at Rotten Tomatoes and Metacritic. There are solutions available to the sites. Rotten Tomatoes already took steps in the right direction to eliminate the obvious trolling but more can be done. It's really just a simple understanding of opt-in polling and gathering data that will allow the sites to compare the population of users that have opted into their poll and the public data of what the larger movie-going audience is or even better, gathering that information over several similar titles and using that data as a baseline.

Other solutions are a tad more extreme. When imdb user comments devolved into a cesspool of trolling, they eliminated them altogether. Steam, a popular transactional video game platform, has been combating review bombing for years and recently started graphing out their user reviews allowing buyers to take the extra step and look for weird clustering of negative reviews.

The benefit of having user scores for Rotten Tomatoes and Metacritic are obvious – it gives their users a voice and encourages engagement on their sites which drives traffic. We’ve already discussed why they are being targeted: they represent an accepted measurement of a film’s quality. But as they say in statistics: one’s a dot, two’s a line and three’s a trend. We’re one film away from establishing that Rotten Tomatoes and Metacrtic can no longer be given the trust they’ve longed earned if they continue to allow their polling to be controlled by the most extreme voices in fandom.

Comments