IMDb User Ratings Page Redesign

Hi @ACT_1​ -

If you are not yet seeing the ratings page changes, you should see them shortly!

I think I see a way out of this conundrum.

First though I want to review some critical assumptions.

That the privacy laws and so forth are at odds with storing demographic data on a user's profile.

That demographic data is in fact stored on the users profile in IMDB's database and not stamped on to a movie's vote data row (a database term).

That users, especially top "critics" want to be able to share their votes with others and let others see them. 

That users want to be able to see a demographic breakdown of movie votes across a variety of demographic variables including gender and age. That users have yearned for even more demographic data say for example ethnicity, religion, or politics/class. 

The way out. Store two datasets that can not be related to each other. 

1) Remove all demographic data from a users profile and any table that can be related to that user

2) Offer a client side advanced voting demographic data entry survey experience. This data does not get stored in the user profile or a table associated with the user in IMDB's database.

3) when a user votes for a movie they can choose to fill out an extensive demographic data survey along with their vote. Most importantly they can click save for later and consent to storing these answers on their Browsers cookies. You can sort out whether that should be encrypted somehow SHA256? etc. 

4) this demographic survey data is combined with the movie vote itself but stored in a completely different store of movie rating data that has NO association to a user profile

5) a user when reviewing a movie can click on something to see the demographic survey results 

I believe the correct way to refer to this survey data would be "anonymous" demographic survey data. 

6) You can consider whether you want to mark on a user's vote that they have already submitted a demographic survey. Its only a bit field 0/1. This bit field is stored on the user's movie vote table not the anonymous demographic vote table. If they have previously submitted the survey you can decide whether users should be allowed to submit one again and perhaps how long they should be allowed to wait before adding another one. Say for example flip the bit field every 10 years as a user might shift from one age related demographic tier to another? 

7) a user will never be able to see the demographic data they assigned to a particular movie vote. this data is stored anonymously in a separate data store

8) a user can change their demographic answers stored on their client side before submitting their vote (and they certainly may want to do so as their age demographic changes (slowly) over decades).

9) additional demographic survey questions can be offered in this advanced survey submission and be displayed on the anonymous demographic report. 

10) over time updates to the app can offer these additional survey questions to the user in that survey experience to enrich the survey reporting

So overall the idea is to continue offering rich demographic reporting on movie votes but introduce a "fire wall" between the user data and the demographic data but prevent a user from flooding the demographic survey data set with more than 1 vote over at least say a 10 year period of time. The demographic report would be able to be referred to as anonymous and as such should not trip any privacy laws. You would expect the total number of demographic survey results to perhaps be vastly less than the regular votes submitted without them but the use of the client side data storage should make it far less painful for users to submit demographic survey answers in the future once they have taken the time to select answers to the questions the first time.

I've suggested a path for us to have our cake and eat it too. Keep the privacy fans and lawyers happy and keep the movie collection analysis of demographic bias transparent. The more the demographic reports can offer us ability to see how these movie ratings split along various demographic lines the more we can understand how various bias impacted the movie ratings. 

I think its one thing to complain but quite another to provide constructive criticism by offering alternative solutions. Please consider this path carefully. Thanks for you taking the time to review this proposal as a path forward. I think you will find plenty of users who are willing to fill out the survey once they appreciate that their participation is what allows the anonymous report to add value to their own research. Perhaps a bit of training and marketing emphasizing the benefit of filling out the surveys and submitting them along with the votes. Even 500-1000 users can offer enormous value in an anonymous demographic report. of course the more the better.

The fun part about this path is that you can start enhancing the survey questions offered and offer even more fascinating reporting than we have ever experienced before.

The bottom line is that you have to be quite strict about never associating this survey data to a particular user's id in the database. Once you start down this path, all sorts of doors should open for you to present a very rich and valuable data set to your consumers. You should no longer have to fear legal action if you have approached this from an anonymous survey gathering perspective. Good luck and do not be afraid of the undiscovered country.

@SidneyLeeJohnson​ 

First though I want to review some critical assumptions.

That the privacy laws and so forth are at odds with storing demographic data on a user's profile.

No sorry, that’s not it at all.  It is perfectly possible for us to securely store and manage this data (and we do).  The two issues are that:

1. A decreasing number of customers are willing to provide this data directly (anywhere, this is not specifically an IMDb issue).
2. Increasing consents will be required to maintain the data from the third party account linkage (and therefore see also #1). 

Thanks for the write-up though. 

@Col_Needham​ Can't get age/gender but CAN get country. For some reason this doesn't quite wash.

@Col_Needham​ I'd pay to have the top1000 return.  I don't care about age and gender ratings but I relied on the top 1000 to help when a movie is hyperregional or prone to some sort of brigading.  

Maybe add it as a feature to imdb pro.  I'd also pay for a second decimal point in the weighted ratings. 

@Col_Needham & was ist bitte das Problem mit den Einwilligungen, für Cookies braucht es diese ja auch... & die meisten User wären wohl bereit, diese zu geben... & was spricht gegen die Darstellung der top1000 voters? Die sind/waren ja immer anonym...

Ich werde den Eindruck nicht los, das dies alles Ausflüchte sind! Die Datenschutzgeseze der EU wurden ja z.B. bereits 2018 verschärft!

Bringen sie zumindest die top1000 zurück oder erklären sie mir, was gegen dies spricht!

@gstine99​ 

I hope the IMDb team will reconsider their decision to remove it.

The issue is that this was not our decision, sorry. 

Could you please make it so that you can see the exact number of votes instead of "3.2k" or something. The exact numbers were helpful. That's not confidential information, is it? It's about user comfort

@Col_Needham​ Not your decision....? whose then?

@klnaco​ This will not be possible, sorry.  Please see our reply above at https://community-imdb.sprinklr.com/conversations/imdbcom/imdb-user-ratings-page-redesign/64220f80ac2f711237475865?commentId=6422251f82212822057e4b28&replyId=6422a9d382212822057e6c99

@Col_Needham​ IMDB's dismissive attitude about this will lead to your going defunct. Not being hyperbolic; it's just true. You-all have shot yourselves in the leg with this nonsensical quality-degrading change. No one cares how people in Sweden rate a film. We want to know by gender and age. If this is a gender issue then allow people to pick Male, Female, and Other

It's pretty clear from this and other changes that IMDb doesn't care what users found helpful or what we'd actually like from the site. They'll give us what they want and tell us how much we enjoy the changes.

I do love this buck passing though. Gosh, BIG TECH can tell when a surfer hunted for obscure information five years ago and still barrage us with ads BUT AMAZON/IMDb can't assign basic demographic data to users? It's really hard to believe. Especially since they CAN tell us the top countries. Something just doesn't make sense.

Additionally, guess they could have decided to do a male/female/not specified type breakdown.

@bderoes​ 

I like the idea of the Top 10,000.

Or even, one might consider the vote of reviewers.

I think it's nice to have more granularity in the ratings. Nationality is certainly not the most revealing variable. Is a UK voter any different than a US voter? I think that it is not.

On the contrary, men and women are different, an 18 yo is different from a 45 yo.

And also, votes change in time: a movie can be valuable when released, but eventually loose its interest (or the other way around).

So for me the point is: do we want to have nice views with useless statistics or do we want to see something helping us to give an a priori evaluation of a movie?

@bderoes​ Thanks for the feedback …

It was never clear whether the age demographic was based on the voter at the age when they voted or the age when you look at the report. 

It was the age when you looked at the report. For example, I have been voting since the ratings poll launched in 1989 (when I was 22) so for a title I have not rewatched (and therefore not revoted) since then, my vote for that title would have moved upwards into older age groups across the years, and finally have been placed in the 45+ age group ever since 2012. 

As for the Top 1000 voters, I hope that IMDb has not only stopped displaying their vote, but is completely ignoring it in their weighting. They either need to expand that to Top 10,000, or dump it altogether. Those numbers were often significantly biased. You don't have to watch a title to rate it, and I strongly suspected that many of those Top 1000 were NOT honest reviewers who appreciate a large variety of genres/content.

This is why the Top 1,000 breakdown has been removed.  You do not need to worry about them having any outsized influence in the overall weighted average, but by definition, we could not filter the ratings from the Top 1,000 breakdown itself, which reduced its accuracy and usefulness. 

For series, create an average rating per season.

Display the actual number of votes, not the abbreviated **k.

Thanks for these.

@SidneyLeeJohnson​ exactly - I am with you! I had no idea this shift was afoot. It is a radical degradation of IMDB. The excuses for making these horrific changes are illogical. I use IMDB demographics ALL the time. I will literally never come to IMDB to find out how people in Turkey rated a film.

I literally created an account to agree with you.

Now ratings are only useful if you're a young white man.

Finding films that appeal to me as ablack woman has now become incredibly difficult and I worry about how things like this will affect the wider media industry.

It's basically further institutionalising white men as the default and resigning everyone else to an other category. In fact it's not doing that, It's basically now saying "everyone agrees that movies featuring POC/not men objectively aren't as good as movies about white men and explosions"

This combined with the genuine sexism and racism in with the review bombing of media heavily featuring people who aren't white or male paints a very dystopian picture.

I'd ask who these changes and this website are supposed to be for but the answer is becoming disturbingly obvious.

@jupiterLILY​ "Ratings are only useful to a young white man".  Why do you single out this group as though they are somehow advantaged.  I'm a baby boomer male so in the past I looked at ratings for the older age range and male. I can't do that any more and miss the feature. What I see in entertainment media is far from " institutionalizing white men as the default". My mind goes to Sunny Hostin on The View who will make any topic about race. Entertainment media is hypervigilant going out of it's way to make minorities, women, and LGBQT folks happy. The issue here is removing demographic filters that we used to have and found useful.  That's all.

How does making up the majority of the voting body not advantage a group?

Somewhat lacking in imagination with that one.

If you're a white dude coming to this website, it's great and it's highly likely that you will not see the problem. You're going to be given a list of movies that appeal to you and all the reccommendations are going to be pretty spot on.

And entertainment media is hypervigilant about making themselves rich. If you think they're trying to make *anyone* happy then I have a bridge to sell you.

There has been a slight push in recent years when studies have realised that not just white men have money, but the vast vast majority of media is still produced by and for white men.

Minorities being able to make some content that appeals to them isn't a bad thing, it's not pandering, it's just people participating in an art form. Not really sure why you're presenting that as a problem.

@jupiterLILY​ 

@nospam77​ 

You know something has been bothering me about this. Why would a forum get rid of what is/was its most popular feature? To borrow from a movie’s tagline let’s follow the money. 
Let’s suppose it’s in a studio’s interest not to have too much detailed rating information out there. As an older male I would avoid movies that were popular with young females. IMDB used to allow me to know this consequently I would avoid seeing  them. 
But if I didn’t know chances are I might go to/rent the movie. 
Please excuse my cynical mind but is too big a stretch to think that  studios might lay on IMDB to withhold demographic information in order to increase viewership? They are THE source of information for IMDB after all and wield a crucial hammer. 

Don't forget that IMDb is owned by Amazon which also now owns MGM -- merged into Amazon MGM Studios. This includes United Artists.

I largely agree with you and I'm cynical for similar reasons. Although, I'm not sure how intentional it is or if it's just like it is in other industries like where it's just the result of not enough of the individuals not questioning their implicit bias. White men are assumed to be the defualt population in science and medicine, it's still not mandatory for them to also use female crash test dummies and don't even get me started on bias in pharmeceutical testing. Why would something as like movies be any different.

Studios definitely have and want this information, they want movies that poll highly with everyone because that equals more money. It feels more like companies are now trying to tell people what they like, instead of finding content that's tailored to them.

Concealing that information from users just reinforces systemic bias though. I found this, and it's from quite a while ago, but it shows you just how many more men are voting on movies than women. Only 2 movies have more than 100,000 votes by women and they squeaked it, the average is 65,000 women voting for every 374,000 men.

Given that they give the impression of being the go to place for movie ratings, that's some really biased data. I know there are similar problems with the countries being surveyed. I'd love to see some adjusted ratings that accounted for the different weights in gender, country etc.

@SidneyLeeJohnson​ Well-stated. I shared similar statements. I cannot imagine what forces led IMDB to compromise their product so profoundly negatively.

The IMDbPro brand is focused on neither the experiences of movie goers, entertainment seekers, movie reviewers nor the audiences of those evaluators, so even invoking the name "IMDbPro" is pointless in this context.

The age and gender breakdown is already available on IMDb Pro, in a different design, and is so far still displayed there.

I think professionals may well be interested in how projects are received by different demographics. (Isn't that all entertainment executives think about?)

Yes, that's a good point, Peter. The demographic data can certainly be of interest to professionals in show business. I was just thinking about how much IMDbPro is so oriented around casting calls, employment opportunities, talent representation, public relations management and, to a small degree, networking. I've observed one person on here, maybe more, who proposed making an ad-free video-on-demand service a feature of IMDbPro, which I thought would be going too far.

@SidneyLeeJohnson​ 

As now noted above, no, unfortunately not.  We simply will not have access to the same data so we cannot use it anywhere, sorry.  We are just waiting for the work to remove it from IMDbPro to be scheduled.

@Col_Needham​ are you passing along to upper level decision makers how horrified people are by this ruination of your product?

Hi @mturulova​ -

You are not seeing the breakdown because the "Rating By Demographic" section has been removed and is no longer available.

Sorry, but this is a really terrible idea.  Why did you do it?

- Ratings by age and by gender were really helpful in deciding which movie to watch.

For example, if some movie has significantly higher rating in older or younger age group, or in female group, it gives you a pretty good picture if you would like it or not, depending which group you are in.

- You removed it and replaced it with ratings for users in specific country (?!?)

How does it help?

Do you really think that _anybody_ might say "Oh, this movie is popular in Turkey or Germany, so that means I might like it"?  Or "well, users in France generally like specific type of movies, so this country rating for France gives me good information what this movie might look like" ?

So, you removed the info that was very valuable to users, and replacing it with something meaningless that has no value for users at all?

Really?

@V7Fabio​ The reasons are covered above -> https://community-imdb.sprinklr.com/conversations/imdbcom/imdb-user-ratings-page-redesign/64220f80ac2f711237475865?commentId=6422251f82212822057e4b28&replyId=6422a9d382212822057e6c99  

We had little choice in the matter, sorry. 

@ron3​ Thanks for the bug report.  We are unable to reproduce this. Do you have a link to the above page and please can you let us know on which browser (including version) and operating system you are seeing this? 

@Col_Needham​ I don't get it--who is "we" and why wouldn't you have a choice?

@Peter_pbn​ Thanks for the suggestion. 

I like the breakdown by country. Otherwise I agree with the comments so far that the old design was more useful, as it provided insight whether a show appealed to young and old, or was perferred by males or females. The gender category might be adjusted to be helpful to a broader non-binary demographic; notwithstanding, the differentiation between those who identify as male/female was very helpful.

I used to consult the breakdown regularly. I miss it.