Duplicate Keywords - List #20 (Proposals for Permanent Merger and Auto-Conversion) (geography keywords)

@bradley_kent​ Fair enough, but that is a separate issue from whether the plural "indians" should be merged into the singular "indian."

I would avoid merging two poorly-defined keywords with each other (in this case "indians" into "indian"). While I'm not familiar with all of the titles to which the "indian" keyword applies, I know that, for example, "The Big Bang Theory" and "Jessie" have characters who are Asian Indians, whereas "King of the Hill" has a character who is an American Indian. 

I agree that the plural keyword "indians" is improper, but it would be better to sort out the titles with that keyword into better keywords rather than assigning them all the vague keyword "indian".

@gromit82​

Keep in mind there are already more than 2,000 keywords with the "indian" keyword:

indian (2047 titles)

Also keep in mind that citizens of the country India are properly referred to as Indians. So "indian" is not an inherently bad keyword, even though many contributors have imprecisely used it to refer to "american-indians."

Do you believe that the ultimate fate for the "indians" keyword should be to block it from being used in the future, rather than to auto-convert "indians" into "indians?" I would probably be okay with that, since the keyword "indians" shouldn't be used in any event (because it is plural).

Do you believe that the ultimate fate for the "indians" keyword should be to block it from being used in the future, rather than to auto-convert "indians" into "indians?"

Probably yes. 

I realize that the people in Asia had the "Indians" name first, but from the perspective of many people in the U.S., the keyword "indian" is inherently ambiguous as between Asian Indians and Native Americans. 

If IMDb had an official dictionary of keywords (something that was proposed years ago but never implemented), so that contributors would know whether the keyword "indian" referred to people from India, as well as whether "cricket" meant the sport or the insect, "turkey" meant the country or the bird, "georgia" meant the U.S. state or the country, "mercury" meant the planet, the element, or the car, etc., that would be great.

But until that happens, we're better off not using ambiguous keywords or expanding the use of those that already exist.

@gromit82​ Okay, I will take "indians" off the list, and will consider listing it in the future as a keyword that should be blocked altogether. 

@gromit82​ @bradley_kent 

Over the past 6 months, I have slowly audited the "indians" keyword, modifying more than 100 instances of this keyword and dropping the total number down to 26 titles at last count. I could use some help with the remaining titles.

https://www.imdb.com/search/title/?keywords=indians&adult=include

@gromit82​ @bradley_kent 

The "indians" keyword has now been blocked.

@bradley_kent​ The naming conventions for these geographic keywords are largely based on user patterns over the years.  Generally speaking, contributors have favored including the provinces in Canada, and the boroughs of New York City, for example, but not the states in places like the USA and Australia.

I do "audit" these keywords before proposing them in these lists. Before posting "paris," for example, I determined as best as I possibly could that all of the instances of "paris" involved Paris, France, and not any other place named Paris.

You imply that the keyword "paris-ottawa-canada" was intended for some of the "paris," keywords, but it does not appear that you actually added "paris-ottawa-canada" to any titles. 

As we have discussed many times before, you and I have different understandings of what it means to "audit" a keyword. I use the word "audit" in its conventional sense -- here, double-checking the keyword to make sure it is ready for a mass change.

When you use the word "audit," you apparently mean manually editing every single instance of that keyword so that it is reduced down to zero. This is not an "audit," in my opinion, but rather a manual, mass-scale edit.

Manual, mass-scale edits involving dozens or even hundreds of instances of a keyword, performed by contributors rather than IMDb staff, can cause misleading patterns in keywords, making it appear that the patterns are coming from the community at large, when in fact they may be coming from the edits of one or two overly zealous contributors. 

Case in point: the overzealous and misleading mass changing of "cigarette" to "cigarette-smoking" over the years: Someone seems to be removing the keyword "cigarette" from the database

Also, as we have discussed, you and I have different understandings of when the "reference-to-" prefix should and should not be used. You describe your interpretations as absolute black and white rules or "truth" versus falsities, when in fact the guidelines are largely silent here, failing to define the limits of when the "reference-to-" prefix is proper, improper, required, not required, etc.

In my opinion, if Paris, France is an important plot point in a title, the "paris-france" keyword can and should be used, even if the title is not set or filmed in that city. But if Paris is mentioned by a character in passing, and the city is not an important plot point, it adds very little to include either a "character-says-" or "reference-to-" keyword for Paris, and if that were done for every little thing mentioned in the dialogue, it would quickly result in keyword spamming.

You are correct that very few people are willing to mass edit hundreds of instances of keywords on a mass scale. And for good reason: it is extremely inefficient and can actually be counterproductive for mass edits to be done by contributors, instead of being fixed on a mass scale by IMDb staff. It also results in "bad" keywords being used all over again by other contributors after they are "audited" (read: edited) by a single contributor. It is putting a band-aid on a systemic problem rather than performing the necessary surgery to fix it. If your approach of contributors manually editing every single instance of every single bad keyword down to zero had been followed, it would make it nearly impossible to achieve any real progress on a systemic level, and would depend on perpetual "auditing" (read: editing) by contributors in the future. 

Yes, we have very different interpretations of what an "audit" means.  To me, it is reviewing every single item.  Almost always, there are exceptions.  As was once again revealed in the Supreme Court hearings for Judge Ketanji Brown Jackson, one must look to each single case if one wants to know the truth.  Generalizations are not a true reporting of the facts.

Keyword audits are something that I have undertaken with the belief that the staff (probably underpaid AND understaffed) does not have the time to do.  I just wish that more contributors were willing to undertake this "grunt" work and not just command and suggest from high while the foot soldiers do the actual fighting.

Oh, and USA states have always been included with city names, just like you request with Las Vegas.

@bradley_kent​ There is a huge difference between checking every instance of a keyword and actually deleting every single instance of a keyword. That is the major difference between our two approaches.

You're right about USA states -- I was overgeneralizing and misspoke. What I should have said was that for USA, the country is not typically included, while for Australia, the state is not typically included (and both patterns have been established over time). 

When I audit a keyword, I not only delete some, but add some and correct some.  And, an audit also provides an opportunity to add, delete and/or correct OTHER keywords.

Also, when I finish auditing, I usually request that the staff block the "bad" keyword from further addition.  Still waiting for some action on such requests before I request more.  (I have a list of quite a few requests for such blocks, but am waiting on some action from the staff on existing requests before doing so.)

The staff is much more responsive to your merges than it is to my requests.

Expediency, and a seemingly easy solution, seems to rule the day.

@bradley_kent​ I realize all of that. My point is that if, after an audit, it is determined that 95% of the instances of a particular keyword can be deleted, merged, changed, or some other action on a mass level by staff, then I will only worry about the 5% of instances that require some other change (e.g., changing to a different keyword). But it is often unnecessary (and in fact it can be counterproductive) to reduce all instances of a bad keyword down to zero.

Here are some examples of what I mean to "audit" a keyword: A few vague keywords that are ready to be merged and/or blocked (trump, roosevelt, washington, vancouver)

Note that for "trump" and "vancouver" in those examples, I didn't bother to reduce the numbers down to zero, which would have only been a complete waste of everybody's time. But for "roosevelt" and "washington," I did reduce those keywords down to zero, which was warranted under the circumstances.

The keyword "paris" is in the category of a keyword that makes absolutely no sense for contributors to reduce down to zero, because after I "audited" that keyword, I determined that 100.0% of the remaining instances involved Paris, France (and not any other place named Paris).

As for a manual "audit" of every single instance of a keyword turning up other edits that could be made, that would always be true of anything on the IMDb site. It is easy to get lost in the keywords by manually editing them on a mass scale. While a mass manual approach can be very appropriate for particular keywords (e.g., "roosevelt"), when that approach is applied to other keywords, it can be easy to lose the forest for the trees. 

@bradley_kent​ said:

Recently did an audit of the "bad" "paris" keyword, and it, once again, illustrated WHY keywords should be audited and not just merged.  Most, of course, are "paris-france," but there are also "paris-ottawa-canada" and, of course, "paris-texas" and "paris-character."

Are you claiming to have actually changed the keyword "paris" to "paris-ottawa-canada," "paris-texas," or "paris-character" for any title(s) during your "audit?" As far as I can tell, you did not do so. 

There were 0 titles with "paris-ottawa-canada" both before and after your "audit."

There were 4 titles with "paris-texas" both before and after your "audit."

There were 10 titles with "paris-character" both before and after your "audit."

Other than changing the keyword "paris" to "paris-france" for a bunch of titles (something that staff would have done anyway) and possibly changing "paris" to "reference-to-paris-france" for one or more titles, it is unclear what was achieved from your "audit," except to confirm my previous conclusion that 100% of the instances of "paris" were in fact intended to refer to "paris-france."

@bradley_kent

Well, this is interesting. Somebody (I'm assuming you) has made a bunch of manual edits to the "belgrade" keywords since I first posted this list:

belgrade (0 titles)  

belgrade-serbia (73 titles)

belgrade-yugoslavia (51 titles)

I am pretty sure the keyword "belgrade-yugoslavia" did not exist when I first posted this list, or else I would have taken it into account at that time.

Although you are correct that Belgrade was once part of Yugoslavia, it also remained a part of Serbia even during that same time, since Serbia was one of the six states or units of Yugoslavia. The idea behind the "belgrade" keywords on my list was to try to have one all-encompassing keyword that refers to Belgrade at all times in history. In theory, that keyword could be just "belgrade," but there would still be a bifurcation problem because some people would use "belgrade," while others would use "belgrade-serbia."

Another good example is cities in Ukraine. These cities have at various points in history been part of Kievan Rus', Russia (including the Tsardom of Russia and then the Russian Empire), the USSR, and Ukraine. Then it starts getting even more complicated considering the various romanized spellings of different languages for these cities. Should there be multiple sets of keywords with different spellings for each of these cities to signify different points in history (e.g., "kyiv-kievan-rus'," "kiev-tsardom-of-russia," "kiev-russian-empire," "kiev-ussr," and "kyiv-ukraine?" 

(Then it gets even more complicated when talking about cities like Sevastopol that are currently part of the disputed Crimean peninsula, which is regarded by the international community as still part of Ukraine but which Russia has annexed/occupied.)

Why go to all that trouble to bifurcate all these keywords, when single keywords like "kyiv-ukraine" will suffice and are accurate, since the geographical concept of Ukraine has been part of the mix for a long time even when it was part of other countries? 

It's the same with Belgrade. Even when it was part of Yugoslavia, it was also at the same time still part of Serbia. 

Interestingly, following your edits, 12 titles currently have both of the keywords "belgrade-serbia" and "belgrade-yugoslavia." 

Meanwhile, the keyword "belgrade" now has 0 titles, but it is not blocked or merged into any other keyword, so unless something is done with this keyword, it will inevitably be started up again. This is what happens when large numbers of specific keywords are privately, manually (and temporarily) edited by a single contributor, rather than permanently modified  by staff following a public discussion. 

I do not know what the solution is, but in this particular area I would prefer consolidating into as few keywords as possible.

Perhaps for cities like this it makes the most sense to make an exception and not use the countries as part of the keywords, keeping it simple with single keywords like "belgrade," "kyiv," "chernobyl," "crimea," etc.

Your assumption is wrong.  It was not me, but I am curious about who it was -- and would like to commend them on the attempt.

Instead of having ONE generalized "city" keyword for ANY city, I, as stated above, go by how the city is acknowledged within a specific title.  Through wars and other changes, I go by the specifics. "berlin-germany," "east-berlin-germany" and/or "west-berlin-germany," for examples, can exist alone or in any combination, again, depending on how there are addressed within a specific title.  

Many cities have changed names (and even spellings) throughout history, and some cities have even disappeared, so their name "as it is addressed in a specific title" should be the determinate.  Anyone researching a cities evolution would need to search ALL its past names.  

Look at the Wikipedia "list of city name changes."  To try to convert each to only one name is a disservice to historical truth.

I live in New York City, or is it New Amsterdam?  Maybe.  Perhaps. But... don't think I was alive back then.    

@bradley_kent​ 

I think you mean "east-berlin-east-germany" and "west-berlin-west-germany." Obviously these are different cities than Berlin, and they each deserve their own keywords.

Same with cities that change names over time, like your New Amsterdam example.

But those are different from when a city retains the same name and geographical unit, but that geographical unit might temporarily become part of a different country and/or the romanized spelling might change. The examples I have given are Belgrade plus a number of Ukrainian places: Chornobyl/Chernobyl, Kyiv/Kiev, Belgrade, Sevastopol, and Crimea (not a city but a region).

Belgrade has been part of Serbia for hundreds of years, even while it was part of the Ottoman Empire, and more recently, part of Yugoslavia. 

The Ukrainian cities I mentioned were still part of Ukraine even while they were part of Russia and the USSR. 

For cities like this, perhaps it does make sense to allow standalone city keywords without a country, like "belgrade" and "kyiv." After all, there are already exceptions for other cities, like "new-york-city," which also does not have a country or state within the keyword.

The alternative is to include countries in all keywords. But perhaps both sets of keywords should remain. For example, "belgrade," "belgrade-serbia," and "belgrade-yugoslavia" could all coexist as keywords.

It starts to get more complicated though with many of the Ukrainian cities, which have different Romanized spellings under Ukrainian control versus Russian control. 

Again, I don't profess to know what the best answer is, but this should be resolved one way or the other. 

Similar to "belgrade," there is also the keyword "istanbul." In the past two months since I started this post, somebody took a lot of time to manually edit "belgrade" down from 72 titles to zero titles and "istanbul" down from 146 titles to zero titles, and then you just mentioned both those cities today. 

@bradley_kent​ p.s. I am taking "belgrade" and "kyiv" off the list for now, and perhaps a new post should eventually be started to discuss and resolve how to handle the unique category of cities I have discussed above like Belgrade, Chornobyl, Sevastopol, Kyiv, etc.

__

Edit: to clarify, I removed "kyiv" from the list, but kept the proposed merger of "kiev-ukraine" into "kyiv-ukraine."

Any work that I have done on city keywords was determined by the specific content of the title in question.  If I couldn't figure that out, I just left it as a city, sans country.  Again, I really think that the determining criteria should be how the city is addressed in a specific title, and that, of course, could be more than one way,