brianrisselada's profile

85 Messages

 • 

1.9K Points

Saturday, February 12th, 2022 1:58 AM

No Status

4

Add music videos to the IMDb Dataset title.basics.tsv.gz

Hello, on February 9, 2022 I downloaded the file title.basics.tsv.gz from the dataset at https://datasets.imdbws.com/

This data set is missing all of the titles in the Music Video category. Can you please fix this so that these appear in this dataset document?

Employee

 • 

17.2K Messages

 • 

310.6K Points

3 years ago

Hi brianrisselada -

Thanks for reporting the missing titles, I have alerted the appropriate technical team to investigate.  As soon as I have an update I will relay the information here.

Cheers!

85 Messages

 • 

1.9K Points

@Michelle​ Hi

Have you heard anything back about this yet?

Thanks!

85 Messages

 • 

1.9K Points

3 years ago

Thank you so much Michelle

85 Messages

 • 

1.9K Points

3 years ago

Also, I didn't check for sure but I think they may be missing from the other dataset files too like crew and ratings.

Employee

 • 

17.2K Messages

 • 

310.6K Points

Hi @brianrisselada​ -

Thanks for this additional comment, I have added this to the ticket.  I can confirm that the ticket is currently still being researched, as soon as I have an update I will be sure to post the details here.

Thanks again for your patience!

85 Messages

 • 

1.9K Points

3 years ago

Hi @Michelle , just checking in again to see if there are any more updates on this.

I appreciate it.

Employee

 • 

17.2K Messages

 • 

310.6K Points

Hi @brianrisselada​ -

The ticket is still being investigated by the applicable tech team.  As soon as I have an update on the status I will let you know here.

Thanks for your patience!

85 Messages

 • 

1.9K Points

Hi @Michelle​ 

I hope you don't mind me just checking in again to just confirm if there have been any new updates on this or time estimates for fixing.

Thanks.

Employee

 • 

17.2K Messages

 • 

310.6K Points

Hi @brianrisselada​ -

I just checked the ticket status and can see that it is still pending review by our tech team.  Unfortunately, I cannot provide a timeline as to when it will be resolved, but I have given a nudge on the ticket to see if I can help speed up the investigation.

Thanks again for your patience!

85 Messages

 • 

1.9K Points

@Michelle​ Thank you!

85 Messages

 • 

1.9K Points

@Michelle​ Just popping in again to see if you think it would be helpful to give another nudge on this ticket since it's been a couple more weeks now.

Employee

 • 

17.2K Messages

 • 

310.6K Points

Hi @brianrisselada​ -

I reviewed the ticket and unfortunately, it is still pending review.  Unfortunately I cannot provide an exact timeframe as to when it will be handled by the tech team, although I will continue to nudge the ticket for progress.

85 Messages

 • 

1.9K Points

@Michelle​ I appreciate nudging it.

It is frustrating that this data that has been freely available that we have been using has been missing a bunch of information for a while now.

(edited)

85 Messages

 • 

1.9K Points

@Michelle​ Just checking in again.

I just downloaded the latest file again and the data is still missing.

Do you know why this is taking so long or why it does not seem to be a high priority?

I like that this new category of Music Video has been created, but to provide these files but not have all of the titles showing up in them makes the files useless in many ways. And it has been like this for several months at least.

Thanks for your help.

85 Messages

 • 

1.9K Points

@Col_Needham Do you have any insight into this?

Thank you.

85 Messages

 • 

1.9K Points

@Michelle or anyone else? Is anyone reading this?

Is there any other way to easily extract this information?

85 Messages

 • 

1.9K Points

@Michelle​ Thought I'd just check in again to see if there is any movement on this.

Thanks.

85 Messages

 • 

1.9K Points

Just checking in again. Haven't heard from @Michelle in over a month so just wondering if any other employees know anything about this.

@Bethanny @Maya @Sally @Fran @Edward @Jon 

2.7K Messages

 • 

47K Points

@brianrisselada​ It seems we hear from IMDb staff less and less these days on this forum. I am waiting for numerous pending posts to be resolved by staff.

85 Messages

 • 

1.9K Points

@keyword_expert​ Thanks for your comment. I actually was wondering if it was just this issue that was no longer being given attention or if it was happening more in general.

85 Messages

 • 

1.9K Points

I'm not sure what else to do. I seem to have been abandoned.

I guess I'll see if there are any other ways to contact employees at IMDb other than through this board.

If anyone reading this is aware of any other ways like a phone number or email or other people who work at IMDb that might be able to help, please let me know.


Thank you.

2.7K Messages

 • 

47K Points

@brianrisselada​ 

I recently received this response from @Michelle on this thread:

Again, I apologize for the delay on handling these larger-scale Keyword updates.  After this coming US Holiday we are hoping to systematically handle these older requests on a more consistent basis.

85 Messages

 • 

1.9K Points

@keyword_expert​ Thank you for sharing.

85 Messages

 • 

1.9K Points

What IMDb staff is encouraging with having an incomplete dataset available here is for people to start mass scraping the site for data now. I know they don't like data scraping on their site but if they don't provide the data any other way that's about the only way available to get it.

85 Messages

 • 

1.9K Points

@Michelle​ It's been three months since I've heard from you. Can you give me a status update on this?

Thanks.

2.7K Messages

 • 

47K Points

3 years ago

Brian, can you clarify what you mean by "Music Video category?"

As far as Genres, there is a Music genre and a Musical genre. Are you referring to one or both of these? (There is not a Music Video genre.)

There is also a "music-video" keyword, which is currently assigned to about 83,400 titles.

https://www.imdb.com/search/keyword/?keywords=music-video

Which of these is missing from the file?

cc: @ACT_1 

___

Edit: I figured it out. You were referring to the music video title type, which largely overlaps with the "music-video" keyword.

https://www.imdb.com/search/title/?title_type=music_video

(edited)

85 Messages

 • 

1.9K Points

@keyword_expert​ Thanks for asking. Let me clarify.

The "Music Video category" I'm referring to is neither a genre category nor a keyword category.

Rather it's in the category that IMDb seems to define as a "Title Type"

For more clarification, the list of categories under Title Type is short enough to list here (you can see them in the Advanced Title Search form https://www.imdb.com/search/title/):

  • Feature Film
  • TV Movie
  • TV Series
  • TV Episode
  • TV Special
  • Mini-Series
  • Documentary
  • Video Game
  • Short Film
  • Video
  • TV Short
  • Podcast Series
  • Podcast Episode
  • Music Video

The category of "Music Video" was recently created. However many of the titles that are currently designated as this category have been in the database for a long time, but used to be designated in other categories in the past, such as "Short Film" or "Video".

The following search URL shows the titles that are missing from the file: https://www.imdb.com/search/title/?title_type=music_video

(edited)

85 Messages

 • 

1.9K Points

@keyword_expert​ Do you know anything else about this issue? I haven't heard back from anyone in a while.

2.7K Messages

 • 

47K Points

I don't know anything except what has been posted in this thread.

85 Messages

 • 

1.9K Points

@keyword_expert​ From your experience do many people use these dataset files? They don't seem too concerned with fixing this very quickly. I thought this was a resource that a lot of people used.

2.7K Messages

 • 

47K Points

@brianrisselada​ I really wouldn't know; I only learned of the existence of these files around the time you posted this thread.

Employee

 • 

17.2K Messages

 • 

310.6K Points

2 years ago

Hi @brianrisselada -

Again, my sincere apologies for the delayed response.  I understand the frustration, especially given that you reported this issue initially back in February. 

I just reviewed the open ticket and can see that it is still being researched.  As this ticket has not been prioritized, I have again inquired on the status with the applicable team, once I receive a response I will post the information here.

85 Messages

 • 

1.9K Points

@Michelle​ Apology accepted. Thank you for the response.

If I don't hear back after a certain amount of time, is it OK if I check in again? If so, what would that amount of time be?

Thank you!

2.7K Messages

 • 

47K Points

@Michelle​ It doesn't seem right that @brianrisselada should have to wait more than a year to learn anything about why this problem exists and what can be done to fix it.

85 Messages

 • 

1.9K Points

@keyword_expert​ Thanks.  I just wish I knew if it was an issue of difficulty or prioritization.  It's hard to imagine why it would be so difficult to fix, but of course I'm no expert. If it's not that difficult, why isn't it a prioritization as this used to be a feature that the staff seemed to emphasize. But maybe no one else is using it, or they don't care that it's now missing data?  I just wish I could get some more info so I knew what to expect and how to keep planning going forward regarding this.

2.7K Messages

 • 

47K Points

@brianrisselada​ It feels like it is taking more effort for staff to read this thread and provide non-update updates than it might take to just fix the problem. Very frustrating.

2.7K Messages

 • 

47K Points

2 years ago

Was this issue ever resolved?

85 Messages

 • 

1.9K Points

@keyword_expert​ I just downloaded the latest file and it is still not fixed. Music videos are still not appearing in the file.

85 Messages

 • 

1.9K Points

2 years ago

@Michelle It's been four months since we last checked in. I just confirmed that the issue still isn't fixed. Would you mind checking into it to see if there's any movement on it or if there's anything else that can be done to try to push it along?


Thank you.

85 Messages

 • 

1.9K Points

2 years ago

@Michelle  Happy new year!

Just checking in on this one again. Have there been any updates?

Thank you.

85 Messages

 • 

1.9K Points

2 years ago

So it's been over a year now since I created this ticket and this still isn't fixed. This is quite disappointing.

Does anyone have any updates?

85 Messages

 • 

1.9K Points

1 year ago

@Michelle Checking in again. Any updates?

Employee

 • 

17.2K Messages

 • 

310.6K Points

Hi @brianrisselada​ -

I am profoundly sorry it has taken over a year to confirm the status on this issue.  I am personally escalating further to the team responsible to confirm either if there is a planned fix or if the datasets will remain as-is without Music Video titles.

85 Messages

 • 

1.9K Points

@Michelle​ I am glad to hear from you! I am incredibly grateful for you following up and and escalating the issue.

Employee

 • 

17.2K Messages

 • 

310.6K Points

Hi @brianrisselada​ -

I have just confirmed with the applicable team manager that due to several factors, including scalability and company focus, unfortunately we will not be updating the IMDb Dataset title.basics.tsv.gz to include Music Video titles.

Sorry it has taken so long to provide you with and answer, especially as it's likely not what you were hoping to hear.

85 Messages

 • 

1.9K Points

@Michelle​ That's extremely unfortunate as the datasets are now incomplete and actually lacking titles that used to be there before.

85 Messages

 • 

1.9K Points

Contributors were responsible for this data and yet the amount of it they can get back keeps shrinking and shrinking over the years.

(edited)

85 Messages

 • 

1.9K Points

@Michelle​ Would I have access to this data if I were a top contributor?

10.6K Messages

 • 

225.3K Points

I think, the folks who run IMDb are worried about rogue websites (and beyond) harvesting data from IMDb. Regardless, the mitigation observed over the past half decade may in turn be exacerbating the problem.

85 Messages

 • 

1.9K Points

@jeorj_euler​ if that were the case then why not just get rid of sharing the data at all instead of advertising it as a resource for contributors but then not fixing it after they make changes that break it?

10.6K Messages

 • 

225.3K Points

So, as not to lose too many contributors on the crowd source end. The amount of utilization of particular data types is also taken into consideration. Public access to underutilized ones tend to be locked off if not the data itself being discarded entirely.

85 Messages

 • 

1.9K Points

@jeorj_euler​ well that data used to be there. All that happened was they added a new type but apparently didn't update their program that spits out the dataset files to include it. I can't imagine it would even take that much time to update. They don't even care about giving the smallest amount of time for our sake though.

2.7K Messages

 • 

47K Points

@brianrisselada​ 

It is disappointing to see IMDb staff reward your diligence and patience with complete disregard of the problem.

They have been doing the same thing on keyword problems the past few months. They even  wrongly closed most of my open threads on this message board, including threads where they haven't even yet decided whether to take action.

All of this is really disappointing, and it has caused me to become disenchanted with the IMDb website and this forum in 2023.

85 Messages

 • 

1.9K Points

6 days ago

@Michelle There by chance aren't any new updates regarding this are there?