Chuq Von Rospach writes about how Apple can implement a system to report unwanted “Rate this app” dialogs.
What you want to build here is a reputation system. Every time this team validates a report, everyone who made that report gets their reputation value incremented. Every time a report is rejected, those that reported gets their reputation value decremented. Over time, you’ll build a data set that will tell you how reliably a person giving a report is in sync with the standards of those judging the reports.