Having pioneered Content Testing in India for over a decade, we often get asked questions from our prospective clients about the intricacies of the process. In this multi-part series (one edition every month), we attempt to answer some of the most common questions we get asked around the various aspects related to Content Testing. 

1. What is the difference between focus screenings, test screenings, focus groups, and other such terms one keeps reading about?

These terms are often used interchangeably. Without getting into technical distinctions, it can be said that all such terms refer to market research being done to test content, such as a movie or a series. We like to refer to this type of work collectively as Content Testing. Our Content Testing products are called Ormax True Value (for Television shows), Ormax Moviescope (for Theatrical films), and Ormax Stream Test (for OTT content).

2. How secure is the process of Content Testing? Do I need to worry about my content getting leaked?

Across almost 1,000 Content Testing projects conducted by us till date, there has not been a single instance of any leak. We follow a strict ‘Privacy & Security’ protocol, which is summarized in the chart below. For films and high-profile shows, we never take physical possession of the material (e.g., the hard disk) ourselves. We will be happy to answer any questions you may have on the protocol.

3. How do you select the test audience?

Over the last 16 years, Ormax Media has built a repository of industry data that captures the gender, age and market profile of core audience of various categories in various languages. Based on this data, we recommend a research design to the producer or platform commissioning the project, and lock the final design in consultation with them. Depending on the scale of the project and the budgets, a project may cover a sample size of 250-1,200 audience, across 2-6 markets.

Ormax Media runs a large online panel of users, who are invited to Content Testing projects if they meet the selection criteria. No panelist can attend more than one research in one category (theatrical/ OTT/ TV) in a period of six months. The panel is incentivised (e.g., Amazon and Bookmyshow gift vouchers), in line with standard practices with such panels globally.

4. At what stage of the content development process should I consider testing?

We can test content at three stages of the development process:

The choice of testing at any particular stage of the development process is determined by the objective of the producers and/or platforms. Evidently, there is a trade-off between accuracy and scope of development and changes, and that becomes a determining factor in deciding the stage of testing.

5. How many films, shows and scripts have you tested so far?

We have tested 408 theatrical films, 252 OTT shows and films, and 333 TV shows at the time of publishing this article, at synopsis, screenplay or pilot episode stages, across the languages mentioned above Hindi, English, Tamil, Telugu, Kannada, Malayalam, Punjabi, Marathi, Bengali and Gujarati languages.

6. How do you measure the performance of a piece of content in a Content Testing research?

Across our content testing tools, content likability is measured by a metric called Ormax Power Rating (i.e., OPR). OPR is a single number on a scale of 0-100 which represents how much a piece of content was liked by the people who consumed it. In principle, it is indicative of the extent of engagement of the audience with the content, and predicts the conversion of initial viewership to final viewership for any content piece.

We collected OPR for all major films and shows after their release among actual viewers of the content. At the testing stage, the objective is to arrive at an early estimate of the actual OPR, to be able to pre-empt the actual performance of the content upon its release.

7. How is Ormax Power Rating (OPR) calculated?

At the testing stage, OPR is calculated through a combination of benefits that audience seeks from content. These are encapsulated as Ormax Power Rating (OPR) drivers, which are factors that can have a positive influence on a show or film’s audience engagement, once they have decided to watch the content in question. A combination of various benefits leads to Ormax Power Rating (OPR), our proprietary metric to measure audience likeability of all content, ranging from theatrical films to streaming originals to linear television shows.

OPR drivers vary by medium (i.e., different drivers for theatrical content vs. OTT vs. TV), and formats (i.e., different drivers for OTT fiction series vs. direct-to-OTT films vs. Documentaries, etc.). Their weights vary by language to account for the varying tastes and preference of different audience sets. The drivers are updated every year, and their weights are updated every quarter.

8. How do I interpret expected performance of the content tested, based on its Test OPR?

Three conceptual OPR performance levels have been identified by us, through extensive benchmarking across mediums and categories:

The performance of the test property is then benchmarked against the spectrum of performance across these categories:

Additionally, multiple forms of additional benchmarking (i.e., against content in the same genre, past work of the same actor, director, production house, etc.) helps contextualize the performance of the content in a more nuanced manner.

9. How do you assess the accuracy of Content Testing, and what has the accuracy been so far?

For films and shows that release, we compare the actual performance to the forecasted performance as a measure of accuracy. For example, if a film was forecasted to do a business of ₹125 Cr but actually did only ₹105 Cr, its deviation is 20/105=19%, and hence, its Accuracy is 81%. This calculation, when done at a cumulative level for all films (or shows) tested, gives us the overall Accuracy of our testing products, which is typically in the 80-85% range.

10. What are the various decisions that Content Testing can help me make?

The infographic below highlights the various strategic and tactical decisions that can be taken in a more informed manner, with the help of Content Testing:

We understand that you may have other queries too. Watch out this space for the next edition of the FAQs on Content Testing. Additionally, we will be happy to connect with you and answer your more urgent queries.