Do you think that you have the right stuff to become a great product reviewer for Apple's New Mac App Store? Well, it appears that Apple wants to upgrade their app reviewing process to provide a real community-based predictive ranking system for products that small business owners and productive consumers could come to rely on for rich reviews beyond just paid hype. However it's a double-edged sword because the public reviewers will also be compensated for their work in a number of ways. Apple's system could be interesting over time, as this system is based on the number of accurate reviews over time. This is a complicated system but one that marketers and commissioned salespeople could understand. But if you think you're cut out for this, then it may be worth investigating.
Online electronic stores are becoming more and more common. Online stores can be directed to the general public or to owners of specific devices. These online stores offer a variety of items in order to serve the needs of as many potential customers as possible. These items can include media files, applications, books, etc. Item quality can vary widely. Popular taste also varies widely. Some items of only standard quality can become extremely successful while other outstanding items go unnoticed. Oftentimes these online stores don't offer adequate preview mechanisms for these items, so potential customers don't know if a particular item could fit their needs and/or wants.
The Small business Owner Example
An example perhaps best illustrates this problem. Consider Jane, a small business owner, who is looking for a mobile accounting program that can sync with her small business accounting program. Jane browses the online store and finds three different applications that purport to meet these requirements. However, Jane is uncomfortable paying for one without knowing if it will really do what she wants it to do.
The promotional information provided by each manufacturer gushes about how fantastic each application is. In order to sort out which one best suits Jane's needs, she must do extensive research to find unbiased information if the online store does not aid her. Often, if Jane is researching online she must browse away from the online store and may decide to purchase the accounting application from another vendor. It is in the interest of the online store to provide as much information as possible for Jane so that she can make an informed decision and purchase the software from the store and not elsewhere. One way to approach this problem is to allow individuals to review applications for functionality, ease of use, safety, speed, accuracy, and/or various other criteria.
If the electronic store delegates this task to a large group of users, or "crowd sources" the task, it will save money which would otherwise pay for full-time staff reviewers. The problem with this approach is that not every reviewer is as trustworthy as the other and customers end up frustrated because they must spend a long time sorting through reviews to determine which ones are sufficiently detailed, unbiased, well-written, and otherwise generally useful. The online store can provide a credibility or trustworthiness score for reviewers based on some metric. An aspect of this patent is to provide a mechanism of identifying quality reviewers by their ability to accurately predict the sales performance of an item in the online store prior to actual sales and/or availability of ranking data.
What is needed in online stores is an improved method to allow customers like Jane to feel confident in their online store purchases without the need for hiring expensive testers or reviewers.
Community-Based Predictive Ranking System Overview
Apple's patent describes systems, computer-implemented methods, and tangible computer-readable media for community-based predictive ranking of items in an electronic store. The method includes receiving a preliminary predictive ranking of an item in an electronic store and feedback about the item from each of a group of individuals. The predictive ranking is predictive of item performance in the electronic store. The method further tracks via a processor an actual ranking of the item over time based on item performance in the electronic store, provides an incentive for individuals in the group of individuals whose associated predictive ranking coincides with the actual ranking of the item, and presents in the electronic store received feedback from at least one individual whose predictive ranking coincides with the actual ranking of the item. Rankings can be directed to different item or customer subdomains in the electronic or online store. Individuals having favorable successful prediction ratios can receive incentives.
Gathering Feedback and Rankings from Reviewers
Apple's patent FIG. 2 below illustrates an example predictive ranking history for applications. In this example, three persons labeled A, B, and C, are reviewers for the electronic store. At time T1 202, the reviewers submit their predictions for the question "Will App1 be in the top 10 in the online store?"
The reviewers have an opportunity to download and test or use all or part of the application. In this manner, the reviewers can provide an intelligent prediction of its ranking. In this example, once the application is actually sold through the online store, its ranking was high. Thus, the answer to the question is yes, indicating that it will at a later point in time be in the top 10 in the store. Persons A and B correctly predict yes and C incorrectly predicts no. The electronic store assigns 1/1 or 100% credibility to A and B and 0/1 or 0% credibility to C.
Next, at time T2 204, the reviewers submit their predictions for the question "Will App 2 be in the top 10 in the online store?" In this example, the eventual answer to this question is no, indicating that it will not be in the top 10 in the store at a later point in time. B and C correctly predict no and C incorrectly predicts yes. The electronic store assigns 2/2 or 100% credibility to B, 1/2 or 50% credibility to A and C.
Next, at time T3 206, the reviewers submit their predictions for the question "Will App 3 be in the top 10 in the online store?" In this example, the objective answer to this question is yes, indicating that it will be in the top 10 in the store at a later point in time. B correctly predicts yes and A and C incorrectly predict no. The electronic store assigns 3/3 or 100% credibility to B, 1/3 or 33% credibility to A, and 2/3 or 66% credibility to C. In this way, a history of successful predictions will lead to a higher credibility score.
Incentives for Reviewers
In some situations, an electronic store's offer of an increased credibility score is not enough to motivate sufficient numbers of individuals to review products. An electronic store can also offer cash incentives to successful reviewers. Monetary incentives are not integral to the invention and other tangible or intangible incentives can be substituted, such as credibility points, exclusive media (such as songs, photos, user icons, etc.), and others.
Predictive Ranking Credibility & Compensation Table
Apple's patent FIG. 3 illustrates an example predictive ranking credibility and compensation table. The steps outlined in this example are non-limiting. This example shows four reviewers 310, A 302, B 304, C 306, and D 308. Each reviewer, after sampling the media in question, makes a prediction of its maximum chart position 312. The sampled media can include audio, video, still pictures, software application, websites, games, and so forth. A predicts that the sampled media will reach number 17, B predicts number 1, C predicts number 50, and D predicts that it will not even break in to the top 100 chart. The actual ranking of the song 314 based on sales performance in the online store is 11.sup.th place. The system can measure actual performance based on sales in the online store, one or more external source (such as the Billboard Top 100 list), some other metric(s), or a combination thereof. A combination of metrics can be weighted or unweighted.
In one variation, the system determines the maximum number of credibility points 316 for this prediction and bases the awarded credibility points on how close to correct each prediction is. For example, the system can calculate a weight 320 based on the distance between the predicted ranking and the actual ranking 318. This chart shows that the weight is calculated by dividing the distance between the predictive ranking and the actual ranking by the sum of all the distances, 55 in this example. Other suitable mechanisms exist for calculating the weight. The system can determine the amount of credibility points based on the weight 322. The example here shows that awarded credibility points are calculated by subtracting the maximum credibility points multiplied by the weight from the maximum credibility points.
Further, the system can provide an actual cash award 324 for reviewers. In some cases, the media producer (i.e. a singer, artist, software publisher, and so forth) provides all or part of the cash award to generate promotional interest in their item in the online store. In other cases, the electronic store can provide all or part of the cash award as a cheaper alternative to hiring full-time reviewers. An electronic store can provide an equivalent to a cash award in the form of store credit or gift cards to the online store. In yet other cases, a portion of advertising revenue provides all or part of the cash award. The award can be based on the amount of advertising revenue associated with a review or ranking. In this example, the total cash award 324 is $10. The cash award can be based on assigned credibility points. To determine the actual cash award for each reviewer, this example multiplies the total cash award by the individual reviewer's awarded credibility points divided by the total awarded credibility points. The electronic store can use other distribution/compensation algorithms. In this chart, reviewer B 304 predicted the song "Cats Four Pounds" would be #1. It was actually #11, a difference of 10 chart positions. The sum of predictive distances for the four reviewers is 55 chart positions, i.e. A's prediction was 6 chart positions off, B's prediction was 10 chart positions off, and C's prediction was 39 chart positions off, for a total of 55 chart positions. The predictive weight for the ranking of reviewer B is 10/55 or 0.181. The system awards 819 credibility points by calculating 1000-(1000*0.181). The total number of credibility points awarded to A, B, and C in this example is 2001. The system calculates B's actual cash award of $4.09 as $10*(819/2001). The shown chart and calculations should be considered non-limiting and only demonstrate one possible approach among many for distributing credibility points, cash awards, and/or other incentives.
A Way to Reduce Prediction Inflation
One possible way to encourage accurate predictions or to reduce prediction inflation is to provide asymmetric incentives and disincentives for going over or under the actual ranking. For example, if the actual chart position is 10, the award for under predicting chart position 15 can be greater than the award for over predicting chart position 5. In this example, the weights are based on the collective performance of all a group of reviewers. The system can assign weights without relying on other reviewers.
Online Store Reviewing System Overview Including an iPhone Interface
Apple's patent FIG. 4 below illustrates a sample online store for gathering feedback and rankings from reviewers. The online store can notify users 404a, 404b and 404c that an item is available for review. Would-be reviewers can request to review a specific item in the online store. Users can interact with the online store wirelessly with mobile devices, with laptop computers, desktop computers, or with any other suitable computing device over any type of wired or wireless data connection.
The online store retrieves the item or a sample of the item to be reviewed from an items database. The online store can decide the amount and form of the item to be reviewed based on the type of item. For example, if the item is a song, the store can deliver the entire song, a 60-second full quality sample, a reduced quality sample of the entire song, etc. If the item is an application, the store can deliver the entire application, a full-featured version of the application set to expire after 7 days, or a limited functionality version of the application, etc. The online store can suitably adapt other items for review.
Once the reviewers receive the item to review from the online store, they can use or sample the item. Reviewers can then use their judgment to predict a ranking for the item in the online store and provide a review or feedback. The ranking can be as simple as a chart or sales position. The ranking can be tied to a specific time frame, such as a reviewer predicting that the item will be in the top 10 chart for at least 5 consecutive weeks. The review can be somewhat more detailed, with a description of features, ease of use, speed, performance, accuracy, etc. The reviewers transmit their predictive ranking and reviews to the online store which stores them in a reviewer database.
Apple's patent FIG. 6 illustrates an exemplary user interface for reviewing and predictively ranking audio files on a mobile device display 600.
Apple's patent application 20100332296 has much more details if you're interested in this future opportunity. Apple credits Sam Gharabally as the sole inventor of patent which was originally filed in Q2 2009.
Notice: Patently Apple presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for further details. For additional information on any patent reviewed here today, simply feed the individual patent number(s) noted in this report into this search engine. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.