Building a Product Value Predictor

Dynamic pricing is based on a simple approach – pick out a product, find identical offerings in competitor catalogs, and monitor prices. If competitors change the pricing for the products in question, use automated (or, sometimes, manual) systems to adjust accordingly.

Since retail and ecommerce customer behavior is heavily driven by prices, such an approach provides a way to make sure the company in question always has the best offer. Various other, more complicated implementations are used, such as measuring supply and demand factors and changing prices accordingly, however, the most simple one is a good starting point.

There are two challenges associated with dynamic pricing. A relatively simple one is acquiring the price from various sources. Websites will each have different layouts (and even those might change frequently) for the same product, necessitating a dedicated scraper and parser for each one.

Yet, getting pricing data is relatively simple when compared to product matching. Finding two identical products is harder than it may seem. First, there are many products that are, in some sense, the same, but have different specifications (e.g., various iPhone models have different capacities for storage while being named identically).

Additionally, while intuitively, we think product titles will be identical, it turns out not to be the case. Names aren’t the only thing retailers include in a title – various features, specifications, and other aspects may be included. As a result, the process of establishing identical products across platforms is not as simple as equating two titles.

There’s also little room for error. Sending out pricing changes after misidentifying products can cost businesses enormous amounts of revenue. Product identification, as such, has to be so well-designed that errors must be minimal. If there’s any doubt on whether the two retrieved ones match, they should be removed.

Businesses, however, still arrive at functional dynamic pricing systems. Some are more complicated, others simpler, but all of them have these two aspects to them – price matching and product identification.

These systems can be expanded to cover a novel area – unstocked product value discovery.

Entering a new product into inventory comes with some measure of risk. It may not be popular and may not generate as much revenue as one would expect. These occurrences are fairly frequent in retail, as predicting consumer demand beforehand is a tricky process.

If a product of interest exists in a competitor’s inventory, some data about its importance can be extracted through the use of a similar system to dynamic pricing. First, either manual or automatic processes should be used to ensure that nothing of the like exists in a company’s inventory.

That product can then be added to the list of suitable candidates. A similar algorithm to the one used in dynamic pricing can be employed to monitor that product and the data associated with it. There are two avenues that can be used to predict performance.

Some businesses show the available stock as public data. These are the best indicators for product performance as they can be collected to evaluate how much of it is being sold on a regular basis. It is even possible in cases where stock data is shown in an abstract format (e.g., colors or bars, etc.), although the signals produced would be much weaker and prone to mistakes.

Not all retailers, however, choose to show remaining stock. Another option, then, is to scrape review numbers and changes in them. Data on other products, however, should also then be acquired to properly evaluate popularity.

Without additional data, review counts could be meaningless as you would have no idea what’s considered “normal” on the specific website. Once a measure of the average amount of reviews left on product categories is established, a proper evaluation of the product can be conducted.

Finally, all of this can be hedged against known top sellers. They can be used as the peak of potential and everything else could be measured accordingly.

It’s important to note, however, that these are abstract popularity signals. Outside of cases where accurate available stock numbers are available, the data provides only pointers instead of action points.

If found, a popular product that has newly arrived at a competitor’s inventory could be included with significantly minimized risk. As data collection is significantly cheaper than product introduction, even misses wouldn’t produce results that have greater negative outcomes than usual.

Dynamic pricing systems can be slightly altered to deliver novel and interesting data, particularly of those products that a company has yet to add to its inventory. If competitors working in the same industry are having success with one such product, data about it can be collected in order to minimize risk.

After a sufficient and thorough analysis, it can be discovered whether that product or service is a worthwhile addition. As such, web scraping can make adding new products or services significantly less risky.