Web Scraping Drives Decision-Making In the UK & US Ecommerce

Published on October 22, 2022

Over 82% of eCommerce organizations are now using web scraping to gather external data to help guide their decision-making. Web scraping overtakes internal data collection (81%), which is the number one method in nearly every other sector. This is one of the key findings from the new Oxylabs white paper, Alternative Data Defines Competition in the US & UK Ecommerce Sectors. It demonstrates how web scraping is shaping the industry.

In cooperation with Censuswide, Oxylabs surveyed 500 UK-based and 501 US-based senior data decision-makers from UK and US eCommerce companies. Oxylabs gathered insights on attitudes towards data collection in the sector.

 Web scraping takes the lead

Web scraping has entrenched itself as the primary vehicle for decision-making. This technology has become the most popular way to collect data in the eCommerce industry.

When asked what methods the respondents use to acquire data, 75% of eCommerce organizations selected web scraping. Also, 74% are using the runner-up, internal data collection. Other methods, such as the procurement of data from third-party aggregators, fall further behind at 65% for traditional, and 59% for alternative data. Such a trend isn’t surprising. Having control of the entire pipeline allows for greater flexibility and adaptability to business goals.

When asked which type of data respondents use to make business decisions, third-party aggregators came in third behind web scraping and internal data. Only 66% view it as the most crucial source of data-driven decision-making. Such a trend might be caused by eCommerce companies and retailers mainly dealing with regular consumers. Third-party aggregators are often oriented towards B2B data, making their offerings more useful to internal decision-making than direct revenue generation.

Gediminas Rickevičius, VP of Global Partnerships at Oxylabs, said: “Web scraping overtaking internal data collection in any industry is a milestone moment. External data collection is where businesses across many sectors will compete fiercely with one another in the future. eCommerce is an important trendsetter in this respect. As external data offers exciting, unique, and profitable insights, getting to them first should be a priority for every business, whether involved in eCommerce or not. The reliance on data is clearly important. Therefore it is no surprise that the budgets are also expected to increase over the coming year.”

Expanding data budgets

As the data access needs increased (cited by 85%, with 30% citing significant increases), so did the budgets for data departments (cited by 73%, with 27% citing substantial increases). Additionally, 70% of respondents expect to continue expanding their data department budgets in the coming 12 months.

Rickevičius continued: “Ecommerce isn’t the only industry heading towards significantly increasing the procurement of external data. These trends are visible throughout all industries. They show that external data, especially its advanced forms, has become an essential part of daily operations. Today’s business optimization is achieved through careful data-driven decision-making. It shows the immense importance of departments that support such processes.”

“In eCommerce, where margins are thin and the competition is fierce, companies will take absolutely any advantage they can get. Web scraping has now proven itself as the way to do so. Through external data, businesses can monitor the competition, adapt to changing environments. They can even predict consumer behavior, bringing it closer to optimal performance,” concluded Rickevičius.

To download your copy of the Oxylabs’ Alternative Data Defines Competition in the UK & US eCommerce sectors white paper, please visit HERE.

Julius Cerniauskas is a Grit Daily contributor and the CEO of Oxylabs, one of the biggest companies in the public web data collection industry, employing over 300 specialists. Since joining the company in 2015, Mr. Cerniauskas has implemented a brand new leadership company structure, taking product and service technology to the next level, as well as securing long-term partnerships with dozens of Fortune 500 companies. Mr Cerniauskas is Lithuania’s technology industry leader who speaks on the topics of web scraping, big data, machine learning, technology trends, and business leadership. Today, he continues to lead Oxylabs as a top global provider of premium proxies and data scraping solutions, helping companies and entrepreneurs to realize their full potential by harnessing the power of data.

Read more

More GD News