black friday shoppers

Holiday Data Spikes and AgilOne Platform Scalability

One of the key differentiators of the Acquia AgilOne CDP is the scalability of the platform. Scalability is one of Acquia AgilOne’s key architectural tenets, powered by our tech stack’s horizontal scalability and elasticity. This is evident from how large, high-demand enterprise brands deliver relevant and personalized customer experiences via the AgilOne CDP, processing hundreds of millions of customer records, billions of transactions and billions of events daily. One illustration of the AgilOne platform’s scalability is how easily we are able to add enormous volumes of historical data into the platform when onboarding each new enterprise client. 

Another key illustration of the AgilOne platform’s scalability is our ability to handle and ingest big spikes of data - as we did during this season’s Black Friday and Cyber Monday holiday’ for our retail clients. Let’s dive further into some interesting aspects of these data spikes. 

The AgilOne real-time pipeline ingests real-time events, including web events (browse as well as checkout/transactional events) and email events. We saw huge spikes across brands in these events and transactions, and our platform successfully ingested these data spikes. This success is not just about data ingestion but is also about scalability in all the other modules like campaigns, analytics and machine learning, which all need to operate on these increased volume spikes. Supporting spikes like this is mission critical to our clients’ bottom lines.   

Data Points in Web Traffic Volumes: 2019 Black Friday / Cyber Monday

  • Multiple brands had spikes of greater than 20 times their regular web traffic volume on Black Friday and again on Cyber Monday; these peaks correlate to the times when they sent large campaigns. AgilOne’s CDP also ingested these spikes in email events, so the campaigns had double the impact on the data ingested into AgilOne.
  • One brand had spikes of greater than 30,000 events per minute (compared to their regular volume of less than 1,000) - this was early morning on Black Friday.
  • Another brand had sustained spikes over time resulting in greater than 1.3 Million events per hour, and greater than 25M events per day (which was on Black Friday).
  • Another brand normally gets 20,000 calls per hour on average. But on Sunday, ahead of Cyber Monday, this brand reached peaks of ~430K calls per hour, i.e., greater than 20 times their average.
  • Even some of our European clients saw about 20 times their normal volume consistently throughout the holiday weekend.
  • In that week, across brands, we processed more than 350 million web events -- more calls in that week than an average month.

Get updates!

Receive the best content about the future of marketing, industry shifts and other thought leadership.

Data Points From Email Traffic Volumes

We saw spikes in email campaigns and the events generated by them, which are all ingested into the AgilOne platform via the extensive ESP connectors we have. Some interesting data points are:

  • We ingested ~3.7 billion email events (which include email send, email open, email click) across all brands during the holiday month. This is 15% more than the email events in the previous month.
  • We have consistently seen higher email click rates, leading to better engagement during the holidays in 2018 and 2019. For example, in 2019, while 29% of the monthly email sends were in the holiday week, 32% of the monthly email clicks were during the holiday week thus showing greater engagement during holidays.
  • We supported multiple intra-day spikes for web and email events on the days of Black Friday and Cyber Monday. 

Data Points from Transactional Volumes

As expected with the increased web and email activity, this resulted in a spike in transactional data as well. We noticed some interesting data points in the transactional data ingestion

  • On Black Friday 2019, on average across brands, we ingested approximately six times our clients’ normal daily transactional volume across online and offline transactions. 
  • Some brands had greater than nine times their normal transactional volume on this day.
  • On average, our clients saw 13% growth in transactional volume when compared to the same period last year. A majority of that growth came from digital sales channels driven by convenient features such as "Buy Online; Pickup in Store."
  • The percentage of digital transactions in the total transaction volume during the week of Black Friday and Cyber Monday was 23% in 2018, then jumped up to 30% in 2019.

Powering Machine Learning, Campaigns and Analytics

Once these data spikes are ingested through the AgilOne data pipeline, this data powers all the CDP modules, which illustrates the scalability in these modules as well. So, these data spikes directly translate to helping marketers achieve better results.

  • AgilOne’s Machine Learning module generates greater than 1 billion predictions across brands
  • AgilOne’s Metrics module powers dashboards and ad hoc reports to deliver insights based on this increased event and transaction volume.
  • AgilOne’s Actions module scales up to help marketers act on these newly ingested data spikes to deliver campaigns.

Gangadhar Konduri

SVP, Acquia Marketing Cloud

With more than 23 years of experience in engineering, product management and leadership, Gangadhar is responsible for Acquia Marketing Cloud’s innovation and growth while leading the global product development teams.

Prior to joining Acquia, Gangadhar was the Chief Product and Technology Officer at AgilOne responsible for engineering, product management, strategy. SaaS Operations and Professional implementation services, and led the transformation of AgilOne to be the premier enterprise CDP, and to the successful acquisition by Acquia.

Previously, Gangadhar was Vice President at Oracle leading several PaaS and SaaS development initiatives while defining, building and launching products in Oracle Cloud and Oracle Fusion Middleware.

Gangadhar and his family live in Palo Alto, California. Gangadhar holds a Master’s degree in EECS from M.I.T., and undergraduate degree in Computer Sciences from I.I.T. Kharagpur, India.