Black Friday Traffic: Top 4 Challenges in the Cloud and How to Overcome Them

Black Friday represents one of the biggest web traffic spikes of the year, concentrated over a short and intense period. This surge is on a predictable upward trajectory as online sales and services continue to grow exponentially each year.

In recent years, when demand has peaked uncontrollably, retailers have turned to implementing a queue-based system, trying to strike a balance between not losing business and ensuring they can offer a smooth, error-free online experience. Re-creating an in-store analogue paradigm to cope with demand runs analogous to what services have promised - infinite burstable resources that can scale on demand to meet any needs.

Cloud infrastructure is clearly well suited to the task of handling traffic surges, without the constraints of a fixed amount of computing resources and almost infinite opportunity to grow. Uncontrolled cloud growth can, however, lead to a nasty shock when the bill arrives.

Security is also a major consideration and is tested during the Black Friday period. Miscreants look to exploit potential breakdowns or ad-hoc changes in usual processes where stress can easily amplify human error. Introducing “quick fixes” to infrastructure configuration or throw-away changes to apps intended to facilitate increased traffic can inadvertently lead to compromised apps, which are made all the more difficult to detect amongst the volume of traffic.

While the focal point for Black Friday is online retailers, a multitude of other systems are involved and can be easily forgotten. Notably, most web apps introduce some form of user intelligence tooling to help make visual sense of web traffic source or identify user experience issues, and these can inadvertently burden and slow down an otherwise well-behaving web app as they struggle under load.

The Divio platform has developed strategies to gracefully accommodate the Black Friday traffic surges and ensure the apps that depend on the platform remain fast and accessible even with unprecedented and unpredictable traffic volumes.

Based on these experiences, we can break down the Divio approach into four common problem areas.

1. Scaling gracefully - the balance between cost, complexity, and performance

Scaling involves the provisioning of cloud infrastructure but also leveraging the features of the infrastructure fully.

Scaling cloud resources can be interpreted as allocating sufficient resources to your app in good time to accommodate increased usage by leveraging the allocated resources accordingly. 

In a typical cloud environment, resources can generally be allocated in two ways: 

• A fixed configuration with predictable monthly costs that require manual adjustment

• A variable configuration that grows as needed with an unknown future cost  

During Black Friday, implementing a combination of cost ceilings and manual adjustment is a common strategy. However, this approach requires a specialised DevOps team on hand and is notoriously difficult to gauge.

Making optimal use of the infrastructure available by fine-tuning what cloud services are used and creating more app instances to deal with the incoming traffic is another approach. Recreating infrastructure and configurations requires an in-depth knowledge of a cloud vendor's offering - differing by vendor and proprietary services and features. It is also a costly endeavour, both in time, complexity and the running of additional infrastructure. 

The Divio platform addresses scalability through a combination of technologies

The Divio Mirrors feature for web app mirroring makes cloning and creating new app instances easy and without the overhead of increased complexity of cloud management. One app code base can serve multiple instances, each with its own isolated cloud resources and predictable cost. By cloning, or mirroring, app instances can serve different needs and traffic sources.

Coupled with Mirrors, the Divio platform uses an intelligent approach to allocating cloud resources which is especially well suited to sudden bursts of traffic.

In a traditional cloud environment, a load balancer distributes requests to existing instances of a web app, distributing the load. Divio couples a load balancer with an intelligent app controller that can create new instances of an app, or a mirror, as necessary whilst working within the parameters of the cost plan - ensuring performance, with a predictable cost that can scale up to meet demand as required.

2. Performance through infrastructure - picking the right technology ahead of time

App performance optimisation is easy to defer during normal traffic loads and only comes into sharp focus during exceptional traffic peaks. Investing time in searching for incremental performance gains tends not to give immediate and apparent benefits yet becomes crucial during Black Friday.

Using the wrong cloud technology or not having the right technology in place is a common mistake and cloud storage is especially susceptible.

Most apps depend on cloud storage buckets to contain images and other metadata, differing by vendor in costs and types. Getting the setup wrong can lead to a poor experience for users and unexpected costs, especially where the storage is not intended to be heavily user-facing.

Putting the right choice of cloud storage behind a cache or CDN (Content Delivery Network) increases performance by moving assets physically closer to the user and also keeps costs to a minimum by reducing unnecessary calls to a storage medium. 

A CDN also plays an important role in adding a layer of security, typically being able to exclude malicious traffic from ever reaching the cloud infrastructure directly.

Another storage medium type is a database - and almost all apps depend on databases in some form to provide helpful functionality, such as product data or user properties. Optimising a database configuration can play an important role in overall app performance, given most app usage will involve an interaction with the database.

The Divio platform makes the best-fit technology choices for you

The Divio platform performs many optimisations behind the scenes, automatically configuring and provisioning the right storage technology and deploying a CDN during the app provisioning process. Media assets are then transparently delivered through the CDN without requiring any additional configuration or considerations.

The database provisioned by Divio is fully managed, meaning the database is constantly monitored, with scaling and optimisation of data performed automatically. Data sharding, a technique for distributing the load across a number of databases, is also employed as necessary when consumption grows.

3. Monitoring and performance analysis - making insights available and actionable

Proactively monitoring app performance and behaviour is a typical hygiene task but is easily overlooked during day-to-day development and app management. A leading cause of runaway cloud costs is often attributed to misbehaving apps - over-consuming resources or overly intensive computing operations that become especially costly during Black Friday. 

The task of monitoring and analysis typically sits within the DevOps function, feeding back insights to development teams which can act upon performance bottlenecks. Making sense of the data being captured and making it actionable for development teams is essential to unlocking the value within. 

Setting up performance monitoring tooling to give easy access to insights is common practice but often comes with additional costs and overheads attached. Introducing performance tooling can even inversely affect performance, especially when dealing with large volumes of logs and data during Black Friday. Digesting and processing data comes at the expense of cloud resources, which could otherwise be spent servicing app instances.

The Divio platform turns app usage data into understandable insights

The Metrics feature is automatically configured and deployed during the app deployment process and, together with logging, provides an easy way to readily identify bottlenecks without needing to understand the underlying cloud infrastructure. 

It uses a variety of data points to ascertain app performance and the services an app is using to develop a simplified and actionable overview which abstracts away cloud technologies.

As a component of the Divio platform, using Metrics does not come at the expense of the resources allocated for the app and is transparently connected to each app environment - such as test, staging or live environments. 

4. Cost management - keeping costs predictable and upfront

Black Friday can be a fraught time for allowing cloud costs to run away, as reflected by cloud vendors typically reporting their highest revenues during the Q4 shopping period. 

• Poorly configured scaling is a particular source of uncertainty - while apps remain available and responsive, instances and cloud consumption can grow uncontrollably, leading to surprise bills over the period. Miscreants can also abuse scaling up to rapidly spiral cloud costs.

• Over-provisioning is another common mistake, leading to an abundance of cloud resources being available and generating costs but potentially underused.

• Comparing cloud vendor costs makes calculating cloud costs notoriously difficult. The proprietary technologies being used make direct comparisons largely inaccurate and unfeasible. 

The Divio platform makes costs predictable and understandable

Apps that are deployed on the Divio platform always reside within a subscription with an upfront and predictable monthly cost.

The three main considerations of a subscription are:

1. Location

2. Memory 

3. Number of instances

These are dedicated entirely to the app being run and not spent on other services that are provisioned, such as a database.

The notion of an instance is a relative term rather than a cloud vendor-specific value. The Divio platform determines which cloud vendor computing type is best suited to the task at hand - serving requests rapidly and maintaining consistent performance. 

Deciding what resources an app needs is easy to ascertain from the Metrics feature. If an app consumes its allocated resources, more instances or memory can be added dynamically, without the need to understand cloud vendor-specific technologies.

The Divio platform can intelligently grow to accommodate Black Friday and other unforeseen surges by orchestrating multiple cloud vendors to provide burstable resources, delivering computing resources to the instances it manages.

A worry-free Black Friday, Cyber Monday and beyond

Divio has built a wealth of experience in dealing with unpredictable traffic surges, notably appearing on the popular Hacker News website multiple times and managing web apps that have been at the forefront of Black Friday traffic surges. 

By using automated platform intelligence, the underlying cloud vendors that Divio orchestrates can effectively deal with unprecedented volumes of unexpected traffic - scaling up infrastructure, using the right technology, leveraging data analysis and offering predictable pricing.

Talk to us to learn more about how we can eliminate Black Friday infrastructure worries.

Join the conversation! Follow our LinkedIn and X/Twitter communities for exclusive insights and to stay up to date with our latest blog posts.