Building a Highly Scalable Intake Form Application

In 2018, Empower Illinois started accepting applications for Illinois’ new Tax Credit Scholarship program. The program, which provides scholarships to low income families, was in extremely high demand.

When the scholarship application opened, over 65,000 families tried to simultaneously submit applications to reserve their place in the first come, first served program.

The huge influx of traffic caused the app to immediately crash. Tens of thousands of families were left staring at error screens, as they waited to see if their child would receive a scholarship. 

Empower originally relied on an off-the-shelf software product to accept scholarship applications. Unfortunately, the product didn’t offer the scalability that it advertised. It could not serve the tens of thousands of families trying to apply.

Without a way to accept applications, Empower turned to LaunchPad Lab to develop a custom solution that could handle over 100,000 applications per minute. 

Within two weeks, LaunchPad developed and launched a new scholarship application. The new application scaled to flawlessly accept every submission on a first come, first served basis.

Our team has learned a lot from developing highly scalable applications. When a form is part of a first come, first served program, you can expect extreme short-term surges in traffic. This type of scaling requires additional considerations beyond traditional scaling architectures. 

Here’s the way we approach the process. 

User Experience (UX) First

When your site is flooded with visitors, you have to have a perfect UX. Any mistakes or questions in the experience will cause visitors to immediately contact your customer support team. It’s unlikely that your support team will be able to answer all of those requests in that short period of time. 

Our design team spends as much time as possible perfecting the form. Every piece of text and input field counts. We look to make the form as simple as possible. Links and buttons should provide clear direction.

Once the initial form design is done, we conduct user tests to make sure that the form is easy to use. These tests can yield valuable feedback that we use to iterate and improve the form.

In addition to the visual design, we test the form in as many devices and browsers as possible. 

Depending on the application, we generally see a minimum of 40% of traffic coming from mobile devices. With so many different types of devices, we keep a number of extra testing phones in our office. We can also run a program to simulate any devices that we don’t have on-hand.

Web browser errors can be a major cause of support requests. Too often people forget to upgrade and use outdated browsers. Browser errors can prevent users from submitting the form. Anticipating these issues, we test the application on as many different browsers as possible.

Surge Architecture

The architecture of the application needs to be able to handle huge surges in traffic. We’ve developed an architecture that allows us to quickly rollout applications that can meet massive traffic loads. The architecture allows us to serve web forms that can handle millions of simultaneous visitors. We process submitted applications using a queuing system to ensure that all applications are saved without overwhelming the database.

Sometimes highly scalable applications only receive surges of traffic during certain periods. In this case, we can scale down the servers and other application resources to manage costs during non-peak times.

It’s important to choose a hosting environment for this architecture that can meet the scaling demands and is extremely reliable. The hosting environment that we trust for this type of architecture is Heroku. Heroku is a Platform as a Service (PaaS) that is part of Salesforce. Heroku allows us to quickly launch the services that we need to accommodate high traffic volumes. It also allows us to easily increase or decrease capacity on demand.

Load Testing

Once the application form and architecture is in place, we test to make sure that it can handle extreme web traffic loads. We perform load testing using a tool called Flood. Flood makes it relatively easy to setup large scale load testing. It also provides in depth reporting metrics on all tests, allowing you to fine tune performance with each test. 

As a general rule, we design load tests using four-times the anticipated peak load. This provides us with a large margin of safety, just in case traffic turns out to be higher than expected. 

The Future of Forms

We believe that companies of all types will start to recognize the value in having highly-scalable web forms. Without a way to quickly create scalable web forms, companies are forced to use the old process of emailing PDFs back and forth. But the time it takes to manually process PDFs is too costly for businesses. It requires a large number of employees to do basic manual work when they could be working on higher value activities. 

It’s time for businesses to adopt modern, scalable web forms to reduce the burden on customers and streamline their business processes.

×

Subscribe to Our Newsletter

If you made it this far, you should sign up for our monthly newsletter. Get our latest articles and insights delivered each month.

First Name*

Last Name*

Email*