Loggly keeps the Speedway Motors e-commerce site in top racing form

Loggly has pulled our error reporting out of the dark ages and become an integral component of our e-commerce solution. It’s given us the confidence to release new external customer-facing applications and web services and know that we can react to potential issues instantly as they arise

Aaron Remaklus Web Development Manager, Speedway Motors

Highlights:

• Delivers dramatically faster responses to operational issues since adopting Loggly
• Successfully launched new platform for transactional website with more than one million pages
• Protects revenue by proactively monitoring application logs

Challenge

Speedway Motors operates a complex e-commerce site, with millions of pages serving up valuable information and how-to’s, as well as a catalog with tens of thousands of products for auto racing and performance enthusiasts. Speedway’s team of web developers is responsible not only for managing all aspects of the site—interfacing with data sources, integrating third-party APIs, and building presentation logic and user experience—but also for troubleshooting operational problems, 24×7.

“We process a lot of orders through the website every day, both in the United States and internationally,” says Aaron Remaklus, web development manager at Speedway. “Also, we look at the customer experience as an important way we can stand apart from our competitors. With so much money at stake, we can’t afford to not know when an issue could be affecting customers.”

In March 2014, Speedway was preparing to replace its old website with a custom-built, distributed e-commerce platform. The new platform would deliver huge functionality improvements, but the Speedway team also knew that it needed to improve its approach to log management. “One thing we had never done well was log errors,” Remaklus explains. “We did not have a central place to see errors that were occurring or to analyze them. As a result, we sometimes did not become aware of operational issues until our users reported them – a far from ideal situation.”

Speedway was determined to implement a proactive system of monitoring its logs, alerting the on-call web developer about potential issues, and analyzing log data to determine root causes. “Our new website was the most significant project our team has done. We simply would not have been able to go live without having a log management solution in place.”

Why Loggly?

As the Speedway team prepared to launch the new site, they started a Loggly trial and evaluated the service in its testing environment, using it to debug code and analyze errors that the team intentionally generated during load testing. The team chose Loggly over several other potential solutions because of its:

• Ease of setup: Speedway was able to build a quick prototype with Loggly that met all of its requirements.
• API: According to Chris Stevenson, one of the web developers who configured Loggly, “We have worked with quite a few third-party APIs, and we found the Loggly API to be noticeably easier to work with.”
• Ease of use: “Overall, we enjoy using Loggly,” Remaklus reports. “We found it easy to create searches, find what we were looking for, and identify problems.”
• Convenience: As a cloud-based solution, Loggly would not require Speedway to maintain additional code or infrastructure.
• Integration with PagerDuty: Speedway wanted to be sure that the right person was notified as soon as certain conditions occurred. “The combination of Loggly and PagerDuty works very well.”

Solution

When the new Speedway Motors site launched in June 2014, Loggly was there to help. All of the site’s internal services log any internal errors or exceptions that occur and send that data to Loggly as JSON. This includes exceptions from third-party services. The logs include a JSON environment field, so that Speedway can aggregate information from all of its environments (development, testing, and production) while limiting alerts to the production site. “The technical aspects of sending logs to Loggly is easy,” says Matt Stubblefield, another member of the Speedway web development team. “The hardest part was figuring out what to log.”

When Speedway first launched the new site, the team took the approach of logging as much data as possible. As they gained experience with the site’s behavior and the capabilities of Loggly, they have made modifications, cutting back logging in certain areas and pushing new information as additional JSON fields.

Proactive Alerting and One-click Access from PagerDuty Get Problems Solved Faster

Remaklus and his team set up saved searches that trigger alerts when events reach pre-set thresholds, for example:

• Failure for product or navigation pages to display properly
• Login or registration failures
• Issues with the site’s page caching service, which could result in slow page loads for visitors
• Timeouts and other exceptions with key third-party services, such as shipping quotes, which factor into the buying decision for a visitor purchasing 100 pounds of auto parts
• Errors or exceptions at any step in the checkout process

All alerts are configured to hit a single PagerDuty endpoint, which notifies the on-call team member. Similarly, Speedway uses AWS CloudWatch to monitor events like spikes in CPU utilization, failed health checks, or auto scaling issues. AWS CloudWatch alerts are also forwarded to PagerDuty.

Speedway team members have one-click access to Loggly within the alert, so they can immediately begin reviewing the relevant logs, determining how many visitors were affected, and finding the root cause. Managing issues in Jira helps them close the loop. “I have never had any problems identifying the source of an issue when I had the logs in Loggly,” Remaklus comments.

Dashboards Provide a Broader View

The Speedway team has begun to use dashboards as a way to monitor the site and plan future development activities. For example, the team tracks potential unsafe requests. In addition, analyzing bot traffic helped Speedway to surface some old URLs of which they weren’t aware and set appropriate redirects. “We have some platform enhancements coming up, and I think Loggly will be instrumental in helping us plan for them,” Remaklus notes.

Loggly Value: A Better Night’s Sleep

With Loggly serving as a catch-all to alert Remaklus and his team to anything that could have an impact on revenue and the customer experience, the new Speedway Motors site has had great success. “I wasn’t expecting to feel so comfortable about being responsible for an application that runs 24x7x365,” Remaklus concludes. “Loggly gives me peace of mind and the ability to sleep at night.”

[gview file=”https://www.loggly.com/wp-content/uploads/2016/09/CS-SPEEDWAY.pdf”]