Building an Unmeltable Architecture at Scale: Loggly Session at AWS re:Invent

Tomorrow afternoon (Friday) at 1:30, Philip O’Toole and I will be giving a presentation at AWS re:Invent. We’ll be covering a lot of ground, but the gist is that we’ll be giving a technical architect’s case study of how Loggly has employed the latest social-media-scale technologies as the backbone of ingestion processing for our multi-tenant, geo-distributed, and real-time log management system.

Cloud-based log management at scale has a lot of challenges:

• A massive stream of incoming events with distinct bursts from customers that can last several hours

• Every log has the potential to be the critical one; you can’t afford to drop a single logged event

• A scalable framework for analysis is a must

• Near real-time indexing and time series index management are fundamental to log management use cases

With lessons learned from our first generation service under our belts, we designed Gen2 Loggly to leverage a number of AWS services including Amazon Route 53 DNS with heartbeat and latency-based routing, multi-region VPCs, Elastic Load Balancing, the Amazon Relational Database Service, and a number of proactive and reactive approaches to scaling our computational and indexing capacity.

If you’re at AWS re:Invent, be sure to attend our session, and reserve your seat now as it’s in the top 25% selected will 800+ registrations and may fill up. Stop by our booth (#821) as well; we’d love to hear your story and answer questions you have. Philip and I will be there along with our engineers, product management, and marketing team… who have some fun new swag for all of you.   If you can’t make the show, we’ll be sure to post our slides up on SlideShare.

The Loggly and SolarWinds trademarks, service marks, and logos are the exclusive property of SolarWinds Worldwide, LLC or its affiliates. All other trademarks are the property of their respective owners.