Logging Challenges and Logging in the Cloud - PodCast
I was invited as a guest to the CloudChaser podcast with Matt Grant.
We talked about a number of interesting topics related to logging, cloud, and security.
Log Management Challenges
We discussed a number of log management challenges from log generation to security in the cloud. Here is a brief list of topics we talked about:
- We first touched upon some issues with log file generation. I am discussing the lack of logging guidelines and the problems that brings with it.
- How are logs analyzed? One of the problems it that it should really be the application owners that look at their logs. From a security point of view, security analysts should look at the overall picture. But they should not be the only ones looking at those logs. It’s impossible for them to understand all the logs on n intimate level.
- Yet another problem is understanding the logs. Visualization is an interesting way of addressing that issue. Especially for reporting and exploration or discovery.
- Large-scale log storage seems to be a problem. Is it? Make sure you setup use-case driven retention policies!
We touched upon a number of other topics. Here is a short list:
- It seems that users are moving more and more into the application layer to collect logs. It’s not just the infrastructure layer anymore!
- Availability, performance, etc. can be a great way of selling your log management budget instead of using security as a selling point.
- Obviously we talked about Logging as a Service and Loggly in specific. A lot of logs are in the cloud or are being moved into the cloud ;)
- Security and regulatory concerns for logging in the cloud are always a fun topic. We discuss this briefly. The upshot is that it often isn’t a show stopper!
But hey, listen yourself!
Ninja 22 Feb, 2011 02:33am
To respond to both, within the health vertical especially (Talking about 2/3 Class medical assets) sensitive information can easily find it’s way into the logs. The SIEM is the key(s) to the castle. the log data with addition to the access and authorization is severely mistaken if you ask me. Older legacy apps routinely place sensitive information as the developers at the time of coding had no intention for that data to ever leave the system. Routinely doing asset discovery, one needs to identify with the owner/sec team/business and determine data classification prior to storage in the “Cloud” or log review by others. Huge challenge :)
Kamal Govindaswamy 18 Dec, 2010 01:19am
Great podcast!
Just have a comment on security or privacy concerns around sending logs to the cloud… I couldn’t agree more with what Raffy said. At least in the area of security logs, I haven’t seen any use cases that require one to be logging any sensitive data. The closest to sensitive data that one may be logging is the userid which one might argue can be considered Personally Identifiable Information (PII) depending on the construct of the userid (userid being a combination of first-name and last-name for example) but once again security logs related to user activities are meaningless if they don’t contain userids.
I don’t see any concern as long as there are adequate access controls to who can view the logs. And I believe those access controls can be established in the cloud just as well as in the traditional infrastructure.