Loggly uses a search language for searching and filtering event results. The search language is accessed by the search command in the shell, and the /search/ endpoint in the APIs. Loggly can also return facet data for a search, which is used by the graph command in the shell, and the /facets/ endpoint in the APIs.
By default, all searches are done on the last hour's worth of data across an entire account's set of inputs and devices.
A query is broken up into terms and operators. Terms are defined as single terms or phrases which occur in your events. Phrases are defined as single terms surrounded by double quotes.
An example of a single term is:
hoover@example> search life
An example of a phrase is:
hoover@example> search "loathe it or ignore it"
Multiple terms can be combined together with boolean operators to form a more complex queries. Here's an example of a boolean operator combined with a phrase and single term:
hoover@example> search "marvin.jpg" AND 404
You can also use wildcards in the searches, but you should limit them to suffix wildcarding. The beavers that run our indexers are angered by the use of prefixed searches. Here's an example of searching for either 'jpg' or 'jpeg':
hoover@example> search jp*g AND 404
If you want to search for things ending in '.jpg', just search for it like so:
hoover@example> search '.jpg'
You can even do partial matches of IP addresses with a wildcard search:
hoover@example> search 10.0.*
You can limit a search by IP, where <ip_address> is the IP address of the device that sent the data to Loggly:
hoover@example> search ip:<ip_address>
You can also limit a search by device name, where <device_name> is the name you gave the device in the Loggly Interface.
hoover@example> search device:<device name>
Things to be aware of
Couple of important things that you should be aware of with regards to the search language
- Does not support case-sensitive searches. Everything is case-insensitive.
- Punctuations are not search-able. For example, '[' or ']' are not indexed, so a search for: '[error]' is the same as a search for 'error'
- Right now, we break on case-changes or changes from numerical to alpha characters. So, a search for 'www' will find results containing 'www111' or 'www112ww'.
Loggly supports limiting event results by fields associated with a given event. The fields available for non-structured data are:
- the device's IP address (or name)
- input name
Fields can be used in a search using the field name followed by a colon (:) and then the desired field value. Searches can contain a combination of field queries and regular terms:
hoover@example> search marvin.jpg AND inputname:webinput
When searching structured data (JSON), you must include the field names.
Boolean operators allow terms to be combined through logic operators. Loggly supports AND, ”+”, OR, NOT and ”-” as boolean operators. Note: Boolean operators must be ALL CAPS and + is short for AND and - is short for NOT.
The boolean operator 'AND' is assumed if you enter two separate terms. The following searches return the same results:
hoover@example> search foo bar
hoover@example> search foo AND bar
You can combine this with a phrase search. To search for both “apache request” and “apache” use the query:
hoover@example> search "apache request" apache
Alternately, you can search for one or the other:
hoover@example> search "apache request" OR apache
Loggly supports using parentheses to group clauses which form sub queries. To search for either “request” or “apache” and “404” use the query:
hoover@example> search (request OR apache) AND 404
You can even throw fields into the mix to further refine your results:
hoover@example> search (request OR apache) AND 404 AND inputname:webinput
Events stored inside Loggly contain a timestamp which is placed on the event when it arrives in our system. Searches done on Loggly default to the last hour. Timestamps in the shell are entered via the context bar at the top of the shell, or by entering them via the 'set' command. Timestamps in the APIs are entered via the 'from' or 'until' parameters.
Note: Time strings must not include white spaces or rabbits.
Loggly's search supports both relative time and absolute time. Here's an example for setting the default relative time range used for the last day in the shell, and then conducting a search across that time range:
hoover@example> set from NOW-1DAY
hoover@example> set until NOW
hoover@example> search 404
Note: Relative time ranges define a sliding window of time. Results using relative time ranges will vary as the times in which you are searching them are also changing. See arrow of time for more information on this subject.
Time ranges can also be defined as absolute times. Here's an example of setting from to an absolute timestamp defined in UTC format:
hoover@example> set from 2010-05-10T12:43:12
Note: Make sure you do not omit the T in the time, and leave any timezones or timezone modifiers off. Loggly assumes all timezone searches are UTC.
The basic timestamp syntax supports addition, subtraction and rounding. '+' and '-' denote addition and subtraction of times, while '/' denotes a rounding function. You can use this syntax to combine both relative and absolute time strings.
Now would probably be a good time to start throwing out a few examples. Remember, these times can be entered in the 'from' or 'until' text boxes at the top of the shell, set at the command line in the shell using the 'set' command, or passed as a parameter in the API using the 'from' or 'until' keys.
Rounded to the hour from right now:
Two years ago from right now:
1 day ago from now:
12 minutes forward from April 3rd, 2010 @ 12:34:44 in the afternoon:
Here's a complete list of all the time modifiers. Note: There is no 'WEEK' modifier!
YEAR, YEARS MONTH, MONTHS DAY, DAYS HOUR, HOURS MINUTE, MINUTES SECOND, SECONDS MILLI, MILLIS, MILLISECOND, MILLISECONDS