Streaming data

Streaming data is almost always being generated, with a timestamp associated to each entry. Splunk's inherent ability to monitor and track data loaded from ever growing log files, or accept data as it arrives on a port, are critical pieces of functionality.

However, streaming data is no different than other data in that it's usefulness erodes, particularly at a detailed level. For instance, consider a firewall log.

In real time, Splunk will capture and index events written to a firewall log file. Normally, there will be many different activity events logged to Splunk in real time. However, many of those events are normal logging events noting activity occurring successfully.

As you consider your source data, its important to consider how long you want to retain data and/or how you would want to archive it. It is also important to understand if you need all the data from the source or only specific kinds of events.