Daniel Nashed's Blog

Weekend research project - Splunk

Did anyone look into Splunk integration before?
I am looking into how I could put e-mail data into Splunk in the best structured way.
The e-mail formats I found in the data definitions for sendmail, postfix & co did not really convince me.

JSON looks like the perfect format

Getting JSON data into Splunk using a raw TCP socket and also the newer HTTP Event Collector isn't rocket science.

• The HTTP Event Collector works well with CURL and LibCurl.
• The TCP socket allows to send raw data -- for example with netcat on Linux or just a raw TCP/IP connection from a C application..

Most customers use the classical integration parsing text logs.
It would make much more sense to send structured data from Domino to Splunk and to do more with the message data.

Did anyone in the community look into integrating with Splunk?
I would be interested to know, before diving into data extraction and creating JSON data for e-mail first.
But this isn't just about e-mail. Using their standard interfaces, you can put any data into Splunk.
-- Daniel

Run Splunk on Docker

Splunk is very easy to install using their Docker container.
And they offer a free server, because they license based on indexed volume per day.
For a test server, the free version is great!
This is a perfect test environment. Only figuring out where to put the PEM based certificate look a moment.
Here is an example run statement:

docker run --name splunk --hostname splunk.nashcom.lab -p 8000:8000 -p 8088:8088 -p 9999:9999 \
              -e "SPLUNK_HTTP_ENABLESSL=true" \
              -e "SPLUNK_HTTP_ENABLESSL_CERT=/opt/splunk/etc/cert.pem" \
              -e "SPLUNK_HTTP_ENABLESSL_PRIVKEY=/opt/splunk/etc/key.pem" \
              -e "SPLUNK_PASSWORD=splunk-secret-password" \
              -e "SPLUNK_START_ARGS=--accept-license" \
              -e "SPLUNK_LICENSE_URI=Free" \
              --stop-timeout=60 \
              -v /local/splunk/var:/opt/splunk/var \
              -v /local/splunk/etc:/opt/splunk/etc \
              -it splunk/splunk:latest


Getting data into Splunk

• Comments [0]