I’ve recently been researching Splunk and have been impressed with its power, flexibility, and ease of use. This blog is not intended to be a step-by-step tutorial, but rather is aimed to show some initial findings, overview one way to integrate Splunk with DNN, and paint the picture of some potential use cases.

So What is Splunk?
If you don’t already know what Splunk is, Splunk is a software company based in San Francisco that produces software for searching, monitoring, and analyzing machine generated big data via a web style interface. Splunk’s software helps organizations with operational intelligence, log management, application management, enterprise security and compliance.

Installing Splunk was simple and after clicking around a little while it was evident that Splunk is an intuitive software. From a UI standpoint, it makes logical sense and the flow is easy to understand. And it didn’t take long to see and understand how powerful it is.

As you may imagine, I began to wonder if and how I could integrate Splunk with DNN.

DNN + Splunk: One Way to Connect the Two
One of Splunk’s powerful features is that it can literally suck in all types, styles, and formats of data. This data can be machine data, log files, or even data from a REST API. There are several mechanisms for getting data into Splunk, but for this scenario, DNN’s web API implementation makes this an easy fit. On the DNN side, a developer can easily create a custom module using web services to expose any DNN data on an endpoint, which Splunk can then access. If you’d like to go the custom module route, check out my other blog series on module development. However, I did not write a custom module to test the integration.

For my initial investigation into Splunk I chose to use DNN Sharp’s API Endpoint module as it allows easy configuration of end points. Splunk is architected to consume any type of data and then it makes that data extremely easy to search, create visualizations and/or alerts with. These searches, visualizations, and alerts can be very basic or very complex in nature.

Another thing to note is that Splunk is architected to do this at scale and can easily parse enormous amounts of data. For example, every time you drink from a Coca-Cola “Freestyle” machine at a fast food restaurant, the data from your drink selection is logged and Splunk helps analyze the data, denote trends, and sends alerts. So yes, those Coke machines (all across the world) are connected IOT devices and Coke is a Splunk customer. See how Coke is using Splunk in the Splunk Conf 2014 Keynote replay session. Imagine how much data that is on a global scaled --> Splunk is helping Coke make sense of it.

Side note: Check out the blog I wrote on using Particle & Splunk to monitor temperature

So, my first goal was simple: see if I could get data from DNN into Splunk.

Sticking along the thought process of “data logs” I figured why not expose the DNN event log on an endpoint and see what I could make happen. Obviously, the event log may not be the best use case as site administrators can clear logs or processes to automatically clear logs sometimes exist. However, for this initial test it is a good candidate. To get the event log data on an end point I used the DNN Sharp API Endpoint module to make a SQL query on the event log view and return it as JSON.

With the event log now sitting out there as JSON on a DNN end point now all I needed to do was get it into Splunk…

Getting REST Data Into Splunk
The Splunk side of this configuration only took a few minutes to configure and keep in mind I’m no Splunk guru (read, it’s easy!). Splunk is similar to DNN in that it’s extensible. Splunk extensions can be found on the Apps and Ad-Ons sections of the Splunk website. I tell you this because ultimately, I followed a blog by Damien Dallimore on getting REST data into Splunk which used a modular input extension and that was all it took. I simply completed the required fields in the Splunk REST Modular Input as shown below.

I chose to poll the data every 60 seconds. With this information inputted I clicked save and returned to the Data Inputs screen of Splunk and chose my newly created data source.

BOOM! I was seeing DNN event log info in Splunk!

Searching, Visualizations, & Alerts in Splunk
With data in Splunk now I needed to proceed to using Splunk to make sense of the data. Splunk’s searching functionality makes it very easy to search for, well... anything you'd like. I’m not yet knowledgeable enough to fully explain all the capabilities, but what I can easily see is that you can select your data source, click on keywords, add them to the source's search criteria and set your desired timeframe for the search. It’s feels as if you have a Google search bar and all your searches are performed on your data source and intellisense & syntax highlighting for your search are provided too!

Once you have a search returning data you can then create visualizations or alerts. And yes, there are tons of visualizations provided by Splunk. These visualizations can be saved as reports or live as “panels” that reside on dashboards. Dashboards can have as many panels as you want and you can have multiple dashboards if you like. Also, you can easily embed these panels into DNN or any other location by clicking the “convert to HTML” link that each panel has. Being able to display this info anywhere you like is a neat feature. Are your mental light bulbs turning on yet?

So, I created a few visualizations based on event log data that was available. I created a number-based-visualization to show a large number that represented a count of 404 errors, a line graph showing the number of failed logins, and a chart showing the 404’s over time. So, in just minutes Splunk was already helping me understand that I have some issues going on with one of my sites. I believe one reason for the 404's is that I've renamed some pages that I think bots are targeting trying to register. Anyways, I've got work to do... don't judge!

Opening Up Possibilities
Now you may be looking at this and thinking to yourself, yeah this is neat, but I could create a custom module to make something similar to this happen. And you would be correct, but keep in mind the potential use cases, flexibility, and scalability of Splunk in comparison to a custom module. You could easily have all your customers as data sources and create dashboards to help you (and your customers) quickly understand what’s going on with your customer's applications. You could also do data mashups of data from a DNN website/web app, some IOT device out in space, and any other data source you can think of to provide valuable insight. And again, Splunk has no problem doing this with massive amounts of data.

With just a little research into Splunk it didn’t take long to get my mind spinning with all the possibilities within DNN and beyond. Think about your current DNN use cases, requirements of your customers, and the exploding IOT market and you’ll soon see the light.

Here are some ideas I had right off the bat:

• Dashboards that visually communicate mission critical data to your client’s management tier
  • Think a specific page of dashboards only visible to your client’s leadership showing application performance dashboards.
  • Think about posting monitors in your client’s offices (or your office) showing performance or usability metrics 
  • Dashboard Examples:
    • Failed Logins – to denote potential bot attacks 
    • New Users – to denote growth or potential bot attacks 
    • 404’s – to denote when pages aren’t found, SEO issues, or other concerns 
    • Etc., etc., etc. 
• Alerts – Any type of alert you can imagine based on the data you expose 
• Proactive Client Consulting 
  • If you represent a web firm you can shift your business from being reactive to proactive by monitoring your client’s data and then reaching out to them when trends are noticed
    • Consider your SEO person suggesting redirects be put in place when 404’s for a specific page continuously occur
    • Consider shutting down account creation when a rapid account creation occurs over a short period of time in the scenario of bot registration attacks 
    • Have you developed custom modules for your clients? You could create visualizations and alerts from the data of those modules to help provide business intelligence & insights to your clients 
• IOT 
  • As I mentioned earlier I also wrote a blog on using Particle & Splunk to monitor temperature. Check it out for more potential use cases. 
  • Many clients are seeking or will be seeking IOT solutions in the future. If you are working with connected devices you could easily setup web services in DNN to log data from these microcontrollers, sensors, motors, or actuators. If you’re logging data into DNN (or any other location) you can easily feed that data to Splunk to parse and analyze the data. Presenting data from IOT devices coupled with data from your DNN instance could potentially provide a holistic picture for your client’s business and deliver critical insights to them.
    • DNN firms like nvisionative are already building IOT based solutions that integrate with DNN. Splunk opens even more possibilities for organizations like this.

Splunk FTW!
As you can see the power and flexibility Splunk provides is really nice. I believe Splunk could be a game-changer especially for those with large amounts of data to parse, anybody in the IOT space, and much more. I hope this blog has provided you with an introductory glimpse into some of the capabilities of Splunk and even got you thinking of potential ways to integrate Splunk into your applications or customer's environments. I am still learning about it and hope you will too. I know that I'm just scratching the surface here in my initial findings.

Find out more about Splunk at http://www.Splunk.com

In the past few years I’ve been getting into IOT. You may have seen tweets or blog about the Tech-Turkey project I’ve been working on or flame throwing pumpkins at Halloween. I’ve learned and used Arduinos, Raspberry Pi’s, and Particle Photons and Electrons. It has been fun to learn more and get into the connected world… the internet of things!

I’ve been keeping ServoCity in business and even recently worked to get a custom PCB created. Every step of the way I’ve been learning different things and realizing just how much more there is to learn. Recently I’ve started learning more about Splunk.

What is Splunk
If you don’t already know what Splunk is, Splunk is a software company based in San Francisco that produces software for searching, monitoring, and analyzing machine generated big data via a web style interface. Splunk’s software helps organizations with operational intelligence, log management, application management, enterprise security and compliance. 

Side note: In my first exploration into Splunk I wrote a blog about using Splunk with DNN that may interest you.

Particle & Splunk
DNN is a web application, but what if I wanted to get data from an IOT device? That’s when we call on Particle. If you’re not familiar with Particle, it makes it really easy to bring real world objects online. Particle is one of my favorite IOT platforms. It makes awesome microcontrollers, provides a nice IDE, has awesome documentation, and a great community. Connecting to Particle’s cloud is straight forward and even southerners can do it! See my presentation at our user group on DNN & Particle.

If you’re not familiar with Splunk, it makes it really easy to pull in data (machine data) and make sense of it. I’m talking about parsing vast amounts of data, creating visualizations and/or alerts and making it simple to understand. Even southerners can use it too!

Both Particle and Splunk are industry leaders and have some really big names behind their companies and as clients of their companies.

So why not bring Particle & Splunk together?

Reading Temperature with Particle
To use the awesomeness that both solutions bring us we’ll first need to read the temperature and post it to a webservice. Here again, Particle makes this easy. I used a basic temperature reader in a bread board layout for this experiment.

Then, in Particle’s IDE I used the basic tutorial level code to read an analog value and post it to a Particle cloud variable. Cloud variables are accessible via web services. That is, I can make a GET request and parse the JSON object to get the data. Epic.

And all you have to do is “flash” the firmware to the device and then hit the API endpoint. In my case I accessed the service and saw the following JSON

Now we were cooking with oil! The next step was to get this data into Splunk.

Getting Particle’s RESTful Data Into Splunk
Getting RESTful data into Splunk is really straightforward thanks to Splunk’s extensibility. Splunk has an extensions gallery that can be found on the Apps and Ad-Ons sections of the Splunk website. I tell you this because ultimately, I followed a blog by Damien Dallimore on getting REST data into Splunk which used a modular input extension and that was all it took. I simply completed the required fields in the Splunk REST Modular Input as shown below.

After clicking save, the data from my Particle temperature reader was showing up in Splunk!

Creating Dashboards from the Particle Data
Once data shows up in Splunk you can literally perform any search query you want on the data and create/configure dashboards, panels, reports, alerts and more. Splunk is very powerful in this regard and scale to infinity. However, for this scenario I just wanted log the temperature over time from one device, as well as the temperature’s highest, lowest, and average. Splunk, again, made this very simple.

After clicking on the “result” field I created some visualizations and voila! Out popped some neat dashboards showing all my data in a way that’s easy to understand.

If you are like me, you kind of want to see things in action. So for those of you like me who are visual learners, here’s a quick video of the solution in action.

An IOT Combination That’s Hard to Beat!
As you can see, both solutions are awesome and the opportunities are endless. Consider the possibilities here… Particle is easy to deploy and post data to the net and Splunk can easily connect, suck in data, and bring instant insights. The more data you give to Splunk the more knowledge you’re going to have. Splunk can handle this at scale too… I mean massive scale. Why not connect thousands of devices and pump all the data into Splunk and tune it to your liking! I believe that’s what they refer to as operational intelligence 😊 Now my mind is spinning with possibilities. Is yours?

The Central High School Eagles of Pageland, South Carolina have a rich tradition and history of success. A few years ago my friend, Jason Fararooei, a video producer from the Charlotte area, took a liking to the program. Over the years, Jason has made some really great videos for the eagles. If you haven’t seen them then check out 3:17 and the Eagle Tribute Video.

With so much recent transition going on at Central, we decided to make another video to try and create energy and enthusiasm around the program. Our hope is that the new head Coach, Trent Usher, will get the program back to where it used to be.

We had quite the eventful weekend last weekend. If you read the “Big & J Hogs Hammer It and Pigs Dig It Helps Get Rid of Nuisance South Carolina Hogs” blog that posted on Monday then you are aware of the local farmer who had reached out to us to assist with his hog problem. Although we expected multiple hogs to come out on the first hunt we only ended up seeing one.

So we returned for another hunt a day or so later…

The hogs had stayed away for a day, but on day 2 they wiped out all the remaining corn that was saturated with Big & J Hog attractant. The farmer notified us of what the hogs had done overnight and so we knew we needed to be back down at the farm sooner than later.

After replenishing the corn, I went down to the farm on a solo hunt as my hunting partners were unable to come on this specific night. The farmer sat with me and we watched the corn pile for a while and were ready to handle business. However, nothing moved just after dark. We sat and strategized what we would do when certain hogs arrived, but nothing was moving. The farmer had to pack it in for the night so I remained on the gun watching the field.

Shortly after the farmer left 3 deer came out and grazed through the field. I watched them for a while in the scope. Then 2 more deer entered the field. Interestingly, the deer did not eat the corn that had the Big & J hog attractant on it (which is a good sign to me!). Eventually the deer exited the field into some nearby woods.

From Reading a Devotional to Shooting a Hog
I was reading a devotional on the bible app and I would stop every couple of minutes and scan the field. I’ve hunted hogs enough to know that the hunt can change in an instant because these hogs don’t hesitate too much when they come into a field and they move more quickly than you might expect. I read and scanned, read and scanned, and towards the end of the devotional I noticed a blob of heat on the corn! While I was reading, a group of hogs, 1 female and several piglets, had gotten out into the middle of the field.

I knew it was game time.

I got in the gun and watched this group for a few minutes. I scanned the edges looking to see if any more were nearby or entering the field. I didn’t see any sign of other hogs coming in so I continued to watch. I knew I was going to shoot the big one, but it was just a waiting game.

I don’t like to shoot in the middle of a white blob of heat because it’s hard to tell exactly what you’re aiming at and sometimes the piglets are taller than you think. Translation: I didn’t want to get a piglet and miss the big one so I waited on the right opportunity to present itself. I needed the big hog to separate herself far enough so that I could get a silhouette of her body and know where I was aiming.

While I watched them feed something funny happened. One of the piglets went behind the female and the larger female cut the piglet a flip! She kicked the piglet and it somersaulted backwards and when it landed it just got right back up and kept rooting. It was pretty funny. I couldn’t believe what I’d witnessed.

A few seconds later the large female advanced forward aggressively and this singled herself out. It was just the sight I was waiting for. I flipped the safety off and squeezed the trigger really slow. The Anderson Arms AM-10 308 that I have has a long trigger pull and in hopes of not flinching on my shot I always try to ensure the gun surprises me when it goes off. I hope for the smooth trigger pull. I put the cross hairs on this hogs shoulder and squeezed off.

The boom echoed through the field and down to the creek.

The large hog instantly fell and within a second the piglets scurried out of the field. Since the large hog was on the ground, my job shooting was essentially done. I waited a while and started loading up the truck.

Loading a Hog By Yourself Ain’t Easy
I took the shot at about 11:58 and with my hunting partners not around it was me… and well me… that had to load the hog up. When I got down to the hog I realized she was bigger than I thought. Getting her in the truck wouldn’t be as easy as it normally is when you have help.

Ultimately, I ended up dragging the hog to the side of the field and then walking up the bumper to the tailgate with one of the hog’s legs in my hand. When I got in the bed of the truck the weight of the hog was very heavy to hold on to so I had to essentially lay down on my stomach and grab the other leg with my other hand. With both legs in hand I then had to figure a way to stand up. It reminded me of a dead lift that we used to do in high school and college football except this was more awkward and off balance. If you would have seen me you would have laughed, but once I got my feet under me I was able to pull the hog in the truck using the tailgate as a lever. I hope that’s the last time I have to load a big hog up by myself!

And since there was no one there to take a pic of me and the hog I had to take a hog selfie!

It was a great hunt and yet another nuisance hog is in the freezer at the processor!

Do You Have Hog Problems?
If you have hog problems we’re happy to help. Learn more about how we are helping land owners and farmers with their hog problems on the SC Hog Removal page.

Another South Carolina Farmer With Nuisance Hog Problems
We’ve recently been in communications with another local farmer who’s crop were being demolished by hogs. On this specific farmer’s land, the hogs showing up and rooting his crop fields was a new occurrence. Frustrated and not exactly sure of how to solve this problem the farmer asked us how quickly we could help him out. Within a day we had game cameras set up and were getting recon on the hog’s pattern on this specific property.

Big & J Hog Products Help the Hunt
In this setup the area where the hogs were showing up was narrow in nature. The field makes kind of a point where the hogs have easy access and had been rooting. This meant we most likely wouldn’t get multiple shots and would need to get the hogs to the middle of this area of the field.

To coax the hogs into the middle of the field we used something that would be memorable for them, Big & J’s new Hog attractant products. We spread both Hogs-Hamer-It and Pigs-Dig-It on top of corn in the middle of this point in the field. And it didn’t take long before we had them coming in and loving what Big & J’s products had to offer!

Only One Hog Came Through
Due to the amount of damage we’d been seeing on this property we anticipated seeing several hogs, but on this hunt, it didn’t play out that way. The wind was not in our favor and was blowing pretty strong. We sat for a while and shot the breeze. Early in the night we had a deer that kept walking through the field and right around midnight we had a solo hog come in and go straight to the Big & J hog attractant marinated corn pile!

For us it’s rare to see a solo hog like this unless it was a really big male. So we waited thinking that more would eventually come out. And we waited and waited and waited. It seemed like forever, but it was probably around 10 minutes or so. Evidently the hog was there by itself. We decided to go ahead and pull the trigger because we didn’t want that one to get out of there before we could get a shot off and nothing else seemed to be showing up.

As you can see on the video below, the Anderson Arms 308 with Pulsar Trail XP50 made quick work of this hog. The hog flopped on the spot and our tracking job was easy! We loaded her up, took some pics, and took her to the processor.

Another nuisance South Carolina hog headed to the freezer.

Do You Have Hog Problems?
If you have hog problems we’re happy to help. Learn more about how we are helping land owners and farmers with their hog problems on the SC Hog Removal page.