New mailing list for Sideview Utils

1) There is finally a mailing list for Sideview Utils updates.  It’s been something people have been asking for for a long time, and it’s finally here.

So if you’ve been having trouble staying up with all the latest fixes, features, and documentation,  sign up now.  Sideview Utils gets updated every couple weeks on average, as you can see from the release notes, and the updates are pretty significant.

2) In other news since it’s friday and friday in Splunktonia is traditionally a day to celebrate ascii-art, you should note that Sideview Utils now has a saved search inside it by the name of “fishies”.   Run it when you get a chance…

Leave a comment

New Cooking with Sideview episodes

I did two videos last December and they’re really the best ones so far so check them out.

In the first, which is the fourth overall, I take a step back and give a broad overview of Sideview Utils. What it is, what it does, why it’s awesome and why you should use it. — Cooking With Sideview Episode 4 – Sideview Utils Overview”

And the other, which is the fifth overall, I go back to doing deep dives and we cover the Pulldown module. — Cooking with Sideview Episode 5 – the Pulldown module.

Both videos are about ten minutes long and very informative so check them out today.

Leave a comment

Fire drill around Splunk 5.0

We just updated all of our apps!  Why, you ask?

Well, Splunk 5.0 was released this morning, and despite having tested everything against earlier 5.0 builds,   when the GA version of Splunk 5.0 released this morning, we had a little problem!

As you (the Sideview user) know,  Sideview Utils and Sideview apps in general patch and extend quite a lot of functionality in Splunk.   And even though Splunk itself is also constantly changing things from release to release, we know what we’re doing enough so that we hardly ever step on eachother’s toes.

Well, 5.0 is a pretty big number and there was some very minor toe-stepping this morning.  What happened this morning is that Splunk changed something relatively minor about how saved search names are passed around in one area,  but they didn’t make the same change in another area.    I’ve already bored you so lets say no more.

What broke was that Sideview-style saved reports and saved searches would no longer load properly when you clicked the links in the navigation menus.   This broke in the Sideview Report Builder,  in Splunk for Cisco CDR, and in basically any sideview UI where you are saving searches and reports.   NOTE: I don’t think that anything in any non-Sideview-owned app broke, but if you see anything broken and particularly if it looks similar,  contact us.

SHORT VERSION:   If you upgrade to Splunk 5.0,  and you use the Sideview Report Builder,  or the Sideview Chart view,  or if you’re a user of ANY of our other apps,    you should at the same time upgrade Sideview Utils and your other Sideview apps to the latest versions available on this site.

Don’t let this dissuade you from upgrading to Splunk 5.0.  A) because it’s awesome,   and B) because upgrading apps these days is  a simple matter of going to “Manager > Apps > Install app from file”,  and checking the “upgrade app” checkbox.

 

Leave a comment

Sideview Utils 2.2 – even more awesome

This was a big release.   The Table and Multiplexer modules introduce a new Sideview mechanism that allows you to greatly expand the range of custom rendering.    I’ll be expanding on this as soon as I can, in the form of a new “Cooking with Sideview” video, so keep an eye out for that.

Until then, you can of course download  the app yourself and check out the examples and the docs.

Also many people seem to have missed this so remember that Sideview Utils is free for internal use as of several months ago!   Unless you’re developing a commercial app or distributing stuff to third parties the Internal Use Agreement should be all you need to get going.

Anyway, here’s why 2.2 is so cool:

Multiplexer::  Multiplexer allows you to custom-render search result rows, and use Splunk and Sideview modules to do that custom rendering.   This sounds kind of like nothing at all, but it’s very very amazing.      Picture being able to write a dashboard where for *EACH* row in your search results you can have a graph.  Or a graph and a table and some HTML.

You can also take a Multiplexer, along with the multiplexed modules, and put it downstream from a Pager module.  This allows your users to effectively page through 10,000 custom charts, each with custom drilldowns and custom headers.   If you understand what I’m saying your head may have exploded.    And the best thing about all this is that it doesn’t require a single line of custom code, let alone a Splunk “event_renderer”.     So if you thought you had to learn python or mako or javascript to do stuff even half as cool as this,  think again.

Table::   The Table module is extremely simple.  And when you need it to be ludicrously powerful, it can do that too.   Sensible defaults mean that it pretty much does the job of SimpleResultsTable but does it better.

Params like rowClass, rowStyle mean you can do a lot of simple changes like row background colors.

Params like hiddenFields and fields mean you have fields that are available in the drilldown data but not actually visible in the table rows.

Params like earliestTimeField, latestTimeField and durationField mean you can set up drilldowns on custom epochtime-valued fields.

And last but not least,   Table allows you to do the same Sideview-style templating that we just went over in the Multiplexer module.    Want to make some of your tablecells display as an icon instead of a piece of text?  no problem.   Want to have a pie chart in each row of your search results table?   No problem.   Want to do all this and still have a custom drilldown?  No problem.

And as always, the new release has a bunch of other more humdrum fixes and improvements.  Check out the release notes,  check out the app homepage, and check back later for another Cooking With Sideview episode about all this.  And thanks as always to the users who gave feedback and asked for things. =)

 

Leave a comment

Sideview Utils 2.1 released

 

A bunch of great stuff just came out in Sideview Utils 2.1

“Runtime Debug” mode in the Sideview Editor.   For the first time Splunk admins have a way to debug tricky form searches and inline drilldowns.   Are the $foo$ tokens not ending up quite where they’re supposed to?  Walk the module hierarchy in your view and inspect the live arguments and searches and timeranges as they’re flowing around in the living view.   Until now to get any real work done you’ve had to insert HTML modules into the actual view,  to debug out the keys you’re interested in,  like $search$ and $myPulldownName$.   Now you can work more quickly and less painfully.

The Lookup Updater.  Have you ever found it limiting that even Splunk admin users have no way to update an existing file-based lookup from Manager?   Well I put together a little prototype view with it’s own little controller that you can use to do just that.  Try it out and let me know what you think. 

Fixes to the core Splunk platform around handling of “All time” searches.  In the Splunk UI,  the lack of a timerange is implicitly the “all time” timerange.   Although in a way it’s  sort of a neat convention, there’s no denying that this creates some frustrating problems for Splunk admins and app developers.    Now however Sideview Utils 2.1 unveils an emerging convention whereby an “all”/”all” timerange is treated as an “explicit” all time timerange that triggers prepopulation, and the old no-timerange timerange still wont trigger prepopulation.   If you have no idea what I’m talking about that’s perfectly fine.    This is just a core improvement.

Short version:  In views that have the Sideview back button support turned on,   back button and forward buttons will now work even across “All Time” timeranges being selected. =)

Bugs.  There are always some.  We always fix them.

Enjoy!

 

Leave a comment

So you’d like to… convert a simplified XML dashboard to Sideview Utils

Intro: There are two flavors of simple XML,  the <dashboard> and the <form>.   With both flavors, in terms of interface functionality they are both just limited shorthand forms of the “advanced XML”, which I just mention to reassure you that the conversion process is relatively reliable and you won’t necessarily lose anything.   And once you’re in the advanced XML, the steps to upgrade to Sideview Utils modules are each relatively simple.

Why Would I Do This? Although the range of user interfaces that you can achieve in the <form> and <dashboard> views is often enough to do what you need, you often hit limitations.  As you get more use cases, more drilldown requests and more team members involved,  Splunk users often feel constrained by one or more of these many limitations.   Often users will first convert to Advanced XML, and then suffer at that level for quite some time before finding out about the Sideview Utils modules.   I wrote this post because I believe if you’re going to convert to advanced XML,  you should just jump right then and there to Sideview Utils.

Prerequisites: 

Sideview Utils has a page “Key Techniques > Overview of the Advanced XML”.    Even if you’ve read that page already, go back now and read it again.    To really be able to comfortably convert the actual syntax for individual modules you should probably read through that module’s documentation in Sideview Utils…  Nonetheless quite possibly you’re about to read on without having done so and that’s fine.

Here’s How:

Step 1>  Convert the view to advanced XML.  You do this by simply going to the simplified <dashboard> or <form> view in your browser and tacking on ?showsource=true   to the URL in your browser’s location bar.   Then on the resulting page scroll down to the very bottom.   You’ll see a big textarea full of XML.   Copy and paste this XML out into a text editor, or Notepad or vi or whatever floats your boat.  This is the corresponding advanced XML for that simplified XML view.

Step 2> Save the raw converted view in Splunk.  You can save it in etc/apps/search/local/data/ui/views/my_converted_view.xml,  and use the FreshMaker to reload from disk,  or if you don’t know what the heck I’m talking about,   from the Search app, go to “Manager > User Interface > Views > Create”,  and paste the XML into the big texarea and give it a name like “my_converted_view” .

Step 3> View the converted view in the Splunk UI.  This step is easy.   Just go to  http://host:port/app/search/my_converted_view.   You’ll see an exact copy of the dashboard or form you just converted.    This is just a sanity check.  On we go.  Now that we have a working base view in advanced XML we’ll convert to Sideview Utils.

Step 4> Put a SideviewUtils module into the view.

Put it at the top, just underneath AppBar and AccountBar. We need this because all Sideview modules use shared code from the “SideviewUtils” module, and that shared module also patches a lot of the code in the core Splunk systems.

<module name="SideviewUtils" layoutPanel="appHeader" />

(after this step and each succeeding step, you should verify afterwards that the view still loads properly, that you don’t have a typo in the XML, and that it functions as expected.  Paranoia is good here because having to retrace your steps is bad.)

Step 5> clean out all the noise.  Unfortunately Splunk’s simplified XML does a lot of strange things that make no sense, or that almost always make no sense.  This step is optional but it will make your view a lot shorter, a lot less nested, and that will mean it is easier to read.

– remove all “groupLabel” param tags.

– remove all “HiddenFieldPicker” modules, and de-nest all the modules that were contained within.

– “de-nest” all of the modules inside all “EnablePreview” modules.  In other words, for each EnablePreview module, make it’s one child,  into a sibling, and have the EnablePreview module be closed like this:

<module name="EnablePreview">
  <param name="enable">True</param>
  <param name="display">False</param>
</module>

– if you have modules nested inside any JobProgressIndicator modules, do the same with those – denest all those modules,  such that the child of JobProgressIndicator ends up being a following sibling instead.

– you can almost certainly remove all “allowTransformedFieldSelect” params.

– you can almost certainly remove all “ViewStateAdapter” modules and de-nest all the modules that were contained within.

– remove all “Gimp” modules.

Wow. We just removed quite a lot.    reload/refresh and test.  Make sure you still have well-formed XML.

OK.  Now we start with the good stuff.

Step 6> replace all “HiddenSearch” modules with “Search” modules. 

First just replace the module names. The param values in Simplified XML will be the same.

Next though, you’ll have to cut the module XML out of the view,  and move it deeper into the nesting,  so that it is below all of the form-field modules that contribute to the search.   It pays to keep your indentation clean although you don’t have to listen to me.

And leave the layoutPanel attribute and autoRun attribute on whatever module is left behind as the topmost <module>.

Step 7> Delete each and every “ConvertToIntention” module. 

This one will feel good.  After doing it, the view will mysteriously work just fine without them.   =)

NOTE: well ok fine, if you have some complex views with more than one SearchSelectLister modules, then you *might* need to replace all the lister modules with equivalents first.   Technically the listers *may* in some cases expect the intentions to still be there.

Step 8> replace all ExtendedFieldSearch modules with TextField

You’ll want to be a little familiar with TextField, and you’ll want to have read the docs pages in Sideview Utils.  However TextField is VASTLY SIMPLER than ExtendedFieldSearch so you’ll be fine.

For example, you would take this beast:

<module name="ExtendedFieldSearch">
  <param name="replacementMap">
    <param name="arg">
      <param name="series"/>
    </param>
  </param>
  <param name="field">sourcetype</param>
  <param name="q">splunkd</param>
  <param name="intention">
    <param name="name">stringreplace</param>
    <param name="arg">
      <param name="series">
        <param name="fillOnEmpty">True</param>
      </param>
    </param>
   </param>

and replace it with this:

<module name="TextField"> 
  <param name="name">sourcetype</param> 
  <param name="label">Sourcetype</param>

It’s quite a bit simpler isn’t it!

Step 9> replace all “SearchSelectLister” code with “Pulldown”.

Same rules apply.  You’ll want to have read the pages about Pulldown.  However Pulldown is VASTLY simpler and more intuitive than SearchSelectLister.   For example you would replace this:

<module name="SearchSelectLister">
  <param name="staticFieldsToDisplay">
    <list>
      <param name="value">*</param>
      <param name="label">Any</param>
    </list>
  </param>
  <param name="search">index=_internal source=*metrics.log group="per_sourcetype_thruput" | top series</param>
  <param name="label">Select series</param>
  <param name="settingToCreate">series_setting</param>
  <param name="searchFieldsToDisplay">
    <list>
      <param name="value">series</param>
      <param name="label">series</param>
    </list>
  </param>
  <param name="searchWhenChanged">False</param>

with this:

<module name="Search">
  <param name="search">index=_internal source=*metrics.log group="per_sourcetype_thruput" | top series</param>
  <module name="Pulldown">
    <param name="name">selectedSourcetype</param>
    <param name="label">Sourcetype</param>
    <param name="valueField">series</param>

This is a hard oen though so there are some things to note carefully.

First, note that the param in the SearchSelectLister was called “series_setting”,  but in the Pulldown and the Search module we have lost the “_setting” part.  This is part of the conversion to bypass all the intention stuff.

Secondly, note that we’re replacing one incredibly complicated ‘SearchSelectLister’ module with two modules — ‘Search’ and ‘Pulldown’.    The listers all have these special “internal” searches, but the Sideview equivalents just use the main search from the module system.  As such you now can put “earliest” and “latest” params on that Search module so you can limit your search in time.    And there are many other benefits.  Fortunately this is the only time we will replace one module with two.

Thirdly, note that if there was not an autoRun attribute on the SearchSelectLister, and there was not an autoRun attribute on any direct ancestor of the SearchSelectLister, then you will have to put an autoRun attribute on the Search module.

Note: if you’re on the older Sideview Utils 1.3.X,  you’ll have to use the older searchFieldsToDisplay param instead of the “valueField” param shown here.  Consult the Pulldown docs appropriate to your Sideview Utils version though and you’ll be fine.


Step 10> replace all StaticSelect with Pulldown

This one’s pretty simple.  Change the “StaticSelect” module name to “Pulldown”,   create a “name” param that has the same value as the “settingToCreate” param,   delete the “settingToCreate” param.  Now go back and delete the “_setting” suffix from the settingToCreate value.

Step 11> replace all ServerSideInclude and StaticContentSample modules with HTML modules. 

This is pretty straightforward, but I’ll say that with the HTML module you can either specify a “src” param in which case it works like ServerSideInclude, or an “html” param, in which case it works like StaticContentSample.

Step 12> replace all Paginator modules with Pager modules

In addition to being a better module and supporting postProcess,   if the “entityName” param is set to “results”, which it will be 99% of the time, you can delete the param entirely because that is the default for Pager.

 

How did we do?

It’s quite possible that your view XML now is about half the size it was right after the conversion to advanced XML.

What next?

remember that page I told you to read at the beginning?   “Key Techniques > Overview of the Advanced XML”.

Even if you’re in the 10 percent of people who actually stopped and read it,   now go back and read it again.

Having just spent some time mucking about in an actual view, now the modules may make more sense.   There is an ‘aha’ moment in the module system, generally when people realize that the XML format is really just a little markup language for functional-description, and that it has nothing to do with “layout”.     But you have to just stare at examples and read the framework docs a few times before the light appears.

 

Leave a comment

Sideview Utils 2.0, now free for internal use!

There’s been another turn in the road around the Sideview Utils licensing.   Full details are in the license agreement, but in short Sideview Utils 2.0 is now available free of charge for internal use only.   (woot!)

To illustrate:

So if you want to use Sideview Utils for some dashboards that you use internally,   go for it.  Likewise if you want to use it in a Splunk app,  but that app will only ever be used internally by people at your company, you’re good.

On the other hand if you’re developing something (like a dashboard or a Splunk app), if you want to sell or distribute that something to other companies, and you want to use Sideview Utils to make it better and build it faster,  well the internal use license agreement is of no use to you.  However!   All is not lost!   If you find yourself in this situation just contact us and we can talk about our OEM agreement.

Wait, why give it away?  

Well,  when Sideview Utils stopped being freely available under the LGPL (as of 2.0),  obviously our download numbers went way down.    However even with that huge and expected falloff in new users there was a surprising rise in interest among companies who were building commercial products on top of Splunk, and who needed an OEM agreement to use Sideview Utils in those products.

As time wore on it became more and more apparent that the OEM side of the business was perfectly viable and also that it was a much more efficient model than selling lots and lots of end-user licenses.    So by giving it away for free to the end-users, restricting them to internal use only, we’re betting that the word will get out and that will end up driving more OEM business.  Simple as that.   That is, if you’re still reading.  If you’re not still reading it probably didn’t seem simple at all.

 

Leave a comment

screencasts are addictive, aka Cooking With Sideview, episode 3

I’m getting the hang of these screencasts, and I just launched one for the Sideview Editor.   Check it out .  You may have seen the screenshot of the Editor on the Sideview Utils page,  but let me give you a tour of it in person.   In the video I introduce the editor,  I first use the editor to browse around in the view and then make some simple edits to a sample dashboard.  Then I move on and wire up a drilldown.  After a few simple edits I’ve made it so that clicks on the table in the example view will take the user to a new view and pass the selected value from the table row.   All without touching a single line of Advanced XML.

Watch the episode

Leave a comment

Cooking with Sideview, episode 2 – docs that live and breathe

Check out the latest screencast.  Here we go and look at the living breathing examples that are embedded in the Sideview Utils documentation.    We walk you through how to read the docs, play with the living examples, and understand the XML source all at the same time.

Along the way you’ll also get a quick sense of the breadth and depth of functionality in Sideview Utils 2.0.

Watch the episode

Leave a comment

Cooking with Sideview, episode 1 – the Sideview report builder

This marks the first of many little screencasts showing some awesome things that are either contained in Sideview Utils 2.0, or that have been built using it.

This particular episode is about the Sideview Report Builder, which like many things just came out in the new version.

So what is the Sideview Report Builder? On the one hand, it’s a very powerful, very user-friendly interface you can use to create your reports, and it comes with Sideview Utils 2.0. On the other hand, the interface was built using the modules and techniques that are packaged in Sideview Utils, so this is also an example to show Splunk app developers how much power there really is in here.

Watch the episode

Leave a comment