This blog is a continuation of the Kibana tutorial blog. If you reached this blog directly, it is highly recommended to learn the basics of Kibana by visiting this link.
Once you have already understood the complete basics of Kibana, now it’s time for you to create your own visualizations and dashboards.
In the second part of the Kibana tutorial Blog we cover the following:
Let’s get started right away by building your own visualization!
Your data is ready and now you are ready to create your own visualization. From the main menu bar as shown below click on the “Visualize”
In this blog, we will explore the most popular visualization: the Time Series Visual Builder, also known as TSVB. Using this visualization, you can create multiple types of visualization such as Time Series, Metrics, TopN, Gauge, Markdown, and Table. The TSVB visualization is like a swiss knife for your visual needs. It enables you to visualize data from multiple data series. At the same time, it supports the highest number of Elasticsearch metric aggregations with numerous types of visualization and custom functions. Time-Series Visualize Builder (TSVB) graph offers you various visualizations under one hood and is thus easy to recommend for building your visualizations.
Once you click on “Create Visualization”, choose the TSVB option from the menu as shown below:
Select the “Data source” for monitoring. Kibana will load with the default indices on the data sources drop-down options. To make it more relevant to your day-to-day monitoring activities, we will look at an example where you are going to monitor the log of AWS ECS metrics.
Once you are on the next page, click on “Panel options” from the tabs menu and fill in the required index pattern (index containing the logs or metrics for visualizing).
Since in this example you are going to measure the AWS ECS metrics, you choose the Index Pattern that you have created to measure the metrics. You can use any of the indexes that you have already created and then set the Time field as @timestamp as shown below.
Now that you have configured the data. It’s time to do the math on it. For doing this, click on the “Data tab” and click on the “Aggregation” dropdown.
Metric aggregations are used to calculate a value for each document based on the details inside the bucket. Each visualization type has unique characteristics providing different ways to present buckets and their associated values. To break it down in simple terms, you are going to select “Max” aggregation (other options displayed in the image below) and then specify the “Field” name for the plot. (Example given below: ecs.cpu_utilized is plotted) and set the “Group by” as “Everything”(Can be changed based on the requirement). What you have done is basically provided instructions to Kibana to work with the data from the index. This visualization would display the Maximum CPU utilization over a while over a time-series graph.
Now that all instructions are ready to go, click on the “Save” button to save your visualization. Before saving the visualization, do enter a “Title” and “Description”. That would be handy for you or someone else working on the dashboard to understand the need for the dashboard. Assigning “Titles” to the visualizations must be unique and specific.
Good naming convention is imperative to keep track of the visualizations
After the save is completed the visualization will look in a similar manner as shown below:
You have now successfully created a Time Series (TSVB). Now let’s move on to create some complex visualizations on Kibana!
Now that you have successfully created a Time Series. We leap ahead from the simple visualization to build something complex and pragmatic. What if you could use the data from the previous few days or hours for our analysis. The offset data would allow you to understand the working of our infrastructure in a more precise manner. For example, when you made some alterations on your dashboards in the previous week and if you want to plot comparison of the current week’s performance of your infrastructure and that of the past week when you made the alterations, you can use the data generated to generate information out of it using Kibana and, eventually, the information becomes knowledge for your team. Let’s proceed and see how you can form such cool visualizations on Kibana!
We consider a week-to-week comparison over a time series graph for metrics-based data where the data at the point of time is compared to the data exactly one week earlier. Below given is the final snapshot of the time-series graph that you can build. The trick is to use the time-series graph that you created earlier and then add an overlapping graph. For this, the data for the overlapping graph should be having an offset of one week. The sample that we operate to build such a visualization is the “Number of Queries” executed on the server.
Note: We are going to work on an existing time-series visualization and will make changes on a similar time-series graph. You can create your own time-series graph and proceed in a similar manner.
For creating a new TSVB refer to the preceding section
At any point in time, visualizations can be edited using the dashboard “Edit” button on the top right-hand corner on the second ribbon from the top. Once you click the edit button, all the individual visualizations should now have a tool button on the top right-hand corner for making changes to the visualization. Click on the “Tool button” and then click on the “Edit Visualization”.
Once you click on the “Edit Visualization” button, a screen will appear similar to the snapshot given below for making changes to the visualization with three sub-options (Data, Panel Options, Annotations) under the time-series graph.
Click on the “Data” menu to explore the options with metrics and other options. Each color tag corresponding to the respective graph is adjacent to the name correlated in the ledger. Choose the options under each tab under metrics. Choose the aggregation type, field names, scale, and the corresponding grouping as per the requirements. For monitoring, you can configure the preferred settings for a time-series graph, given in the above snapshot, along with the changes in the field name required.
To insert overlaying graphs with a particular offset, click on the “Plus” button from the menu on the tab on the top right corner.
TIP: The menu can also be used for hiding/unhiding the series, cloning the series, creating a new series, or deleting the series
Once you click on the plus sign from the menu, a new series is ready to be configured. Make the adjustments as per the requirements for your graph.
The next step is setting an offset for the time series. Click on the “Options” tab and then enter the offset that you want to plot for the time series
1w = 1 week, 1h =1 hour, etc., you can set the offset as per your requirements.
TIP: Utilize the Panel options whenever possible to build snappy dashboards
Using Panel options allows Kibana to pinpoint the exact data set on which your visual will query your data. Click on the “Panel options” tab to set the panel filter. Once you click the panel tab, fill in the details about the panel filter. For this example, the status dataset of the MySQL module is going to be utilized. We mention the particular dataset that you need to pinpoint the query (example: event.dataset:mysql.status). Once all the details have been entered. Click save and return on the right corner of the second ribbon from the top.
You have now created a Week-to-Week Comparison TSVB graph.
The dashboard is essentially a collection of visualizations grouped in one screen. So now it is time for you to bring together all the work and combine our excellent visualizations to form our dashboards. Since dashboards are the combination of the different visualizations that you have created over the period, you should answer the question about the intended use of the dashboards.
Design answers pertaining to dashboard goal should be clearly defined
Dashboards should not lose their meaning by being too overwhelming and overcrowded with many metrics.
While building your dashboards, you should also keep in mind the issue that may arise due to multiple data sources. Generating vast visuals with un-correlated data will kill the ability to drill down. Performing a drill-down on Kibana will add a filter to your data sources that will only be relevant to the selected source effectively, making the visualizations useless. As a result, for dashboards where drill-down capabilities are required, it is critical to ensure that your visualizations have the same data source. It is critical to ensure that your visualizations have the same data source!
TIP: You should try to keep your dashboards to a single page to avoid scrolling
To create your dashboards click on “Dashboards” from the main menu bar. Once you click on the Dashboard tab, you will arrive at a screen where you have an option for creating a new dashboard. Click on the “Create dashboard” button. The page will then be redirected to a new page where you have the option to create new visualizations and then add them, or you can add the visualization from the existing list of visualizations. Since you have already generated a couple of visualizations, you should opt for the “Add an existing” option.
A new pop-up appears on the right side of the screen to add the panel. You can either search for the specific visualization or go through page indexes to find your required visualization.
You have the freedom to select as many visualizations as per your requirement for your dashboard.
TIP: Keep your most visited visuals on the top of any dashboards to save your time
You can reshuffle each of your visualizations as per your needs. Hold and drag the visualization from the top banner and place it anywhere on your dashboards. Once all the arrangements of your visualizations look good. You are now ready to save the visualization. Click on the “Save” button from the list.
As you click the “Save” button, you will have a pop-up on the screen asking you to fill in the name and the description of the visualization and click on the “Save” button.
TIP: Name your visuals in a unique manner to increase your work efficiency
Provide a unique name for the dashboard so that it is easy for you to track the names among the multiple dashboards that you want to create in the future. It is a good practice to mention the details of the dashboard in the description box.
The dashboards that you have created till now do not allow you to establish correlations among various visualizations. Each visualization acts as a stand-alone. There were no fields or any other correlated data in the visuals to create correlated visualizations and interactive dashboards.
To create a dashboard where you can see and apply the filter to the live dashboard, you must create a search object and then add it as a visualization to apply the filter.
To create a search object, you start by clicking on the “Discover” tab and then “Enter the index” of the logs/metrics you want to build your search object. For instance, Mysql logs have been put in play. Below given is the screenshot of the index pattern on which you want to build our search objects. Select the required index.
Once you select the required index. You will then reach a screen as shown below.
Select the “Column” as per your requirements and that you want to display on the visualization on your dashboard. Use the “Plus” sign on each of the fields as shown below:
Now that you have selected all the required fields, click on the “Save” button on the top end corner with a “Unique name” for the visualization that would appear on your dashboard.
Now that the visualization and the search object are ready, you have all that you need to create a new interactive dashboard. Reach out to the required dashboard, which you want to modify as an interactive one, and click the “Edit” button to add the new search panel to your dashboard. Then click on the “Add” option from the list of options.
Once you click “Add”, a pop-up will appear on the right side of the screen to add the search panel. Search for the required panel name and then select the type of panel as “Saved Search” as shown below:
Click on the “Respective Panel Name” to add the search panel as a visualization on the dashboard. Now that you have created a new search object on the dashboard, you can now use the search filter from the search panel.
Reach out to the search object that you have added to the dashboard from the previous steps.
For filtering out the data, based on the values under the respective column names, hover your pointer over the required field name and click on the “(+)Plus button”. Once you have selected the required filter all the graphs and metrics on your dashboard will be displayed with the applied filter. Check on the top right corner to confirm if the settings are applied across all the visualizations.
If you want to eliminate any fields from the search, reach out to the search panel again & click on the “(-)minus” sign. A sample of the search object in Kibana is given below:
Going through the two blogs you must have already understood that Kibana is a very powerful tool for building visualizations and dashboards. Along the way, you have explored the basics of Kibana along with the advanced visualization and dashboarding capabilities. You also came across some advanced use cases of Kibana such as creating a search object and dashboards with co-related visualization. We have only scratched the surface of visualizations and dashboarding through these blogs. There are many more ways to explore your data, many more ways to change the data to useful insights that would help you and your team. Happy Exploring!
PacketAI is the world’s first autonomous monitoring solution built for the modern age. Our solution has been developed after 5 years of intensive research in French & Canadian laboratories and we are backed by leading VC’s. To know more, book a free demo and get started today!