官术网_书友最值得收藏!

The app key-value store

The app key-value store is a feature provided by Splunk Enterprise to manage and maintain the state of the application. Using an app key-value store, users can save and retrieve data from Splunk apps.

System requirements

The app key-value store feature is only available in the 64-bit distribution of Splunk Enterprise. It is not available in the 32-bit version of Splunk. It uses the 8191 port by default, but it can be configured from Server.conf located at $SPLUNK_HOME\etc\system\local by modifying the [kvstore] code block.

Uses of the key-value store

The following are some of the uses of a key-value store:

  • It can be used to manage the app state of the user interface by storing the session/application state information
  • It creates a checkpoint of the uploaded data in case of modular inputs
  • It enlists the environment variable used, accessed, or modified by users
  • It is the metadata storage of the user
  • It caches results from search queries

Components of the key-value store

The key-value store saves data in the collections of the key-value pair. The key-value store files are located on the search heads. The following are the various components of the key-value store:

  • Collections: Collections are containers for data storage similar to a database table.
  • Records: Records store the entry of data in the collection.
  • Fields: Fields contain the value of data in the JSON format file. Fields correspond to the key name similar to columns in the database table.
  • _key: This is the reserved field that contains a unique ID for each record. It is an autogenerated field that is not explicitly specified.
  • _user: This is also a reserved field that is used to map the user ID of each record.
  • Accelerations: This is used to improve search performance that contains the accelerated fields.

Let's take a look at how to create a key-value store collections via a config file. To use a key-value store, we need to create a key-value store collection using the following steps:

  1. Create a collections.conf file in the application's default or local directory, as follows $SPLUNK_HOME\etc\apps\APPNAME\default\collections.conf or $SPLUNK_HOME\etc\apps\APPNAME\local\collections.conf.
  2. Modify collections.conf by specifying the name of the collection and optionally, the schema of the data. Listed in the following sublist is the description of the parameters which need to be configured in collections.conf file:
    • [collection_name]: This is the collection name
    • enforceTypes: This is set to True or False to enforce the data types of values when inserting them into the collection.
    • field.name: This is an optional field. The available data types are string, time, Boolean, and number. If the data type is not set explicitly, then it is set to JSON.

Any change in collections.conf needs a restart of the Splunk instance to apply the changes on the search heads. Refer to the following example for better understanding:

[AndroidCollections]  #collection_name

The screenshot that follows shows a code snippet of the sample JSON data:

Components of the key-value store

The following screenshot is the code snippet of the enforce data type for the preceding JSON data:

Components of the key-value store

The following screenshot shows the sample code snippet for hierarchical JSON data:

Components of the key-value store

The following screenshot shows how a data type can be enforced on hierarchical data using a dot (.) notation:

Components of the key-value store

Managing key-value store collections via REST

The Splunk REST API can be used to create, read, delete, update, and manage key-value store data and collections. The Splunk REST API accesses Splunk via the management port (by default, 8089). The following are the REST endpoints for the key-value store:

  • storage/collections/config:
    • GET: This fetches a list of collections in a specific app
    • POST: This creates a new collection in a specific app
  • storage/collections/config/{collection}:
    • GET: This fetches information about a specific collection
    • DELETE: This deletes a collection
    • POST: This updates a collection
  • storage/collections/data/{collection}:
    • GET: This fetches records from a specific collection
    • POST: This inserts a new record into a specific collection
    • DELETE: This deletes all records from a specific collection
  • storage/collections/data/{collection}/{id}:
    • GET: This fetches records in a collection by a key ID
    • POST: This updates records in a collection by a key ID
    • DELETE: This deletes a record in a collection by a key ID
  • storage/collections/data/{collection}/batch_save:
    • POST: This runs one or more save (insert and replace) operations in a specific collection

Examples

There are various notations used in the following examples, such as username, password, IPAddress, and others. Users need to replace them with their own corresponding values to execute the examples. The following are the examples:

  • Fetching a list of collections for an android app:
    curl -k -u username:password \ 
    https://IPAddress:8089/servicesNS/nobody/android/storage/collections/config
  • Creating a new collection called AndroidCollections in the android app:
    curl -k -u username:password \ -d name= AndroidCollections \ 
    https://IPAddress:8089/servicesNS/nobody/android/storage/ collections/config
  • Defining a collection schema:
    curl -k -u username:password \ 
    https://IPAddress:8089/servicesNS/nobody/android/storage/ collections/config/ AndroidCollections \ 
    -d field.Devicename = string \ 
    -d field.DeviceID = number \
    -d field.DeviceInfo.DeviceBuild = string \
    -d field.DeviceInfo.DeviceAndroidVersion = string 
  • Adding data of the hierarchical JSON format to a collection:
    curl -k -u username:password \
    https://IPAddress:8089/servicesNS/nobody/android/storage/ collections/config/ AndroidCollections \
    -H 'Content-Type: application/json' \
    -d '{ "Devicename" : "Test Device", "DeviceID" : 9661, "DeviceInfo" : { "DeviceBuild" : "Test build 9661C", "DeviceAndroidVersion" : "Marshmallow 6.0", "DeviceIMEI" : 12345678909876, "DeviceMAC" : "AA:BB:CC:DD:EE:FF" }} '
  • Getting all data from the collection:
    curl -k -u username:password \
    https://IPAddress:8089/servicesNS/nobody/android/storage/ collections/config/ AndroidCollections
  • Getting a specific range of records from collections, for example, records from 10 to 15:
    curl -k -u username:password \
    https://IPAddress:8089/servicesNS/nobody/android/storage/ collections/config/ AndroidCollections?sort=Devicename&skip=10&limit=5
  • Getting a record of a specific key ID:
    curl -k -u username:password \
    https://IPAddress:8089/servicesNS/nobody/android/storage/ collections/config/ AndroidCollections/KEYID

    Where the key ID is the unique _key of collections for which the record is to be fetched.

  • Deleting the record of the specific key ID:
    curl -k -u username:password –X DELETE \
    https://IPAddress:8089/servicesNS/nobody/android/storage/ collections/config/ AndroidCollections/KEYID
  • Deleting all records of the AndroidCollections collection:
    curl -k -u username:password –X DELETE \
    https://IPAddress:8089/servicesNS/nobody/android/storage/ collections/config/ AndroidCollections

Replication of the key-value store

In case of a distributed environment, the key-value store can be replicated to a large number of search heads by enabling replication. By default, the key-value store is not replicated to indexers in distributed deployment of Splunk.

To enable replication, the collections.conf file is to be modified and we need to add replicate = true to the file.

主站蜘蛛池模板: 乾安县| 阿鲁科尔沁旗| 习水县| 伊宁县| 交口县| 台中县| 枝江市| 肥乡县| 舟曲县| 清镇市| 湾仔区| 中方县| 南城县| 文昌市| 独山县| 垦利县| 西畴县| 平凉市| 侯马市| 曲麻莱县| 镇雄县| 定结县| 泗水县| 枝江市| 鄱阳县| 杨浦区| 泉州市| 翁牛特旗| 曲沃县| 边坝县| 蒙山县| 陆河县| 普宁市| 永春县| 吴忠市| 那坡县| 五大连池市| 马公市| 白玉县| 瑞安市| 富锦市|