This is apparently no need the full Elastic Stack (ELK) here because we only need ES cluster for the database. Cortex XSOAR will read/write to ElasticSearch via API so no need ingestion or report function.
As described on ElasticSearch documentation, we will need at least 3 nodes (servers) to deliver resiliency to the cluster. The setting of these 3 server will be similar, so let’s jump to the 1st one.
For your information, ElasticSearch includes 4 different types of nodes:
· Data nodes — stores data and executes data-related operations such as search and aggregation
· Master nodes — in charge of cluster-wide management and configuration actions such as adding and removing nodes
· Client nodes — forwards cluster requests to the master node and data-related requests to data nodes
· Ingest nodes — for pre-processing documents before indexing
And please be noted that we will use only Master node and Data node in our cluster with XSOAR.
Step 1: have yourself a clean Linux server. In this guide, I use Ubuntu 18.04
Step 2: Install Java
sudo apt-get update
sudo apt-get install default-jre
And check java version
#java --version
openjdk 11.0.10 2021-01-19
OpenJDK Runtime Environment (build 11.0.10+9-Ubuntu-0ubuntu1.18.04)
OpenJDK 64-Bit Server VM (build 11.0.10+9-Ubuntu-0ubuntu1.18.04, mixed mode, sharing)
Step 3: Import ElasticSearch PGP key
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add –
Step 4: Install apt-transport-https to get from ES server
sudo apt-get install apt-transport-https
Step 5: Add ES 7.x repo
echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-7.x.list
Step 6: Install ElasticSearch
sudo apt-get update && sudo apt-get install elasticsearch
Step 7: configure ElasticSearch cluster parameter
Please make sure you finish this step before starting your ElasticSearch service.
sudo vi /etc/elasticsearch/elasticsearch.yml
Add these lines to your configuration file on Node-1
#Set cluster name identical in all of the 3 servers
cluster.name: xsoar-db
#Node name will be node-1, node-2 and node-3
node.name: node-1
#Enable all node to be eligible for master
node.master: true
# Enable all node to be data
node.data: true
#
# ---------------------------------- Network -----------------------
#
#IP address of the node
network.host: 172.17.4.4
#
#Leave default port 9200
http.port: 9200
#
# --------------------------------- Discovery ----------------------
#Add list of 3 nodes IP here for discovery when ES starts
discovery.seed_hosts: ["172.17.4.4", "172.17.4.5", "172.17.4.3"]
#
#Select master nodes
cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]
#
Add these lines to your configuration file on Node-2
#Set cluster name identical in all of the 3 servers
cluster.name: xsoar-db
#Node name will be node-1, node-2 and node-3
node.name: node-2
#Enable all node to be eligible for master
node.master: true
# Enable all node to be data
node.data: true
#
# ---------------------------------- Network -----------------------
#
#IP address of the node
network.host: 172.17.4.5
#
#Leave default port 9200
http.port: 9200
#
# --------------------------------- Discovery ----------------------
#Add list of 3 nodes IP here for discovery when ES starts
discovery.seed_hosts: ["172.17.4.4", "172.17.4.5", "172.17.4.3"]
#
#Select master nodes
cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]
#
Add these lines to your configuration file on Node-3
#Set cluster name identical in all of the 3 servers
cluster.name: xsoar-db
#Node name will be node-1, node-2 and node-3
node.name: node-3
#Enable all node to be eligible for master
node.master: true
# Enable all node to be data
node.data: true
#
# ---------------------------------- Network -----------------------
#
#IP address of the node
network.host: 172.17.4.3
#
#Leave default port 9200
http.port: 9200
#
# --------------------------------- Discovery ----------------------
#Add list of 3 nodes IP here for discovery when ES starts
discovery.seed_hosts: ["172.17.4.4", "172.17.4.5", "172.17.4.3"]
#
#Select master nodes
cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]
#
Step 8: Start ElasticSearch
sudo service elasticsearch start
Step 9: Verify the service
Make sure the service is up and running
systemctl status elasticsearch
Then verify your cluster status
curl -XGET 'http://localhost:9200/_cluster/state?pretty'
You should get similar output
{
"cluster_name" : "xsoar-db",
"cluster_uuid" : "v0cPjnYDTFirhLLJFWAnrw",
"version" : 404,
"state_uuid" : "mNoaeOarR46GrjYflLEA4A",
"master_node" : "M6c0B5AkT_y50si1Rtrb6A",
"blocks" : { },
"nodes" : {
"nvydIDDtS4O5ODAOZgFGJQ" : {
"name" : "node-2",
"ephemeral_id" : "M5jnrwd4SauGcZgtTs7bfg",
"transport_address" : "172.17.4.5:9300",
"attributes" : {
"ml.machine_memory" : "8349163520",
"ml.max_open_jobs" : "20",
"xpack.installed" : "true",
"ml.max_jvm_size" : "1073741824",
"transform.node" : "true"
}
},
"M6c0B5AkT_y50si1Rtrb6A" : {
"name" : "node-1",
"ephemeral_id" : "7ar0kYPtS_CxSSrmC4Q7YQ",
"transport_address" : "172.17.4.4:9300",
"attributes" : {
"ml.machine_memory" : "8349163520",
"xpack.installed" : "true",
"transform.node" : "true",
"ml.max_open_jobs" : "20",
"ml.max_jvm_size" : "1073741824"
}
}
},
Now you have every essential component, ready for XSOAR app installation?
Comments