Elasticsearch
This article describes how to create and maintain connections to your Elasticsearch cluster.
Before you can write your transformed data to an index in Elasticsearch, you should first establish a connection to your Elasticseach cluster.
An Elasticsearch connection can be created as simply as follows:
CREATE ELASTICSEARCH CONNECTION my_elasticsearch
CONNECTION_STRING = 'elasticsearch://search-elastic1-tiq6hb7ltwvcu2vwp3tbixyzey.us-east-1.es.amazonaws.com:443?cluster.name=elastic_main&ssl=true'
USER_NAME = 'your_user'
PASSWORD = 'your_password'
COMMENT = 'My new Elasticsearch connection';
Note that an Elasticsearch connection must specify the cluster name it is connecting to within the connection string.
This means that in order to connect to multiple clusters within your account, you need to create at least one connection per cluster.
For the full list of connection options with syntax and detailed descriptions, see: Elasticsearch connection with SQL
Once you've created your connection, you are ready to move on to the next step of building your data pipeline: Output to an Elasticsearch index
Certain connection options are considered mutable, meaning that in some cases, you can run a SQL command to alter an existing Elasticseach connection rather than create a new one.
For example, take the Snowflake connection we created previously:
CREATE ELASTICSEARCH CONNECTION my_elasticsearch
CONNECTION_STRING = 'elasticsearch://search-elastic1-tiq6hb7ltwvcu2vwp3tbixyzey.us-east-1.es.amazonaws.com:443?cluster.name=elastic_main&ssl=true'
USER_NAME = 'your_user'
PASSWORD = 'your_password'
COMMENT = 'My new Elasticsearch connection';
To change the host you are connecting to but keep everything else the same without having to create an entirely new connection, you can run the following command:
ALTER ELASTICSEARCH CONNECTION my_elasticsearch
SET CONNECTION_STRING = 'elasticsearch://your_new_host:443?cluster.name=elastic1&ssl=true';
If you no longer need a certain connection, you can easily drop it with the following SQL command:
DROP CONNECTION my_elasticsearch;
However, note that if there are existing jobs that are dependent upon the connection in question, the connection cannot be deleted.
Last modified 14d ago