Elastic Search 入门

1 简介

Elastic Search 是一个基于 Lucene 的开源搜索引擎，它可以快速地储存、搜索和分析海量数据。

2 快速上手（ES7）

2.1 安装 elasticsearch

# https://www.elastic.co/guide/en/kibana/current/docker.html
docker network create elastic
docker pull docker.elastic.co/elasticsearch/elasticsearch:7.14.1
sudo mkdir /usr/share/elasticsearch/
sudo chmod 777 /usr/share/elasticsearch/
mkdir config
vi config/elasticsearch.yml
# add
cluster.name: elastic-single-node
network.host: 0.0.0.0
discovery.type: single-node
xpack.security.enabled: true # 安全机制开启
bootstrap.memory_lock: true # 锁定物理内存地址，避免es使用swap交换分区

docker run -d --restart=always --name elasticsearch --net elastic -p 9200:9200 -p 9300:9300 \
  -v /usr/share/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml \
  -v /usr/share/elasticsearch/data:/usr/share/elasticsearch/data \
  -e "ES_JAVA_OPTS=-Xms5120m -Xmx10240m" \
  docker.elastic.co/elasticsearch/elasticsearch:7.14.1

# set password
docker exec -it elasticsearch bash
./bin/elasticsearch-setup-passwords interactive
# set all password 123456

2.2 安装 kibana

kibana 是一个基于 elasticsearch 的管理工具，它以可视化的方式，让我们更好的管理数据。

docker pull docker.elastic.co/kibana/kibana:7.14.1
sudo mkdir /usr/share/kibana/
sudo chmod 777 /usr/share/kibana/
cd /usr/share/kibana/
mkdir config
vi config/kibana.yml
# add
server.publicBaseUrl: "http://0.0.0.0:5601" # TODO TEST
server.host: "0"
server.shutdownTimeout: "5s"
elasticsearch.hosts: [ "http://elasticsearch:9200" ]
elasticsearch.username: kibana_system
elasticsearch.password: 123456
i18n.locale: zh-CN

docker run -d --restart=always --name kibana --net elastic -p 5601:5601 \
  -v /usr/share/kibana/config/kibana.yml:/usr/share/kibana/config/kibana.yml \
  docker.elastic.co/kibana/kibana:7.14.1
# open in http://localhost:5601

3 快速上手（ES8）

3.1 安装 elasticsearch

资源有限，这里演示仅仅为单机单节点。

# 创建网络
docker network create elastic

# 下载镜像
docker pull docker.elastic.co/elasticsearch/elasticsearch:8.2.3

# 启动 elasticsearch
docker run -d --restart=always --name elasticsearch --net elastic -p 9200:9200 -p 9300:9300 \
  -v elastic-search-data:/usr/share/elasticsearch/data -e "ES_JAVA_OPTS=-Xms10g -Xmx10g" \
  docker.elastic.co/elasticsearch/elasticsearch:8.2.3

# 设置超级管理员 elastic 的密码
docker exec -it elasticsearch bin/elasticsearch-reset-password -u elastic -i
# 输入 y
# 再输入密码两次

# 拷贝出安全证书
docker cp elasticsearch:/usr/share/elasticsearch/config/certs/http_ca.crt .

# 测试连接
curl --cacert http_ca.crt -u elastic https://localhost:9200
# 输入密码

3.2 安装 kibana

kibana 是一个基于 elasticsearch 的管理工具，它以可视化的方式，让我们更好的管理数据。

# 下载镜像
docker pull docker.elastic.co/kibana/kibana:8.2.3

# 启动 kibana
docker run -d --name kibana --net elastic -p 5601:5601 docker.elastic.co/kibana/kibana:8.2.3
# open in http://localhost:5601

docker exec -it elastic-search bin/elasticsearch-create-enrollment-token --scope kibana
# 复制 token，填写到 kibana 的页面里

docker exec -it kibana bin/kibana-verification-code 
# 复制 code，填写到 kibana 的页面里

# 使用 elasticsearch 的账号密码进行登录

4 安装插件

4.1 安装 IK 分词插件

# download the same version package in  https://github.com/medcl/elasticsearch-analysis-ik/releases/tag/v7.14.1
wget https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.14.1/elasticsearch-analysis-ik-7.14.1.zip

# copy to container
docker cp elasticsearch-analysis-ik-7.14.1.zip elasticsearch:/usr/share/elasticsearch/plugins/

# unzip plugin
docker exec -it elasticsearch /bin/bash
cd plugins
mkdir analysis-ik
mv elasticsearch-analysis-ik-7.14.1.zip analysis-ik/
cd analysis-ik
unzip elasticsearch-analysis-ik-7.14.1.zip
rm -f elasticsearch-analysis-ik-7.14.1.zip

# restart elasticsearch
docker restart elasticsearch

# test analysis
curl -XGET "http://elasticsearch:9200/intent/_analyze?pretty=true" -H 'Content-Type: application/json' -d'
{
  "analyzer": "ik_smart", 
  "text": "上海自来水来自海上"
}'

5 常见接口

5.1 Create Index

curl -XPUT "http://localhost:9200/intent" -H 'Content-Type: application/json' -d'
{
  "settings": {
    "number_of_replicas": 1,
    "number_of_shards": 5
  },
  "mappings": {
    "properties": {
      "question": {
        "type": "text",
        "analyzer": "ik_max_word",
        "search_analyzer": "ik_max_word"
      },
      "intent": {
        "type": "text"
      }
    }
  }
}'

5.2 Insert Data

curl -XPOST "http://localhost:9200/intent/_doc" -H "Content-Type: application/json" -d'
{
  "example": "abc",
  "intent": "123"
}'

5.3 Search Data

curl -XGET "http://localhost:9200/intent/_search" -H 'Content-Type: application/json' -d'
{
  "query": {
    "match": {
      "question": "这款是偏大的吗"
    }
  }
}'

5.4 Delete Data

curl -XDELETE "http://localhost:9200/intent"

6 常见问题

# 错误
ERROR: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

# 解决方法
# 1. 临时修改
sysctl -w vm.max_map_count=262144

# 2. 永久修改
sudo vim /etc/sysctl.conf
# 添加
vm.max_map_count=262144
# 重启生效
sudo reboot

7 参考

• 全文搜索引擎 Elasticsearch 入门教程
• Elasticsearch Python API Document: v8.2.3
• Install Elasticsearch with Docker
• 利用 docker 部署 elasticsearch 集群（单节点多实例）
• Elastic Search - Similarity
• Set up minimal security for Elasticsearch