Dockerized Elasticsearch nodes unavailable for Liferay 7.1

2
votes

Liferay is not able to recognize my Elasticsearch cluster when starting. Here is my docker-compose configuration:

version: '2.2'
services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.1.1
    container_name: es01
    environment:
      - node.name=es01
      - discovery.seed_hosts=es02
      - cluster.initial_master_nodes=es01,es02
      - cluster.name=liferay-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata01:/usr/share/elasticsearch/data
    ports:
      - "9299:9200"
      - "9399:9300"
    expose:
      - "9299"
    networks:
      - esnet
  es02:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.1.1
    container_name: es02
    environment:
      - node.name=es02
      - discovery.seed_hosts=es01
      - cluster.initial_master_nodes=es01,es02
      - cluster.name=liferay-cluster2
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    ports:
      - "9298:9200"
      - "9398:9300"
    expose:
      - "9298"      
    volumes:
      - esdata02:/usr/share/elasticsearch/data
    networks:
      - esnet

volumes:
  esdata01:
    driver: local
  esdata02:
    driver: local

networks:
  esnet:

com.liferay.portal.search.elasticsearch6.configuration.ElasticsearchConfiguration.config file content

transportAddresses="127.0.0.1:9299"
logExceptionsOnly="false"
operationMode="REMOTE"
indexNamePrefix="myprefix-"
clusterName="liferay-cluster"

When starting docker-compose, I'm able to access my two ES clusters on: http://127.0.0.1:9299/ and http://127.0.0.1:9298/

However, when liferay starts it is unable to access ES nodes:

NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{vUNCF_HNRtu_tYUjkqhXvg}{127.0.0.1}{127.0.0.1:9299}]]

Anyone tried this configuration ? Any help would be appreciated. Thanks :-)

3
dockerelasticsearchliferayliferay-7liferay-7.1
According to the DXP7.1 compatibility matrix - elasticsearch 7.1 is not compatible with 7.1 DXP. It might be a reason, please rather try to use 6.1.X or 6.5 if you have FixPack5+ – Andre Albert
I updated my docker-compose with the indicated ES version, (I did a docker pull of ES 6.1.4 image) but the issue still persists. Liferay is displaying the same error when starting. – javaxiss
Does your Liferay also run in a Container? Can you wget the elastic from liferay host? – Andre Albert
Liferay is not running in a container. ES is running well since 127.0.0.1:9299 is returning the json response on my browser. – javaxiss

3 Answers

2
votes

I've found a solution. It could help if someone is trying to do the same.

As, I said in my comment to @ibexit, I'm running two dockerized ES clusters and two separate Liferay portals (not in containers) on the same machine (development mode).

I changed th transport address in Liferay OSGi config file, since it must match the transport tcp port where ES is running:

transportAddresses="127.0.0.1:9301"
logExceptionsOnly="false"
operationMode="REMOTE"
indexNamePrefix="myprefix-"
clusterName="liferay-cluster"

I also added the property network.publish_host=127.0.0.1 in my ES clusters (without this property Liferay was not able to detect ES nodes)

Here is my docker-compose.yml:

Using ES 6.1.4

version: '2.2'
services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.1.4
    container_name: es01
    environment:
      - node.name=es01
      - cluster.name=liferay-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - transport.tcp.port=9301
      - network.publish_host=127.0.0.1
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata01:/usr/share/elasticsearch/data
    ports:
      - "9201:9200"
      - "9301:9301"
    networks:
      - esnet
  es02:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.1.4
    container_name: es02
    environment:
      - node.name=es02
      - cluster.name=liferay-cluster2
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - transport.tcp.port=9302
      - network.publish_host=127.0.0.1
    ulimits:
      memlock:
        soft: -1
        hard: -1
    ports:
      - "9202:9200"
      - "9302:9302"
    volumes:
      - esdata02:/usr/share/elasticsearch/data
    networks:
      - esnet

volumes:
  esdata01:
    driver: local
  esdata02:
    driver: local

networks:
  esnet:

network.publish_host did the trick !

0
votes

Per default, elasticsearch binds the transport and http ports to localhost (local) only. So your ports exposed by docker are not working. You need to bind to a specific ip or using 0.0.0.0 for all or site as explained here: https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html#network-interface-values

Please have in mind, that enabling this will startup the node in production mode followed by several bootstrap checks. Please see the docs if you need more informations on this topics or search SO.

0
votes

My working local setup with docker compose and two elasticsearch nodes in docker containers and Liferay running on host. I am using elasticsearch 6.8.2 images modified according to Liferay docs available from docker hub with url: https://hub.docker.com/repository/docker/ktorek/liferay7-elasticsearch.

I am using gradle workspace. So I've configured: configs/local/osgi/configs/com.liferay.portal.search.elasticsearch6.configuration.ElasticsearchConfiguration.config:

operationMode=REMOTE
clusterName=docker-cluster
transportAddresses=127.0.0.1:9300,127.0.0.1:9301

I've killed a lot of time with transportAddresses configuration as it's documented to use square brackets and square quotes transportAddresses=["192.168.1.1:9300","192.168.1.2:9300"] but it does not work. Configuration listed above contains actual working configuration syntax.

My docker-compose.yml:

version: '3.7'
services:

  es01:
    container_name: "es01"
    image: ktorek/liferay7-elasticsearch:latest
    environment:
      - node.name=es01
      - node.data=true
      - cluster.name=docker-cluster
      - xpack.security.enabled=false
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - "discovery.zen.ping.unicast.hosts=es02"
    ports:
      - "9300:9300"
      - "9200:9200"
    networks:
      - mynetwork
    volumes:
      - es01-data:/usr/share/elasticsearch/data
    ulimits:
      memlock:
        soft: -1
        hard: -1

  es02:
    container_name: "es02"
    image: ktorek/liferay7-elasticsearch:latest
    environment:
      - node.name=es02
      - node.data=true
      - cluster.name=docker-cluster
      - xpack.security.enabled=false
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - "discovery.zen.ping.unicast.hosts=es01"
    ports:
      - "9301:9300"
      - "9201:9200"
    networks:
      - mynetwork
    volumes:
      - es02-data:/usr/share/elasticsearch/data
    ulimits:
      memlock:
        soft: -1
        hard: -1
networks:
  mynetwork:
    name: mynetwork
    driver: bridge
    ipam:
      config:
        - subnet: 172.30.29.0/24