Promote System Overview Doesn't Show All Machines

[DavidHa]

Issue

The Promote administrative System Overview table only shows the Leader node, or a subset of the nodes in the Promote cluster.



However, the promote.status command reports that everything is working correctly:
 

[promote@CST-DEMO-PRO-1 ~]$ promote.status
ID                  NAME                        MODE                REPLICAS            IMAGE                                     PORTS
0bghzbfpxjla        promote_registrator         global              4/4                 quay.io/yhat/registrator:2020.4.0
0qudq20bmce4        promote_kibana              replicated          1/1                 quay.io/yhat/kibana:2020.4.0              *:85->5601/tcp
20vddh5u9y5o        promote_logstash            global              4/4                 quay.io/yhat/logstash:2020.4.0            *:5001->5000/tcp, *:5400->5400/tcp, *:9600->9600/tcp, *:5002->5002/udp
2q928z6880fa        promote_portainer           replicated          1/1                 portainer/portainer:1.22.2                *:9000->9000/tcp
4syx12g0nxbt        promote_db                  replicated          1/1                 quay.io/yhat/promote-db:2020.4.0          *:5432->5432/tcp
bcc1qknamvji        promote_elasticsearch       replicated          1/1                 quay.io/yhat/elasticsearch:2020.4.0       *:9200->9200/tcp, *:9300->9300/tcp
cnfidv40evbr        promote_docker-events-mon   global              4/4                 quay.io/yhat/docker-events-mon:2020.4.0
jt3piu5y9f4l        promote_consul              replicated          1/1                 quay.io/yhat/consul:2020.4.0              *:8500->8500/tcp
kwroflyqsjk0        promote_nginx               global              4/4                 quay.io/yhat/nginx-proxy:2020.4.0
l96vmcuv0spy        promote_influxdb            replicated          1/1                 quay.io/yhat/influxdb:2020.4.0            *:8086->8086/tcp
mg2tlt75rf8y        promote_logspout            global              4/4                 quay.io/yhat/logspout-logstash:2020.4.0
pd1c8nmskxaj        promote_registry            replicated          1/1                 registry:2.6.2                            *:5000->5000/tcp
wpkwasc0u4ov        promote_app                 replicated          1/1                 quay.io/yhat/promote-app:2020.4.0         *:3001->3001/tcp
y93r7no24zd8        promote_telegraf            global              4/4                 quay.io/yhat/telegraf:2020.4.0

Cause

The cause of this problem is that the time of each node in the cluster must be in sync with the Leader node.  This can easily be checked with the "date" command at the Linux command prompt.  Machine times can slowly diverge over time if they are not in sync with a NTP server.

Resolution

Work with your IT team to configure the NTP service to keep machines in sync with an NTP server.  Example procedure below:
 

1. Install the NTP package:

sudo yum install ntp

2. Edit the NTP configuration file:

sudo vi /etc/ntp.conf

Add the NTP servers to the configuration file:

# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
server 0.pool.ntp.org
server 1.pool.ntp.org
server 2.pool.ntp.org

 
3.  Start the NTP Service:

sudo service ntpd start

4. Configure the NTP service to auto start on system boot:

sudo chkconfig ntpd on   

5. Repeat steps 1-4 on all machines the Promote Cluster.

After completing the above steps, the "date" command should return consistent results on all machines, and the System Overview panel should then display all machines the Promote Cluster.