Can not start Mongos Using systemd service

1
votes

I am unable to start mongos using systemd service

Mongodb Sharding cluster setup

  1. Config replica set (3 servers) mongo-config-1, mongo-config-2, mongo-config-3
  2. Two Sharding replica set (2 data nodes - primary, secondary and 1 arbitrer)

/etc/mongos.conf

sharding: configDB: config0/mongo-config-1:27017,mongo-config-2:27017,mongo-config-3:27017 net: bindIp: 0.0.0.0

/etc/systemd/system/mongos.service

[Unit]
Description=High-performance, schema-free document-oriented database
After=network.target
Documentation=https://docs.mongodb.org/manual

[Service]
User=mongodb
Group=mongodb
ExecStart=/usr/bin/mongos --config /etc/mongos.conf --fork --logpath /var/log/mongodb/mongos.log
PIDFile=/var/run/mongodb/mongos.pid
# file size
LimitFSIZE=infinity
# cpu time
LimitCPU=infinity
# virtual memory size
LimitAS=infinity
# open files
LimitNOFILE=64000
# processes/threads
LimitNPROC=64000
# locked memory
LimitMEMLOCK=infinity
# total threads (user+kernel)
TasksMax=infinity
TasksAccounting=false

# Recommended limits for for mongod as specified in
# http://docs.mongodb.org/manual/reference/ulimit/#recommended-settings

[Install]
WantedBy=multi-user.target

Excerpt from /var/log/mongodb/mongos.log

2018-03-28T13:22:05.019+0000 I CONTROL  [main]
2018-03-28T13:22:05.019+0000 I CONTROL  [main] ** WARNING: Access control is not enabled for the database.
2018-03-28T13:22:05.019+0000 I CONTROL  [main] **          Read and write access to data and configuration is unrestricted.
2018-03-28T13:22:05.019+0000 I CONTROL  [main]
2018-03-28T13:22:05.021+0000 I SHARDING [mongosMain] mongos version v3.6.3
2018-03-28T13:22:05.021+0000 I CONTROL  [mongosMain] git version: 9586e557d54ef70f9ca4b43c26892cd55257e1a5
2018-03-28T13:22:05.021+0000 I CONTROL  [mongosMain] OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016
2018-03-28T13:22:05.021+0000 I CONTROL  [mongosMain] allocator: tcmalloc
2018-03-28T13:22:05.021+0000 I CONTROL  [mongosMain] modules: none
2018-03-28T13:22:05.021+0000 I CONTROL  [mongosMain] build environment:
2018-03-28T13:22:05.021+0000 I CONTROL  [mongosMain]     distmod: ubuntu1604
2018-03-28T13:22:05.021+0000 I CONTROL  [mongosMain]     distarch: x86_64
2018-03-28T13:22:05.021+0000 I CONTROL  [mongosMain]     target_arch: x86_64
2018-03-28T13:22:05.021+0000 I CONTROL  [mongosMain] db version v3.6.3
2018-03-28T13:22:05.021+0000 I CONTROL  [mongosMain] git version: 9586e557d54ef70f9ca4b43c26892cd55257e1a5
2018-03-28T13:22:05.021+0000 I CONTROL  [mongosMain] OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016
2018-03-28T13:22:05.021+0000 I CONTROL  [mongosMain] allocator: tcmalloc
2018-03-28T13:22:05.021+0000 I CONTROL  [mongosMain] modules: none
2018-03-28T13:22:05.021+0000 I CONTROL  [mongosMain] build environment:
2018-03-28T13:22:05.021+0000 I CONTROL  [mongosMain]     distmod: ubuntu1604
2018-03-28T13:22:05.021+0000 I CONTROL  [mongosMain]     distarch: x86_64
2018-03-28T13:22:05.021+0000 I CONTROL  [mongosMain]     target_arch: x86_64
2018-03-28T13:22:05.021+0000 I CONTROL  [mongosMain] options: { config: "/etc/mongos.conf", net: { bindIp: "0.0.0.0" }, processManagement: { fork: true }, sharding: { configDB: "config0/mongo-config-1:27017,mongo-config-2:27017,mongo-config-3:27017" }, systemLog: { destination: "file", path: "/var/log/mongodb/mongos.log" } }
2018-03-28T13:22:05.022+0000 I NETWORK  [mongosMain] Starting new replica set monitor for config0/mongo-config-1:27017,mongo-config-2:27017,mongo-config-3:27017
2018-03-28T13:22:05.022+0000 I SHARDING [thread1] creating distributed lock ping thread for process mongos-router-2:27017:1522243325:-5142424883419352228 (sleeping for 30000ms)
2018-03-28T13:22:05.025+0000 I NETWORK  [shard registry reload] Successfully connected to mongo-config-3:27017 (1 connections now open to mongo-config-3:27017 with a 5 second timeout)
2018-03-28T13:22:05.025+0000 I NETWORK  [mongosMain] Successfully connected to mongo-config-1:27017 (1 connections now open to mongo-config-1:27017 with a 5 second timeout)
2018-03-28T13:22:05.026+0000 I NETWORK  [ReplicaSetMonitor-TaskExecutor-0] Successfully connected to mongo-config-2:27017 (1 connections now open to mongo-config-2:27017 with a 5 second timeout)
2018-03-28T13:22:05.026+0000 I ASIO     [NetworkInterfaceASIO-ShardRegistry-0] Connecting to mongo-config-3:27017
2018-03-28T13:22:05.026+0000 I ASIO     [NetworkInterfaceASIO-ShardRegistry-0] Connecting to mongo-config-1:27017
2018-03-28T13:22:05.026+0000 I ASIO     [NetworkInterfaceASIO-ShardRegistry-0] Connecting to mongo-config-1:27017
2018-03-28T13:22:05.026+0000 I ASIO     [NetworkInterfaceASIO-ShardRegistry-0] Connecting to mongo-config-3:27017
2018-03-28T13:22:05.030+0000 I ASIO     [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to mongo-config-3:27017, took 4ms (2 connections now open to mongo-config-3:27017)
2018-03-28T13:22:05.034+0000 I ASIO     [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to mongo-config-1:27017, took 8ms (2 connections now open to mongo-config-1:27017)
2018-03-28T13:22:05.034+0000 I ASIO     [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to mongo-config-1:27017, took 8ms (2 connections now open to mongo-config-1:27017)
2018-03-28T13:22:05.034+0000 I ASIO     [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to mongo-config-3:27017, took 8ms (2 connections now open to mongo-config-3:27017)
2018-03-28T13:22:05.036+0000 I NETWORK  [shard registry reload] Starting new replica set monitor for shard1/mongo-shard1-server-1:27017,mongo-shard1-server-2:27017
2018-03-28T13:22:05.036+0000 I NETWORK  [shard registry reload] Starting new replica set monitor for shard2/mongo-shard2-server-1:27017,mongo-shard2-server-2:27017
2018-03-28T13:22:05.041+0000 W SHARDING [replSetDistLockPinger] pinging failed for distributed lock pinger :: caused by :: LockStateChangeFailed: findAndModify query predicate didn't match any lock document
2018-03-28T13:22:05.042+0000 I NETWORK  [ReplicaSetMonitor-TaskExecutor-0] Successfully connected to mongo-shard1-server-1:27017 (1 connections now open to mongo-shard1-server-1:27017 with a 5 second timeout)
2018-03-28T13:22:05.047+0000 I NETWORK  [ReplicaSetMonitor-TaskExecutor-0] Successfully connected to mongo-shard1-server-2:27017 (1 connections now open to mongo-shard1-server-2:27017 with a 5 second timeout)
2018-03-28T13:22:05.054+0000 I NETWORK  [ReplicaSetMonitor-TaskExecutor-0] Successfully connected to mongo-shard2-server-1:27017 (1 connections now open to mongo-shard2-server-1:27017 with a 5 second timeout)
2018-03-28T13:22:05.061+0000 I NETWORK  [ReplicaSetMonitor-TaskExecutor-0] Successfully connected to mongo-shard2-server-2:27017 (1 connections now open to mongo-shard2-server-2:27017 with a 5 second timeout)
2018-03-28T13:22:07.031+0000 I ASIO     [NetworkInterfaceASIO-ShardRegistry-0] Connecting to mongo-config-2:27017
2018-03-28T13:22:07.040+0000 I ASIO     [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to mongo-config-2:27017, took 9ms (1 connections now open to mongo-config-2:27017)
2018-03-28T13:22:07.040+0000 I ASIO     [NetworkInterfaceASIO-ShardRegistry-0] Connecting to mongo-config-2:27017
2018-03-28T13:22:07.041+0000 I FTDC     [mongosMain] Initializing full-time diagnostic data capture with directory '/var/log/mongodb/mongos.diagnostic.data'
2018-03-28T13:22:07.042+0000 I ASIO     [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to mongo-config-2:27017, took 2ms (2 connections now open to mongo-config-2:27017)
2018-03-28T13:22:07.043+0000 I NETWORK  [mongosMain] waiting for connections on port 27017
2018-03-28T13:22:07.045+0000 I CONTROL  [signalProcessingThread] got signal 15 (Terminated), will terminate after current cmd ends
2018-03-28T13:22:07.046+0000 W SHARDING [signalProcessingThread] error encountered while cleaning up distributed ping entry for mongos-router-2:27017:1522243325:-5142424883419352228 :: caused by :: InterruptedAtShutdown: interrupted at shutdown
2018-03-28T13:22:07.046+0000 I FTDC     [signalProcessingThread] Shutting down full-time diagnostic data capture
2018-03-28T13:22:07.046+0000 W SHARDING [Uptime reporter] failed to refresh mongos settings :: caused by :: InterruptedAtShutdown: Failed to refresh the balancer settings due to InterruptedAtShutdown: interrupted at shutdown
2018-03-28T13:22:07.046+0000 I CONTROL  [signalProcessingThread] shutting down with code:0

Note: I am able to start mongos using command line

sudo /usr/bin/mongos --config /etc/mongos.conf --logpath /var/log/mongodb/mongos.log
2
databasemongodbshardingsystemd
What linux distro are you using? How did you install mongodb, was it just a standard install from their repository? – jordanwillis
@jordanwillis I am using Ubuntu 16.04, I was trying to run mongos using systemd just like you run mongod using systemctl start mongod since the documentation does not mention how to do that. I was trying to figure it out myself – Hemant Mann

2 Answers

1
votes

Problem here is that systemd don't know that this mongos process is forking and that's why kill (-15) signal is sent to that process.

Check here and add 

[Service]
Type=forking
0
votes

If you specify forking and pidFile in the configuration file it is not necessary specifing the forking option in the systemd.

processManagement:
  fork: true
  pidFilePath: "/var/run/mongos.pid"

It is important that pidfile match in both config files.