环境说明

由于是实验环境，故在一台主机上开设不同的端口来启动mongodb服务进行sharding集群部署。

ENV:

  • MongoDB shell version v3.4.5
  • MongoDB server version: 3.4.5
  • Centos 7 x86_64

集群规划

Sharding集群架构图

shard节点

shard要配置成Replica Set模式

ip port description
192.168.1.167 27001 primary(主节点)
192.168.1.167 27002 secondary(从节点)
192.168.1.167 27003 secondary(从节点)

Config Server节点

config要配置成Replica Set模式

ip port description
192.168.1.167 27010 primary(主节点)
192.168.1.167 27011 secondary(从节点)
192.168.1.167 27012 secondary(从节点)

Mongos Server节点

mongos服务不需要存储数据，开启单实例mongos

ip port description
192.168.1.167 27017 mongos路由

创建数据文件夹

shard server

以shard-1为例：

1
2
3
4
5
6
7

mkdir -p /home/shard/sh1/db && mkdir -p /home/shard/sh1/log

cd /home/shard/sh1
touch mongod.conf

cd /usr/lib/systemd/system/
touch mongodsh1.service

config server

以config-1为例：

1
2
3
4
5
6
7

mkdir -p /home/shard/conf1/db && mkdir -p /home/shard/conf1/log

cd /home/shard/conf1
touch mongod.conf

cd /usr/lib/systemd/system/
touch mongodconf1.service

mongos

mongos服务不需要存储数据，故不需要创建数据目录

1
2
3
4
5
6
7

mkdir -p /home/shard/mongos/log

cd /home/shard/mongos
touch mongos.conf

cd /usr/lib/systemd/system/
touch mongos.service

配置shard server

sharding要求shard server必须是replica set集群方式部署

启动前确保所有的mongod服务都已关闭：killall mongod

配置mongod.conf

mongod.conf

  • replication选项
  • sharding选项
1
2
3
4
5
6

# replica set
replication:
   replSetName: rs       # 指定副本集集群名称
# shard
sharding:
   clusterRole: shardsvr # 指定sharding集群中mongod服务的角色:有两种选择(configsvr | shardsvr)

配置mongod服务脚本

mongod.service - 修改文件(夹)的所有者

需要修改的部分如下：

1
2
3
4
5
6
7
8
9

User=root
Group=root
Environment="OPTIONS=--quiet -f /home/shard/sh1/mongod.conf"
ExecStart=/usr/bin/mongod $OPTIONS run
ExecStartPre=/usr/bin/mkdir -p /var/run/mongodbsh1
ExecStartPre=/usr/bin/chown root:root /var/run/mongodbsh1
ExecStartPre=/usr/bin/chmod 0755 /var/run/mongodbsh1
PermissionsStartOnly=true
PIDFile=/var/run/mongodbsh1/mongod.pid

开启服务

1
2
3

systemctl start mongodsh1   
systemctl start mognodsh2   
systemctl start mongodsh3

配置Replica Set

初始化Replica Set

检查shard节点的Replica Set是否部署成功

配置config server

sharding要求config server必须是replica set集群方式部署

配置mongod.conf

mongod.conf

  • replication选项
  • sharding选项
1
2
3
4
5
6

# replica set
replication:
   replSetName: rs       # 指定副本集集群名称
# shard
sharding:
   clusterRole: configsvr # 指定sharding集群中mongod服务的角色:有两种选择(configsvr | shardsvr)

配置mongod服务脚本

mongod.service

  • 修改文件(夹)的所有者

需要修改的部分如下：

1
2
3
4
5
6
7
8
9

User=root
Group=root
Environment="OPTIONS=--quiet -f /home/shard/sh1/mongod.conf"
ExecStart=/usr/bin/mongod $OPTIONS run
ExecStartPre=/usr/bin/mkdir -p /var/run/mongodbsh1
ExecStartPre=/usr/bin/chown root:root /var/run/mongodbsh1
ExecStartPre=/usr/bin/chmod 0755 /var/run/mongodbsh1
PermissionsStartOnly=true
PIDFile=/var/run/mongodbsh1/mongod.pid

开启服务

1
2
3

systemctl start mongodconf1   
systemctl start mognodconf2  
systemctl start mongodconf3

配置Replica Set

初始化Replica Set

检查config节点的Replica Set是否部署成功

启动mongos服务

配置mongod.conf

mongod.conf

  • storage选项
  • sharding选项
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /home/shard/mongos/log/mongod.log

# Where and how to store data.
#storage:
#  dbPath: /home/shard/mongos/db
#  journal:
#    enabled: true

# how the process runs
processManagement:
  fork: true  # fork and run in background
  pidFilePath: /var/run/mongodbos/mongod.pid  # location of pidfile

# network interfaces
net:
  port: 27017
  bindIp: 192.168.1.167  # Listen to local interface only, comment to listen on all interfaces.

# authorization
#security:
  #authorization: enabled
#  authorization: disabled

# shard
sharding:
  configDB: cf/192.168.1.167:27010,192.168.1.167:27011,192.168.1.167:27012

注：

  • mongos服务不需要存储数据, 故不用配置storage选项
  • configDB选项的config server地址列表必须用一下格式：<setname>/<server_addr>, <server_addr>
    • PS：cf/192.168.1.167:27010,192.168.1.167:27011,192.168.1.167:27012

配置mongod服务脚本

mongod.service

  • 修改文件(夹)的所有者

注：mongos服务脚本的服务实例改为 mongos

启动mongos服务

启动mongos服务前， 确保config server & shard server均已运行

  • 查看所有的config & shard server均已开启：

    1

    ps -aux | grep mongod

  • 开启服务

    1

    systemctl start mongos

注：经测试mongos服务，通过mongos.service脚本开启失败，具体原因尚未可知…

mongos服务的另一种运行方式

1

mongos -f /home/shard/mongos/mongos.conf

Over here, all server has been running.

向集群中添加分片

连接到 mongos server

1
2

mongo 192.168.1.167:27017
mongos>

添加shard节点

1
2
3

sh.addShard("192.168.1.167:27001")
sh.addShard("192.168.1.167:27002")
sh.addShard("192.168.1.167:27003")

为集群数据库开启分片

1

sh.enableSharding("shtest")

对一个集合开启分片

1

sh.shardCollection("shtest.user", {userid: "hashed"})

测试分片

FAQ

Issue: confiDB option

  • mongos –port 27017 –configdb 192.168.1.167:27010,192.168.1.167:27011,192.168.1.167:27012 –logpath /home/shard/mongos/log/mongod.log –fork

    • BadValue: configdb supports only replica set connection string

Issue: No primary detected for set configsvr

  • Shard 必须是复制集

Issue: sh.addShard(“192.168.1.167:27001”)

  • error:

    1
2
3
4
5

    {
    "code" : 96,
    "ok" : 0,
    "errmsg" : "host is part of set rs; use replica set url format <setname>/<server1>,<server2>, ..."
}

  • success:

    1
2

    sh.addShard("rs/192.168.1.167:27001")
{ "shardAdded" : "rs", "ok" : 1 }

See Also

Thanks to the authors 🙂