MongoDB分片集群还原

提示：扫一扫查出行
【扫一扫了解最新限行尾号】
复制提示

从mongodb 3.0开始，mongorestore还原的时候，需要一个运行着的实例。早期的版本没有这个要求。

1.为每个分片部署一个复制集

(1)复制集中的每个成员启动一个mongod

mongod --dbpath /mdb/data/s11 --logpath /mdb/mlog/s11.log --fork --port 27017 --replSet s1 --smallfiles &
mongod --dbpath /mdb/data/s12 --logpath /mdb/mlog/s12.log --fork --port 27018 --replSet s1 --smallfiles &mongod --dbpath /mdb/data/s21 --logpath /mdb/mlog/s21.log --fork --port 27019 --replSet s2 --smallfiles &
mongod --dbpath /mdb/data/s22 --logpath /mdb/mlog/s22.log --fork --port 27020 --replSet s2 --smallfiles &

(2)通过mongo连接到实例，运行：

mongo --port=27017
>rs.initiate()
>rs.add("11.11.11.195:27018")

mongo --port=27019
>rs.initiate()
>rs.add("11.11.11.195:27020")

2.部署config服务器

mongod --dbpath /mdb/data/sc --logpath /mdb/mlog/sc.log --fork --port 27021 --configsvr &

3.启动mongos实例

mongos --logpath /mdb/mlog/ss.log --fork --port 30000 --configdb 11.11.11.195:27021

4.集群添加分片

登陆路由器
./mongo --port 30000增加片节点
sh.addShard("s1/11.11.11.195:27018")
sh.addShard("s2/11.11.11.195:27020")

mongos> sh.status()
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("5722c003710922b361783847")
}
shards:
{ "_id" : "s1", "host" : "s1/11.11.11.195:27018,11.11.11.195:27017" }
{ "_id" : "s2", "host" : "s2/11.11.11.195:27020,11.11.11.195:27019" }
balancer:
Currently enabled: yes
Currently running: no
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
No recent migrations
databases:
{ "_id" : "admin", "partitioned" : false, "primary" : "config" }mongos>

5.关闭mongos instances
分片集群启动后，关闭mongos实例

6.还原分片数据

mongorestore --drop /mdb/bin/s1 --port 27017

2016-04-29T10:20:20.643+0800 building a list of dbs and collections to restore from /mdb/bin/s1 dir
2016-04-29T10:20:20.657+0800 reading metadata file from /mdb/bin/s1/snps/elegans.metadata.json
2016-04-29T10:20:20.658+0800 reading metadata file from /mdb/bin/s1/test/system.users.metadata.json
2016-04-29T10:20:20.658+0800 reading metadata file from /mdb/bin/s1/snps/system.users.metadata.json
2016-04-29T10:20:20.658+0800 restoring snps.elegans from file /mdb/bin/s1/snps/elegans.bson
2016-04-29T10:20:20.658+0800 restoring test.system.users from file /mdb/bin/s1/test/system.users.bson
2016-04-29T10:20:20.666+0800 restoring snps.system.users from file /mdb/bin/s1/snps/system.users.bson
2016-04-29T10:20:22.974+0800 restoring indexes for collection snps.system.users from metadata
2016-04-29T10:20:22.975+0800 finished restoring snps.system.users
2016-04-29T10:20:23.073+0800 restoring indexes for collection test.system.users from metadata
2016-04-29T10:20:23.074+0800 finished restoring test.system.users
2016-04-29T10:20:23.644+0800 [##......................] snps.elegans 1.6 MB/13.3 MB (11.8%)
2016-04-29T10:20:26.644+0800 [##############..........] snps.elegans 7.9 MB/13.3 MB (59.3%)
2016-04-29T10:20:29.239+0800 restoring indexes for collection snps.elegans from metadata
2016-04-29T10:20:29.660+0800 finished restoring snps.elegans
2016-04-29T10:20:29.660+0800 done

mongorestore --drop /mdb/bin/s2 --port 27019

2016-04-29T10:20:44.153+0800 building a list of dbs and collections to restore from /mdb/bin/s2 dir
2016-04-29T10:20:44.165+0800 reading metadata file from /mdb/bin/s2/snps/elegans.metadata.json
2016-04-29T10:20:44.165+0800 restoring snps.elegans from file /mdb/bin/s2/snps/elegans.bson
2016-04-29T10:20:44.184+0800 restoring indexes for collection snps.elegans from metadata
2016-04-29T10:20:44.186+0800 finished restoring snps.elegans
2016-04-29T10:20:44.186+0800 done

然后关闭所有分片实例

7.还原config server数据

mongorestore --drop /mdb/bin/config_server --port 27021

2016-04-29T10:26:24.294+0800 building a list of dbs and collections to restore from /mdb/bin/config_server dir
2016-04-29T10:26:24.296+0800 reading metadata file from /mdb/bin/config_server/config/changelog.metadata.json
2016-04-29T10:26:24.296+0800 reading metadata file from /mdb/bin/config_server/config/locks.metadata.json
2016-04-29T10:26:24.297+0800 restoring config.locks from file /mdb/bin/config_server/config/locks.bson
2016-04-29T10:26:24.297+0800 reading metadata file from /mdb/bin/config_server/config/actionlog.metadata.json
2016-04-29T10:26:24.302+0800 reading metadata file from /mdb/bin/config_server/config/chunks.metadata.json
2016-04-29T10:26:24.302+0800 restoring config.chunks from file /mdb/bin/config_server/config/chunks.bson
2016-04-29T10:26:24.303+0800 restoring indexes for collection config.locks from metadata
2016-04-29T10:26:24.303+0800 restoring config.actionlog from file /mdb/bin/config_server/config/actionlog.bson
2016-04-29T10:26:24.303+0800 restoring indexes for collection config.chunks from metadata
2016-04-29T10:26:24.304+0800 restoring config.changelog from file /mdb/bin/config_server/config/changelog.bson
2016-04-29T10:26:24.304+0800 finished restoring config.locks
2016-04-29T10:26:24.306+0800 restoring indexes for collection config.actionlog from metadata
2016-04-29T10:26:24.306+0800 reading metadata file from /mdb/bin/config_server/config/shards.metadata.json
2016-04-29T10:26:24.306+0800 restoring config.shards from file /mdb/bin/config_server/config/shards.bson
2016-04-29T10:26:24.307+0800 finished restoring config.chunks
2016-04-29T10:26:24.307+0800 finished restoring config.actionlog
2016-04-29T10:26:24.307+0800 restoring indexes for collection config.shards from metadata
2016-04-29T10:26:24.308+0800 restoring indexes for collection config.changelog from metadata
2016-04-29T10:26:24.308+0800 reading metadata file from /mdb/bin/config_server/config/databases.metadata.json
2016-04-29T10:26:24.308+0800 restoring config.databases from file /mdb/bin/config_server/config/databases.bson
2016-04-29T10:26:24.308+0800 reading metadata file from /mdb/bin/config_server/config/lockpings.metadata.json
2016-04-29T10:26:24.308+0800 finished restoring config.shards
2016-04-29T10:26:24.308+0800 reading metadata file from /mdb/bin/config_server/config/collections.metadata.json
2016-04-29T10:26:24.308+0800 finished restoring config.changelog
2016-04-29T10:26:24.309+0800 restoring config.lockpings from file /mdb/bin/config_server/config/lockpings.bson
2016-04-29T10:26:24.309+0800 restoring config.collections from file /mdb/bin/config_server/config/collections.bson
2016-04-29T10:26:24.325+0800 reading metadata file from /mdb/bin/config_server/config/mongos.metadata.json
2016-04-29T10:26:24.325+0800 restoring indexes for collection config.databases from metadata
2016-04-29T10:26:24.326+0800 restoring config.mongos from file /mdb/bin/config_server/config/mongos.bson
2016-04-29T10:26:24.326+0800 restoring indexes for collection config.lockpings from metadata
2016-04-29T10:26:24.327+0800 restoring indexes for collection config.collections from metadata
2016-04-29T10:26:24.327+0800 finished restoring config.databases
2016-04-29T10:26:24.327+0800 finished restoring config.lockpings
2016-04-29T10:26:24.328+0800 reading metadata file from /mdb/bin/config_server/config/version.metadata.json
2016-04-29T10:26:24.328+0800 restoring config.version from file /mdb/bin/config_server/config/version.bson
2016-04-29T10:26:24.328+0800 reading metadata file from /mdb/bin/config_server/config/settings.metadata.json
2016-04-29T10:26:24.328+0800 restoring config.settings from file /mdb/bin/config_server/config/settings.bson
2016-04-29T10:26:24.328+0800 finished restoring config.collections
2016-04-29T10:26:24.328+0800 reading metadata file from /mdb/bin/config_server/config/tags.metadata.json
2016-04-29T10:26:24.328+0800 restoring config.tags from file /mdb/bin/config_server/config/tags.bson
2016-04-29T10:26:24.366+0800 restoring indexes for collection config.tags from metadata
2016-04-29T10:26:24.366+0800 restoring indexes for collection config.settings from metadata
2016-04-29T10:26:24.366+0800 restoring indexes for collection config.version from metadata
2016-04-29T10:26:24.375+0800 restoring indexes for collection config.mongos from metadata
2016-04-29T10:26:24.376+0800 finished restoring config.settings
2016-04-29T10:26:24.376+0800 finished restoring config.tags
2016-04-29T10:26:24.376+0800 finished restoring config.mongos
2016-04-29T10:26:24.376+0800 finished restoring config.version
2016-04-29T10:26:24.376+0800 done

8.启动mongos instance

mongos --logpath /mdb/mlog/ss.log --fork --port 30000 --configdb 11.11.11.195:27021

2016-04-29T10:27:56.855+0800 W SHARDING running with 1 config server should be done only for testing purposes and is not recommended for production
about to fork child process, waiting until server is ready for connections.
forked process: 25444
child process started successfully, parent exiting

9.如果shard的主机名发生了变化，需要更新config数据库

mongos> db.shards.find()
{ "_id" : "s1", "host" : "s1/genome_svr1:27501,genome_svr2:27502,genome_svr2:27503" }
{ "_id" : "s2", "host" : "s2/genome_svr4:27601,genome_svr5:27602,genome_svr5:27603" }
mongos> db.shards.update( { "_id": "s1" }, { $set: { "host": "s1/11.11.11.195:27017,11.11.11.195:27018" } }, { multi: true })
mongos> db.shards.update( { "_id": "s2" }, { $set: { "host": "s2/11.11.11.195:27019,11.11.11.195:27020" } }, { multi: true })
mongos> db.shards.find()db.shards.find()
{ "_id" : "s1", "host" : "s1/11.11.11.195:27018,11.11.11.195:27017" }
{ "_id" : "s2", "host" : "s2/11.11.11.195:27020,11.11.11.195:27019" }
mongos>

10.重启所有的shard mongod实例

11.重启其它的mongos实例

12.验证集群环境

mongos> db.printShardingStatus()
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("553f0cc819d7841961ac8f4b")
}
shards:
{ "_id" : "s1", "host" : "s1/11.11.11.195:27018,11.11.11.195:27017" }
{ "_id" : "s2", "host" : "s2/11.11.11.195:27020,11.11.11.195:27019" }
balancer:
Currently enabled: yes
Currently running: no
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
No recent migrations
databases:
{ "_id" : "admin", "partitioned" : false, "primary" : "config" }
{ "_id" : "snps", "partitioned" : true, "primary" : "s1" }
snps.elegans
shard key: { "snp" : 1 }
chunks:
s1 1
s2 1
{ "snp" : { "$minKey" : 1 } } -->> { "snp" : "haw100000" } on : s2 Timestamp(2, 0)
{ "snp" : "haw100000" } -->> { "snp" : { "$maxKey" : 1 } } on : s1 Timestamp(2, 1)
{ "_id" : "test", "partitioned" : false, "primary" : "s1" }mongos>