What happens when one shard in a MongoDB cluster goes out of disk space? -

What happens when one shard in a MongoDB cluster goes out of disk space? -

tl;dr happens when 1 shard in mongodb cluster goes out of space? new documents written remaining shards, , cluster whole go on function normally? or such scenario can never happen balancer create sure shards as distributed?

context: having standalone mongod instance (a) running on machine 500gb disk space , running out of disk space. i've since sharded database not run out of disk space - cluster has 2 shards of (a & b).

after successful sharding, can see mongodb has taken ~100gb of space on b chunks beingness migrated new shard. however, can see that, has taken additional 30gb of additional space on shard (the original standalone mongod instance). after research, seems ~28gb of taken movechunk directory, guess safe remove?

mongodb version - 2.6.5

first off, read through wrote here balancing of chunks thought of how works. tl;dr version no, balancer not care how total shard is, cares making chunk counts on each shard same.

to reply headline question: when 1 shard runs out of space, inserts chunk ranges live on shard fail. bad state too, , might crash (though mongodb has gotten improve @ handling type of problem).

basically advice is: don't allow happen - can't predict state database might end in when run out of space, improve not point seek deal afterward (more on later).

regarding movechunk files: yes, it safe remove them 1 time shards balanced.

next, should read on how space used/reused in mongodb - when sharded , documents started moving new shard, deleted on old one. not mean space on disk reclaimed, , shard go on see new documents added (depending on chosen shard key). hence, after clean may still see info growth.

what getting @ may run out of space, , want avoid that. mention running single node means need take downwards time reclaim disk space. hence see 2 possible paths:

convert standalone replica set (see guide) , rotate through nodes reclaiming space (minimal downwards time conversion) take shard downwards , run repair (lengthy downwards time, bounded disk speed)

i take first path. 1 time have done necessary work, can go single node if wish (single node replica set = no downwards time, standalone mongod = downwards time convert replica set)

mongodb