Saturday 15 February 2014

ruby - How do I find out how many Mongo documents were actually inserted? -



ruby - How do I find out how many Mongo documents were actually inserted? -

i have function looks this:

def insert_multiple_cakes(cake_list) ensure_indexes insert_list = cake_list.map { |cake| mongofy_values(cake.to_hash) } inserted = db[cake_collection].insert(insert_list, w: 0) homecoming inserted.length end

the goal of function insert cakes cake_list mongo database. cake exists in database should ignored. function should homecoming number of cakes inserted, if cake_list contains 5 cakes , 2 of cakes exist in database, function should homecoming 3.

my problem after hr of experimenting, have concluded following:

if write concern (the :w option) 0, insert phone call silently ignores duplicate inserts, , homecoming value contains all input documents, weren't inserted. doesn't matter set :continue_on_error or :collect_on_error, homecoming value always contains documents, , list of collected errors empty.

if write concern 1, insert phone call fails mongo::operationfailure if there duplicates among input documents. doesn't matter set :continue_on_error or :collect_on_error to, insert fails when there duplicates.

so seems me way accomplish iterate on input list, perform search every document , filter away exist. application going deal (at least) thousands of inserts @ time, plan much i'd jump off bridge.

have misunderstood something, or ruby client perhaps bugged?

to demonstrate, function want , works:

def insert_multiple_cakes(cake_list) ensure_indexes collection = db[cake_collection] # filters away cakes exists in database. filtered_list = cake_list.reject { |cake| collection.count(query: {"name" => cake.name}) == 1 } insert_list = filtered_list.map { |cake| mongofy_values(cake.to_hash) } inserted = collection.insert(insert_list) homecoming inserted.length end

the problem performs gazillion searches should have 1 insert.

documentation mongo::collection#insert

you can (source):

coll = mongoclient.new().db('test').collection('cakes') mass = coll.initialize_unordered_bulk_op bulk.insert({'_id' => "strawberry"}) bulk.insert({'_id' => "strawberry"}) # duplicate key bulk.insert({'_id' => "chocolate"}) bulk.insert({'_id' => "chocolate"}) # duplicate key begin bulk.execute({:w => 1}) # default don't alter 0 or won't errors rescue => ex p ex p ex.result end

ex.result contains ninserted , reason each 1 failed.

{"ok"=>1, "n"=>2, "code"=>65, "errmsg"=>"batch item errors occurred", "ninserted"=>2, "writeerrors"=> [{"index"=>1, "code"=>11000, "errmsg"=> "insertdocument :: caused :: 11000 e11000 duplicate key error index: test.cakes.$_id_ dup key: { : \"strawberry\" }"}, {"index"=>3, "code"=>11000, "errmsg"=> "insertdocument :: caused :: 11000 e11000 duplicate key error index: test.cakes.$_id_ dup key: { : \"chocolate\" }"}]}

ruby mongodb

No comments:

Post a Comment