python - Return map results as keyed dictionary -
in python 2.7 have dictionary big numpy
matrices, keyed name of matrix:
import multiprocessing mp import numpy np names = ['a', 'b', 'c'] size = 40 matrices = {k:np.random.random([size,size]) k in names}
i want run matrix algebra on each matrix, using multiprocessing
library's pool.map
which, according documention, parallel equivalent of normal python map
(so if can built-in map, can pool.map
too...)
def hard_maths(matrix): homecoming np.dot(np.linalg.inv(matrix), matrix).round(0) pool = mp.pool() results = pool.map(hard_maths, matrices.itervalues()) pool.close() pool.join()
this gets me results list
.
but how can matrix (a
, b
or c
) produced result? in ideal world, results
end dictionary keyed same keys matrices
, values beingness results.
note: don't want have pass entire matrices
list function each time, creates big info overhead swamps effects of multiprocessing.
just pass name value, homecoming both:
def hard_maths(name_matrix): name, matrix = name_matrix homecoming name, np.dot(np.linalg.inv(matrix), matrix).round(0) pool = mp.pool() results = pool.map(hard_maths, matrices.iteritems()) pool.close() pool.join()
on linux, multiprocessing implemented forking. long matrix exists when pool created, in kid process space , can cut down overhead passing name
def hard_maths(name): homecoming name, np.dot(np.linalg.inv(matricies[name]), matrix).round(0) pool = mp.pool() results = pool.map(hard_maths, matrices.iterkeys()) pool.close() pool.join()
python python-2.7 map
No comments:
Post a Comment