Wednesday 15 August 2012

python - Return map results as keyed dictionary -



python - Return map results as keyed dictionary -

in python 2.7 have dictionary big numpy matrices, keyed name of matrix:

import multiprocessing mp import numpy np names = ['a', 'b', 'c'] size = 40 matrices = {k:np.random.random([size,size]) k in names}

i want run matrix algebra on each matrix, using multiprocessing library's pool.map which, according documention, parallel equivalent of normal python map (so if can built-in map, can pool.map too...)

def hard_maths(matrix): homecoming np.dot(np.linalg.inv(matrix), matrix).round(0) pool = mp.pool() results = pool.map(hard_maths, matrices.itervalues()) pool.close() pool.join()

this gets me results list.

but how can matrix (a, b or c) produced result? in ideal world, results end dictionary keyed same keys matrices , values beingness results.

note: don't want have pass entire matrices list function each time, creates big info overhead swamps effects of multiprocessing.

just pass name value, homecoming both:

def hard_maths(name_matrix): name, matrix = name_matrix homecoming name, np.dot(np.linalg.inv(matrix), matrix).round(0) pool = mp.pool() results = pool.map(hard_maths, matrices.iteritems()) pool.close() pool.join()

on linux, multiprocessing implemented forking. long matrix exists when pool created, in kid process space , can cut down overhead passing name

def hard_maths(name): homecoming name, np.dot(np.linalg.inv(matricies[name]), matrix).round(0) pool = mp.pool() results = pool.map(hard_maths, matrices.iterkeys()) pool.close() pool.join()

python python-2.7 map

No comments:

Post a Comment