Monday 15 September 2014

write multiprecision binary file with matlab -



write multiprecision binary file with matlab -

i write table 1 integer followed 3 doubles in binary format. of course of study can loop

for i=1:sz fwrite(fid, integ(i), 'int'); fwrite(fid, doubl(i,:), 'double'); end

but quite slow arrays few 1000000 entries. efficient way handle (without having write .mex)?

unfortunately must maintain [int32 float64 float64 float64] format, since file format specification used program.

edit: fastest way , respect exact order , type of variable convert table of nx3 double array of (2xn)x3 int32, reshape , concatenate arrays write in 1 go.

outfile4 = 'test1.bin' ; tic4 = tic ; % // reshape table2write = int32(zeros(2*ncol+1,npt)) ; table2write(1,:) = integ.' ; %' k=1:ncol ixline = (k-1)*2+2 ; table2write( ixline:ixline+1 , : ) = reshape( typecast(doubl(:,k),'int32') , 2 , [] ) ; end % // write fid = fopen( outfile4 , 'w' ) ; count = fwrite(fid , table2write , 'int32' ) ; fclose( fid ) ; elapsed4 = toc(tic4)

which result in:

elapsed4 = 0.794346687070910

read below see definition of test variables , faster method deform array

original answer: if can afford reorganize file, can gain tremendous amount of time.

consider next example:

outfile1 = 'e:\temp\z_todelete\test1.bin' ; outfile2 = 'e:\temp\z_todelete\test2.bin' ; npt = 0.5e6 ; integ = int32( randi(32000,npt,1) ) ; doubl = rand(npt,3) ; %% // write file mixed precision tic1 = tic ; fid = fopen( outfile1 , 'w' ) ; k = 1:npt fwrite(fid, integ(k), 'int'); fwrite(fid, doubl(k,:), 'double'); end fclose( fid ) ; elapsed1 = toc(tic1) %% // write file sequentially tic2 = tic ; fid = fopen( outfile2 , 'w' ) ; fwrite(fid, integ, 'int'); fwrite(fid, doubl, 'double'); fclose( fid ) ; elapsed2 = toc(tic2)

on system, output:

elapsed1 = 19.7780466501241 elapsed2 = 0.0309073378234669

so letting matlab handle writing of total arrays, 1 precision @ time extremely more efficient specifying line line write.

the downside reading of 1 single record saved file may little more complex, can write function given index go read integer, skip rest of them, read 3xdoubles.

if cannot afford multiplex/demultiplex data, can consider converting int double , writing total array:

tic3 = tic ; = [double(integ) doubl] ; fid = fopen( outfile2 , 'w' ) ; fwrite(fid, a, 'double'); fclose( fid ) ; elapsed3 = toc(tic3)

this still lot faster initial "mixed precision" solution

elapsed3 = 0.483094789081886

it take less time convert them integer when read them spent writing mixed precision values. downside of method slight increment in file size (~ 14%).

matlab binary fwrite

No comments:

Post a Comment