write multiprecision binary file with matlab -
i write table 1 integer followed 3 doubles in binary format. of course of study can loop
for i=1:sz fwrite(fid, integ(i), 'int'); fwrite(fid, doubl(i,:), 'double'); end
but quite slow arrays few 1000000 entries. efficient way handle (without having write .mex)?
unfortunately must maintain [int32 float64 float64 float64] format, since file format specification used program.
edit: fastest way , respect exact order , type of variable convert table of nx3 double
array of (2xn)x3 int32
, reshape , concatenate arrays write in 1 go.
outfile4 = 'test1.bin' ; tic4 = tic ; % // reshape table2write = int32(zeros(2*ncol+1,npt)) ; table2write(1,:) = integ.' ; %' k=1:ncol ixline = (k-1)*2+2 ; table2write( ixline:ixline+1 , : ) = reshape( typecast(doubl(:,k),'int32') , 2 , [] ) ; end % // write fid = fopen( outfile4 , 'w' ) ; count = fwrite(fid , table2write , 'int32' ) ; fclose( fid ) ; elapsed4 = toc(tic4)
which result in:
elapsed4 = 0.794346687070910
read below see definition of test variables , faster method deform array
original answer: if can afford reorganize file, can gain tremendous amount of time.
consider next example:
outfile1 = 'e:\temp\z_todelete\test1.bin' ; outfile2 = 'e:\temp\z_todelete\test2.bin' ; npt = 0.5e6 ; integ = int32( randi(32000,npt,1) ) ; doubl = rand(npt,3) ; %% // write file mixed precision tic1 = tic ; fid = fopen( outfile1 , 'w' ) ; k = 1:npt fwrite(fid, integ(k), 'int'); fwrite(fid, doubl(k,:), 'double'); end fclose( fid ) ; elapsed1 = toc(tic1) %% // write file sequentially tic2 = tic ; fid = fopen( outfile2 , 'w' ) ; fwrite(fid, integ, 'int'); fwrite(fid, doubl, 'double'); fclose( fid ) ; elapsed2 = toc(tic2)
on system, output:
elapsed1 = 19.7780466501241 elapsed2 = 0.0309073378234669
so letting matlab handle writing of total arrays, 1 precision @ time extremely more efficient specifying line line write.
the downside reading of 1 single record saved file may little more complex, can write function given index go read integer
, skip rest of them, read 3xdoubles
.
if cannot afford multiplex/demultiplex data, can consider converting int
double
, writing total array:
tic3 = tic ; = [double(integ) doubl] ; fid = fopen( outfile2 , 'w' ) ; fwrite(fid, a, 'double'); fclose( fid ) ; elapsed3 = toc(tic3)
this still lot faster initial "mixed precision" solution
elapsed3 = 0.483094789081886
it take less time convert them integer when read them spent writing mixed precision values. downside of method slight increment in file size (~ 14%).
matlab binary fwrite
No comments:
Post a Comment