javascript - Send MediaStream object with Web Audio effects over PeerConnection -

javascript - Send MediaStream object with Web Audio effects over PeerConnection -

i'm trying send audio, obtained getusermedia() , altered web sound api, on peerconnection webrtc. web sound api , webrtc seem have ability i'm having problem understanding how can done. within web sound api, audiocontext object contains method createmediastreamsource(), provides way connect mediastream obtained getusermedia(). also, there createmediastreamdestination() method, seems homecoming object stream attribute.

i'm getting both sound , video getusermedia() method. i'm having problem how pass stream object (with both sound , video) methods (ex: createmediastreamsource())? first need extract, somehow, sound stream (getaudiotracks) , find way combine video? or pass , leaves video unaffected? can sound altered 1 time (before added peerconnection)?

the createmediastreamsource() method takes mediastream object parameter, takes first audiomediastreamtrack object used sound source. can used mediastream object received getusermedia() method if object contains both sound , video. instance:

var source = context.createmediastreamsource(localstream);

where "context", in above code, audiocontext object , "localstream" mediastream object obtained getusermedia(). createmediastreamdestination() method creates destination node object has mediastream object within "stream" attribute. mediastream object contains 1 audiomediastreamtrack (even if input stream source contained both sound , video or numerous sound tracks): altered version of track obtained stream within source. instance:

var destination = context.createmediastreamdestination();

now, before can access stream attribute of newly created destination variable, must create sound graph linking nodes together. example, lets assume have biquadfilter node named filter:

source.connect(filter); filter.connect(destination);

then, can obtain stream attribute destination variable. , can used add together peerconnection object send remote peer:

peerconnection.addstream(destination.stream);

note: stream attribute contains mediastream object altered audiomediastreamtrack. therefore, no video. if want video sent well, you'll have add together track stream object contains video track:

var audiotracks = destination.stream.getaudiotracks(); var track = audiotracks[0]; //stream contains 1 sound track localstream.addtrack(track); peerconnection.addstream(localstream);

keep in mind, addtrack method not add together track if there 1 in mediastream object same id. therefore, may have first remove track obtained in source node.

the sound should able altered @ time adjusting values within intermediate nodes (between source , destination). because stream passes through nodes before beingness sent other peer. check out example on dynamically changing effect on recorded sound (should same stream). note: have not tested code yet. though works in theory, there may cross browser issues since both web sound api , webrtc in working draft stages , not yet standardized. assume work in mozilla firefox , google chrome.

reference

media capture , streams web sound api

javascript html5 webrtc web-audio