Mixed microphone voice and local sound
Hi everyone, I'm building an app where users can rhyme with each other.
And in a rhyme battle when you go to perform your rhyme it always plays an instrumental beat.
I would like that when it's the user's turn to rhyme, they can play an instrumental beat and it comes out along with their voice on the mic.
My question is if anyone would know how I can mix the user's voice along with the beat and output on the same audio channel, giving the necessary synchronization for the rhyme to come out with quality.
Could someone give a north on how to do this with Daily.js?
Answers
-
(FYI, most of these links have pretty NSFW language)
I've done a few projects here and there with shared music performance and WebRTC. In apps like this, it's all about the latency.
Even though WebRTC is very low latency between participants, the delays can start adding up depending on where the audio is coming from. If quality of performance is what you're going for, and you're OK with swapping beats between rounds like 8 Mile, you can use an AudioContext to play the beat client-side and combine it with the mic. So if Alice and Bob are battling, for round 1, Alice is playing the beat locally and rhyming over it, and that combined audio track gets sent to Bob. Then, when it's Bob's turn, his browser is playing the beat and combining it with his rhymes, and sending that to Alice.
If you need to go back and forth very quickly, that can be tricky. (Sorry.) You might be able to do that by playing the beat from the server and letting each participant rhyme over it that way. Daily has a beta feature called Remote Media Player that you could use for something like that. It's a way to play a video (or live stream) directly into a call instead of someone having to screenshare it, for example.
If you decide to try Remote Media Player, you could also use a cloud recording to get a sense of how the latency looks at the server.
1 -
On second thought, I don't love the Remote Media Player approach. Here's a latency test that I definitely won't regret making publicly available:
6
