Replace Audio with External audio files

When to consider replacing Audio on a Streem call recording?

By default, Streem videos will use Audio from the mobile device microphone and Expert microphone (typically desktop). This native audio recording suffices for most users.

The exception is typically where Streem Customers in Call Center environments are already in direct communication with a consumer and disconnecting the call to perform a Streem call can result in loss of information or repeat calls in case the user has trouble initiating a session.

Example Scenarios

1. Experts is on a call with the consumer using the Same device the consumer will use for the Streem call. The phone call is using the microphone from the device, but Recording may still contain Expert audio. Resulting recording has bad or incomplete audio. Expert needs the Call center audio to be inserted in the video to ensure the conversation is retained.

2. Experts is on a call with the consumer and consumer will use an alternate mobile device for the Streem call. The Streem call is NOT using the microphone from the device, but consumer is engaged with Call Center agent on the phone so Audio will not be clear or will be missing Expert side of discussion. Expert needs the Call center audio to be inserted in the video to ensure the conversation is retained.

Your Expert and Consumer stay connected via phone and still use Streem video calls.
Your Experts retain visibility into the success of the consumer being invited. Consumers who have difficulties starting a Streem call be re-invited or assisted without requiring a second phone call.
Your call center audio is used as a source of truth for the Expert queues.

🚧

Make sure you test in Sandbox before Production

Reach out to your account representative if you need Sandbox (eg.. ENV: 'sandbox-us)access. All dev testing should occur in that environment while prepping for audio replacement.

While Replacing audio on a Streem video is made easy through our API, reverting back to the original audio is incredibly difficult. Developers should consider audio replacement as permanent and irreversible. A video file may have it's audio replaced several times and this would be the case on each. once replaced, the prior audio is no longer available.

Replacing Streem recording audio with External audio recording

Prerequisites:
*** The process to Replace a recording with an external audio file requires that a file be downloaded from an API or other accessible URL provided to the Streem audio/video muxing system.
The URL provided to Streem for your replacement audio must be Internet accessible and should be a Signed-URL to preserve security and Privacy of your Consumer's data. The Streem API does NOT accept a blanket location or secondary auth credentials, only direct direct Signed-URLs will be accepted.

*** Audio Files are only accepted in WAV format. We will consider adding new media types by request.

*** You MUST know the Date, Time, Timezone of the audio file start. This is critical for calculating where your Streem video will start in the conversation. Streem stores information using UTC server time so calculation across timezones may be needed as part of your audio replacement request.

How To

POST /v1/rooms/rm_1I2RN0RuR6YhDDO5QFfAYu/artifacts/external-audio
See documentation here: TBD

Post Body
{
"filename": "test_audio_1.wav",
"room_sid": "{{room_sid}}",
"codec": "h264",
"start_time": "2021-03-01T17:20:06.0448Z",
"audio_file_url": "{{audio_file_url}}"
}

PARAMETERS

filename: This is required and must end in a .wav extension

room_sid: With out this value your call does not know where to look for the video media

codec: currently unused and can be excluded

start_time: There are two use cases supported for muxing and these relate to the start_time parameter:

• The video starts at the same time as the audio file. The start_time should be the same as started_recording_at in the video recording track. You can get this from GET /v1/rooms/{room_sid}/artifacts
• The video will be paired with a longer audio track and you need to dictate where in the audio the data audio/video should sync. The start_time should be started_recording_at from the video recording track MINUS the number of seconds into the audio the video should start. The muxer will calculate this offset and create an audio clip that is synced to the video recording track. If the audio is longer than the video it will clip the audio at the start and end of the video.

audio_file_url: This is the SignedURL where Streem can collect your intended audio replacement file

Validating your results

Streem will provide access to the default audio and video that was captured during the Streem Call.
When Muxing is complete, Streem will replace the recording in your call logs with the new video, containing new audio.

For audit or Development purposes (not needed for validation of each replacement) you can place a call to GET /v1/rooms/{room_sid}/artifacts
Search for "type": "RECORDING" with "recording_source": "SOURCE_MUXER"
The Status of a successful audio replacement will carry "status": "STATUS_FULFILLED"