Ephemeral tokens

[mahomedalid]

In the OpenAI documentation there is a mention about ephemeral tokens, this is when we do streaming from the web we use a token that has a validity of 1min, hence this token is generated by the backend which has the real key.

See: https://platform.openai.com/docs/guides/realtime-webrtc#creating-an-ephemeral-token

I am trying to see if Azure OpenAI supports this endpoint (v1/realtime/sessions) but I have not find any information. Neither the code has a hint, or at least one that I can find.

Do someone has any insights?

[luisquintanilla]

cc: @trrwilson do you know?

[kareemkudus]

Looking for the same information!

I believe that this repo and related ones (aoai-realtime-multi-assistants) cannot be deployed live as is because keys are exposed on the client side.

[mahomedalid]

@kareemkudus I see this example in dotnet:

https://github.com/Azure-Samples/aoai-realtime-audio-sdk/blob/5e9fff2040de4657432042cdb7cf553c1076a008/samples/middle-tier/dotnet-aspnetcore-mvc/README.md

Probably we would need to implement some kind of proxy (websocket in the server) for RT in nodejs. thoughts @trrwilson ?

[kareemkudus]

Thanks @mahomedalid , that's helpful!

I found this OpenAI (not Azure) example helpful as well, for understanding how we need to set this up: https://github.com/openai/openai-realtime-agents

As you can see, the authentication token is created on the server, and then passed back to the client (/api/session/route.ts)

If I am understanding correctly, we will need to use the same sort of configuration, but use DefaultAzureCredential for authentication (as was done in the example you sent), rather than a call to https://api.openai.com/v1/realtime/sessions to get an ephermal token, as in the OpenAI example I sent here?

[kareemkudus]

@mahomedalid I took a closer look, it looks like the example you sent does not involve the use of ephemeral tokens...It doesn't seem that Azure supports ephemeral tokens at this time, so I don't think we will be able to follow the design pattern in that OpenAI example I sent.

[mahomedalid]

@kareemkudus correct, that approach does not use ephemeral token, this is how it works:

I am not aware of any example in nodejs of such backend tho, and Idk how complex could be.

[kareemkudus]

Thanks!

I'm not either...I'm also reluctant to switch to this sort of architecture when OpenAI is recommending WebRTC (rather than websockets) for connection directly from a client to realtime API

[mahomedalid]

No news on this?

[kareemkudus]

I haven't seen anything, any ideas who we might get in touch with?

[juliannicolas90]

has anyone been able to solve this?

[Devon-Henry]

I managed to get a working WebRTC demo on a Next JS app using ephemeral tokens. The current Microsoft implementation is in Preview. This doc should help! (not using this library)

https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/realtime-audio-webrtc