Gemini supports the OpenAI API now. Did anyone try to access it using simple-openai?

[the-gigi]

See https://developers.googleblog.com/en/gemini-is-now-accessible-from-the-openai-library/

It will be very cool if simple-openai can add Gemini to its supported providers.

According to the example, all it takes is a different base URL and model name. No complex adaptation like Azure OpenAI.

[the-gigi]

I plan to try it in the next couple of weeks. I'll report back here and if it works as expected it can be added to the documentation as yet another supported provider.

[the-gigi]

At the moment I'm unable to use simple-openai as is to access Gemini due to BaseSimpleOpenAI constructor being package-private.
See #247

[the-gigi]

There is one more issue that is different with Gemini. Its API key is a token that you need to refresh every hour. Currently, when you create an OpenAI instance it takes an API key as a string and uses it for as long as this instance is alive.

This can be handled by the user by creating a new instance OpenAI instance periodically. But, this is pretty cumbersome and potentially requires a lot of changes in many call sites if you call site just receive an OpenAI instance at initialization and then just use it.

The API key is converted to an auth header passed to CleverClient

I can see several ways to address this concern:

1. add an apiKey setter that will create a new CleverClient, which is identical to the existing one except the auth header contains the new API key (no changes need to CleverClient).
2. similar, but also add updateHeaders method to CleverClient that will allow updating the auth header of the existing CleverClient without re-creating a whole new CleverClient (requires changes to CleverClient
3. Use the existing requestInterceptor (requires no changes to somple-openai or cleverclient)

I think that using the request interceptor is a valid approach unless this use case of a dynamci API key is something that simple-openai should support as first-class citizen.

Note that specifically for Gemini and request interceptor is required regradless because it need to strip the "/v1" version that simple-openai automatically attaches to the base url for some calls. So, extending this request interceptor to handle the auth header too doesn't add a lot of complexity.

@sashirestela WDYT?

[the-gigi]

I see you're working on a PR for improving support for multiple providers:
#246

Please consider support for dynamic API key 🙏

[sashirestela]

@the-gigi Thanks for your research about Gemini!

3. Use the existing requestInterceptor (requires no changes to somple-openai or cleverclient)

I think that using the request interceptor is a valid approach unless this use case of a dynamci API key is something that simple-openai should support as first-class citizen.

Note that specifically for Gemini and request interceptor is required regradless because it need to strip the "/v1" version that simple-openai automatically attaches to the base url for some calls. So, extending this request interceptor to handle the auth header too doesn't add a lot of complexity.

Agree, Request Interceptor is the most appropriate approach for this new provider.

Please consider support for dynamic API key 🙏

Request Interceptor will be the approach to handle this particular Gemini functionality, no additional support will be included.

[the-gigi]

@sashirestela I was able to implement the support for Gemini in a fork. This includes the dynamic API key handling with no changes to the core classes, so core simple-openai doesn't need to provide special support for dynamic API keys.

The SimpleOpenAIGemini class takes Supplier<String> apiKeyProvider as an argument, instead of a fixed String apiKey like other providers. Then in the requestInterceptor it invokes it to get a fresh token and setting the auth header for the request.

I also implemented ChatGeminiDemo. Everything works except for

        /* requires GCS file path. Didn't test */
        //demo.addTitleAction("Call Chat with Vision (External image)", demo::demoCallChatWithVisionExternalImage);

This is due to requirement by Gemini to use Google cloud storage files, so the wikipedia URL doesn't work.

See the-gigi@40fe114

Let me know if you want me to open a PR, or you can just use it as inspiration.

[sashirestela]

@the-gigi It is awesome that you have verified the feasibility of the Gemini API with your POC.

• Could you create an issue labelled as enhancement for this, please?
• Any PR for this should be worked after the next release and taking the changes related to providers (I have even more changes in progress). The next release should be delivered today in the next hours.

[the-gigi]

Done. #248

I don't think I can label issues.

[the-gigi]

@sashirestela I rebased my fork on 3.13.0 and updated the OpenAIGemini + ChatGeminiDemo classes to the new approach. Let me know if it's OK to open a PR

the-gigi@87cdaaa

[sashirestela]

Please, rebase to release_3.14.0 which has the latest refactor to Demo code structure. Are you sure that Gemini Vertex implemnents all the services? If that is the case, you should add Demo code for each of them. Take as a reference the Mistral implementation demo code.

[the-gigi]

Oh, wow. You move fast :-) I'll rebase to 3.14.0 and check out the Mistral demo.

[the-gigi]

@sashirestela I rebased my fork on release_3.14.0

I also simplified it to only support chat completion for now.

main...the-gigi:simple-openai:main