xef.ai: Modern AI for Scala and Kotlin

xef.ai: Modern AI for Scala and Kotlin

In recent months, the world of modern AI has changed drastically. The impact of the general availability of Large Language Models, like ChatGPT, is hard to overstate. These only complement existing techniques in image generation, classification, speech recognition, and many others. At the time of writing this post, Hugging Face Hub, the leading repository of models, has more than 200,000 of them! The challenge is how to integrate these techniques into a broader system. AI is not a goal in itself but a tool in your journey to offer the best experience to your customers.

The possibilities are endless: Summarize the most important events in the customer’s account. Generate queries to the database from a natural language description. Help customer service provide a sequence of steps to resolve an issue. Identify the important parts of a PDF invoice. These are tasks where AI can make an impact now.

At Xebia, we’re eager to explore the possibilities of this new world. However, the ecosystem is quite fragmented, usually requiring a different library for each different service. This is where xef.ai enters the game: a project whose goal is to offer a convenient interface to these modern AI techniques so your team can quickly move from ideas to proofs of concepts to integrating AI into your business processes. Our initial release targets Kotlin and Scala, with more languages to come in the future.

xef.ai primer

To give a more concrete example, this is all the code required to query OpenAI’s ChatGPT. The ai block initializes all required connections so you can directly prompt a message.

fun whatToWear(weather: String): List<String> = ai {
promptMessage("Knowing that today is $weather, what clothes should I wear?")
}.getOrThrow()

Language models are very powerful but lack knowledge about current events or any information within your system. To give the best results, your AI pipelines need to be aware of other information. With xef.ai, you can always enlarge the context with the required information, which is considered in subsequent queries. Here’s a variation of the code above, where a regular Internet search is performed to find the weather forecast before going to the AI.

fun whatToWear(place: String): List<String> = ai {
context(search("Weather in $place")) {
promptMessage("Knowing this forecast, what clothes should I wear?")
}
}

Prompt engineering, that is, how to write the best queries to obtain the desired result, is a nascent field. Websites like Learn Prompting provide useful tips. With the convenience of xef.ai, all your energy can be spent on improving these prompts instead of fighting with tons of service APIs.

Deserialization support

One of the pain points of Language Models is that they give their answers in plain text. As a result, an additional step is usually required to convert that answer into a value or object in the form required by your system. xef.ai tackles that problem and provides direct deserialization from answers into your own classes.

@Serializable data class Book(val title: String, val author: String)

fun books(topic: String): List<Book> = ai {
prompt("Give me a selection of books about $topic")
}.getOrThrow()

Currently, we build special prompts that guide language models into producing structured output, which we then try to parse. Whereas this works most of the time, we’re investigating more powerful approaches like ReLLM and jsonformer.

Future features

Our goal with xef.ai is to become a hub for modern AI in the JVM ecosystem. We are learning from sibling projects like LangChain, but we’re trying to keep a minimal and idiomatic interface.

We’ve started integrating with Large Language Models like ChatGPT and Dolly, but we intend to include other areas like image generation, translation, or speech recognition. xef.ai is open source, so we’re happy to hear any feedback, issues, or features you may want to see.

Happy hacking with modern AI!