Skip to content
Merged
87 changes: 87 additions & 0 deletions temporal-spring-ai/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,63 @@ public class MyTools {

Auto-detected and executed as Nexus operations, similar to activity stubs.

## Migrating from plain Spring AI

The plugin is designed so that bringing an existing Spring AI service onto Temporal is a localized change. Outside Temporal, you probably have something like:

```java
@Service
class AssistantService {
private final ChatClient chatClient;

AssistantService(ChatModel chatModel) {
this.chatClient = ChatClient.builder(chatModel)
.defaultSystem("You are a helpful assistant.")
.defaultTools(new WeatherTools(), new MyTools())
.build();
}

String respond(String goal) {
return chatClient.prompt().user(goal).call().content();
}
}
```

Inside a Temporal Workflow it becomes:

```java
@WorkflowInterface
interface AssistantWorkflow { @WorkflowMethod String respond(String goal); }

class AssistantWorkflowImpl implements AssistantWorkflow {
private final ChatClient chatClient;

@WorkflowInit
AssistantWorkflowImpl(String goal) {
WeatherActivity weather = Workflow.newActivityStub(WeatherActivity.class, opts);
this.chatClient = TemporalChatClient.builder(ActivityChatModel.forDefault())
.defaultSystem("You are a helpful assistant.")
.defaultTools(weather, new MyTools())
.build();
}

@Override
public String respond(String goal) {
return chatClient.prompt().user(goal).call().content();
}
}
```

Three substitutions:

| Outside Temporal | Inside a Temporal workflow |
|---|---|
| `ChatModel chatModel` (injected) | `ActivityChatModel.forDefault()` |
| `ChatClient.builder(chatModel)` | `TemporalChatClient.builder(activityChatModel)` |
| `new WeatherTools()` for a plain POJO tool | `Workflow.newActivityStub(WeatherActivity.class, ...)` for a durable tool |

Plain `@Tool` POJOs, `@SideEffectTool`-annotated classes, and Nexus service stubs all work the same way — see **Tool Types** above.

## Media in messages

If you attach media (images, audio, etc.) to a `UserMessage` or an `AssistantMessage`, prefer passing it by URI rather than raw bytes:
Expand All @@ -130,6 +187,36 @@ Raw `byte[]` media gets serialized into every chat activity's input *and* result

Override the cap by setting the system property `io.temporal.springai.maxMediaBytes` before your worker starts (pass a positive integer; `0` disables the check). For anything larger than a small thumbnail, the URI route is the right answer — have an activity write the bytes to blob storage, then pass only the URL into the conversation.

## Activity options and retry behavior

`ActivityChatModel.forDefault()` and `ActivityChatModel.forModel(name)` create the chat activity stub with sensible defaults: a 2-minute start-to-close timeout, 3 attempts, and `org.springframework.ai.retry.NonTransientAiException` + `java.lang.IllegalArgumentException` classified as non-retryable so a bad API key or invalid prompt fails fast.

Override with `ActivityChatModel.forModel(name, ActivityOptions)`:

```java
ActivityOptions opts = ActivityOptions.newBuilder(ActivityChatModel.defaultActivityOptions())
.setStartToCloseTimeout(Duration.ofMinutes(10))
.setTaskQueue("reasoning-models")
.build();
ActivityChatModel chatModel = ActivityChatModel.forModel("reasoning", opts);
```

For repeated per-model overrides, declare a `ChatModelActivityOptions` bean and auto-configuration wires the map into the plugin. See that class's javadoc for the pattern.

`ActivityMcpClient.create()` / `create(ActivityOptions)` behave the same way with a 30-second default timeout.

## Known limitations

- **Streaming (`chatClient.stream(...)`)** — not currently supported. Use `.call()` instead.
- **`defaultToolContext(Map<String, Object>)`** — not supported; tool context holds mutable state that can't safely cross the activity boundary. Pass required context as activity parameters or workflow state.
- **Child workflow stubs as tools** — not supported. Wrap a plain `@Tool` method that starts the child workflow via `Workflow.newChildWorkflowStub(...)` and call through to it yourself.
- **Media `byte[]` size** — inline bytes are capped at 1 MiB per payload (see "Media in messages" above). Prefer URI-based media.
- **Provider-specific `ChatOptions` via `ChatClient.defaultOptions(...)`** — works as long as your `ChatOptions` subclass overrides `copy()` to return its own type (every real provider class does this). A subclass inheriting the default `copy()` loses its identity before the plugin sees it — same behavior as outside Temporal.

## Observability

`TemporalChatClient.builder(chatModel, observationRegistry, customConvention)` accepts a Micrometer `ObservationRegistry` for Spring AI-side chat client metrics. Temporal-side metrics (activity durations, retries) are emitted by the SDK's `MetricsScope` — see the [Temporal Java SDK observability docs](https://docs.temporal.io/develop/java/observability) for how to wire an OpenTelemetry or Prometheus exporter onto your workers. The two layers compose: Spring AI observations cover what the caller does; Temporal metrics cover what the scheduled activity does.

## Optional Integrations

Auto-configured when their dependencies are on the classpath:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,8 @@ public class ChatModelActivityImpl implements ChatModelActivity {
* @param chatModel the chat model to use
*/
public ChatModelActivityImpl(ChatModel chatModel) {
this.chatModels = Map.of("default", chatModel);
this.defaultModelName = "default";
this.chatModels = Map.of(ChatModelTypes.DEFAULT_MODEL_NAME, chatModel);
this.defaultModelName = ChatModelTypes.DEFAULT_MODEL_NAME;
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,14 @@
*/
public final class ChatModelTypes {

/**
* The name used for the default chat model when no {@code modelName} is specified on an activity
* input or when {@link io.temporal.springai.model.ActivityChatModel#forDefault()} is called.
* Lives here rather than on {@code SpringAiPlugin} so both the activity impl and the plugin can
* reference it without the activity package importing the plugin package.
*/
public static final String DEFAULT_MODEL_NAME = "default";

/**
* Maximum size, in bytes, of a single {@link MediaContent#data()} byte array carried across the
* chat activity boundary. Bytes above this threshold land inside workflow history events, which
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

import io.temporal.common.SimplePlugin;
import io.temporal.springai.activity.ChatModelActivityImpl;
import io.temporal.springai.model.ChatModelTypes;
import io.temporal.worker.Worker;
import java.util.Collections;
import java.util.LinkedHashMap;
Expand Down Expand Up @@ -43,9 +44,6 @@ public class SpringAiPlugin extends SimplePlugin {

private static final Logger log = LoggerFactory.getLogger(SpringAiPlugin.class);

/** The name used for the default chat model when none is specified. */
public static final String DEFAULT_MODEL_NAME = "default";

private final Map<String, ChatModel> chatModels;
private final String defaultModelName;

Expand All @@ -56,8 +54,8 @@ public class SpringAiPlugin extends SimplePlugin {
*/
public SpringAiPlugin(ChatModel chatModel) {
super("io.temporal.spring-ai");
this.chatModels = Map.of(DEFAULT_MODEL_NAME, chatModel);
this.defaultModelName = DEFAULT_MODEL_NAME;
this.chatModels = Map.of(ChatModelTypes.DEFAULT_MODEL_NAME, chatModel);
this.defaultModelName = ChatModelTypes.DEFAULT_MODEL_NAME;
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import io.temporal.springai.activity.ChatModelActivityImpl;
import io.temporal.springai.activity.EmbeddingModelActivityImpl;
import io.temporal.springai.activity.VectorStoreActivityImpl;
import io.temporal.springai.model.ChatModelTypes;
import io.temporal.worker.Worker;
import java.util.*;
import java.util.stream.Collectors;
Expand Down Expand Up @@ -92,7 +93,7 @@ void singleModelConstructor_usesDefaultModelName() {
ChatModel chatModel = mock(ChatModel.class);
SpringAiPlugin plugin = new SpringAiPlugin(chatModel);

assertEquals(SpringAiPlugin.DEFAULT_MODEL_NAME, plugin.getDefaultModelName());
assertEquals(ChatModelTypes.DEFAULT_MODEL_NAME, plugin.getDefaultModelName());
assertSame(chatModel, plugin.getChatModel());
}

Expand Down
Loading