transcribe() on a 20-second clip”. The days get lost later: the file is 90 MB and the endpoint refuses it, the summary comes back wrapped in a sentence the JSON parser chokes on, and nobody has worked out what an hour of meetings costs.
This article builds the whole chain — recording, transcript, Minutes record, spoken read-back — and spends most of its length on what goes wrong. It starts from the smallest working call and assumes nothing about audio APIs, only that you have met Spring AI’s ChatClient (the ChatClient primer covers it). The code is a module of the companion repository, spring-ai/audio.
Versions this was built and run against. Spring AI 2.0.2 (GA; on Maven Central 22 October 2026 —2.1.0-RC1is a release candidate, not GA), Spring Boot 4.1.2 (GA, same day;4.2.0-RC1is a release candidate), Eclipse Temurin JDK 25.0.5+8, OpenAI Java SDK 4.80.0 (the client Spring AI 2.0.2 uses underneath). Versions were read frommaven-metadata.xml; the API shapes were read from the shipped jars withjavap.
What is real here and what is not. No OpenAI key was available, so no real speech model, transcription model or language model produced any output in this article. The two audio endpoints are a mock HTTP server running locally; the real Spring AI classes and the real OpenAI SDK talk to it over HTTP, so the requests you will see are exactly what they send. The “recording” is thirty synthetic tones, one per scripted sentence, and the mock “transcribes” by decoding each tone’s pitch back into the sentence. The chat model is a stub that replays canned JSON. So: request shapes, status codes, limits and retry behaviour are exercised for real; transcript accuracy, latency and speech quality are not measured at all. Each output block below says which kind it is.
Spring AI treats audio as two small interfaces, and both defaults are deprecated
Speech to text is aTranscriptionModel: you give it an AudioTranscriptionPrompt wrapping a Spring Resource (a file, a byte array) and optional settings, and get back text. Text to speech is a TextToSpeechModel: a TextToSpeechPrompt wrapping a string goes in, bytes of audio come out. Each has an OpenAI implementation, OpenAiAudioTranscriptionModel and OpenAiAudioSpeechModel, and your own code should depend on the interfaces so a different provider can be swapped in. MinutesService does exactly that.
The class names and packages below come from javap on the 2.0.2 jars, and the defaults from a test that prints the constants. This cost nothing to check and is worth doing, because the audio classes live in org.springframework.ai.audio.* while the OpenAI ones stay in org.springframework.ai.openai; a guess from the chat classes lands in the wrong package. Captured in output/01-api-shape.txt:
speech-to-text interface : org.springframework.ai.audio.transcription.TranscriptionModel
speech-to-text OpenAI : org.springframework.ai.openai.OpenAiAudioTranscriptionModel
text-to-speech interface : org.springframework.ai.audio.tts.TextToSpeechModel
text-to-speech OpenAI : org.springframework.ai.openai.OpenAiAudioSpeechModel
transcription DEFAULT_TRANSCRIPTION_MODEL = whisper-1
transcription DEFAULT_RESPONSE_FORMAT = text
speech DEFAULT_SPEECH_MODEL = gpt-4o-mini-tts
speech DEFAULT_VOICE = alloy
speech DEFAULT_RESPONSE_FORMAT = mp3
speech DEFAULT_SPEED = 1.0
both DEFAULT_TIMEOUT = PT1M
both DEFAULT_MAX_RETRIES = 3whisper-1 for transcription and gpt-4o-mini-tts for speech. OpenAI’s own guides, read on 26 October 2026, say whisper-1 (with gpt-4o-transcribe, gpt-4o-mini-transcribe and the diarize model) is deprecated and shuts down on 26 February 2027, that gpt-transcribe is the recommended replacement, and that tts-1, tts-1-hd and gpt-4o-mini-tts shut down on 6 January 2027. So a project that follows the Spring AI defaults today is building on models with a published end date. Name the model explicitly, always.
| Model string | Role | Status per OpenAI, read 26 Oct 2026 | Price |
|---|---|---|---|
gpt-transcribe | transcription | recommended | $0.0045 / minute |
whisper-1 (Spring AI default) | transcription | shuts down 26 Feb 2027 | $0.006 / minute |
gpt-4o-transcribe, gpt-4o-mini-transcribe | transcription | shuts down 26 Feb 2027 | $0.006 / $0.003 per minute |
gpt-4o-mini-tts (Spring AI default), tts-1, tts-1-hd | speech | shuts down 6 Jan 2027 | $0.60 in / $12 audio out per 1M tokens; $15 and $30 per 1M characters |
The speech endpoint has no announced successor. The text-to-speech guide says to migrate togpt-realtime-2.1-minithrough the Realtime API, and adds that it “isn’t a supported model for/v1/audio/speech”. The read-back step in this article uses the Speech API and works until that shutdown date; after it, moving to the Realtime API is a change of workflow, not of a model name. I have not tried the Realtime API, and I did not look for a Spring AI class for it. See the text-to-speech guide.
Going deeper: auto-configuration properties and the OpenAI-compatible servers
With Spring Boot auto-configuration you do not build the models by hand. The property names below are read from spring-configuration-metadata.json in spring-ai-autoconfigure-model-openai 2.0.2: spring.ai.openai.audio.transcription.{api-key, base-url, model, language, prompt, response-format, temperature, timestamp-granularities, max-retries, timeout} and spring.ai.openai.audio.speech.{api-key, base-url, model, voice, response-format, speed, instructions, max-retries, timeout}. The module in this article builds the models in code instead, because the tests point them at a local port.
Setting base-url is also how you aim at any server that implements the same two paths — a self-hosted Whisper server, for instance. The mock in this repository is exactly that, and it is the reason the tests need no key. Note the SDK appends /audio/transcriptions to the base URL, so the base URL ends in /v1.
Spring AI 2.0.2 has no languages or keywords option for gpt-transcribe, only the singular language; the guide says languages replaces language for that model and that you must not send both. I checked the builder with javap; I did not test what the real endpoint does with a lone language, so treat the language=en in the captured request below as a mock demonstration.
Both endpoints are plain HTTP uploads with a hard limit on each side
A beginner’s mental model is enough here. Transcription is one HTTP POST with the audio as a multipart file part; the answer is text. Speech is one HTTP POST with a JSON body holding your text, a voice and a format; the answer is the audio bytes. Neither is a conversation, and neither remembers the previous call, which matters in a moment. The diagram is the shape of the program you are about to read. The two limits in it are the whole difficulty: the transcription endpoint accepts files “up to 25 MB” in the formatsmp3, mp4, mpeg, mpga, m4a, wav, webm (speech-to-text guide), and the speech endpoint takes at most 4096 characters per request (API reference). Everything else in the article is about living with those two numbers.
The smallest call: wrap the bytes in a Resource that has a file name
The first working call is short. The one trap is thattranscribe(Resource) sends the resource’s file name as the multipart file name, and the service decides the format from its extension. A bare ByteArrayResource has no file name, so NamedBytes adds one:
public class NamedBytes extends ByteArrayResource {
private final String filename;
public NamedBytes(byte[] bytes, String filename) {
super(bytes);
this.filename = filename;
}
@Override
public String getFilename() {
return filename;
}
}whisper-1 and once with gpt-transcribe, and prints what the mock received. This is a mock server (output/02-transcribe-small.txt):
model option: whisper-1
request : POST /v1/audio/transcriptions (multipart, 1920524 bytes)
fields : {model=whisper-1, response_format=text}
file part : name=standup.wav type=application/octet-stream bytes=1920044
transcribe(Resource) -> "Okay, let's get started with the Falcon weekly sync. First item is the release date for version two point four."
with language=en and prompt, fields now: {model=whisper-1, language=en, prompt=Falcon weekly sync, response_format=text}
response metadata class: OpenAiAudioTranscriptionResponseMetadataresponse_format is text, so the reply is a bare string and there are no segment timestamps; asking for verbose_json and word timestamps is possible through the options builder but the guide ties timestamp_granularities to whisper-1, the model that is going away. The file part’s content type is application/octet-stream, which tells you the extension is doing all the work. And the second call, with language and prompt set, shows both arriving as ordinary form fields — the prompt field becomes important when the file is split.
What happens when the name is wrong is the first failure mode. Same mock, same bytes, three names (output/02b-filename.txt):
file name standup.wav -> ok: 52 chars
file name standup.flac -> BadRequestException: 400: mock: Invalid file format. Supported formats: [mp3, mp4, mpeg, mpga, m4a, wav, webm]
file name standup -> BadRequestException: 400: mock: Invalid file format. Supported formats: [mp3, mp4, mpeg, mpga, m4a, wav, webm].flac is not in the list, and one with no extension has no format at all. Rename or convert before you upload.
Going deeper: why the file name is the format
The multipart upload carries the bytes and a filename; there is no separate “format” field. Spring AI passes whatever Resource.getFilename() returns. If you read from a Spring MultipartFile, wrap it so the original filename survives; if you read from a database blob, store the extension next to it.
The same mock records the exact request in FakeAudioServer.Seen, which is how the field list above was produced, so you can reuse the class to assert what your own code sends without a network.
A 30 MB recording is refused whole, and an hour of WAV is far past the limit
The recording in the tests is 946 seconds — just under sixteen minutes — as a 16 kHz, mono, 16-bit WAV. That is 32,000 bytes per second, so the file is 30,278,444 bytes, over the limit. Sent in one request to the mock, which enforces 25 MiB (26,214,400 bytes; the documentation says “25 MB” without saying which, so the code budgets 24,000,000 bytes and is under both readings), the Spring AI client throws an exception from the OpenAI SDK, not a Spring one. This is the start of output/03-large-file.txt, from the mock server:1) the whole file in one request (mock limit 26,214,400 bytes):
exception: com.openai.errors.UnexpectedStatusCodeException
message : 413: mock: Maximum content size limit (26214400) exceeded (30278929 bytes read)size of 60 minutes: 16 kHz mono 16-bit WAV = 115.2 MB | 64 kbit/s mp3 = 28.8 MB | 32 kbit/s mp3 = 14.4 MB
minutes that fit in 25,000,000 bytes: WAV 13.0 | 64 kbit/s 52 | 32 kbit/s 104ffmpeg call (ffmpeg -i meeting.wav -ac 1 -b:a 32k meeting.mp3); I did not run that command for this article, so it is documentation-sourced, not executed.
When it is still too big, cut in the middle of a silence, not at a byte count
Some recordings are longer than any sensible bitrate allows. Then you must split them, and the OpenAI guide has one instruction about how: avoid splitting “in the middle of a sentence, which can remove context and reduce accuracy”. The lazy split is every N bytes. The diagram shows why that is the wrong default: with a 24,000,000-byte budget, the blind cut lands at 750.0 seconds, in the middle of sentence 23 (counting from 0). The cut that looks backwards for the longest quiet stretch lands at 733.5 seconds, in the gap before it.cutKind, because a silent fallback hides exactly the bug you are trying to avoid (an excerpt of split; the helper methods are in the file):
public List<Chunk> split(byte[] wav) {
Wav.Info info = Wav.parse(wav);
if (wav.length <= maxBytes) {
return List.of(new Chunk(0, 0, info.seconds(), wav, "none (fits)"));
}
int maxFrames = (int) ((maxBytes - 44) / info.bytesPerFrame());
int win = info.sampleRate() * WINDOW_MS / 1000;
int total = info.frames();
List<Chunk> out = new ArrayList<>();
int start = 0;
while (start < total) {
int limit = Math.min(total, start + maxFrames);
if (limit == total) {
out.add(chunk(out.size(), wav, info, start, total, "end of file"));
break;
}
int cut = quietCut(wav, info, win, start + maxFrames / 2, limit);
String kind = "mid-silence";
if (cut < 0) { cut = limit; kind = "HARD CUT (no silence found)"; }
out.add(chunk(out.size(), wav, info, start, cut, kind));
start = cut;
}
return out;[inaudible]; I chose that to make the damage countable, and a real model would degrade less predictably. From output/04-blind-split.txt:
chunk 0: 0.0 s -> 750.0 s 24,000,000 bytes boundary falls INSIDE sentence 23 ("Everyone should check the new rotation sheet before the end of this week.")
chunk 1: 750.0 s -> 946.2 s 6,278,488 bytes boundary falls at end of file
script lines recovered: 29 of 30; [inaudible] markers: 22) MinutesService.transcribe: budget 24,000,000 bytes per chunk, silence of at least 2.0 s, RMS below 500
chunk 0: 0.0 s -> 733.5 s 23,472,022 bytes cut: mid-silence
chunk 1: 733.5 s -> 946.2 s 6,806,466 bytes cut: end of file
requests the mock saw: 2
- file=part-00-meeting.wav bytes=23,472,022 prompt=(none)
- file=part-01-meeting.wav bytes=6,806,466 prompt="...the on call rotation changes next month."
script lines recovered: 30 of 30, in order: true
[inaudible] markers: 0prompt ending with the tail of the first chunk’s transcript. That is what the guide means by “carrying context from a previous chunk”: the endpoint is stateless, so the only way a name or a half-finished topic crosses the boundary is as text in the next request’s prompt. It is also why the chunks are sent in order, one at a time, and not in parallel. The cost of that choice is wall-clock time; the gain is that a surname spelled correctly in chunk one is likelier to be spelled the same way in chunk two. The mock ignores the prompt, so this run shows the mechanism, not the benefit.
The hard-cut fallback is a warning, not a fix. If a recording has no two-second silence in the window (a call with a constant hum, music under the talk), the chunker cuts at the byte limit and labels the chunk HARD CUT (no silence found). Log that label. The first symptom of an unlucky cut is a dropped or garbled sentence at the seam, and nothing else in the pipeline will tell you.
Going deeper: tuning the chunker, and what this version does not do
Three parameters matter: the byte budget (24,000,000 here), the minimum silence (2.0 s) and the RMS threshold (500 on a 16-bit scale). The test recording has room noise of about ±40 against tones of 8000, so any threshold in between works; real recordings have laughter, keyboards and air conditioning, and the threshold must be tuned on yours. A more robust approach measures the noise floor first and sets the threshold relative to it.
The chunker reads and writes only 16-bit PCM WAV, on purpose: it keeps the logic visible and testable with no native binary. For mp3 or m4a input, decode to WAV first, or let ffmpeg‘s segment muxer do the cutting; I did not run that route here. Two limitations: it keeps the whole recording in memory (a 2-hour WAV is about 230 MB), and it does not overlap chunks. Overlapping by a few seconds and de-duplicating the seam is a common refinement; I did not build or test it.
If you only need speaker-labelled output on long audio, the guide documents a diarization model and a chunking_strategy setting for audio over 30 seconds, but that model is on the same deprecation list and Spring AI’s builder exposes chunkingStrategy and known-speaker options for it. I did not exercise any of that.
Turning the transcript into action items is a typed call that fails in three ordinary ways
With a transcript in hand, the second step is a plain structured-output call, the same mechanism as in the structured output article. You describe what you want as a Java record, andentity(Minutes.class) does two things: it appends a JSON Schema and an instruction to answer in JSON to your prompt, and it parses the reply into the record. An excerpt of Minutes, whose remaining method builds the spoken text:
public record Minutes(String title, List<String> decisions, List<ActionItem> actionItems) {
/** {@code due} is free text ("Friday", "14 November") because a meeting rarely says an ISO date. */
public record ActionItem(String owner, String task, String due) {} public Minutes summarise(String transcript) {
// transcript goes in as the user message itself, not through a {placeholder}: a transcript with
// braces in it would otherwise be read as template variables.
return chat.prompt().system(SYSTEM).user(transcript).call().entity(Minutes.class);
}{placeholder}. A transcript with braces in it, a pasted JSON snippet or a code review, would otherwise be read as template variables. The chat model in the test is a stub replaying a canned reply, so what the output below proves is the prompt Spring AI built and the parse, not what a language model would extract. From output/05-structured-output.txt:
messages sent to the model:
[SYSTEM] 222 chars
[USER] 2799 charsYour response should be in JSON format.
Do not include any explanations, only provide a RFC8259 compliant JSON response following this format without deviation.
Do not include markdown code blocks in your response.
Remove the ```json markdown from the output.
Here is the JSON Schema instance your output must adhere to:parsed: Falcon weekly sync
decisions : 2
action item : Rahul | Fix the login timeout bug | due: before the release on the fourteenth of November
action item : Priya | Send the revised budget to finance | due: Friday
action item : Sana | Open tickets for both export endpoint issues | due: today
spoken form:
Falcon weekly sync. 2 decisions. Ship version 2.4 on the fourteenth of November. The data pipeline migrates to the new cluster next sprint. 3 action items. Rahul: Fix the login timeout bug, due before the release on the fourteenth of November. Priya: Send the revised budget to finance, due Friday. Sana: Open tickets for both export endpoint issues, due today.required, so a prompt that says “if no due date was said, use an empty string” is what keeps the model from inventing one, and why due is a free-text string: a meeting rarely says an ISO date. And the schema is part of the prompt text, appended to the user message, so your system prompt and the transcript share space with it.
Now the failures. Models do not always answer with bare JSON. Three replies a model can plausibly give, fed to the same parser from a stub, from output/05b-bad-replies.txt:
wrapped in a markdown fence with a sentence before it -> StreamReadException: Unrecognized token 'Here': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false...
truncated: closing brace missing -> UnexpectedEndOfInputException: Unexpected end-of-input: expected close marker for Object (start marker at [Source: REDACTED (`StreamReadFeatu...
prose instead of JSON -> StreamReadException: Unrecognized token 'I': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false') ...entity(), not as an empty record, so a caller that does not catch them turns a bad summary into an HTTP 500. These exact texts are stub inputs I wrote to show the exception types; how often a real model produces each is not something this run measured.
The generated JSON Schema, verbatim
From output/05-structured-output.txt (stub model, real Spring AI schema generation):```{
"$schema" : "https://json-schema.org/draft/2020-12/schema",
"type" : "object",
"properties" : {
"actionItems" : {
"type" : "array",
"items" : {
"type" : "object",
"properties" : {
"due" : {
"type" : "string"
},
"owner" : {
"type" : "string"
},
"task" : {
"type" : "string"
}
},
"required" : [ "due", "owner", "task" ],
"additionalProperties" : false
}
},
"decisions" : {
"type" : "array",
"items" : {
"type" : "string"
}
},
"title" : {
"type" : "string"
}
},
"required" : [ "actionItems", "decisions", "title" ],
"additionalProperties" : false
}```Going deeper: retrying a bad reply, and keeping action items honest
Spring AI 2.0 has an advisor that validates the reply against the schema and re-asks on failure; the structured output article shows it, including a captured run where every retry fails and nothing throws. Use it here rather than writing a retry loop around summarise().
Validation proves the shape, not the facts. A model can return a perfectly valid action item for something nobody agreed to. The system prompt here asks for explicitly agreed items only, but the reliable check is a golden set of transcripts with known action items, scored in CI; the testing article builds that. Also remember the transcript can contain instructions (“ignore the above” said aloud by a joker); treat it as data, and see the prompt-injection article before this feeds anything with tools.
The read-back has its own limit, and the model’s output is not always speakable
Text to speech is the simpler half. Minutes.toSpokenText() turns the record into a few plain sentences (no bullets, no markdown), andreadAloud sends it. The test SpeechTest prints the request JSON the mock received, once with all defaults through call(String) and once with explicit options. This is a mock server; the “audio” it returns is a beep whose length follows the text length. From output/06-speech.txt:
speech.call(String) with every default:
request json : {"input":"Three action items.","model":"gpt-4o-mini-tts","voice":"alloy","response_format":"mp3","speed":1.0}
response : 20310 bytes
with TextToSpeechPrompt and options:
request json : {"input":"Rahul: fix the login timeout bug.","model":"gpt-4o-mini-tts","voice":"coral","instructions":"Calm, like a project manager reading notes.","response_format":"wav","speed":1.1}
response : 35244 bytes, starts with RIFF
input of 5000 characters -> BadRequestException: 400: mock: the maximum length of input is 4096 characters, got 5000gpt-4o-mini-tts, voice alloy, mp3, speed 1.0. Voice names are strings in the builder; the guide lists thirteen names for the newer model and a smaller set for tts-1. The instructions field (“speak calmly”) is documented as not working with tts-1 or tts-1-hd, and the mock accepts it from either, so that rule is only enforced by the real service. The last line is the limit: 4096 characters, enforced here by the mock because the API reference states it.
A meeting with thirty action items overruns 4096 characters. splitForSpeech cuts on sentence ends so no clip stops mid-word, from output/06b-speech-split.txt:
text: 11051 characters, 120 sentences
part 0: 4038 characters, ends with " take space."
part 1: 4047 characters, ends with " take space."
part 2: 2964 characters, ends with " take space."wav you can join them by appending the PCM and fixing the header (the Wav class has wrap for that); joining mp3 clips by concatenation usually plays but leaves odd frame boundaries, and I did not test it. OpenAI’s usage policy, quoted in the guide, requires telling end users that the voice is AI-generated.
A retry uploads the whole file again
The OpenAI SDK under Spring AI retries failed requests itself, three times by default (DEFAULT_MAX_RETRIES = 3 in the output earlier). For a chat call that is harmless. For a transcription it means the file goes over the wire again each attempt. The test configures the mock to answer the first two requests with a failure and counts what arrives, for a 120-second clip of 3,840,044 bytes. A rate limit (429) is retried; a bad request (400) is not. This is the mock server, from output/07-retries.txt:
maxRetries=0, server fails the first 2 requests with 429 -> RateLimitException requests: 1, bytes uploaded: 3,840,523
maxRetries=0, server fails the first 2 requests with 400 -> BadRequestException requests: 1, bytes uploaded: 3,840,523
maxRetries=3, server fails the first 2 requests with 429 -> ok requests: 3, bytes uploaded: 11,521,569
maxRetries=3, server fails the first 2 requests with 400 -> BadRequestException requests: 1, bytes uploaded: 3,840,523max-retries and owning the retry yourself, at the chunk level, where you know which chunks already succeeded. And a 400 is never worth retrying: if a chunk is refused for size or format, the next attempt is the same bytes. Whether a failed request is billed on the real service I do not know; the mock cannot say.
An hour of meeting costs about thirty cents, and nearly all of it is the transcription
Cost has three line items, and only one of them is large. With prices as inputs, the formulas are:transcription = minutes x pricePerMinute
summary = tokensIn x in/1e6 + tokensOut x out/1e6 tokensIn = words / 0.75
read-back = characters x pricePerMillionChars / 1e6 (tts-1 family, per character)
| textTokens x textIn/1e6 + audioTokens x audioOut/1e6 (gpt-4o-mini-tts, per token)gpt-transcribe $0.0045 per minute, gpt-4o-transcribe $0.006 and gpt-4o-mini-transcribe $0.003 per minute, from the pricing page; whisper-1 $0.006 per minute and tts-1 $15 per million characters from their model pages; gpt-4o-mini at $0.15 in and $0.60 out per million tokens from its model page, used here as a stand-in chat model (the model page’s price header was ambiguous about standard versus batch pricing, so read the summary column as illustrative). The pricing page did not list whisper-1 or any speech model on that day, which is itself a sign of where they are in their lifecycle. The word and token rates are assumptions, not measurements: 150 spoken words a minute, 0.75 words a token, 600 output tokens, a 120-word read-back. Output from output/08-cost.txt:
assumptions: 150 words/minute spoken, 0.75 words/token, summary output 600 tokens, read-back 120 words
prices : gpt-transcribe $0.0045/min | whisper-1 $0.006/min | gpt-4o-mini $0.15 in / $0.60 out per 1M | tts-1 $15 per 1M chars
minutes words gpt-transc whisper-1 summary tts-1 total(gt)
15 2,250 0.0675 0.0900 0.0008 0.0108 0.0791
30 4,500 0.1350 0.1800 0.0013 0.0108 0.1471
60 9,000 0.2700 0.3600 0.0022 0.0108 0.2830
120 18,000 0.5400 0.7200 0.0040 0.0108 0.5548gpt-transcribe and the summary is two-tenths of a cent. The decision that moves the bill is the transcription model and the minutes you send, not the chat model you pick for the summary. Two practical savings follow: do not transcribe silence (trim the start and end, and long dead gaps), and do not re-transcribe a recording to retry a failed summary — keep the transcript and re-run only the cheap step. The speech read-back at 720 characters is about a cent on the per-character price.
Going deeper: what the table leaves out
Not modelled: the retry multiplier above (if a failed upload is billed, three attempts at a 60-minute recording is three times the line); token-priced transcription (gpt-4o-transcribe also lists per-token prices, with the per-minute figure labelled an estimate); batch discounts; and, if you run gpt-4o-mini-tts, its token pricing, for which the model page lists $0.60 per million text tokens in and $12 per million audio tokens out. Audio-token counts for a given text are not something I measured, which is why the read-back line uses the per-character formula.
The observability article shows how to turn the chat step’s token usage into a cost metric per endpoint. Spring AI also publishes observations for transcription and speech models (the observation packages are in the spring-ai-model jar); I confirmed the classes exist but did not wire them to a meter here.
The whole chain in one test
All three parts together, driven by MeetingEndToEndTest: the 30 MB recording goes in, two requests go to the mock transcription endpoint, the transcript reaches the stub chat model as the user message, the parsed record is spoken, and one speech request goes out. Mock audio endpoints and a stub chat model, from output/09-end-to-end.txt:1) transcribe: 30,278,444 bytes -> 2 request(s) -> 1701 characters
2) summarise : transcript of 292 words reached the chat model as the user message: true
2 action items: [ActionItem[owner=Rahul, task=Fix the login timeout bug, due=before the release], ActionItem[owner=Priya, task=Send the revised budget to finance, due=Friday]]
3) read back : 1 clip(s), 221910 bytes, spoken text: Falcon weekly sync. 1 decision. Ship version 2.4 on the fourteenth of November. 2 action items. Rahul: Fix the login timeout bug, due before the release. Priya: Send the revised budget to finance, due Friday.
speech request: {"input":"Falcon weekly sync. 1 decision. Ship version 2.4 on the fourteenth of November. 2 action items. Rahul: Fix the login timeout bug, due before the release. Priya: Send the revised budget to finance, due Friday.","model":"gpt-4o-mini-tts","voice":"coral","response_format":"wav","speed":1.0}scripts/run-all.sh in the module regenerates every file in output/.
Should you build this? Probably not as your own pipeline if a meeting tool you already use records, transcribes and summarises — the plumbing here is worth owning only when the audio must stay inside your system, when you need the typed action items to flow into your own tracker, or when you are learning where these APIs bite. Two things to settle before relying on it: the models behind Spring AI’s defaults are scheduled to shut down in early 2027, and the speech endpoint has no announced successor on the same path. And nothing in this article tested transcript accuracy or summary quality against a real model; the mock proves the plumbing, not the results. Measure on your own recordings, with consent from the people on them, before anyone acts on generated minutes.
Further reading
- Companion code: spring-ai/audio, with every output file under output/.
- Structured Output in Spring AI 2.0 —
entity()and the validation advisor. - Multimodal Spring AI: Extract Structured Data from Images — the same local-server technique for images.
- Observability for Spring AI — tokens and cost as metrics.
- Anthropic Claude vs OpenAI vs Gemini in Spring AI 2.0 — swapping providers behind the interfaces.
- Spring AI 1.x to 2.0 migration guide.
- OpenAI: speech to text guide, text to speech guide, pricing.
- Spring AI reference: transcriptions, speech and structured output converter.
No Comments yet!