Skip to content

fix: prevent server crash when clients send 'tools' param (fixes #530)#538

Open
octo-patch wants to merge 1 commit intomicrosoft:mainfrom
octo-patch:fix/issue-530-unsupported-param-crash
Open

fix: prevent server crash when clients send 'tools' param (fixes #530)#538
octo-patch wants to merge 1 commit intomicrosoft:mainfrom
octo-patch:fix/issue-530-unsupported-param-crash

Conversation

@octo-patch
Copy link
Copy Markdown

Fixes #530

Problem

When clients like Open WebUI send a POST /v1/chat/completions request with a tools parameter (standard OpenAI function-calling API), the llama-server binary crashes with SIGABRT instead of returning a proper error response:

terminate called after throwing an instance of 'std::runtime_error'
  what():  Unsupported param: tools
Error occurred while running command: ... died with <Signals.SIGABRT: 6>

Root Cause

In examples/server/server.cpp, the handle_chat_completions function calls oaicompat_completion_params_parse(), which throws std::runtime_error for unsupported parameters (tools, tool_choice). This exception is not caught locally, so it propagates past httplib's global exception handler and calls std::terminate(), crashing the server process.

Solution

Wrap the oaicompat_completion_params_parse() call in a try-catch block that:

  1. Catches any std::exception thrown during parameter parsing
  2. Returns an HTTP 400 (invalid request) error response with the error message
  3. Returns early, keeping the server running for subsequent requests

The change is minimal (7 lines in examples/server/server.cpp) and does not affect normal request handling.

Changes

  • Updated 3rdparty/llama.cpp submodule to octo-patch/llama.cpp at commit 7525084 which adds the try-catch fix in examples/server/server.cpp

Testing

Verified that the fix pattern matches how other error conditions are handled in the same file (e.g., the --embeddings check at the top of handle_chat_completions uses the same res_error + return pattern).

After this fix, sending a request with tools will return HTTP 400 with a clear error message instead of crashing the server.

…icrosoft#530)

When clients like Open WebUI send a POST to /v1/chat/completions with
a 'tools' parameter (for function calling), the llama-server binary
crashes with SIGABRT instead of returning a proper error response.

Root cause: oaicompat_completion_params_parse() throws std::runtime_error
for unsupported params ('tools', 'tool_choice'), but the exception is not
caught in handle_chat_completions(), causing it to propagate past httplib's
exception handler and terminate the process.

Fix: wrap the oaicompat_completion_params_parse() call in a try-catch that
returns HTTP 400 (invalid request) with the error message, keeping the
server alive for subsequent requests.

Change: update 3rdparty/llama.cpp submodule to octo-patch/llama.cpp
at commit 7525084 which contains the fix in examples/server/server.cpp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unsupported param: tools

1 participant