Fundamentals: Bounds

Users can restrict query processing overheads by setting thresholds on several processing cost metrics:

max_seconds: maximum query runtime
max_calls: maximum number of LLM calls
max_tokens: maximum number of processed LLM tokens

Thresholds are set either in the web interface or via parameters in the REST interface.

Warning: samtSQL checks periodically whether user-defined thresholds are reached. Query termination is not instantaneous and query evaluation costs may slightly exceed the user-defined bounds.

If query evaluation reaches any of the user-defined bounds, samtSQL terminates query evaluation and returns a partial result. AI operators return a default value of SQL NULL for all rows on which they were not evaluated on.

Queries should be written with partial results in mind, handling SQL NULL results for AI operators in the most appropriate manner, given the query intent. See Partial Query Results for best practices on handling partial results.