1. Create an API key
Create or sign in to your IteraCompute account, then create a key in the Console.
Integration guideOpenAI-compatibleUse a model with LlamaIndex through IteraCompute.
Power retrieval and agent workflows.
Connect in three steps
2. Configure the base URL
Use OpenAILike with api_base and explicitly configure the model context window and tool-calling capability.
3. Select the model
Use the exact API Model ID for your chosen model. Check its context and capabilities before enabling agent or vision features.
Choose your model
Example configuration
Keep API keys in your server environment or tool credential store. Replace the example key field with your own key where applicable.
Python · pip install llama-index-llms-openai-like
import os
from llama_index.llms.openai_like import OpenAILike
from llama_index.core.llms import ChatMessage
llm = OpenAILike(
model="qwen/qwen3.8-27b",
api_base="https://api.iteracompute.com/v1",
api_key=os.environ["ITERACOMPUTE_API_KEY"],
context_window=131072,
is_chat_model=True,
is_function_calling_model=True,
max_tokens=512,
additional_kwargs={"chat_template_kwargs": {"enable_thinking": False}},
)
response = llm.chat([ChatMessage(role="user", content="Hello!")])
print(response.message.content)Version and verification
Configuration reference · reviewed 2026-10-15. A reproducible end-to-end tool test has not been recorded here. Record your installed version and test date when running the steps below.
Success criterion
Run llm.chat and check non-empty message.content. Test retrieval and tool execution separately.
Record the installed version
python -m pip show llama-index-llms-openai-likeTest the configured tool itself after the API check below. An HTTP 200 or a cURL result alone does not verify that tool.
Test the connection
cURL · Chat Completions
# Bash / zsh / WSL; set ITERACOMPUTE_API_KEY in your environment first.
curl --fail-with-body https://api.iteracompute.com/v1/chat/completions \
-H "Authorization: Bearer $ITERACOMPUTE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"qwen/qwen3.8-27b","messages":[{"role":"user","content":"Reply with exactly: Hello!"}],"max_tokens":512}'Check that choices[0].message.content contains visible text; HTTP 200 alone is insufficient. Reasoning may consume the output budget. For free Qwen, use the first free call example to disable thinking, then send a prompt inside your tool.
Selected model capabilities
These are API capabilities of the selected model. Feature availability in your workflow also depends on the tool and its settings; reasoning controls vary by model.
- Chat CompletionsSupported
- Tool callingSupported
- ReasoningSupported
- Image inputSupported
- StreamingSupported
Troubleshooting
Unknown model metadata
Use OpenAILike from llama_index.llms.openai_like and provide context_window and is_chat_model=True. Its model metadata must match the selected account tier.
Tools or context mismatch
Set is_function_calling_model only for a model that supports tools. Free Qwen context is 131072, including input and requested output; max output is 8192.
Retrieval or answer missing
First verify llm.chat returns non-empty message.content. A retrieval pipeline also needs a configured embedding model and index; a chat connection does not verify retrieval.
429 / Request limits
Check account-wide monthly usage and TPM. Honor Retry-After and avoid parallel retries. Free Qwen has no automatic paid overage; other models use their published prices.
401 / 403: check the key and model permission. Free Qwen requires no balance or card. For paid models, also check available balance.
Keep exploring
Start building with LlamaIndex.
Connect your key, choose a model and send your first request.