Documentation
Model setup
Match an available runtime to the work and the machine.
Local runtime
Install a compatible local model runtime and download a model that fits the machine. Verify its service responds before selecting it in the application. Keep the model identifier and endpoint accurate.
Provider route
Use a supported API provider with your own credentials. Provider costs and quotas are separate from the application download. A free model may be unavailable, rate-limited or unable to perform a particular task.
Capacity
Account for model memory, context length and other applications. Integrated GPU memory is shared with the system. GPU detection is not the same as verified acceleration or stable task completion.
Switching models
Inspect which model actually served the operation and any fallback notices. Avoid changing a running task’s assumptions without checking its state. Privacy and spending restrictions should remain in force across fallback routes.