Qualcomm AI Inference Suite: Getting Started is Easy

Developing solutions with AI inference at scale doesn’t have to be hard. Get started today using the Qualcomm AI Inference Suite, hosted on Cirrascale using our Python SDK.

Qualcomm has partnered with Cirrascale to provide a free-to-try AI inference service that makes it easy for developers to connect and run inference workloads from code. You don’t need to spin up containers or fiddle with complex cloud infrastructure. All you need to get started is to sign up and obtain an API key.

Once you have a key, it is simply a matter of choosing how to connect to the inference endpoint.  If you are most comfortable with Python, an SDK is provided along with documentation so that you can connect using familiar Python coding conventions. Many other programming languages and tools (20+) are supported if you prefer to use OpenAI compliant APIs.

Sample scenario

Let’s create a sample scenario to showcase use.  You want AI to tell you if a given customer review is positive, negative, or neutral. Once you have that data, you could later analyze which products or services are liked and which are not.

Python SDK example

Going step by step, we’ll build up our sample code.

Import functions you’ll need from the Imagine SDK:

from imagine import ChatMessage, ImagineClient

Set your API key and endpoint in env variables or in code:

myendpoint = "https://aisuite.cirrascale.com/apis/v2"
myapikey = "useyourownkeyhere"

Create your client for the API using the endpoint and key:

client = ImagineClient(endpoint=myendpoint, api_key=myapikey)

Set up your request, choosing which LLM to use:

mymodel = "Llama-3.1-8B"

feedback = "Feedback: ' We loved this hotel, we will be coming back.'"

mycontent = "You are an expert at sentiment analysis. Please evaluate the following customer feedback and answer only with positive, negative, or neutral. Only answer positive, negative, or neutral in lowercase. " + feedback

Call the chat function with your request:

mymessage = ChatMessage(role="user", content=mycontent)

chat_response = client.chat(
    messages=[mymessage],
    model=mymodel,
)

Print out the result:

print(chat_response.first_content)

Try changing the text of the feedback variable to different sets of text about any product and give it a try.

Try it out

As you can see, the process of using inference on a scalable platform like the Qualcomm AI Inference Suite is as easy as using any other simple API, but it provides access to very fast results using even larger models.  Even though this is just a small example, it could be integrated into a microservice to provide lots of customer sentiment analysis as part of a production data pipeline.

Like what you are seeing? Connect with fellow developers, get the latest news and prompt technical support by joining our Developer Discord. Qualcomm Blog. R. S.