Skip to main content
FriendliAI enhances AI application performance and optimizes cost savings with scalable, efficient deployment options, tailored for high-demand AI workloads.

​
Installation and setup

Install the friendli-client python package. 
pip install -U langchain_community friendli-client
Sign in to Friendli Suite to create a Personal Access Token, and set it as the FRIENDLI_TOKEN environment variable.

​
Chat models

See a usage example. 
from langchain_community.chat_models.friendli import ChatFriendli

chat = ChatFriendli(model='meta-llama-3.1-8b-instruct')

for m in chat.stream("Tell me fun things to do in NYC"):
    print(m.content, end="", flush=True)

​
LLMs

See a usage example. 
from langchain_community.llms.friendli import Friendli

llm = Friendli(model='meta-llama-3.1-8b-instruct')

print(llm.invoke("def bubble_sort(): "))