Workers AI Update: Stable Diffusion, Code Llama + Workers AI in 100 cities

Workers AI expands: Stable Diffusion, Code Llama, and bigger global footprint

Cloudflare has added two new models to its Workers AI platform, which now runs inference across more than 100 cities on its global network. The additions bring Stable Diffusion XL 1.0 for text-to-image generation and Meta's Code Llama, a code-focused language model built on Llama 2.

Stable Diffusion XL 1.0, released by Stability AI this summer, is designed to produce images without imposing a particular aesthetic on the output, giving developers freedom over style. The model also delivers improved color accuracy, contrast, lighting, and shadows compared to its predecessor, with native 1024x1024 resolution output.

Calling the model from a Worker is straightforward:

curl -X POST \
"https://api.cloudflare.com/client/v4/accounts/{account-id}/ai/run/@cf/stabilityai/stable-diffusion-xl-base-1.0" \
-H "Authorization: Bearer {api-token}" \
-H "Content-Type:application/json" \
-d '{ "prompt": "A happy llama running through an orange cloud" }' \
-o 'happy-llama.png'

Developers can also interact with the model directly through the API. The same applies to Code Llama, which understands and generates code in Python, C++, Java, PHP, TypeScript/JavaScript, C#, and Bash. It works not just for writing new code but also for parsing and explaining unfamiliar codebases.

Code Llama usage from a Worker:

import { Ai } from '@cloudflare/ai';

// Enable env.AI for your worker by adding the ai binding to your wrangler.toml file:
// [ai]
// binding = "AI"

export default {
  async fetch(request, env) {
    const ai = new Ai(env.AI);
    const response = await ai.run('@hf/thebloke/codellama-7b-instruct-awq', {
      prompt: 'In JavaScript, define a priority queue class. The constructor must take a function that is called on each object to determine its priority.'
    });
    return Response.json(response);
  }
}

Or via curl:

curl -X POST \
"https://api.cloudflare.com/client/v4/accounts/{account-id}/ai/run/@hf/thebloke/codellama-7b-instruct-awq" \

-H "Authorization: Bearer {api-token}" \-H "Content-Type: application/json" \
-d '{ "prompt": "In JavaScript, define a priority queue class. The constructor must take a function that is called on each object to determine its priority." }

Or with Python:

#!/usr/bin/env python3

import json
import os
import requests

ACCOUNT_ID=os.environ["ACCOUNT_ID"]
API_TOKEN=os.environ["API_TOKEN"]
MODEL="@hf/thebloke/codellama-7b-instruct-awq"

prompt="""In JavaScript, define a priority queue class. The constructor must take a function that is called on each object to determine its priority."""
url = f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/{MODEL}"
headers = {
  "Authorization": f"Bearer {API_TOKEN}"
}
payload = json.dumps({
  "prompt": prompt
})

print(url)
r = requests.post(url, data=payload, headers=headers)

j = r.json()
if "result" in j and "response" in j["result"]:
   print(r.json()["result"]["response"])
else:
   print(json.dumps(j, indent=2))

Inference reaches 100 cities ahead of schedule

Workers AI originally launched in September with inference in just seven cities, with a goal of reaching 100 by the end of the year. That target has already been met. For developers, this means inference tasks are more likely to run geographically closer to end users, reducing latency. Cloudflare says the footprint will continue expanding through 2024.

Also new: Mistral 7B

Earlier in the week, Cloudflare also announced Mistral 7B, a language model that delivers strong performance relative to its size. It's available through the same Workers AI text generation API used by other models.

Developers can experiment with these models and share feedback via the Cloudflare Developer Discord. Cloudflare also plans a series of AI developer workshops covering fundamentals like embeddings, vector databases, and getting started with LLMs on Workers AI; sign-ups are open.