Eight New Models Land in the Workers AI Catalog

Cloudflare has expanded the Workers AI model catalog with eight new additions, bringing the total to over 20 models available to developers. The release focuses on text generation, code generation, and a new content-safety classification option.

The new models are all quantized variants using Activation-aware Weight Quantization (AWQ), a technique designed to reduce the memory footprint of large language models while minimizing the precision loss typically associated with quantization.

Text generation models:
@hf/thebloke/llama-2-13b-chat-awq
@hf/thebloke/zephyr-7b-beta-awq
@hf/thebloke/mistral-7b-instruct-v0.1-awq
@hf/thebloke/openhermes-2.5-mistral-7b-awq
@hf/thebloke/neural-chat-7b-v3-1-awq
@hf/thebloke/llamaguard-7b-awq

Code generation models:
@hf/thebloke/deepseek-coder-6.7b-base-awq
@hf/thebloke/deepseek-coder-6.7b-instruct-awq

Highlights Among the New Releases

Cloudflare reports that deepseek-coder-6.7b scores roughly 15% higher than comparable Code Llama models on popular coding benchmarks, a result attributed to training data that includes both English and Chinese code generation datasets. The openhermes-2.5-mistral-7b model, a fine-tune of Mistral 7B, demonstrates how targeted fine-tuning can lift performance by around 10% on many LLM benchmarks when compared to its base model.

Adding an Input/Output Safeguard Model

One of the more distinct additions is llamaguard-7b-awq, a Meta model designed for input and output safety in AI applications. Developers can use it to detect when a prompt or response falls into potentially unsafe categories and to define those categories themselves. Categorization rules can be customized to fit an application's specific use case. Cloudflare notes that its example categories are provided for informational purposes only, and no guarantees are made regarding Llama Guard's performance on any given task.

Getting Started With the New Models

The models are available for experimentation in the AI section of the Cloudflare Dashboard, with full documentation and the complete catalog in the Developer Docs. Usage follows the existing Workers AI pattern: make an API call or use a Workers binding to run inference from a Worker or Pages application. Cloudflare states that additional models, a pricing announcement, and higher usage limits are planned for the coming weeks. Developers can share feedback and projects on the Cloudflare Discord server.