Instructor¶

Instructor makes it easy to get structured data like JSON from LLMs like GPT-3.5, GPT-4, GPT-4-Vision, and open-source models including Mistral/Mixtral, Anyscale, Ollama, and llama-cpp-python.

Instructor's Rust Client is in active development. This means that the API and package might change moving forward. We are looking for active contributors to the repository in the meantime to help flesh out more of the features.

Roadmap¶

Here is a rough roadmap of features we'd like to implement

Struct -> JSON parsing

• Strings
• Handle Booleans
• Integers
• Handle String Enums
• Lists
• Nested Structs
• Union Types (Eg. Struct1 | Struct 2 )

Validators

• Support different types of integers (Eg. u8, u32, u64 -> Add a validator automatically which checks that we have a value > 0 and < max value)
• Validation Context (Eg. We can validate citations by passing in original passage )

Clients

• OpenAI
• Anthropic
• Cohere
• Gemini
• Mistral
• Llama-cpp

CLI

• Support Batch jobs using Instructor
• Support Fine-Tuning jobs using instructor
• Monitor Usage

Getting Started¶

To install instructor-ai, you'll need to add the following to your cargo.toml file

instructor-ai = "0.1.8"
instruct-macros = "0.1.8"
openai-api-rs = "4.1.0"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
instruct-macros-types = "0.1.8"

Getting started with structured extraction is then as simple as declaring a new struct with the InstructMacro and importing the ParamterInfo and StructInfo types.

use std::env;
use instruct_macros::InstructMacro;
use instruct_macros_types::{ParameterInfo, StructInfo};
use instructor_ai::from_openai;
use openai_api_rs::v1::{
    api::Client,
    chat_completion::{self, ChatCompletionRequest},
    common::GPT3_5_TURBO,
};
use serde::{Deserialize, Serialize};

fn main() {
    let client = Client::new(env::var("OPENAI_API_KEY").unwrap().to_string());
    let instructor_client = from_openai(client);

    #[derive(InstructMacro, Debug, Serialize, Deserialize)]
    // This represents a single user
    struct UserInfo {
        // This represents the name of the user
        name: String,
        // This represents the age of the user
        age: u8,
    }

    let req = ChatCompletionRequest::new(
        GPT3_5_TURBO.to_string(),
        vec![chat_completion::ChatCompletionMessage {
            role: chat_completion::MessageRole::user,
            content: chat_completion::Content::Text(String::from(
                "John Doe is a 30 year old software engineer",
            )),
            name: None,
        }],
    );

    let result = instructor_client
        .chat_completion::<UserInfo>(req, 3)
        .unwrap();

    println!("{}", result.name); // John Doe
    println!("{}", result.age); // 30
}