SGLang is a fast serving framework for large language models and vision language models.

Nexa SDK is a comprehensive toolkit for supporting ONNX and GGML models. It supports text generation, image generation, vision-language models (VLM), auto-speech-recognition (ASR), and text-to-speech (TTS) capabilities.

The Cradle framework is a first attempt at General Computer Control (GCC). Cradle supports agents to ace any computer task by enabling strong reasoning abilities, self-improvment, and skill curation, in a standardized general environment with minimal requirements.

An AI-powered file management tool that ensures privacy by organizing local texts, images. Using Llama3.2 3B and Llava v1.6 models with the Nexa SDK, it intuitively scans, restructures, and organizes files for quick, seamless access and easy retrieval.

[NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

LLM Agent Framework in ComfyUI includes Omost,GPT-sovits, ChatTTS,GOT-OCR2.0, and FLUX prompt nodes,access to Feishu,discord,and adapts to all llms with similar openai/gemini interfaces, such as o1,ollama, grok, qwen, GLM, deepseek, moonshot,doubao. Adapted to local llms, vlm, gguf such as llama-3.2, Linkage neo4j KG, graphRAG / RAG / html 2 img

A family of lightweight multimodal models.

[CVPR 2024 🔥] GeoChat, the first grounded Large Vision Language Model for Remote Sensing

Custom ComfyUI nodes for Vision Language Models, Large Language Models, Image to Music, Text to Music, Consistent and Random Creative Prompt Generation

ScreenAgent: A Computer Control Agent Driven by Visual Language Large Model (IJCAI-24)

A streamlined and customizable framework for efficient large model evaluation and performance benchmarking

[NeurIPS'24 Spotlight] EVE: Encoder-Free Vision-Language Models

Official code for Paper "Mantis: Multi-Image Instruction Tuning" (TMLR2024)

Ptera Software is a fast, easy-to-use, and open-source software package for analyzing flapping-wing flight.

Seamlessly integrate state-of-the-art transformer models into robotics stacks

RAI is a multi-vendor agent framework for robotics, utilizing Langchain and ROS 2 tools to perform complex actions, defined scenarios, free interface execution, log summaries, voice interaction and more.

LLaRA: Large Language and Robotics Assistant

JoyCaption is an image captioning Visual Language Model (VLM) being built from the ground up as a free, open, and uncensored model for the community to use in training Diffusion models.

[NeurIPS 2023 Oral] Quilt-1M: One Million Image-Text Pairs for Histopathology.

DenseFusion-1M: Merging Vision Experts for Comprehensive Multimodal Perception