Experiment Control Flow Using MADSci

MADSci is a modular, autonomous, and scalable framework for scientific discovery and experimentation. It aims to provide:

• Laboratory Instrument Automation and Integration via the MADSci Node standard. Developers can implement device-specific Node modules in any language that can then be integrated into a MADSci system using a common interface standard (currently supports REST-based HTTP communication)
• Workflow Management, allowing users to define and run flexible scientific workflows that can leverage one or more Nodes to complete complex tasks.
• Experiment Management, conducting flexible closed loop autonomous experiments by combining multiple workflow runs, as well as any compute, decision making, data collection, and analysis as needed.
• Resource Management, allowing robust tracking of all the labware, consumables, equipment, samples, and assets used in an autonomous laboratory.
• Event Management, enabling distributed logging and event handling across every part of the autonomous lab.
• Data Management, collecting and storing data created by instruments or analysis as part of an experiment.
• Location Management, coordinating multiple different representations of locations in the laboratory and their interactions with resources and nodes.

Diagram of a MADSci Laboratory's Infrastructure

MADSci is currently in beta. Most of the core functionality is working and tested, but there may be bugs or stability issues (if you run into any, please open an issue so we can get it fixed). New releases will likely include breaking changes, so we recommend pinning the version in your dependencies and upgrading only after reviewing the release notes.

MADSci is made up of a number of different modular components, each of which can be used independently to fulfill specific needs, or composed to build more complex and capable systems. Below we link to specific documentation for each system component.

• Common: the common types and utilities used across the MADSci toolkit
• Clients: A collection of clients for interacting with different components of MADSci
• Event Manager: handles distributed event logging and querying across a distributed lab.
• Workcell Manager: handles coordinating and scheduling a collection of interoperating instruments, robots, and resources using Workflows.
• Location Manager: manages laboratory locations, resource attachments, and node-specific references.
• Experiment Manager: manages experimental runs and campaigns across a MADSci-powered lab.
• Experiment Application: extensible python class for running autonomous experiments.
• Resource Manager: For tracking labware, assets, samples, and consumables in an automated or autonomous lab.
• Data Manager: handles capturing, storing, and querying data, in either JSON value or file form, created during the course of an experiment (either collected by instruments, or synthesized during anaylsis)
• Squid Lab Manager: a central lab configuration manager and dashboard provider for MADSci-powered labs.

All MADSci components are available via PyPI. Install individual components as needed:

# Core components
pip install madsci.common          # Shared types and utilities
pip install madsci.client          # Client libraries
pip install madsci.experiment_application # Experiment Logic

# Manager services
pip install madsci.event_manager    # Event logging and querying
pip install madsci.workcell_manager # Workflow coordination
pip install madsci.location_manager # Location management
pip install madsci.resource_manager # Resource tracking
pip install madsci.data_manager     # Data capture and storage
pip install madsci.experiment_manager # Experiment management

# Lab infrastructure
pip install madsci.squid           # Lab manager with dashboard
pip install madsci.node_module      # Node development framework

We provide pre-built Docker images for easy deployment:

• ghcr.io/ad-sdl/madsci: Base image with all MADSci packages. Use as foundation for custom services.
• ghcr.io/ad-sdl/madsci_dashboard: Extends base image with web dashboard for lab management.

Try MADSci with our complete example lab:

git clone https://github.com/AD-SDL/MADSci.git
cd MADSci
docker compose up  # Starts all services with example configuration

Access the dashboard at http://localhost:8000 to monitor your virtual lab.

MADSci uses environment variables for configuration with hierarchical precedence. Key patterns:

• Service URLs: Each manager defaults to localhost with specific ports (Event: 8001, Experiment: 8002, Resource: 8003, Data: 8004, Workcell: 8005, Location: 8006, etc.)
• Database connections: MongoDB/PostgreSQL on localhost by default
• File storage: Defaults to ~/.madsci/ subdirectories
• Environment prefixes: Each service has a unique prefix (e.g., WORKCELL_, EVENT_, LOCATION_)

See Configuration.md for comprehensive options and example_lab/ for working configurations.

We're working on bringing the following additional components to MADSci:

• Auth Manager: For handling authentication and user and group management for an autonomous lab.
• Transfer Manager: For coordinating resource movement in a lab.

1. Example Lab: Complete working lab with virtual instruments (robot arm, liquid handler, plate reader)
2. Example Notebooks: Jupyter notebooks covering core concepts and implementation patterns, included in the example lab
3. Configuration examples: See example_lab/managers/ for manager configurations

Starting a basic lab:

# Use our example lab as a starting point
cp -r example_lab my_lab
cd my_lab
# Modify configurations in managers/ directory
docker compose up

Creating custom nodes:

# See example_lab/example_modules/ for reference implementations
from madsci.node_module import AbstractNodeModule

class MyInstrument(AbstractNodeModule):
    def my_action(self, param1: str) -> dict:
        # Your instrument control logic
        return {"result": "success"}

Submitting workflows:

# See example_lab/workflows/ for workflow definitions
from madsci.client.workcell_client import WorkcellClient

client = WorkcellClient("http://localhost:8005")
result = client.submit_workflow("path/to/workflow.yaml")

• Python 3.9+: Required for all MADSci components
• PDM: For dependency management and virtual environments
• Docker: Required for services and integration tests
  • Alternatives: Rancher Desktop, Podman
• just: Task runner for development commands
• Node.js/npm: Only needed for dashboard development

# Clone and initialize
git clone https://github.com/AD-SDL/MADSci.git
cd MADSci
just init  # Installs all dependencies and sets up pre-commit hooks

# See all available commands
just list

# Start example lab for testing
just up

# Testing
pytest                    # Run all tests
just test                 # Alternative test runner
pytest -k workcell        # Run specific component tests

# Code Quality
just checks               # Run all pre-commit checks (ruff, formatting, etc.)
ruff check               # Manual linting
ruff format              # Manual formatting

# Services
just build               # Build Docker images
just up                  # Start example lab
just down               # Stop services

# Dashboard Development
cd ui/
npm run dev             # Start Vue dev server
npm run build           # Build for production

Manager Implementation: Each manager service follows this structure:

• Settings class inheriting from MadsciBaseSettings
• FastAPI server with REST endpoints
• Client class for programmatic interaction
• Database models (SQLModel/Pydantic)

Testing:

• Integration tests use Docker containers via pytest-mock-resources
• Component tests are in each package's tests/ directory
• Use pytest -k EXPRESSION to filter tests

Configuration:

• Environment variables with hierarchical precedence
• Each manager has unique prefix (e.g., WORKCELL_, EVENT_)
• See Configuration.md for full details

For VS Code users, use the included .devcontainer for instant setup:

• Automatic dependency installation
• Pre-configured development environment
• Docker services ready to run