Deprecated: Function get_magic_quotes_gpc() is deprecated in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 99

Deprecated: The each() function is deprecated. This message will be suppressed on further calls in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 619

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1169

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176
8000 feat(mlx): add mlx backend by mudler · Pull Request #6049 · mudler/LocalAI · GitHub
Nothing Special   »   [go: up one dir, main page]
Skip to content

feat(mlx): add mlx backend #6049

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 14 commits into from
Aug 22, 2025
Merged

feat(mlx): add mlx backend #6049

merged 14 commits into from
Aug 22, 2025

Conversation

mudler
Copy link
Owner
@mudler mudler commented Aug 12, 2025 •
edited
Loading

Description

This PR:

Notes for Reviewers

Currently WIP

What works:

Signed commits

  • Yes, I signed my commits.

@netlify
Copy link
netlify bot commented Aug 12, 2025 •
edited
Loading

✅ Deploy Preview for localai ready!

Name Link
🔨 Latest commit a4bdaab
🔍 Latest deploy log https://app.netlify.com/projects/localai/deploys/68a81144786af50008a84e7b
😎 Deploy Preview https://deploy-preview-6049--localai.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions
Copy link

yamllint Failed

Show Output 
::group::backend/index.yaml
::warning file=backend/index.yaml,line=95,col=6::95:6 [comments] missing starting space in comment
::error file=backend/index.yaml,line=140,col=24::140:24 [trailing-spaces] trailing spaces
::warning file=backend/index.yaml,line=723,col=6::723:6 [comments] missing starting space in comment
::warning file=backend/index.yaml,line=928,col=3::928:3 [comments-indentation] comment not indented like content
::endgroup::

Workflow: Yamllint GitHub Actions, Action: __karancode_yamllint-github-action_2, Lint: backend

@mudler mudler mentioned this pull request Aug 21, 2025
Closed
@mudler
Copy link
Owner Author
mudler commented Aug 21, 2025

due to Blaizzy/mlx-vlm#470 mlx-vlm installed here is behind and does not support many new models, but this is a start!

@mudler
Copy link
Owner Author
mudler commented Aug 21, 2025

arg, more failures as it's a really old version. we might need to drop the mlx-vlm from this PR for now

mudler added 13 commits August 21, 2025 23:46
@mudler
chore: allow to install with pip
6d0af39 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler
WIP
44eaf6c 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler
Make the backend to build and actually work
d933847 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler
List models from system only
8ed272e 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler
Add script to build darwin python backends
4662a9f 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler
Run protogen in libbackend
1c86dfd 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler
Detect if mps is available across python backends
1fbe95f 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler
CI: try to build backend
cfaecba 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler
Debug CI
d9d0439 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler
Fixups
0a8975d 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler
Fixups
ab9ab20 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler
Index mlx-vlm
c59b6a9 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler
Remove mlx-vlm
92456f5 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler mudler changed the title feat(mlx): add mlx and mlx-vlm backends feat(mlx): add mlx backends Aug 21, 2025
@mudler mudler changed the title feat(mlx): add mlx backends feat(mlx): add mlx backend Aug 21, 2025
@mudler mudler marked this pull request as ready for review August 21, 2025 21:49
@mudler
Drop CI test
a4bdaab 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler mudler merged commit 1d830ce into master Aug 22, 2025
68 of 70 checks passed
@mudler mudler deleted the feat/mlx branch August 22, 2025 06:42
This was referenced Aug 22, 2025
Merged
Closed
@BrewTestBot BrewTestBot mentioned this pull request Sep 3, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

enhancements

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Mlx backend Backend: MLX Using mlx for apple silicon

1 participant

@mudler
0