Use Language Models in Your Rust Application (Free, Open-Weight, Self-Hosted)

preview_player
Показать описание
A concise look at the Kalosm crate, including how it fits in with the rest of the libraries of the LLM ecosystem.

  1. Code to the Moon
  2. kalosm
  3. llm
  4. language models
  5. large language models
  6. rust
Рекомендации по теме
How Large Language 0:05:34

How Large Language Models Work

How to Choose 0:06:57

How to Choose Large Language Models: A Developer’s Guide to LLMs

Large Language Models 0:07:58

Large Language Models explained briefly

Use Language Models 0:08:48

Use Language Models in Your Rust Application (Free, Open-Weight, Self-Hosted)

run AI on 0:00:40

run AI on your laptop....it's PRIVATE!!

Should You Use 0:06:40

Should You Use Open Source Large Language Models?

What are Large 0:05:30

What are Large Language Models (LLMs)?

LLM Explained | 0:04:17

LLM Explained | What is LLM

Control Your Browser 0:20:33

Control Your Browser with AI: Automate Anything in Minutes!

What is Ollama? 0:07:14

What is Ollama? Running Local LLMs Made Simple

From Idea to 1:05:55

From Idea to Implementation - How to Use Language Models in Your Business

Large Language Models 0:25:20

Large Language Models (LLMs) - Everything You NEED To Know

Connecting LLMs to 0:06:18

Connecting LLMs to tools

Choose the right 0:04:04

Choose the right AI model for your use case

Feed Your OWN 0:18:53

Feed Your OWN Documents to a Local Large Language Model!

Introduction to large 0:15:46

Introduction to large language models

[1hr Talk] Intro 0:59:48

[1hr Talk] Intro to Large Language Models

How to Make 0:17:52

How to Make Small Language Models Work. Yejin Choi Presents at Data + AI Summit 2024

How I use 2:11:12

How I use LLMs

Five Steps to 0:06:56

Five Steps to Create a New AI Model

Fine Tuning Large 0:08:01

Fine Tuning Large Language Models with InstructLab

The Rise of 0:02:03

The Rise of Small Language Models to Cut AI Costs

Building AI Applications 0:04:42

Building AI Applications with Large Language Models

What is Retrieval-Augmented 0:06:36

What is Retrieval-Augmented Generation (RAG)?

Комментарии
Автор

Wow, there really are so many applications! Amazing video❤

virusblitz
Автор

Awesome content! Excited to try this crate. Thank you for sharing

northicewind
Автор

if you are an engineer and serious about LLMs in Rust learn Candle. Ive been able to research and develop optimizers and architectures with Candle. Its an involved framework but is feature rich and complete in more places than any other framework. It makes design decisions that are reminiscent of torch and tensorflow and is very low level and not polyglot compared to alternatives.

first-thoughtgiver-of-will
Автор

Technically if you want to deploy an LLM powered application you would not just rely on a library like Transformers. You also need a performant server that can A. Do adaptive batching and B. Implement efficient “multiplications” to work optimally on multiple batches. So while kalosm looks dope, you would not want to use Rust if you actually need a python library like vLLM or NVIDIA’s triton (it’s oss). Folks at Kyutai (the company founded by the same guy who contributes to Candle) said they use rust for inference. So they must have implemented all of that in rust :)

But if you don’t need that much load, of course use kalosm. I’m pretty sure you can find a crate that implements batching for tokio.

__sassan__
Автор

I think the sweet spot for using a library like Kalosm would be integrating it into a Tauri app and downloading + executing the ML model on the user's local machine. Pair Kalosm & Tauri with a Leptos frontend, and you have a really compelling native desktop (or mobile) app!

An installable PWA using a similar stack (minus Tauri of course) would also be a great use case for Kalosm + Leptos..

DaM_Cdn
join shbcf.ru