Faster and Lighter Model Inference with ONNX Runtime from Cloud to Client
19:55
Faster and Lighter Model Inference with ONNX Runtime from Cloud to Client
Aug 3, 2022
Microsoftmarkdefalco
Quantization: What Everyone Gets Wrong (Accuracy Myths)
0:44
Quantization: What Everyone Gets Wrong (Accuracy Myths)
65 views2 weeks ago
YouTubeCode & Capital
FP32 FP16 FP8 TENSOR INT #chatgpt #llm #google #tech #ytshorts #yt #youtube #youtubeshorts
0:16
FP32 FP16 FP8 TENSOR INT #chatgpt #llm #google #tech #ytshorts #yt #youtube #youtubeshorts
61 views1 month ago
YouTubeAmit_Chopra_assruc
Google magic bullet - TurboQuant #ai #gpu #google #chips #cuda #quantization
0:41
Google magic bullet - TurboQuant #ai #gpu #google #chips #cuda #quantization
1.3K views1 month ago
YouTubeNeural AI Flair
From 15GB to 4.7GB: Quantizing AI Models Locally
13:42
From 15GB to 4.7GB: Quantizing AI Models Locally
7.7K views1 month ago
YouTubeNeuralNine
Tikhomirov M.M. - Training of large language models - 8. Inference, quantization
1:08:05
Tikhomirov M.M. - Training of large language models - 8. Inference, quantization
218 views2 weeks ago
YouTubeteach-in
Why Inference is hard..
15:14
Why Inference is hard..
232 views3 weeks ago
YouTubeCaleb Writes Code
6:29
Inference Optimization: Making AI Faster & Cheaper (Latency, Throughput & GPUs)
56 views1 month ago
YouTubewecite
7:29
Model Quantization Explained 8 bit, 4 bit & Inference Optimization #genai #aigenerated
32 views2 months ago
YouTubeSmartSkale
2:36
I added KV caching and INT8 KV quantization to our transformer inference, improving throughput by 35x.All of this was done from scratch in Rust + CUDA, on top of a homemade ML framework.On a 4-token prompt with 252 generated tokens:- Original: 0.76 tok/s- KV cache fp32: 27.21 tok/s- KV cache int8 (quantized): 27.29 tok/sTry it out yourself here: https://t.co/kFS9Z0fs4hIn practice:- KV caching gave us about a 35x end-to-end speedup- INT8 KV cache kept roughly the same speed as fp32 but cut KV cac
48.8K views3 weeks ago
x.comReese Chong
0:23
This is the clearest explanation of how LLM quantization works:It lets engineers compress a model by 4x and run it 2x faster without quality loss.I stumbled on this piece by Sam Rose and honestly wish I had it when I first tried to understand quantization.He breaks it down from absolute zero - bits, floats, how weights are stored - all the way to actually benchmarking quantized models. With interactive demos you can play with.Here's the core idea in 60 seconds:→ An LLM is billions of numbers (we
7K views2 weeks ago
x.comSukh Sroay
Z-Image-Turbo INT8 — AI Playground & API - deAPI.ai
3 weeks ago
deapi.ai
Model Precision and Deployment Choices in Object Detection | Mohammad Zaid posted on the topic | LinkedIn
4 views3 weeks ago
linkedin.com
Edge ML Development for i.MX Processors
Aug 21, 2024
nxp.com
15:47
Discrete Math - 1.4.2 Quantifiers
226.9K viewsFeb 25, 2020
YouTubeKimberly Brehm
5:23
Rules of Inference for Quantified Statements (Part 1)
169.5K viewsJan 19, 2021
YouTubeNeso Academy
41:59
Sampling Theorem Quantization and Binary Coding
7.1K viewsApr 11, 2021
YouTubeEngineering with Bingabr
15:08
PREDICATE LOGIC and QUANTIFIER NEGATION - DISCRETE MATHEMATICS
551.3K viewsJul 17, 2017
YouTubeTrevTutor
12:35
The Mathematics of Quantum Computers | Infinite Series
715.7K viewsFeb 16, 2017
YouTubePBS Infinite Series
0:51
Quantization explained
504 views3 months ago
YouTubeChip Talks AI
9:57
What is LLM Quantization ?
3.2K viewsMar 19, 2025
YouTubeNew Machina
2:11
NVIDIA Tesla T4 Introduction to Inference
3.7K viewsApr 18, 2019
YouTubeBoston Limited
1:16:40
Lecture 30: Quantized Training
3.3K viewsOct 7, 2024
YouTubeGPU MODE
5:22
1.2.2 Quantifying Information
26.3K viewsJul 12, 2019
YouTubeMIT OpenCourseWare
53:01
22. Sampling and Quantization
30.8K viewsMar 15, 2013
YouTubeMIT OpenCourseWare
12:30
41. Adaptive Quantization
28.1K viewsApr 24, 2018
YouTubeitechnica
12:10
Optimize Your AI - Quantization Explained
465.1K viewsDec 28, 2024
YouTubeMatt Williams
1:36
What Is Quantization? | Decoding LLM File Names
1.3K views4 months ago
YouTubeAnaconda, Inc.
1:01
Towards Unified INT8 Training for Convolutional Neural Network
803 viewsJul 17, 2020
YouTubeComputerVisionFoundation Videos
1:03:51
DSP Lecture 23: Introduction to quantization
39.8K viewsNov 24, 2014
YouTubeRich Radke
See more