Awesome Knowledge Distillation

MEAL V2: Boosting Vanilla ResNet-50 to 80%+ Top-1 Accuracy on ImageNet without Tricks. In NeurIPS 2020 workshop.

A minimalist SOTA LaTeX OCR model with only 20M parameters, running in browser. Full training pipeline available for self-reproduction. | 超轻量SOTA LaTeX公式识别模型，仅20M参数量，可在浏览器中运行。训练全流程代码开源，以便自学复现。

irresponsible innovation. Try now at https://chat.dev/

Build an Autonomous Web3 AI Trading Agent (BASE + Uniswap V4 example)

Learn How To Observe, Manage, and Scale, Agentic AI Apps Using Azure AI Foundry - with this hands-on workshop

Fine-Tune End-to-End Distillation Models with Azure AI Foundry Models and Foundry Local

[ICML2023] Revisiting Data-Free Knowledge Distillation with Poisoned Teachers

[ICML 2024]Exploration and Anti-exploration with Distributional Random Network Distillation

An R package providing functions for interpreting and distilling machine learning models

TeEFusion: Blending Text Embeddings to Distill Classifier-Free Guidance (ICCV 2025)

Distillation Knowledge for training Multi-exit Model

Easily generate synthetic data for classification tasks using LLMs

It is envisaged to eliminate these light constituents by distillation (flash or stripping). A preliminary study of the operating conditions of the process can be done in pseudo-binary: we assimilate the C7 cut to n-heptane and the light ones to ethane. We wish to construct the diagrams [T-x-y] and [x-y], [h-x-y] of the ethane-n-heptane binary u…

A transformer-based masked language model for learning amino acid sequence representations. The model uses self-attention mechanisms with custom gating and incorporates protein features for enhanced sequence understanding. Trained using BERT-style masking on peptide sequences to learn contextual amino acid embeddings.

CISPA Summer Internship

🤖 Build an autonomous Web3 AI trading agent for ETH-USDC on the BASE blockchain with Uniswap V4; learn by modifying and enhancing this hands-on project.

A comprehensive toolkit and knowledge base for fine-tuning state-of-the-art LLMs. This repository provides hands-on examples, benchmark datasets, and best practices for domain adaptation, instruction-following, and specialized task performance in Natural Language Processing.

Embedding markdowns of sites using distilled roberta v1 to ChromaDB

Improving Typhoon Center Location Models by Augmented Typhoon Image and Distillation Methods