Open Source · Hacker News ·
AirLLM 70B inference with single 4GB GPU
AirLLM is an open-source project that enables inference with 70-billion-parameter language models on a single GPU with 4GB of memory. The approach is intended to reduce GPU memory requirements for running large models.