A lightweight SPEED-Bench client for benchmarking an already-running llama-server through its OpenAI-compatible API. It is primarily meant to evaluate speculative decoding (draft model, n-gram, MTP, ...

Welcome to the AI Applications in Accelerators Tutorial & Mini-Hackathon repository. This project hosts the computational challenge, simulation environment, benchmark notebooks, and documentation for ...