What is Roofline?

Roofline is a software suite for edge AI deployment, offering an SDK, runtime, and performance dashboard. It is based on a next-generation AI compiler built on MLIR, which translates models from major AI frameworks into optimized executables for various hardware backends. The product aims to unlock the full potential of edge SoCs by enabling heterogeneous execution across CPU, GPU, and NPU.

How to use Roofline?

To use Roofline, developers integrate the SDK into their workflow via a simple Python interface. They can import models from frameworks like PyTorch, TFLite, or ONNX, and compile them for target hardware with just a few lines of code. The runtime executes the compiled models on devices, and the performance dashboard provides transparency on performance metrics. Users can switch between target devices seamlessly and access documentation for detailed guidance.

Core features of Roofline

Ahead-of-time compilation for resource-constrained edge devices
Full support for LLM pipeline including tokenizer and integrations
Quantization support for INT8, blocked, symmetric, asymmetric, and dynamic schemes
Heterogeneous execution across CPU, GPU, and NPU
Simple Python interface for model compilation and device switching
Performance dashboard for tracking real-world performance
Support for a wide range of models and hardware backends