The AI accelerator runs generative AI models efficiently, achieving up to 40 TOPS with less than five watts of power consumption.
Edge AI specialist Hailo has launched the accelerator family. The development promises to operate generative AI models within a power envelope of less than five watts. The company claims that the internal testing indicates that the new Hailo-10 accelerators can achieve up to 40 tera-operations per second (TOPS) of compute performance for on-device machine learning (ML) and artificial intelligence workloads. The company asserts that this is double the performance of Intel’s Core Ultra NPU, all while consuming half the power. In practical terms, this enhanced performance enables the Hailo-10 to run the Llama2-7B LLM at up to 10 tokens per second, and the Stable Diffusion 2.1 image generation model can produce one image every five seconds, with both operations using less than 5 watts of power.
The Hailo-10 enables users to perform tasks such as real-time translation, summarization services, generating software code, or producing images and videos from text prompts directly on their PCs or other edge systems. This functionality comes without overloading the CPU or depleting the battery.
“As Gen AI on the edge becomes immersive, the focus turns to handling large LLMs [Large Language Models] in the smallest possible power envelope — essentially less than five watts,” claims Hailo’s chief executive officer and co-founder Orr Danon of the company’s latest launch.
“We designed Hailo-10 to seamlessly integrate gen AI capabilities into users’ daily lives,” Danon continues, “freeing users from cloud network constraints. This empowers them to utilize chatbots, copilots, and other emerging content generation tools with unparalleled flexibility and immediacy, enhancing productivity and enriching lives.”
The company has announced its first product in this new line, the Hailo-10H M.2 Generative AI Acceleration Module, which features 8GB of LPDDR4 on-module memory and is compatible with x86 or Arm aarch-64 hosts running Microsoft Windows.
For more information, click here.
