Apple has made a significant breakthrough in using large language models (LLMs) on limited-memory devices like iPhones. By leveraging innovative flash memory usage techniques, Apple AI researchers have developed a methodology that enables efficient creation of LLM-based chatbots and other AI models in limited memory conditions.
Traditionally, LLM-based applications posed a challenge for devices with limited memory, such as iPhones, primarily due to high data and memory requirements. However, Apple’s innovative approach utilizes the abundant flash memory resources available on these devices to store AI model data.
In their work titled “Snapshot LLM: Efficient Inference from Large-Language-Model on Limited Memory,” Apple researchers explain two key techniques that are employed to maximize flash memory throughput and minimize data transfer:
1. Windowing Method: This technique involves reusing processed data instead of continually loading new data. This eliminates the need for constant memory retrieval and simplifies the process, leading to increased speed and smoother operation.
2. Row-Column Clustering: Similar to reading a book in larger chunks, this technique effectively groups data, allowing for quick retrieval from flash memory. As a result, the AI’s ability to understand and generate language significantly improves.
According to the article, these techniques allow for running AI models that are twice the size of the available memory on iPhones. This translates to a noticeable speed increase of 4-5 times on standard central processing units (CPUs) and an impressive improvement of 20-25 times on graphics processing units (GPUs).
The implications of this revolution are immense, as it paves the way for advanced Siri functionalities, live language translation, and advanced AI features in photography and augmented reality on future iPhones. Moreover, this technology enables running complex AI assistants and chatbots on devices, aligning with Apple’s aspirations in this field.
Apple’s commitment to generative AI is also evident in their development of the “Ajax” generative model intended to rival OpenAI’s GPT series. Operating on a massive 200 billion parameters, internally referred to as “Apple GPT,” Ajax underscores Apple’s strategy of deep AI integration across all areas of their ecosystem.
While reports suggest that newer OpenAI models may surpass Ajax’s capabilities, it can be expected that Apple will integrate generative AI into the iOS system by around 2024. The combination of cloud-based AI and on-device processing allows for Apple’s vision where AI seamlessly integrates into the user experience of their devices.
FAQ
What is LLM?
LLM (Large Language Model) is a large language model that utilizes artificial intelligence techniques to understand, generate, and process natural language. These advanced models can be used in various applications such as chatbots, language translation, content generation, and many others.
What are the benefits of using flash memory in devices with limited memory?
Flash memory offers large capacity and fast data access, which is extremely useful in the case of devices with limited memory, such as iPhones. By employing techniques like windowing and row-column clustering, flash memory can be efficiently utilized for storing AI model data, leading to increased speed and smooth operation of AI applications on these devices.
Source: Article inspired by content from example.com
The source of the article is from the blog toumai.es