Apple has released a new open-source AI model, called “MGIE,” that can edit images based on natural language instructions. MGIE, which stands for MLLM-Guided Image Editing, leverages multimodal large language models (MLLMs) to interpret user commands and perform pixel-level manipulations. The model can handle various editing aspects, such as Photoshop-style modification, global photo optimization, and local editing.

MGIE is the result of a collaboration between Apple and researchers from the University of California, Santa Barbara. The model was presented in a paper accepted at the International Conference on Learning Representations (ICLR) 2024, one of the top venues for AI research. The paper demonstrates the effectiveness of MGIE in improving automatic metrics and human evaluation, all while maintaining competitive inference efficiency.

For Apple, MGIE also highlights the company’s growing prowess in AI research and development. The consumer tech giant has rapidly expanded its machine learning capabilities in recent years, with MGIE being perhaps its most impressive demonstration yet of how AI can enhance everyday creative tasks.

While MGIE represents a major breakthrough, experts say there is still plenty of work ahead to improve multimodal AI systems. But the pace of progress in this field is accelerating quickly. If the hype around MGIE’s release is any indication, this type of assistive AI may soon become an indispensable creative sidekick.

The VentureBeat report goes on to address 'How does MGIE work; What can MGIE do?; How to use MGIE?; and, Why is MGIE so important. Below are a few MGIE examples.