MorphGUI: Real-time GUIs Customization with Large Language Models
Human-centred AI
MorphGUI is a framework that leverages Large Language Models to enable users to customize interfaces using natural language without altering the application's core functionalities. Its structured dual-input design separates functional goals ("what it should do") from visual specifications ("how it should appear"), allowing users to modify interface elements while maintaining the integrity of the underlying system logic. Powered by a dynamic component architecture, MorphGUI applies updates at runtime, preserving both the application state and baseline functionality.
Customization Interface
The "Enhance With AI" panel in MorphGUI enables users to personalize and modify their interface through AI-assisted customization. It features two main sections, Static Settings and Dynamic Settings, allowing configuration of both fixed and adaptive interface components. Within the Dynamic Settings tab, the Personalize section introduces component-level targeting for modifying or adding UI elements. Users specify What it should do to define the element's functional behavior and How it should appear to control its styling and layout. The panel also provides three main actions: Generate to apply the defined changes, Previous to roll back to a previous configuration, and Restore to return the interface to its last stable state.
Workflow & Real-time Updates
MorphGUI combines selected component context, user inputs, and constraints into structured prompts for the LLM, then validates specifications and transforms them into executable UI code. The dynamic component bridges generated code and the app's runtime, handling dependency resolution and compilation in real-time effectively rendering the new UI.
Architecture
The system architecture implements a modular client-server model that enables real-time GUI customization through natural language interaction. The Front-end is a web-based interface built on a component-driven framework that communicates bidirectionally with Dynamic Components for runtime rendering and update of interface elements. These components interface with the Server layer via RESTful APIs to support synchronous data exchange and asynchronous updates. The Server, developed using Node.js and the Express.js framework, orchestrates communication with the AI module,a large language model that parses natural language inputs and translates them into executable interface specifications. The Database layer, implemented with SQLite, manages persistent storage of user preferences, interface states, and component configurations, enabling rollback and version control functionalities. The User Preferences subsystem provides contextual adaptation by maintaining individualized configuration data that influence interface behavior and visual presentation across sessions.
User Study Results
The user study evaluated MorphGUI in terms of usability, learning curve, and interaction effectiveness when customizing graphical interfaces through natural language. Participants performed a series of interface modification tasks using a calendar application. Each participant started with base calendar layout and used MorphGUI to reproduce one or more target interfaces of increasing complexity. The easy task involved straightforward visual or structural edits (e.g., changing colors, repositioning elements), while the complex task required functional and aesthetic modifications, such as adding components, reorganizing layouts, or combining multiple requests into a single prompt.
Results from the System Usability Scale (SUS) and custom Likert-scale items indicate that participants found MorphGUI intuitive, well-integrated, and easy to use, with low perceived complexity. Most participants disagreed with statements suggesting high cognitive demand or inconsistency, confirming a low learning overhead and high user confidence.
Correlation analyses revealed no significant relationship between users' familiarity with large language models (LLMs) and overall task performance (Pearson r = -0.03, p = 0.883), suggesting that even participants unfamiliar with LLMs were able to effectively complete customization tasks. However, a mild positive trend (r = 0.24, p = 0.165) was observed between prompt descriptiveness and task success, implying that clearer and more detailed natural language prompts tended to produce better customization outcomes. Additionally, analysis of template usage patterns showed that participants primarily relied on the "what" field for functional modifications and the "how" field for styling and layout changes, supporting MorphGUI's dual-input design principle.
Publication
Tommaso Calò, Andrea Sillano, Luigi De Russis. In press. MorphGUI: Real-time GUIs customization with large language models. International Journal of Human-Computer Studies, DOI: https://doi.org/10.1016/j.ijhcs.2025.103695