Using AI for XR development improves developer experience and accelerates WebXR prototyping.
XR development traditionally requires a broad range of skills; creating a high barrier to entry and slowing down prototyping cycles. With the release of Canvas in the Gemini web app, the Android XR team explored making immersive computing accessible to developers and users at large.
To evaluate the pipeline, Google’s team collaborated to build an immersive biology simulation. The generative model calculated the complex interactions between blood cells and rendered them visually using WebGL and Three.js.
According to Google, the initial output “looked like a living textbook illustration,” but the format changed when the developers converted that content to XR using WebXR APIs. “Exploring car-sized blood cells at a sub-cellular scale demonstrated Gemini’s capability to build fully realised, interactive XR experiences,” explained the company.
Manual environment creation consumes heavy developer hours. A task that costs a senior XR engineer an entire day of development can be completed by Canvas using AI in under a minute. Accelerating this process allows technical architects to validate concepts before committing budget to full production cycles.
The output becomes a web link that developers can share with anyone. Relying on standard web links simplifies distribution and avoids the deployment friction associated with compiled application packages. These tools allow development teams to build complex 3D prototypes and interactive models faster than traditional methods.
Hardware requirements and system implementation
Testing these environments does require specific enterprise-grade hardware. Developers will need a Samsung Galaxy XR headset and access to Gemini 3 Pro.
The integration starts in the Gemini web interface. Platform engineers can start with the pre-built XR Blocks Gem, or build a custom version by downloading the XR Blocks ‘ultra-prompt’. This ultra-prompt gives Gemini advanced perception capabilities for realistic AR physics.
Configuration requires defining a system persona. Users upload the file to a new Gem in the Gemini web interface and add a description like: “You are a lead spatial computing architect specialising in high-performance WebGL rendering, efficient asset optimisation, and secure enterprise data visualisation.”
Establishing this context then allows Gemini to interpret the visual scene and use image-generation tools to create 3D textures. To launch the experience, users open Gemini in Chrome on the Galaxy XR, start a chat with the XR Gem, and select Canvas.
Iterative development using AI within XR
Once inside the environment, developers build interactive components using plain text commands. An example includes creating a dandelion that reacts to touch. The system accepts commands like “create a snowy environment” to change surroundings instantly, or “add a leopard there” to place specific objects.
Engineers can give characters backstories and agency within the scene. If the result requires adjustment, developers can try again or layer simple ideas together to create complexity.
For even faster iteration, users can prompt Gemini to embed Gemini Live within the experience. This setup allows developers to talk to Gemini directly, or even continue creating without leaving the immersive session.
By adopting WebXR APIs over proprietary game engines, enterprise teams maintain vendor neutrality and adhere to open web standards. Removing the 3D rendering bottleneck lets development teams dedicate 100 percent of their time to application logic and data security.
See also: AWS CEO describes AI undermining software fears as ‘overblown’
Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events including the Cyber Security & Cloud Expo. Click here for more information.
Developer is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.
