Microsoft Photos in Windows 11: 5 AI Features Changing Photo Editing Forever

Microsoft's Photos app in Windows 11 has quietly evolved into a powerhouse of AI-driven editing tools, transforming casual users into photo editing wizards. The latest updates leverage cutting-edge machine learning to automate complex tasks that once required expensive software or professional skills. Here's how these five groundbreaking features are redefining digital photography for Windows users.

1. Generative Erase: The Magic Wand for Unwanted Objects

The new Generative Erase feature uses diffusion models (similar to those in DALL·E) to seamlessly remove objects and reconstruct backgrounds. Unlike traditional clone stamp tools, this AI analyzes the entire image context to fill gaps convincingly. In our tests, it successfully removed photobombers, power lines, and even large foreground objects while maintaining realistic textures.

How it works:
- Select the object with brush or lasso tools
- AI predicts surrounding pixels and generates plausible replacements
- Processes locally on-device for privacy (no cloud upload required)

2. Background Removal 2.0: Precision Edge Detection

Building on the previous version, the updated background remover now handles tricky edges like hair, fur, and transparent objects with remarkable accuracy. The AI distinguishes foreground subjects at the pixel level, outperforming many third-party apps we compared it against.

Pro tip: Combine this with the new relighting feature to instantly create professional product shots or portrait studio effects.

3. AI-Powered Relighting: Virtual Studio Lighting

This breakthrough feature analyzes depth information and surface angles to simulate directional lighting. You can:
- Change light source positions post-capture
- Adjust intensity and warmth
- Add rim lighting for dramatic portraits

Our experiments showed particularly impressive results with food photography and indoor portraits, where it could "re-light" scenes to appear as if shot with professional studio setups.

4. OCR Search: Find Text in Your Photo Library

The integrated Optical Character Recognition (OCR) now indexes all text in your photos, enabling searches like "receipt from March" or "menu with lobster." Microsoft's AI reads handwritten notes, street signs, and even stylized fonts with surprising accuracy.

Privacy note: All processing occurs locally—your images aren't sent to Microsoft servers for analysis.

5. Smart Album Curation

Leveraging the same AI that powers Windows 11's Widgets, the Photos app now automatically:
- Groups similar shots into stacks
- Identifies best versions based on focus and composition
- Creates themed albums (e.g., "Beach Vacation 2023")
- Detects and highlights important documents

Under the Hood: The AI Architecture

Microsoft revealed these features are powered by:
- ONNX Runtime for efficient ML execution
- DirectML for GPU acceleration
- Proprietary computer vision models trained on diverse image datasets

The app intelligently switches between CPU and GPU processing based on task complexity and hardware capabilities.

Performance Benchmarks

We tested the AI features on various Windows 11 devices:

Limitations and Workarounds

While impressive, we found some constraints:
- Batch processing isn't available for AI features
- RAW file support varies by camera model
- Extreme low-light images challenge the relighting AI

Pro solution: For professional workflows, export to Photoshop or Lightroom after using these tools for initial edits.

The Future of AI Photo Editing

Microsoft's Photos app roadmap suggests upcoming features like:
- AI-powered restoration of old photos
- Style transfer filters
- Automated color grading presets
- Depth-aware filters

These developments position Windows 11 as a surprisingly capable platform for casual and prosumer photo editing—all built into the operating system at no additional cost.