ChatGPT, Amazon's Alexa, and Google's Gemini can dramatically cut the time you spend researching products online, but they fail when it comes to verifying details or confirming that an item is genuinely a good deal. That's the central finding from a new hands-on test by PCMag, which examined three distinct AI-powered shopping approaches: deep research tools in ChatGPT and Gemini, Alexa's product question feature within Amazon, and Gemini's Virtual Try-On for clothing on Google Shopping.
The investigation reveals a critical gap: while these assistants can aggregate information, summarize reviews, and highlight key specs in seconds, they often make factual errors, misattribute product features, and cannot be relied upon to catch misleading listings or fake reviews. For consumers eager to hand off the tedium of online shopping research to AI, the message is clear—you're still on your own for the final verification step.
What the PCMag Test Uncovered
PCMag's reporters ran multiple product searches across the three platforms, comparing the output to manual research. The deep research modes in ChatGPT and Gemini generated multi-page reports that pulled data from across the web, often correctly identifying the most popular models in a category and summarizing expert opinions. However, when asked for hard facts—like the exact weight of a laptop or whether a particular camera supports 4K 120fps video—both assistants occasionally returned wrong numbers or conflated specs from different product generations.
Alexa's integration inside Amazon allowed voice or text queries on product pages, such as "Does this air fryer have a dehydrate function?" While Alexa drew from the product listing and customer questions, it sometimes missed answers that were buried in the description or gave overgeneralized responses that didn't match the specific model. In one instance, Alexa incorrectly confirmed a feature that was only available in a higher-tier variant.
Gemini's Virtual Try-On for clothing on Google Shopping fared better visually, letting users see how garments looked on a range of body types. But the tool couldn't convey fabric quality, fit accuracy, or durability—information that often requires digging into detailed reviews or manufacturer specifications. When asked to cross-check size charts between brands, Gemini sometimes applied generic conversion rules, leading to potential sizing errors.
What This Means for You
For Everyday Shoppers
If you've been tempted to let AI find your next laptop, kitchen gadget, or pair of jeans, the technology can genuinely shorten the research phase. Instead of opening twenty browser tabs and reading through dozens of reviews, you can ask ChatGPT or Gemini to compare the top three models, summarize pros and cons, and even highlight which one is best for your needs. That can save hours.
But you must treat the AI's output as a starting point, not the final word. Double-check any specification that matters to you, especially if it's a dealbreaker feature. If an AI says a vacuum cleaner has a HEPA filter, verify it on the manufacturer's product page. If it claims a phone has an SD card slot, look at the official specs. The time saved in research should be reinvested into manual verification of the details you care about most.
For Power Users and Admins Buying in Bulk
IT professionals or office managers procuring equipment in quantity face higher stakes. AI tools might quickly generate a shortlist of monitors or keyboards that fit a budget, but they aren't reliable enough to ensure compliance with enterprise requirements—like TAA compliance, EPEAT certification, or exact port configurations. A single wrong specification could lead to a costly return or deployment delay. Treat AI as a pre-filter, but always confirm critical specs against manufacturer spec sheets or with a vendor representative before purchasing.
For Sellers and Marketers
If you run an e-commerce business, the rise of AI shopping assistants means your product listings must be more accurate and complete than ever. These tools scrape and interpret your listings, and mistakes or omissions will be amplified. Ensure your product titles, descriptions, and tech specs are consistent across platforms. Include detailed attribute tables that AI can parse easily. The more structured your data, the less likely an assistant will misattribute a feature—and the less likely you'll face returns from misled customers.
How We Got Here
The integration of AI into shopping isn't new—retailers have used recommendation engines and chatbots for years. But the current wave began with the public release of large language models and generative AI. In late 2023, OpenAI introduced "deep research" capabilities in ChatGPT, which could scour the web and compile reports. Google followed with similar functionality in Gemini. Amazon, which already anchored Alexa in its shopping ecosystem, rolled out more granular product Q&A features that could answer contextual questions about items.
These launches promised a future where AI would slash the cognitive load of comparison shopping. Early tests were promising but often flawed. Models hallucinated features, misread review sentiment, and sometimes recommended products that were out of stock or discontinued. The technology has improved, but as PCMag's latest test shows, it's still not trustworthy for verification.
This pattern mirrors broader challenges with AI in other domains: large language models are excellent at synthesizing information that appears plausible, but they lack true understanding and can't reliably verify facts. In shopping, where the details make all the difference, that limitation is particularly costly.
What to Do Now
Despite the caveats, you can still use AI shopping tools effectively if you follow a few principles:
- Use AI for broad comparison, not final decisions. Ask assistants to identify the top five air purifiers under $300 and summarize what reviewers say about each. That's a task that would take you an hour—AI can do it in seconds.
- Always spot-check key claims. Pick the three most important features for your purchase and verify them personally. If the AI says a laptop weighs 2.5 pounds, find the official specs. If it claims a blender has a glass pitcher, check the product Q&A on the retailer's site.
- Combine multiple tools for different strengths. For clothing, use Gemini's Virtual Try-On to gauge appearance on different body types, but then turn to detailed text reviews for fit and fabric quality. For electronics, you might use ChatGPT for initial research but then consult a site known for thorough, hands-on reviews.
- Share feedback with the platforms. When Alexa gives a wrong answer, use the feedback button. When ChatGPT misstates a spec, let it know. These systems improve through human correction, and your input can help future shoppers.
- For Alexa shoppers specifically: Stick to questions that are likely answered in the product listing or customer Q&A. Avoid open-ended subjective questions like “Is this the best laptop for students?”—Alexa isn't a reviewer.
The Outlook
AI shopping assistants will get better, but the fundamental verification problem won't disappear soon. Next-generation models may gain the ability to click into product pages in real time and cross-reference details, but until they can truly understand nuance and spot deceptive claims, the final verification will remain a human task.
In the near term, expect tighter integration between AI assistants and retailer databases. Amazon, for instance, could give Alexa direct access to structured product specifications rather than relying on natural-language extraction. Google might combine Virtual Try-On with user-submitted photos and return data. OpenAI may allow ChatGPT to query live inventory APIs. Each step will narrow the reliability gap.
For now, the practical takeaway is clear: let AI do the heavy lifting of research, but keep your own hands on the wheel when it's time to click "buy."