Google did not waste its opportunity to showcase every advancement it has made in its products, and particularly on AI features, during its widely followed I/O developer conference, which took place this week.
Developing and strengthening the company’s artificial intelligence capabilities was clearly a priority in the past months for the search giant as rivals like OpenAI, Microsoft (MSFT), and even Meta Platforms are racing fast to gain dominance in this nascent market.
Major AI upgrades to the firm’s core product offerings like Search, Android, and Workspace apps like Gmail were announced during the conference. There is no doubt that AI is among Google’s current strategic priorities.
Both short-term releases and visionary conceptual products, features, and services were announced during the conference. One theme clearly connected all of these announcements – Google wants to power its entire ecosystem with the latest and most powerful large language models (AI) developed by its AI arm – Google DeepMind. This includes hardware, software, mobile, and web search.
On the practical side for users, upgrades to Google’s flagship Gemini AI service will soon allow more sophisticated, conversational interactions with the company’s virtual assistant on Android phones and tablets. But Google also pulled back the curtain on more speculative innovations like an augmented reality platform called “Project Astra” that could power AI-guided vision and speech capabilities.
Among the most practical enhancements that users will soon enjoy are the latest upgrades performed on Gemini, the company’s sophisticated generative AI solution that will now include a virtual assistant for Android-powered devices.
Moreover, Google (GOOG) pulled the curtain on innovations in the field of augmented reality (AR) with a new platform called “Project Astra” that brings users closer to AI-guided vision and speech.
Search Engine Gets a Long-Awaited AI-Assisted Makeover
Google Search, the company’s crown jewel, is getting revamped with new generative AI capabilities. The company previewed an “AI Organized” version of its classic search results page that combines related topics, insights, and media by tapping on AI models.
These new AI-curated rankings will initially roll out for searches related to dining, recipes, movies, hotels, books, and shopping. However, the AI overhauls go well beyond just presenting information differently.
Google also revealed that its AI-powered search engine will start drafting custom textual overviews to summarize complex topics directly in the results rather than simply linking out to other sites.
This could have major implications for how users consume information online and for websites aiming to drive traffic through Search.
Along those lines, the company is launching new multimodal search features that blend text prompts with images, documents, and other uploads to generate relevant insights tailored to each specific query context.
This richer integration of Gemini across various Google services enables entirely new AI-powered search capabilities that surpass the traditional text matching that made the search engine so popular and widely adopted.
Gmail Is About to Get a Lot Better With the Power of Gemini AI
Gmail is among the first Google apps to get AI-powered writing, analysis, and task assistance via new Gemini integrations. The company showed off how users could ask Gemini to read through a backlog of emails on a certain topic, identify key points and action items, and draft reply messages for streamlined workflows.
Gemini is also coming to Google Docs, Sheets, Slides, and other Workspace productivity tools to give users a hand when drafting, editing, analyzing documents and data, answering follow-up questions dynamically, finding the write formulas to perform certain operations, creating pitch decks, and sorting large data sets as needed.
While Gmail already offered AI writing suggestions through its “Help me write” feature, the new Gemini functionality represents a major leap in terms of user experience. The integration will make the firm’s existing AI assistants far more capable — and embedded — across all of Google’s core business and productivity tools.
Upgrading Smartphone Interactions with Gemini Live
Looking ahead, one of Google’s flashier announcements was a preview of “Gemini Live,” a new feature that aims to deliver more natural, voice-enabled AI conversations on smartphones.
With Live, Gemini will be able to engage in back-and-forth spoken dialogue, seeing and responding to a user’s physical environment through their phone’s camera while pulling in relevant info to answer follow-up questions. It promises an experience that is closer to human-like interactions.
Google’s execs used Live in various scenarios in real-time and showcased how the solution managed to follow the conversation’s thread while retrieving pertinent facts from its knowledge base to lend a helping hand on different tasks.
The company claimed that Live taps on new generative AI techniques that provide more advanced computer vision and speech recognition compared to existing conversational AI assistants.
However, it’s unclear if or when Live may actually be available since the demo was still an early prototype. The concept showcases Google’s vision for an AI assistant that has some degree of situational awareness by relying on intelligent cameras and sensors to make its assessments and provide relevant responses.
Getting the AI Basics Right with Gemini Updates
Google also focused on upgrading Gemini’s architecture in the near term. The Gemini 1.5 Pro model is getting a highly anticipated bump in its contextual understanding window from 1 million to a staggering 2 million tokens.
Having a larger context window will allow Gemini to ingest, process, and generate output from much lengthier documents, videos, websites, code repositories, and other data sources compared to current models. This is a key factor in providing more informed and authoritative outputs.
Google cited examples like being able to break down entire digitized textbooks by using Gemini 1.5 Pro to create interactive lectures and lessons. The model is also being optimized for highly specialized fields like code analysis to accelerate software development workflows with AI assistance.
Updates are coming as well for Gemini Flash (better for speed/latency) and the new Gemini Nano model – embedded directly into the Chrome browser for more privacy-centered on-device AI processing. Google also stressed its efforts around AI security with tools for watermarking and detecting synthetic media among other safeguards.
Introducing ‘Project Astra’ for Augmented Reality Computing
Perhaps the most ambitious initiative revealed at the I/O Conference was Project Astra, a cutting-edge experiment developing AI that could eventually interpret and interact with the real world by using camera sensors and computer vision.
Astra builds on the smartphone-based Gemini Live concept by potentially extending to augmented reality glasses or other wearable computers of the future. Google hinted that the system may allow a user to orchestrate complex and multi-step tasks by using voice and gestures while getting relevant context and assistance from intelligent software.
For example, Astra could theoretically guide someone through household repairs by identifying tools, reading instruction manuals aloud, illustrating 3D models, and walking through each step by using spatial mapping and AR displays.
Sadly, for now, Astra is still in the development stage and is probably years away from being brought to consumer product applications.
AI Features Brought to Android OS Include Circle to Search and Camera Optimization
Mobile solutions were in focus as well during the I/O Conference as Google outlined how its entire ecosystem will become more AI-centric. New features and products showcased improved AI accessibility and capabilities like a Gemini voice call monitoring service that will be able to flag potential scams and optimizie a phone’s camera performance by using artificial intelligence.
The previously announced “Circle to Search” for Android is getting more robust and contextual, allowing users to ask the Gemini assistant for help with virtually anything that they highlight or circle with finger or hand gestures.
Moreover, in a relatively unusual integration, Google is adding AI models directly to Android for on-device malware scanning that can disable infected apps or add permissions and automatically revoke them if threats are detected.
Developers will also enjoy an expanded toolkit to build custom Gemini-powered virtual assistants called “Gems” optimized for various domains, workflows, apps, and services. In essence, these are Gemini-powered customized AI chatbots.
Soothing AI Anxieties with Ethics Focus and Privacy Controls
There’s a lot to say and share from the Google I/O Conference, perhaps more than a single article can cover. However, what seems to be clear is that the company is making AI its strategic priority at the moment and its intentions are way more ambitious than originally thought.
The Google ecosystem will be immediately revamped by Gemini through various features that will be available either immediately or within the next few months.
Also read: Countless Massive Google Announcements Shock the Industry – Here’s Everything We Know
Google highlighted that it is particularly concerned with adding new safeguards that allow it to moderate the output that can be created by its generative AI models to prevent users with ill-gotten intentions to exploit the software for purposes that are harmful to society like spreading misinformation or creating malicious software.
Regardless, the firm’s intense AI focus — with Gemini poised to touch everything within the Google ecosystem including email, document workflows, smartphone cameras, and eventually AR environments — will likely spark concerns and scrutiny regarding data collection, personally identifiable information, and AI model training processes.
This could ultimately emerge as Google’s biggest challenge in navigating the generative AI revolution: retaining users’ trust as AI capabilities grow increasingly intimate, omnipresent, and influential in today’s digitally augmented lives.