November 30, 2022.
The date when ChatGPT was released.
The date that marked the beginning, according to some, of the AI revolution.
Everything seemed normal until SEO professionals noticed something: a significant number of their queries were being resolved by ChatGPT instead of Google.
This led many to wonder:
Even Google recognized the urgency of the situation, declaring a "code red" in what many considered an exaggerated reaction to the threat against their business model.
In response to this challenge and as part of efforts to integrate AI into their products, Google introduced the Search Generative Experience (SGE) on May 10, 2023.
One year later, on April 25, 2024, this is how Google’s CEO Sundar Pichai presented the SGE: “For nearly a year, we’ve been experimenting with SGE in Search Labs across a wide range of queries. And now we’re starting to bring AI overviews to the main search results page. We are being measured in how we do this, focusing on areas where gen AI can improve the Search experience, while also prioritizing traffic to websites and merchants… We’ve already served billions of queries with our generative AI features… Most notably, based on our testing, we’re encouraged that we’re seeing an increase in Search usage among people who use the new AI overviews, as well as increased user satisfaction with the results…”
In short, SGE represents a significant evolution in Google Search by incorporating AI capabilities to enhance user experience. This includes:
SGE will change the way users interact with Google.
Powered by an entirely new algorithm, SGE differs fundamentally from traditional Google search and will, therefore, completely redefine SEO.
This change brings uncertainty and fear, but that’s not a solution.
We must embrace this change as inevitable.
We must test, analyze, and experiment to gain insights and make data-driven decisions.
With this growth mindset, we have conducted a comprehensive research study on search generative experience.
In this article, we will present the insights and findings that provide a deeper understanding of SGE.
For this research to be most effective, we analyzed keywords across different verticals:
In total, we analyzed 100,000 keywords.
We’ll now break down the research study for each vertical.
As for all other verticals, granularity helps us to have more precise data and avoid overbroad conclusions. We analyzed 28,000 eCommerce keywords across 14 industries and 10 modifiers.
Industries:
Keyword modifiers:
SGE result is not available
SGE is enabled only after clicking the "Generate" button
SGE is enabled automatically
The layout of SGE results varies significantly across different keyword types and intents.
We segmented each Search Generative Experience (SGE) result into various components to thoroughly analyze its characteristics, visibility, and other crucial metrics influencing user experience and interaction with Google Search.
Let’s take a look at the sample SGE result for the keyword “door mats”:
Summary text
Google provides concise summarizations for queries within SGE results, offering a snapshot overview of the topic or product.
These succinct descriptions aim to quickly inform users, while selected references (within Citation parts with links) offer a starting point for deeper exploration.
This refinement of information into bite-sized texts is part of what we examined, assessing their role in improving the search experience.
Product Listings
This part is the single most significant change for eCommerce SERPs
Google now directly aggregates product data from its extensive database, including external sources, to display products within search results.
This integration allows users to view essential product information on Google's interface itself, streamlining the shopping experience by minimizing the need for additional site navigation.
Each listed item features a succinct description along with an interactive element. Clicking this element expands the view, providing detailed product insights within the search engine results page.
Single Product Link
This view showcases product images, descriptions, and direct links to up to three or more vendor sites, depending on the product's popularity.
Google displays the price point next to each vendor link, highlighting any special offers or discounts.
Additionally, Google consolidates user reviews from various platforms, allowing users to easily navigate and assess the product's reputation, informed by a broad spectrum of consumer feedback.
Product Listing Expansion
The SGE result initially offers a snapshot of three products. As the results are expanded, the SGE view provides an overview of 10 or 25 products. Further details on this will be covered later in the research.
Follow-up Questions
This feature transforms user interaction with Google. These prompts intuitively align with the user's next logical step in their search or purchasing journey, facilitating a seamless and guided experience.
“Ask a follow up” Feature
This segment of SGE results showcases the transformative power of generative AI in search.
It allows users to ask more complex questions beyond keywords, enhancing the depth and breadth of information discovery on the web.
Carousel Links
Google displays links to external sources only in the carousel and citation sections (of course, including product links to vendor sites).
These collections serve as a comprehensive index of resources that Google has utilized to compile the SGE result.
Featuring prominently in these sections can be a significant objective for SGE SEO strategies, as they are the primary points through which Google directs traffic to external sites.
SGE Enablement
Over 81% of 28,000 analyzed eCommerce keywords (or, more precisely, 22,716 keywords) had SGE results enabled.
Such a large magnitude of impact.
Types of SGE results
Loading Time
LLMs work behind the scenes to generate SGE results - and as per Google’s employees, this process will get faster over time.
The graph presents a bimodal distribution, characterized by two prominent peaks. These peaks suggest the presence of two significant frequencies or values within the data set.
The reason for this is obvious: SGE loading time is mainly influenced by the type of SGE result (“button” generated versus automatically generated).
Here are the statistics behind the graph:
Loading time for automatically generated SGE results
.png)
Here are the statistics behind the graph:
Loading time for button-generated SGE results
.png)
Here are the statistics behind the graph:
Text Length (Characters) statistics
.png)
Text Length (Words) statistics
.png)
Analysis of Vertical Movement for Top Traditional Organic Results, Measured in Pixels
This data shows unequivocally - when displayed, SGE results take up significant space on the screen - and will significantly affect the CTR of organic results in the traditional SERP!
SGE button-generated results
Automatically generated SGE results
Follow Up Questions
In 99.38% of cases, the number of follow-up questions per SGE results is 3.
In this image, we can see the distribution of question modifiers across all analyzed follow-up questions:
The most popular question modifier is “What,” and here are the sample questions for various keywords:
The Elvion team will further analyze how Google chooses which questions to ask in this section and their semantic connection to the keyword and the purchasing journey.
There are three sections where Google directs traffic to external sources:
Let’s dive deeper into each section.
Statistical Overview of Link Counts in Carousel Section
.png)
Important note: The carousel section (and therefore links) was not fully available in 27.56% of cases - and in 75% of cases where it was available, the number of links within the carousel section was 4 or less!
Statistical Overview of Link Counts in Carousel Section
.png)
On average, citation sections have more links. However, those links are mostly only displayed after clicking the dropdown button.
Statistical Overview of Unique Link Counts in SGE
This answers the question of how many links are in the total Google features in the SGE section.
.png)
This makes everything more competitive, doesn't it?
The insight we've all been eagerly waiting for.
Out of 79,992 SGE links we analyzed from 22,716 SGE-enabled results, 79.69% don’t rank in traditional organic SERP, while only 10.26% rank in the top 10 results!
However, this distribution makes sense in the context of SGE. And we'll explain how.
Out of 22,716 keywords with SGE-enabled results, we got 79,992 SGE links (URLs ranking in the carousel and/or citation section) from 10,738 unique domains. However, one sample of analyzed keywords contained the “amazon” modifier, which naturally triggers results from Amazon’s domain. We systematically omitted those keywords to eliminate potential bias and uphold the rigorous validity of our analysis.
After that exclusion, we got 70,075 SGE links from 10,297 unique domains. Here are the results of the visibility analysis based on that data set:
This brings us to an important finding: most websites with the highest visibility in the SGE results are not even eCommerce websites.
Let’s now see how that is connected to the rankings data.
Previously, you would get eCommerce category pages if you searched for any broad short-tail eCommerce keyword.
But now, Google doesn’t need category pages for product discovery.
Google wants users to browse and discover products on its platform.
Google is becoming more like a marketplace rather than solely a search engine.
This is the breaking point—besides its Shopping Graph data, which has over 35 billion product listings, Google uses one more thing for SGE product listings: external sources.
And those external sources are listicles and review articles.
This is the statistics behind that pattern.
Interesting, right?
Let’s now take a look at specific cases where articles serve to rank specific products.
SGE results for eCommerce keywords feature listicles and review types of articles, while traditional SERPs feature category pages. That’s the main reason why over 79% of SGE links don’t rank in Google’s search results.
* Note: as for all visibility studies, we systematically omitted keywords featuring the 'amazon' modifier to eliminate potential bias and uphold the rigorous validity of our analysis.
These are the results of our visibility analysis:
As results suggest, Amazon has by far the highest visibility among all other websites.
Do you think this is fair?
We think not, but it will be a reality, and other eCommerce sites will have to perfect their product pages to increase visibility.
As we already noted - granularity helps us to have more precise data and avoid overbroad conclusions.
SGE enablement
Types of SGE results
Average loading time for different types of SGE results
Average text length (words) In SGE results
Analysis of Vertical Movement for Top Traditional Organic Results
Average number of links per SGE result
Statistical overview of SGE links not ranking in organic search results
Statistical overview of enabled product listings within SGE Results
Home & Furniture
Fashion & Apparel
Electronics & Appliances
Technology & Gadgets
Sports & Outdoor Equipment
Beauty & Personal Care
Media & Entertainment
Garden & Outdoor Living
Toys, Games & Hobbies
Pet Supplies & Care
Automotive & Accessories
Arts, Collectibles & Antiques
SGE enablement
Type of SGE results
Average loading time for different types of SGE results
Average text length (words) In SGE results
Analysis of Vertical Movement for Top Traditional Organic Results
Average number of links per SGE result
Statistical overview of SGE links not ranking in organic search results
Statistical overview of enabled product listings within SGE Results
“Best” in carousel links’ titles analysis
“Review” in carousel links’ titles analysis
We analyzed 10,000 software-related keywords across different modifiers:
Sample SGE result for the keyword “project management software” (we had to cut a part from the image due to the SGE result’s size!):
They facilitate the discovery of various software solutions and emphasize key decision-making factors in an accessible format.
Summary text
The summary text includes a citation button, which, when selected, reveals all the sources referenced to compile that information.
Selection of software options
This section presents a vetted selection of software options, each accompanied by a concise description to give a snapshot of the product's features and utility.
However, Google does not include direct links to the products' websites but provides images sourced from third-party sites.
This approach emphasizes a visual and descriptive introduction to the software without immediate redirection away from the search results page.
Citation Links
Citation sections are the only places where Google displays links to external sources.
These collections serve as a comprehensive index of resources that Google has utilized to compile the SGE result.
Featuring prominently in these sections can be a significant objective for SGE SEO strategies, as they are the primary points through which Google directs traffic to external sites.
As shown later in the presentation, unlike SGE results for other types of keywords, Google prioritizes citation sections above carousel links for software-related keywords.
Follow-up Question Features
SGE Enablement
Over 95% of 10,000 analyzed software keywords had SGE results enabled.
However, not all of the SGE-enabled keywords generated meaningful SGE results.
Namely, 5.7% of SGE-enabled keywords have SGE results enabled, but SGE is not generated. We call them false positive results.
This is what a false positive result looks like:
Types of SGE results
Loading Time
.png)
Revisiting the data, we see two peaks and two significant frequencies or values within the data set. This pattern correlates with the observation that automatically generated SGE results load faster than those generated on-demand (button-generated).
Here are the statistics behind the graph:
Loading time for automatically generated SGE results
Here are the statistics behind the graph:
Loading time for button-generated SGE results
.png)
Here are the statistics behind the graph:
Text Length (Characters) statistics
.png)
Here are the statistics behind the graph:
Text Length (Words) statistics
.png)
Analysis of Vertical Movement for Top Traditional Organic Results, Measured in Pixels
SGE button-generated results
.gif)
Automatically generated SGE results
.gif)
Follow Up Questions
In 99.16% of cases, the number of follow-up questions per SGE results is 3.
In this image, we can see the distribution of question modifiers across all analyzed follow-up questions:
The most popular question modifier is “What” (as for all other keyword types) and here are the sample questions for various keywords:
Note: since only 1.44% of SGE-enabled results had featured carousel sections, we didn’t make analyses for carousel and citation sections separately, but rather one, for all types of SGE links.
A different distribution compared to the eCommerce segment.
Out of 61,668 SGE links analyzed:
Out of 8,968 keywords with SGE-enabled results, we got 61,668 SGE links (URLs ranking in the carousel and/or citation section) from 15,427 unique domains.
Here are the results of the visibility analysis based on that data set:
Again, the most popular types of webpages Google features within SGE results are listicles and review articles. Here is the brief statistictics for citation links:
SGE enablement
False positive results
Average text length (words) In SGE results
Analysis of Vertical Movement for Top Traditional Organic Results
Average number of links per SGE result
Statistical overview of SGE links not ranking in organic search results
We analyzed 10,000 YMYL keywords across three main categories:
YMYL Health
YMYL Finance
YMYL Society
Sample SGE result for the keyword “abscess tooth symptoms”.
Summary text
The YMYL SGE results offer a structured overview, presenting critical information in clearly defined sections.
The summary text systematically categorize details into distinct boxes that address various aspects of the user's query.
This layout is designed to deliver essential information with precision, separating symptoms, treatments, and other details (depending on the YMYL query type) into segmented units for quick reference.
Citation Links
Similar to other keyword types, YMYL SGE results also have a dedicated section that cites the sources and references supporting the provided information.
This collection of links serves as a foundation for the credibility of the SGE content, providing users with a way to access the original information and more context.
Disclaimer Section
Given the very sensitive nature of YMYL queries, SGE prominently features a disclaimer advising users to seek professional assistance.
This serves as an ethical reminder that while the information provided is thorough, it cannot substitute for expert advice.
Follow-up Question Features
SGE Enablement
And this is how SGE enablement looks like across three main YMYL types of queries:
It turns out that Finance YMYL is by far the most “resistant” group. These are examples of Finance YMYL keywords for which there is no SGE result:
On the other side, both Health YMYL and Society YMYL have only around 24% keywords that aren’t SGE enabled.
The YMYL Health keyword section is unique because we analyzed several modifiers within it. Here is the graph with the results:
The modifiers “uses” and “dosages” are the “strictest”, with “zero tolerance” for SGE results, followed by “side effects”, where only 0.93% of keywords had SGE results enabled.
Types of SGE results
Most SGE results are generated automatically, which is unique only to the YMYL segment of keywords. Now, let's see this data across YMYL groups:
YMYL health SGE results are in 96.41% of cases automatically generated! This is an obvious trend, and is similarly present in the other two YMYL groups.
Loading Time
Due to the fact that most results are automatically generated, there aren't two significant frequencies or values within the data set. We spot only one, with a minority continuing without creating a significant peak.
Here are the statistics behind the graph:
Text Length (Characters) statistics
Here are the statistics behind the graph:
Text Length (Words) statistics
Analysis of Vertical Movement for Top Traditional Organic Results, Measured in Pixels
SGE button-generated results
Automatically generated SGE results
Follow Up Questions
In 97.25% of cases, the number of follow-up questions per SGE results is 3.
In this image, we can see the distribution of question modifiers across all analyzed follow-up questions:
Here are the sample questions for various keywords with “What” as a modifier:
Again, since only 52 out of 6,085 SGE-enabled keywords featured carousel link section, we didn’t make separate analyses for carousel and citation sections separately, but rather one, for all types of SGE links.
Out of 37,231 SGE links we analyzed in total:
Google doesn't play around when it comes to YMYL queries - only authoritative sites that otherwise rank can create an SGE result!
We analyzed 9,255 SGE links (URLs ranking in the carousel and/or citation section) from 2,760 unique domains.
Here are the results of the visibility analysis based on that data set:
We analyzed 18,751 SGE links (URLs ranking in the carousel and/or citation section) from 2,269 unique domains.
Here are the results of the visibility analysis based on that data set:
We analyzed 9,225 SGE links (URLs ranking in the carousel and/or citation section) from 4,654 unique domains.
Here are the results of the visibility analysis based on that data set:
Interesting that LinkedIn is in second position, isn't it?
We were also surprised. However, when we looked at the actual SGE results, we saw quite a few articles from the LinkedIn Pulse section. Here are some examples:
We analyzed 30,000 informational keywords across multiple modifiers and industries:
Sample SGE result for the keyword “how to wash pillows”:
Summary text
SGE Links (Citation and Carousel Section)
Follow-up Questions
“Ask a follow up” Feature
Video Section
Over the past few years, Google has pushed more YouTube videos into organic SERPs as they are more eligible to satisfy user intent for informational keywords.
The video is prominently displayed within the SGE results and grabs the most attention of all the other elements.
Moreover, when a user interacts with a video, it is displayed within Google's ecosystem, not redirecting the user away from the search page. This feature indicates Google's goal to maintain users on its platform, utilizing SGE as an effective means to this end.
SGE Enablement
Over 95% of 30,000 analyzed informational keywords (or, more precisely, 28,562 keywords) had SGE results enabled.
However, 4.15% of SGE-enabled keywords have SGE results enabled, but SGE is not generated (false positive results).
Types of SGE results
Loading Time
Again, we see two peaks and two significant frequencies or values within the data set.
Here are the statistics behind the graph:
Loading time for automatically generated SGE results
Here are the statistics behind the graph:
Loading time for button-generated SGE results
Here are the statistics behind the graph:
Text Length (Characters) statistics
Here are the statistics behind the graph:
Text Length (Words) statistics
Here are the statistics behind the graph:
Analysis of Vertical Movement for Top Traditional Organic Results, Measured in Pixels
SGE button-generated results
On average, the first results in the organic traditional SERP move by 1082.03px.
Here are the statistics:
Automatically generated SGE results
On average, the first results in the organic traditional SERP move by 399.73px before the full expansion of the SGE result, and by 1008.06px after it.
Here are the statistics:
Follow Up Questions
In 95.08% of cases, the number of follow-up questions per SGE results is 3.
Here is the distribution of question modifiers across all analyzed follow-up questions:
Statistical Overview of Link Counts in Carousel Section
Note: only 526 (1.96%) out of all SGE-enabled keywords had links in the carousel section.
Statistical Overview of Link Counts in Citation Section
Statistical Overview of Unique Link Counts in SGE Results
We got 160,244 SGE links (URLs ranking in the carousel and/or citation section) from 39,199 unique domains.
Here are the results of the visibility analysis based on that data set:
Note: the distribution of the domains across SGE links and results is mainly influenced by the type of keywords analyzed. This type of analysis serves us to see which websites are more eligible to be shown within SGE results.
SGE enablement
Type of SGE results
Analysis of Vertical Movement for Top Traditional Organic Results
Average number of links per SGE result
Statistical overview of SGE links not ranking in organic search results
We analyzed 10,000 local keywords across two main groups:
Sample SGE result for the keyword “24 hour restaurants near me”:
Summary text
SGE Links (Citation and Carousel Section)
Local Pack Results
Google's 'Local Pack' feature is designed to answer localized search queries, prominently displaying results that align with geographic relevance.
This enriched version of the typical SERP local pack includes concise descriptions of each listed place.
Interaction with any of these listings triggers a pop-up window, offering comprehensive insight into the place's Google My Business profile.
Google Maps
Consistent with Google's approach to localized search queries, a Google Maps segment is automatically activated.
This feature emphasizes geographic accuracy and enhances the search experience's relevance by visually representing the proximity of results.
SGE Enablement
Over 86% of 10,000 analyzed local keywords (or, more precisely, 8,630 keywords) had SGE results enabled.
However, 3.2% of SGE-enabled keywords have SGE results enabled, but SGE is not generated (false positive results).
Types of SGE results
Loading Time
Due to the frequency and dominance of button-generated SGE results over automatically enabled ones, the average loading time is drastically higher compared to other types of keywords and verticals.
Text Length (Characters) statistics
Text Length (Words) statistics
Analysis of Vertical Movement for Top Traditional Organic Results, Measured in Pixels
SGE button-generated results
On average, the first results in the organic traditional SERP move by 1,114px.
Here are the statistics:
Automatically generated SGE results
On average, the first results in the organic traditional SERP move by 399.67px before the full expansion of the SGE result, and by 1209.8px after it.
Here are the statistics:
Local Pack Results Statistics
Follow Up Questions
In 98.71% of cases, the number of follow-up questions per SGE results is 3.
Here is the distribution of question modifiers across all analyzed follow-up questions:
Statistical Overview of Link Counts in Carousel Section
Statistical Overview of Link Counts in Citation Section
Statistical Overview of Unique Link Counts in SGE Results
We analyzed 26,813 SGE links (URLs ranking in the carousel and/or citation section) from 7,763 unique domains.
Here are the results of the visibility analysis based on that data set:
"Near Me" keywords definitely trigger more SGE results than "In-person visit" ones:
And now, very interesting data: "Near Me" keywords are generated on demand (by clicking a button) in 99% of cases, while "In-person visit" ones have an even distribution:
Here we can see the data from the comparative analysis:
We analyzed 2,000 time-sensitive keywords, such as: "austin weather today," "basketball results live," "charlotte events tomorrow," "comedy shows chicago tonight," etc.
In this segment, we have absolutely the same SGE elements as for all other keyword types: summary text, SGE links (from citation and carousel sections), follow-up questions, and "ask a follow-up" feature.
SGE Enablement
Over 91% of 2,000 analyzed time-sensitive keywords (or, more precisely, 1,816 keywords) had SGE results enabled.
However, 2.86% of SGE-enabled keywords have SGE results enabled, but SGE is not generated (false positive results).
Types of SGE results
Loading Time
Again, as for local keywords, the vast majority of SGE results were generated by clicking a button. That definitely affects the average load time.
Also, due to the smaller sample, we do not see significant frequencies on the graph but a more random distribution.
Text Length (Characters) statistics
Text Length (Words) statistics
Analysis of Vertical Movement for Top Traditional Organic Results, Measured in Pixels
SGE button-generated results
On average, the first results in the organic traditional SERP move by 1052.5px.
Here are the statistics:
Automatically generated SGE results
Local Pack Results Statistics
Follow Up Questions
In 99.43% of cases, the number of follow-up questions per SGE results is 3.
Here is the distribution of question modifiers across all analyzed follow-up questions:
SGE Links Analysis - Statistical Overview of Unique Link Counts in SGE Results
We analyzed 4,340 SGE links (URLs ranking in the carousel and/or citation section) from 1,170 unique domains.
Here are the results of the visibility analysis based on that data set:
We analyzed 10,000 keywords across multiple modifiers and industries:
SGE Enablement
Over 90% of 10,000 analyzed service-oriented and professional keywords (or, more precisely, 9,045 keywords) had SGE results enabled.
However, 9.3% of SGE-enabled keywords have SGE results enabled, but SGE is not generated (false positive results).
Types of SGE results
Loading Time
We see two peaks and two significant frequencies or values within the data set.
Here are the statistics behind the graph:
Loading time for automatically generated SGE results
Here are the statistics behind the graph:
Loading time for button-generated SGE results
Text Length (Characters) statistics
Text Length (Words) statistics
Analysis of Vertical Movement for Top Traditional Organic Results, Measured in Pixels
SGE button-generated results
On average, the first results in the organic traditional SERP move by 1216.33px.
Here are the statistics:
Automatically generated SGE results
The SGE result is so large that we had to reduce the image to fit in this preview!
On average, the first results in the organic traditional SERP move by 400.49px before the full expansion of the SGE result, and by 1099.68px after it.
Here are the statistics:
Follow Up Questions
In 99.58% of cases, the number of follow-up questions per SGE results is 3.
Here is the distribution of question modifiers across all analyzed follow-up questions:
Statistical Overview of Link Counts in Carousel Section
Statistical Overview of Link Counts in Citation Section
Statistical Overview of Unique Link Counts in SGE Results
We analyzed 31,843 SGE links (URLs ranking in the carousel and/or citation section) from 11,664 unique domains.
Here are the results of the visibility analysis based on that data set:
This keyword segment is very similar compared to eCommerce and Software when it comes to the preferred type of pages in SGE results (listicles, review pages, and marketplace websites).
“Best” & “Review” In The SGE Links’ Titles - Statistics
The analyzes clearly show that SGE results are dominant in all keyword categories:
Keywords within the "startup" group have the most false positives, pointing to the fact that LLMs and SGE cannot keep up with the latest information (more on this in future research).
Analysis of Vertical Movement for Top Traditional Organic Results, Measured in Pixels, Across Different Keyword Groups
Without excessive deviations, the values for each category match the global average values:
Once again, the "startup" keyword category has the largest outliers, with nearly 47% of SGE links not ranking in the Google organic SERP.
Our study utilizes a JavaScript library called Puppeteer, which simulates a real user by responding to our commands. We use Google Chrome as the browser, and the Puppeteer servers are located in San Francisco, USA. The browser viewport is set to 1920 x 1080 pixels.
The workflow is as follows:
Subsequently, the height of the section is measured. If a button is present, Puppeteer clicks it and continues to gather further information. This includes collecting the text from the SGE section, gathering all links (classified as either carousel or citation), collecting suggested queries, hero clips, and cited locations. Following this, the page is screenshotted again. Finally, all product listings are collected if they are present.
