BERT , the largest update of the Google algorithm in 5 years, will allow us to better understand the intention of searching for users in context- dependent queries .
BERT = Bidirectional Encoder Representations from Transformers
Using Artificial Intelligence and machine learning to provide more relevant answers , it is estimated that BERT will alter the ranking in 10% of Google’s search results .
According to the article Understanding searches better than ever before , published on October 25, 2019 on the Google blog by Pandu Nayak, vice president of the Google search department, BERT will help you better understand 1 in 10 searches .
In their own words, language comprehension thanks to this machine learning algorithm is allowing a significant improvement in the way Google understands queries , which represents …“… the greatest advance in the last five years, and one of the greatest advances in the history of the search.”
Below you can see the answers to the most frequently asked questions about BERT :
What is BERT?
BERT’s announcement was posted on the official Google account on Twitter :
Google SearchLiaison✔@searchliaison Meet BERT, a new way for Google Search to better understand language and improve our search results. It’s now being used in the US in English, helping with one out of every 10 searches. It will come to more counties and languages in the future.
BERT , an abbreviation for Bidirectional Encoder Representations from Transformers, is a language representation technique based on neural network architecture for Natural Language Processing (NLP) that enables training of a state-of-the-art or conversational question response system .
In other words, BERT helps the rest of Google’s algorithms to better understand the natural language used by people.
This new algorithm does not replace other existing algorithms such as Rank brain , but works in collaboration with others.
The objective is to improve the understanding of the searches performed by users, relating each of the words in a sentence with the rest , to understand the way people express themselves and offer more accurate and accurate search results .
BERT considers the complete context of each word by observing the previous and subsequent ones, to understand the intention of the search queries
BERT outperforms previous methods because it is the first unsupervised system for Natural Language Processing (NLP), which is important because a large amount of plain text data is publicly available on the web in many languages .
Some of the models that can be built with BERT are so complex that they exceed the limits of traditional hardware , so, for the first time, Google is using the latest TPUs in the cloud to serve search results and get more relevant information quickly.
This breakthrough is the result of Google’s research over the past year on transformers : models that process words in relation to all other words in a sentence , rather than one by one in order.
If you are interested in the subject, you can consult more in-depth information in the Github repository and on the Cornell University website .
Below you can see the talk (in English) ” Deep Learning for Solving Important Problems ” byJeff Deanon the use of Artificial Intelligence in ” The Web Conference ” of May 209 in San Francisco.
The whole video is tremendously interesting, but you can go straight to the 20th minute if you just want to see the part where he talks about BERT .
How does BERT affect SEO?
The implications for SEO are evident since the BERT models will be applied to both the queries and the ranking in Google searches and to the featured fragments , that is, it will alter the order and appearance in which the results appear when a search is made. search .
The claim that this is the biggest advance in the last 5 years seems to refer to the launch of Rankbrain in 2015, the first precedent for the use of artificial intelligence by Google to understand user queries .
In its beginnings RankBrain affected 15% of the searches that were conducted in English , but now it is present in all the searches that are performed in Google , intervening in most of them.
For its part, BERT goes further, allowing to improve the understanding of the queries Google receives depending on the context to return more relevant results , which will allow users to make queries with a more natural language .
The complexity of this new algorithm exceeds the limits of conventional hardware , so perhaps its launch is related to the recent news that Google has reached quantum Supremacy using a programmable superconducting processor .
According to Google’s calculations, for now BERT is only used in 10% of English-language queries in the United States , so the current impact may be less than that of a large-scale algorithm update .
In any case, this first step is not an exception , but clearly marks the path for SEO in the coming years.
Google estimates that the new algorithm will improve the understanding of one in ten queries on classification issues , and this percentage will increase gradually as in the case of Rankbrain .
To deepen the implications that BERT will have, you can read this fantastic article (in English): Algorithm analysis in the age of embeddings by Manshu Ydoxy in November 2018.
SEO for BERT
It is still early to make conjectures, but we must be careful to study how BERT affects the positioning of each website in the coming weeks.
We are not talking only that there will be fluctuations in the positions of the SERPS , but that many results will cease to be displayed for queries in which BERT decides that they are not relevant , and others will appear instead .
Given the emergence of voice searches through virtual assistants , in which Google only returns one response , it seems clear that in the near future the first position of the SERPs will take almost all visits.
As for the possibility of optimizing SEO for BERT , it does not seem possible to do so beyond trying to write for people and not for algorithms .
At this time Google understands the user’s search intention and has the ability to respond with the most relevant information , so that the rest of the ” signals ” used so far will lose importance over time.
When will BERT arrive?
BERT has begun operating this week (October 21-27, 2019) in the United States and will soon be extended to more languages and countries , as it is being tested to improve worldwide searches .
A feature of Artificial Intelligence systems is that they can be based on the learning of a language and apply them to others, so you will use learning models of the improvements in English to apply them to other languages .
Google is currently using a BERT model to improve the featured fragments in the two dozen countries where this feature is available, noting significant improvements in some languages such as Korean , Hindi and Portuguese .
Examples of BERT in Google searches
Thanks to machine learning , BERT improves the way Google understands the most conversational searches or those in which prepositions such as ‘for’ or ‘a’ matter a lot in meaning .
Own Pandu Nayak, vice president of Google Search, gave an example in a press event using the question ” How old was Taylor Swift Kanye When Went on stage? ”(How old was Taylor Swift when Kanye took the stage?)
Without using BERT , Google showed videos of the 2009 event during which the rapper interrupted the pop star’s acceptance speech at the MTV Video Music Awards .
However, using BERT , Google returns as a first result a fragment of a BBC article , which says: “A 19-year-old Swift had just defeated Beyoncé to win Best Female Video for her country-pop teen anthem You Belong With Me . ”, Highlighting” 19-year-old “to emphasize the answer to the question.
BERT is not perfect
In the same event, Nayak also recognized that there are areas where BERT does not help well .
In one example, for the ” tartan ” search , BERT promoted the results of the dictionary because it is a technology that focuses solely on the text .
In searches prior to BERT , the search engine showed images of tartan cloth , which is a more convincing result .
A Google spokesman said the tartan problem was corrected before the launch of BERT .
In another case, when you search for “ What state is south of Nebraska? ” BERT displays the Wikipedia page for southern Nebraska , instead of the ideal result, which would be the Wikipedia page for Kansas .
BERT evaluation
Below are some of the examples that show Google’s evaluation process to demonstrate BERT’s ability to understand search intentions , detecting the subtle nuances of language that machines don’t understand like humans .
2019 USA traveler to usa need a visa
In the search ” 2019 USA traveler to usa need a visa “, BERT helps Google understand that the query ” 2019 The traveler from USA to the UK needs a visa ” is about a American who travels to the United States, and not vice versa .
In the image, the ” Before ” result shows that Google’s algorithms did not understand the importance of this connection and returned results on USA citizens traveling to UK .
In the ” After ” result offered by BERT , the word “a” and its relation to the other words in the query are particularly important to understand the meaning and offer a much more relevant result for this query.
Do estheticians stand a lot at work?
In the query ” do estheticians stand a lot at work? “, Google interpreted the word ” stand ” in the sense of ” stand alone “, which returns irrelevant search results .
Using BERT Google can better interpret how the word is used and understand that the query ” Why do estheticians work hard? ” Is related to work demands as an esthetician , to show a more useful answer.
Can you get medicine for someone pharmacy
In the query ” Can you get medicine for someone pharmacy “, ” Can you get medicine for someone in a pharmacy? ” The BERT model understands that ” for someone ” is an important part of this consultation, while previously the results are general about prescriptions .
Parking on a hill with no curb
A query like ” parking on a hill with no curb “, which shows fragments highlighted in the answer, confused Google’s algorithms because it gave too much importance to the word ” curb ” and could ignore the word ” no “, without understanding the critical That was that word to adequately answer the question .
In the example, the query ” parking on a hill without a curb ” returned the results of parking on a hill with a sidewalk.
Math practice books for adults
Referring to ” math practice books for adults “, ” Mathematical practice books for adults “, the results page above included a book in the category ” Young adult “, while BERT can better understand that ” adult ” is out of context and choose a most useful result .
Google BERT: New Google update
Google announced what they called the most important update in five years. The BERT update affects 10% of search queries. What is BERT and how will it affect SEO? BERT stands for Bidirectional Encoder Representations from Transformers.
Google is making the biggest change in its search system since the company introduced RankBrain almost five years ago. The company said this will affect 1 in 10 queries in terms of changing the results that are classified for those queries.
Do you want 5 links TOTALLY FREE?
Just by signing up for my newsletter, I give you 5 authority links so you can implement in your projects.
Sign up and receive in 5 minutes to start earning visits thanks to link building
BERT is a major Google update
BERT is the technique based on Google’s neural network for training prior to natural language processing (NLP).
According to Google, this update will affect complicated search queries that depend on context.
This is what Google said:
“These improvements are aimed at improving language comprehension, particularly for more natural / conversational language consultations, since BERT is able to help the Search to better understand the nuances and context of the words in the Searches and to do Match those queries better with useful results.
Particularly for longer queries or long tails , more conversational, or searches where prepositions like “for” and “to” matter a lot to the meaning, Google will be able to understand the context of the words in your query. You can search in a way that “seems natural”.
What is the BERT algorithm?
The patent search algorithm expert Bill Slawski described the Bert algorithm like this:
“Bert is a natural language processing pre-training approach that can be used in large text. It handles tasks such as entity recognition, part of voice tagging, and question-answers among other natural language processes. Bert helps Google understand the natural language text of the Web. Google has opened this technology, and others have created variations of BERT. ”
The BERT (Bidirectional Encoder Representations from Transformers) algorithm is a deep learning algorithm related to natural language processing. It helps a machine to understand what the words of a sentence mean, but with all the nuances of the context.
BERT improves understanding of search queries
The Google BERT update improves the way Google understands search queries. BERT analyzes search queries, not web pages. Now SEO becomes more important in terms of using words in precise ways. It is possible that the Google BERT update does not help careless content.
It is already deployed: BERT started operating this week and will be available shortly. It is being developed for English consultations now and will be extended to other languages in the future.
Beware of featured fragments: This will also affect featured shards. Google said BERT is being used globally, in all languages, in featured snippets.
When does Google use BERT?
Google said BERT helps to better understand the nuances and context of words in searches and to match those queries better with more relevant results. It is also used for featured fragments, as described above.
RankBrain is not dead
RankBrain was Google’s first artificial intelligence method to understand queries in 2015. It examines both the queries and the content of the Google index web pages to better understand the meaning of the words.
BERT does not replace RankBrain, it is an additional method to understand content and queries. It is an additive for the Google ranking system. RankBrain can and will continue to be used for some queries.
But when Google thinks that a query can be better understood with the help of BERT, it will use it. In fact, a single query can use multiple methods, including BERT, to understand the query.
How is that?
Google explained that there are many ways in which you can understand what the language of the query means and how it relates to the content of the web. For example, if you write something wrong, Google’s spelling systems can help you find the right word to get what you need.
And / or if you use a word that is synonymous with the actual word that is in the relevant documents, Google can match them.
BERT is another signal that Google uses to understand the language. Depending on what you are looking for, any of these signals or a combination of them could be more used to understand your query and provide a relevant result.
Google Headquarters in Silicon Valley
Can it be optimized for BERT?
It is unlikely, at the moment. Google has told us that SEOs cannot really optimize for RankBrain. But it does mean that Google is improving in the understanding of natural language.
You just have to write content for users, as you always do. This is Google’s efforts to better understand the search engine query and make it better match the most relevant results.
Why we care We care, not only because Google said this change “represents the biggest leap forward in the last five years, and one of the biggest leaps forward in the history of the search.”
We are also interested because 10% of all queries have been affected by this update. That is a great change. Unconfirmed reports of algorithm updates have been seen mid-week and earlier this week, which may be related to this change.
Gutting BERT: New Google NLP Model
Today I come to bring all the information that I know or I can provide within my little knowledge about NLP, that for nothing I am an expert in this area go ahead the truth, you know that I like to be sincere, although I am trying to learn everything I can hehe, and as you know I like to try to understand what happens with Google’s algorithms, let’s see what BERT is, a little more in depth than what I have seen in United States for now.
Bert affects approximately 1 in 10 searches, it is the new open source neural network, it is designed to pre-train deep bi-directional representations of text. It is able to process words in each sentence, and not word for word as they did until now, which implies a better understanding of the complete context of each sentence, since prepositions change what we want to express and / or seek, in theory should improve these results by improving understanding.
One of the biggest challenges of the NLP (natural language processing), was the lack of data to train them since they were a few thousand or hundreds of thousands of examples labeled by humans, then came the NLP models based on deep learning since They train with much larger amounts, reaching millions or billions of web data without annotations (pre-training), this pre-training can then be adjusted with small data sets such as answering questions, feeling analysis, which gives as result great improvements in accuracy compared to training these sets from scratch. Well, this is where BERT comes in, it is a new technique for prior NLP training.
● https://github.com/google-research/bert
What is BERT?
It is a neural network based on the transformer architecture. The model trains with a corpus of 3,300 million words (800m of words from BooksCorpus and Wikipedia (2,500M of words). To give you an idea, Bert is able to discover relationships and sequences within the phrase for an entity, It is where the transformer concept makes a difference.
I quote verbatim:
“The transformer is the first transduction model ( conversion of input sequences into output sequences) that relies entirely on self-attention to calculate the representations of its input and output without using RNNs or convolution aligned with the sequence”.
“Self-attention, sometimes called intra-attention, is a mechanism of attention that relates different positions of a single sequence to calculate a representation of the sequence.”
Prateek Joshi
Does Bert replace or replace RankBrain?
No, Bert is an additional method to debug and understand the content, queries and therefore improve the classification system in the index. RankBrain will remain active, simply Google will use one more method, since it really uses several, and not only used RankBrain, such as spelling corrections, as far as I know (think it’s like a hybridized algorithm that constantly changes and evolves), uses among others a method that is based on weighted editing distance (Damerau – Levenshtein distance), using a minimum number of possible modifications to find a correctly written alternative, also corrects phonetic errors, usually replaces a part of the wrong word with a phonetically equivalent character sequence, until you locate the correct spelling,
Do you think maybe the corrections it makes when looking for? Or maybe you meant?
More in-depth with BERT: pre-training and adjustment
Pre-training occurs with two tasks that must learn on the network at the same time the meaning of the tokens and the structure of their tickets. Accept a vector of up to 512 tokens that can be divided into two segments (a and b).
The jump is in the relation and direction of the words, instead of from left to right or from right to left, they use two unsupervised tasks. In the first task they mask a percentage of tokens (15% normally) at random from the sentence / s and then predict (decode).
Clarification about 15%:
Masked words were not always replaced by masked tokens because the token would never appear during fine tuning.
Then, the researchers used the following technique:
● 80% of the time the words were replaced by the masked token
● 10% of the time the words were replaced by random words
● 10% of the time the words were not modified
General Pre-Training and Bert Adjustment Procedures
Unlike automatic decoders instead of reconstructing the entire entry, they only do so on masked words. That is, sentences with masked tokens and you are asked to produce the vocabulary ID of the missing word. It is “similar” to the word breaks of Word2Vec, except that the latter is never represented in a sentence, only the central word in the window where the target word is chosen.
The second task is based on the prediction of the following sentence (NSP), which is based on an understanding of the relationship between two sentences. This is very important for the part of QA (Question Answering) and NLI (Natural Language Inference) getting the final model between 97-98% accuracy in predicting the next sentence. Here the BERT model is presented with two sentences, coded with segment A and B inlays as part of an entry. Other times, the model is presented with two consecutive sentences from the corpus, sometimes a second sentence is not a sequencer, and is chosen at random from the data set. This pre-training task is intended to improve the performance of the dual segment component of the model.
In all pre-training tasks the entries are represented by a vector that contains a token insert, segment and position.
● Token inlays help transform words into vector representations.
● Segment inlays help you understand the semantic similarity of different parts of the text.
● Position inlays mean that identical words in different positions will not have the same output representation.
In addition to the output layers, they use the same architectures in pre-training as in adjustment.
Differences between BERT vs OPENAI GPT vs ELMo
BERT as we have spoken uses a deep bidirectional transformer, OpenAI GPT uses a transformer from left to right (unidirectional), ELMo (superficially bidirectional) uses the concatenation of LSTM from left to right and from right to left independently trained to then generate functions for tasks later.
BERT apart is the only one that works by relating contexts in both directions in each layer. BERT and OpenAI GPT are fine tuned and ELMo is based on approach characteristics.
The differences apart from the pre-training and bidirectional tasks are also in the training:
● GPT is trained with BooksCorpus (800m of words) and Bert with BooksCorpus and Wikipedia (2,500M of words).
● GPT was trained for 1 million steps in lots of 32,000 words and BERT for 1 million steps in lots of 128,000 words.
● GPT uses the same learning ratio in all fine tuning experiments and BERT chooses the learning rate that is best for the development set (It should be noted that for each of these settings, the entire model must be adjusted, not just the layers superior)
● GPT uses a sentence separator () and classifier token () that are only entered at the time of adjustment. While BERT learns () and () and the A / B prayer inlays during pre-training.
Can I optimize for BERT?
No, if you have understood what is going on and what it does, the question does not really make sense as it did not have it for Rankbrain.
BERT examples in action
The difference in understanding between the meaning of the search and the result.
My conclusion about BERT for now
Upgrade:
It is clear that Google and the SEOS are quite alarmist with each update, it seems that BERT is the GREAT update of the century, I do not think so, it is a great step in terms of improving understanding, but for now it is only deployed in English , it will be replicated in other languages, it will mainly affect longs tails, and the QA, if your website responds mainly to longs tails simply, check that the intention has not changed / improved, we are going to do the only thing I would do, it would constantly review CTRS ( which you should really be doing already), and see if there is a notable drop, analyze if it is because you now understand the search better and your answer was not right / well focused. We will see in what else this algorithm affects since it surely not only affects longs tails and some main kw can be affected.
