How search engines work

lots of book on shelves in a library
Last updated:
Originally published:

Every business owner and marketer needs to know about SEO. From content to PR and technical optimisation, it all adds up to ranking higher in your search engine of choice.

But, how do the search engines actually work?

What is a search engine?

A search engine is a website we go to when looking for answers on the internet.

The most popular search engine at the time of writing is Google. Others in the Australian market include Bing, Yahoo and DuckDuckGo.

A search engine is like the filing system in a huge library.  That library (the internet) has billions of books in it.  Those books all have multiple chapters and pages, leading to the possibility of trillions of results.

If you want to search for a term, then Google (or your chosen search engine) will find websites that contain those keywords, or similar / closely related ones.

The search engine will attempt to return the most relevant results first, even if that result is from a third party.

As an example, at the time of writing, a Google search for “how much are tickets to the Eiffel Tower” gives the following result:

A Google search result for “how much are tickets to the Eiffel Tower”

A Google search result for “how much are tickets to the Eiffel Tower”

Note in this example that the search query “how much are tickets to the Eiffel Tower” returns a result not from a seller of tickets to that monument, but to an article from The Huffington Post titled 18 Things You Need to Know Before Visiting the Eiffel Tower.

Why has Google chosen to show this article, rather than linking to a local ticket seller, or even an aggregator such as Trip Advisor or Kayak?

For whatever reason (and we have no doubt there are many), Google has decided that anyone asking this particular question should read this particular article.

When clicking through to that result, you’ll note that the article clearly states that “Eiffel Tower ticket costs vary wildly” while it then explains the intricacies of the ticket prices, taking into account the age of the participants as well as how high up the tower you wish to go. It also offers a link to actually purchase tickets.

Note that this experiment is being done from Melbourne, Australia in January 2021. The top search result in Google has not changed for me in the two years I’ve been running the test. I will say that I am somewhat surprised that even a worldwide pandemic hasn’t budged the HuffPost off the top spot. When running this experiment in some online classes through our lockdown last year, every participant had the same result.

Fascinating.

The top search result may well be different for you, depending on many factors, some of which we’ll look into now…

Search engine algorithms are all different

Optimising for one search engine may not improve your rankings on all.

As a quick example, the first organic result on Bing for the same Eiffel Tower query links through to an info page from howmuchdoesitcost.com:

Bing search result for How much are tickets to the Eiffel Tower

Bing search result for How much are tickets to the Eiffel Tower

While the DuckDuckGo result actually sends you to the official Eiffel Tower website.

DuckDuckGo search result for How much are tickets to the Eiffel Tower

DuckDuckGo search result for How much are tickets to the Eiffel Tower

Sometimes a Google result can be so great that you don’t even need to click through to anywhere.

Google search results page for the search query of "where is the capital of australia"

Google search results page for the search query of “where is the capital of Australia”

Search engine bots

Wall-E robot

Search engines have ‘spiders’ or ‘bots’ that go out and crawl the internet.  When a bot finds a new link, it will ‘follow’ it and then ‘index’ the content.  If that indexed page has links on it, then the bot will follow them too.  The bots quite literally create a web of links that they follow, and the re-follow when they revisit the page.

There is obviously a fair bit of intricacy to this, which we cover in our SEO for Australian Small Business eBook.

Search Engine Algorithms

system of equations

Search engines all have different algorithms that calculate how different websites will rank.

An algorithm is defined as “a process or set of rules to be followed in calculations or other problem-solving operations, especially by a computer”.

These algorithms use different ranking factors that determine how your site will rank for a certain search term.  We don’t know all the ranking factors, but we do know some.  We don’t know how much weight all the ranking factors have, but again, we do know some.

Google isn’t exactly going to advertise the secret sauce that makes them the most powerful search engine in the world, are they?

Ranking Factors

golden trophy

All the major search engines have multiple ranking factors, and it’s impossible to cover them all. One of the reasons for that is no one knows what they all are!

We do know a lot of them though, and we cover most of the ones that count for a lot in our book.

Some ranking factors include:

As well as a lot more.

Many of these overarching ranking factors will likely never change, although their weighting might. Optimising for these ranking factors is essential to rank higher in Google and other search engines.

It’s important to note that ranking factors and their weightings can differ dramatically between industries. It can be impossible to measure how successful something is going to be until you’ve tried it.

We have worked with clients where simply optimising their images and making their website faster has had a huge impact on their rankings while updating to more readable copy has done little.

We’ve also worked with businesses where the exact opposite can be said.

When working out what sort of ranking factors to try and rank well for, it’s always a good idea to have a think about what’s going to be good for your customer.

Most great SEO comes down to great User Experience (UX).

Rankbrain

illustrated brain with wires

Rankbrain is the ‘smarts’ behind Google’s powerful search algorithm.  While you don’t need to remember its name, you do need to know what it does and why it does it.

Rankbrain uses machine learning to help match search terms together.

Ten years ago, doing a Google search for “Italian restaurant in Carlton” would only list web pages that had the specific search term of “Italian restaurant in Carlton”.  Now with the power of machine learning and artificial intelligence, your results are more like to show results like:

  • Italian inspired cuisine in Carlton
  • Melbourne’s top 10 Italian restaurants
  • Homemade style Italian food on Lygon St.

Google knows that the above results are all talking about Italian restaurants in Carlton, even though the specific search term isn’t mentioned.

Why is this good for you?  So, you can write like the human that you are and appeal to your fellow human customers!

Experience, Authority & Trust

many people holding hands

Over the years of algorithm updates, it’s become clear that the most important factors that you must exhibit are experience, authority and trust.  In SEO circles, these are often referred to as the acronym E-A-T.

Experience, authority and trust are obvious in their definition.  We cover off many techniques that will help you and your website to prove all three in our SEO for Australian Small Business eBook.

Really, when it comes down to it – put your customer first and your SEO ranking will likely follow.

SEO is personal

many friends arm in arm

SEO is as much about your user as it is about you.

If everyone reading this were to Google the same search term, we’re all going to get different results.

We’re all geographically apart, different ages, demographics as well as a host of other factors.

So, for example, if everyone reading this article Googled “Hairdresser near me”, we’re all going to get quite different results.

Google will want to give you the result it thinks is best for you.  Maybe it will be a local map result or a string of videos or something else entirely.  Whether you’re on mobile or desktop also influences what search results you’ll see.

Your past search behaviour also influences the kinds of search results you’ll be served.

There’s no such thing as number 1

So, what’s the takeaway from this?

Although every search engine is different, there are distinct similarities too.  The algorithms and technology that go into search engines have grown in sophistication dramatically over the years.

There is no guaranteed number one result on Google because it all depends on the who is making the search query, where they are, what they have previously searched for, as well as their intent.

While ranking as number 1 for your business name is critical (and very easy), the same can’t be said for other phrases and keywords in your niche.

If someone blindly claims that they can rank you number one in Google for any search term, they’re probably not the SEO agency to be talking to.

Pay what you like for our "SEO for Australian Small Business" eBook

Thanks for reading How search engines work.

In recognition of the small businesses who have been doing it tough in recent times, you can now pay what you like for our SEO for Australian Small Business eBook.

Minimum price is $5

Check out SEO for Australian Small Business
Avatar for Sam HemphillNow a digital veteran, Sam Hemphill initially trained as a drummer and high school teacher and spent 10 years as a touring musician and tour manager. During that time, he...

Read more about Sam Hemphill