You see it and use it multiple times throughout the day, but do you know how it works?
Surprisingly, many webmasters severely lack knowledge about one of the most important aspects of the internet – the search engines! We all know they exist, we know people use them, and we know that they drive traffic to our websites. But few people actually carry an understanding about how they work.
If you can educate yourself about the way search engines function, you’ll be more capable of optimizing your site properly.
Search engines send automated “bots” to “crawl” the web through hyperlinks.
As we’ll discuss later, without external hyperlinks to your website, the bots really have no way of finding your website. It’s still possible that they’ll discover your site eventually, but without some kind of effort on your part, it may be unlikely.
Only about half of the existing pages on the internet have been crawled by the search engines, if that gives you any indication of the importance of backlinks.
Once the bots crawl your website, it becomes indexed in a huge database along with all the other indexed pages on the internet. There are literally billions of pages stored in this database. Yet, as you’ve probably noticed, it takes barely a second or two to get results after performing a search.
When searches are made, the engine quickly scans through relevant documents and provides results based on the most accurate possible matches. Generally a match is determined by the presence of that particular keyword on the webpage. Thus, on-page optimization is extremely important.
Results Will Always Differ
Google and the other search engines will provide differing results depending on whether you type the phrase as-is (purple umbrellas), in quotes (“purple umbrellas”), with the + symbol (purple + umbrellas), or other variations.
After the SE has found matches for the search query, a special algorithm scans each of the results to determine relevance to the keyword phrase. Results are provided to the user in order from most relevant to least relevant. So what can you learn from this information? A few things:
Your page must be relevant to the search term (listing the keyword several times throughout the course of the website, on-site optimization)
You need external hyperlinks (backlinks) pointing to your website to act as a “gateway” for the bots to access your site.
As far as relevance goes, Google and the other search engines take all of these into consideration:
On-site optimization
Age of the domain (the older the better)
Page Rank (PR)
Alexa ranking (the ‘popularity’ of your website based on the amount of traffic it gets)
Number of backlinks, particularly from high PR authority websites that are related to your website (if you have a blog about real estate, a high PR link from a real estate website will be more valuable than a high PR link from a website about dog grooming)
Linking structure of the website (easy navigation)
So looking at all of these factors, you can see why Amazon.com would take the #1 spot for “buy books” rather than your three-week-old Blogger blog with 4 backlinks.
Stumbling Blocks for the Search Engines
Sometimes the search engines are prevented from crawling your website for one reason or another. You’ll want to make sure you eliminate these.
Your site requires a cookie for navigation. Bots can’t carry cookies the way a regular browser user can.
Framed websites. Back in the day (10+ years ago) when I first started designing & programming websites, I loved using frames. But I had no idea that it would hinder my sites’ ability to be ranked in the search engines.
Long, complicated URLs such as http://www.website.com/page.php? ID=HuUj=987sj=%site%
Login pages
Redirect pages (Google hates these)
Poor linking structure on your website. Each page on your domain should be linked to from the home page (or a sitemap) or the bots may have a difficult time crawling it (because they won’t be able to find it).
Link Structure
Ideally you would like to have a very convenient linking structure, in which each page on your domain is accessible from every other page. This is why WordPress blogs are favored by the search engines, because their layout allows for this kind of linking structure.
You accidentally have the “I would like to block search engines” option selected in the “Privacy” tab in your WordPress dashboard. Sometimes this option is selected automatically, so always double-check that it is turned off. Similar options exist for other sites like Blogger.
You have a robots.txt file preventing the search engines from crawling certain pages. Usually this is done purposefully to prevent unwanted content from being indexed, but sometimes it might be a mistake.
Your page is heavily coded with Javascript or contains Flash as its primary method of delivering content.
Keyword Research
If you want your website to rank in the search engines, you’ll need to optimize it for particular keyword phrases. Keyword research is the process of finding out which keywords will be the easiest and most beneficial to optimize for.
Generally, you’ll be looking at two things:
Search volume: How many people search for the keyword in a given month.
Competition: How many competing websites are also optimized for this keyword phrase.
Why is keyword research important? Well, you can’t just pick some keywords and keyword phrases off the top of your head and expect to easily rank for them. If your website is just getting off the ground, you will find it exceptionally difficult to rank for a keyword like “lose weight” or “quit smoking” or even “cat litter”.
However, if you conduct proper keyword research, you may be able to find similar keyword phrases that have a decent search volume and little competition. Those are the keywords that you want to target.
I could probably go on and on for one hundred pages about keyword research, but I know you have a busy schedule, so I’m just going to get straight to the point.
To easily conduct keyword research (for free), all you need are two resources at your disposal: The Google Adwords Keyword Tool, and the Google search engine itself.
Open up the Keyword Tool and input your desired keywords. How do you know which keywords to list? Well, let’s say you have a website about jewelry. The keyword phrases you choose could be:
buy homemade jewelry buy homemade necklaces buy homemade earrings online jewelry store
Basically, anything related to the content of your website. Generally, the more targeted your keywords are, the better. So if you specialize in turquoise jewelry only, don’t bother optimizing for “jewelry” (an exceptionally hard keyword to rank for) – just “turquoise jewelry”.
Additionally, don’t try to rank for “pearl jewelry” if that’s not what your website covers. You want to attract visitors who are specifically looking for “turquoise jewelry” – they will come to your website and see that you are offering them exactly what they are looking for. The overall quality of your traffic will be better and your conversions will be higher.
Click “Get keyword ideas” and you’ll be given a long list of related keyword phrases and their search numbers.
Browse through the keyword list and copy down all the keyword phrases that interest you. Generally it’s safe to assume that the higher the search volume, the higher the competition, and vice versa. However, this isn’t always the case. It’s still possible to stumble upon some real gems – keywords with high search volume and little competition.
Let’s take a look at this keyword: discount jewelry store. As you can see, it gets 1600 searches per month.
Now go to Google and type it in with quotes, like this: “discount jewelry store”
The quotes tell Google to only show websites that are optimized for the entire keyword phrase, in order. Without quotes, Google searches for all websites that contain the words discount, jewelry, and store, but not necessarily together, and not necessarily in order. So keeping the phrase in quotes is important because it’ll give you a more accurate idea of the competition.
As you can see, Google tells us that there are 19,600 competing websites for this keyword phrase. Generally anything under 50,000 is considered pretty good. The lower the better.
Another search you can do is allintitle: “discount jewelry store” and allinanchor: “discount jewelry store”.
The first tells you how many websites are optimized with the keyword phrase in the title and the second tells you how many contain the keyword as anchor text in a link. This, in a sense, is your “true” competition, because these people purposely optimizing for your keyword.
Finally, it’s important to look at the specific pages that turn up in the top 10 – 20 results. Check their Page Rank. You can download a free plugin for Firefox called “Seo For Firefox”. There will be a tiny icon at the bottom of your browser that will tell you the PR of any website you’re browsing.
It’s also important to see what KIND of pages show up in the results. Are they high authority domain names like Amazon, Ebay, etc? Secondary pages (http:// www.website.com/discount-jewelry-store.html)?
Ranking Factors
The best way to tell if a keyword will be easy to rank for is to look for Web 2.0 sites, social bookmarks, and RSS feeds in the top 10. Ezinearticles, GoArticles, Weebly, Squidoo, Hubpages, Livejournal, WordPress, Blogger, and Quizilla are all examples of Web 2.0 sites.
Furthermore, keep an eye out for results like this:
This is a page from a social bookmarking website. Other sites like this include Mixx, Digg, Propeller, Reddit, and delicious. Also look for Yahoo! Answers and Youtube videos.
If your search (without quotes) looks something like the screenshot below, you’ve probably found a winner. It doesn’t necessarily mean that you won’t have to work hard or face challenges, but it’s certainly a lot easier to beat these types of websites than it is to beat high authority aged domain names with lots of backlinks.
So there you have it: a very very BASIC guide to keyword research and competition analysis. Once you’ve found a suitable keyword or keywords, you’ll be ready to perform some on-site optimization.
Stay Tuned for Part 2 of this ongoing Blog Series.
You see it and use it multiple times throughout the day, but do you know how it works?
Surprisingly, many webmasters severely lack knowledge about one of the most important aspects of the internet – the search engines! We all know they exist, we know people use them, and we know that they drive traffic to our websites. But few people actually carry an understanding about how they work.
If you can educate yourself about the way search engines function, you’ll be more capable of optimizing your site properly.
As we’ll discuss later, without external hyperlinks to your website, the bots really have no way of finding your website. It’s still possible that they’ll discover your site eventually, but without some kind of effort on your part, it may be unlikely.
Only about half of the existing pages on the internet have been crawled by the search engines, if that gives you any indication of the importance of backlinks.
Once the bots crawl your website, it becomes indexed in a huge database along with all the other indexed pages on the internet. There are literally billions of pages stored in this database. Yet, as you’ve probably noticed, it takes barely a second or two to get results after performing a search.
When searches are made, the engine quickly scans through relevant documents and provides results based on the most accurate possible matches. Generally a match is determined by the presence of that particular keyword on the webpage. Thus, on-page optimization is extremely important.
Google and the other search engines will provide differing results depending on whether you type the phrase as-is (purple umbrellas), in quotes (“purple umbrellas”), with the + symbol (purple + umbrellas), or other variations.
After the SE has found matches for the search query, a special algorithm scans each of the results to determine relevance to the keyword phrase. Results are provided to the user in order from most relevant to least relevant. So what can you learn from this information? A few things:
Sometimes the search engines are prevented from crawling your website for one reason or another. You’ll want to make sure you eliminate these.
Ideally you would like to have a very convenient linking structure, in which each page on your domain is accessible from every other page. This is why WordPress blogs are favored by the search engines, because their layout allows for this kind of linking structure.
If you want your website to rank in the search engines, you’ll need to optimize it for particular keyword phrases. Keyword research is the process of finding out which keywords will be the easiest and most beneficial to optimize for.
Generally, you’ll be looking at two things:
The best way to tell if a keyword will be easy to rank for is to look for Web 2.0 sites, social bookmarks, and RSS feeds in the top 10. Ezinearticles, GoArticles, Weebly, Squidoo, Hubpages, Livejournal, WordPress, Blogger, and Quizilla are all examples of Web 2.0 sites.
Furthermore, keep an eye out for results like this:
This is a page from a social bookmarking website. Other sites like this include Mixx, Digg, Propeller, Reddit, and delicious. Also look for Yahoo! Answers and Youtube videos.
If your search (without quotes) looks something like the screenshot below, you’ve probably found a winner. It doesn’t necessarily mean that you won’t have to work hard or face challenges, but it’s certainly a lot easier to beat these types of websites than it is to beat high authority aged domain names with lots of backlinks.
So there you have it: a very very BASIC guide to keyword research and competition analysis. Once you’ve found a suitable keyword or keywords, you’ll be ready to perform some on-site optimization.