Is Your Website Being Indexed Properly by Google?

One of the most common problems experienced when trying to rank in Google, is that your website is not currently being indexed correctly. If this is the case, it means Google is failing to access your web pages to index your site’s content effectively.

To check whether your site is efficiently crawled and listed, you will need to log into your Google Webmaster Tools and check the “Google index” tab. There you will find the total number of pages the search engine has indexed. If you see a drop in the number of these pages, you are likely to experience a decrease in traffic levels.

Finding the Reason Behind Your Indexing Issues

If you’ve taken a look at your Webmaster Tools and it’s clear that not all your pages are being found by Google’s crawlers, now is the time to take a closer examination at the possible problems Google is experiencing with your website.

Does Your Site Have Crawler Errors?

To find out if Google is indexing your site fully, begin by heading to your Google Webmaster Tools dashboard and checking your Crawler Error messages. The most-likely error message you will find is a 404 HTTP Status Code warning. It signals that the URL cannot be found.

Other crawling errors include:

• Robots.txt – A   poorly scripted Robots.txt file can be detrimental to your Google indexing. This text file is like a set of instructions telling a search engine crawler not to index parts of your website. If it includes a line such as “User-agent: *Disallow: /” this basically tells every single crawler it experiences to ‘get lost’ – including Google.
• .htaccess – This invisible file can do nasty things if incorrectly configured on your site. Most FTP clients allow you to toggle hidden/seen files so that you can access it if required.
• Meta Tags – If you have pages that aren’t being indexed, be sure they don’t have the following meta tags in the source code: <META NAME=”ROBOTS” CONTENT=”NOINDEX, NOFOLLOW”>
• Sitemaps – If you receive a Sitemaps crawling error, it means your website sitemap is not updating properly; your old sitemap is being repeatedly sent to Google instead. When you’ve tackled any issues signalled by the Webmaster Tools, make sure you run a fresh sitemap and re-submit it.
• URL Parameters – Google allows the option to set URL parameters when it comes to dynamic links. However, incorrect configuration of these can result in pages that you do want picked up being dropped instead.
• DNS or Connectivity issues – If Google’s spiders simply can’t reach your server, then you may encounter a crawler error. This could be for a variety of reasons such as your host is down for maintenance or had a glitch of their own.
• Inherited Issues – If you have bought an old domain or moved your website to an old website’s location it is possible the previous site had a Google penalty. This will inhibit indexing of the new site. You will have to file a reconsideration request with Google.
• If you are considering using a historic domain for your site, be sure to take a look at its history before purchasing. You can make use of the Internet Archive’s Wayback Machine to see pages that were previously hosted on your domain.

Does Your Site Have Syntax Errors or Structural Complications?

Google is very tolerant when it comes to HTML mark-up mistakes within webpages, but it is possible that syntax errors can prevent indexing (in extreme cases). Check your site’s HTML with the W3C’s HTML Validator to see a report of errors you need to correct.

Google advises you make your site structure as logical as possible. Every page should be reachable from at least one test link. You can use a text browser, like Lynx, to look at your site much the same way the spiders see it. Remember, the parts of your site that use frames, JavaScript, Flash, session IDs, cookies and DHTML may be missed by crawlers.

Does Your Site Have Inbound Links?

To be indexed with Google, your website needs to have at least one quality inbound link from another website already indexed in the search engine. This is a common reason it takes a lot of new websites a while to be successfully indexed.

One way to create some quick links is to update social networks with your website URL or add a link on an existing related website that you own. Social media profiles that carry high weight include: Facebook pages, Twitter profiles, Google+ profiles/pages, LinkedIn profiles, YouTube channels, and Pinterest profiles.

Offsite content is another excellent way to build links that will help your site get indexed properly. Offsite content is content relevant to your site that is hosted elsewhere: guest posts on other blogs in your niche. Just keep in mind, you need to make sure these external sites are all high quality, as links from ‘spammy’ sites will do your website harm instead of good. The best way to ensure your links are high quality is to make sure that they have ‘natural links’, links that develop as part of the dynamic nature of the internet where other sites link to content they find valuable.

See Google’s Webmaster Guidelines for a more in-depth understanding of what they consider to these to be.

Has Google Penalized You?

One of the most difficult obstacles in proper indexation by Google is a Google penalty. There are a number of reasons why you might encounter a penalty from Google, but if you do not deal with the issue they raise, you may be deindexed (removed from their search engines).

Avoid Google Penalties by Steering Clear of The Following Techniques:

• Automatically generating content
• Link schemes
• Plagiarizing or duplicating content
• Cloaking
• Sneaky redirects
• Hidden links & text
• Doorway pages
• Content scraping
• Affiliate programs with little content value
• Using irrelevant keywords
• Pages that install trojans, viruses, & other adware
• Abusing rich snippets
• Automating queries to Google

Recovering from Google penalties requires hard work and due diligence to remove links on your part; you will need to submit a reconsideration request before your site is effectively indexed and ranked once more.

Fix Your Indexing

Most of these checks are quick and easy to make, so don’t let your SEO and link building efforts go to waste – make sure your website is indexed correctly by Google. It’s surprising how many websites make some of the smallest mistakes, and it prevents their site from being indexed correctly. In the end, it hurts their website’s rankings, which hurts their traffic, which hurts their sales.

Panda 4.1 Google’s 27th Panda Update Is Rolling Out

Google has announced that the latest version of its Panda Update a filter designed to penalize “thin” or poor content from ranking well has been released.

Google said in a post on Google+ that a “slow rollout” began earlier this week and will continue into next week, before being complete. Google said that depending on location, about 3%-to-5% of search queries will be affected.

Anything different about this latest release? Google says it’s supposed to be more precise and will allow more high-quality small and medium-sized sites to rank better. From the post:

Based on user (and webmaster!) feedback, we’ve been able to discover a few more signals to help Panda identify low-quality content more precisely. This results in a greater diversity of high-quality small- and medium-sized sites ranking higher, which is nice.
New Chance For Some; New Penalty For Others
The rollout means anyone who was penalized by Panda in the last update has a chance to emerge, if they made the right changes. So if you were hit by Panda, made alterations to your site, you’ll know by the end of next week if those were good enough, if you see an increase in traffic.

The rollout also means that new sites not previously hit by Panda might get impacted. If you’ve seen a sudden traffic drop from Google this week, or note one in the coming days, then this latest Panda Update is likely to blame.

About That Number
Why are we calling it Panda 4.1? Well, Google itself called the last one Panda 4.0 and deemed it a major update. This isn’t as big of a change, so we’re going with Panda 4.1.

We actually prefer to number these updates in the order that they’ve happened, because trying to determine if something is a “major” or “minor” Panda Update is imprecise and lead to numbering absurdities like having a Panda 3.92 Update.

But since Google called the last one Panda 4.0, we went with that name — and we’ll continue on with the old-fashioned numbering system unless it gets absurd again.

For the record, here’s the list of confirmed Panda Updates, with some of the major changes called out with their AKA (also known as) names:

Panda Update 1, AKA
Panda 1.0, Feb. 24, 2011 (11.8% of queries; announced; English in US only)
Panda Update 2, AKA
Panda 2.0, April 11, 2011 (2% of queries; announced; rolled out in English internationally)
Panda Update 3, May 10, 2011 (no change given; confirmed, not announced)
Panda Update 4, June 16, 2011 (no change given; confirmed, not announced)
Panda Update 5, July 23, 2011 (no change given; confirmed, not announced)
Panda Update 6, Aug. 12, 2011 (6-9% of queries in many non-English languages; announced)
Panda Update 7, Sept. 28, 2011 (no change given; confirmed, not announced)
Panda Update 8 AKA
Panda 3.0, Oct. 19, 2011 (about 2% of queries; belatedly confirmed)
Panda Update 9, Nov. 18, 2011: (less than 1% of queries; announced)
Panda Update 10, Jan. 18, 2012 (no change given; confirmed, not announced)
Panda Update 11, Feb. 27, 2012 (no change given; announced)
Panda Update 12, March 23, 2012 (about 1.6% of queries impacted; announced)
Panda Update 13, April 19, 2012 (no change given; belatedly revealed)
Panda Update 14, April 27, 2012: (no change given; confirmed; first update within days of another)
Panda Update 15, June 9, 2012: (1% of queries; belatedly announced)
Panda Update 16, June 25, 2012: (about 1% of queries; announced)
Panda Update 17, July 24, 2012:(about 1% of queries; announced)
Panda Update 18, Aug. 20, 2012: (about 1% of queries; belatedly announced)
Panda Update 19, Sept. 18, 2012: (less than 0.7% of queries; announced)
Panda Update 20 , Sept. 27, 2012 (2.4% English queries, impacted, belatedly announced
Panda Update 21, Nov. 5, 2012 (1.1% of English-language queries in US; 0.4% worldwide; confirmed, not announced)
Panda Update 22, Nov. 21, 2012 (0.8% of English queries were affected; confirmed, not announced)
Panda Update 23, Dec. 21, 2012 (1.3% of English queries were affected; confirmed, announced)
Panda Update 24, Jan. 22, 2013 (1.2% of English queries were affected; confirmed, announced)
Panda Update 25, March 15, 2013 (confirmed as coming; not confirmed as having happened)
Panda Update 26 AKA
Panda 4.0, May 20, 2014 (7.5% of English queries were affected; confirmed, announced)
Panda Update 27 AKA
Panda 4.1, Sept. 25, 2014 (3-5% of queries were affected; confirmed, announced)
The latest update comes four months after the last, which suggests that this might be a new quarterly cycle that we’re on. Panda had been updated on a roughly monthly basis during 2012. In 2013, most of the year saw no update at all.

Of course, there could have been unannounced releases of Panda that have happened. The list above is only for those that have been confirmed by Google.

What 7 Things Everybody Needs to Know About Google Search Optimization

As one of the most visited websites around the globe, with over 12 billion searches sent through its engine per month and owning 67.5 percent of the U.S. search market, Google can play a huge role in the popularity of your own site. Whether it's business-related or you're just trying to establish yourself as a presence and brand online.

SEO may seem like magic to an outsider. It is a structured approach and measured practice of optimizing your pages (and website) to improve your find-ability factor in search engines. If done right you can get listed on top of the first page of results for your specific keywords. While nothing in life is really free, your listings that are shown in the organic, natural results will not ding your wallet each time a searcher clicks. (That's the model for paid advertising, like Google Adwords).

Here are 7 things that everybody should know about Google search optimization.

1. Google Is Always Evolving

Google is constantly changing and evolving at a high speed, and it has had a steady level of growth over the past decade with new software, patents, products, web properties, and corporate acquisitions accumulating over time.

"Among Google's goals: Improve the user experience by delivering relevant, fresh, quality content and, at the same time, crack down on those using questionable search engine optimization techniques to gain unjustified ranking position." - CIO.com

The index also changes and that alters operations. Due to these changes it's impossible to ever completely comprehend the Google algorithms or make any sort of permanent change to your site that will always comply with the standards that Google is keeping. Add to this the fact that Google's main business is paid advertising and naturally wants businesses and users to engage there.

2. Everybody Wants A Piece Of The Google Action

This might seem obvious. You know at the back of your mind that all of your competitors are also vying for that top spot on the search list.

"If your business has an active, well optimized and maintained website with high quality unique content, then you may find yourself a step ahead already." - SEJ

This isn't enough, however. If your company has no real online presence; has not built trust via quality links or has a random selection of social media profiles, blogs, and widgets that provide little value or tries to manipulate Google to try to acquire high positions unnaturally, you could be losing out. Your competitor may have already overtaken you, and web traffic is decreasing.

3. Quality Matters Most To Google

Believe it or not, one of the most important concepts that you must understand when it comes to Google is that their standards for the quality of content are very high.

Despite the millions of website operators who strive for better rankings within the search engine, the most important parties to Google are their (paid) users, which means that they want to provide content that satisfies and makes readers come back again and again.

4. Structure Your Data When You Can

We know that although not everything needs to be structured and organized into some specific schema, Google prefers structured data as much as possible as it's easier to analyze and requires less computing resources. Google responds by greatly increasing your chances for visibility and traffic. Especially when they know the best data to present to readers, whether it's a product listing, video, news or event announcement.

5. Don't Buy Links, Work For Them

It may be possible to build up the number of links back to your site by purchasing them in some way, but this is viewed as a negative behavior by Google.

The truth is that if you buy your links, then they aren't likely a good representative of how honest your content is. Google is cracking down on this behavior and give out penalties where they see it occurring. Even if you were not aware of it. You can earn links the hard way through great quality information and products that people want to see. It may then be shared naturally.

6. Get On Board With Google+

It hasn't appeared to be one of the most popular social media sites on the internet, but using Google+ does tend to have a correlation with search engine rankings. Your personal network will see it. Even if the use of this tool only provides a miniscule difference in the grand scheme of things, every little bit helps in your online presence building.

7. Get Moving

The content of your site is the most important aspect that Google wants you to focus on for SEO benefit. Rankings may and will come over time, but there are hundreds of factors to the Google algorithm. Content and links are critical, but so is the speed at which your website and pages operate.

Success or failure of your website is ultimately measured in visibility, traffic, stickiness (how long visitor stay and if/when they come back) your message and positioning, as well as conversions (sales). With so many moving parts, make sure your content is readily available and that your pages load fast. Improve your site speed and make users and Google happy in the process.

WHAT'S NEXT?

Before you add a second story to your house or construct a brand new home, be clear on where you are and what you want. For Google SEO, that means getting an SEO Audit.

Google’s Push For HTTPS Is More About PR Than Search Quality

Is it worth it for webmasters to switch to HTTPS in light of Google's recent announcement?

Earlier this month, Google announced that its search ranking algorithm will now consider whether a site is HTTPS. Does this mean you should now go out and make the switch to HTTPS, or is this just political jousting with no real search relevance on Google’s part?

What Is HTTPS, Anyway?
HTTPS stands for Hyper Text Transfer Protocol Secure. It’s a variation of the popular HTTP used to transfer web pages across the internet. The difference (the “S”) is that HTTPS adds a layer of security by encrypting the data.

A normal website is accessed by putting http:// before the domain name, such as http://facebook.com. If the site supports HTTPS, the URL will look like https://facebook.com. Typically, browsers will add a padlock icon and will highlight the address bar in green when a site uses HTTPS.

The Push For Security
Over the past few years, Google has pushed for improved security on its site as well as sites in general across the internet, and for good reason. Between the NSA’s spying and routine security breaches that pilfer millions of passwords from popular sites, it’s not a bad idea for a company like Google to take security seriously.

We saw the beginnings of this a few years ago when Google began encrypting search referral terms for logged-in users, which led to a lot of frustration for marketers who no longer had access to keyword data in their analytics packages. This frustration was compounded late last year when Google moved to 100% secure search — whether searchers were logged in to Google or not.

Now, we see another step toward security with Google announcing a potential rankings boost for sites that run HTTPS.

How Will This Impact Your Rankings?
Several years ago, Google announced that site speed would be considered a ranking factor in its search algorithm. As a result, many sites rushed to improve their site load time. While users certainly appreciated the speed improvement, hardly anyone noticed a direct impact to their rankings. Why was that?

Page Speed is what’s called a “modifier.” If two web pages have very similar quality and relevance scores, Google considers which page loads faster as the deciding factor on which ranks higher. The loading speed of the page modified the ranking score only ever so slightly.

Similarly, HTTPS looks to be a modifier, from what I’ve seen. Ninety-nine percent of searches will happen without HTTPS even being looked at; but, in those rare cases when two search results are otherwise “equal,” HTTPS might push one over the edge for the higher ranking.

This Is About Politics, Not Search Quality
Google has a phase it likes to use: “HTTPS Everywhere.” In fact, that’s what they named this year’s I/O Conference. The idea is that if every site implemented HTTPS, the web would be that much more secure; but, it’s a red herring. Here’s why:

HTTPS only protects against a very limited number of site vulnerabilities, specifically wiretapping and man-in-the-middle type attacks – in other words, spying. It makes the NSA’s job of tracking and spying on internet users more difficult, but it doesn’t protect against hackers, denial-of-service attacks and scripting, server or database exploits.

Essentially, HTTPS is useful for sites that collect and transmit personal information. Banks, e-commerce sites, even social networks need to have HTTPS in place to make sure consumers’ sensitive information is protected.

For all the blogs, news sites, brand brochure-type sites or any information site that doesn’t require a member login, HTTPS is useless. It’s like the post office telling you to that all your mail needs to be written in secret code. That’s fine for the military, but do your Christmas greeting cards really need to be written in unbreakable secret code? Probably not. It’s just as pointless to require HTTPS on sites that do not transfer sensitive information.

That’s why it doesn’t make sense for Google to consider using HTTPS as a ranking signal for the majority of sites and queries. If used at all, it will always be a very lightweight signal used on a very narrow set of queries, acting only as a tie breaker between two identically ranked pages.

No, this announcement is not about search quality. It’s about Google trying to get back at the NSA for making it look bad during the PRISM scandal, and it is doing this under the guise of a social cause — internet privacy under the “HTTPS Everywhere” banner.

It’s a classic “greater good” story. Google says HTTPS will be a ranking signal so that everyone runs out and switches to HTTPS. What they’re not saying is that this change will only affect a minuscule number of sites. For everyone else, they’ve wasted time and energy switching to HTTPS for no reason – but that’s okay, because it serves the greater good of improving privacy for the internet as a whole.

What Should I Do?
What should you do about switching to HTTPS? When in doubt, do what’s best for users.

If you run a site in the e-commerce, financial, search, social networking or related fields, you should already be running HTTPS on it. In fact, if your site utilizes a member login or any type of shopping cart, you should really switch to HTTPS.

On the other hand, if you’re running a blog, brochure site, news site, or any sort of information site where users don’t provide you with any personal information, I would recommend not using HTTPS. It costs money; it takes resources to implement; it slows down your site; it’s not needed; and it won’t hurt your rankings.

Long story short: if you make the switch, do it for the users and not because Google said it’s a ranking signal, because it really isn’t.

HTTPS Sites Secure Ranking Boosts in Google

“We want to convince you that all communications should be secure by default.”

Those were the words uttered by Webmaster Trends Analyst Pierre Far at the Google I/O event this summer, when he and a Google colleague talked “HTTPS everywhere.” And this week, Google Search is taking a very convincing stance on the matter: HTTPS is now a ranking signal in its algorithm.

From Google’s announcement:

Over the past few months we’ve been running tests taking into account whether sites use secure, encrypted connections as a signal in our search ranking algorithms. We've seen positive results, so we're starting to use HTTPS as a ranking signal. For now it's only a very lightweight signal — affecting fewer than 1% of global queries and carrying less weight than other signals such as high-quality content — while we give webmasters time to switch to HTTPS. But over time, we may decide to strengthen it, because we’d like to encourage all website owners to switch from HTTP to HTTPS to keep everyone safe on the web.

On Google+, Webmaster Trends Analyst John Mueller answered questions from the community, like, "What if you have an informational site – does it apply to you, too?"

Mueller said this:

Some webmasters say they have "just a content site," like a blog, and that doesn't need to be secured. That misses out two immediate benefits you get as a site owner:

1. Data integrity: only by serving securely can you guarantee that someone is not altering how your content is received by your users. How many times have you accessed a site on an open network or from a hotel and got unexpected ads? This is a very visible manifestation of the issue, but it can be much more subtle.

2. Authentication: How can users trust that the site is really the one it says it is? Imagine you're a content site that gives financial or medical advice. If I operated such a site, I'd really want to tell my readers that the advice they're reading is genuinely mine and not someone else pretending to be me.

On top of these, your users get obvious (and not-so-obvious) benefits.

Moving a site from HTTP to HTTPS could have technical problems if not implemented carefully. Google gives tips on how to handle the move here.

And, in its help files, it also talks about best practices for setting up HTTPS, which include helping the search engines see the site as secure by following these tips (more details exist on the help page itself):

Redirect your users and search engines to the HTTPS page or resource with server-side 301 HTTP redirects.

Use relative URLs for resources that reside on the same secure domain.

Use protocol relative URLs for all other domains or update your site links to link directly to the HTTPS resource.

Use a web server that supports HTTP Strict Transport Security (HSTS) and make sure it's enabled.

If you have questions or concerns, Google is directing people to the Webmaster Help Forums. For example, this search for “HTTPS” in the forums pulls up several conversations already happening on the matter. The announcement said that in the coming weeks, Google would be publishing detailed best practices on this issue.