Home
Internet
Internet News
Google casts a big shadow on smaller web sites

Google casts a big shadow on smaller web sites

By Steve Lohr and Claire Cain Miller, The New York Times | Updated: 5 November 2012 11:35 IST

Starting in February, Jeffrey G. Katz grew increasingly anxious as he watched the steady decline of online traffic to his company's comparison-shopping Web site, Nextag, from Google's search engine.

In a geeky fire drill, engineers and outside consultants at Nextag scrambled to see if the problem was its own fault. Maybe some inadvertent change had prompted Google's algorithm to demote Nextag when a person typed in shopping-related search terms like "kitchen table" or "lawn mower."

But no, the engineers determined. And traffic from Google's search engine continued to decline, by half.

Nextag's response? It doubled its spending on Google paid search advertising in the last five months.

The move was costly but necessary to retain shoppers, Mr. Katz says, because an estimated 60 percent of Nextag's traffic comes from Google, both from free search and paid search ads, which are ads that are related to search results and appear next to them. "We had to do it," says Mr. Katz, chief executive of Wize Commerce, owner of Nextag. "We're living in Google's world."

Regulators in the United States and Europe are conducting sweeping inquiries of Google, the dominant Internet search and advertising company. Google rose by technological innovation and business acumen; in the United States, it has 67 percent of the search market and collects 75 percent of search ad dollars. Being big is no crime, but if a powerful company uses market muscle to stifle competition, that is an antitrust violation.

So the government is focusing on life in Google's world for the sprawling economic ecosystem of Web sites that depend on their ranking in search results. What is it like to live this way, in a giant's shadow? The experience of its inhabitants is nuanced and complex, a blend of admiration and fear.

The relationship between Google and Web sites, publishers and advertisers often seems lopsided, if not unfair. Yet Google has also provided and nurtured a landscape of opportunity. Its ecosystem generates $80 billion a year in revenue for 1.8 million businesses, Web sites and nonprofit organizations in the United States alone, it estimates.

The government's scrutiny of Google is the most exhaustive investigation of a major corporation since the pursuit of Microsoft in the late 1990s.

The staff of the Federal Trade Commission has recommended preparing an antitrust suit against Google, according to people briefed on the inquiry, who spoke on the condition they not be identified. But the commissioners must vote to proceed. Even if they do, the government and Google could settle.

Google has drawn the attention of antitrust officials as it has moved aggressively beyond its dominant product - search and search advertising - into fields like online commerce and local reviews. The antitrust issue is whether Google uses its search engine to favor its offerings like Google Shopping and Google Plus Local over rivals.

For policy makers, Google is a tough call.

"What to do with an attractive monopolist, like Google, is a really challenging issue for antitrust," says Tim Wu, a professor at Columbia Law School and a former senior adviser to the F.T.C. "The goal is to encourage them to stay in power by continuing to innovate instead of excluding competitors."

Speaking at a Google Zeitgeist conference in Arizona last month, Larry Page, the company's co-founder and chief executive, said he understood the government scrutiny of his company, given Google's size and reach. "There's very many decisions we make that really impact a lot of people," he acknowledged.

The main reason is that Google is continually adjusting its search algorithm - the smart software that determines the relevance, ranking and presentation of search results, typically links to other Web sites.

Google says it makes the changes to improve its service, and has long maintained that its algorithm weeds out low-quality sites and shows the most useful results, whether or not they link to Google products.

"Our first and highest goal has to be to get the user the information they want as quickly and easily as possible," says Matt Cutts, leader of the Web spam team at Google.

But Google's algorithm is secret, and changes can leave Web sites scrambling.

Consider Vote-USA.org, a nonprofit group started in 2003. It provides online information for voters to avoid the frustration of arriving at a polling booth and barely recognizing half the names on the ballot. The site posts free sample ballots for federal, state and local elections with candidates' pictures, biographies and views on issues.

In the 2004 and 2006 elections, users created tens of thousands of sample ballots. By 2008, traffic had fallen sharply, says Ron Kahlow, who runs Vote-USA.org, because "we dropped off the face of the map on Google."

As founder of a search-engine optimization company and a recipient of grants that Google gives nonprofits to advertise free, Mr. Kahlow knows a thing or two about how to operate in Google's world. He pored over Google's guidelines for Web sites, made changes and e-mailed Google. Yet he received no response.

"I lost all donations to support the operation," he said. "It was very, very painful."

A breakthrough came through a personal connection. A friend of Mr. Kahlow knew Ed Black, chief executive of the Computer & Communications Industry Association, whose members include Google. Mr. Black made an inquiry on Mr. Kahlow's behalf, and a Google engineer investigated.

The problem, Mr. Kahlow learned, was that the site's state Web pages also had information for national candidates - reasonable to a person seeking voting information at one digital location. But to Google's algorithm, duplicate content on a site suggests a shady shortcut to try to make a site look bigger than it is.

Mr. Kahlow fixed that, so that someone on a state page must go to another page to see the ballot and information about national candidates. Vote-USA.org quickly moved off Google's black list, and during this election season people have been viewing 333,000 pages a day.

Last year, Mr. Kahlow said, F.T.C. investigators asked him if he thought his site was a target of discrimination. No, he replied. But since then, he has watched Google promote its tools for finding where to vote and sample ballots, just like his site offers.

"At that time, I didn't believe it was intentional, but I'm having second thoughts," he says. "I'm sure they're aware of the amount of money that's being spent in politics and I'm sure they'd like to get their fingers in the pie."

Google will not comment on its relations with specific Web sites. But a person briefed on its interactions with Vote-USA.org said Google had found additional duplication, like Michigan candidates showing up on the Iowa Web site. The person also said that hardly any other Web sites linked to Vote-USA.org, one of the most important elements in the algorithm.

Early last year, operators of small local news sites nationwide found that their number of readers had plummeted. Why? Their Web sites had disappeared from Google News, which in many cases was their No. 1 source of traffic.

Their owners wrote e-mail messages to Google and scoured online forums, to no avail.

"There was no explanation why or place you could go for more information," says Hal Goodtree, editor and publisher of CaryCitizen, a local news site in Cary, N.C.

Google News, he says, brings traffic and credibility. "You're legit if you're on Google News," he explains, "so it was painful."

Lance Knobel, co-founder of Berkeleyside, a local news site in Berkeley, Calif., complained about the problem on Twitter, and a Berkeleyside reader who was a Google engineer eventually saw it. He promised Mr. Knobel he would look into it, unofficially.

Twelve hours later, Berkeleyside reappeared in Google News. Google said dropping the site was an error.

In North Carolina, Mr. Goodtree heard nothing for months about what had happened. In June, he received an e-mail from Google saying it was "unable to provide specifics" about why the site had been dropped.

Then, in August, Google wrote, "We recently reviewed your site again and have decided to add it to Google News once more." No further explanation was given. Traffic to CaryCitizen jumped 24 percent. Mr. Knobel, in California, says of Google: "It's a totally opaque corporation."

Mr. Cutts of Google says it is impossible for the company to respond individually to every Web site owner with a question, because of sheer scale; there are 240 million domain names and people search Google more than 3.3 billion times a day.

Yet Google is trying to do more, he says. Last year, it started hosting video chats for Web site owners to ask questions. It has also started publishing blog posts about changes in the algorithm and broadcast eight minutes of a top-secret meeting about it.

"We try very hard to make sure we communicate with Webmasters," Mr. Cutts says. For instance, he says, he reassured Gawker Media that it would remain in search results after Hurricane Sandy caused it to crash.

Google does not compete with sites like CaryCitizen and Berkeleyside in the labor-intensive chore of reporting and writing local news. But local news sites, notes Mr. Knobel of Berkeleyside, do compete in their way with Google for local advertising.

As the company builds up Google Plus Local, its local business listing and review service, Mr. Knobel says Google's search engine could give the Internet giant "a tremendous advantage." That would result, he says, if a person seeking information about a local business is steered to Google Plus Local rather than to small sites like Berkeleyside or sizable services like Yelp.

At the Arizona conference, Mr. Page was asked about Google's competition with other Web businesses. "That's always a hard question," he replied, but not a new one. He pointed to Google Maps, which was introduced in 2005. "If you think back then," he said, "we had the same kind of criticisms, like 'Oh, there's already MapQuest.' Anybody heard of them? No one uses them anymore."

Today, MapQuest has half the number of monthly visitors as Google Maps, according to comScore.

Google's overriding goal, Mr. Page suggests, is continual improvement of its product, which, he says, means adding more services that collect and parse data. Some competitors may suffer. But, he adds, "our job is to serve users."

Google has argued that forcefully to regulators because in antitrust, consumer benefit weighs heavily. But this is an evasion to competitors like Mr. Katz of Nextag, who has been questioned by regulators. He says Google increasingly presents itself as a commerce site as much as a search engine, pointing to changes that occurred mainly in the last year or two.

Today, if you Google a phrase like "patio furniture," the results are not just a list of blue links to other Web sites. Instead, you will find a few ads, shaded in beige to identify them. On the right side of the page are product pictures and links to merchants, part of Google Shopping. Below Google Shopping is a Google map with locations of local stores. Then, sprawled down the page, are the blue links.

The effect, Mr. Katz says, is that Google commerce services get more of the prime real estate near the top of the screen. That means that even if Nextag ranks high in the organic search results, which costs him nothing, a user is less likely to see the link. He and his team also say changes in Google's algorithm in recent months have hurt Nextag.

His Google traffic now costs more. Two years ago, 60 percent of Nextag's traffic from Google was from free search and 40 percent paid - people clicking on ads Nextag bought. Today, it is 30 percent free and 70 percent paid.

But his company has also shifted its strategy to become less vulnerable to Google's charge into commerce. It has invested heavily in its underlying technology to help Web sites attract visitors, especially ones most likely to buy their goods.

Wize Commerce, the Nextag owner, is based in San Mateo, Calif.; it is profitable and employs 450 people. The revised plan, Mr. Katz says, "gives us a shot at being a very healthy company," less dependent on Google.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.