Tuesday, 17 February 2009

From the height of this place

I originally wrote this email for internal consumption; Presidents' Day here in the US and President Obama's recent inaugural address got me thinking about the future of the Internet, Google, and the challenges that lie ahead. The note borrows from a host of US presidential inaugural addresses to illustrate some of its points (thanks to former President Clinton for the title). Quite a few Googlers suggested I share it externally, so here it is, with just a few minor edits. - Jonathan Rosenberg

Dear Googlers -

Today is Presidents' Day here in the United States, when we honor the birthdays of two of our country's greatest leaders, George Washington and Abraham Lincoln. A few weeks ago many of us were lucky to witness, either in person or via TV or the web, a masterful inauguration speech by the newest President, Barack Obama. The speech was rife with poignant points and subtle historical allusions: "We the people" came directly from the US Constitution, while "all are equal, all are free, and all deserve a chance to pursue their full measure of happiness" echoes both the Declaration of Independence and Abraham Lincoln's Gettysburg Address. (Many of these nuances were only revealed to me upon reading the transcript.)

As expected, President Obama aptly captured the wary mood of the nation. After all, we are in the midst of what is likely the worst economic situation of our lifetimes. In the US alone, 2.6M people lost their jobs in 2008, followed by nearly 600,000 more last month, and on the Monday following the inauguration companies around the world, including Caterpillar, Pfizer, ING, and Phillips announced job cuts totaling over 75,000. Add to that our dependence on fossil fuels, the resulting (and accelerating) climate change, and national security concerns, and you can feel the gravity of this pivotal moment. Eric Schmidt has called these times 'uncharted waters': none of us has been here before.

President Obama asserted that we will face the moment with what he called new instruments and old values, values that have been "the quiet force of progress throughout history" and which must, once again, define our character. While this reference to the national character of the US was no doubt inspiring for Americans, the mention of "new instruments" was far more relevant to Google. In a way, I felt like he was talking about the Internet, which is the most powerful and comprehensive information system ever invented.

Consider its predecessors. The famed Library at Alexandria (that's Egypt, not Virginia - some of you have GOT to get out more ;) ) was built circa 323 BC for an educated public, which actually meant very few people since the skills of literacy were deliberately withheld from the majority of the population. For several centuries monks were the keepers of the written word, painstakingly transcribing and indexing books as a means of interpreting the word of God. They were prized as much for their ability to write small, which saved on expensive paper, as for their piety.

The first universities came about in the 4th century AD, the first formal encyclopedias didn't appear until the 16th century, the first truly public libraries appeared in the 19th century and proliferated in the 20th. Then suddenly comes the Internet, where, from the most remote villages on the planet, you can reach as much information as is held in thousands of libraries. Access to information has completed its journey from privileged to ubiquitous. At Google we are all so immersed in daily introspective exercises like product reviews, our GPS [Google Product Strategy] meetings, and budget exercises that it's easy to forget this.

We shouldn't. In fact, since the challenges the world faces are, to a large degree, information problems, I believe the Internet is one of the "new instruments" that the President and the world can count on. And how do a great many people use the Internet? What is the first place many of them go when they conduct research, seek answers, do their work and communicate with their friends and family? Google. Ours is much more than a passing role in this next phase of history, rather we have the responsibility and duty to make the Internet as great as it can possibly be. Fortunately, that is pretty much what we all set out to do every day anyway, but now there's just a little extra pressure. Not your average 9-to-5 job.

At Google we are all technology optimists. We intrinsically believe that the wave upon which we surf, the secular shift of information, communications, and commerce to the Internet, is still in its early stages, and that its result will be a preponderance of good.

As we look toward the pivotal year ahead, here are a few observations on the future of the Internet for all of us to assess, consider, and carry as we do our work. (I have occasionally borrowed the inaugural words of previous presidents, sometimes cited, as with Bill Clinton's phrase which I appropriated for my title, and sometimes not.) To paraphrase President Obama, these things will not happen easily or in a short span of time, but know this my colleagues: they will happen.

All the world's information will be accessible from the palm of every person
Today, over 1.4 billion people, nearly a quarter of the world's population, use the Internet, with more than 200 million new people coming online every year. This is the fastest growing communications medium in history. How fast? When the Internet was first made available to the public, in 1983, there were 400 servers. Twenty five years later: well over 600 million.

In many parts of the world people access the Internet via their mobile phones, and the numbers there are even more impressive. More than three billion people have mobile phones, with 1.2 billion new phones expected to be sold this year. More Internet-enabled phones will be sold and activated in 2009 than personal computers. China is a prime example of where these trends are coming together. It has more Internet users than any other country, at nearly 300 million, and more than 600 million mobile users — 600 million! Twenty-five years ago, Apple launched the Mac as "the computer for the rest of us." Today, the computer for the rest of us is a phone.

This means that every fellow citizen of the world will have in his or her pocket the ability to access the world's information. As this happens, search will remain the killer application. For most people, it is the reason they access the Internet: to find answers and solve real problems.

Our ongoing challenge is to create the perfect search engine, and it's a really hard problem. To do a perfect job, you need to understand all the world's information, and the meaning of every query. With all that understanding, you then have to produce the perfect answer instantly. Today, many queries remain very difficult to answer properly. Too often, we force users to correct our mistakes, making them refine their searches, trying new queries until they get what they need. Meanwhile, our understanding of the interplay between high-quality content, search algorithms, and personal information is just beginning.

Why should a user have to ask us a question to get the information she needs? With her permission, why don't we surf the web on her behalf, and present interesting and relevant information to her as we come across it? This is a very hard thing to do well, as anyone who has been presented with a where-the-heck-did-that-come-from recommendation on Amazon or Netflix can attest to, but its potential is huge.

While we're working on improving the quality of search, the web is exploding. Our infrastructure has to keep up with this growth just to maintain our current level of quality, but to actually make search smarter, our index and infrastructure need to grow at a pace FASTER than the web. Only then will we be able to reject the idea that we have to choose between latency, comprehensiveness, and relevancy; we will have the ability to preserve all our ideals.

Solving search is a long-term quest for perfection, but the transition of information from scarce and expensive to ubiquitous and free will conclude far sooner. We will then bear witness to a true democratization of information, a time when almost everyone who wants to be online will be online, able to access virtually every bit of the world's information. This is great for our business, even greater for all the users. In fact, it's difficult to overestimate how important that moment will be. As Harry Truman said, "Democracy of information alone can supply the vitalizing force to stir the peoples of the world into triumphant action." (OK, I added the "of information" part!)

Everyone can publish, and everyone will
One thing that we have learned in our industry is that people have a lot to say. They are using the Internet to publish things at an astonishing pace. 120K blogs are created daily — most of them with an audience of one. Over half of them are created by people under the age of nineteen. In the US, nearly 40 percent of Internet users upload videos, and globally over fifteen hours of video are uploaded to YouTube every minute. The web is very social too: about one of every six minutes that people spend online is spent in a social network of some type.

Publishing used to be constrained by physical limitations. You had to have a printing press and a distribution network, or a transmitter, to publish to any sort of critical mass, so broadcasting was the norm. No more. Today, most publishing is done by users for users, one-to-one or one-to-many (think of Twitter, Facebook, Wikipedia, and YouTube). Free speech is no longer just a right granted by law, but one imbued by technology.

The era of information being more powerful when hoarded has also passed. As our economist Hal Varian has noted, in the early days of the Web every document had at the bottom, "Copyright 1997. Do not redistribute." Now those same documents have at the bottom, "Copyright 2009. Click here to send to your friends." Sharing, not guarding information, has become the golden standard on the web, so not only can anyone publish, but virtually everyone does. This is both good and bad news. No one argues the value of free speech, but the vast majority of stuff we find on the web is useless. The clamor of junk threatens to drown out voices of quality.

Meanwhile, those voices are struggling. The most obvious example is newspapers, which have historically been the backbone of quality original reporting, a post they have mostly maintained throughout the Internet explosion. But news isn't what it used to be: by the time a paper arrives in the morning it's already stale. As written communication has evolved from long letter to short text message, news has largely shifted from thoughtful to spontaneous. The old-fashioned static news article is now just a starting point, inciting back-and-forth debate that often results in a more balanced and detailed assessment. And the old-fashioned business model of bundled news, where the classifieds basically subsidized a lot of the high-quality reporting on the front page, has been thoroughly disrupted.

This is a problem, but since online journalism is still in its relative infancy it's one that can be solved (we're technology optimists, remember?). The experience of consuming news on the web today fails to take full advantage of the power of technology. It doesn't understand what users want in order to give them what they need. When I go to a site like the New York Times or the San Jose Mercury, it should know what I am interested in and what has changed since my last visit. If I read the story on the US stimulus package only six hours ago, then just show me the updates the reporter has filed since then (and the most interesting responses from readers, bloggers, or other sources). If Thomas Friedman has filed a column since I last checked, tell me that on the front page. Beyond that, present to me a front page rich with interesting content selected by smart editors, customized based on my reading habits (tracked with my permission). Browsing a newspaper is rewarding and serendipitous, and doing it online should be even better. This will not by itself solve the newspapers' business problems, but our heritage suggests that creating a superior user experience is the best place to start.

Of course, the greatest user experience is pretty useless if there's nothing good to read, a truism that applies not just to newspapers but to the web in general. Just like a newspaper needs great reporters, the web needs experts. When it comes to information, not all of it is created equal and the web's future depends on attracting the best of it. There are millions of people in the world who are truly experts in their fields — scientists, scholars, artists, engineers, architects — but a great majority of them are too busy being experts in their fields to become experts in ours. They have a lot to say but no time to say it.

Systems that facilitate high-quality content creation and editing are crucial for the Internet's continued growth, because without them we will all sink in a cesspool of drivel. We need to make it easier for the experts, journalists, and editors that we actually trust to publish their work under an authorship model that is authenticated and extensible, and then to monetize in a meaningful way. We need to make it easier for a user who sees one piece by an expert he likes to search through that expert's entire body of work. Then our users will be able to benefit from the best of both worlds: thoughtful and spontaneous, long form and short, of the ages and in the moment.

We won't (and shouldn't) try to stop the faceless scribes of drivel, but we can move them to the back row of the arena. As Harry Truman said in 1949, "We are aided by all who want relief from the lies of propaganda — who desire truth and sincerity."

When data is abundant, intelligence will win
Putting the power to publish and consume content into the hands of more people in more places enables everyone to start conversations with facts. With facts, negotiations can become less about who yells louder, but about who has the stronger data. They can also be an equalizer that enables better decisions and more civil discourse. Or, as Thomas Jefferson put it at the start of his first term, "Error of opinion may be tolerated where reason is left free to combat it."

The Internet allows for deeper and more informed participation and representation than has ever been possible. We see this happening frequently, particularly with our Geo products. The Surui tribe in the Amazon rain forest uses Google Earth to mark the boundaries of their land and work with authorities to stop illegal logging. Sokwanele, a civic action group in Zimbabwe, used the Google Maps API on their website to document reported cases of political violence and intimidation after the controversial Presidential election in March 2008. Armed with this map, the group can better convey and defend their argument that elections in Zimbabwe are neither free nor fair. The stakes couldn't be higher for these people. We can give them a fighting chance.

Everyone should be able to defend arguments with data. To let them do so, we need tools like the Sitemaps protocol, which opens up large volumes of data previously trapped behind government firewalls. Most government websites can't be crawled, but with Sitemaps, thousands of pages have been unlocked. In the US, several states have opened up their public records through Sitemaps, and the Department of Energy's Office of Science & Technology Information made 2.3 million research findings available in just twelve hours.

Information transparency helps people decide who is right and who is wrong and to determine who is telling the truth. When then-Senator Clinton incorrectly stated during the 2008 Presidential campaign that she had come under sniper fire during her 1996 trip to Bosnia, the Internet set her straight. This is why President Obama's promise to "do our business in the light of day" is important, because transparency empowers the populace and demands accountability as its immediate offspring.

But as powerful as it can be in politics, data has the potential to be even more transformational in business. Oil fueled the Industrial Revolution, but data will fuel the next generation of growth. One of the largely unheralded by-products of the Internet era is how it has made the power of the most sophisticated analytical tools available to the smallest of businesses. Traditionally, business software packages have treated data reporting as a second class citizen. Here is my cool new feature, they say. Oh, you want to know how many people use it? You want the flexibility to organize and assess this data in ways that work best for you? Well, let us tell you about the analytics module! It's only tens of thousands of dollars more (not counting the 18% annual maintenance fee in perpetuity ... sucker!!)

Fortunately that's not Google, nor can it ever be. All of our products should reflect our bias toward giving our customers, users, and partners as much data as possible - and letting them do with it what they wish. Then they can run their business like we do, by making decisions based on facts, not opinions. Here at Google the words of every colleague, from associates to vice presidents, carry the same weight so long as they are backed by data. (If you don't think we live up to this standard then please feel free to correct me ... but you better have the facts to prove it!!!)

Hal Varian likes to say that the sexy job in the next ten years will be statisticians. After all, who would have guessed that computer engineers would be the cool job of the 90s? When every business has free and ubiquitous data, the ability to understand it and extract value from it becomes the complimentary scarce factor. It leads to intelligence, and the intelligent business is the successful business, regardless of its size. Data is the sword of the 21st century, those who wield it well, the Samurai.

In 1913, Woodrow Wilson stated, "... and yet, it will be no cool process of mere science ... with which we face this new age of right and opportunity." Perhaps, but from our perspective the cool process of mere science, fueled by ubiquitous data and intelligence, will be quite sufficient to power new generations to success.

The vast majority of computing will occur in the cloud
Within the next decade, people will use their computers completely differently than how they do today. All of their files, correspondence, contacts, pictures, and videos will be stored or backed-up in the network cloud and they will access them from wherever they happen to be on whatever device they happen to hold. Access to data, applications, and content will be seamless and device-agnostic. Convergence isn't something that occurs at the device level, which was the vision we all had in the 90s as we struggled to invent that perfect gadget that did it all (witness my own unfortunate progeny, the Apple Newton, which ended tragically). Rather, devices will proliferate in many directions, but all of them will converge on the cloud. That's where our stuff, not to mention civilization's knowledge, will live.

This doesn't mean that the access device simply becomes a juiced up version of an old 3270 terminal. To the contrary, smart programmers will figure out ways to use all that power in your hands to create great applications, and to let you run them whether or not you are connected. But it shouldn't take three minutes for the device to boot, and losing it shouldn't be a catastrophe. You'll just get a new one and it will sync instantly; all your contacts, pictures, music, files, and other stuff will automagically just be there, ready for you to log in and say "be mine".

Still, these examples simplify and understate the true impact of what is going on with the transition to cloud computing. As Hal has noted, we are in a period of "combinatorial innovation", when there is a great availability of different component parts that innovators can combine or recombine to create new inventions. In the 1800s, it was interchangeable parts. In the 1920s, it was electronics. In the 1970s, it was integrated circuits. Today, the components of innovation are found in cloud computing, with abundant APIs, open source software, and low-cost, pay-as-you-go application services like our own App Engine and Amazon's EC2. The components are abundant and available to anyone who can get online.

The power of innovation and the cloud are driving two trends. First, because the tools of innovation are so easy and inexpensive to access, and consumers are so numerous and easy to reach, the consumer market now gets the greatest innovations first. It's easy to forget that just twenty years ago the best technology was found in the workplace: computers, software, phone systems, etc. Thirty years ago all you software geniuses working on Search, Ads, and Apps would have been programmers at IBM; forty years ago, at NASA. Now, the best technology starts with consumers, where a Darwinian market drives innovation that far surpasses traditional enterprise tools, and migrates to the workplace only after thriving with consumers. Think of Google Video for Business, which started out as YouTube and then evolved to the enterprise. How many businesses out there have even conceived of how useful this can be to them? Not many, perhaps because only a year ago the costs of having such an internal service were prohibitive. No longer.

Second, it used to be that every growing business would at some point have to make a big investment in computers and software for accounting systems, customer management systems, email servers, maybe even phone or video conferencing systems. Today, all of those services are available via the network cloud, and you pay for it only as you use it. So small businesses can scale up without making those huge capital investments, which is especially important in a recession. Access to sophisticated computer systems, and all the value they can deliver, was previously the realm of larger companies. Cloud computing levels that playing field so that the small business has access to the same systems that large businesses do. Given that small businesses generate most of the jobs in the economy, this is no small trend.

We still have a long way to go in making web-based applications robust enough for businesses. Things like latency, data reliability, and security all have to be equal to or better than the currently available alternatives. The user experience needs to be fast, easy, and rich — "like reading a magazine," Larry has said. This is why we are building Chrome, Gears, V8 and more. Users now expect these apps to work perfectly for them all the time, and we need to meet that expectation.

The real potential of cloud computing lies not in taking stuff that used to live on PCs and putting it online, but in doing things online that were previously simply impossible. Combining open standards with cloud computing will enable businesses to conduct commerce in brand new ways. For example, there is a great opportunity to take advantage of (to quote Hal again) "computer-mediated transactions". Computers now mediate virtually every commercial transaction, recording it, collecting data, and monitoring it, which means that we can now write and enforce contracts that were previously impossible. When you rent a car, you could be offered a thirty percent discount for agreeing not to exceed the speed limit, a deal that they could actually enforce with GPS reporting! Would you take it?

Another example is machine language translation. As more people do more things online computer systems will have the opportunity to learn from the collective behavior of billions of humans. Translation will get a tiny bit smarter with each iteration. There are over 400,000 books in the modern version of the aforementioned Library of Alexandria, nearly half of them in Arabic. The culture and history that's in those books is not available to you unless you read Arabic, which of course most people don't. But soon, with the power of the cloud, they'll be able to read them anyway. This is why translation is important, because it gives us the ability, to quote John Adams from 1797, to "encourage schools, colleges, universities, academies, and every institution [to propagate] knowledge, virtue, and religion among all classes of the people."

Lit by lightning
Virtually every American president since George Washington has used his Inaugural Address to speak not just of the coming four years, but also of his vision for future generations. Similarly, we manage Google with a long-term focus. We live and run our business in these uncertain times, but our eyes are always on the future, on the better tomorrow that the Internet and all of its promise shall help bring to fruition. I hope that the four predictions that I have presented here will elicit your curiosity and illuminate the significance of the changes that lay ahead. They may inevitably come to pass, but their impact on us, our neighbors, our countries, and our world is not inevitably good. Hence, our challenge.

We are standing at a unique moment in history which will help define not just the Internet for the next few years, but the Internet that individuals and societies around the world will traverse for decades. As Googlers our responsibility is nothing less than to help support the future of information, the global transition in how it is created, shared, consumed, and used to solve big problems. Our challenge is to steer incessantly toward greatness, to never think small when we can think big, to strive on with the work Larry and Sergey began over ten years ago, and from this task we will not be moved. In a world that feels like it is lit by lightning, speed wins, and we have a responsibility to our users to not retreat, to not be content to stand still, to not be complacent or near-sighted. The Internet has had a profound and remarkable impact in the past decade. Now, from the height of this place, let's appreciate its implications and pursue its promise.

It seems only fitting to conclude this Presidents' Day treatise, which began by quoting our 44th president, with a statement from our first. And so, having thus imparted to you my sentiments as they have been awakened by the occasion which brings us together, I shall take my present leave.