keys to success
 

A new economy: the rise of publicly funded projects.

24 February, 23:18, by Gregory Magarshak Tags: , , , , , , ,

For a long time, debates have ranged between capitalism and socialism, each side staunchly embracing their ideology and demonizing the other. This scene is played out in American politics, with conservatives and libertarians arguing for freer markets and smaller government, while liberals and democrats focus on equal rights, quality of life, and democracy.

We can have both. There are countries in Europe whose social programs are executed better than they are in the United States. Education and medicine are often free. In America, moves to improve the situation for the lower class are often met with derision as “socialist” and “redistribution of wealth”. But the issue goes much deeper than that.

Often, the issue is not simply socialism vs capitalism, but it is between open and free information vs. proprietary information controlled by a business entity. Ironically, modern capitalism relies on government policing and enforcement of monopolies in the form of copyright and patent licensing. The same people, who talk about reducing government intervention and promoting individual liberty through free market capitalism, find it necessary to have the government step in and help crack down on copyright and patent infringements. The RIAA is complaining about declining revenues and prices for music becoming lower. The truth is, whether it’s labor unions or the RIAA, monopolies allow the incumbents to try and hold on to the status quo, to keep the keep the cost of entry high for little guys, and to keep the prices high. While those who dislike socialism in all its forms might decry government protection for labor unions, they would usually take the opposite stance on copyright issues.

When copyrights and patents get involved, things can get pretty ridiculous. Eolas patented “active content” in browsers. Amazon patented “one-click” purchasing. Patents threatened de facto standards like GIF and HTML5 video. Paul Allen’s lawuit against a large portion of the web is completely, or at least close to completely, legal and intended by the Patent system, which grants 17-year monopolies: the ideas were non-obvious and useful in when they were patented, but does that mean the entire industry now has to pay a royalty to the guy who filed a patent? The British government passed a law censoring and disconnecting households and businesses from the internet without the usual machinations of due process.The US government is poised to pass the COICA bill which would allow the government to control the DNS system and essentially shut off suspected copyright infringers from the internet. When it comes to internet technology, it is a challenge for lawmakers to keep up with the times. Is government-enforced monopoly a bad idea in principle, because it contradicts individual liberty and prevents competition?

In the software world, we have seen that open source has been a powerful force and accomplished many good things. Proprietary software is starting to look outdated (compare IE to the other browsers). Because of that fateful decision by Netscape to release their code as free software, forming the Mozilla corporation, we all benefit. Firefox and WebKit are products of open source software. Business like Apple and Google benefit from this, too. They can’t make money on selling the browser, but new markets have been created in their economic sector, and their relevance is continuing, whereas Microsoft — which did not embrace open source software — is becoming less relevant.

Open Source did not have to be a communist-run government effort. It happened because of dedicated, thoughtful people who changed the world. They did it not by taking away the freedom of businesses to conduct business, but by building an alternative ecosystem, which is now thriving in ways which do not rely on hiding information and prosecuting those who would share it.

Can we apply the same principles to other industries: drugs, books, movies, songs, etc? Perhaps. There have been efforts to make “open source” licenses for patents similar to the GNU license in copyrights. (Personally, I think they are a little too strong. A patent license would be analogous to a GNU license if it required the user of the patent to cause all their patents which are derived from the used patent or used in conjunction with it by the actual business to be shared under the same license — that is something that can be proven in court anyway.)

Basically, the system would provide an alternative to the proprietary system we have now, which encourages innovation through promises of possible great riches. There has been a lot of new science and economics focused on what motivates people, and there is a great video that shows we are motivated by more than that:

Lots of smart people are motivated by accomplishment. That is a huge force in economics, and if we are able to nurture it and build up a supportive culture around it, together with its own values and focus on improving the world, we can harness it to produce new innovation in the world more cheaply and effectively. The Free Software movement was modeled on the activities of scientific researchers. Many of them have gone into academia because they were driven by passion for the work they did in their field, not by the promise of riches. Once the basic money is out of the way — with a large margin for comfort (3x-4x your living expenses), smart people are driven by making a difference in the world. Patents don’t factor in. Jonas Salk, who invented the polio vaccine, said his sole focus had been to develop a safe and effective vaccine as rapidly as possible, with no interest in personal profit. When he was asked in a televised interview who owned the patent to the vaccine, Salk replied: “There is no patent. Could you patent the sun?”

So what would an alternative system look like? It would be funded by people (“walkathon to find the cure for X”) and later, by corporations (similar to Google and Apple who contribute open source foundations). The breakthroughs, innovations, and ideas would be protected by copyleft licenses and patentleft licenses. The open source industries will probably produce better and better utilitarian products (drugs, wikipedias, etc.). Eventually, people would be able to base their purchasing decisions on publicly available information from the industry itself, instead of annoying advertisements (“ask your doctor about X”). Because the passion and goodwill of the innovators is being leveraged,  R&D would be much cheaper. Because people would be able to freely build on the work of others, wherever it may have been done, lots of beneficial derivative products will arise, which may prove useful in the near future when drugs might become customize to individual genomes, body chemistry, etc. Distribution networks (think Apple’s iphone store, or Netflix, but for other industries) will compete on accessibility, richness of selection, etc. and will themselves probably contribute to the open source industries. The distribution networks will have to be regulated similar to the Meat Inspection Act of 1906 to ensure minimum safety and usability for the public.

When it comes to art, however, works produce by an open-source community, through lots of little contributions, might be too utilitarian for our tastes. As a society, we have always had heroes in art — famous writers, playwrights, composers, directors, painters, and so forth. It’s true, you won’t find famous heroes on Wikipedia (unless you look at the revision history, you won’t even know who the authors are). That’s what the creative commons licenses are for. For example, if you copy someone’s work, you have to leave the attribution to the original author. This works well for works of art with one author, but what about community open source projects, where there are many authors? Well, Rasmus Lerdorf, John Resig, and Guido von Rossum are all heroes for starting and heading up development of their own programming languages. Recognition is something that you don’t need to enforce on everyone. I don’t even need to credit Rasmus when using PHP. It’s common knowledge.

So I believe that the future holds some exciting promise: open source industries will arise in various disciplines. In everything except art which needs heroes, free information can bring much more to humanity than proprietary information protected by government-granted monopolies, because it has the potential to stimulate and harness people’s untapped passions and motivations, without the considerations of monetary profit. (For example, if finding a cure for the disease is far less lucrative than research of a way to simply manage the disease indefinitely, the open source community would still have many people going for the cure, because there is no conflict of interest. They don’t have to recoup the 700 million dollars spent by their company on amassing proprietary information, that cost has been amortized across the whole industry.) The open source solutions may eventually surpass the proprietary ones, which may seem outdated (as IE is today). If the result would be anything it is like in the software industry, would be a good world for all of us. As for art, I think there will also be a mix of community-created books, plays, songs, etc. But we will still need heroes, and that will happen through how the projects are set up. But, for things like movies and TV shows, we are still going to have to use the same actors throughout the whole movies — and that means the movies probably won’t be made using open source. Luckily, there are movie theaters.

Towards a new model for funding innovation and content for the public good

08 February, 21:08, by Gregory Magarshak Tags:

There is a great talk by the RSA about what drives us: http://www.youtube.com/watch?v=u6XAPnuFjJc … until recently, it was not clear from an economics point of view why there would be so much innovation and work done entirely for free. But smart people are driven by purpose and making a difference in the world, and as soon as they have enough money to take care of their needs (rent, food, etc.) with a large safety net (2x-3x) they will focus on contributing to make a difference, and will create a lot of value without needing any more money “right now”. I think this might be very important in the area of funding entrepreneurs.

I realize it’s controversial, but I did want to throw it out there that ever since digital copying has become essentially free, our ideas about copyright need to be re-examined. To illustrate why, I wrote a post on hacker news showing how one can make a legal pyramid scheme by relying on copyright:

http://news.ycombinator.com/item?id=2039527

Copyright and patent protection are essentially monopolies enforced by the government — no one except the inventor or copyright holder may take an action without first clearing it with them. Currently, that is the way it is, but the systems were created in times long ago, when making a copy of a book was much more expensive. Today, when making a copy of an e-book or song in digital format is “essentially” free, I think it’s important for us as a society to revisit the issue of intellectual property and see how much of it was done to protect the distributors (publishers, customer support, the cost of actually making copies etc.) and how much of it was to incentivize innovation.

I wonder, too, if the patent system for software is a smart idea, considering the pace at which software evolves — every time I ever read about a patent suit in the news, it seems really harmful to the actual innovation in the industry, and I have to wonder how many landmines a typical startup is actually infringing on, if every patent holder followed a policy of diligently suing all infringing startups out of existence. It’s well known that a lot of innovation comes from startups, and intellectual monopolies that last 17 years in a fast-moving field like software hurt startups more than large corporations that have money to throw around. It is in the interest of large corporations to use whatever means they have at their disposal to eliminate their competition or buy it — including large patent portfolios.

Sure, copyright and patents may have its problems, but without it, how can we have the creation of new content?

Movies, songs, drugs, software, and startups… some of them go on to benefit lots of people. But it takes $ to make them.

There’s the research and development phase,
There’s the distribution system

The question is, how can we as a society have the benefit of good products without enforcing some sort of monopoly (copyright or patent based) which causes ever invasive methods of policing things like copying bits?

Well, if everyone was free to copy the bits, where would the money come from to create new movies, books, drugs, etc?

One possibility is subscription services. All those “race for the cure” and other fundraisers, or St. Jude’s hospital, etc. ask you to give $ to support research to cure some disease. If we learned one thing from open source and scientific journals, it’s that work done in the open produces better and more accessible results in the long run, for the entire public. So if we raise money for cancer research, say, investing that money in proprietary labs might be seen as irresponsible. That money should be put into “open source” type efforts to create drugs for the public good.

If we apply this model (of open source foundations) to movies and songs, though, we’d completely change the culture. Look at Wikipedia. It is collaboratively edited and arguably a great resource. And yet, there are no rockstars on wikipedia, there are nothing like a great american novel or a famous author. It’s good content produced collectively.

For utilitarian things like drugs, and maybe even technology startups, the “open source foundation” is an interesting model, and certainly one that Google and Apple have benefited from (e.g. WebKit now powers Safari and Chrome, which made Android and iOS possible). We as a society have benefited from open source — mobile browsers are just one example.

But for art, our society wants to have heroes, it wants to have great movies. If there were 20 different versions of The Matrix, some that made more sense than others, it would be a completely different society. We’d need curation during the actual development process, to overcome the glut of content. We’d need “releases” to be ordained and distributed on distribution networks, while less popular forks fall by the wayside. We’d have more of a Linux-type world than a MacOS type world.

In a way, American Idol, America’s Next Great Restaurant, etc. are a similar model. The show pulls in ratings and can support itself with advertising. The curation during the show (done by the public, no less!) all but guarantees that the winners will be popular. The winners are then spun into a brand.

So I think the future of innovation and content creation in this world will have some of these elements: open source foundations, public curation during development, and the revenue will come from subscriptions to distribution networks, which will non-exclusively deliver the curated “releases” and will compete on ease of use. Content will be easily searchable, and accessible. Sure, private copying will take place. But the CULTURE of paying a subscription for easy access to new releases, the culture of curating the best brands (singers, movies, etc.) through competitions will keep the economy going.

The main question is, will that culture encourage good things to be produced for the public, and will it enable entry for new and fresh talent to step up and be identified? My hunch is, very much YES.

Why Do Dating Sites Charge?

08 June, 04:17, by Gregory Magarshak

Okay, let’s talk about business models, and what it takes to make a venture successful these days. They say that 98% of new businesses fail, and some say 80% — well, whoever says that is probably very wrong. In fact, around 51% of small businesses survive their first 5 years. So take heart, entrepreneur, and dare to achieve!

Anyway, let’s start by considering the title of this post. Why do dating sites continue to charge their users, and get away with it, when free social networking sites like Facebook have been around for years? And for that matter, why do investors pour millions of dollars into companies whose services are absolutely free to use?

Choosing Your Market

The first thing to consider when you are running a business is where the money is going to come from. There are lots of sources but they all have something in common: they have a need that your business will fulfill. So before you start a business, you’ve got to nail down exactly what market you will be getting your revenue from, and what you will be offering them.

Next, look at the competition in that market, and see how people are being served today. What’s interesting that there is often a decade-long cycle. Some of the most promising markets to enter are the ones where there is one large, incumbent leader that you can disrupt by providing a fresh new approach or business model. That market could have been very crowded several years ago, but now that there is one leader, it could be the perfect time to make your move:

  • The noise and cynicism about an overcrowded market has died down
  • The incumbent is increasingly viewed as someone who should be challenged
  • If you do things right (read: better for the users), you will get lots of favorable media coverage
  • You’ll be compared to the leader and your main strengths will be constantly reiterated and inspire many people to engage with your product or help your company out

In short, one great place to look is a market where there are few players, and starting a lean, user-friendly competitor. Just think of the myriad examples in the last 10 years on the web.

Another great place to look is new platforms coming out. Just in the last two months there was the WebOS competition (where you could get a boost), Facebook’s Open Graph, Twitter’s ad platform, and just now, the iPhone 4.0 launch! Each of things brings tons of new opportunities for you to take advantage of:

  • New forms of advertising. You can help develop this new content and get lots of clients.
  • New languages and technologies. If the platform had a very well-publicized launch, and all your friends heard about it, chances are it will be in demand. Learning a new SDK is a barrier to entry, and experience has shown that the first people to cross that barrier are usually handsomely rewarded.
  • Hire or attract developers! If you’re technically minded, this may be a good time to go solo. But there is a budding community around the new platform, and if you engage with it, you can quickly find ways to do profitable things in that space, even if you don’t do the development yourself.
  • While some people are thinking about Business-To-Consumer models, you can do a similar idea and make it Business-To-Business to get a reliable revenue flow, like was done by Scvngr and Yammer.

Pick something you can be really passionate about and something that until now wasn’t possible. If you play things right, others will become interested. When you launch, you can get publicity, and be in a good position to raise venture capital in order to expand and dominate the newly formed market.

And if you ever become the leader, be careful :) Well, at that point you should refer me to your blog.

Spreading The Word

Funny enough, there are a few simple concepts that capture how successful businesses make their money. If you focus on nailing these points in your business model, you should make some serious money (unless something else goes horribly wrong).

Start by considering who is going to use your product or service, how they are connected to each other, and how and why they would deliver your message to others. Let’s see a couple of examples:

  • You (and your employees) prepare high school teens for the SAT. How do high school teens connect to each other, and how will word about your tutoring service spread?
  • Staffing companies post jobs on your job board. Job seekers browse your job board. How does word about your job board spread?
  • You launch a site in 2005 that lets people post videos for others to see. How does word about your site spread?
  • You make a site that lets people send messages to all their followers. How does word about your site spread?

Okay, so let’s say you’re making a dating site. Your target users are singles, presumably. The major challenge with making your site viral is that singles are a “sparse community” — out of all the social connections that your typical user would has, few of them are single. Or even if they are, your typical user doesn’t know which are and which are not.

Or let’s say you’re making an iPhone application. Your target users are iPhone users. Sure, there are 50 million iPhones out there but how does your typical user know which of his contacts has an iPhone?

Let’s contrast this with facebook. It started as a social network for college students in Harvard. That is a much more tightly knit social network. Each typical college student knows a lot of other college students. They go to the same school together. Thus, introducing facebook in the “niche market” of college students is a wise idea, because that niche market is a “dense community”.

The takeaway is that users of your product should come from a tightly knit community, until you reach critical mass. So you should strike partnerships with communities where your potential users congregate. Each community means lots of users and word of mouth about your product. Once a few communities sign up, supporting your case for raising money should be straightforward.

The key to entering these communities is to differentiate yourself from others trying to do the same thing. You can do that in all the ways I described in “Choosing Your Market”. Additionally, you can offer incentives to the community itself, by cross-promoting with it. As long as you can do this in a way that benefits your users, you will have a win-win situation all around. This is a great foundation to start a business.

If we look at popular social products that grew exponentially — ICQ, AIM, Skype, Facebook, Twitter — we find something common to all of them: the product enabled users to communicate in a way that they really wanted, and in order to do that, the other person had to get the product, too. Eventually, if the desire was great enough, whether due to actual usefulness, peer pressure, or just “coolness”, a person would get the product. Then it became a question of user retention. But the most important thing that retains a user is not the product — it’s the way other people use the product to create information that the user eventually sees. That is what is useful to the user. Design that properly, so that people use it in natural ways, and you will have a great product.

Making Money

So what does this have to do with business models and the title of this post? Everything. Just because you make a website, or open a store, doesn’t mean you will become profitable or even gain a local customer base. In order to engage with its customers, a business must first attract them. And the ways to do that involve some combination of the following two things:

  1. Word of mouth
  2. Direct advertising

These two things usually lead to different business models — although you might wind up with a hybrid. Direct advertising means that your business has to spend a certain amount of money to acquire a user — the user acquisition cost. You could take out superbowl ads, or print flyers and hand them out, but at the end of the day you’ll have to advertise yourself on some level, if only to your initial userbase. Word of mouth is the means by which existing users recruit new users to your product or service. This leads to the “viral coefficient” I described in my earlier blog post — the magic number that can reduce your user acquisition cost by 10, 100, 1000 or more.

On the other hand, anytime you are doing advertising, you should get that money back in actual revenue. Otherwise, when you scale up your advertising campaign you will lose a proportional amount of money — which can quickly get prohibitive. The easiest way to guarantee you’ll get the money back is to charge the customers you acquire some amount that will at least cover the cost of acquiring those customers. Another way is to get publicity — by doing something cheap that will get a lot of attention. But if you start mixing in word-of mouth, taking out small buckets of advertising, tracking the viral links, and measuring the response — you will be able to use this data to get bigger and bigger growth spurts for the same amount of money, until eventually, your business is seriously growing.

So if we take a look at dating sites, the reason most of them charge their users a monthly fee is that they are not getting most of their customers through word-of-mouth. And that’s because singles are a “sparse community”. If you wanted to make a dating site that was viral, you’d try to cross-promote with organizers of singles events, and you would be able to undercut pretty much every paid dating site out there. Throw in a couple more features (I’ll leave them to your imagination), and you’ve got yourself a real modern business model for dating. That’s one of our products that will be coming out this fall, actually. Investors, you know what to do :)

The science of scaling

07 October, 20:30, by Gregory Magarshak

Alright, let’s look forward towards the time your business is going to be successful. If your website is like most, you’ll be generating less than $100 a year from the average user. That means your business will depend on scaling up the number of users, in order to pay for its development costs and realize the serious profits.

As you know, some companies (at the time of this writing, Twitter) haven’t even announced a business model that will make them profitable, yet they already have millions of dollars in funding, which they are using to amass millions of loyal users. So clearly, whether or not your business is actually profitable, there’s a point at which it may be very attractive to various investors. I will deal with the business model in a later article. The point I want to make is that if there’s one thing that can give you a good chance of raising money, it’s “traction”. Coming and saying “the site is growing at x% per week and currently has y members. We need money to scale up our servers and build features A, B and C” sounds much better than “I just know this site will become popular once we launch it.”

How to get more members

These days, your site will attract users via a combination of direct advertising and viral marketing. The more targeted the advertising, the better. One of the best deals is pay-per-click advertising where your ad strikes the right balance between weeding out people who would not convert to bona-fide members on your site (by making the text of your ad very specific) and generating such poor click-through rates that it’s buried by google and facebook (because it makes them too little profit).

The rest of your user base will come from viral channels. To find viral channels, look at any social network, such as: facebook, twitter, email, and tightly-knit communities such as colleges. Try to choose viral channels that are not saturated with spam, so people are paying attention to the message they receive from a friend (or someone they signed up to follow, such as a twitter user). Finally, create a viral loop where a user of your site has a strong incentive to link their friends to your site (or something specific within your site) using one of these viral channels, and encourage them to check it out.

There are several types of viral growth:

  • The most powerful is the grass-roots type, where people tell their friends about something. Because there is no hierarchy of popularity required, any receiver of a message can turn around and become a sender.
  • Another type of viral growth is the “PR” type viral growth. This is where your message bloggers, newspapers, media companies and other producers of content. They have user-bases that subscribe to what they say and respect it. One guy with a mailing list can get you 1000 eyeballs. Because it is a hierarchy, there are politics involved, and getting this done well is an art. You should probably hire a PR person for this.
  • Finally, there’s the search-engine based viral growth. I say viral because these days, search engines look at inbound links and other related factors to determine a site’s popularity and relevance. The more people link to your site on their webpages, the higher it ranks on search engines. Here, the network you’re using is the network of internet websites. And the viral channels are the links, embedded in blog articles or what-not.

Optimizing your virality

You can easily tell how quickly your site is growing by looking at the number of users. But there are other metrics that you should probably be measuring as well, in order to optimize your site.

One such metric is the viral coefficient. We’ve all heard the “myth” of exponential growth… that all you need is 1 guy to bring 5, and 5 guys to bring 25, and so forth.  Well, in practice it seems more like a powerful function (e.g. u = t^2 rather than u = 5^t). I’m going to define viral growth in a different, but more useful way. Imagine you attract a user to your site using some means over which you have direct control, such as pay-per-click advertising. You can track this using a parameter embedded in the link (e.g. http://mysite.com/myapp/?adwords=8). This user may send out links (invites) to their friends using some viral channels (such as email, or whatever). These links will contain an id associated with this user (e.g. http://mysite.com/myapp/coolpage?inviter=23498). Whenever someone follows this link, you store this id in their session. If they wind up signing up for your site for the very first time, you give a point to the original user (who came in through your ad) for having brought a new user to the fray.

The viral growth coefficient, then, is the average number of new members that an “original” member (who was signed up after following an ad) brings in through viral efforts on their part. Notice that for this coefficient, it doesn’t matter whether those new members in turn brought other members. The viral growth originating from one member could die out after bringing in a total of 20 people. But that one person brought 20 more without your having to advertise to them.

You can go further and measure other viral metrics if you wish. In my opinion, the one above is the most vital to your business model (which I will discuss in another post). But other possibilities include: the average number of users a user invites and the average conversion rate on each of those invitations. These can’t always be measured — for example, if a person shares a link on facebook, sends out a tweet, posts it to a blog, etc. you don’t really know how many people they have “invited”.

Other metrics you’ll want to measure and optimize:

  • Conversion rate from a visitor (someone who visits your site and is not a bot) to a member. You measure this by starting a session for each visitor, and if they wind up signing up for the first time, you count this as a conversion. If they simply log in, you use that for the following metric:
  • Frequency of logging in. Every time a user visits the site and logs in, you can update their average frequency. Set a minimum coarseness level of, say, 1 day — meaning that if they log in 7 times on Monday, 3 times on Tuesday, and then once on Friday, you will store the frequency as “3 days out of 5″.

These are obviously very important metrics. You’ll need them to increase your site’s audience. The first one directly affects the viral coefficient and therefore your user acquisition cost (UAC). After all, people are only successfully “brought in” through viral means if they actually sign up as members.  The second metric measures how often people return to your site. Each day is a new day the member can spread the word about your site, create content for others, invest their time into it, build a reputation on it, and in general build mindshare among your market. You should reward your members for doing this. You should also reach out to them by sending them notifications and updates.

I’m going to write a separate article on user retention and engagement strategies. I’m a big believer in giving people a good experience and not simply milking their time and attention through some kind of addiction. In other words, it’s not enough to ruthlessly bring people back to your site and make them stay there for a while, but you should strive to make it healthy and enjoyable for them. Internet addiction may or may not be a disorder.

What to watch out for

In my opinion, most businesses fail because they couldn’t survive until they got enough revenue to cover their expenses. This could be because the business was not viable (say, for legal reasons), or it turned out customers just weren’t going to pay as much as the founders anticipated, or because the costs of delivering service to a member were much higher than the founders anticipated, or because the costs of acquiring a member were too high. The saddest one is where each member would have brought in a great net profit, but there simply wasn’t enough initial investment to get over the hump, and the business could not raise more money. It ran out of funding and had to cut corners or fold. (Clearly, there may be other reasons for a businesses to fail, but most failures can be traced back to some form of this.)

Therefore, the first — most fundamental thing that can go wrong is that your business model will require incredible luck to actually make a profit. I’ll talk about science of business models in a future article, but if your business model relies on a long shot, then you’re risking an insane amount from the get-go. Usually this means there is no fall-back plan. You either make it or you spectacularly fail.

On the other hand, there are business models that can survive a lot the sensitivity analysis, and remain profitable for most reasonable values you can throw at the variables. The great thing about websites and viral growth is that you can analyze a “small” sample of people to estimate what the larger population is going to do. You can do market studies to make an educated guess as to how much a typical user is going to generate. Once you build your prototype and your metrics, you can start measuring the viral coefficient and trying to improve it, or user retention, or user conversion. Usually, these are all fixable problems — as long as the business model can remain profitable for a large range of values.

The next thing that can go wrong is that you will run out of money before covering your initial costs (development, legal fees, hiring, etc.) . You will have no money left over for scaling up your servers, for example. As long as you can maintain the existing servers, this shouldn’t be a huge problem either. What you do is temporarily disable invites until you can raise more money. You already have traction — in fact you are fast becoming a victim of your own success. You’ll find many people willing to rescue that kind of damsel in distress — as long as you can prove your business model is empirically working and generating the profits you expected. Still, by cutting off viral growth, you run the risk of changing the culture of your site. If your site depends on spreading the word about stuff, you’re going to get lots of users who arrive but can’t sign up. On the other hand, it might be a blessing in disguise, as in a classic case of reverse psychology, you tell them to enter their email and be notified when you’ll accept more members into your “elite” society.

Perhaps a little ironically, one of the biggest problems you can have is growing too fast. Too many users, or too much content accruing, for you to scale up fast enough. That is what we deal with in the next section, and it is the main focus of this article. If you really ignore this problem in the beginning, you can really paint yourself into a corner, as unfortunately happened with pixelotto.com. The site started slowing down and experiencing a lot of scaling problems due to the traffic it was receiving. It was built in Ruby On Rails, and my guess is — at the time, a single server at a hosting company may have not been enough to handle all the traffic.

Another thing that can go wrong once your site becomes popular is security flaws. If you have 1 million users and people hear all the time about your site, and suddenly it goes DOWN, or starts acting very strange, that is a big problem. That is over 1 million people and their friends saying your site sucks. Now hopefully, they are saying your site’s been great and it’s just acting weird. But the best you can hope for is to restore the site from a backup, to the state it was in 2 days ago (when presumably everything was fine), and then FIX the vulnerability as soon as possible. This is very, very expensive, and time sensitive. If it takes you a month to fix the vulnerability, you’ve alienated over a million people and the word’s going to spread. Worse still, if the hackers were able to steal some kind of important data, you’re never going to be able to live it down. That is why it is absolutely crucial to secure your site while it is still small. The good news is, you just have to develop it in the right way. I’m going to devote an entire article to this alone.

Another thing that can go wrong is that your datacenter is hit by a tornado or a truck. Or more probably, one of your machines suffers a disk failure. That is why it is nice to build on a cloud of interchangeable machines these days, and to back up your database (and images of your app server).

As you can see, most things that can go wrong kick in when you have a lot of users and it’s hard to turn things around. That is why you should do all your thrashing early — as Seth Godin said in his lecture at The 99%. There, he also said that you’re not paid to write code, you are paid to ship. When you run out of money, you ship. Ideally, of course, you should ship before you run out of money. You should launch your site, build metrics, analyze how you can improve them, and then have a period where you may or may not need to raise more money. But

Scaling your back-end

Alright, now we get to the meat of the article. How do you build your website to handle all this scaling in traffic? Sometimes you’ll be growing smoothly and then get a huge spike in traffic from being “slashdotted,” or mentioned on the yahoo front page.

The first thing to know is about is shared-nothing architecture. If your application tier (the web server and your PHP applications, say) has a shared-nothing architecture, each box is self-sufficient and can handle a request as well as any other box. Then you can simply scale up with traffic, even to the point where you’re just bringing up new nodes on Amazon’s elastic compute cloud with your application server’s image (already containing the operating system, web server, etc. ready to go). You can then have a router do load balancing by routing incoming requests among all the web servers. Any code where you don’t store any state, such as the web server or your PHP script, can be put on a shared-nothing box.

The persistence (data model) tier is not going to be shared-nothing. This is where your web application boxes will send queries of their own, in order to retrieve data. If you’ve had to deal with relational databases, you’re probably familiar with the idea of database normalization. Those principles are designed to promote data consistency and free you from having to update duplicate copies of data all over the place. However, they do not promote scaling. To scale a data layer, you will usually apply some variation of horizontal partitioning. This is where data is stored in different partitions (often referred to as shards) which may or may not be on distinct machines. You can still use relational databases, but when the app server makes a query, it is routed to the machine that has all the data that is being queried. Splitting things up for easier querying is an instance of data warehousing.

To partition a table horizontally, first you must pick a field by which to partition. Good candidates include the user id (if it is present) because human users should only be allowed to execute a limited number of actions in a minute (throttling). Similarly you can use the API key if you have an external API. If none of these are present, then consider the primary key as the key to partition on.

Joins become harder to do across partitions. The whole point of partitioning is to spread the load among different machines, so instead of doing a join, you will usually wind up getting a list of IDs and then send out 5-10 separate queries to grab one row for each ID (which may reside on different machines). It’s like you would query the database without using joins at all — this allows scaling smoothly through horizontal partitioning. The one exception is when two tables are sharded by the exact same key, and it is being used as the key for the join. For example, when you get a user and all their photos. If the photos are sharded by the id of the user who uploaded them, their row can be placed on the same machine as the row of the user. So you can still do joins but only between restricted subsets of your tables.

Key-value databases are the easiest to partition horizontally, because the lookup is always happens based on a string key, and there are no joins. The key usually looks something like “tableName_keyValue”. This is how memcache works, and it can scale to huge numbers of machines. Still, you can improve performance even further if you store related items under the same key. For example, when you want to show the profile for a certain user, just store all the information under “user_profile_831551″ or something to that effect.

If you don’t want to deal with all this, you can check out new services like the Google app engine. Because of the way it’s designed, you can write the apps and let it worry about the scaling. Then again, you won’t get a lot of the relational database goodness you’ll have if you use MySQL, etc. Here is an article that makes a good case for it. Currently it only supports Python and Java, though.

Make client-centric apps

There was a time when web browsers had no client side scripting, and everything was sent from the server every time. The page had to be re-rendered every time. Sure, you could cache the results, but you still took a hit in the sheer number of requests (back then it wasn’t usually in the millions), and all that memory having to maintain the cache, as well as the view state, on the server.

These days, many people still start with “dumb” webpages and then add javascript on top of it later. The theory goes that your site should work even without javascript. I know because until recently, I used to code this way too. I’d make everything work without jvascript, and then tried to figure out ways to “ajaxify” the resulting site as much as possible. I even made a framework called PAL to do that (among other things).

Fast-forward to now, and I know that this kind of thing is not only harder (to code, to document, to educate new developers in) but also doesn’t scale nearly as well. Consider two example:

  • Let’s say you have a 5-star rater that uses AJAX. The AJAX response could either a) return the new value for the rater to display, or b) re-render the rater on the server side and return the markup to replace the existing one. Clearly, the second one consumes more resources for both computation and bandwidth.
  • Let’s say you have a form that you’re submitting and no JS support. If the POST request is processed successfully, you’ll redirect to the success page.  If there are errors, you want to re-display the entire page with the form. Now you’ve built all that, and want to add ajax support. Suddenly your form can be smarter: if the server returns errors (using JSON, say) then you display them next to their respective fields. If the server succeeds, it returns just enough information for your component to understand. Most of your previous response logic is no longer needed if there is Javascript.

So I propose writing things the other way: make your components do as much as possible on the client side, like Google encourages with GWT. You don’t have to use GWT, you can use YUI, jQuery UI, or any other javascript UI framework. The point is — make it so that a lot of the view state and interaction is on the client side. Think of the server side as a web service. Your client-side components will consume web services (from twitter, facebook, or your own site). They will be rendered once by the server, and the rest of the time they will update themselves with javascript, possibly through calling webservices. In addition, you might want the components to construct themselves entirely through javascript. This would allow you to build re-usable widgets that others can install on their sites, promoting viral growth through blogs, etc. The only thing you’ll have to worry about then is security (the subject of another article).

Finally, cache as much as you can. Caching rarely hurts. Do it at the browser level, at the webserver level (use a CDN to deliver static content), the app level (when someone is not logged in, show them relatively static data which a server-side cron job updates every few minutes), and the persistence level (memcache in front of your database, to store the results of queries as well as entire profiles).

If you do these things, you should be able to scale. Keep in mind that slow and steady viral growth can actually be a good thing. It gives you time to raise money, to re-architect things correctly, before optimizing your virality metrics and buying your next round of ads.

Oh and I have to mention: check out http://highscalability.com .

Starting a company

06 October, 21:45, by Gregory Magarshak Tags: , , ,

Let’s start at the beginning. No one becomes great on their own. Along the way. you will meet lots of people (which can be very fun) and many of them can help you. But you need to know what you are doing, what kind of help you need, and how to present it to others. You’ll want to open a company.

The company is a separate entity dedicated to undertaking the projects you want to do. It provides a great way to structure things so others can get involved. Once you develop a clear mission and business plan, people will know what they are dealing with. When you raise money (by obtaining credit or selling shares), it will be clear what the deal is. After your company becomes profitable, you are able to sell shares, transfer control to others and exit. If your business fails, your personal life can go on just fine if you do things correctly.

In this blog, I will mostly try to distill what works, so you can take it and use it right away. In my experience, the difference between ongoing success and failure often has to do with focusing on the right things. So when it comes to topics I have figured out, I won’t mince words and will focus only on what you need to know to succeed.

Would you invest into this?

Most entrepreneurs start out with some amount money, time and other resources they can dedicate to the business. The business is an investment — and the founders are its first investors. So think like one. What are you going to get from this business, and how can you manage the risk? If you aren’t willing to invest into this business, don’t expect anyone else to be so generous.

Talk about your business idea with others and seriously consider their feedback. You may indeed have a great idea but might need to present it in a way that makes other people understand why it’s a good opportunity. Through feedback, you will refine both your presentation and the original idea. Before you commit to doing your business, make sure you can explain exactly how it will become profitable. As you get good at this, many people might get excited enough to want to get involved with your company and your project. This is great news for you because it a) shows that you have something valuable, and b) you get to choose who to bring on board based on what you need.

Make a name for yourself

It’s well known that a proven track record counts for a lot in this world. If you built and sold several successful companies, people will be more likely to fund your next venture. Not only that, but your name will be out there, associated with those successes, and everywhere you go, people will respect you.

Therefore, it is a good strategy to achieve your first major success early, and parlay it into other opportunities. You may think your idea is so great that you want to retain as much control of your company as possible. Remember, though, you are already the founder. It is more important that your venture succeed, because then you will be known for having founded a successful company. Chances are, if you came up with this idea, you will come up with more — and even if you don’t (which is unlikely), you can always invest in others who come to you with ideas. And they will love you for it. You’ll get your next ideas funded — and even if they fail, you can still find people who are impressed enough with your original, successful venture, to get involved.

Your first project should maximize the chances of success. You’ll have plenty to play with in your next projects if you succeed.

Chances are, though, it will take a lot for you to quit your job and devote yourself full-time to your company. When you have zero track record and connections, those kinds of resources aren’t raised overnight. So you have three options:

  1. Get a track record and connections by helping launch something successful first.
  2. Find someone who does have these things, get them excited about your project, and get them on board.
  3. Keep working and saving up money and resources, so you can afford to quit your job and devote yourself to the business.

The first two don’t carry nearly as much risk as the third, because at the end of the day, you have not risked everything you had on the business. Although all three let you maximize your chances of success, the first two allow you to minimize your risk.

It takes money to make money

Here is an interesting suggestion to consider: ideas are a dime a dozen compared to a track record of carrying out a successful venture. Perhaps you should consider first doing an idea that is straightforward and succeeds (such as taking advantage of a newly formed market, which I will discuss in a future blog post). It shows you can work with others. It gets you connections. And in the end, your name is associated with success.

Seriously, consider a business where people can make a lot of money, and think about partnering with someone who already had a lot of success in that business. You can get off to a running start in your business career by leveraging the success and goodwill of people who were already successful.

In business, knowing people and being able to sell them on your vision is a crucial skill.

Milestones

Right now your idea may be just a gleam in your eye. Break it down into milestones: opening the company, completing the business plan, getting the people in place that can help you, getting the money in place. Those milestones will take time and effort to accomplish. But once reached, each milestone will unlock further possibilities for you.

There are few things more frustrating than not having enough resources to get basic things done. This includes time and money. For each milestone, make sure you have twice the amount of time and money you think you need to accomplish it. It is much better to say “I will definitely get the prototype completed in less than 6 months” than to say “I will hopefully get the prototype completed in 3 months.” The first one inspires confidence and gives you a guarantee you can work with.

So go ahead, pursue your idea, but do it in milestones — and don’t commit yourself fully to the next milestone until you can set aside double the time and money you think it’ll take to achieve it. At worst you’ll be pleasantly surprised. At best, you’ll find that all the extra time and money let you meet people along the way that you wouldn’t have been able to if you were back in your lab stressing out. People who can seriously help you.

As you reach milestones and get to know more people, your original idea may change. A good business venture should survive lots of these variations without becoming stupid or impossible to continue. Re-evaluate and refine your business plan at each juncture. Your final business may turn out to be very different from your original idea, and it’s okay. It is your first business. The most important thing is that it succeeds.

Naming the company

Lucky Apps started as a company that would make facebook applications. It was originally named during a brainstorming session I had with my girlfriend. The requirements were:

  • Easy to pronounce and very hard to misspell
  • Positive and helpful associations
  • Available as a .com domain name

It’s often hard to find a good domain name that fits these requirements, and harder still to find one that’s composed of actual words. That’s why we have all these sites like Kazaa, Wufoo, Flikr, Bebo, and so forth. Still, these names are short, compared to, say, experts-exchange.com . (Without the dash, it can be mispronounced in a pretty funny way.)

Also consider conflicts in the following areas:

  • Trademarks — you may get sued by a company in the exact same business if they have a very similar name. This happens much more rarely on the web because the courts recognize it’s a crowded space, but still. A trademark normally costs around $300 to obtain if you do it yourself.
  • Search engines — when people search for your company by its exact spelling, what comes up? For example, if your company consists of two words (like Lucky Apps), chances are it will be harder to get results near the top in google. If you are making a software product, similar considerations apply. Ruby On Rails and symfony are a better names, search-engine-wise, than Cake.

Be organized

Finally, get tools that will help you organize your company’s documents, finances, and more. Some things can bring with them a lot of headaches — as I found out the hard way. For example, when your company directly hires employees in New York State, you must get Workers’ Compensation insurance, unemployment insurance, and report your income four times a year. Be aware of the law (at the city, state and federal level) when you do something — you can often get the information via free consultations with lawyers and recruiters.

Products that can be very useful include a calendar/organizer, project management software, budgeting software, collaboration software and a revision control system for all your important files. They allow you to work with your information effectively, make the right connections, realize what’s actually happening, and get things done. Good tools are able to tremendously reduce one of the major sources of stress, namely the necessity to keep ad hoc information in your head rather than having easy access to it.

The other major source of stress is regret: you expect something to happen, and it doesn’t happen. That’s why my advice is to go easy on yourself — double the time and other resources you are willing to commit to a project. Make your expectations of yourself be lenient — assume there may be unforseen obstacles along the way. When it comes to others, though, make your expectations of them reasonable and firm. Once they are on board, it is their responsibility to manage your expectations if something comes up. But that is a topic for another time.