Creative Ideas.

Look at where we are headed, in terms of the internet, what we pay attention to, where we spend our time, and how we communicate. The world is heading towards increased openness of information, trust in large companies, and increased addiction to real time updates. The web is moving from computer browsers to cell phones and even gaming consoles are getting involved. It’s new, terrifying and exciting all at once.

To see friend of mine told me today that he can view most of his browsing history on Google, with analysis of his past trends and a comparison to the rest of the world. That’s because browsers allow any website to see the contents of your address bar for every other window and tab, so as long as you have Google open somewhere, it can gather statistics on you. It’s reassuring that Google’s unofficial motto is “don’t be evil”, but sometimes it’s unnerving to know just how much information they have on you.

Search engines have been around for years. There’s a new juggernaut in town, and that is the power of the network effect: the more people use a service, the more others are compelled to use it. What might be a little scarier is the power of the network effect in the hands of proprietary companies.

I am intimately familiar with these issues, since I’ve been studying what makes viral phenomena work. One of the main factors is user perception. Many years ago there was a site called “whimit.com –  Russians Online”. It was a social network — perhaps way ahead of its time. It was free to the users and made around $100,000 a year in its heyday. When facebook came along, it did things slightly differently — it was visually clean, and for college students. It also kept adding features, like the news feed. When whimit users realized that most of their non-Russian friends were on facebook, they migrated over.

I met some tourists on the train the other day. The man was from England, the woman from Spain. At the end we wanted to keep in touch, so I asked them both if they had facebook, and we exchanged names. Why ask their numbers? How quaint. Almost “everyone” has facebook now. And it seems much easier to ask their “facebook” — which happens to be their full name — and soon see all the friends photos notes and other items they let people see online.

It is interesting to ask what sites or companies will be the dominant players in 5 or 10 years. When we looked at products, it seemed pretty realistic that new, better products could come along and people would migrate over. If a better search engine came out, we could just use it. However, with the new reliance we have on social websites and our email box, it is hard to imagine moving to anything else. When all your friends are on facebook, how can you leave? The more a site becomes a part of our lives, the more stuck we get in . The web really starts to get sticky.

Not too long ago, I read an interesting viewpoint: Facebook calls itself a social utility that connects you with friends. In fact, it is becoming a utility, just like your water, electricity, or telephone. It already does more than your telephone. Consider this: even if facebook never figured out how to turn a profit, and kept losing 100 million dollars every year, investors would still fund it year after year. Because it is too big to fail. Imagine one day waking up and going to check your facebook and seeing “we’re sorry, but we have run out of funding. Facebook no longer exists. Thank you for having taken part!” The fact is, the more invested we are in something, the harder it is to leave. Similarly for marketers and new startups. There are very few places where so many people signed up with their real name, age, sex and actively update their friend connections. (Google and Yahoo come to mind, but facebook’s the only one where people are used to constantly giving out their real name.)

The fact is, facebook is fast becoming too big to fail. Google is very very big and does a lot of things right. But if Google suddenly ceased, companies buying adwords and publishers getting revenue from adsense would be hurt, but the average user could, say, go use Yahoo. If facebook ceased to exist one day, there would be a revolt. And facebook’s user base is already larger than most countries…

The utility bill is pretty high for these new sites, and goes up the more real-time, media-heavy, and popular it is. Twitter reached a deal with many cellphone carriers to be able to multicast text messages. Facebook stores each and every photo uploaded, indefinitely — even if you click “delete” on the site. That alone is terabytes of new information to store every day. Youtube’s burn rate in 2007 was around to $1 million a month, and is probably higher now. Online video and content delivery is growing, and has sparked debates such as the one about net neutrality and who should foot the bill for all that bandwidth being used. Since buying Youtube, Google may be losing money with all those bills (copyright-related lawsuits don’t seem to be a big problem for it). But youtube is a cultural tool now, with presidential debates being aired on it. It seems our future internet will continue consist of free services with huge numbers of users — even if they lose money — supported by advertising, targeted and tailored to our browsing habits as much as possible.

Finally, let’s consider a potential implication of all this. Suppose facebook simply decides to turn a profit by charging its users for use of the site. One day you wake up and find on your home page something about a new Terms of service:

“Facebook has always strived to provide its users with up-to-date information about their friends. In order to continue providing services to our 800 million users, we can no longer keep all the features free.”

Can you afford to opt out of this “social utility” when all your friends are making updates on it? Let’s face it — the more useful a service is, the more invested you are in it — the more reluctant you’d be to stop using it. Even if it means having to pay. Yes, facebook could present an option for you to pay $1 a month for it, and later jack up the price to $10 a month once everyone gets used to the idea. After all, many of us have already given them our credit card info in order to send virtual gifts to a friend; and many more will enter it as part of the new Facebook Payments feature. What’s $1 a month, anyway? Okay, later they might raise the prices like the MTA (subways and buses in New York City). Then it really will be the social utility.

Sure, lots of users may revolt and form groups like “Stop Facebook From Charging”… they may even become 1 or 2 million strong. But they will probably be found among all the other groups, like “Don’t Let The Democrats Rule Us All”, and will be too poorly organized to stop the other 230,000,000 users from continuing to use the site at $1/month. Whoever wields the network effect can build a powerful web, indeed.

And maybe in the end, isn’t the internet becoming a public service we are all paying for anyway, in one way or another?

Posted on June 5th, 2009 by admin
Filed under: Uncategorized | No Comments »

When you’re designing a site that becomes popular, you have to be prepared. Malicious users can write bots that will attack your site in various ways. Most common attacks involve making requests to the same URLs repeatedly, with different query parameters. These URLs may send out expensive requests to other servers, or otherwise overload your server. So how can we protect against this?

Well, one way is to require authentication. For example, a user cannot do anything major, such as add database records or files on your server, without being logged in. Or an external application cannot call your API without using an API key. In this case, you can throttle the requests using a proxy sitting between your server and the requests which are coming in. This way, for example, the holder of a particular API key can only make 10 requests a second, or 1 million requests a month, max.

Of course, authentication is only good as long as it does its job. If the users can create thousands of accounts, then your authentication needs to be better. Use two-factor authentication, or better yet, two-factor authentication with some kind of physical “dongle” to tie an account to, such as a cellphone. This is a whole other topic. But even if your authentication gets compromised, it’s better to require authentication to secure those requests that can overload your server in large numbers. You can always “ban” those users / api keys.

There are also things you can do if the user is not authenticated. First of all, require them to have a temporary session. (Use cookies — if users don’t have cookies enabled for your site, you can keep appending the session ID to the URL — PAL does this automatically.) New sessions are easy enough to obtain, though, unlike full-blown accounts on a site. The user might even vary their IP dynamically, and testing for abuse is really hard — there might be a whole company using your site through a single proxy. What you would like to do is to enforce that every form is submitted by a human being. That way people can’t have bots that keep simulating the form submissions, and overloading your server.

One way to do that is form signing. When you generate a form,  store a random (high-entropy) id inside the session. Output this id as a hidden field in the form. The form handler script will check for the presence of the id it’s given inside the session. If it’s not there, the handler script should complain to the user, and log this incident in the security logs (along with the IP, etc.) That will ensure at least that requests to the handler are not being issued directly, but through the form. You can also be sneaky and store the current time and browser used in the session. If the browser suddenly changes, the security bells should ring too.

This does not tell computers and humans apart, but only ensures the action is being executed because a form has actually been requested and submitted. (You will also need to make sure the browser does not cache the form, by judiciously using ETags, etc.)

On top of this, to really tell computers and humans apart, use some kind of CAPTCHA in the form.* There are lots of good ones, including reCaptcha.net, using which you can secure your site and help digitize books at the same time

So, to sum it up:
1) Use authentication and throttling, to be able to deal with crises easily.
2) For users who are not authenticated, use CAPTCHAs and form signing, to ensure requests have been submitted by humans through forms, and to be alerted to potential problems before they happen.

* One simple but overlooked form of captcha involves having a visible, offscreen field that should NOT get filled out. This type of captcha is no good for our purposes, because it protects from bots trying to randomly submit forms on lots of sites. Here we want to secure against determined attackers who KNOW and USE our site.

Posted on January 26th, 2009 by admin
Filed under: Uncategorized | No Comments »

The more we design social applications here at luckyapps, the more experience we get with the various elements. At some point, the approach becomes straightforward, almost formulaic.

If you are designing a database for your next website (or, more generally, your social backend which can power multiple interfaces), I would like to offer some guidelines based on what has worked for us.

First, some basic concepts:

• Database schema - the structure of the tables, as well as the code of the stored procedures, triggers etc.
• Primary key - a unique index on one or more fields, by which records are typically found.
• Foreign key - a set of fields, which is not the primary key, which match the primary key in another table, i.e. their values for a row are used to look up the related row in the foreign table.
• Extending a table - if two tables have the same primary key (same fields), then one table can be considered an extension of the other, sort of like the class “extends” keyword in Java or PHP.
• One-to-many relation - if a table A has a foreign key into another table C, then there can be any number of A rows corresponding to a row of C.
• Binary relation - if a table B has foreign keys into two tables A and C, then we say that A and C have a many-to-many relation and that B is a binary relation table. For example, an author can write many articles, and an article can have several authors.
• Relation table - a table can have foreign keys into more than two other tables. If it has foreign keys into N tables, then we call it an N-ary relation table.

Here are the steps to creating your database:

1. Start off by making the tables that represent the basic objects you will definitely need. Make sure you have third normal form. Group tables into separate databases. This will make it easier for you to scale your site later. One consideration for your partition, though, is that you should never have to join across databases. This won’t work, as far as I know, with MySQL when the databases are on different machines.
2. Create fields for the primary keys of each table. Think about how you will find items in this table.
3. For each table, add the fields each object should definitely have. Pay attention to whether they should support NULL or not. Mark unsigned integers as unsigned. Make booleans into enum — you might extend them later. These fields should pretty much satisfy all the conditions of Third Normal Form.
4. Think about what users would ever insert into your database. If something can be added by a user, it belongs in a database row. This may turn some enums into foreign keys.
5. After this, there will also be fields which may be relevant for some rows but not others. If you will need to search by this field (i.e. create an index on it), then create a field. Otherwise, create an extra_json field of type TEXT, and store all the other stuff in there. The other way to solve this problem is by extending the database row.
6. Now, add indexes to whatever fields you will definitely need to search by. (Primary keys are already indexed.) You can always add more indexes later. Make these indexes BTREE indexes, unless you don’t ever need sorting and you will select a lot of rows, in which case you can use HASH.
7. Since inserts and deletes happen more rarely than selects, precompute things like row counts. That means, if a table A has a foreign table into B, then B should have an “A_count” field. Add two triggers: one for insert on A, and one for delete on A, which update B.
8. For voting or other crowdsourcing activities, create a table of things from which users can choose (e.g. “tag” table), a table of things to vote for (e.g. “tag_on_item” table), a table of votes (”tag_on_item_vote”, which would have tag_id, on_item_id, by_user_id). When users vote for a row, delete their previous votes for that row before inserting the new one.
9. For ranking things, we developed a very nice algorithm that doesn’t tie up the server, and lets ranking of millions of users happen incrementally in the background. It’s just one of the functions in our framework, PAL.

So, if you follow this outline, you should have a very nice database. As you need to scale it up, you should start using memcached to take the load of all those SELECT statements off the database. Also, horizontally partition your tables onto multiple machines by using MySQL proxy.

Posted on January 14th, 2009 by admin
Filed under: Uncategorized | No Comments »

This is the latest in our semi-professional advice series As luckyapps meets with more clients and completes more projects for them, we accumulate lessons which we want to share with other entrepreneurs, because they are useful for all of us.

This time, we look at sellers of online products such as e-books, and we take a page out of their playbook (pun intended). You have a product or service that your customers must pay for. When visitors arrive at the product’s site, or when your potential clients meet with you for a consultation, you will typically share something about the product or service they will get once the money is paid and they are your customer.

Front-load your value proposition.

Many people feel a bit secretive at this point and don’t want to share the best information until the customer decides they want to pay. It is commendable that they want to give their paying customers something of proportionally greater value, but it does not have to be the information. At this point, your goal is to make the visitor want to buy the book, or the group of people you are meeting with want to become your paying customers. This is the time to be totally open with your ideas and gladly share some of your best material. This is especially important if you have a lot of competitors who are offering similar products or services.

The customer is right there, in dialogue with you. Impress them with the quality of what you offer them during that opportunity. They will want to see more. If you are selling an e-book, offer its two best chapters for free. Better to do that than to offer two average chapters, with a promise that the best is yet to come if they only buy.

In the end, when the client spends their money and you spend your time and effort fulfilling the order or contract, what has really been established is a relationship. Make it a good one.

Posted on January 6th, 2009 by admin
Filed under: Uncategorized | No Comments »

After years of organizing the NY Tech meetup together, Scott Heiferman and Dawn Barber are looking for someone to step in and take the reins. The new organizer will have to solve many issues, including the prospect of a larger venue for the growing group, corporate sponsorship, and increasing the opportunities for its members. These are the issues I’m considering, since I volunteered to run for the next organizer (along with 10 other people). I’m going to be presenting on Tuesday. Wish me luck!

Posted on December 6th, 2008 by admin
Filed under: Uncategorized | No Comments »

… since the last blog entry.

I’ve finished PAL with its plugin system, and built many projects on top of it, including:

• http://apps.facebook.com/editphotos/ <– proof of concept for Photo Artist, using our new viral photo technology
• A facebook application for MAC cosmetics
• A Flash application for Bluefly
• and more.

This month, luckyapps is working on:

• Holiday applications for facebook, and several non-profit organizations
• An improved gifting engine for our clients
• A casual network game engine is in the works. More about that later.

Happy holidays!

Posted on December 6th, 2008 by admin
Filed under: Uncategorized | No Comments »

I’ve now gotten my framework Pal to the point where it is in very good shape. I’ve already made several projects with it, such as this one. I decided to think seriously about adding a robust plugin system, that would let me re-use and upgrade plugins between projects, and facilitate the development process.

Architecture diagram:

Development process diagram:

As people produce more functionality on top of Pal, it should consistently and neatly fit the two diagrams above. This lets development happen smoothly, and allows functionality to be re-used between projects.

So right now I’ve just got through designing the basics of the plugin system for Pal. It makes the following possible:

• Plugins can be developed and used across multiple projects. The user of a plugin will be able to decide whether particular instances of a plugin share their databases/model or not.
• Plugins will be versioned, with several versions coexisting side-by-side. Upgrading a plugin will mean installing a new version.
• Keeping track of dependencies between plugins and their versions.
• Separate projects will be able to use separate versions of a plugin. Installation and upgrading will involve importing the web assets of the plugin, executing .sql files to upgrade the schema, and .php scripts to do anything else automatically.
• A control panel (basically a Pal project) for managing Pal projects, plugins, and modules in a consistent way. It will allow adding, removing, exploring and changing properties of many different items in a consistent way.
• Integration with automated testing, with scripts to handle setup and teardown for staging.

My current design does all that. Now it’s time to start implementing it

Posted on May 9th, 2008 by admin
Filed under: Uncategorized | No Comments »

Working on facebook applications in 2007 has taught me a lot about the web in general. The landscape has changed. In some ways, making successful apps on the web has now become a bit of a science. Here are three key insights about how a successful project should typically be designed:

Business Aspects

A very successful web application should achieve the following three objectives:
1) It should be viral - people would help spread it themselves
2) It should be sticky - people would enjoy coming back and using it over and over
3) It should monetize - usage of your application should translate into revenues of some kind.

There are many ways to achieve each one of the three things. Almost all applications on facebook aim for the first two steps — 1 and 2, but most do not seem to have seriously been designed for 3). That is why, when they are at 200,000 users a day, they are still not making a lot of money. But are they worth a lot?

Actually, having just 1) and 2) is good, too. Imagine if your first project resulted in stable traffic of 200,000 visitors per day with no sustaining effort on your part. That is 200,000 people (many of them returning visitors) every day who enjoy using your web application for one reason or another. Now, if you implemented a project that really helps the user and monetizes, you could drive this traffic toward that project. Many useful products don’t spread virally at all. Consider Adobe Photoshop. Services on average spread more virally than products (because you can usually somehow involve your friends) but here’s the thing:

If you took an existing viral and sticky project and promoted a monetizing project, you’d have a sustainable money making machine.

Keep in mind, however, that there is a limit to this. Those 200,000 visitors are most likely mostly returning visitors, so you might have at most 1-5 million TOTAL visitors. That means you’re not going to be making a billion dollars from making a one-shot-wonder like a software program and selling it to your users. However, you should keep the momentum going. Each one of your products and services should continue the good experience for the user and provide opportunities for you to later introduce something new to your users.

Of course, the best projects achieve all three — viral, sticky and monetize — starting from their very conception. That way people start coming, staying and spending money from the very beginning. If I was a VC, I’d strongly look for the actual mechanisms that would achieve 1, 2, and 3, and make sure they are all viable and have a good chance of working. These are the main factors these days, that can make or break your product. (Sure, there are completely different ways to market, such as SEO and Google Adwords, but I am talking about startups, exponential growth and major possibilities.)

If you apply this “successful combination” analysis to previous products, you can see its power in determining how things play out. For example, Whimit - Russians Online (and still is) was a social networking site that launched way before facebook.

Viralality comparison: One of the major problems was, it was limited to Russians. So, the initial growth in the Russian communities was explosive. But because of this limitation, it didn’t become much more viral than that. At some point, it had around 3 million members. It lost most of its visitors to facebook because people had lots of non-russian friends, and started to get pulled away to this new site.  TheFacebook, on the other hand, started in Harvard and got half of the student population using it within a month. It’s kind of funny that it started as just a follow up to a hotornot clone. The hotornot clone was done first, but what people really always wanted to do was connect with each other and share pictures and express themselves. Especially students! (They already had yearbooks, so it was an easy concept to understand.) I believe facebook would have taken off just as much if it had started at NYU or Berkeley. (See myyearbook.com) Of course, it helped that it had an enterprising founder and smart guys taking it to the next levels.

Stickiness comparison: And in thefacebook.com, for the first time, they had a site that did all these things, and rapidly improved and gave them more and more features. Whimit didn’t. It was pretty static and — although it had unique features, like polls — it didn’thave many privacy controls, searching, events, discussion forums, modularity… facebook was just built smarter and eventually offered a better user experience. This is another reason why whimit lost to facebook. Facebook was way more sticky.

Monetization: Okay, whimit had already lost. But it didn’t help that users’ profiles were spammed with promoters advertising clubs. Facebook keeps a tight lid on spam, and its advertisements started with low-key “flyers” in the bottom left of a page. Anyone could post a flyer, and it was pretty democratic. I think facebook could make way more revenue than it currently does, but this already generates at least ten million dollars a month in revenue, thanks to their 40+ million users.

The number of your friends on a site also contributes very mcuh to its stickiness. If more of your friends are doing actions on a particular site, you feel your drive for social interaction being satisfied better through that site.

How To Be Viral?

Here’s the short answer:

• Help your users invite their friends through viral channels - email, AIM, word of mouth, print media, facebook invites.
• Give them a compelling reason to spread word about your product. This can be done artificially — by giving them some kind of contrived points they can spend in your app, or intrinsically — because they would benefit from the network effect of having more of their friends using the app. Both are good and there is no clear line where “intrinsic benefit” and “contrived benefit” begin and end.

The problem, though is no matter how good your application is, it will have an extremely hard time growing virally through channels which are clogged/ignored by users. This can be seen currently in the facebook application platform. Because applications many hijacked the crowd mentality and some abused things outright (such as forcing invites), the invite channel got comlpetely spammed, and many people lost faith in applications that even their friends recommended. No matter how good and sticky your application is, if it sends an invite to a user with a lot of friends (and those are the ones you want to reach most), it might wind up 537th out of 839 invites. What are the chances anyone will care?

Facebook could have avoided this by letting the users control the exponential factor of growth for applications. users could rate an application on its about page, and if the rating was unfavorable, the maximum number of daily invites would be lowered for it; and if everyone loved the app, it should be able to spread by even 100 invites a day. This simple system would have kept the invites channel full of mostly relevant stuff, proportional to how many other users liked this app.

As it is, facebook implemented a stopgap measure based on how many people reject your invitations. Guess what! The people most qualified to decide  whether an application is great or not are the people who have used it. Their ratings should affect the maximum rate of growth of the app, not the responses of the poor friends who are still being deluged by invites.

Another thing that would have been great is to have a button that says “ignore future invites from this application”, with an option to unblock the app later. Facebook may have that — I’m not entirely sure — but it’s not prominent. Instead, they have a big button that lets you delete all your invites. Hardly helpful for restoring faith in that viral channel.

In short — at this time, the invite channel on facebook is clogged, so you should be using other means to spread your social apps.

Architecture

Finally, let’s realize that not all web apps much reside solely on facebook. Instead, let’s leave our options open, and grow our own businesses. The web used to be just web sites. You can think of them as standalone programs. You could embed widgets into any website, but the widget didn’t really have much personalized information, since it didn’t really know who you were very well. Nowadays, social networks like facebook, and other websites that log in their users, have enabled you to write applications on top of them that actually know a lot about the person and even their friends. So the new web architecture is becoming something like this:

BACKEND + INTERFACES

Interfaces include your web site, facebook apps, myspace apps, and widgets embeddable in websites like blogs. Interfaces can be personalizable or generic. On a site where you know who the logged-in user is, you can display information from your service relevant to them (such as what their friends are saying). On a site where you don’t know who the logged-in user is, you can display the most recent information, to tantalize them into clicking a link to your site, or remaining in the widget and logging in through there.

In short, build all your backends to be independent of facebook. That means, don’t store the user id as just “facebook uid”. This goes for all other objects. Anticipate you may have users coming from elsewhere, and build accordingly!

Posted on March 28th, 2008 by admin
Filed under: Uncategorized | No Comments »

“Hey, dude. What are you doing tonight?” “You wanna do something?” “Where can we go…” “well, I know this nice place…” “Want to invite some girls?” “Nah, not tonight, man.”

This happens all the time. People want to get together. They want to do stuff. Sometimes, they’re busy and they need to stay in. Other times, they want to see a movie, they want to check out a new place, they want to meet new people, they want to be social.

FreeMeet — the social calendar solves all those problems.

I’ve been bouncing around lots of ideas, lots of programming, lots of time and resources. I’ve tried a lot and did yet not have one resounding success. Since my last post about becoming focused, I have become more so. Now, when I’m putting together a business plan for freemeet, I realized just how wonderful that service can be — it can improve not just my life, but the lives of everyone around me. I can actually build a business that makes life better for a lot of people. They say a great business addresses a pain that the customer feels. Well, lonely people, social people, guys, girls, and those who feel that “pain” I have felt for the past several years — some times more than others — I can help you.

To accomplish this, I will focus on freemeet, more than anything else at this time.

Enough shedding tears about the past. If you’re gonna feel something, enjoy that you’re feeling.

I’m really going to enjoy this business.

On March 25th, FreeMeet is born.

Posted on March 24th, 2008 by admin
Filed under: Uncategorized | 1 Comment »

Here is another one of my creative ideas, that I wanted to do for quite a while. Personally, I don’t have the time to make it right now. But I would really like to see it done.

So as with my other ideas, if you’re going to do it, let me know. Cause I’d like to take part in some small way

How do you prevent your sites from being overrun by automated spambots? One way is to let only logged-in users affect the databases. But at some point people have to sign up, right? And maybe you might want visitors to try things out before making an account. For all this, there was a quick method to weed out the bots: CAPTCHA.

CAPTCHA stands for Completely Automated Public Turing test to tell Computers and Humans Apart (whew). Captchas started out over ten years ago and are still being used today. We come across them from time to time on the internet:

If a form is being submitted, someone’s got to type in the letters. The idea is, a bot can’t do it. That’s because the search space of all possible solutions is too big to randomly guess, and solving the problem, and because solving the problem is too hard for a computer program.

Well, the news is out that the major CAPTCHAs have all been cracked. Jeff Attwood made a great blog post about it recently, and you can find the story about the latest crack here. That’s GMail being cracked. This doesn’t mean your passwords are stolen, but it does mean spammers can make thousands of GMail accounts to play with. And GMail.com isn’t unlikely to be blacklisted.

If you make your own CAPTCHA, it is unlikely someone is going to crack that particular one. Even if it’s just an slightly distorted image of a word, and you expect them to enter the word ‘blah’. That’s because it’s in the spammers’ interest to crack something once and gain a lot of benefits. They are very unlikely to make a bot that can crack crack every single CAPTCHA on the internet, but they will definitely focus on Google and Yahoo.

But because these CAPTCHAs at google and yahoo are all automatically generated (that’s the CA part), these people have figured out how to get back the text they started with, with impressive accuracy: 20% in google’s case.

There are, of course, much more secure ways to make sure people don’t abuse the system. You can, for example, use two-factor authentication, and simply verify the user’s email or cellphone number (typically people do not have more than 1 or 2 cellphones, so verifying a cellphone is a great way to make sure a user has only 1 account — although it does rule out some people who don’t have one).

Here Comes Paptcha

So here is a slight variation on CAPTCHA that makes a lot of difference:

Let people submit photos with text in them. The letters in the text, once submitted, can be mixed together to make words. Or words can be mixed to make sentences.

The best OCR software is still rather rather bad with real-life situations. Text is skewed, printed in many different fonts, distorted a bit, and so forth. Here are some examples below:

Street signs:

T-shirts:

Coffee Mugs:

But perhaps computers will be able to read these. So an even easier thing would be to combine these pictures with a question. The space of solutions is big, and the questions are pretty intractable.

Change the C to a P and it’s called PAPTCHA — Partially Automated Public Test to tell Computers and Humans Apart. Whew!

But how would it work?

Here’s how:

• A fun site where people would submit thousands of new PAPTCHAs every day
• We’d make sure they are people by two-factor authentication, or by existing PAPTCHAs.
• An API that would allow sites to place these PAPTCHA challenges on their forms
• New PAPTCHAs would be tested. Unlike CAPTCHAs, PAPTCHAs might have more than one popular answer. Viewers would be presented with one or two heavily tested PAPTCHA (tested means that there were at least 1000 attempts, say, and 1 or 2 answers which are by far most common), which would be used to prove they are a human. And alongside the tested PAPTCHAs, they would get one PAPTCHA that is being tested.
• But they wouldn’t know which is which. The PAPTCHAs would all be strung together into one sentence or word. The challenge would be considered solved if the tested PAPTCHA subchallenges were all completed successfully.
• We could use some bayesian estimation to determine which answers are correct and when a PAPTCHA is sufficiently tested to be used. We would probably end up with a linear combination of some sort — I haven’t done any mathematical analysis on this yet.

Of course, these PAPTCHAs, like CAPTCHAs, will need to be implemented carefully. There are other security threats, such as session hijacking which sites need to guard against in order to be as secure as possible. But PAPTCHAs will prevent bots from doing much of anything, until of course hackers come up with ways of reading even 1% of all text from different kinds of photos in the world. In that case, they might as well get some venture capital

Let’s Build It!

This is one of the upcoming battles of Humans vs Machines. It should be fun to watch. This project has a lot of practical benefits, so if you want to make it, contact me.

Why contact me?

Because I registered paptcha.com

Posted on March 18th, 2008 by admin
Filed under: Uncategorized | No Comments »