Friendly Bit - Web development blog

Partial XMLHttpRequest responses?

2012-01-08T15:28:49+01:00

We all know how to make an AJAX request, and fetch some data. But as soon as you need to fetch data incrementally, have the server push data to you, you have to resort to all sorts of complicated stuff. Websockets; with all their different versions and shady support, different kinds of polling, hidden iframes, ActiveX for IE?

The simplest way, that almost workds is partial XMLHttpRequest responses. I first read about them as progressive xmlhttprequests on Kyle Schulz blog, but really think that method should get more recognition.

Note: I've only tested this with Webkit, against Twitter's Streaming API, with a XMLHttpRequest that allows cross-domain requests. I think it works with Firefox too, but it will definitely not work in IE. Sorry.

var xhr = new XMLHttpRequest();
var url = "<streaming-url-on-you-own-domain-or-CORS>";
xhr.open("GET", url, true);
xhr.send();
// Define a method to parse the partial response chunk by chunk
var last_index = 0;
function parse() {
    var curr_index = xhr.responseText.length;
    if (last_index == curr_index) return; // No new data
    var s = xhr.responseText.substring(last_index, curr_index);
    last_index = curr_index;
    console.log(s);
}
// Check for new content every 5 seconds
var interval = setInterval(parse, 5000);
// Abort after 25 seconds
setTimeout(function() {
    clearInterval(interval);
    parse();
    xhr.abort();
}, 25000);

The biggest problem with this method is that the responseText property keeps filling up with data. The longer you receive data, the bigger the data in memory will be. The only way I can see this fixed (today) is to simply kill the connection after a certain amount of data has been received, and open it up again.

I would love to see a better way to do this, from native javascript, without all the numerous hacks that are out there. If you know of a way that fills these requirements, please let me know:

Easy to implement on the client side. Ideally I would like to use XMLHttpRequest, and just get a callback each time the client sends data, with the NEW data specified as a callback parameter.
Easy to implement on the server-side. I can set some headers if you make me, but ideally I would like to use this against existing Streaming APIs (like Twitter's), without adding custom stuff.
As cross-browser, cross-platform as possible.

Is there a way to get this working? It's so annoying to see something that's a curl one-liner, be 100s of lines of code with web technologies…

curl https://stream.twitter.com/1/statuses/filter.json?track=<your-keyword> -u <your-twitter-nick>

Is the web really that far behind?

Geolocation and Google maps

2011-06-18T14:11:11+02:00

Image by heiwa4126 via Flickr

Google Maps has has geolocation support for a long time, but I still find people surprised of how it all works. So here's a short writeup, skip it if you already know all about geolocation.

Lets start at the Google Maps frontpage. Among the zoom controls, above the little old man, there's a button in the form of a small circle. It does not look like much, but what it's doing is incredibly cool. Click it and the map moves to where you are, with an accuracy of ~20 meters. Sci-fi crazy!

How does it work? In the newer browsers (all but IE6, IE7, or IE8) may ask you for your positioning information from the browser. It usually shows up as a bar at the top of the browser. The browser then gathers two specific forms of positioning information from your computer: your IP address and the signal strength of any wireless network near you. That information is then sent, if you approve it, to Google, which returns the coordinates you are at the moment.

Google can do this because they've been driving around with their street view cars all over the world, and measured signal strength of WiFi networks (public and private) at each specific location. Now they can use that information, combined with your local signal strength conditions, to triangulate their way to where you are. And it's both accurate and fast.

If your wireless reciever is turned off, or you're at a stationary computer, all calculations are based on the IP number. These kind of lookups are quite arbitrary and inaccurate, I just get to the nearest big city when trying to use it over a non-wireless line. But mobile connections are slowly taking over landlines, so I guess this problem will solve itself automatically.

The best kind of technology throws you forward in time, and make your realize that we're actually making progress. Geolocation has feel to it.

Looking for the techy explaination of geolocation? Have a look at Robert Nyman's writeup.

Will newspapers die?

2011-02-09T00:41:17+01:00

A journalism student in Toronto, Canada, asked me some questions via e-mail about my old article about newspapers and online reading. Instead of just sending an e-mail out in the void I thought I'd persist my answers here:

Do you think it would be best for newspaper publications to get rid of their print and go strictly online only? Why or why not?

I think they should keep their print papers for a while longer. It's a simple question of profit and catering to what users want. If you have thousands of users that want their paper, and it's economically justifiable, just keep it.

The tricky part is determining how much loss in paper subscribers you should tolerate before shutting down the huge printing presses. I have no good answer to that other than to know your numbers. Paper profit needs to cover paper printing and distribution. Journalism costs can be split between channels. Do the math.

Many people believe newspapers are dying and eventually will no longer exist. What are your thoughts on this?

I think they will persist, just not their paper versions. Many of them are looking at dwindling subscription numbers, year after year, and are trying to figure out what to do. How do you move to the digital worlds while keeping quality, relevance, and profit.

The iPad is one way to get people to pay for news again, and some newspapers are seeing success doing that. Over a longer haul just the iPad won't be enough, you need to be able to deliver your news to wherever your customers want them. Generally, this means being able to get your content on all kinds of devices, both mobile and desktop. The only format that can really do that, and still deliver a somewhat consistent design across devices, is HTML and CSS. Building one native app for each platform is just not worth it.

So it's simple really: newspapers that figure out how to transition to digital (while still getting paid enough to afford quality journalism) will persist, all others will (slowly) die. This means an end to both paper-only newspapers, and those that fail to find the right payment model.

My iPad – a short review

2010-04-25T20:33:32+02:00

About two weeks ago, I got my hands on an iPad. For those of you who have been living under a rock for the last months, and iPad is something that looks like a big iPhone, but behaves much like a small laptop.

Since people who just spent over $500 for a toy, are very subjective in their judging (buying something bad would make you look stupid!), you have to take this review for what it is: Me trying to justify buying an expensive toy. To help my mind balance things a bit, I'm going to talk about two good things, and two bad things about the iPad.

Good: Looks, UI and surfing the web#

[](/files/post-media/ipad_and_banana.jpg)

An iPad, here shown next to a old banana for comparison. Click for bigger image.

The area where the iPad really shines is when it comes to the look and feel. It's simply gorgeous, feels good in your hand, and makes all your friends instantly drool (like you expect them to). The screen is as sharp (or sharper) than a computer monitor, and its brightness could light up a dark room.

Working with it is just as pleasant. You click, swipe, pinch and type, like you've done nothing nothing else in your life. As someone who haven't used an iPhone before, I'm impressed. Those that have, say that the iPad is even snappier than the iPhone. Some application are of course better fit for a touch interface than others. Google Maps is one of them, when a large display combined with panning and zooming makes the old arrow and zoom buttons on the traditional Google Maps interface feel like stone-age.

They keyboard surprised me by being even better than I expected. In landscape mode it's almost as large as a laptop keyboard, and in some cases even faster than a traditional keyboard since you don't have to push, just touch the buttons. The biggest keyboard problem though is of course that you usually only have one hand available. When you have two hands, it's great.

Another area where the iPad really works is as a window to the web. Safari comes pre-installed (Microsoft monopoly anyone?) but works like a charm. Pages load quickly, and swiping to scroll and pinch to zoom works just like it should.

Web standards are well supported, like you expect from Safari. But some new problems emerge when looking at HTML5 demos though:

The Safari inside the iPad isn't as performant as a desktop browser. So there's lots of lagging when trying to rotate things in advanced ways.
Even though Apple has decided to ban Flash and pick a fight with Adobe, many new apps are built to take advantage of the mouse moving around. On the iPad, there's no mouse pointer, so you have to simulate one by touching the screen and moving the finger around.
Keyboard navigation is strange, because most of the time, you don't have a keyboard. The keyboard only shows up only when you focus a text field, at other times, you're lost.

These things sound like annoyances, and they are, but they are also quite minor. For all the surfing you usually do, the iPad works just like it should.

Bad: A horrible Developer Platform#

As you've probably heard during the last few weeks, Apple have no clue whatsoever about how to cultivate a vibrant developer community. Let's look at the facts:

To distribute your app you need to join the iPhone Developer Program, that costs you $99-$299 per year.
To download the SDK you need to fill out a huge multi-step form (where almost all fields are required), and where you're asked to give away lots of personal information.
When you "publish" your app it has to go through a manual approval process, which could take months if you are unlucky.
You need to approve to a ridiculous legal statement preventing you from using some programming languages, and making calls to some APIs.

To quote:

"3.3.1 — Applications may only use Documented APIs in the manner prescribed by Apple and must not use or call any private APIs. Applications must be originally written in Objective-C, C, C++, or JavaScript as executed by the iPhone OS WebKit engine, and only code written in C, C++, and Objective-C may compile and directly link against the Documented APIs (e.g., Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited)."

Needless to say, I'm never going to be develop anything for the Apple Platform. Apple and its developers is much like one big company (with strict rules). The only difference is that instead of a salary the developers get to pay to work for them. Good for Apple, horrible for developers.

Good: Games that use the touch interface#

There are lots of good games that have started to take advantage of the big touch surface to make fun interaction styles possible.

Tap Tap Radiation builds on you tapping the screen in pace with the music playing. The place to hit move around, just like the markers that indicate where to tap.
Asphalt 5 is a beautiful racing game where you use the iPad as a steering wheel and tilt it back and fourth to steel. Since the accelerometer is so sensitive it works remarkably well.
N.O.V.A. - Near Orbit Vanguard Alliance is a first person shooter that uses gestures to control who you shoot and how. Advanced gestures makes it possible to throw grenades that fly around corners, and targeting multiple enemies at once.

Simply put, there are lots of fun games for the iPad, and the reason is experimentation with the touch interface. Combine this with several days of battery power, and you have many hours of fun ahead of you.

Bad: Downloading apps through iTunes#

[](/files/post-media/ipad_closeup.jpg)

A closer look at the iPad, still with the banana present. Click for bigger image.

Sadly, the only way to get apps legally into the iPad is through iTunes. Since my iPad was bought in the US, the App Store icon leads to a "Your request could not be completed" with no further info. So it seems that the App Store is only launched in the US. Silly me, that thought the web was an international thing.

Anyway, it is possible to get apps into the iPad without getting a fake american App Store account. You just use iTunes to download the apps to a desktop computer, and then sync your app to the iPad. It's easy.

Well, strike that, using the iTunes store is not easy. Not being a previous iPhone using I had thought that downloading apps from iTunes was like shopping on Amazon. IT'S NOT EVEN CLOSE. This is how Apple built iTunes:

Take one fully functional e-commerce web site.
Make it slow so that people like me that surf on a 100 Mbit/s fibre connection have to wait several seconds after each click.
Make sure you can only access the store through a special downloadable app, that requires you sign several legal agreements, and leave away personal information, before you even bought anything.
Remove the possibility to sort through listings, remove any information what the list is currently sorted by, and hide options to only show apps for the device they know you just connected (the iPad).
Tada! You've built iTunes!

Needless to say, each visit to the iTunes App Store is a bad experience, and no something you want to do often.

Summary#

Situations where I see myself using my iPad:

In bed, watching public television through Svtplay. They have a great app, that is mainly adapted for iPhone, but scales up when used on a bigger screen.
Casually surfing the web from the couch. This includes checking my e-mail (which is IMAP over SSL, which works fine), checking my feeds through Google Reader, checking Facebook through the iPhone app (which can be pixel-doubled to fit the screen nicely), and twitter through twitterific.
Surfing all kinds of maps, and wikipedia in some social context. If I ever get into an argument about where a country is exactly, I'm pull up my iPad instantly and check it.

And that concludes this iPad review.

[](/files/post-media/ipad_off_and_banana.jpg)

iPad rests here, next to banana.

As usual I'm looking forward to your thoughts in the comments. Thanks for reading!

Why people skip newspapers and read news on the web instead

2009-06-25T22:44:12+02:00

You can't be involved with what's happening on the internet without coming in contact with the "newspaper crisis" somehow. From a business perspective it's simple really: Much fewer people buy newspapers (on paper) nowadays. Please note that this has very little to do with advertisement or business models, I'm talking about newspapers from the user perspective here.

Internet is really a commodity nowadays. People process loads of information on the web every day, and this of course affects how they expect newspapers to behave. Every time I hold a big newspaper in my hands I'm surprised at how inferior it is compared to reading news on the web.

Newspapers have problems with references#

If I find an interesting news story on the front page, it's a mess finding the full article in there. The references are done with page numbers, but with page numbers that are local to a certain part of the paper. "Culture, page 7". And the culture part is stacked inside the part I'm reading, so I first have to find that one, then find the page numbers (which are removed from pages with ads), and then finally find the article I wanted.

The same is true for related articles. If I read an article I like, it's quite likely that I want to read other articles on the same subject. Newspapers solve this today by placing similar articles close to each other, and hope that you see them. This is of course limited, and gets harder when pages sizes shrink.

Compare this with clicking a link on the web. If I find an interesting article teaser, I click it, and am instantly taken to the full article. If that article was indeed as interesting as the teaser suggested, I'm often presented with similar articles, from similar categories, and can click them to move there.

Newspapers are slow#

Even the most frequently published papers are only distributed once per day. This simply means that papers can't compete on speed, being first with a certain story. Even if you happen to get your hands on a story at the perfect time, a paper still have to be both printed, and distributed to people. This takes hours.

What's worse, morning newspapers brand themselves as dealing with "today's news", when in fact it's the news from yesterday. This hasn't been a problem before, since there was no faster way to get news. Now there is.

If speed is important to you, you can easily subscribe to news via e-mail, Twitter or RSS, and be instantly updated.

Newspapers are static#

Articles in a newspaper are, once published, not possible to update and improve. They are left for the wind, even though there are inaccuracies or important clarifications to be made. Any conversation sparked won't be there.

This is of course the strongest argument for internet news. A big article will be different if you look at it later the same day. Comments and updates based on feedback are able to improve articles considerably.

Newspapers don't have enough unique content#

Big parts of daily newspapers contain poor rewrites, or plain copies, of articles from elsewhere. The reasoning is probably that they are trying to be exhaustive, give a broad view of what has happened. Problem is, they are hiding their own unique content behind loads of reposts of other's content.

The same happens on the web, but instead of copying the article you link to it. Then you get to read the news from the real source, and dig in deeper if you want to. Additionally, there's safegards that stop people from copying other people's content. For instance, Google have special filters for filtering out duplicate content.

Newspapers are not relevant enough#

The biggest reason why I don't subscribe to any newspapers is that they are not relevant enough. I'm not one bit interested in sports, and still, during big sporting events newspapers push them to the front page, over interesting internet-related news; things I find interesting.

The key here is of course to realize that relevancy is in the eye of the beholder. Only I know what I find interesting, and relevant. Why trust someone else's relevancy ranking when I can easily get my own ranking online? Even if I don't want to tailor-pick RSS feeds and build my own custom news feed, there's someone out there that has more similar taste than the four major newspapers here.

So, what should newspapers do?#

Well, they have two options: One, they could keep writing articles, hoping that the quality will be high enough to turn the trend, or Two, they could start thinking of how they deliver news. The expectation of how news should be served is changing. It now needs to be delivered…

… filled with references for digging deeper
… faster than once a day
… in a manner where people's contributions enhance it
… with a quality stamp that ensures that you're reading something you couldn't get anywhere else
… personalized to my own specific taste. No sports.

Is there any other way to do this than focusing aggressively on the web, and less on dead trees?

Follow the 10 ground rules, or fail on the web

2008-11-20T18:58:16+01:00

When you work with the web, there are a couple of basics you really need to get right. These are things I see people misunderstand over and over again, with a varying degree of failiure as a result. Not only individuals make these mistakes, even really big organizations make them, and suffer as a result. So lets get at it.

1. Everyone is anonymous on the web, if they want to#

If you depend on trying to identify who a particular user on the web is, you will fail in every way possible. This is what lots of law firms try to do, especially in the context of file sharing. You will never ever be able to know who someone online is, if they don't want you to.

Technical reasons: IP-numbers are easy to fake, but using proxy servers of different sorts. New services are constantly popping up, that makes this available for unexperienced users. Just push them a little more, send a couple of more cease and desist letters and even ordinary home users will start to cloak who they are. It's possible, it's easy.

Social reasons: It's very easy to surf using someone else's internet connection. Ways vary from just connecting to an unencrypted wireless network, to just using the free connection at a library or school. And it's getting worse: people are starting to move around while using the web (mobile), hopping from one connection to the next as their geographic position permits. The link between internet connections and users are getting blurrier as we speak.

Despite the above, there are things you can identify and use. One of those is categorizing users into user types. This is what Google (and lots of others) are successfully doing with their advertising system. "Based on what you've showed interest in previously, we think you will like this". This enables you to personalize and customize your content to a particular user type, something that holds lots of economical value (if that's what you're after).

2. Give your content away for free, or watch someone else do it#

The web is a gigantic content creation machine, and except for very niched content, you can find anything on it. If you are in the business of selling content online, you will see your profits steadily decline. The competition is just too intense when anyone can create content.

How do you tackle that? Well, put your previously costly content online for free, to drive people to your site. If the content is good, your will see a massive surge in number of users. Well on the site, you will need to figure out how to make money of them, either by showing them ads or getting them to buy something physical.

Industries that depend on online content creation are really living on the edge right now, and for a future prediction, just look at the music industry. You still have some time to rethink your bussiness model.

3. Linking is the core of the web, make people want to link to you#

It's called a web because of the links. Trying to limit or control other sites linking to you is fundamentally breaking the web. It's also downright stupid, since every external link is a vote for your site, and a sure way for people to find you. Having an "link policy", controlling how people may link to you, is a sure way to make a fool out of yourself online. Not to say that people won't care one bit about it.

Instead, you should encourage people to link to you. How? By first giving every piece of content a linkable address. Many flash- and frame-based sites miss this fact, and instead require users to click through your frontpage to find the content they want. This is a sure way to turn users away. Just make your content linkable, it's not hard.

Then, make sure the content is in such a high quality that users want to link to it. This rules out saying what everyone else is saying, in the same way everyone else is saying it. You need to be different, and have genuinely interesting content, that no one else has. This is hard, but a problem you need to solve to make it on the web.

4. Link to external sites with good content, it's all about servicing your users#

There's billions of sites a user can go to, and you need to work hard to servicing your users if you want their attention. Sometimes your content is so extensive, that no matter how hard you work, users will still want more. This is a perfect situation where you should link to external sites and give your users what they want.

Many newspaper make this mistake, and end up with users leaving their site for Google, trying to find more information about a particular piece of content. They have instead linked to good follow-up content directly, given the user what they wanted, and be accessible via the most used button on the web, the back-button. Happy users that have easy access to your site, or annoyed users that leave for Google, your choice.

5. People will copy your content, and there's nothing you can do about it#

Some people are worried about the copying of their content. And if they see that as an issue, they better be worried. You see, each time someone reads your texts, or looks at your media, it's copied from your server to the users computer. Web browsers are really effective copyright infringement machines, and by just clicking a link, you copy that page.

So if you are worried about users copying your content, please don't put it online at all.

Other people think that they can have their content on the web, and still prevent people from copying it. This is pure lunacy, and all ways to prevent copying have failed. People even miss the most obvious way people can copy, by hitting the "Print screen" button on their keyboards. When it comes to video and audio it's a bit more annoying, but everything that comes out of your speakers, or is displayed on the screen, is recordable. So stop worrying about copying, you can't stop it.

Instead, know and accept that people will copy your content, and just ask for a simple link back if they do. That way, small pieces of your content on other sites will act as marketing for the whole site.

If you want this formalized as a license, there is the creative commons attribution license, which I use on this site (see footer).

6. Use the web to communicate with your users, or watch your impact fade#

Almost everything is moving to the web, including most of your business and private life. This of course has implications for web sites. Back in 1999, a couple of smart people sat together and tried to formulate how the future would look for businesses. One of the fundamental truths they found was "Markets are conversations". In essence, this means that you sell stuff by talking about it with customers. It can be through marketing or through recommendation, but it is talking. When that communication moves to the web, you too better start talking on the web. And quickly.

This movement of websites for two-way instead of one-way communication has been given the techy name Web 2.0. And it turns out, this new web is a terribly effective way of communicating. The more people realize this, the more of the conversations will move to the web.

The question is, when hoards of people come to your site to talk to you about your product, are your ready? Or will you give them: "Sorry, this store is closed, but we have this nice brochure you can read…"

7. Communicate with your users in natural language, marketing speech has no place on the web#

It's easy to find marketing speech about products today. Companies compete with each other to make sure as much of your field of vision as possible is filled with their message. When people are subjected to that much information they have no interest in, there are consequences. On the early web, the consequences were banner blindness, people simply started ignoring some banner formats. On the modern web, people have started ignoring marketing language use instead. The words "Best", "Revolutionize" and "Cheapest", instantly makes people stop reading.

Why? Because companies writing about their own products don't have anything to compare to! They can't say what's best, because they don't use their competitor's products. They don't know if the product will revolutionize the users life, because they know nothing about the user. And any user can look up how cheap they are on a price comparison site.

So what to do? Talk to them like you would to a friend, in the afternoon, after two beers. This will make sure you don't turn customers away by trying to push to much crap on them. You wouldn't do it to your friend? Then don't do it to potential customers. Just changing your language has huge positive implications, but let me just sum it up like this: it builds trust.

8. Be honest about what your strengths are, liars are easily uncovered#

Never has it been so easy to look up facts. Liars are easily uncovered by a couple of soft tappings on a keyboard. What's even better, is that even subjective things can be checked. If someone yells: "Our product is the best one!", you can find out if that's the case in a few seconds. Just read a couple of reviews, look at comparison charts, and you will know.

Lying in person is rather bad, but lying on the web is worse. All the facts are so close. A customer is literally seconds away from finding out if you're bullshitting them or telling the truth. This makes it vital not to lie on the web.

If you want to brag anyway, make sure your add a "we think" before all your bold claims, and back them up with solid arguments. Because if you say "We've found no cheaper product than our own", and I find out there are others that in fact are, you're not a liar, you're just badly informed.

9. Care about search engines, and double your number of users#

You are not the only site on the web, and people will spend most of their time on sites other than your own. This means, that if you don't think about how people find your site, all your content work is in vain.

So how are people found? I have yet to see a customer of mine that gets less than 50% of their users from search engines. Usually 60-70%. People tend not to understand the implications of this. A e-store could potentially lose over half their sales by misbehaving in Google's eyes. This is huge.

You need to care about search engines. Luckily, they are well synced with the ground rules you're reading. If you make your content linkable, unique, interesting, and talk to them in a language they understand, you will be successful. All serious search engine optimization guides will say the same.

The last step, and what many miss, is to keep track of how people find you, and what they find. Then feed that information back to your content producers. That way, you can produce content that people actually like.

Or, you could lose half your sales. Your choice.

10. Encouraging and acting upon feedback is currently the best form of marketing#

Asking for feedback is unfortunately still uncommon. Unfortunate because there are so many out there that could benefit from knowing how they can do better. It's also fortunate, because it means that you have a chance to be different, and stand out, just by asking people what they want.

Dell IdeaStorm is a great example of this. They are simply saying: "Hi, we want your feedback", and instantly they have media coverage everywhere. Additionally, they get great feedback on how to improve their business.

Of course, this means you need to act upon feedback too. Telling people that you listen, and then don't is a much worse offense than not listening at all. So find yourself a good chair, hand out the megaphones, and listen carefully.

Where to now?#

Thanks for reading this far. This article comes from a frustration over how some companies, and individuals, think the web is still a place were they can makes their own rules. I would therefore like to set the foot down, and tell people what I've learned about the web, about what works and not.

I also know that there are other people out there that needs to read this. Do you know some of them? I then appreciate if you could send them the link. More people need to read this.

Thanks.

What is Web 2.0? Really.

2007-03-10T00:59:52+01:00

Web 2.0 is really hot right now. One of Sweden's biggest newspapers recently wrote a long article on their debate section. They had started linking back to blogs that linked to them, in a little box next to the article. The problem was that they had got into trouble with what blogs to link to. After all, you can't just link to anything, right?

Aside from starting to think about the implications of blog links, I got another interesting question in my head. What is Web 2.0 really? Most people working with interface development would say that Web 2.0 is everything that uses AJAX. But the newspaper didn't use AJAX at all, and still they claim links to blogs is Web 2.0. Time for some research!

The hunt for a definition#

The phrase "Web 2.0" was first put into widespread use at an O'Reilly conference in 2004. The organizers wanted to talk about a change that has happened on the web, and just bumping the version of the web seemed like a good idea. Paul Graham found this first try at defining Web 2.0 at the conference:

"While the first wave of the Web was closely tied to the browser, the second wave extends applications across the web and enables a new generation of services and business opportunities."

Note that there's no mention of AJAX there. Hell, there's no mention of users either! They must have meant something else, and the definition might have changed over the years since then. Let's keep looking for a good definition.

Tim O'Reilly comments on the issue two years later, in a clarification on his blog:

Web 2.0 is the business revolution in the computer industry caused by the move to the internet as platform, and an attempt to understand the rules for success on that new platform. Chief among those rules is this: Build applications that harness network effects to get better the more people use them.

Tim acknowledges the change in meaning and talks about "network effects" here, something that starts to look a little bit more like what my idea of Web 2.0 is. But isn't there still something missing? To me, that looks only like a small part of what I call Web 2.0. Let's keep looking…

Gartner has tried to convince companies of the merits of Web 2.0 for a rather long time. In one of their many (business oriented) PowerPoint presentations they attempt to define "three anchor points around Web 2.0":

Technology and Architecture — Consisting of Web-oriented architecture (WOA) and Web platforms.

Community — Looks at the dynamics around social networks and other personal content public/shared models, wiki and other collaborative content models.

Business Model — Web service-enabled business models, mashup/remix applications and so forth.

I believe we're getting closer. Gartner is making business models a separate point which I don't agree with. Many of the biggest Web 2.0 sites didn't have a business model when they started (and many still don't). Digg has troubles covering their hosting cost with the tiny bit of money they acquire from their ads. Del.icio.us still doesn't make any direct money (although they got sold to Yahoo, and they surely use the data…). My point is, a business plan isn't one third of Web 2.0, it's something sites add afterwards if things work out.

So let me state my own (slightly more general) definition of Web 2.0:

"A collection of ideas and techniques that can be used to make more interactive sites"

The thing is, just doing something in AJAX does not mean it gets the Web 2.0 stamp of approval. You need several of the ideas or techniques and you need to combine them in clever ways. So lets agree on the definition above and got look for ideas.

Ideas that are part of Web 2.0#

I'd like to gather a whole bunch of ideas under the Web 2.0 roof. My selection is of course not all there is. Googling could uncover more I'm sure, but I think this is a good start.

User generated Content#

Many companies still view the web as a one-way medium, an extension of a paper catalog, it's main advantage being that it can be distributed to more people. It's not like that any more! There are people that want to add content to your site, that want to contribute their ideas and thoughts. Why are you denying them to help you?

User generated content is content that your users are willing to give you. It can be everything from a simple "like it" or "don't like it", to fully featured articles written by users. The two most used ways of contributions are the simple ones: votes and comments. The author is still in full control of the content but users are given a chance to chip in with minor corrections. This is the least you can do. It's what I do on this blog; let you chip in by commenting.

But if you have users that are passionate about your subject, users that are willing to use your site to get their ideas out, you should really endorse that! Allow them to post articles on your site, allow them to regroup and re-prioritize according to their own wishes. It is technically possible, even rather easy to do; you just have to do it.

Six simple things your users can help you with:

Reviews of your products
Comments on your articles
Stories about how people use your product
Multimedia using your product
Questions about your products
Articles in an area you choose

Radical trust#

Another approach many sites have taken can be summarized as "radical trust". It builds on the simple idea that users know what they want better than you do. So let them order, categorize, sort, select your data as they like!

This is exactly what wikis do. They acknowledge that there are more users doing good than doing bad. If that's the case, why not let them in on your content directly, letting them edit and improve it like they want. They few people that do bad can be hunted away with a combination with versioning (saving old versions of content that's easy to restore), and some simple monitoring. Wikis trust users, do you?

Syndication of content and services#

Web 2.0 is not only about "user to website" interactivity. It's also about letting other sites and tools interact with your site directly. This is often summarized as "syndication".

It's a very fancy word for a simple concept. Let me try to explain.

Somewhere you have some kind of database with your content, be it products in your e-store or posts on your blog. Usually you take that content, add some structure (HTML) to it, and send it to the user.

Another website that wants to access the same information could parse the HTML and try to understand what it means, something called "screen scraping" (or microformats ;). The problem with that method is that it's very dependent on that the webmaster doesn't decide to change the HTML. The other problem is that computers and humans often want different types of information. A computer that is going to parse a list of your products doesn't need navigation like humans do. What you do is send your data directly to computers instead, without messing it up with HTML. Formats include: RDF, RSS, or perhaps custom XML through a Web Service.

Thing is, when you start syndicating your data you make it easier for others build services based on it. Now people get several entrances into your content instead of the one you produced. Again, your users are helping you reach more people.

Allowing full RSS feeds is another way of syndicating, I'm switching right now. Do you syndicate?

Long tail#

The theory of the long tail is one of the ideas that are usually associated with Web 2.0. It's a business model that many companies that are successful on the web use. As with most business models they are invented after the fact, and as such I'm not sure it really belongs on this list, but people always bring it up, so let’s go. I'll let you choose yourself.

Wikipedia describes the long tail like this:

"Products that are in low demand […] can collectively make up a market share that rivals or exceeds the relatively few current bestsellers and blockbusters, if the store or distribution channel is large enough"

Most physical stores need to aim for the bestsellers to sell anything. There's simply too few interested in odd products, to make it worth hiring store space and personnel to sell it.

Web based stores are in a different position. Selling another product usually means just adding another page to your site, no extra store space or personnel needed. In addition people are prepared to wait a couple of days before receiving their products. That time span means you can skip warehouses and not start producing your products until you have an order ready.

No more is there a need to estimate what people will be interested in and pre-order them. For the right kind of product the internet is a huge benefit. Amazon did it with books, iTunes did it with music, do you do it?

Collective intelligence#

You have let each user customize their experience and add content of their own. Now it's time to organize that content to better help the everyone benefit from it. There's hundreds of ways of doing it, but here's a couple of things that you can present to your users:

The most popular Swedish article right now… - Crawl all Swedish blogs and keep track of what they link to.
Others that bought this book also bought… - Query your shipment table. Pick a product, select the number of times each other product appeared together with said product, and recommend that product to your customers. Amazon does this and it's said that it has increased their profits considerably.
You probably think this movie is a four out of five - A Swedish movie site give its users personalized ratings of movies. They find users that have similar taste as you have (based on your previous grades) and then checks what those people have rated the movie as.
This is probably spam - Write a word filter that learns what is considered spam when users mark them. Share that filter with fellow users and let their markings stop your spam. A form of collective sieve filtering, Akismet does this.

None of them are technologically hard to do, you just Google and copy other people's samples. Why not?

Techniques involved in Web 2.0#

But there are techniques involved too. Let's go through a few of the important ones.

AJAX (and other javascript)#

Everyone talks about AJAX together with Web 2.0, but I think it's important they are kept separate.

AJAX is just a technology that helps prevent (full) page reloads. Instead you connect to the server silently in the background and receive your data that way. What's the revolutionary about this technique? Nothing. It has been in use for at least 5 years. They new thing about it is that people started using it to build better interfaces.

Javascript is language that enables AJAX, and playing with reloads is not all it can do. Through some nifty use you can change attributes on any HTML element on the page. Move things around, react to mouse movement, fade and animate, it's your choice. This means a lot of new controls become possible, ranging from simple sliders to interactive maps.

Why do most accessibility people hate it? Because most developers don't know enough about accessibility. And when those start to use AJAX they disregard accessibility completely. Javascript and AJAX have different goals and I think a good compromise is making sure the basic functions of the site (buy a product and pay for it) works without javascript, but enabling it adds additional features.

When was the last time you used javascript to enhance your site? What was the last control you invented?

Feeds (RSS, Atom)#

Feeds are great for syndication of content. There are many different feed formats to choose from but they all have one purpose: to communicate pure data, skipping all design.

A feed is simply a list of feed items, each with an unique identifier. A user adds the address to their "feed reader" and it starts polling you, asking for updates. I have my reader set to just a couple of minutes, making sure I quickly notice changes in people's feeds.

The good thing about feeds is that they make it easy to follow several at once. There's no annoying different designs in the way if you don't want to (you can always just visit the site if you want design). Feeds are getting more and more of a commodity; you should already be allowing your users the possibility to subscribe your content. Do you?

Tags#

Tags is another hip concept. It deals with the collective intelligence idea and how to categorize content efficiently. A tag consists of a phrase of some kind that describes a piece of content. This blog post could have the tag "javascript".

There's several ways you can use them. One is the just fix what tags are allowed and use them as regularly groups you can assign content to. But allowing more than one tag enable you to do more than just split things into groups, you can instead pick all contents bits that have the same tag. You can go further, allowing custom tags that the users can pick themselves. That gives you a wide array of descriptive words for your content, free to play around with. For example, if many users pick the same word, that one is probably a better descriptor.

Picking many bits of content and analyzing all tags tied to them can be easily done with a tag cloud. In that you simply print all tags used after each other, and make those used often bigger. Doing this on a whole site is an effective way of giving users a snapshot of what you write about, something I know I like.

Can tags help you solve a categorization problem you have?

Summary#

Now. To build a Web 2.0 site, pick from both of the lists above. You Google for more ideas and you work hard to interact with your users as much as possible. Web 2.0 is the combination that happens when your ideas and technology finally just works.

Your users want to help you, do you let them in?