1. You type an URL into address bar in your preferred browser.
2. The browser parses the URL to find the protocol, host, port, and path.
3. It forms a HTTP request (that was most likely the protocol)
4. To reach the host, it first needs to translate the human readable host into an IP number, and it does this by doing a DNS lookup on the host
5. Then a socket needs to be opened from the user’s computer to that IP number, on the port specified (most often port 80)
6. When a connection is open, the HTTP request is sent to the host
7. The host forwards the request to the server software (most often Apache) configured to listen on the specified port
8. The server inspects the request (most often only the path), and launches the server plugin needed to handle the request (corresponding to the server language you use, PHP, Java, .NET, Python?)
9. The plugin gets access to the full request, and starts to prepare a HTTP response.
10. To construct the response a database is (most likely) accessed. A database search is made, based on parameters in the path (or data) of the request
11. Data from the database, together with other information the plugin decides to add, is combined into a long string of text (probably HTML).
12. The plugin combines that data with some meta data (in the form of HTTP headers), and sends the HTTP response back to the browser.
13. The browser receives the response, and parses the HTML (which with 95% probability is broken) in the response
14. A DOM tree is built out of the broken HTML
15. New requests are made to the server for each new resource that is found in the HTML source (typically images, style sheets, and JavaScript files). Go back to step 3 and repeat for each resource.
16. Stylesheets are parsed, and the rendering information in each gets attached to the matching node in the DOM tree
17. Javascript is parsed and executed, and DOM nodes are moved and style information is updated accordingly
18. The browser renders the page on the screen according to the DOM tree and the style information for each node
19. You see the page on the screen
20. You get annoyed the whole process was too slow.

I, too, get annoyed when the above steps take longer than one tenth of a second. But now at least I have some documentation to look at, while waiting the remaining fractions of a second before the page renders.

Spoiled we are, all of us.

(Feel free to add more steps, through the comments…)

Many of Tapestry’s problems comes from their design decision to parse the HTML you’re trying to output. Yes, Tapestry parses your HTML, and adds stuff to it dynamically. This is nothing new, anyone that has played with ASP.NET knows how hidden form elements get stuffed in here and there. This is a nightmare for an interface developer, we need exact control over HTML to do our jobs well.

Tapestry does horrible things to HTML:

• Several extra javascripts and CSS files are referenced in the head tag. There is no simple way to get rid of these.
• A meta tag which states that the tapestry did generate this page, is added.
• The two above are added to the head, and if a head tag doesn’t exist, it gets added. Never mind that your current little HTML snippet was just a little part of a page, that was being fetched with AJAX, Tapestry will add a head tag for you.
• More javascript, and a hidden div with a firebug-like console, is appended to the end of the body element.
• Self-closing tags are expanded to a start tag and an end tag (<input /> gets transformed to <input></input>), which means you can’t use XHTML together with Tapestry.
• All whitespace is removed by default, and you have to disable the “feature” on a tag-by-tag basis (“Please don’t strip whitespace inside this list, IE6 goes crazy then”). This is a mess for interface developers, who know that whitespace matters for rendering. It was even pointed out to Tapestry developers early on, but was ignored.
• There are ways to do loops, if:s, reference variables and generate URL:s, but all of these are based on the HTML parsing. And as a good parser, it skips all comments. This means using tapestry template code inside a HTML comment will not work. When do you need that? Conditional comments! So what if you want an URL generated to your IE6 stylesheet? No go. You’ll have to write a custom component that does that for you.
• Changes the id of all your forms to “form” and adds name=”form” (which is invalid HTML).
• Adds a hidden “t:formdata” field to forms, much like the dreaded ASP.NET viewstate.
• One of the javascript files added is prototype, a javascript framework which isn’t compatible with jQuery. So you have to rewrite your javascript code to work in “No conflicts mode” if you want it to work with Tapestry.

No. Tapestry was clearly not built with an interface developer in mind. Why is it so hard for many web framework developers to just talk with people that work with HTML, CSS, and Javascript? Please ask us before implementing stuff like the above. We’ll gladly give you our viewpoint.

You also want people to contribute to your sites with content somehow. Text is not enough, you want all kinds of “rich” content, and you want people to be able to place them how they want on your page. I mean, we live in a new era after all.

Problem is, this new era thingie talk makes people forget the basics. Even though this is the future, and we’re all bored with “just” hypertext, we can’t allow random people to add HTML to our sites. Why? Ask myspace about the Samy worm from 2005, a little piece of clever front-end code that took their servers flying. What did they do wrong? They tried to let untrusted people embed HTML on their profile pages.

But they even had this clever filter, which cleaned the code first! Well, long story short, using a div with the style attribute set to a background image that points to a javascript url actually execute the javascript in IE6. Didn’t think of that did you? So what about the fact that both IE6 and IE7 execute vbscripts in urls prefixed with “vbscript:”, just like with javascript. Didn’t know that? Ohhh. Perhaps then you shouldn’t be trying to let unknown people embed HTML on your site?

Many of these security issues stem from the fact that ordinary well-educated computer scientists don’t know enough about interface development, HTML, CSS and javascript, and discount them as being something that “anyone” could do. “Even my aunt made this puppy site in like 3 days, how hard can it be?!”.

I’ve always thought that interface development have an undeserved reputation of being easier than “real programming”; something that you can let rookies work with, something that no “real” programmer would ever want. “No, I want to work with hard stuff, not that webby stuff”.

If you are in a position where people actually think that, perhaps security could be the way to lift interface development to its proper status. So why not read up on Cross Site Scripting (XSS), look at examples of vulnerabilities, and pick a couple of examples of big sites that are vulnerable. While you’re at it, why not read up on Cross-site request forgery (CSRF) too?

These are real issues that someone needs to take into account when building websites. My guess is that the fancy computer scientists will have a very long way to go before grasping this stuff. “Why does IE6 parse ‘java   script:’ as if there where no space in the middle?”. You know who’s not surprised? Every damn interface developer out there.

So. Go out there, and teach them silly math people how it’s done, and what your HTML, CSS and javscript-knowledge is worth. Show them that logic isn’t the way we do things around here, that anything can happen when IE6 boots. And… whatever you do… think very hard before letting people embed HTML on your site.

After asking friends how to solve the problem they get the answer that they now have to learn PHP. Or ASP. Or JSP. Or some other strange language they don’t need. And install this thing here, and that thing there. What does your server host support? Oh, no, you need to configure stuff better. No, that setting is insecure… You know the drill, I’m sure you’ve walked someone through it sometime.

So, a way to include a piece of HTML into a random page would clearly benefit beginners learning HTML. But that’s not all:

• Browsers would be able to cache components of a page, and therefor load new pages using common components faster.
• Less work learning new template languages just to find that language X does not have a way to include things.
• Possible to learn componentization by looking at existing sites and learn from them. This is an area we need to be better in.
• Easy to make fallbacks by linking directly to the corresponding HTML snippet.

So, how would this be implemented? We need a tag that acts as a kind of include, so what about <object>? Just point to the HTML file you want and voilá, it gets included. Since this is HTML it would work exactly the same across all server-side languages.

Luckily, this is already in the HTML5 spec. At the bottom of the object specification. Iinterestingly, I found it after writing this article… Great minds think alike :)

In this example, an HTML page is embedded in another using the object element.

<figure>
 <object data="clock.html"></object>
 <legend>My HTML Clock</legend>
</figure>

Good! And it even seems to work in current browsers (Thanks Siegfried). I’ve tested it in Firefox, Opera, and Safari, and it works the same in all of them. Internet Explorer 6, 7, 8 (beta 2) just ignores it altogether.

The problem is, the current implementation is to handle them just like iframes. And there’s of course lots of problems with that approach:

• Currently, an object element without a height and width gets rendered as a 300 x 150 pixel block. There is no reason whatsoever to do this when including HTML. This must change for this to be usable.
• The included HTML needs to be stylable with the CSS rules on the page it’s included from. Currently, this does not work, included HTML is treated as an iframe. Must be changed if this is to be usable.
• Are HTML components full HTML pages? Do they include a doctype and a <head>, and do those get included? I assume only HTML inside <body> gets included in the new page. No CSS. No JS linked to in <head>.
• Clicking links inside included HTML should be handled as if the HTML was on the current page. This follows the same concept as if the HTML was included in the server side, and is needed if this is ever going to be used for a menu.

So, what do you think, is this a good idea? Personally, I’m hoping more concepts from template languages start to move into HTML.

Update: Thanks to the comments (thanks zcorpan) I now have found exactly the above in HTML5. It’s called <iframe seamless>. It meets all the points in my list, and I’m now really looking forward to the first implementation.

Good code never repeats itself. The number of techniques to avoid it are numerous, and all new languages compete in trying to remove as much repetition as possible (Especially the dynamic ones).

Good design repeats itself, good code does not.

With interface development, you face the conflict above over and over again. You get a design that (rightly) reuses the same concepts over and over, and you need to implement them in code that makes you write the same logic only once. This same time both when writing the code and later when fixing bugs in it, and deep inside, all programmers know that it’s the correct way to do things.

I’m currently working in a .NET project (EPiServer CMS 5), and is faced with a design that uses the same kind of boxes all over the site. The boxes only differ by color and content, so things like shadows and rounded corners are clear repetition that I want to do only once. I’ll do the shadows and corners with CSS, but for that I need a couple of wrapper divs. Divs that I only want to specify once, and then reuse.

The prequisites are:

1. I want a flexible solution, so I’m not tied to a specific HTML structure (number of divs, or even if I use the div tag or not).
2. No HTML in properties that get sent to user-controls
3. No HTML in code-behind (a common way in .NET to split logic (code-behind) and templates (ASP.NET and HTML))

What I came up with was templated user controls. They provide a way to write controls that wrap any other controls you may have, and add content around them. And it’s easy to write and user. This is how the one I wrote is used:

<MyProject:Box runat="server">
    <Contents>
        <h2>Random header...</h2>
        <asp:Repeater runat="server">...</asp:Repeater>
        ...
    </Contents>
</MyProject:Box>

It simply wraps anything inside it (in this case a heading tag and a asp repeater), and lets me do whatever I want with them. In my case, I wanted to add some generic HTML around lots of different content, but you could do anything you wanted.

This is how the above was implemented. First the code-behind:

using System.Web.UI;

namespace MyProject.templates.units
{
    [ParseChildren(true)]
    public partial class Box : System.Web.UI.UserControl
    {
        private ITemplate contents = null;

        [TemplateContainer(typeof(TemplateControl))]
        [PersistenceMode(PersistenceMode.InnerProperty)]
        [TemplateInstance(TemplateInstance.Single)]
        public ITemplate Contents
        {
            get
            {
                return contents;
            }
            set
            {
                contents = value;
            }
        }

        void Page_Init()
        {
            if (contents != null)
                contents.InstantiateIn(PlaceHolder1);
        }
    }
}

… and then the “code-front”:

<%@ Control Language="C#" AutoEventWireup="true" CodeBehind="Box.ascx.cs" Inherits="MyProject.templates.units.Box" %>
<div class="box">
    <div class="boxwrapper">
        <asp:Placeholder runat="server" ID="PlaceHolder1" />
    </div>
</div>

I think this is a really useful way to write user controls, especially for those of you that work as interface developers in a .NET world. Asking the people around me I found that quite a few didn’t know how templated user controls worked, so I hope I will be of use to some of you out there. Happy coding!

1. Do you need HTML or CSS for this?

Lots of times when I get stranded on hard problem and sit back, I realize that I really shouldn’t be trying to use CSS to solve all layout problems. Assuming a rounded corner box will have to scale in all directions (up to 8 images!), when it really is fixed width and height (1 image), is an indescribable waste of everyone’s time. Look closely at the design you’re trying to implement: does it really have to be as flexible and scalable as you think? Shifting to that mindset saves me hours of work per day.

One thing that many developers forget is that not all designers care for exact pixels. If something is really hard to develop, walk over to their table and suggest a change. You’ll soon start to notice that designers are real people too, and that they have as much flexibility as you do. It’s just as easy to call them up, so don’t let shyness waste your time.

2. Try to set properties once

When you know you have to do something (see above), try to do things once. This is really tricky in both HTML and CSS, languages that have no variables, and generally makes code reuse a pain. But it is possible.

With CSS; set widths of elements once, and trust inner elements to fill their parent. That’s the default behavior of block level elements, and you can set display: block, or width: 100% on most other elements. Don’t set heights at all if you can avoid them, and let elements expand to fill their containers. Combine similar CSS selectors to avoid typing things twice, and make new ones for common properties when few properties differ. Use floats and clearing instead of absolute positioning, and you’ll won’t have to micromanage positions all the time.

With HTML, make sure that you really know the template language. I’ve fighted with ASP.NET, JSP, JSTL, Smarty, Wordpress Themes, RHTML, and Django’s Templating Language, and now know enough about all of them to be productive. You should never have to type the same structure out twice, even in HTML. If your template language have loops, if-clauses and includes you really have no excuse to copy large chunks of HTML around. Refactor your HTML and make your own life easy.

3. Learn your text editor

When code reuse isn’t possible, use search/replace in your favorite text editor.

Replacing all instances of one HTML tag name with another is about replacing “element>” with “element2>” (without the opening bracket). That way you replace both opening and ending tags at once. Check to see if your editor supports regular expressions in its search/replace tool. If it does you can save lots of time by learning it. Matching a HTML tag is “<[^>]+>”, a starting angel bracket, one or more non-brackets, followed by an ending angel bracket. Naive regexps can make you match the starting bracket of one tag and the end of another, which makes for a mess.

There’s lots of little tricks like that, if you learn them you save massive amounts of time.

4. Make sure your environment supports you

There’s more than the text editor that you have to work with. One essential part of front-end development is trial and error. It sounds silly but no matter what your skill is, sometimes you must resort to trying and testing your way to a solution. With HTML and CSS, this means being able to quickly make a change, switch window, and see that change in the browser. If that cycle takes more that 5 seconds, you’re losing valuable time.

Don’t use a virtual machine (even if it’s only for IE6) that you can’t quickly Alt-Tab out of. If you’re stuck using one, you need to have the text editor on that machine too. It’s worth your time fixing it.

Don’t accept build processes that makes you wait more than 5-10 seconds before you see your changes. With trial and error coding you will then waste 50% of your capacity doing nothing. Most often you can bypass this by copying just the HTML and CSS to the right place, without touching other source-code. I’ve managed to do this in all kinds of crappy environments, and it’s always worth it.

So.

I hope some of the above can help you save some time each day. The little things add up you know. What’s your best time-saver?

ASP.NET developers made a decision once to include some hidden form fields, called viewstate, on all aspx pages. When deciding how to include those fields they chose an input with the type set to hidden. When the Sharepoint developers wanted to do something similar they decided on another route:

<input type="text" name="__spDummyText1" style="display:none;" size=1/>
<input type="text" name="__spDummyText2" style="display:none;" size=1/>

So we have a visible textbox, hidden with CSS, that has a size of 1 character. No wait, we have two textboxes and they both have very descriptive names (NOT). What the heck were they thinking? Why not type=”hidden”, and remove size? Or, I know, don’t add them at all, they are only dummy right? Note that they are empty.

Another strange interface decision is the one to include a skiplink in the page. Skip links are links that help screen readers (and others) to navigate to important areas quickly. There’s many different ways to implement them, here’s the Sharepoint way:

<A href="javascript:;" onclick="javascript:this.href='#mainContent';" class="ms-skip">

The .ms-skip class positions it far off screen to the left, and the javascript… wait, javascript on a skip link? Why on earth did they not add #mainContent to the href attribute directly? Only they know…

Next in this post is a more philosophical piece of HTML. Look at this:

<span dir="none"><table...>...</table></span>

I’ve heard about setting text direction to rtl (right to left), and ltr (left to right, for some eastern languages), but never to none. Where do text go that has no direction? Inwards? And what about having a table inside a span element?

ASP.NET also have user controls, a way to abstract out important pieces of code, to make reuse easy. Sharepoint of course uses this convenient tool regularly, and comes with a couple of useful ones:

<SharePoint:SPRememberScroll runat="server" id="TreeViewRememberScroll"
onscroll="javascript:_spRecordScrollPositions(this);" Style="overflow:
auto;height: 400px;width: 150px; "> ...

In the documentation for SPRememberScroll you can learn… absolutely nothing about it. It seems to strangely interact with a TreeView somehow, and even worse, it has a width and height! I guess it’s good that it’s separate from the TreeView itself, so we can remove it from there if we want to forget where we’ve scrolled to. What?

Sharepoint 2007, note the year there, also both have the classical spacer gif and a new one; the spacer span:

Spacer gif: <td width="4px"><IMG SRC="/_layouts/images/blank.gif" width=4 height=1 alt=""></td>
Spacer span: <span style="width: 3px; height: 5px;"> </span>

Starting with the spacer gif: Luckily they have their width set twice, both on the image and the cell. Note the “4px” there, you can’t use CSS units when you’re writing HTML, HTML only knows about pixels. You’re glad you wrote the width twice now, right? The spacer span is a fairly new thing I guess, the strange thing here is that you can’t set a width and height on inline elements. It just doesn’t work, sorry.

If you’ve built your user controls really bad, you might need a way to let people customize its HTML to make sense. Sharepoint has a couple of different ways to do this, one that they use themselves like this:

<SharePoint:DelegateControl runat="server" ControlId="PublishingConsole"
PrefixHtml="&lt;tr&gt;&lt;td colspan=&quot;4&quot; id=&quot;mpdmconsole&quot; class=&quot;ms-consolemptablerow&quot;&gt;"
SuffixHtml="&lt;/td&gt;&lt;/tr&gt;">

If you manage to read that, you can see that they are injecting a table row somewhere inside their control (which of course generates a table). Is it just me that feels the urge to add an ending table tag to SuffixHtml just to see what happens?

Another truly interesting interface construct is the datepicker. It’s one of these ordinary little calendar icons that opens in a little box and lets you pick a date. How did the Sharepoint guys (or girls) implement that?

...<A href="#" onclick='clickDatePicker(<long id here>, <long url here>, ""); return false;' >
<IMG src="/_layouts/images/calendar.gif" border="0" alt="Select a date from the calendar.">
</IMG></A></td>
<IFRAME SRC="/_layouts/images/blank.gif" FRAMEBORDER=0 SCROLLING=no
style="DISPLAY:none;POSITION:absolute; width:200px; Z-INDEX:101;"
title="Select a date from the calendar."></IFRAME><td>...

Ok, lots of code to analyse here, and I’ve still only cut out the middle of a couple of nested tables. A link with an onclick that opens the datepicker is fairly standard, all parameters needed are encoded into a long url sent to that method. There’s an image to click, with a strange img ending tag, but that could have been a mistake. The strange thing here is the iframe that points to a blank image. It’s inserted into a table right between two cells (yes, you read that right). That’s just insane. If you for some sensational reason have to use an iframe to display some HTML, output it javascript instead of just setting the src with js. Seriously people. Nice accessibility feature though, to have the alt text both on the image and on the iframe. Twice as accessible!

Another not-so-fine interface implementation is the following: They want to show a heading and a list of links. Personally, I would have used a heading tag, and an ul with links in. Sharepoint does it like this:

<div style="margin-left: 4px;">
<div class="level-header"><span id="header" class="headertitle">Division</span></div>
<div class="level-item-pos level-item level-bullet"><span id="header">
    <a href="<url>">Information Technology</a></span></div>
<div class="level-item-pos level-item level-bullet"><span id="header">
    <a href="<url>">Research &amp; Development</a></span></div>
...

They break their own scheme of prefixing all their classes with “ms-” (something that severely messed up my header), they repeat ids, they use only divs and spans, and inline styles.

Do I need to say more? Want to spread the word? Vote on reddit.

One of the principles of microformats is to “design for humans first, machines second”. Still, almost all formats are about adding span tags and class or rel attributes to existing XHTML. Humans will never see those, or benefit from them, unless there’s some kind of machine parsing done on top. Microformats were first built by people working with the blog search engine technorati, one of the reasons being to make it easier for technorati to aggregate data from those blogs. So machines it is.

Thing is, if you’re going to give information to machines, why not use vCard instead of the equivalent microformat hCard? hCard is just a translation of vCard into XHTML. vCards open in your e-mail program, allowing you to save the contact information there, hCards don’t open anywhere. vCards are also just as easy (probably easier) to crawl and parse as microformats.

So what I’m saying is, could we please use real formats instead of microformats?

Update: This article was too fuzzy, so let me clarify: This discussion is about embedded formats vs. formats. The “vs.” come from the fact that lots of sites that implement microformats choose not to implement the corresponding format, which in some cases lead to people not being able to use the extra information.

Controls that keep the HTML customizable

Splitting pages into “controls” is a common way to build sites in ASP.NET. There’s lots of different kinds of controls available, and I can’t list all of them or name their differences. But I know I don’t want to compile a .cs file every time a change some HTML. That pretty much rules out anything but user controls (as far as I know, but any kind of control that fills the above requirements would do). User controls have two sides, one for C# code (code-behind), and one for tags (HTML and calls to other controls).

There’s also the issue of built-in stuff. ASP ships with many pre-built controls that help developers push out things quickly, the problem is that almost all of them neglect the HTML (DataList, GridView, …). They only allow setting a few properties, sometimes a CSS class, and assume that’s OK. It isn’t. I need full control over both the CSS and the HTML to be productive. There’s one tag in particular is built with my kind of mindset, and that’s the Repeater. A repeater is used like a for loop, but outputs no HTML itself. Instead it allows you to specify some code to output before the loop, inside the loop, every other step, and afterwards. That goes a long way, and that’s often enough. If the predefined templates are insufficient, like if you want the last item to be rendered differently, you’re out of luck.

I think I agree with the MVC Framework guys in how to solve missing parameters of default controls and bad output: Use C# code inside your templates. Why is this different than the spaghetti code of old ASP? Because you only use layout related logic, nothing else. People tend to think mixing C# and HTML looks messy, but isn’t it stranger to make two very different things look the same?

There’s also tags like asp:hyperlink, asp:label, and so on; tags that essentially does exactly the same as the HTML they’re hiding. Please don’t use them, there’s no abstraction layer needed for HTML.

Don’t repeat yourself – not even in the interface code

Often when developers move from writing C# code to writing interface code they throw their coding skill out the window. You never copy a C# method just to make it output a different number – you make that number a parameter to the method instead. In the same way, you should never copy a piece of HTML from one place to another – move that code to a common masterpage or usercontrol instead. I’ve seen this many times, even from experienced developers. Never copy code, it makes for nightmare maintainance.

Also, use nested masterpages. So if you have three pages you need to build, and they have some code in common, you break that common code our into a masterpage. If two of the pages have even more in common you break that out into another masterpage, and let that one use the first one you made. Each Aspx page should only contain what makes it different from the other pages! Repeated HTML can also come in the form of small snippets of code, maybe 10 lines. You might not be able to break that out into a masterpage because it shows up all over the place. For those cases, break the snippet out into a usercontrol instead, and call that. Simple, and DRY.

Disable viewstate

HTTP is by default stateless. That means that each HTTP request to the server knows nothing about all the other requests made. For scalability and performance, that’s a great thing. By default, ASP.NET makes web development stateful. Every form field you fill out gets saved into a hidden form field, called viewstate, and sent with every request. But I’m not talking about performance here, I’m talking about interface development:

To be able to save state across requests there’s a hidden form field, and to be able to send that with every request there needs to be a form covering all your form fields. I can’t help to think that handling statefulness in the HTML (the view) is the wrong place to do it. Aside from taking away some of my control over the HTML, it also has accessibility issues. JAWS, a popular screen reader, switch to a different reading mode when inside of forms, and while it’s possible to get to all content inside a form, it’s harder.

So, start by disabling viewstate by default. If you must have it for a certain control, only enable it for that control.

Summary

There’s lots of stuff you can do to make the HTML look better with ASP.NET, and that’s a good thing. The bad thing is that it isn’t that way by default. Now get out of here and go build some good HTML :)

If you instead had called it “licenseAgreement”, you would have been better off. Right? Perhaps not. Because we’re forgetting about CMS:es. Those often have some kind of templates, where you as an interface developer define how the HTML should look. But as a interface developer, you have no idea of what kind of content editors will put into your templates. A couple of clicks later your “semantic id” is wrong.

I think this is one of the reasons semantics hasn’t gotten a hold of the CMS world. CMS:es have no idea of what kind of content people will store in them. Their main business goal is to make something generic, that doesn’t assume semantics.

This is why “wrapper” is such a common class name.

This is not only a sane way to approach standards, but it’s also a way to say “We’re listening and are open to change based on your feedback” (Although saying the reason is the new Interoperability standards and not user feedback is rather weak).

This move is great for both for the web in general and for the Internet Explorer Team. Let there be celebration!

Sharepoint 2007 has been “updated” to support masterpages, a concept from .NET 2.0. Sharepoint’s implementation of masterpages has several problems, but none of them even come close to the biggest problem of them all: default.master.

Default.master is the page that ships with Sharepoint and is used everywhere by default. A quick glimpse at it would make any seasoned web developer feel sick, and I quickly replaced it with something homemade. If you keep reading you will see why. Let’s get going with the first line:

<HTML xmlns:o="urn:schemas-microsoft-com:office:office"
dir="ltr" __expr-val-dir="ltr">

Things start out really bad, without a doctype on the first line. This mean that all default pages will render in quirks mode, making rendering unreliable across browsers. Next they set an XML namespace, which after some googling points to very thorough documentation (pun intended). So they mean that it’s an XHTML document? No, because XHTML has lowercase tags, and here they use uppercase. The attribute __expr-val-dir just doesn’t exist.

A hardcoded link to the 4000 line “core.css” (filled with classnames you have to override by hand), and five external hardcoded javascripts then follow. Then 30 lines of inline CSS and javascript. Then comes the next real beauty, the body tag:

<BODY scroll="yes" onload="javascript:
   if (typeof(_spBodyOnLoadWrapper) != 'undefined')
   _spBodyOnLoadWrapper();">

The funniest thing here is the scroll attribute. Let me paraphrase htmlcodetutorial.com (found through quick googling):

SCROLL, which is only recognized by MSIE, indicates if there be scroll bars on the page. The default value is YES, so the only reason to use SCROLL is to set it to NO. SCROLL serves little purpose except to confuse and annoy readers by removing scroll bars which are there for a good reason.

I couldn’t have said it better myself. There’s no reason, whatsoever, to use the scroll attribute. Ever. Especially with “yes” set. Pause for deep breaths. Then we have the inline javascript. Did you know that you could put javascript: inside of onload? Despite all odds, it seems to work. _spBodyOnLoadWrapper() is by the way Sharepoint’s way of letting you add javascript to pages. You first do a _spBodyOnLoadFunctionNames.push("YourFunctionName");, and then that code will get run. Insane.

Then we have the typical ASP.NET form, encapsulating the whole page, followed by 18 hidden HTML fields (including the infamous viewstate). Directly afterwards, we have a 60 line mix of inline and external javascript, contained in 7 different code blocks (two that’s missing the type and language attribute).

The next discovery can be found three nested tables later:

<span id="TurnOnAccessibility" style="display: none">
   <a onclick="SetIsAccessibilityFeatureEnabled(true);
   UpdateAccessibilityUI();return false;" href="#"
   class="ms-skip">Turn on more accessible mode</a>
</span>
<a onclick="javascript:this.href='#mainContent';" href="javascript:;">
class="ms-skip" AccessKey="J">Skip to main content</a>

I’ll wait until you regain consciousness. Hi, welcome back. Yes, they are using display: none and javascript calls to enable “accessibility mode”. I mean seriously, screen readers ignore things with display: none set, and we can certainly not always trust that javascript is enabled. I also love the skiplink, and how it sets its own href attibute to an internal link. Note how this code is (mostly) lowercase.

And this is where it starts to get ugly. Every Sharepoint page has a Personal menu and a Site menu on it, containing things you want to do with your account or the current site. Fair enough. This is the (rather lengthy) code for the Personal menu:

<span style="display:none">
   <menu type='ServerMenu' id="zz3_ID_PersonalActionMenu"
   largeIconMode="true">
      <ie:menuitem id="zz4_ID_PersonalInformation"
      type="option" iconSrc="/_layouts/images/menuprofile.gif"
      onMenuClick="javascript:GoToPage([url]);return false;"
      text="My Settings" description="Update your user information, regional settings, and alerts."
      menuGroupId="100"></ie:menuitem>
      ... [three more "ie:menuitem"] ...
   </menu>
</span>
<span title="Open Menu">
   <div  id="zz8_Menu_t" class="ms-SPLink ms-SpLinkButtonInActive"
   onmouseover="MMU_PopMenuIfShowing(this);MMU_EcbTableMouseOverOut(this, true)"
   hoverActive="ms-SPLink ms-SpLinkButtonActive"
   hoverInactive="ms-SPLink ms-SpLinkButtonInActive"
   onclick=" MMU_Open(byid('zz3_ID_PersonalActionMenu'), MMU_GetMenuFromClientId('zz8_Menu'),event,false, null, 0);"
   foa="MMU_GetMenuFromClientId('zz8_Menu')"
   oncontextmenu="this.click(); return false;" nowrap="nowrap">
      <a id="zz8_Menu" accesskey="L" href="#" onclick="javascript:return false;"
      style="cursor:pointer;white-space:nowrap;"
      ... [five more eventhandlers] ...
      menuTokenValues="MENUCLIENTID=zz8_Menu,TEMPLATECLIENTID=zz3_ID_PersonalActionMenu"
      serverclientid="zz8_Menu">
         Welcome Emil Stenström<img src="/_layouts/images/blank.gif" border="0" alt="Use SHIFT+ENTER to open the menu (new window)."/>
      </a>
      <img align="absbottom" src="/_layouts/images/menudark.gif" alt="" />
   </div>
</span>
<script type="text/javascript" language="javascript">var _spUserId=14;</script>

There’s several things worth a note here (in a bad way). The menu tag Sharepoint uses is in fact a real HTML tag, deprecated in HTML 4.01. Inside of that we have a tag called ie:menuitem that I really can’t understand. What on earth is ie:menuitem? Did they intentionally not make it work in other browsers? Everything is wrapped inside a display: none; As you see, tags are also cluttered with event handlers, both real and made up ones. The contextmenu is blocked twice, and a spacer gif is used to add alternate text to the link. The menu icon is hardcoded into everything.

It goes on in the same manner, with tables with the “TOPLEVEL” attribute (and no value), divs with a WebPartId set, PlaceHolder tags rendered directly out in the HTML (instead of parsed by ASP.NET), to end with 200 lines of mixed inline javascript and css. Everything embedded in several layers of nested tables.

In conclusion

Default.master contains the worst code I’ve ever seen, and it’s really disappointing to see that from a product with “2007″ in it. Microsoft have failed in every possible way when it comes to the interface code, and I believe the only way out is to rebuild Sharepoint from scratch (not likely to happen). Having to work with Sharepoint is a real pain, and I honestly don’t recommend it to anyone. Put your curiosity to use into something more interesting, like the ASP.NET MVC Framework instead. Thanks for listening.

I believe the best articles that talks about the switch are these:

1. Legacy, by James Bennet, puts this move in a bigger context. I believe his most important conclusion is that Microsoft are now clearly positioning themselves towards big old corporate intranets, reiterating “Don’t break the web”.
   
2. Has Internet Explorer Just Shot Itself in the Foot?, by Andy Budd, talks about how Microsoft marginalizes their own browser. Most people won’t know about the switch, and end up in IE7 mode, so there’s no real need for anyone to upgrade. IE7s rendering engine needs to be with us, forever.
3. Microsoft koan, by Mark Pilgrim, elegantly points out that strange code (unknown doctypes, including html5) will be rendered in standards mode, and ordinary code (proper doctype) will be rendered in IE7 mode. Seems twisted?

The real strangeness

What I can’t understand in all of this is how IE7-specific sites can be such big of a problem. Because that what all of this fuss is really about. That’s why we need a new switching mode and they need to ship the IE7 rendering engine with IE8.

So, now there’s a couple of different sites.

• IE6 specific sites. My guess is that these are the majority of sites out there. These will break as much in IE8 as they did in IE7. “Don’t break the web” does not apply to these sites.
• IE7 specific sites. These are much fewer and got updated rather recently (badly, to work only in IE7). The new switch is there for these sites. With the switch these sites won’t break.
• Sites built to the standards. These were also updated rather recently, but unlike the above sites they were built to the standards. Fixes for IE6 and IE7 were properly only sent to just those browsers, and will not be sent to IE8 (conditional comments anyone?). If you targeted IE7 with your conditional comment, IE8 will not get that fix, breaking these type of sites. Well, unless IE8 will execute IE7 conditional comments, when that engine does the rendering.

So let’s look back at the previous articles, keeping in mind the IE team mantra: “Don’t break the web“, the very reason why this fix exists.

First article (1): My guess is that many corporate intranets belong in the first group of sites. They haven’t been updated in a while, and won’t be either. This group of sites will be just as unhappy with IE8 as they were with IE7. This can’t be about those sites. IE team mantra: “Keep breaking the web just as much as we broke it before“.

Second article (2): This switch will increase the number of sites in the IE7 specific group. This will be at the expense of the standards group. When a developer not knowing about the switch sees that his site works in IE7 and IE8 (which will be the same), he will stop working on it. IE team mantra: “Freeze the web to IE7“.

Third article (3): Since unknown doctypes will send IE8 into standards mode, there will be even more breakage than before. What? Well, sites with broken doctypes, that previously sent IE6 and IE7 into quirks mode and relied on that, will break. IE Team mantra: “Break the web“.

Additionally, modern developers that don’t read the IE blog, and don’t know about the new switch, will keep getting IE7s strange behaviour on their sites. And since the switch also can be made via a HTTP header, even copying the full HTML and CSS of a site might break it when looking at it locally. IE team mantra: “Make people read our blog, or go insane trying to develop to the standards“.

What it all comes down to

This is a terrible idea, both technically and in how it was presented. If the same idea had been presented in a more open manner, “we’re thinking about solving this problem like this… what do you think?“, people would have been much more willing to help, and had probably came up with a better solution.

WaSP really acted badly by not first talking to the community, or at least discussing things internally, and instead just presented how IE8 will be built. Also, since this won’t help developing standards based websites, and instead help IE7 specific sites, I’m not sure what the WaSP has to do with this issue whatsoever.

On the constructive side of things, I think the best way forward for IE8 is to change the switch to be opt-in, letting IE7 specific sites switch to IE7 rendering if they want to. That’s makes sure IE7 sites that break are easy to fix, and does not break havoc on the rest of the web. Don’t set the default to worse rendering than you can produce.

With the above change, I’m looking forward to another modern web browser, IE8.

“Customization”

Let’s start. Sharepoint 2007 very, very hard to customize. If your job is to customize the interface in any major way, you have a lot of work ahead of you. You might think you’re a much better interface developer than me, but you still have a lot of work ahead of you. You need to accept that.

When you read about Sharepoint 2007 on other sites you will find that people are really ecstatic about it. Just check the comments on how to customize the calendar. Reading carefully you will find that he’s comparing it with Sharepoint 2003, not with other products, or even a standard ASP.NET site. That’s a big difference. Sharepoint 2003 was much worse, but that does not mean things are good now. This is a common pattern, people that like Sharepoint compare it to the previous version, not with better ways of doing things.

As most CMS:es, Sharepoint gets sold to customers that don’t know programming. They see a product that can handle all the features they want: blogs, wikis, forums, calendars, document libraries, workflows, and so on. All features you could ever want is included, from what is called “out of the box”. The problem is that the system is built so that all of those “out of the box” features are hard to customize. Things like changing how the wiki renders, is really hard. Often it’s easier to rebuild that feature from scratch, than customizing what Sharepoint ships.

This is the main reason, from my point of view, that you should pick another CMS.

Specifics

I don’t believe you trust me on my words, so I thought I’d be more specific now, and talk about the issues that make Sharepoint 2007 hard to work with (customize). Let me stress that these are not all the issues, just the major ones that an interface developer has to tackle.

Development

From what I’ve read from larger projects, the recommended development environment for a Sharepoint site is a virtual machine. For us, we needed a 36 Gb image, a size that required me to get a new external hard drive, just to store it. Also, running in a virtual machine means that you only can use about half of your RAM, which means everything gets slow. As if that wasn’t enough, a new virtual machine means you have to reinstall all your programs, browsers, plugins again. Web development shouldn’t have to be like that.

Accessibility

Sharepoint has gotten a lot of bashing for not being accessible, so they’ve added some features to account for that. Some of those are really good individually, but looking at the general mess that is the HTML that Sharepoint spits out, the pretty page above fades in importance. One major problem is that you still need javascript enabled for many of the accessibility features. Enabling “Accessibility mode”, among other things, makes dropdown menus appear as new windows (popups) instead of dynamically writing HTML to the page. And this is after they “fixed” it.

Page size

A standard Sharepoint site uses lots of default Javascript and CSS in two files called core.js and core.css. All in all they add up to over 250 kb. Yes, that’s not a misspelling or extra zero, a default page in sharepoint takes at least 250 kb. This is not counting the HTML, a huge, table-ridden, inline-javascripted, mess of things.

If you go through lots of trouble, you can strip those files out, but that means you can’t use any default components. Since much of Sharepoint depend on those, you don’t really want to go that way. It’s a mess, and Microsoft themselves recommend you try to remove some of it to boost performance. Preferably by using AJAX to load the js file later. Seriously.

Javascript and Browser dependencies

Except from core.js, the javascript is spread out all over the place. As usual in many CMS:es, javascript is not treated as a real programming language, so there’s no structuring whatsoever. Sometimes it’s script tags, sometimes inline, sometimes external mini files, and most often automatically generated code. It’s a real mess, and is a big reason for the HTML size being so large. Many of the scripts do not work in browsers other than IE, which means you really need IE to properly use the admin interface.

From the browser compatability info you can read that there are level 1 and level 2 browsers:

“Level 2 web browsers support all of our basic functionality so that users can both read and write in our content sites, perform site administration, etc. Some functionality may not be available, some functionality may be available in a more limited form, and some visual rendering may not be as optimized as our Level 1 browsers. We also will not support Level 2 web browsers on the Central Administration site.”

These are the Level 2 browsers: Firefox, Netscape, Safari. Opera isn’t even in the list. Is this the new, modern way, of handling cross browser compatibility?

Changing the CSS

I previously said that most CSS is included in a file called core.css. The file in itself if huge, with thousands of class names, and no structuring whatsoever. Heather Solomon has made an attempt to make things workable by making images of what each class specifies. It’s not even nearly a complete reference, and helps some, but using Firebug is a faster way. Another thing the CSS sheet above does, is show how completely unorganized the CSS is. Have a look at it.

As with core.js you can’t really remove core.css, since lots of built-in functionality depends on it. So what you need to do is start deciding which of those rules you need to overwrite and which not. Have fun!

To add to the insult, if you just add your own CSS as an external file and try to link to that on your own pages, core.css is appended as the last stylesheet in the source code. This of course means that all your changes get overwritten, and you need to hack around things by using the <link> tag directly (instead of sharepoint’s own csslink-control), or add your custom files via the admin interface. Since core.css is just a list of simple class names, you can also use CSS Specificity to make your rules count, but that isn’t mentioned in any of the troubleshooting blog posts. None the less, it’s tedious do work around.

Share point and Master pages

Most of Sharepoint is upgraded to use .NET 2.0, and its concept of Master pages and aspx pages. For those of you that don’t develop in .NET the two can be explained as a templating mechanism, where each Master page has general placeholders where you can insert aspx pages. In each aspx page you specify one master page, and all of the placeholders you want to fill with content. It’s a pretty good system.

Sharepoint taps into this system by making up two strings “~masterurl/custom.master”, and “~masterurl/default.master”, that you use instead of specifying your own masterpages. Users then specify masterpages in the admin interface, potentially breaking the entire site. Also, Sharepoint assumes there’s just one masterpage per site, so if you thought you could use one masterpage for twocolumn layouts, and another for threecolumn layouts, you’re wrong. The solution here is to break out of the Sharepoint way and hardcode real urls instead of doing things the Sharepoint way.

There’s also things you can’t change. The admin interface has a masterpage called application.master, which controls pretty much everything that has to do with built-in Sharepoint things. You don’t ever change this file, since it’s quite easy to seriously break things then. The problem is that this file also specify the look and feel of the admin interface, and if you’re building a site where people are expected to add stuff, they will spend time in the admin interface. There isn’t a good way around this, and you probably end up just changing some colors via CSS and let it be.

Default .NET controls

The default master pages that ship with Sharepoint is horrendous and I’ll post more details about them in a later post. But even if you roll your own master pages from scratch you still have the standard .NET-problem with controls. Using controls in .NET is a way to package functionality that you can easily add to your pages. A developer just needs to copy a DLL file, and call the control, optionally sending in some parameters. Easy.

The problem with this approach is that a control consists both of logic and HTML. So if you want some built-in logic that renders, say, the breadcrumb trail, you also get the HTML the author of the control deemed appropriate. In Sharepoint, a breadcrumb trail can be a doubly nested table, with display: none; controls to enable accessibility, and a doubly span nested link. It’s just a terrible mess, but there’s no other way to get to the breadcrumb logic, except from building a new control from scratch. Be prepared to work with terrible HTML.

In summary

If you want to examine the points above by yourself, there’s a MOSS example site up at wssdemo.com.

This article is a interface developer view at Sharepoint 2007. If you look at the system from other viewpoints, I’m sure there are good things there. That’s not my point. What I’m trying to say is that Sharepoint 2007, is beyond repair when it comes to interface customization, and you should do everything in your power to avoid working with it. If I can give one interface developer the arguments to convince a customer to pick another system, this article was worth it. Thanks for reading.

First we have these people calling themselves developers. And boy do they know programming… and math… and… no that’s all. Many have a masters degree in computer science, a degree that pretty much tells you that they once gave five years of their lives to programing, how cool is that!? It’s not unusual to find people that think in those terms too. You can recognize them by their 10 year old, unmatched, clothes and you’ll rarely talk more than 1 minute with them. Very efficient.

Then we have the designers. Either real females, or people very close to females, that just can’t stop talking about this abstract thingie-thongie that just like, you know, exploded in a big burst of colors and like… “EHEM! Sorry!”, you hear them say, regaining consciousness and flying back to earth. Their clothes, at least you think they are clothes, often look like they where meant for something else. Perhaps for building steel pumps, or killing animals… or both. Well, at least these clothes are matched… or very purposefully not at all matched (not at aaaall like the developers). You can talk with these people for hours, probably even days, and still only touch upon the delicate topic of “a glass of water”. Oh, and they “create” things too. Very deep.

So where the heck does the interface developer come into all of this? Well, imagine you have a real project at hand. Real people that need some new website to do their work. Real deadlines, money that switches hands. Smiling business people that promises things and shake hands. Seems like something you know of?

Now, imagine bringing a developer and a designer together in a room, for like a month, working together with this new website. Concentrate hard and try to see the images before your very eyes. Will they accomplish anything?

1. Hell no! In 5 minutes they will have started arguing about who should be the first one to open the door, and in what way. One talking about the most efficient way to turn the handle, and the other one defending herself by talking about cosmic beams from Saturn. After the first day the developer will have sore cheeks, after being constantly bitch slapped while hiding under his desk. The third day the designer will sit shaking in a corner after the electric shock induced by the steel helmet the developer built. None of them will survive the fourth day, let alone the project.
2. Very yes! When you meet them after the month has passed you find two harmonic people that smiles to each other and really seem to have connected. They explain to you that the demonstration will be in two parts, first the designer and then the programmer. The designer shows you an app that really touches you, at least you think so, although you not fully understand it. The programmer shows you the other part of the app, one that instantly shows the number 369. They finish the presentation and look at you. You stand up, walk out, lock yourself into a nearby free conference room, and shout with all the power in your lungs: “HOW THE F*#CK AM I GOING TO DO MY TAXES WITH THIS SHIT?!#”. When you come back you find them congratulating each other for making the deadline.

You see? There’s something missing there. Some kind of link between the two. Someone that can walk into a room with two screaming people and calmly say: “Hey, you both kinda run linux (SuSE vs. Mac OS X), and have a “cold” desktop background (default plain blue vs. modern interpretation of Vasnetsov Snegurochka), what a coincidence!?”. There needs to be someone that can walk into the same room the next day and explain that they are talking about two different kinds of “abstract methods”. There needs to be a link.

But there’s more: An interface developer is silly enough to learn languages that doesn’t even have variables. Even though he know that his code will be sent as uncompressed plaintext and will be rendered by broken excuses for programs, he won’t cry himself to bed every night.

He will also gladly adapt a design to something it wasn’t really intended for, and he even will make it look somewhat good. He will skip adding those fabulous shadows, to make the deadline on time, and still think of himself a good person.

You see, even though “interface” reflect the visual graphic things, and “developer” relate to hard logic code, interface developers are real people, not a combination of two halves. I am one.